Measuring Justice


How should chronic problems in a U.S. criminal justice system that costs $212 billion a year and employs 2.4 million people be identified — and fixed?

One place to look is the courts, and that is the challenge for Measures for Justice, an organization started in 2011 by journalist and law professor Amy Bach of Rochester, N.Y., whose 2010 book, Ordinary Injustice: How America Holds Court explored the everyday workings of local U.S. criminal courts.

In a chat with Ted Gest, TCR Washington bureau chief, she discusses the reasons for the wide differences in services, quality and efficiency in courts across the nation, and why these should matter to every American.

The Crime Report: Why did you decide to focus on this problem?

Bach: I was a journalist for five years before attending Stanford Law School. After graduation, I sat in courts across the country writing a series of articles and eventually a book. What fascinated me is how hard working, well-intentioned legal professionals could have a blind spot to a pattern of errors, like the prosecutor who had no idea that he hadn't taken on a domestic violence case in 21 years. “Has it been that long?” he said. Yes. That long.

Or the public defender who pleaded 48 people guilty in one day. He could do it because he didn't know the facts of their cases. And it was more efficient to take a plea. “Nobody could say that they didn't have their day in court,” he said.

To provide context for these stories, I needed statistics comparing one county's court to another. This isn't being done in America. I later learned that criminal courts at the county level are one of the few public institutions that are unable to measure their performance consistently and uniformly. The delivery of nearly every public resource—education, medical care, drinking water, even the atmosphere—is subject to evaluation by a watchdog apparatus. There is virtually no comparable assessment of local legal systems. After seeing these issues and talking to people about why this hadn't been done before, I decided to found Measures for Justice to create a criminal justice index to measure how basic services are being performed in local systems across the country. Our brain trust – the MFJ Data Council – is a group of experts with diverse experience as practitioners, criminologists, and government employees who want to create measures that will make a difference.

TCR: What kinds of cases are typically considered in what you call the “lower criminal courts”?

Bach: They are the ones that touch most peoples' lives. Crimes like drug possession, assault, driving while intoxicated, theft, and of course, murder are all addressed or adjudicated in lower criminal courts. The cases include felonies and misdemeanors, but generally not violations like speeding tickets.

TCR: Are these the kinds of courts that average citizens ever would find themselves in as a defendant?

Bach: Yes. Roughly 21.4 million cases come into the local and state courts each year. Unless you've committed a federal crime or your case has been appealed, this is where justice happens for most people in the U.S.

TCR: Based on your observations, how much do these courts vary in terms of quality, from place to place?

Bach: In our quest to develop the index, we have found enormous differences in terms of performance across counties that beg for explanation. Our Data Council, which is a group of experts on criminal justice performance measurement, conducted an “Illustrative Measures Study” to help provide a “proof-of-concept” for our work.

In the study, we took 12 important indicators and found publicly available data to populate them. The disparity in some of the results was wide. Some metropolitan areas have average bails that are 35 times as high as others. And there's large disparity in the percentage of people who can't afford to make bail in different locales. There are also huge variations in terms of caseloads for public defenders, prosecutors, and advocates for victims. These wide variances can serve as a red flag that something in the system isn't as effectively working

TCR: Can you describe the measuring sticks you plan to use in the three main areas you mention: public safety, fairness and accuracy, and fiscal responsibility?

Bach: These areas represent three broad goals that Measures for Justice and its Data Council believe all criminal justice systems should accomplish. Of course, many of the goals overlap. Public safety encompasses a broad spectrum of issues because much of fairness and accuracy ends up being public safety. For example, we created a measure to show prosecution rates of domestic violence arrests. Getting those who have violently offended off the street is a public safety issue as much as it's an issue of fairness and accuracy for everyone involved.

Fiscal responsibility really is embedded in all of our measures. For example, high recidivism rates can be an enormous financial burden for a county as well as a state. Overall, we have to give counties information so that they are not left alone scrambling to control the costs of corrections systems.

TCR: Do you think you can capture the quality of an individual performer in the system like a judge, lawyer or clerk, in terms of numbers or some kind of verbal characterization?

Bach: No, you can't. The Measures for Justice Index is not aimed at identifying a particular “bad apple,” and we don't believe that a single individual could overturn a system or culture, or cause its demise. Our focus is identifying and addressing systemic problems.

TCR: Over the years, some groups around the country have organized court-watching teams, where citizens spot check courts to see how they function. How does what you are doing differ?

Bach: There is a story in my book about a court-watching group in upstate New York which praised a judge for his courteousness and kind demeanor. They sat and watched him for weeks. Eventually, this same judge was kicked off the bench by the New York State Judicial Conduct Commission. The Commission found a pattern of lapses that happened because an entire court turned a blind eye to problems—by setting extremely high bails for nothing crimes, such as $25,000 to a homeless guy for riding his bike on the sidewalk without a bell, and by allowing many defendants to plead guilty without attorneys. What revealed these stories was the collection and analysis of data. I do think these groups are useful for checking on things like a judge's demeanor. But good citizens, lawyers, etc. can't see these problems on their own. We need good data.

TCR: How are you developing the index?

Bach: Ultimately, we need to test the measures in a wide variety of counties to ensure they are meaningful and replicable across the United States. We are conducting our first pilot in Milwaukee County, Wisconsin, in partnership with the U.S. Justice Department's Bureau of Justice Assistance. Milwaukee has been the site of several extremely important studies. It may be one of the most data-friendly counties in America, which provides a good starting point. We have a plan to test the measures. The plan lays out some of the qualities we will look for when identifying pilot sites as well as a general approach to entering and working with a county. The aim of the pilots is to hone in on what the right measures are and develop a process for how we can collect this data in each county. We hope to cover 50 county courts eventually.

TCR: Do you plan to give overall number rankings or totals for named courts, or by category of function being measured?

Bach: It's something we're considering, but it's not our first priority. Ranking is important because it makes the results interesting and important to insiders and outsiders alike. But we want more than just one number. We want to be able to give stakeholders reliable information that can be used to drive improvements to local criminal court systems. We want to give them information they really need.

TCR: Has any index like this been attempted for any other government function? If yes, do you think it has worked to correct problems or at least inform policymakers of them?

Bach: That's a great question. In education, outcome measures such as teacher-student ratios and college admission rates are standard ways of assessing opportunity for students in different communities, and winning government funding. The Dartmouth Atlas of Health Care, founded in the 1970s, brings to light a wide variation in professional decision-making and allocation of resources in health care. Today, it is a common source for decisions in health care policy and federal resource allocation. So, yes, performance measurement is increasingly being used in many disciplines to address problems. There are still some kinks, but these are slowly being resolved.

TCR: Many Americans probably perceive that unless they happen to be involved personally in a case, what happens in the lower criminal courts doesn’t affect them much. Why should they care?

Bach: There are many reasons to care. First, anyone can be a victim of crime. And even if you personally are never arrested, all of us have or will have friends and family that will be touched by the system. About 65.7 million people in the U.S. have criminal records, and that's a conservative estimate. The numbers for victims would be nearly as big. Once you do have a felony record, the “collateral consequences” are immense. People can't find homes or jobs. This ultimately affects our economy and tax dollars.

Finally, states and people in general are concerned about our mass incarceration problem because it's costing us so much money. In 2011 there were 12.4 million arrests; 11.8 million admissions in jails and 608,166 admissions to prison. These high numbers are the direct result of how criminal courts perform every day in the 3,143 counties in America.

TCR: In a similar vein, during this year's 50th anniversary celebrations of the Supreme Court’s Gideon vs. Wainwright ruling, critics pointed out that many courts still don’t provide attorneys in many criminal cases. How might your effort affect that?

Bach: Good data can absolutely help highlight which courts aren't providing lawyers in criminal cases. Consider what happened in Pennsylvania's Luzerne County in 2009: advocates heard stories that children who came to juvenile court were without lawyers to protect their rights – and were being sent to jail for almost nothing. Like lampooning a principal on a MySpace page.

Philadelphia's Juvenile Law Center brought a suit in the state Supreme Court alleging that for 2005 and 2006, more than 50 percent of juveniles appeared in court without attorneys. At nearly ten times the state average, these numbers were so blatant, they could not have happened unless prosecutors, public defenders, and judges had all turned a blind eye.

In the end, two judges were charged with taking millions of dollars in kickbacks from private detention facilities in exchange for putting kids in prison who didn't belong there. What saved the day for these children was not appellate review, but a data set that exposed not just a problem here, a problem there, but a pattern of lapses. It was the kind of data that allowed outsiders to spot a tragedy in the absence of a crisis. In essence: a mini justice index that compared one county's adherence to providing lawyers to the rest of the counties in the state.

Stories like this show that measurement matters. We need to take care of the “indigent” the way we take care of everyone else. The biggest problem is with the way we measure—or fail to measure – basic legal services. No measurement. No change. It's that simple.

Ted Gest is president of Criminal Justice Journalists and Washington Bureau Chief of The Crime Report. He welcomes comments from readers.

Comments are closed.