Learning How To Learn (Together) From Justice Errors

Print More
james doyle

James Doyle

Criminal justice reform is having its moment in the public square. The gatekeepers—editors, publishers, producers, bloggers, and the “most-followed” social media posters—have decided to grant criminal justice issues some attention.

These media moments always fade. How can reformers exploit the opportunity this one presents? Can something useful be left behind?

Since 2011, the National Institute of Justice (NIJ), the research arm of the Justice Department, has been exploring the feasibility of promoting “sentinel event reviews” modeled on those that helped transform safety in industry, aviation, and medicine, in order to develop a “forward-looking” approach to preventing errors such as wrongful convictions, wrongful releases and cold cases that stay cold too long.

Like safety leaders in aviation and medicine, NIJ was also interested in learning the lessons provided by “near miss” and “good catch” experiences where only good luck or a last minute intervention staved off a disaster.

Last week, the NIJ Sentinel Event Initiative Team released Paving the Way: Lessons Learned in Sentinel Event Reviews, its report on the lessons learned by three pioneering “beta sites.”

These sites—Milwaukee, Baltimore, and Philadelphia—had been selected to explore how to mobilize a new routine of “non-blaming, all-stakeholders” reviews of negative outcomes: outcomes that are likely the result of compound errors, signal underlying weaknesses, and provide keys to preventing adverse outcomes in the future.

(Because NIJ promised the three sites that it would protect the anonymity of their review participants as far as possible, Paving the Way does not describe the local events reviewed or recount the analyses that the stakeholder groups produced. Douglas Starr’s piece in the online edition of The New Yorker shows some of the protagonists in action.)

The findings that a sentinel event review program might generate are not designed to be ends in themselves; it is the practice of routinely conducting reviews that may have the potential to nourish a new criminal justice culture of continuous quality improvement and to promote public trust.

Perhaps surprisingly, this novel application of a core idea borrowed from other fields wasn’t a tough sell to criminal justice stakeholders.

This may be because NIJ involved diverse stakeholders early in the development process. At a later point, an eclectic array of commentators from the policing, prosecution, defense, research, crime survivor, and policy communities weighed in supporting the concept in a 2014 NIJ Special Report: Mending Justice, Criminal Justice Sentinel Events.

Pretty much everyone agreed that when things go wrong, a “bad apple” or a lone component is never the complete explanation—that things in the surrounding systems environment set the stage and played a role, and will again unless identified and dealt with. The challenge isn’t to protect a perfect system from dangerous humans; it is to build resilience into a system that is operated by ordinary, fallible people facing production pressure and resource scarcity.

Everyone thought applying the aviation and patient safety approach was a good idea.

Not everyone thought it could be done.

No one thought it would be easy to do it.

Paving the Way shows that jurisdictions can in fact carry out non-blaming, all-stakeholders reviews aimed at cutting future risk, not at fixing blame.

The study does not find that everyone, everywhere, can (or will) conduct a sentinel event review every time something goes wrong.

Some jurisdictions will never conduct a review. Even in receptive jurisdictions, some events are too complex, too eccentric, too old—or simply too hot to handle.

But Paving the Way shows that some jurisdictions can install a routine of examining some cases, some of the time.

That could be a big step forward.

The “beta site” experiment was not the typical contemporary reform process. Most often, new “best practices,” guidelines, checklists, or statutes, are handed down from the Olympian heights of academic centers, Technical Working Groups, think tanks, or law reform Commissions.

The policy wonks, researchers, and reformers who dominate those bodies tend to see the frontline people who do the actual work on the streets or in the courtrooms as either passive objects of study (who should just hold still and have their “outputs” counted) or as dangerous toxins (who have to be punished or quarantined).

In contrast, Paving The Way asks what would happen if we evaluated the front line players—who, after all, will be there long after the academic Dieties and the media have gone home—not as wearisome nuisances or wayward children but as indispensable resources.

The beta site experiences mined in Paving The Way show that inaugural sentinel event reviews in subsequent jurisdictions won’t be launched easily, and that they can’t be launched at all unless they mobilize the talents and dedication of early adopters in the frontline practice communities.

Paving The Way vindicates Michael Jacobson’s assertion in his essay in Mending Justice: “If you want to learn something, do something.”

Nevertheless, it indicates that the most promising route to culture change in criminal justice is not “top-down” but collaborative—that if any vector has to be chosen it should be “bottom-up.”

The report organizes the participating frontline players’ advice about the decisions that subsequent adopters will face and the considerations that should be weighed in making those decisions, while never wavering in from the recognition that local answers will depend on local circumstances, or attempting to dictate choices.

Where do you start? The questions of leadership buy-in and resource gathering are discussed.

What event do you choose? The key importance of initial focus on mitigating the legal risk to the players, attracting broad system participation, and weighing the pluses and minuses of older and more recent events is discussed. The broad range of “near misses,” “good catches,” and other “high frequency, low impact” events is outlined.

Who should be on the team? The role of the sharp-end players is outlined. The experience with the advantages and disadvantages of non-traditional participation by community, crime survivors’ and academic communities is shared.

Where will challenges arise? Paving the Way provides a table listing the gritty logistical and interactional problems that have to be expected.

Other questions are concisely considered. Who should lead? What is the researcher’s role? How do you structure the SER? Set ground rules? These and other questions gain tremendously from the contributions of the array of stakeholders who took part at the three “beta sites.”

The result is an intensely “real world” document that neither ignores real problems nor obsesses over phantom dangers.

There’s something exhilarating about watching criminal justice veterans participate in this exploration, and successfully face, then master, a range of problems and roadblocks. One of the great virtues of Paving The Way is that it conveys the sense that, for the participants, the challenging sentinel event review process can be not only meaningful, but personally rewarding.

They are proud of themselves. There is even a suggestion that the experience might be fun.

The overall effect of reading about the development of the lessons the stakeholders’ now share is something like watching Sherman’s army march through South Carolina, building its own roads as it marches to the sea. There are lots of corporals and privates, developing lots of creative (and indispensable) workarounds and adaptations.

And, of course, now that the road has been built, it can be traveled more easily the next time.

In the end, the lessons relayed in Paving the Way are not lessons about permanent fixes to tidy problems. Every fix, after all, will face immediate assault from its environment the moment it is announced. Caseload pressure will rise; politicians will scream; budgets will fall.

The lessons of Paving the Way, derived from hard, unglamorous work volunteered by a lot of hard-working professionals, are about how we might try continually learning lessons as a way of life.

If the trail these sites have blazed is followed, and sentinel event reviews are added to our routine of sheepish press conferences, reciprocal finger-pointing and million-dollar lawsuits following an exoneration, we will have fewer wrongful convictions.

And we will need fewer exonerations in the future.

Editor’s Note; For another view of the Sentinel Review concept, please see the blogpost in Grits for Breakfast “Exoneration Review Commission To-Do List

James Doyle, a Boston defense lawyer and author, was a 2011-2014 Visiting Fellow at the National Institute of Justice. He served as editor of the Mending Justice report. The opinions expressed here as his own. He welcomes comment from readers.

Comments are closed.