Beware the ‘Professional’ Reformer

Print More
james doyle

James Doyle

It is a little startling to confront the sheer number of people who now earn their living as “criminal justice reformers.” This month saw the launch of the Council on Criminal Justice, the latest collection of “experts, innovators and influencers” intent on rethinking the justice system.

Reform has professionalized, and the profession is organized around discrete tasks that can be expressed in grant announcements, awarded, completed, evaluated, and (when promising) publicized.

Has this process fostered a vision of criminal justice change no individual participant would completely endorse, and which many participants would repudiate? Will it take on a life of its own?

Water is wet, but you can’t see “wetness” in any single molecule of H2O. Wetness is an “emergent” property that is visible only on the system level.

A similar principle can be applied to analyzing the emergent properties of the broad range of initiatives being advanced in criminal justice reform today.

The reform dynamic can amount to a one-way conversation, directed at practitioners. The practitioners will always need reforming; the reformers will always provide it.

This is not inevitable, but it bears watching.

A case in point: Risk assessment tools have emerged as a major element of efforts to to mobilize data as a means of reducing pretrial incarceration.

One such tool, developed by Arnold Ventures, has been the focus of recent debate. Attacked in a recent New York Times op ed column, it has also been vehemently defended.

Arnold promotes the Public Safety Assessment (PSA), an actuarial tool that it has developed and tested as a mechanism for safely minimizing pretrial detentions. Like other predictive tools, the PSA relies on a series of risk factors ( 11 factors and sub-factors),  all derived from the defendant’s past interactions with the criminal justice system, to estimate the danger of a failure to appear or a new crime.

Unlike many of the new predictive tools, the PSA’s assumptions, algorithms, and mechanics are available for public review.

The hope is that the PSA will filter out racial biases and lower pretrial incarceration rates—more specifically, that judges who substitute the PSA’s objective factors for their own preconceptions will discover that release is the right choice (or at least will have their backbones stiffened to do what they are afraid to do without numerical cover).

The PSA is an element of Arnold’s $48 million investment in the National Partnership for Pretrial Justice. 

The list of Arnold’s partners (“grantees” would be another word) in that effort reads like Homer’s epic catalog of the Greek ships drawn up before Troy.

An armada of grant-seeking, grant-executing, grant-evaluating, criminal justice institutions is represented in the National Partnership. Just to name a few of the individual vessels: RAND Corporation, Vera Institute of Justice, Urban Institute, Stanford University, RTI International, the National Center for State Courts, and the New York City Criminal Justice Agency.

Arnold’s website provides a comprehensive list of the individual research projects being undertaken by the think tanks. There is an astonishing amount of useful research underway.

Still, although many worthy efforts can be listed, the shape of the struggle—of the “greater than” zone in “greater than the sum of its parts”—hasn’t been given very much attention.

Will this really come down to an effort to prescribe, from the heights of the think tanks, in detail, and in advance, the steps to be taken by people on the distant frontlines?

Portraits in Risk

The signature tool of a contemporary reformer is data. But despite the heroic efforts of pioneers such as Amy Bach, the reformers frequently find themselves making do with the data they have rather than utilizing the data they need.

In PSA and similar efforts, the data is supplied by a system of portraiture that reduces the African-American men who are the intended beneficiaries of the actuarial approach to history-based scores.

There is a grim irony in the fact that the group whose experience in America is characterized by a unique and acute vulnerability will now be depicted exclusively in terms of the risk they present to others.

In this scoring, defendants are all past, no future; the only relevant aspect of their pasts implicates decisions (Arrest? Convict?) made about the defendants by the criminal system itself.

Meanwhile, the history-based scoring eliminates any need to take account of the iatrogenic injuries (loss of job, loss of children, disrupted treatment, loss of housing) that detention will inflict.

And since it considers only “static” historical factors, and not “dynamic” factors (such as treatment possibilities or employment counseling), the scoring also releases the system’s architects and operators from any need to consider their own problematic failures in risk reduction practice.

Besides—unlike a drug treatment program, or mental health counseling—PSA is free. The temptation for a local system under pressure to hope (or pretend) that it is a silver bullet that resolves all pretrial problems is substantial.

In the end, you are left with the mock-learned taxonomy of subject peoples that the British developed in London and then applied in colonial India through the Criminal Tribes Act.

 The Message in the Numbers

Debating the PSA’s statistical elegance won’t quite address all of the issues.

The risk here is that we will think the work is done when we eliminate the creation of false differences (assumptions based on racial stereotypes, for example).

But there is another mistake we can make about difference: we can ignore genuine differences (in law enforcement activity, or individual family, medical, or employment histories) when they exist.

The actuarial process says to the defendant, his family, and his community, “We see you (or your son, or your neighbor) as an integer, not a person.”

In fact (since everyone involved in the bail hearing knows everyone else’s race) it says to one vulnerable group, “We see you as one of the African-American integers.”

Would they choose to be depicted that way? Has anyone asked them?

It is hard to believe this approach doesn’t eventually erode trust in the law.

A research report by the Center for Court Innovation tactfully explains why no criminal history score can escape the influence of the racial weighting of the pre-existing determinations that constitute its data pool, and suggests workarounds to mute the effect.

No one actually believes that reformers generated a pure numerical tool that is then transmitted and faithfully mobilized on the front lines. PSA is explicitly not a substitute for judges.

As Prof. Mona Lynch has shown in a February, 2019 article in the American Bar Foundation’s Law & Social Inquiry journal, instead of transforming defendants into a set of criminal history points that help determine sentence assignment from a table, the quantified criminal histories simply provide a starting point: another opening for the frontline practitioners’ development of their own “narrative of the number.”

There is no reason to be confident that (with or without a score) the race of the defendant has no impact at an arraignment. For example, experience with Kentucky judges who were asked to employ a precursor risk assessment tool showed a heightened willingness to override risk scores where moderate-risk black defendants were evaluated.

Whatever biases the scores’ stylized representations of the defendants bleed out of the process, the practitioners can put right back in. In a context where a “false negative” (the release of a Willie Horton) terrifies the judge and a “false positive” (the detention of a Kalief Browder, who would not have committed a new crime) will never be discovered, at least one gravitational pull is baked in.

Judges care about community safety; they care about their own job security too.

It is a truism that “culture eats strategy for lunch.” You should expect courthouse culture to gnaw at your algorithm.

Frontline practitioners are not a transparent medium through which the message from the researchers and policy-makers passes; they refract the message to the community, and they refract the message back from the front.

People in Systems

Recently Jeremy Travis, Arnold Ventures’ Executive Vice-president for Criminal Justice, writing in The Crime Report, called for a new lens. He argued that the reform world needs, “a more expansive research agenda [that] would be focused on people, families and communities, not systems.”

That welcome observation might call for a friendly amendment.

The question isn’t people or systems; the question is people in systems. That means we have to spend some time understanding what sort of system people are entangled in.

Travis is quite right that a linear, sequential, model of a system and the research it attracts are inadequate. The criminal system is not a Newtonian mechanism of springs, gears, and switches where every action has an inevitable reaction that can be known in advance. It can’t be fixed by optimizing an individual component. It probably can’t be permanently “fixed” at all.

But criminal justice is a system: it is a complex socio-technical system. Its operators and the people whose lives they affect are subject to swirling clouds of influences which are not “causes” with inevitable effects. These influences impact the probabilities; they do not flip a switch. They are often over-lapping and sometimes conflicting. Everyone’s work is affecting everyone else’s work, all the time.

The work that is done often has to be very different from the work that can be defined in advance. Workarounds, triage, covert work rules, and “practical drift” adjustments multiply.  Very often “workmanship”—not simply following the rules laid down in the researchers’ grid—is what matters.

A great deal can be learned from measuring outputs, but it would be good to remember that outputs cast at best an oblique light on processes.

And there is a reading of the outputs that suggests that the frontline practitioners could be treated as a resource to be mobilized, rather than an audience to be instructed.

For example, a careful study of the “holistic” defense practices of the Bronx Defenders—which, in their highly individualized approach to defendants and their issues constitute a mirror opposite of the actuarial scoring solution—showed an 8.6 percent reduction in pretrial detention. As Silicon Valley De-Bug’s Participatory Defense initiative shows, there is every reason to think that the community, if it were treated as an element of the frontline, could make an important difference too.

My argument here is not that all researchers everywhere are always ignoring the realities of frontline life. I realize that there are numerous contacts between the research and the practice communities.

And my argument here is not for ignoring the researchers’ data-derived vision of the justice system in favor of the frontliners’ narratives.

The argument is for an explicit, determined focus on recognizing that we need both lenses, and that their two approaches are in tension. We can try to make sure that inevitable tension is a productive one.

There’s a lot to be learned, but it can’t be learned (or taught) top-down. Everyone got into this situation together; together is the only way we will get out.

We should do what we can to nourish the capacity for everyone to learn from each other— and all of the time.

James M. Doyle is a Boston defense lawyer and author, and a frequent contributor to The Crime Report. He welcomes readers’ comments.

One thought on “Beware the ‘Professional’ Reformer

Leave a Reply

Your email address will not be published. Required fields are marked *