We tend to think of the criminal justice system as a sequence of screens. First, there’s the cop screen; then, the prosecutor screen; next, the trial screen.
Then, the appeals.
Traditionally, this way of thinking envisions a model of “defense-in-depth:” a one-way process in which each of the increasingly fine screens is designed to inspect the output of the screen just before it, and to catch any errors. But the interactions between the criminal justice “screens” is never as simple as that; the screens have a complex relationship to each other and to their surrounding environments.
A current debate over the handling of eyewitness evidence provides a good example. It illuminates the influence that a “downstream” inspection screen (here, the prosecutors) can exercise over “upstream” practices (here, police evidence evaluation). Whenever an eyewitness testifies, “that’s the guy,” the jurors face a problem.
A constant drumbeat of exoneration stories in the news media tells them that eyewitnesses—even sincere eyewitnesses—are sometimes wrong, and with tragic results.Even so, the jurors reason, this eyewitness was there on the night of the crime; we, the jurors, weren’t. Jurors naturally hesitate to say that they question the witness’s accuracy. How do the jurors decide these cases? Research indicates that jurors often rely heavily on the eyewitness’s subjective level of certainty in deciding which eyewitness should be believed.
“I’ll never forget that face,” is more likely to yield a guilty verdict than “I’m pretty sure.”
But the same research indicates that, in general, a witness’s confidence in his or her own accuracy is a pretty weak indicator of reliability. And the research also indicates that a witness’s level of confidence is easily boosted (without the witness realizing that this is happening or why) by the feedback inevitably provided during the routine course of trial preparation—in witness interviews, for example, or even by cross-examination.
Psychologists feel that only “pristine” levels of uncontaminated confidence, measured at the moment of an identification, give useful information about the likelihood of error. They contend that post-feedback confidence levels can be dangerously misleading.
So, in one well-known exoneration case, by the time Jennifer Thompson wrongly identified Ronald Cotton at trial as the man who raped her, she was “certain” she was right. That’s what she told the jury, and the jury convicted Cotton.
But when Thompson first saw Cotton in a photo array she had said only that he was one of two men who “looked like” the rapist. When modern eyewitness evidence “best practices” were first assembled, the practice of taking an immediate statement of the eyewitness’s confidence at the time of the identification was an uncontroversial addition to the investigative protocols. It seemed like a good solution. Taking an immediate statement preserved what was useful in a witness’s uncontaminated estimation of his or her own confidence while it simultaneously provided a benchmark that protected against the misuse of later, and inflated, confidence statements. Gathering and documenting the eyewitness’s “pristine” confidence statement turned out to be a useful practice within the police investigatory “screen.” It allowed detectives to evaluate witnesses at an early stage, to compare witnesses to each other, to share information about witnesses and, at a supervisory level, to keep track of ongoing interactions.
Maybe most importantly, having the earliest statement of confidence provided at least a partial inoculation against the dangers of tunnel vision. It allows law enforcement practitioners to ask themselves “How can we be so certain it’s Ronald Cotton (and not Bobby Poole, the real rapist in the Cotton case) when my eyewitness only says Cotton ‘looks like’ the rapist?”
That question could—although it doesn’t always—prompt a look at alternative suspects, or a more intensive search for forensic evidence. It can open new possibilities for a self-critical investigation.
The police need that immediate statement. Prosecutors who see themselves as case evaluators, not just as courtroom combatants, could use it too.
For some prosecutors, however, the immediate documentation of confidence levels offers nothing but danger.
A statement that says only “looks like” or (even worse) “seventy percent sure” seems to these prosecutors to translate automatically into a reasonable doubt in every juror and an acquittal in every trial. In its least attractive versions, this argument reflects the desire to exploit the fact that almost all eyewitnesses have become more confident—and many have become absolutely certain—in their identifications by the time they take the witness stand.
But for other prosecutors, this argument is just a mirror image of the argument made by their courtroom adversaries, the defenders.
If confidence by itself shouldn’t be misused to find accuracy as the defenders argue; lack of confidence shouldn’t be misused to find inaccuracy. Lack of confidence at a photo array, after all, might have many explanations: it might indicate nervousness, not doubt.
When prosecutors know about an early statement of uncertainty, they will probably have to turn the statement over to the defense. The defense can be counted on to wave the report around in front of the jury. The jury seems to some prosecutors to be doomed to misuse it.
So, the requirement of taking and documenting an immediate confidence statement disappears in recent prosecutor-generated statements of “best practices” for eyewitness cases. Any temptations prosecutors might have felt to “Don’t turn it over” have been resolved by instituting a policy of “Don’t write it down.”
These prosecutors, concerned about the misuse of information at the later, adversary trial screen, have reached forward into the preceding, police investigatory screen to eliminate their courtroom problem. They would deprive the investigators of the only information derived from eyewitness confidence—the immediate “pristine” measurement—that the science tells us can be of much use.
This particular intervention in rewriting police “best practices” seems especially unnecessary.
After all, the prosecutors have, within their own litigation screen, a vehicle for preventing the misuse of confidence (or uncertainty) statements. Whenever circumstances indicate that a statement of uncertainty is more prejudicial than probative, a judge can keep the statement from jurors on that basis.
Some judges will understand the issue right away; others will need some educating. Neither a preference for not having to make that argument nor pessimism about its chances when it is made really justifies interfering with police documentation.
But as MIT researcher Susan Silbey pointed out at the recent Quattrone Center for the Fair Administration of Justice symposium on the systemic nature of criminal justice issues, this tension between the need for information and the fear that information will be misused is a chronic feature of any safety system.
Dr. Silbey used as an example her review of the effort to construct a database to monitor safety practices in university laboratories. Even apparently simple questions about whether to include a data field reflecting dates of events became problematic when some participants argued that the useful information that the dates would provide, could also be dangerous, since the dates could be used in assessing fines.
In the end, safety was compromised by the inability to resolve the trade-offs without sacrificing useful data.
It is an axiom in most fields that end-of-process inspection, while necessary, is, on its own, a feeble method of quality control. The inspection process is usually captured by the people being inspected, and their primary concern is their own safety and security.
When we see the “don’t ask; don’t tell” impulse at work in the eyewitness confidence statement debates, we should be asking ourselves whether we aren’t seeing that familiar dynamic played out in criminal justice.
And we should ask ourselves how we can reform criminal justice culture to ensure that we are constantly working to focus sustained attention on improving reliability in all of the stages, rather than just relying on the trial at the end.
EDITORS NOTE: For more on the systemic approach to addressing criminal justice errors, please see James Doyle’s March 14, 2014 Viewpoint column, “The Paradigm Shift in Criminal Justice.”
James Doyle, a Boston defense lawyer and author, was a 2011-2014 Visiting Fellow at the National Institute of Justice. The opinions expressed here as his own. He welcomes comment from readers.