When Bad Science Produces Bad Evidence


On March 11, 2004, powerful bombs set by terrorists on four Madrid commuter trains killed 191 people and wounded 1,800. Two months later, the Federal Bureau of Investigation (FBI) arrested Brandon Mayfield, an Oregon lawyer, after fingerprints found on a bag near the explosion site allegedly matched his.

Mayfield had not been to Spain. Nor had he been outside the U.S. for over a decade. But the FBI kept insisting the fingerprints were a “100 percent match”—until Spanish police tied another suspect to the fingerprints. And even after Mayfield was released two weeks later, the FBI continued to insist that their fingerprint-matching process was infallible.

It was a costly mistake. Mayfield eventually received a $2 million settlement from the U.S. government. But to David A. Harris, it also was—or should have been—a teachable moment. Harris, a professor of law at the University of Pittsburgh, believes the FBI's insistence on the accuracy of their analysis, even when the evidence failed to bear it out, reflects a law enforcement culture that relies too much on quasi-scientific forensic evidence—even while it resists the application of genuine advances in science-based investigative techniques. In his newest book, Failed Evidence: Why Law Enforcement Resists Science, released in September, Harris points out that such resistance not only hampers chances of a fair trial, but reduces the effectiveness of modern policing.

In a conversation with The Crime Report's Deputy Editor Graham Kates, Harris discussed why mistaken identity cases based on a misunderstanding of science, like the arrest of Mayfield, are all too common; how even the standards of interpreting DNA evidence have changed over the last two decades; and why it sometimes takes a tough-on-crime advocate to force change.

THE CRIME REPORT: You say a lot of forensic investigation techniques are not scientifically proven. What do you mean by that?

DAVID HARRIS: The 2009 National Academy of Sciences report, “Strengthening Forensic Science in the United States,” was a particularly strong statement on the state of forensic science. It makes a distinction between things that are strongly data based, and the rest of forensic science, which is not data based, which did not grow out of science, but out of police investigation.

You have a whole bunch of forensic tools that are used every day around the country, like fingerprinting and tool marks, all the way down to things like bite marks and hair and fiber, and the report basically said, “Those things are not science.”

They are not necessarily wrong. In fact they are usually correct, but they are based on human interpretation and human experience, not on data. Which means we are not in position to say things like '100 percent match,' which of course we say all the time with those sorts of forensic methods.

It's fine to claim that you can always match a fingerprint because you're the most experienced examiner, but in the wake of Brandon Mayfield and many others, it's just not a claim that you're going to be able to sell anymore, and we shouldn't expect it to. We now know better.

After the Mayfield case, you had agents saying they could match fingerprints with zero error rate. That's not tenable.

TCR: Scientific method is a pretty widely accepted concept. Why isn't it the standard in courts?

DH: Judges are not scientists. In fact I might say that many in the legal profession don't have a natural inclination towards science. We don't understand how the process works. When judges see something like fingerprinting, where people who have been doing it for a long time say they can get a 100 percent match, they are not really equipped to understand why that isn't science.

There's even greater skepticism about social science. Typically, people think of anything that doesn't have beakers and test tubes as easily manipulated, and therefore lacking in value. And that's just not true. As long as any kind of inquiry is done according the principles of scientific method, you can get valid results.

TCR: You say that part of the problem is 'cognitive barriers.' Can you explain?

DH: A lot of what's called behavioral economics has highlighted that in the last couple of decades. It tells us that a lot of our thinking processes don't always serve us well.

The biggest problem is probably cognitive dissonance: the idea that when you have two opposing thoughts, it's really hard to cope with that, and it creates a tension, a dissonance. Usually we resolve those things by simply rationalizing one set of thoughts away.

I'll give you an example: Suppose I smoke two packs of cigarettes a day and I also know — since I'm awake, alive and reasonably intelligent — that smoking gives you lung cancer and can cause emphysema. These are two opposing thoughts and I have to resolve that tension. I might rationalize it away and say, 'Smoking's bad for you, but it's also bad to be obese and I know that when I smoke I eat less.' I've taken a different risk factor and reduced it, so I've figured out a way that the dissonance doesn't exist.

We've seen much the same thing with science in this field. In law enforcement there's a strong tendency for police and prosecutors to say, 'We're on the side of the angels, we wear the white hats, we put the bad guys in jail and we stand up for victims.'

So what you have is two opposing thoughts. One, we're the good guys, we put the bad guys in jail. Two, in opposition, sometimes even when we do our best, things go horribly wrong, because of how we've been trained.

So they have to rationalize one away. What I've heard time and time again is, 'Well actually, 300 is a very small number of cases.' Or, 'See, the system worked. It caught these 300 cases.'

TCR: A lot of cops say they'd like to use DNA more than they currently do, but labs are backlogged and it's hard to get access for cases as it is.

DH: It's true, there are backlogs in many labs. The last report on this I heard, labs in L.A. were so backed up, that they were basically going to a system of triage.

This is going to change in the next few years, with the adoption of some more sophisticated and faster DNA analysis systems. Some of which are already in use in the New York State crime labs, in a few labs in Pennsylvania, and in the U.K. It's basically an entirely computer-based system, and can actually get much more information from the DNA that we have. Right now DNA analysis is based on a 20 year-old-standard, and we're throwing out of information. This has a huge impact on cases where you have a mixture of DNA, and that's actually much more common than people think.

Right now, DNA is going through its own DNA moment. We went through this with fingerprinting and identification of suspects, in which DNA pointed out how a much better job needed to be done. Now we're getting to that point with DNA analysis itself, where new DNA techniques are going to become available and law enforcement is going to have to open up to those things.

TCR: In your book you advocate for the recording of interrogations. It's a practice the NYPD just endorsed, and supposedly will be using across the board. Does that mean the practice is about to become the industry standard?

DH: This has been going on in other police departments since at least the early '90s— but not many. The NYPD's adoption of this will have an impact, because it is one of the flagship police departments in the country.

Among police officers that have done this, that have used this as part of their system, I don't hear any dissenting voices at all. It's really pretty remarkable.

The best comment I've ever heard about it is from a police officer from Minnesota, who said something like, “This was the best reform that anybody ever shoved down our throats.”

TCR: So why is there so much resistance in other departments?

DH: It's so interesting isn't it? It's one of the really good examples of why this process is going so slowly. They think of themselves as in that interrogation room doing the lord's work and sometimes you have to get tough, but they worry that juries won't like it.

But there's no evidence that juries feel bad about what goes on in those recordings. What happens is there are fewer motions to suppress and the cases end up in guilty pleas much more often.

We have to have law enforcement figures who have been doing this for years, saying that once you start you won't want to go back. That this is where the future is.

TCR: Speaking of that, you write about thinking for the future instead of focusing on past mistakes.

DH: Up to this point, I think it's fair to say that much of the reform in the area of wrongful conviction has, necessarily, focused on fixing the wrongs of the past, and getting people out of prison who don't belong there. That has to continue, but what we have to do now is take a different focal point on reform. We have to be looking forward.

The way we do that is not to ignore the wrongs of the past. First, we build conviction integrity units in every prosecutor's office. A unit within every office, whose job it is to investigate cases in which there are serious allegations of wrongdoing or wrongful convictions. If you have this as part of the built-in wiring, plumbing and air conditioning of every office, then you can focus your attention on mistakes going forward.

You can form the reform effort around preventing mistakes going forward. If we make the ferreting out of wrongful convictions simply part of what we do every day, we'll have addressed the cognitive dissonance I mentioned before.

TCR: Aren't there issues with recruiting attorneys to staff conviction integrity units in some offices? Law enforcement officers have been known to look down on their colleagues in internal affairs.

DH: The idea that there would be people who would not be held in high regard, or that it would be a dumping ground can be combated by making people rotate through. It would have to come from the leadership of every office that it's a high priority to ensure integrity.

The leading example is Craig Watkins in Dallas. He set one of these up right away when he was elected, because Dallas has such a horrible history of wrongful convictions. He knew that getting this right was the only thing that would ensure people of the integrity of his office.

TCR: Politically, what's the key to getting reform done?

DH: People think that reform is more likely to come from the political left, but where I've seen success is where there's been a champion on the political right.

This is sort of the theme that you see over and over again in successful statewide efforts. You tend to see a champion on the right working with people on Innocence Projects or something like that.

My favorite example is a guy named I. Beverly Lake, who was the Chief Justice of the North Carolina Supreme Court. He described himself to me as a “conservative's conservative.” Nobody could question his tough on crime bona fides, but he became concerned when North Carolina had some of these cases of wrongful convictions.

He told me that he was concerned that if people don't accept the system's integrity, they'll never get another death verdict out of jurors in North Carolina. Now, that's not where I come from, politically or philosophically, but you can see the reasoning.

So he's the guy who pulled everyone together in North Carolina. He got all the investigators and public defenders around the table, and because of that, North Carolina is the only state that has an Innocence Inquiry Commission: a statewide body that is designed to hear claims of actual innocence.

Graham Kates is Deputy Editor of The Crime Report. He welcomes comments from readers.

Comments are closed.