Eyewitness Blues, Part 2

The Controversy

Before reading this post, please read Part 1, which presents the evidence that sequential lineups are more accurate than simultaneous lineups.

Why do the police prefer the simultaneous method? First, it produces more identifications, which is their immediate goal. Since they are likely to believe that whomever they have arrested is the guilty party, they argue that the greater number of misses by the sequential method results in guilty suspects being allowed to go free, roam the streets, commit further crimes, etc. They are not as sensitive to the costs of false alarms since they underestimate how often they occur.

In his statement refusing to adopt sequential lineups, Mr. Bucar refers to a meeting of the National Academy of Sciences (NAS) in which there was “dissent among scientists who claim that the analysis methods used in the research between 2009 and 2011 was faulty and when corrected will actually show that . . . the simultaneous method is superior.” NAS's Committee on Science, Technology and Law held three meetings on eyewitness identification in 2013 and 2014. The program of these meetings, along with slide shows submitted by the participants, is available online. However, I couldn't determine the exact nature of the controversy to which Mr. Bucar refers. I am puzzled by his reference to analysis methods used between 2009 and 2011, since I don't detect any change in methodology. NAS is preparing a report on eyewitness identification to be released in the near future. I've signed up to receive a copy and I'll let you know their recommendations.

Meanwhile, here's my best guess as to what Mr. Bucar is concerned about. One of the great unknowns in the real world use of eyewitness identification is the base rate of culprit-present lineups. What percentage of lineups actually contain the prepetrator? Is it 50%? 75%? 90%? Neither the identification rate nor the conviction rate really answer this question.

In the studies referred to earlier, the base rate is 50%, since the researchers run an equal number of participants with culprit-present and culprit-absent lineups. In the real world, the lower the percentage of culprit-present lineups, the greater the danger of false alarms. For example, suppose the police conduct a near-random sweep of the neighborhood and show eyewitnesses many people who are not identified as suspects by any other type of evidence. Under these circumstances, it becomes more important to use the sequential method.

The police probably believe that close to 100% of their lineups are culprit-present. If they are right, the number of true identifications—or more likely, lucky guesses—lost in these culprit-present lineups using the sequential method might exceed the number of false identifications avoided in the (presumably) smaller number of culprit-absent lineups. Using Bayes' theorem, Steven Clark has identified the crossover point at which the misses exceed the false alarms using the sequential method. If the true base rate is greater than 85%, the simultaneous method will identify more guilty suspects.

Of course, no one will ever know what the true base rate of culprit-present lineups is. Your guess is likely to be influenced by whether you trust the fairness and efficiency of police investigative procedures or are more skeptical. One of the more important contributions of DNA testing is that it has greatly increased our estimate of the number of innocent people who are convicted of crimes in this country.

However, simply comparing the rate of false alarms in the simultaneous condition to the rate of misses in the sequential condition doesn't do justice to the true superiority of the sequential method because a false alarm is a more costly error for society than a miss. In both cases, the real culprit escapes detection, but when a false alarm occurs an innocent suspect is likely to be arrested and charged, with costs to that individual that range from considerable inconvenience to the complete ruin of his or her life. This is presumably the basis of Lord Blackstone's famous quote: “It is better that ten guilty men escape than that one innocent suffer.”

In a classic case of bad timing, Allegheny County police recently released a man arrested based on a mistaken identification from a culprit-absent lineup using the sequential procedure. Another man confessed to the crime. Of course, no one claims the sequential procedure is foolproof. Our best guess is that, when the culprit is absent, it will produce false alarms almost one-third (32%) of the time, which is disturbing, but not as bad as the 54% false alarm rate using the simultaneous procedure.

You may also be interested in reading:

Eyewitness Blues, Part 1

Book Review:  Failed Evidence, by David Harris

Another Miss By the Supremes