Eyewitness Blues, Part 2

The Controversy

Before reading this post, please read Part 1, which presents the evidence that sequential lineups are more accurate than simultaneous lineups.

Why do the police prefer the simultaneous method? First, it produces more identifications, which is their immediate goal. Since they are likely to believe that whomever they have arrested is the guilty party, they argue that the greater number of misses by the sequential method results in guilty suspects being allowed to go free, roam the streets, commit further crimes, etc. They are not as sensitive to the costs of false alarms since they underestimate how often they occur.

In his statement refusing to adopt sequential lineups, Mr. Bucar refers to a meeting of the National Academy of Sciences (NAS) in which there was “dissent among scientists who claim that the analysis methods used in the research between 2009 and 2011 was faulty and when corrected will actually show that . . . the simultaneous method is superior.” NAS's Committee on Science, Technology and Law held three meetings on eyewitness identification in 2013 and 2014. The program of these meetings, along with slide shows submitted by the participants, is available online. However, I couldn't determine the exact nature of the controversy to which Mr. Bucar refers. I am puzzled by his reference to analysis methods used between 2009 and 2011, since I don't detect any change in methodology. NAS is preparing a report on eyewitness identification to be released in the near future. I've signed up to receive a copy and I'll let you know their recommendations.

Meanwhile, here's my best guess as to what Mr. Bucar is concerned about. One of the great unknowns in the real world use of eyewitness identification is the base rate of culprit-present lineups. What percentage of lineups actually contain the prepetrator? Is it 50%? 75%? 90%? Neither the identification rate nor the conviction rate really answer this question.

In the studies referred to earlier, the base rate is 50%, since the researchers run an equal number of participants with culprit-present and culprit-absent lineups. In the real world, the lower the percentage of culprit-present lineups, the greater the danger of false alarms. For example, suppose the police conduct a near-random sweep of the neighborhood and show eyewitnesses many people who are not identified as suspects by any other type of evidence. Under these circumstances, it becomes more important to use the sequential method.

The police probably believe that close to 100% of their lineups are culprit-present. If they are right, the number of true identifications—or more likely, lucky guesses—lost in these culprit-present lineups using the sequential method might exceed the number of false identifications avoided in the (presumably) smaller number of culprit-absent lineups. Using Bayes' theorem, Steven Clark has identified the crossover point at which the misses exceed the false alarms using the sequential method. If the true base rate is greater than 85%, the simultaneous method will identify more guilty suspects.

Of course, no one will ever know what the true base rate of culprit-present lineups is. Your guess is likely to be influenced by whether you trust the fairness and efficiency of police investigative procedures or are more skeptical. One of the more important contributions of DNA testing is that it has greatly increased our estimate of the number of innocent people who are convicted of crimes in this country.

However, simply comparing the rate of false alarms in the simultaneous condition to the rate of misses in the sequential condition doesn't do justice to the true superiority of the sequential method because a false alarm is a more costly error for society than a miss. In both cases, the real culprit escapes detection, but when a false alarm occurs an innocent suspect is likely to be arrested and charged, with costs to that individual that range from considerable inconvenience to the complete ruin of his or her life. This is presumably the basis of Lord Blackstone's famous quote: “It is better that ten guilty men escape than that one innocent suffer.”

In a classic case of bad timing, Allegheny County police recently released a man arrested based on a mistaken identification from a culprit-absent lineup using the sequential procedure. Another man confessed to the crime. Of course, no one claims the sequential procedure is foolproof. Our best guess is that, when the culprit is absent, it will produce false alarms almost one-third (32%) of the time, which is disturbing, but not as bad as the 54% false alarm rate using the simultaneous procedure.

You may also be interested in reading:

Eyewitness Blues, Part 1

Book Review:  Failed Evidence, by David Harris

Another Miss By the Supremes

Eyewitness Blues, Part 1

The Issue

On July 31, Pittsburgh public safety director Stephen Bucar had an unpromising first day on the job. A week before, Allegheny County District Attorney Stephen Zappala sent him a letter requesting changes in the way the Pittsburgh city police conduct eyewitness identifications. Mr. Zappala said his office would no longer prosecute cases unless (1) the identification was made using a sequential rather than a simultaneous lineup and (2) the lineup was conducted by an officer not involved in the investigation. His demand may have been motivated by an abstract desire for justice, but it's more likely the result of two recent lawsuits filed by men who were mistakenly arrested based on faulty identifications using the simultaneous method.

But the Pittsburgh police, notorious for their resistance to change, said something like, “That's not how we do things around here.” Unfortunately, Mr. Bucar decided to back them up, citing what he believes to be conflict in the scientific literature over whether sequential or simultaneous lineups are more likely to lead to error. This is disappointing news for those of us who had hoped new Mayor Bill Peduto would bring more rationality to Pittsburgh government.

Stipulating that the identification be conducted by an officer not involved in the case assumes that this officer won't know which person in the lineup is the suspect. If so, this double blind procedure prevents bias in which the administrator unintentionally communicates the identity of the suspect to the eyewitness. Since Mr. Bucar did not mention blind testing in his statement, I presume he has no objection to it.

When an eyewitness views a lineup, there are four possibilities. The culprit is either present in or absent from the lineup, and the eyewitness either does or does not make an identification. If the culprit is present and the eyewitness correctly identifies the culprit, this is called a hit. When the eyewitness either identifies someone else or makes no identification, this is a miss. When the culprit is absent and the eyewitness erroneously makes an identification, this is a false alarm. Failure to identify anyone is a correct rejection. Obviously, the goal is to maximize hits and correct rejections, thereby minimizing misses and false alarms.

	Culprit Present	Culprit Absent
Identification	Hit	False Alarm
No Identification	Miss	Correct Rejection

In the traditional simultaneous procedure, the eyewitness is shown an array of (usually) six photographs and asked which one, if any, is the perpetrator. In the sequential procedure, the eyewitness is shown the photographs one at a time, and is asked to state whether each one is the perpetrator. In this procedure, eyewitnesses usually don't know how many photos they will be shown.

In studies that compare these methods, participants unexpectedly witness a simulated crime, either live or on video, and are later asked to make an identification. In those studies with complete designs, the perpetrator is present in half the lineups, while in the other half, he or she is absent. Half of each of these lineups are conducted using the simultaneous and the sequential procedure. Since the researchers know whether the culprit is present in each lineup, they can determine whether a correct identification was made.

Several decades of research show the sequential procedure to result in greater accuracy. In 2011, Nancy Steblay and her colleagues published a meta-analysis of all 72 known studies (from 23 labs involving over 13,000 participants) comparing simultaneous and sequential lineups. Simultaneous lineups produce higher choosing rates (76%) than sequential lineups (61%), suggesting than the simultaneous method encourages guessing. More importantly, identifications from a sequential lineup contain fewer errors.

To illustrate this, here are the data from the 23 studies with complete designs (as described above). Since all four conditions are present in these studies, it is possible to compare their average results.

Hits from culprit-present lineups

          Simultaneous                     52%

          Sequential                         44%

          Difference                           8% fewer hits in sequential

False alarms from culprit-absent lineups

          Simultaneous                     54%

          Sequential                          32%

          Difference                          22% fewer false alarms in sequential

There is a tradeoff. If all lineups were culprit-present, the simultaneous method might appear to be superior, since it produces 8% more identifications. However, since the simultaneous method encourages guessing, some researchers argue that this 8% difference should be treated as lucky guesses rather than true identifications. When the culprit is absent, the simultaneous method leads to 22% more false alarms. Therefore, the sequential method is more accurate.

Why is the simultaneous method more error prone? When an eyewitness is asked to view a lineup, he or she probably infers that the police think they have arrested the perpetrator. In a simultaneous array, eyewitnesses make a relative judgment. They compare the photos to one another and choose the one which most closely resembles their recollection of the perpetrator. In the sequential method, however, because they don't know how many suspects they will be shown, they must make an absolute judgment, comparing each suspect to their independent memory of the perpetrator.

Here is a 61-min lecture by researcher Gary Wells that I posted once before. Between minutes 14 and 30, he talks about the relative judgment process and the need for sequential lineups.

To be continued.

You may also be interested in reading:

Eyewitness Blues, Part 2

Book Review:  Failed Evidence, by David Harris

Another Miss By the Supremes