Skip to main content

Advertisement

Log in

The Role of Site Variance in the American Judicature Society Field Study Comparing Simultaneous and Sequential Lineups

  • Original Paper
  • Published:
Journal of Quantitative Criminology Aims and scope Submit manuscript

Abstract

Objectives

Police departments often use photo lineups for eyewitness identification purposes. A widely adopted lineup reform designed to reduce eyewitness misidentifications involves switching from the standard simultaneous photo presentation format to a sequential format. These two lineup procedures were recently tested in the American Judicature Society (AJS) field study, which was conducted in four different police jurisdictions. The results from two phases of that investigation reached opposite conclusions as to which lineup procedure is superior, and the purpose of our current investigation was to elucidate the role of site variance in shaping those contrasting conclusions.

Methods

In previous analyses, the field study data were either (1) aggregated across all four study sites or (2) drawn from only one study site (Austin, Texas). Here, we analyze the data separately for the Austin study site, where 69 % of the eyewitnesses were tested, and the other three study sites combined, where 31 % of the eyewitnesses were tested.

Results

The results indicate significant site variance between the Austin and non-Austin study sites. In addition, the results suggest that aggregating the data across sites played a determinative role in creating the apparent disagreement about which lineup procedure is diagnostically superior.

Conclusions

Once large differences across the AJS study sites are taken into consideration, there is no longer any disagreement about which lineup procedure is superior. The simultaneous procedure is diagnostically superior to the sequential procedure, but the sequential procedure sometimes induces more conservative responding (a result that can and often does masquerade as diagnostic superiority).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. In actuality there were two purposes of the Phase II study; the first was to examine the case outcomes (both actual judicial outcomes of the cases, and rated evidentiary strength), and the latter was to experimentally test whether the identification of a suspect in a lineup influenced criminal justice decision makers in their ratings of the other evidence in the case.

  2. Because known innocents are used as fillers in lineups, they are not at risk of prosecution.

  3. Teams of case evaluators made up each of one police investigator, prosecutor/DA, defense attorney, and judge rated the same case independently and then discussed their reasons for these ratings. After that, all raters were allowed to change their ratings before we computed an average score for evidentiary strength. Additionally, the teams of raters stayed the same in a given day in which from 3 to 12 cases were rated on average, but every day, teams were changed based on the availability of the raters, so that the teams were well counterbalanced.

  4. Amendola et al. (2014) decided to conduct their experimental study strictly in Austin because (a) the vast majority of cases were drawn from Austin; (b) the limited number of cases in two of the sites; (c) cost-benefit analysis of conducting the study in sites where there were insufficient cases for independent analysis; and (d) minimization of error that could be induced from site variance.

  5. One specific concern was the variance in completion rates for lineups across sites, with a much more rapid rate of study completion in Austin as compared to other sites, indicating some hesitancy in fully complying with the experiment or its protocols in other sites (e.g. in one site there were technical concerns over software issues thereby reducing willingness to rely on the software for administering lineups, as well as slowing down the rate of completed pristine lineups).

  6. After viewing the six photos in a sequential lineup, a witness may request a second lap through the photos before making a final identification decision. A second lap is typically not allowed in lab studies, but it is almost invariably allowed in actual practice. About 16 % of witnesses in the AJS field study requested a second lap, and all of our analyses include their final decisions. Wells et al. (2015a) reported that the filler ID difference between simultaneous and sequential lineups was significant when the analysis was limited to lap 1 decisions, but that result is relevant only to lab studies, not to how sequential lineups are used in practice and were actually applied in these cases in the field.

  7. Another anticipated concern, as the limited number of cases in two of the sites would also limit statistical power.

  8. It is not entirely clear to us how they computed those expected values, but it is not our purpose here to dispute their validity.

References

  • Amendola KL, Wixted JT (2015a) Comparing the diagnostic accuracy of suspect identifications made by actual eyewitnesses from simultaneous and sequential lineups in a randomized field trial. J Exp Criminol 11:263–284

    Article  Google Scholar 

  • Amendola KL, Wixted JT (2015b) No possibility of a selection bias, but direct evidence of a simultaneous superiority effect: a reply to Wells et al. J Exp Criminol 11:291–294

    Article  Google Scholar 

  • Amendola KL, Valdovinos MD, Hamilton EE, Slipka MG, Sigler M, Kaufman A (2014) Photo arrays in eyewitness identification procedures: presentation methods, influence of ID decisions on experts’ evaluations of evidentiary strength, and follow-up on the AJS Eyewitness ID Field Study. Washington, DC, Police Foundation. http://www.policefoundation.org/sites/g/files/g798246/f/201403/FINAL%20EWID%20REPORT–Police%20Foundation%281%29-1_0.pdf

  • Carlson CA, Carlson MA (2014) An evaluation of perpetrator distinctiveness, weapon presence, and lineup presentation using ROC analysis. J Appl Res Mem Cogn 3:45–53

    Article  Google Scholar 

  • Clark SE (2012) Costs and benefits of eyewitness identification reform: psychological science and public policy. Perspect Psychol Sci 7:238–259

    Article  Google Scholar 

  • Dobolyi DG, Dodson CS (2013) Eyewitness confidence in simultaneous and sequential lineups: a criterion shift account for sequential mistaken identification overconfidence. J Exp Psychol Appl 19:345–357

    Article  Google Scholar 

  • Green DM, Swets JA (1966) Signal detection theory and psychophysics. Wiley, New York

    Google Scholar 

  • Gronlund SD, Carlson CA, Dailey SB, Goodsell CA (2009) Robustness of the sequential lineup advantage. J Exp Psychol Appl 15:140–152

    Article  Google Scholar 

  • Gronlund SD, Carlson CA, Neuschatz JS, Goodsell CA, Wetmore SA, Wooten A, Graham M (2012) Showups versus lineups: an evaluation using ROC analysis. J Appl Res Mem Cogn 1:221–228

    Article  Google Scholar 

  • Gronlund SD, Wixted JT, Mickes L (2014) Evaluating eyewitness identification procedures using ROC analysis. Curr Dir Psychol Sci 23:3–10

    Article  Google Scholar 

  • Innocence Project (2015) Understand the causes: the causes of wrongful conviction. New York, Innocence Project. http://www.innocenceproject.org/causes-wrongful-conviction. Accessed 16 March 2015

  • Lindsay RCL (1999) Applying applied research: selling the sequential line-up. Appl Cogn Psychol 13:219–225

    Article  Google Scholar 

  • Lindsay RCL, Wells GL (1985) Improving eyewitness identifications from lineups: simultaneous versus sequential lineup presentation. J Appl Psychol 70:556–564

    Article  Google Scholar 

  • Macmillan NA, Creelman CD (2005) Detection theory: a user’s guide, 2nd edn. Erlbaum, Mahwah, New Jersey

    Google Scholar 

  • Mickes L, Flowe HD, Wixted JT (2012) Receiver operating characteristic analysis of eyewitness memory: comparing the diagnostic accuracy of simultaneous and sequential lineups. J Exp Psychol Appl 18:361–376

    Article  Google Scholar 

  • National Research Council (2014) Identifying the culprit: assessing eyewitness identification. The National Academies Press, Washington, DC

    Google Scholar 

  • Police Executive Research Forum (2013) A national survey of eyewitness identification procedures in law enforcement agencies. http://policeforum.org/library/eyewitness-identification/NIJEyewitnessReport.pdf

  • Rotello CM, Heit E, Dubé C (2015) When more data steer us wrong: replications with the wrong dependent measure perpetuate erroneous conclusions. Psychon Bull Rev 22:944–954

    Article  Google Scholar 

  • Steblay NK, Dysart J, Fulero S, Lindsay RCL (2001) Eyewitness accuracy rates in sequential and simultaneous lineup presentations: a meta-analytic comparison. Law Hum Behav 25:459–473

    Article  Google Scholar 

  • Steblay NK, Dysart JE, Wells GL (2011) Seventy-two tests of the sequential lineup superiority effect: a meta-analysis and policy discussion. Psychol Public Policy Law 17:99–139

    Article  Google Scholar 

  • Steblay NK, Dysart JE, Wells GL (2015) An unrepresentative sample is unrepresentative regardless of the reason: a rejoinder to Amendola and Wixted. J Exp Criminol 11:295–298

    Article  Google Scholar 

  • Weisburd D, Taxman FS (2000) Developing a multicenter randomized trial in criminology: the case of HIDTA. J Quant Crim 16(3):315–340

    Article  Google Scholar 

  • Wells GL, Steblay NK, Dysart JE (2011) A test of the simultaneous versus sequential lineup methods: an initial report of the AJS national eyewitness identification field studies. Des Moines, Iowa, American Judicature Society. http://www.popcenter.org/library/reading/PDFs/lineupmethods.pdf

  • Wells GL, Steblay NK, Dysart JE (2015a) Double-blind photo-lineups using actual eyewitnesses: an experimental test of a sequential versus simultaneous lineup procedure. Law Hum Behav 39:1–14

    Article  Google Scholar 

  • Wells GL, Steblay NK, Dysart JE (2015b) The flaw in Amendola and Wixted’s conclusion on simultaneous versus sequential lineups. J Exp Criminol 11:285–289

    Article  Google Scholar 

  • Wixted JT, Mickes L (2012) The field of eyewitness memory should abandon “probative value” and embrace receiver operating characteristic analysis. Perspect Psychol Sci 7:275–278

    Article  Google Scholar 

  • Wixted JT, Mickes L (2014) A signal-detection-based diagnostic feature-detection model of eyewitness identification. Psychol Rev 121:262–276

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Karen L. Amendola or John T. Wixted.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Amendola, K.L., Wixted, J.T. The Role of Site Variance in the American Judicature Society Field Study Comparing Simultaneous and Sequential Lineups. J Quant Criminol 33, 1–19 (2017). https://doi.org/10.1007/s10940-015-9273-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10940-015-9273-6

Keywords

Navigation