Quality & Quantity

, Volume 49, Issue 4, pp 1735–1745 | Cite as

Predicting correctness of eyewitness statements using the semantic evaluation method (SEM)

  • Farhan SarwarEmail author
  • Sverker Sikström
  • Carl Martin Allwood
  • Åse Innes-Ker


Evaluating the correctness of eyewitness statements is one of the biggest challenges for the legal system, and this task is currently typically performed by human evaluations. Here we study whether a computational method could be applied to discriminate between correct and incorrect statements. The semantic evaluation method (SEM) is based on latent semantic analysis (Landauer and Dumais Psychol Rev 104: 211–240, 1997),—a method for automatically generating high dimensional semantic representations of words and sentences. The verbal data was extracted from the recorded narratives from a prior eyewitness study investigating the role of repeated retellings on subsequent recall accuracy and confidence (Sarwar et al. Cognit Psychol 25(5):782–791, 2011). Participants watched a film of a kidnapping and then either retold the events to a single listener, or discussed the content with a confederate at five separate times over a 20-day period. Their subsequent written recall was analyzed using the SEM. The results show that accuracy can be predicted from quantification of the semantic content of eyewitness memory reports using SEM. This result also held true when data was separated into three distinct categories and the SEM was trained and tested on different categories of data.


Eyewitnesses’ correctness Semantic evaluation method  Semantic spaces Consistency 


  1. Allwood, C.M., Ask, K., Granhag, P.A.: The cognitive interview: effects on the realism in witnesses’ confidence in their free recall. Psychol. Crime Law 11(2), 183–198 (2005). doi: 10.1080/10683160512331329943 CrossRefGoogle Scholar
  2. Arvidsson, D., Sikström, S., Werbart, A.: Changes in self and object representations following psychotherapy measured by a theory-free, computational, semantic space method. Psychother. Res. 21(4), 430–446 (2011). doi: 10.1080/10503307.2011.577824 CrossRefGoogle Scholar
  3. Berman, G.L., Cutler, B.L.: Effects of inconsistencies in eyewitness testimony on mock-juror decision making. J. Appl. Psychol. 81(2), 170–177 (1996). doi: 10.1037/0021-9010.81.2.170 CrossRefGoogle Scholar
  4. Berman, G.L., Narby, D.J., Cutler, B.L.: Effects of inconsistent eyewitness statements on mock-jurors’ evaluations of the eyewitness, perceptions of defendant culpability and verdicts. Law Hum. Behav. 19(1), 79–88 (1995)CrossRefGoogle Scholar
  5. Bothwell, R.K., Deffenbacher, K.A., Brigham, J.C.: Correlation of eyewitness accuracy and confidence: optimality hypothesis revisited. J. Appl. Psychol. 72(4), 691–695 (1987). doi: 10.1037//0882-7974.14.4.671 CrossRefGoogle Scholar
  6. Brewer, N., Burke, A.: Effects of testimonial inconsistencies and eyewitness confidence on mock-juror judgments. Law Hum. Behav. 26(3), 353–364 (2002). doi: 10.1023/a:1015380522722 CrossRefGoogle Scholar
  7. Brewer, N., Potter, R., Fisher, R.P., Bond, N., Luszcz, M.A.: Beliefs and data on the relationship between consistency and accuracy of eyewitness testimony. Appl. Cognit. Psychol 13(4), 297–313 (1999). doi: 10.1002/(sici)1099-0720(199908)13:4<297:aid-acp578>;2-s
  8. Brewer, N., Wells, G.L.: The confidence-accuracy relationship in eyewitness identification: effects of lineup instructions, foil similarity, and target-absent base rates. J. Exp. Psychol. Appl. 12(1), 11–30 (2006). doi: 10.1037/1076-898x.12.1.11 CrossRefGoogle Scholar
  9. Castelli, P., Goodman, G. S., Edelstein, R. S., Mitchell, E. B., Alonso, P. M. P., Lyons, K. E., & Newton, J. W.: Evaluating eyewitness testimony in adults and children. In: Weiner, I. B., Hess, A. K. (eds.) The handbook of forensic psychology, 3rd ed. (pp. 243–304). Wiley, Hoboken, NJ (2006)Google Scholar
  10. Colwell, K., Hiscock, C.K., Memon, A.: Interviewing techniques and the assessment of statement credibility. Appl. Cognit. Psychol. 16(3), 287–300 (2002)CrossRefGoogle Scholar
  11. Cowan, N., AuBuchon, A.M.: Short-term memory loss over time without retroactive stimulus interference. Psychon. Bull. Rev. 15(1), 230–235 (2008). doi: 10.3758/pbr.15.1.230 CrossRefGoogle Scholar
  12. Ebbinghaus, H.: Memory: a contribution to experimental psychology. Retrieved from
  13. Godoy-Cervera, V., Higueras, L.: Criteria-based content analysis (CBCA) in statement credibility assessment. Papeles del Psicologo 26, 92–98 (2005)Google Scholar
  14. Hofmann, T. :Probabilistic latent semantic indexing. Paper presented at the proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, Berkeley (1999)Google Scholar
  15. Hollin, C.R., Clifford, B.R.: Eyewitness testimony: the effects of discussion on recall accuracy and agreement. J. Appl. Soc. Psychol. 13(3), 234–244 (1983)CrossRefGoogle Scholar
  16. Juslin, P., Olsson, N., Winman, A.: Calibration and diagnosticity of confidence in eyewitness identification: comments on what can be inferred from the low confidence-accuracy correlation. J. Exp. Psychol. 22(5), 1304–1316 (1996). doi: 10.1037/0278-7393.22.5.1304 Google Scholar
  17. Karlsson, K., Sikström, S., Willander, J.: The semantic representation of event information depends on the cue-modality: the organization and selection of event information revisited. Plos One 8, 1–8 (2013)Google Scholar
  18. Koriat, A., Goldsmith, M.: Monitoring and control processes in the strategic regulation of memory accuracy. Psychol. Rev. 103(3), 490–517 (1996)CrossRefGoogle Scholar
  19. Kulkofsky, S.: Credible but inaccurate: Can Criterion-Based Content Analysis (CBCA) distinguish true and false memories? In: Smith, M. J. (ed.) Child sexual abuse: Issues and challenges, (pp. 21–42). Nova Science Publishers, Hauppauge, NJ (2008)Google Scholar
  20. Landauer, T.K.: LSA as a theory of meaning. In: Landauer, T.K., McNamara, D.S., Dennis, S., Kintsch, W. (eds.) Latent Semantic Analysis. Lawrence Erlbaum Associates Inc., Hillsdale (2007)Google Scholar
  21. Landauer, T.K., Dumais, S.T.: A solution to Plato’s problem: the Latent Semanctic Analysis Theory of the acquisition, induction, and representation of knowledge. Psychol. Rev. 104, 211–240 (1997)CrossRefGoogle Scholar
  22. Luus, C.A.E., Wells, G.L.: The malleability of eyewitness confidence: co-witness and perserverance effects. J. Appl. Psychol. 79(5), 714–724 (1994)CrossRefGoogle Scholar
  23. Marsh, E.J.: Retelling is not the same as recalling: Implications for memory. Curr. Dir. Psychol. Sci. 16(1), 16–20 (2007). doi: 10.1111/j.1467-8721.2007.00467.x CrossRefGoogle Scholar
  24. Marsh, E.J., Tversky, B., Hutson, M.: How eyewitnesses talk about events: implications for memory. Appl. Cognit. Psychol. 19(5), 531–544 (2005)CrossRefGoogle Scholar
  25. Pennebaker, J.W., Francis, M.E., Booth, R.J.: Linguistic Inquiry and Word Count (LIWC). Lawrence Erlbaum Associates, Mahwah, NJ (2011)Google Scholar
  26. Roll, M., Mårtensson, F., Sikström, S., Apt, P., Arnling-Bååth, R., Horne, M.: Atypical associations to abstract words in Broca’s aphasia. Cortex 48(8), 1068–1072 (2012). doi: 10.1016/j.cortex.2011.11.009 CrossRefGoogle Scholar
  27. Ruby, C.L., Brigham, J.C.: The usefulness of the criteria-based content analysis technique in distinguishing between truthful and fabricated allegations: a critical review. Psychol. Public Policy Law 3(4), 705–737 (1997)CrossRefGoogle Scholar
  28. Sahlgren, M.: An Introduction to Random Indexing. Paper presented at the SICS2007, Stockholm university, Stockholm. p. 1–9.(2007)Google Scholar
  29. Sarwar, F., Allwood, C.M., Innes-Ker, Å.: Effects of communication with a non-witness on eyewitnesses’ recall correctness and meta-cognitive realism. Appl. Cognit. Psychol. 25(5), 782–791 (2011). doi: 10.1002/acp.1749 CrossRefGoogle Scholar
  30. Shaw III, J.S., Garven, S., Wood, J.M.: Co-witness information can have immediate effects on eyewitness memory reports. Law Hum. Behav. 21(5), 503–523 (1997)CrossRefGoogle Scholar
  31. Shaw III, J.S., McClure, K.A.: Repeated postevent questioning can lead to elevated levels of eyewitness confidence. Law Hum. Behav. 20(6), 629–653 (1996). doi: 10.1007/BF01499235 CrossRefGoogle Scholar
  32. Sporer, S.L., Penrod, S., Read, D., Cutler, B.: Choosing, confidence, and accuracy: a meta-analysis of the confidence accuracy relation in eyewitness identification studies. Psychol. Bull. 118(3), 315–327 (1995)CrossRefGoogle Scholar
  33. Undeutsch, U.: Statement reality analysis. In: Trankell, A. (ed.) Reconstructing the past: the role of psychologists in criminal trials. Norstedt, Stockholm (1982)Google Scholar
  34. Undeutsch, U.: Courtroom evaluation of eyewitness testimony. Appl. Psychol. 33(1), 51–66 (1984). doi: 10.1111/j.1464-0597.1984.tb01416.x CrossRefGoogle Scholar
  35. Weingardt, K.R., Leonesio, R.J., Loftus, E.F.: Viewing eyewitness research from a metacognitive perspective. In: Metcalfe, J., Shimamura, A.P. (eds.) Metacognition: Knowing About Knowing, pp. 155–184. MIT Press, Cambridge (1994)Google Scholar
  36. Wells, G.L.: What do we know about eyewitness identification? Am Psychol. 48(5), 553–571 (1993)CrossRefGoogle Scholar
  37. Wells, G.L., Lindsay, R.C.L., Ferguson, T.J.: Accuracy, confidence, and juror perceptions in eyewitness identification. J. Appl. Psychol. 64(4), 440–448 (1979). doi: 10.1037/0021-9010.64.4.440 CrossRefGoogle Scholar
  38. Wells, G.L., Olson, E.A.: Eyewitness testimony. Annu. Rev. Psychol. 54(1), 277–295 (2003)CrossRefGoogle Scholar
  39. Wells, G.L., Olson, E.A., Charman, S.D.: The confidence of eyewitnesses in their identifications from lineups. Curr. Dir. Psychol. Sci. 11(5), 151–154 (2002). doi: 10.1111/1467-8721.00189 CrossRefGoogle Scholar
  40. Wise, R.A., Pawlenko, N.B., Safer, M.A., Meyer, D.: What US prosecutors and defence attorneys know and believe about eyewitness testimony. Applied Cognitive Psychology 23(9), 1266–1281 (2009). doi: 10.1002/acp.1530 CrossRefGoogle Scholar
  41. Vrij, A.: Detecting Lies and Deceit: Pitfalls and Opportunities, 2nd edn. Wiley, Chichester (2008)Google Scholar
  42. Vrij, A., Akehurst, L., Soukara, S., Bull, R.: Will the truth come out? The effect of deception, age, status, coaching, and social skills on CBCA scores. Law Hum. Behav. 26(3), 261–283 (2002). doi: 10.1023/a:1015313120905 CrossRefGoogle Scholar
  43. Yarmey, A.D.: The effects of dyadic discussion on earwitness recall. Basic Appl. Soc. Psychol. 13(2), 251–263 (1992)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  • Farhan Sarwar
    • 1
    Email author
  • Sverker Sikström
    • 1
  • Carl Martin Allwood
    • 2
  • Åse Innes-Ker
    • 1
  1. 1.Department of PsychologyLund UniversityLundSweden
  2. 2.Department of PsychologyUniversity of GothenburgGothenburgSweden

Personalised recommendations