Artificial Intelligence and Law

, Volume 21, Issue 3, pp 303–340 | Cite as

Automatic deception detection in Italian court cases

Article

Abstract

Effective methods for evaluating the reliability of statements issued by witnesses and defendants in hearings would be an extremely valuable support to decision-making in court and other legal settings. In recent years, methods relying on stylometric techniques have proven most successful for this task; but few such methods have been tested with language collected in real-life situations of high-stakes deception, and therefore their usefulness outside lab conditions still has to be properly assessed. In this study we report the results obtained by using stylometric techniques to identify deceptive statements in a corpus of hearings collected in Italian courts. The defendants at these hearings were condemned for calumny or false testimony, so the falsity of (some of) their statements is fairly certain. In our experiments we replicated the methods used in previous studies but never before applied to high-stakes data, and tested new methods. We also considered the effect of a number of variables including in particular the homogeneity of the dataset. Our results suggest that accuracy at deception detection clearly above chance level can be obtained with real-life data as well.

Keywords

Deception detection Stylometry Criminal proceedings 

References

  1. Adams SH (1996) Statement analysis: what do suspects’ words really reveal? FBI Law Enforc Bull 65(10):12–20Google Scholar
  2. Alparone F, Caso S, Agosti A, Rellini A (2004) The Italian LIWC2001 dictionary. LIWC.net, AustinGoogle Scholar
  3. Artstein R, Poesio M (2008) Inter-coder agreement for computational linguistics. Comput Linguist 34(4):555–596CrossRefGoogle Scholar
  4. Bachenko J, Fitzpatrick E, Schonwetter M (2008) Verification and implementation of language-based deception indicators in civil and criminal narratives. In: Proceedings of the 22nd international conference on computational Linguistics—volume 1, COLING ‘08, pp 41–48, Stroudsburg, PA, USA. Association for Computational LinguisticsGoogle Scholar
  5. Bond CF, De Paulo BM (2006) Accuracy of deception judgments. Pers Soc Psychol Rev 10(3):214–234CrossRefGoogle Scholar
  6. Buller D, Burgoon J (1996) Interpersonal deception theory. Commun Theory 6:203–242CrossRefGoogle Scholar
  7. Chinchor N (1992) Muc-4 evaluation metrics. In: Proceedings of the 4th conference on message understanding, MUC4 ’92, pp 22–29, Stroudsburg, PA, USA. Association for Computational LinguisticsGoogle Scholar
  8. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297Google Scholar
  9. Coulthard M (2004) Author identification, idiolect, and linguistic uniqueness. Appl Linguist 25(4):431–447CrossRefGoogle Scholar
  10. Davatzikos C, Ruparel K, Fan Y, Shen D, Acharyya M, Loughead J, Gur R, Langleben D (2005) Classifying spatial patterns of brain activity with machine learning methods: application to lie detection. NeuroImage 28(3):663–668CrossRefGoogle Scholar
  11. De Paulo BM, Lindsay JJ, Malone BE, Muhlenbruck L, Charlton K, Cooper H (2003) Cues to deception. Psychol Bull 129(1):74–118CrossRefGoogle Scholar
  12. Ekman P (2001) Telling lies: clues to deceit in the marketplace, politics, and marriage. W.W. NortonGoogle Scholar
  13. Feng S, Banerjee R, Choi Y (2012) Syntactic stylometry for deception detection. In: Proceedings of the 50th annual meeting of the association for computational linguistics (volume 2: Short Papers), pp 171–175, Jeju Island, Korea. Association for Computational LinguisticsGoogle Scholar
  14. Fitzpatrick E, Bachenko J (2009) Building a forensic corpus to test language-based indicators of deception. Lang Comput 71(1):183–196Google Scholar
  15. Fitzpatrick E, Bachenko J (2012) Building a data collection for deception research. In: Proceedings of the workshop on computational approaches to deception detection, pp 31–38, Avignon, France. Association for Computational LinguisticsGoogle Scholar
  16. Forman G (2003) An extensive empirical study of feature selection metrics for text classification. J Mach Learn Res 3:1289–1305MATHGoogle Scholar
  17. Fornaciari T, Poesio M (2011) Sincere and deceptive statements in Italian criminal proceedings. In: Proceedings of the international association of forensic linguists 10th biennial conference, pp 126–138, Cardiff, Wales, UKGoogle Scholar
  18. Frank MG, Feeley TH (2003) To catch a liar: challenges for research in lie detection training. J Appl Commun Res 31(1):58–75CrossRefGoogle Scholar
  19. Frank MG, Menasco MA, O’Sullivan M (2008) Human behavior and deception detection. In: Voeller JG (ed) Wiley handbook of science and technology for homeland security. Wiley, New YorkGoogle Scholar
  20. Ganis G, Kosslyn S, Stose S, Thompson W, Yurgelun-Todd D (2003) Neural correlates of different types of deception: an fMRI investigation. Cereb Cortex 13(8):830–836CrossRefGoogle Scholar
  21. Giannone C, Basili R, Del Vescovo C, Naggar P, Moschitti A (2009) Kernel-based relation extraction from investigative data. In: Proceedings of the third workshop on analytics for noisy unstructured text data, AND ’09, pp 93–100, New York, NY, USA. ACMGoogle Scholar
  22. Gokhmann S, Hancock J, Prabhu P, Ott M, Cardie C (2012) In search of a gold standard in studies of deception. In: Fitzpatrick E, Bachenko J, Fornaciari T (eds) Proceedings of the EACL workshop on computational approaches to deception detection, pp 23–30Google Scholar
  23. Hancock JT, Curry LE, Goorha S, Woodworth M (2008) On lying and being lied to: a linguistic analysis of deception in computer-mediated communication. Discourse Process 45(1):1–23CrossRefGoogle Scholar
  24. Hauch V, Blandón-Gitlin I, Masip J, Sporer SL (2012) Linguistic cues to deception assessed by computer programs: a meta-analysis. In: Fitzpatrick E, Bachenko J, Fornaciari T (eds) Proceedings of the workshop on computational approaches to deception detection, pp 1–4, AvignonGoogle Scholar
  25. Ireland ME, Slatcher RB, Eastwick PW, Scissors LE, Finkel EJ, Pennebaker JW (2011) Language style matching predicts relationship initiation and stability. Psychol Sci 22(1):39–44CrossRefGoogle Scholar
  26. Jensen ML, Meservy TO, Burgoon JK, Nunamaker JF (2010) Automatic, multimodal evaluation of human interaction. Group Decis Negot 19(4):367–389CrossRefGoogle Scholar
  27. Karatzoglou A, Meyer D, Hornik K (2006) Support vector machines in r. J Stat Softw 15(9):1–28Google Scholar
  28. Koppel M, Schler J, Argamon S, Pennebaker J (2006) Effects of age and gender on blogging. In: AAAI 2006 spring symposium on computational approaches to analysing weblogsGoogle Scholar
  29. Levine TR, Feeley TH, McCornack SA, Hughes M, Harms CM (2005) Testing the effects of nonverbal behavior training on accuracy in deception detection with the inclusion of a bogus training control group. West J Commun 69(3):203–217CrossRefGoogle Scholar
  30. Lord RD (1958) Studies in the history of probability and statistics.: Viii. de morgan and the statistical study of literary style. Biometrika 45(1/2):282–282CrossRefGoogle Scholar
  31. Lutoslawski W (1898) Principes de stylomtrie. Revue des tudes grecques 41:61–81Google Scholar
  32. Luyckx K, Daelemans W (2008) Authorship attribution and verification with many authors and limited data. In: Proceedings of the 22nd international conference on computational linguistics—volume 1, COLING ’08, pp 513–520, Stroudsburg, PA, USA. Association for Computational LinguisticsGoogle Scholar
  33. Merikangas JR (2008) Commentary: functional mri lie detection. J Am Acad Psychiatry Law 36(4):499–501Google Scholar
  34. Mosteller F, Wallace D (1964) Inference and disputed authorship: the federalist. Addison-Wesley, ReadingMATHGoogle Scholar
  35. Newman ML, Pennebaker JW, Berry DS, Richards JM (2003) Lying words: predicting deception from linguistic styles. Pers Soc Psychol Bull 29(5):665–675CrossRefGoogle Scholar
  36. Niederhoffer KG, Pennebaker JW (2002) Linguistic style matching in social interaction. J Lang Soc Psychol 21(4):337–360CrossRefGoogle Scholar
  37. Peersman C, Daelemans W, Van Vaerenbergh L (2011) Age and gender prediction on netlog data. Presented at the 21st Meeting of Computational Linguistics in the Netherlands (CLIN21), Ghent, Belgium.Google Scholar
  38. Pennebaker JW, Francis ME, Booth RJ (2001) Linguistic inquiry and word count (LIWC): LIWC2001. Lawrence Erlbaum Associates, MahwahGoogle Scholar
  39. Pepe G (ed) (1996) La falsa donazione di Costantino. Tea storica. TEAGoogle Scholar
  40. Porter S, Woodworth M, Birt AR (2000) Truth, lies, and videotape: an investigation of the ability of federal parole officers to detect deception. Law Hum Behav 24(6):643–658CrossRefGoogle Scholar
  41. Sasaki Y (2007) The truth of the F-measure. Teach Tutor mater, pp 1–5Google Scholar
  42. Schmid H (1994) Probabilistic part-of-speech tagging using decision trees. In: Proceedings of international conference on new methods in language processingGoogle Scholar
  43. Simpson JR (2008) Functional mri lie detection: too good to be true? J Am Acad Psychiatry Law 36(4):491–498Google Scholar
  44. Solan LM, Tiersma PM (2004) Author identification in american courts. Appl Linguist 25(4):448–465CrossRefGoogle Scholar
  45. Stein B, Koppel M, Stamatatos E (2007) Plagiarism analysis, authorship identification, and near-duplicate detection pan’07. SIGIR Forum 41:68–71CrossRefGoogle Scholar
  46. Strapparava C, Mihalcea R (2009) The lie detector: explorations in the automatic recognition of deceptive language. In: Proceeding ACLShort ’09—proceedings of the ACL-IJCNLP 2009 conference short papersGoogle Scholar
  47. Undeutsch U (1967) Beurteilung der Glaubhaftigkeit von Aussagen [Veracity assessment of statements]. In: Undeutsch U (ed) Handbuch der psychologie: vol 11. Forensische Psychologie. Hogrefe, Gottingen, pp 26–181Google Scholar
  48. Undeutsch U (1982) Statement reality analysis. In: Trankell A (ed) Reconstructing the past: the role of psychologists in criminal trials. Kluwer, Deventer, pp 27–56Google Scholar
  49. Undeutsch U (1984) Courtroom evaluation of eyewitness testimony. Appl Psychol 33(1):51–66CrossRefGoogle Scholar
  50. Vaassen F, Daelemans W (2011) Automatic emotion classification for interpersonal communication. In: 2nd workshop on computational approaches to subjectivity and sentiment analysis (WASSA 2.011)Google Scholar
  51. Vrij A (2008) Detecting lies and deceit: pitfalls and opportunities. Wiley series in psychology of crime, policing and law, 2nd edition. Wiley, ChichesterGoogle Scholar
  52. Vrji A (2005) Criteria-based content analysis—a qualitative review of the first 37 studies. Psychol Public Policy Law 11(1):3–41CrossRefGoogle Scholar
  53. Walczyk JJ, Roper KS, Seemann E, Humphrey AM (2003) Cognitive mechanisms underlying lying to questions: response time as a cue to deception. Appl Cogn Psychol 17(7):755–774CrossRefGoogle Scholar
  54. Wang JT, Spezio M, Camerer CF (2010) Pinocchio’s pupil: using eyetracking and pupil dilation to understand truth telling and deception in sender-receiver games. Am Econ Rev 100(3):984–1007CrossRefGoogle Scholar
  55. Yang Y, Liu X (1999) A re-examination of text categorization methods. In: Proceedings of the 22nd annual international ACM SIGIR conference on research and development in information retrieval, SIGIR ’99. ACM, New York, pp 42–49Google Scholar
  56. Yang Y, Pedersen JO (1997) A comparative study on feature selection in text categorization. CiteSeerX—Scientific Literature Digital Library and Search Engine [http://citeseerx.ist.psu.edu/oai2] (United States)
  57. Zhou L, Shi Y, Zhang D (2008) A statistical language modeling approach to online deception detection. IEEE Trans Knowl Data Eng 20(8):1077–1081CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  1. 1.Center for Mind/Brain SciencesUniversity of TrentoTrentoItaly
  2. 2.School for Computer Science and Electronic EngineeringUniversity of EssexColchesterUK

Personalised recommendations