From Text to Speech: A Multimodal Cross-Domain Approach for Deception Detection

  • Rodrigo Rill-García
  • Luis Villaseñor-Pineda
  • Verónica Reyes-Meza
  • Hugo Jair Escalante
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11188)


Deception detection -identifying when someone is trying to cause someone else to believe something that is not true- is a hard task for humans. The task is even harder for automatic approaches, that must deal with additional problems like the lack of enough labeled data. In this context, transfer learning in the form of cross-domain classification is a task that aims to leverage labeled data from certain domains for which labeled data is available to others for which data is scarce. This paper presents a study on the suitability of linguistic features for cross-domain deception detection on multimodal data. Specifically, we aim to learn models for deception detection across different domains of written texts (one modality) and apply the new knowledge to unrelated topics transcribed from spoken statements (another modality). Experimental results reveal that by using LIWC and POS n-grams we reach a in-modality accuracy of 69.42%, as well as an AUC ROC of 0.7153. When doing transfer learning, we achieve an accuracy of 63.64% and get an AUC ROC of 0.6351.


Linguistic analysis Cross-domain classification Multimodal data analysis Deception detection 


  1. 1.
    Pérez-Rosas, V., Mihalcea, R.: Cross-cultural deception detection. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 440–445. Association for Computational Linguistics (2014)Google Scholar
  2. 2.
    Ekman, P.: Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage, Revised edn. WW Norton & Company, New York City (2009)Google Scholar
  3. 3.
    Newman, M.L., Pennebaker, J.W., Berry, D.S., Richards, J.M.: Lying words: predicting deception from linguistic styles. Pers. Soc. Psychol. Bull. 29(5), 665–675 (2003)CrossRefGoogle Scholar
  4. 4.
    Pérez-Rosas, V., Abouelenien, M., Mihalcea, R., Burzo, M.: Deception detection using real-life trial data. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 59-66. ACM (2015)Google Scholar
  5. 5.
    Abouelenien, M., Pérez-Rosas, V., Zhao, B., Mihalcea, R., Burzo, M.: Detecting deceptive behavior via integration of discriminative features from multiple modalities. IEEE Trans. Inf. Forensics Secur. 12(5), 1042–1055 (2017)CrossRefGoogle Scholar
  6. 6.
    Abouelenien, M., Pérez-Rosas, V., Zhao, B., Mihalcea, R., Burzo, M.: Gender-based multimodal deception detection. In: Proceedings of the Symposium on Applied Computing (2017).
  7. 7.
    Wu, Z., Singh, B., Davis, L.S., Subrahmanian, V.: Deception detection in videos. arXiv preprint arXiv:1712.04415 (2017)
  8. 8.
    Sapkota, U., Bethard, S., Montes, M., Solorio, T.: Not all character N-grams are created equal: a study in authorship attribution. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 93-102 (2015)Google Scholar
  9. 9.
    Toutanova, K., Klein, D., Manning, C., Singer, Y.: Feature-rich part-of-speech tagging with a cyclic dependency network. In: Proceedings of HLT-NAACL 2003, pp. 252–259 (2003)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Rodrigo Rill-García
    • 1
  • Luis Villaseñor-Pineda
    • 1
  • Verónica Reyes-Meza
    • 2
  • Hugo Jair Escalante
    • 1
  1. 1.Instituto Nacional de Astrofísica, Óptica y ElectrónicaPueblaMexico
  2. 2.Centro Tlaxcala de Biología de la ConductaUniversidad Autónoma de TlaxcalaTlaxcalaMexico

Personalised recommendations