Abstract
We suggest a reconsideration of broad and sweeping claims that research has demonstrated that nonverbal behavior (NVB) are not indicators of deception. We reexamine several methodological characteristics of a seminal meta-analysis that is often cited as non-evidence and caution the field from drawing overgeneralized conclusions about the role of NVB as indicators of deception based on that reexamination. We review studies since then that have provided ample evidence for NVB to provide such information, and then offer additional considerations about this topic to provide additional context. Our views are based on the nature of evolved human communication mechanisms, the complexity of mental contents, and differences between unconscious and conscious mediation of behavior and words.
This is a preview of subscription content, log in via an institution to check access.
Source: Matsumoto and Hwang (2018b). Note: Error bars are SEs. The Y-axis refers to combined number of coded negative facial expressions of emotions (i.e., combined frequencies of anger, contempt, disgust, fear, and sad expressions)
Similar content being viewed by others
Notes
Hartwig and Bond (2014) conducted a meta-analysis and concluded that contextual issues such as those we raise here did not affect the ability for NVB to differentiate truthtellers from liars. Their meta-analysis, however, focused on an analysis of multiple NVB cues in a multi-channel, multimodal approach, which is different than what DePaulo et al. (2003) did and which we discuss more below.
To be sure, examining multiple NVB as predictors of any mental state will produce better fitting models than examining single NVB because of the larger number of predictors, raising reasonable concerns about increased Type I error. This problem is compounded when sample sizes do not have sufficient power and findings based on many multivariate procedures are optimized for individual data sets, which leads to concerns about replicability and generalizability. Some procedures exist to mitigate but not necessarily eliminate such concerns (e.g., recomputing analyses using randomized data). Matsumoto and Wilson (2023) discuss this and other methodological issues associated with examinations of multiple NVB.
As with NVB, abundant caution should be applied when considering the diagnosticity of cognitive cues to deception. For instance, in discussing a review of these cues by Vrij and Granhag (2012), Frank and Svetieva (2012) wrote (p. 132):
“The accuracy results for the studies cited by V&G are not very different from accuracy levels reported in the traditional deception literature. For example, the ‘telling the story backward’ technique produced 58% overall accuracy, versus 46% for the control (Vrij et al., 2008); asking a person to ‘look you in the eye’ was not diagnostic when judges saw audiovisual presentations of the lies and truths, but was diagnostic when judges heard audio only presentations—but again that difference was meek—54% versus 50% (Vrij et al., 2010). Likewise, when asked to draw an image related to things seen by a truth teller but not a liar, the unanticipated questions showed no discriminability between liars and truth tellers in their spoken accounts, although the drawings they produced of the scene were strongly diagnostic (over 80%; Vrij et al., 2009)”.
The notion of a one-to-one correspondence between a behavioral indicator of deception itself may be a straw-person ala Pinocchio. Our position is that such indicators exist but none with a one-to-one correspondence with veracity or deception.
References
Abouelenien, M., Perez-Rosas, V., Mihalcea, R., & Burzo, M. (2014). Deception detection using a multimodal approach. In Proceedings of the 16th International Conference on Multimodal Interaction, Istanbul.
Abouelenien, M., Pérez-Rosas, V., Zhao, B., Mihalcea, R., & Burzo, M. (2017). Gender-based multimodal deception detection. In Proceedings of the Symposium on Applied Computing, Marrakech, Morocco. https://doi.org/10.1145/3019612.3019644
Burzo, M., Abouelenien, M., Perez-Rosas, V., & Mihalcea, R. (2018). Multimodal deception detection. In The handbook of multimodal-multisensor interfaces: Signal processing, architectures, and detection of emotion and cognition (vol. 2, pp. 419–453). Association for Computing Machinery and Morgan https://doi.org/10.1145/3107990.3108005
Cartmill, E. A., & Goldin-Meadow, S. (2016). Gesture. In D. Matsumoto, H. C. Hwang, & M. G. Frank (Eds.), APA handbook of nonverbal communication (pp. 307–333). American Psychological Association.
Davis, M., Markus, K. A., Walters, S. B., Vorus, N., & Connors, B. (2005). Behavioral cues to deception vs. topic incriminating potential in criminal confessions. Law and Human Behavior, 29(6), 683–704. https://doi.org/10.1007/s10979-005-7370-z
DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003). Cues to deception. Psychological Bulletin, 129(1), 74–118. https://doi.org/10.1037/0033-2909.129.1.74
Diana, B., Elia, M., Zurloni, V., Elia, A., Maisto, A., & Pelosi, S. (2015). Multimodal deception detection: A t-pattern approach. In Proceedings of the 2015 ACM on Workshop on Multimodal Deception Detection, Seattle.
Dunbar, N. E., Jensen, M. L., Tower, D. C., & Burgoon, J. K. (2014). Synchronization of nonverbal behaviors in detecting mediated and non-mediated deception. Journal of Nonverbal Behavior, 38(3), 355–376. https://doi.org/10.1007/s10919-014-0179-z
Duran, N., Dale, R., Kello, C., Street, C., & Richardson, D. (2013). Exploring the movement dynamics of deception. Frontiers in Psychology, 4, 140. https://doi.org/10.3389/fpsyg.2013.00140
Ekman, P. (1985). Telling lies: Clues to deceit in the marketplace, politics, and marriage (1st ed.). W. W. Norton & Co, Inc.
Ekman, P. (2009). Telling lies: Clues to deceit in the marketplace, politics, and marriage (4th ed.). W. W. Norton & Co, Inc.
Ekman, P., & Fridlund, A. J. (1987). Assessment of facial behavior in affective disorders. In J. D. Maser (Ed.), Depression and expressive behavior (pp. 37–56). Lawrence Erlbaum Associates.
Ekman, P., Friesen, W. V., & Ancoli, S. (1980). Facial signs of emotional experience. Journal of Personality and Social Psychology, 39, 1125–1134. https://doi.org/10.1037/h0077722
Ekman, P., Friesen, W. V., & O’Sullivan, M. (1988). Smiles when lying. Journal of Personality & Social Psychology, 54(3), 414–420. https://doi.org/10.1037/0022-3514.54.3.414
Ekman, P., Matsumoto, D., & Friesen, W. V. (1998). Facial expressions in affective disorders. In P. Ekman & E. Rosenberg (Eds.), What the face reveals: Basic and applied studied of spontaneous expression using the Facial Action Coding System (FACS) (pp. 331–341). Oxford University Press.
Ekman, P., O’Sullivan, M., Friesen, W. V., & Scherer, K. R. (1991). Invited article: Face, voice, and body in detecting deceit. Journal of Nonverbal Behavior, 15(2), 125–135. https://doi.org/10.1007/BF00998267
Fiedler, K., & Walka, I. (1993). Training lie detectors to use nonverbal cues instead of global heuristics. Human Communication Research, 20(2), 199–223. https://doi.org/10.1111/j.1468-2958.1993.tb00321.x
Frank, M. G., Ekman, P., & Friesen, W. V. (1993). Behavioral markers and recognizability of the smile of enjoyment. Journal of Personality & Social Psychology, 64(1), 83–93. https://doi.org/10.1037/0022-3514.64.1.83
Frank, M. G., Maroulis, A., & Griffin, D. J. (2013). The voice. In D. Matsumoto, M. G. Frank, & H. S. Hwang (Eds.), Nonverbal communication: Science and applications (pp. 53–74). Sage Publications.
Frank, M. G., & Svetieva, E. (2012). Lies worth catching involve both emotion and cognition. Journal of Applied Research in Memory and Cognition, 1(2), 131–133. https://doi.org/10.1016/j.jarmac.2012.04.006
Hartwig, M., & Bond, C. F. (2014). Lie detection from multiple cues: A meta-analysis. Applied Cognitive Psychology, 28(5), 661–676. https://doi.org/10.1002/acp.3052
Hartwig, M., Granhag, P. A., Stromwall, L. A., & Vrij, A. (2005). Detecting deception via strategic disclosure of evidence. Law and Human Behavior, 29, 469–484. https://doi.org/10.1007/s10979-005-5521-x
Iverson, J. M., & Goldin-Meadow, S. (1998). Why people gesture when they speak. Nature, 396(19), 228. https://doi.org/10.1038/24300
Jensen, M. L., Meservy, T. O., Burgoon, J. K., & Nunamaker, J. F. (2010). Automatic, multimodal evaluation of human interaction. Group Decision and Negotiation, 19(4), 367–389. https://doi.org/10.1007/s10726-009-9171-0
Johnson, M. K. (1988). Reality monitoring: An experimental phenomenological approach. Journal of Experimental Psychology: General, 117(4), 390–394. https://doi.org/10.1037/0096-3445.117.4.390
Johnson, M. K., & Raye, C. L. (1981). Reality monitoring. Psychological Review, 88(1), 67–85. https://doi.org/10.1037/0033-295X.88.1.67
Koper, R. J., & Sahlman, J. M. (1991). The behavioral correlates of real-world deceptive communication. In Annual Meeting of the International Communication Association, Chicago.
Luke, T. J., Hartwig, M., Brimbal, L., Chan, G., Jordan, S., Joseph, E., Osborne, J., & Granhag, P. A. (2013). Interviewing to elicit cues to deception: Improving strategic use of evidence with general-to-specific framing of evidence. Journal of Police and Criminal Psychology, 28, 54–62. https://doi.org/10.1007/s11896-012-9113-7
Matsumoto, D., Frank, M. G., & Hwang, H. S. (2013). Nonverbal communication: Science and applications. Sage Publications.
Matsumoto, D., & Hwang, H. C. (2018a). Clusters of nonverbal behaviors differ according to type of question and veracity in investigative interviews in a mock crime context. Journal of Police and Criminal Psychology, 33(4), 302–315. https://doi.org/10.1007/s11896-017-9250-0
Matsumoto, D., & Hwang, H. C. (2018b). Microexpressions differentiate truths from lies about future malicious intent. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2018.02545
Matsumoto, D., & Hwang, H. C. (2020). Clusters of nonverbal behavior differentiate truths and lies about future malicious intent in checkpoint screening interviews. Psychiatry, Psychology and Law. https://doi.org/10.1080/13218719.2020.1794999
Matsumoto, D., Keltner, D., Shiota, M. N., O'Sullivan, M., & Frank, M. G. (2008). What’s in a face Facial expressions as signals of discrete emotions. In M. Lewis, J. M. Haviland, & L. Feldman Barrett (Eds.), Handbook of emotions (pp. 211–234). Guilford Press.
Matsumoto, D., & Wilson, M. (2023). Incorporating consciousness into an understanding of emotion and nonverbal behavior. Emotion Review. https://doi.org/10.1177/17540739231163177
Meissner, C. A. (2021). “What works?” Systematic reviews and meta-analyses of the investigative interviewing research literature. Applied Cognitive Psychology, 35(2), 322–328. https://doi.org/10.1002/acp.3808
Meservy, T. O., Jensen, M. L., Kruse, J., Burgoon, J. K., & Nunamaker, J. F. (2005a). Automatic extraction of deceptive behavioral cues from video. In P. Kantor, G. Muresan, F. Roberts, D. D. Zeng, F.-Y. Wang, H. Chen, & R. C. Merkle (Eds.), Intelligence and Security Informatics. International Conference on Intelligence and Security Informatics, Berlin, Heidelberg.
Meservy, T. O., Jensen, M. L., Kruse, J., Burgoon, J. K., Nunamaker, J. F., Twitchell, D. P., Tsechpenakis, G., & Metaxas, D. (2005b). Deception detection through automatic, unobtrusive analysis of nonverbal behavior. IEEE Intelligent Systems, 20(5), 36–43. https://doi.org/10.1109/MIS.2005.85
Patterson, M. L., Fridlund, A. J., & Crivelli, C. (2023). Four misconceptions about nonverbal communication. Perspectives on Psychological Science. https://doi.org/10.1177/17456916221148142
Pennebaker, J. W., & Chew, C. H. (1985). Behavioral inhibition and electrodermal activity during deception. Journal of Personality and Social Psychology, 49(5), 1427–1433. https://doi.org/10.1037/0022-3514.49.5.1427
Perez-Rosas, V., Abouelenien, M., Mihalcea, R., & Burzo, M. (2015). Deception detection using real-life trial data. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, Seattle.
Porter, S., & ten Brinke, L. (2008). Reading between the lies: Identifying concealed and falsified emotions in universal facial expressions. Psychological Science, 19(5), 508–514. https://doi.org/10.1111/j.1467-9280.2008.02116.x
Porter, S., ten Brinke, L., & Wallace, B. (2012). Secrets and lies: Involuntary leakage in deceptive facial expressions as a function of emotional intensity. Journal of Nonverbal Behavior, 36, 23–37. https://doi.org/10.1007/s10919-011-0120-7
Scott, S., & McGettigan, C. (2016). The voice: From identity to interactions. In D. Matsumoto, H. C. Hwang, & M. G. Frank (Eds.), APA handbook of nonverbal communication (pp. 289–306). American Psychological Association.
ten Brinke, L., MacDonald, S., Porter, S., & O’Connor, B. (2012a). Crocodile tears: Facial, verbal and body language behaviours associated with genuine and fabricated remorse. Law and Human Behavior, 36(1), 51–59. https://doi.org/10.1007/s10979-011-9265-5
ten Brinke, L., & Porter, S. (2012). Cry me a river: Identifying the behavioral consequences of extremely high-stakes interpersonal deception. Law and Human Behavior, 36(6), 469–477. https://doi.org/10.1037/h0093929
ten Brinke, L., Porter, S., & Baker, A. (2012b). Darwin the detective: Observable facial muscle contractions reveal emotional high-stakes lies. Evolution and Human Behavior, 33(4), 411–416. https://doi.org/10.1016/j.evolhumbehav.2011.12.003
Vrij, A. (2019). Deception and truth detection when analyzing nonverbal and verbal cues. Applied Cognitive Psychology, 33(2), 160–167. https://doi.org/10.1002/acp.3457
Vrij, A., Akehurst, L., Soukara, S., & Bull, R. (2006). Detecting deceit via analyses of verbal and nonverbal behavior in children and adults. Human Communication Research, 30(1), 8–41. https://doi.org/10.1111/j.1468-2958.2004.tb00723.x
Vrij, A., Edward, K., Roberts, K. P., & Bull, R. (2000). Detecting deceit via analysis of verbal and nonverbal behavior. Journal of Nonverbal Behavior, 24(4), 239–263. https://doi.org/10.1023/A:1006610329284
Vrij, A., & Granhag, P. A. (2012). Eliciting cues to deception and truth: What matters are the questions asked. Journal of Applied Research in Memory and Cognition, 1(2), 110–117. https://doi.org/10.1016/j.jarmac.2012.02.004
Vrij, A., Hartwig, M., & Granhag, P. A. (2019). Reading lies: Nonverbal communication and deception. Annual Review of Psychology, 70(1), 295–317. https://doi.org/10.1146/annurev-psych-010418-103135
Vrij, A., Leal, S., Granhag, P. A., Mann, S., Fisher, R. P., Hillman, J., & Sperry, K. (2009). Outsmarting the liars: The benefit of asking unanticipated questions. Law and Human Behavior, 33(2), 159–166. https://doi.org/10.1007/s10979-008-9143-y
Vrij, A., Mann, S., Leal, S., & Fisher, R. (2010). ‘Look into my eyes’: Can an instruction to maintain eye contact facilitate lie detection? Psychology, Crime & Law, 16(4), 327–348. https://doi.org/10.1080/10683160902740633
Vrij, A., Mann, S. A., Fisher, R. P., Leal, S., Milne, R., & Bull, R. (2008). Increasing cognitive load to facilitate lie detection: The benefit of recalling an event in reverse order. Law and Human Behavior, 32(3), 253–265. https://doi.org/10.1007/s10979-007-9103-y
Wright Whelan, C., Wagstaff, G. F., & Wheatcroft, J. M. (2014). High-stakes lies: Verbal and nonverbal cues to deception in public appeals for help with missing or murdered relatives. Psychiatry, Psychology and Law, 21(4), 523–537. https://doi.org/10.1080/13218719.2013.839931
Yu, X., Zhang, S., Yan, Z., Yang, F., Huang, J., Dunbar, N. E., Jensen, M. L., Burgoon, J. K., & Metaxas, D. N. (2015). Is interactional dissynchrony a clue to deception? Insights from automated analysis of nonverbal visual cues. IEEE Transactions on Cybernetics, 45(3), 492–506. https://doi.org/10.1109/TCYB.2014.2329673
Author information
Authors and Affiliations
Contributions
Both authors discussed the content of the paper. The first author then created the first draft of the paper, and both authors subsequently contributed equally to the revision of the manuscript to final.
Corresponding author
Ethics declarations
Conflict of interest
The author(s) are employees of Humintell, a company that engages in research and training related to behavioral indicators of mental states and deception.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Matsumoto, D., Wilson, M. Behavioral Indicators of Deception and Associated Mental States: Scientific Myths and Realities. J Nonverbal Behav 48, 11–23 (2024). https://doi.org/10.1007/s10919-023-00441-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10919-023-00441-w