Skip to main content

Imaging Facial Physiology for the Detection of Deceit

Abstract

Previous work has demonstrated the correlation of increased blood perfusion in the orbital muscles and stress levels for human beings. It has also been suggested that this periorbital perfusion can be quantified through the processing of thermal video. The idea has been based on the fact that skin temperature is heavily modulated by superficial blood flow. Proof of this concept was established for two different types of stress inducing experiments: startle experiments and mock-crime polygraph interrogations. However, the polygraph interrogation scenarios were simplistic and highly constrained. In the present paper, we report results derived from a large and realistic mock-crime interrogation experiment. The interrogation is free flowing and no restrictions have been placed on the subjects. Additionally, we propose a new methodology to compute the mean periorbital temperature signal. The present approach addresses the deficiencies of the earlier methodology and is capable of coping with the challenges posed by the realistic setting. Specifically, it features a tandem CONDENSATION tracker to register the periorbital area in the context of a moving face. It operates on the raw temperature signal and tries to improve the information content by suppressing the noise level instead of amplifying the signal as a whole. Finally, a pattern recognition method classifies stressful (Deceptive) from non-stressful (Non-Deceptive) subjects based on a comparative measure between the entire interrogation signal (baseline) and a critical subsection of it (transient response). The successful classification rate is 87.2% for 39 subjects. This is on par with the success rate achieved by highly trained psycho-physiological experts and opens the way for automating lie detection in realistic settings.

This is a preview of subscription content, access via your institution.

References

  • Baker, S. and Matthews, I. 2001. Equivalence and efficiency of image alignment algorithms. In Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 1090–1097.

  • Cooley, J.W. and Tukey, J.W. 1965. An algorithm for the machine calculation of complex Fourier series. Mathematics of Computation, 19:297–301.

    MATH  MathSciNet  Article  Google Scholar 

  • Committee to Review the Scientific Evidence on the Polygraph—Board on Behavioral, Cognitive, and Sensory Sciences and Committee on National Statistics—Division of Behavioral and Social Sciences and Education—National Research Council of the National Academies. 2003. The Polygraph and Lie Detection, National Academies Press, Washington D.C.

  • Doucet, A., DeFreitas, N., and Gordon, N. (Eds.) 2001. Sequential Monte Carlo Methods in Practice. Springer-Verlag.

  • Drummond, P. and Lance, J. 1987. Facial flushing and sweating mediated by the sympathetic nervous system. Brain, 110:793–803.

    Google Scholar 

  • Dupros, F., Fitzgibbon, W.E., and Garbey, M. 2004. shape A filtering technique for system of reaction diffusion equations. Technical Report UH-CS-04-02, Department of Computer Science, University of Houston.

  • Ekman, P. 1986. Telling Lies. Berkeley Publishing Group.

  • Ekman, P. and Maureen, O. 1991. Who can catch a liar? American Psychologist, 46(9):913–20.

    Article  Google Scholar 

  • Ekman, P. 1996. Why don’t we catch liars? Social Research, 63(3):801–17.

    Google Scholar 

  • Ekman, P. 2001. Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage, 2nd ed, W.W. Norton and Company.

  • Eveland, C., Socolinsky, D., and Wolff, L. July 2003. Tracking human faces in infrared video. Image and Vision Computing, 21:578–590.

    Google Scholar 

  • Frank, M.G. and Ekman, P. 1997. The ability to detect deceit generalizes across different types of high-stake lies. Personality and Social Psychology, 72:1429–1439.

    Article  Google Scholar 

  • Garbey, M., Merla, A., and Pavlidis, I. 2004. Estimation of blood flow speed and vessel location from thermal video. In Proceedings of the 2004 IEEE Computer Vision and Pattern Recognition, Vol. 1, pp. 356–63.

    Google Scholar 

  • Inbau, F.E., Reid, J.E., Buckley, J.P., and Jayne, B.C. 2001. Criminal Interrogations and Confessions. Aspen Publishers, 4th edn.

  • Isard, M. and Black, A. 1998a. Condensation—conditional density propagation for visual tracking. International Journal of Computer Vision, 19(1):5–28.

    Article  Google Scholar 

  • Isard, M. and Blake, A. June 1998b. ICONDENSATION: Unifying low-level and high-level tracking in a stochastic framework. In Proceedings of the 5th European Conference on Computer Vision, Freiburg, Germany, Vol. 1, pp. 893–908.

  • Isard, M. and MacCormick, J. 2001. BraMBle: A Bayesian multiple-blob tracker. In Proceedings of the 8th International Conference on Computer Vision, Vol. 2, pp. 34–41.

    Google Scholar 

  • Kleinmuntz, B. and Szucko, J.J. 1984. Lie detection in ancient and modern times. American Psychologist, 39:766–776.

    Article  Google Scholar 

  • Knight, J. April 15 2004. The truth about lying. Nature, 428:692–94.

    Article  Google Scholar 

  • Kolsch, M. and Turk, M. 2004. Fast 2D hand tracking with flocks of features and multi-cue integration. In Proceedings of the 2004 IEEE Conference on Computer Vision and Pattern Recognition, Vol. 10, pp. 158–164.

    Google Scholar 

  • MacCormick, J. and Isard, M. 2000. Partitioned sampling, articulated objects, and interface-quality hand tracking. In Lecture Notes in Computer Science, Springer-Verlag, London, U.K., Vol. 1843, pp. 3–19.

    Google Scholar 

  • MacCormick, J. and Blake, A. 2000. A probabilistic exclusion principle for tracking multiple objects. International Journal of Computer Vision, 39(1):57–71.

    MATH  Article  Google Scholar 

  • Martini, F.H., Ober, W.C., Garrison C.W., Welch, K., and Hutchings, R.T. 2001. Fundamentals of Anatomy and Physiology, Prentice Hall, 5th edn, Upper Saddle River, New Jersey.

    Google Scholar 

  • Matthews, I., Ishikawa, T., and Baker, S. June 2004. The template update problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(6):810–815.

    Article  Google Scholar 

  • Pavlidis, I., Levine, J., and Baukol, P. June 16 2000. Thermal imaging for anxiety detection. In Proceedings of the 2000 IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications, Hilton Head Island, South Carolina, pp. 104–109.

    Google Scholar 

  • Pavlidis, I., Eberhardt, N.L., and Levine, J. January 3 2002. Human behavior: Seeing through the face of deception. Nature, 415(6867):35.

    Article  Google Scholar 

  • Pavlidis, I. and Levine, J. November/December 2002. Thermal image analysis for polygraph testing. IEEE Engineering in Medicine and Biology Magazine, 21(6):56–64.

    Article  Google Scholar 

  • Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.F. 1992. Numerical Recipes in C. 2nd edn. Cambridge University Press, New York, New York.

    Google Scholar 

  • Smith, S.W. 1999. The Scientist’s and Engineer’s Guide to Digital Signal Processing. 2nd edn. California Technical Publishing, San Diego, California.

    Google Scholar 

  • Sokolov, E.N. and Cacioppo, J.T. 1997. Orienting and defense reflexes: Vector coding the cardiac response. In Attention and Orienting: Sensory and Motivational Processes, P.J. Lang, and R.F. Simons, (Ed.) Lawrence Erlbaum Associates, Mahwah, New Jersey, pp. 1–22.

    Google Scholar 

  • Verma, R., Schmid, C., and Mikolajczyk, K. October 2003. Face detection and tracking in a video by propagating detection probabilities, IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10):1215–28.

    Article  Google Scholar 

  • Yankee, W.J. 1965. An investigation of sphygmomanometer discomfort thresholds in polygraph examinations. Police, 9:12–18.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to P. Tsiamyrtzis.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Tsiamyrtzis, P., Dowdall, J., Shastri, D. et al. Imaging Facial Physiology for the Detection of Deceit. Int J Comput Vision 71, 197–214 (2007). https://doi.org/10.1007/s11263-006-6106-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-006-6106-y

Keywords

  • deception detection
  • polygraph
  • thermography
  • facial tracking
  • contact-free physiological monitoring