Abstract
Previous work has demonstrated the correlation of increased blood perfusion in the orbital muscles and stress levels for human beings. It has also been suggested that this periorbital perfusion can be quantified through the processing of thermal video. The idea has been based on the fact that skin temperature is heavily modulated by superficial blood flow. Proof of this concept was established for two different types of stress inducing experiments: startle experiments and mock-crime polygraph interrogations. However, the polygraph interrogation scenarios were simplistic and highly constrained. In the present paper, we report results derived from a large and realistic mock-crime interrogation experiment. The interrogation is free flowing and no restrictions have been placed on the subjects. Additionally, we propose a new methodology to compute the mean periorbital temperature signal. The present approach addresses the deficiencies of the earlier methodology and is capable of coping with the challenges posed by the realistic setting. Specifically, it features a tandem CONDENSATION tracker to register the periorbital area in the context of a moving face. It operates on the raw temperature signal and tries to improve the information content by suppressing the noise level instead of amplifying the signal as a whole. Finally, a pattern recognition method classifies stressful (Deceptive) from non-stressful (Non-Deceptive) subjects based on a comparative measure between the entire interrogation signal (baseline) and a critical subsection of it (transient response). The successful classification rate is 87.2% for 39 subjects. This is on par with the success rate achieved by highly trained psycho-physiological experts and opens the way for automating lie detection in realistic settings.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baker, S. and Matthews, I. 2001. Equivalence and efficiency of image alignment algorithms. In Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 1090–1097.
Cooley, J.W. and Tukey, J.W. 1965. An algorithm for the machine calculation of complex Fourier series. Mathematics of Computation, 19:297–301.
Committee to Review the Scientific Evidence on the Polygraph—Board on Behavioral, Cognitive, and Sensory Sciences and Committee on National Statistics—Division of Behavioral and Social Sciences and Education—National Research Council of the National Academies. 2003. The Polygraph and Lie Detection, National Academies Press, Washington D.C.
Doucet, A., DeFreitas, N., and Gordon, N. (Eds.) 2001. Sequential Monte Carlo Methods in Practice. Springer-Verlag.
Drummond, P. and Lance, J. 1987. Facial flushing and sweating mediated by the sympathetic nervous system. Brain, 110:793–803.
Dupros, F., Fitzgibbon, W.E., and Garbey, M. 2004. shape A filtering technique for system of reaction diffusion equations. Technical Report UH-CS-04-02, Department of Computer Science, University of Houston.
Ekman, P. 1986. Telling Lies. Berkeley Publishing Group.
Ekman, P. and Maureen, O. 1991. Who can catch a liar? American Psychologist, 46(9):913–20.
Ekman, P. 1996. Why don’t we catch liars? Social Research, 63(3):801–17.
Ekman, P. 2001. Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage, 2nd ed, W.W. Norton and Company.
Eveland, C., Socolinsky, D., and Wolff, L. July 2003. Tracking human faces in infrared video. Image and Vision Computing, 21:578–590.
Frank, M.G. and Ekman, P. 1997. The ability to detect deceit generalizes across different types of high-stake lies. Personality and Social Psychology, 72:1429–1439.
Garbey, M., Merla, A., and Pavlidis, I. 2004. Estimation of blood flow speed and vessel location from thermal video. In Proceedings of the 2004 IEEE Computer Vision and Pattern Recognition, Vol. 1, pp. 356–63.
Inbau, F.E., Reid, J.E., Buckley, J.P., and Jayne, B.C. 2001. Criminal Interrogations and Confessions. Aspen Publishers, 4th edn.
Isard, M. and Black, A. 1998a. Condensation—conditional density propagation for visual tracking. International Journal of Computer Vision, 19(1):5–28.
Isard, M. and Blake, A. June 1998b. ICONDENSATION: Unifying low-level and high-level tracking in a stochastic framework. In Proceedings of the 5th European Conference on Computer Vision, Freiburg, Germany, Vol. 1, pp. 893–908.
Isard, M. and MacCormick, J. 2001. BraMBle: A Bayesian multiple-blob tracker. In Proceedings of the 8th International Conference on Computer Vision, Vol. 2, pp. 34–41.
Kleinmuntz, B. and Szucko, J.J. 1984. Lie detection in ancient and modern times. American Psychologist, 39:766–776.
Knight, J. April 15 2004. The truth about lying. Nature, 428:692–94.
Kolsch, M. and Turk, M. 2004. Fast 2D hand tracking with flocks of features and multi-cue integration. In Proceedings of the 2004 IEEE Conference on Computer Vision and Pattern Recognition, Vol. 10, pp. 158–164.
MacCormick, J. and Isard, M. 2000. Partitioned sampling, articulated objects, and interface-quality hand tracking. In Lecture Notes in Computer Science, Springer-Verlag, London, U.K., Vol. 1843, pp. 3–19.
MacCormick, J. and Blake, A. 2000. A probabilistic exclusion principle for tracking multiple objects. International Journal of Computer Vision, 39(1):57–71.
Martini, F.H., Ober, W.C., Garrison C.W., Welch, K., and Hutchings, R.T. 2001. Fundamentals of Anatomy and Physiology, Prentice Hall, 5th edn, Upper Saddle River, New Jersey.
Matthews, I., Ishikawa, T., and Baker, S. June 2004. The template update problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(6):810–815.
Pavlidis, I., Levine, J., and Baukol, P. June 16 2000. Thermal imaging for anxiety detection. In Proceedings of the 2000 IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications, Hilton Head Island, South Carolina, pp. 104–109.
Pavlidis, I., Eberhardt, N.L., and Levine, J. January 3 2002. Human behavior: Seeing through the face of deception. Nature, 415(6867):35.
Pavlidis, I. and Levine, J. November/December 2002. Thermal image analysis for polygraph testing. IEEE Engineering in Medicine and Biology Magazine, 21(6):56–64.
Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.F. 1992. Numerical Recipes in C. 2nd edn. Cambridge University Press, New York, New York.
Smith, S.W. 1999. The Scientist’s and Engineer’s Guide to Digital Signal Processing. 2nd edn. California Technical Publishing, San Diego, California.
Sokolov, E.N. and Cacioppo, J.T. 1997. Orienting and defense reflexes: Vector coding the cardiac response. In Attention and Orienting: Sensory and Motivational Processes, P.J. Lang, and R.F. Simons, (Ed.) Lawrence Erlbaum Associates, Mahwah, New Jersey, pp. 1–22.
Verma, R., Schmid, C., and Mikolajczyk, K. October 2003. Face detection and tracking in a video by propagating detection probabilities, IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10):1215–28.
Yankee, W.J. 1965. An investigation of sphygmomanometer discomfort thresholds in polygraph examinations. Police, 9:12–18.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tsiamyrtzis, P., Dowdall, J., Shastri, D. et al. Imaging Facial Physiology for the Detection of Deceit. Int J Comput Vision 71, 197–214 (2007). https://doi.org/10.1007/s11263-006-6106-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-006-6106-y