International Journal of Computer Vision

, Volume 71, Issue 2, pp 197–214 | Cite as

Imaging Facial Physiology for the Detection of Deceit

  • P. Tsiamyrtzis
  • J. Dowdall
  • D. Shastri
  • I. T. Pavlidis
  • M. G. Frank
  • P. Ekman
Article

Abstract

Previous work has demonstrated the correlation of increased blood perfusion in the orbital muscles and stress levels for human beings. It has also been suggested that this periorbital perfusion can be quantified through the processing of thermal video. The idea has been based on the fact that skin temperature is heavily modulated by superficial blood flow. Proof of this concept was established for two different types of stress inducing experiments: startle experiments and mock-crime polygraph interrogations. However, the polygraph interrogation scenarios were simplistic and highly constrained. In the present paper, we report results derived from a large and realistic mock-crime interrogation experiment. The interrogation is free flowing and no restrictions have been placed on the subjects. Additionally, we propose a new methodology to compute the mean periorbital temperature signal. The present approach addresses the deficiencies of the earlier methodology and is capable of coping with the challenges posed by the realistic setting. Specifically, it features a tandem CONDENSATION tracker to register the periorbital area in the context of a moving face. It operates on the raw temperature signal and tries to improve the information content by suppressing the noise level instead of amplifying the signal as a whole. Finally, a pattern recognition method classifies stressful (Deceptive) from non-stressful (Non-Deceptive) subjects based on a comparative measure between the entire interrogation signal (baseline) and a critical subsection of it (transient response). The successful classification rate is 87.2% for 39 subjects. This is on par with the success rate achieved by highly trained psycho-physiological experts and opens the way for automating lie detection in realistic settings.

Keywords

deception detection polygraph thermography facial tracking contact-free physiological monitoring 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Baker, S. and Matthews, I. 2001. Equivalence and efficiency of image alignment algorithms. In Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 1090–1097.Google Scholar
  2. Cooley, J.W. and Tukey, J.W. 1965. An algorithm for the machine calculation of complex Fourier series. Mathematics of Computation, 19:297–301.MATHMathSciNetCrossRefGoogle Scholar
  3. Committee to Review the Scientific Evidence on the Polygraph—Board on Behavioral, Cognitive, and Sensory Sciences and Committee on National Statistics—Division of Behavioral and Social Sciences and Education—National Research Council of the National Academies. 2003. The Polygraph and Lie Detection, National Academies Press, Washington D.C.Google Scholar
  4. Doucet, A., DeFreitas, N., and Gordon, N. (Eds.) 2001. Sequential Monte Carlo Methods in Practice. Springer-Verlag.Google Scholar
  5. Drummond, P. and Lance, J. 1987. Facial flushing and sweating mediated by the sympathetic nervous system. Brain, 110:793–803.Google Scholar
  6. Dupros, F., Fitzgibbon, W.E., and Garbey, M. 2004. shape A filtering technique for system of reaction diffusion equations. Technical Report UH-CS-04-02, Department of Computer Science, University of Houston.Google Scholar
  7. Ekman, P. 1986. Telling Lies. Berkeley Publishing Group.Google Scholar
  8. Ekman, P. and Maureen, O. 1991. Who can catch a liar? American Psychologist, 46(9):913–20.CrossRefGoogle Scholar
  9. Ekman, P. 1996. Why don’t we catch liars? Social Research, 63(3):801–17.Google Scholar
  10. Ekman, P. 2001. Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage, 2nd ed, W.W. Norton and Company.Google Scholar
  11. Eveland, C., Socolinsky, D., and Wolff, L. July 2003. Tracking human faces in infrared video. Image and Vision Computing, 21:578–590.Google Scholar
  12. Frank, M.G. and Ekman, P. 1997. The ability to detect deceit generalizes across different types of high-stake lies. Personality and Social Psychology, 72:1429–1439.CrossRefGoogle Scholar
  13. Garbey, M., Merla, A., and Pavlidis, I. 2004. Estimation of blood flow speed and vessel location from thermal video. In Proceedings of the 2004 IEEE Computer Vision and Pattern Recognition, Vol. 1, pp. 356–63.Google Scholar
  14. Inbau, F.E., Reid, J.E., Buckley, J.P., and Jayne, B.C. 2001. Criminal Interrogations and Confessions. Aspen Publishers, 4th edn.Google Scholar
  15. Isard, M. and Black, A. 1998a. Condensation—conditional density propagation for visual tracking. International Journal of Computer Vision, 19(1):5–28.CrossRefGoogle Scholar
  16. Isard, M. and Blake, A. June 1998b. ICONDENSATION: Unifying low-level and high-level tracking in a stochastic framework. In Proceedings of the 5th European Conference on Computer Vision, Freiburg, Germany, Vol. 1, pp. 893–908.Google Scholar
  17. Isard, M. and MacCormick, J. 2001. BraMBle: A Bayesian multiple-blob tracker. In Proceedings of the 8th International Conference on Computer Vision, Vol. 2, pp. 34–41.Google Scholar
  18. Kleinmuntz, B. and Szucko, J.J. 1984. Lie detection in ancient and modern times. American Psychologist, 39:766–776.CrossRefGoogle Scholar
  19. Knight, J. April 15 2004. The truth about lying. Nature, 428:692–94.CrossRefGoogle Scholar
  20. Kolsch, M. and Turk, M. 2004. Fast 2D hand tracking with flocks of features and multi-cue integration. In Proceedings of the 2004 IEEE Conference on Computer Vision and Pattern Recognition, Vol. 10, pp. 158–164.Google Scholar
  21. MacCormick, J. and Isard, M. 2000. Partitioned sampling, articulated objects, and interface-quality hand tracking. In Lecture Notes in Computer Science, Springer-Verlag, London, U.K., Vol. 1843, pp. 3–19.Google Scholar
  22. MacCormick, J. and Blake, A. 2000. A probabilistic exclusion principle for tracking multiple objects. International Journal of Computer Vision, 39(1):57–71.MATHCrossRefGoogle Scholar
  23. Martini, F.H., Ober, W.C., Garrison C.W., Welch, K., and Hutchings, R.T. 2001. Fundamentals of Anatomy and Physiology, Prentice Hall, 5th edn, Upper Saddle River, New Jersey.Google Scholar
  24. Matthews, I., Ishikawa, T., and Baker, S. June 2004. The template update problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(6):810–815.CrossRefGoogle Scholar
  25. Pavlidis, I., Levine, J., and Baukol, P. June 16 2000. Thermal imaging for anxiety detection. In Proceedings of the 2000 IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications, Hilton Head Island, South Carolina, pp. 104–109.Google Scholar
  26. Pavlidis, I., Eberhardt, N.L., and Levine, J. January 3 2002. Human behavior: Seeing through the face of deception. Nature, 415(6867):35.CrossRefGoogle Scholar
  27. Pavlidis, I. and Levine, J. November/December 2002. Thermal image analysis for polygraph testing. IEEE Engineering in Medicine and Biology Magazine, 21(6):56–64.CrossRefGoogle Scholar
  28. Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.F. 1992. Numerical Recipes in C. 2nd edn. Cambridge University Press, New York, New York.Google Scholar
  29. Smith, S.W. 1999. The Scientist’s and Engineer’s Guide to Digital Signal Processing. 2nd edn. California Technical Publishing, San Diego, California.Google Scholar
  30. Sokolov, E.N. and Cacioppo, J.T. 1997. Orienting and defense reflexes: Vector coding the cardiac response. In Attention and Orienting: Sensory and Motivational Processes, P.J. Lang, and R.F. Simons, (Ed.) Lawrence Erlbaum Associates, Mahwah, New Jersey, pp. 1–22.Google Scholar
  31. Verma, R., Schmid, C., and Mikolajczyk, K. October 2003. Face detection and tracking in a video by propagating detection probabilities, IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10):1215–28.CrossRefGoogle Scholar
  32. Yankee, W.J. 1965. An investigation of sphygmomanometer discomfort thresholds in polygraph examinations. Police, 9:12–18.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  • P. Tsiamyrtzis
    • 1
  • J. Dowdall
    • 2
  • D. Shastri
    • 2
  • I. T. Pavlidis
    • 2
  • M. G. Frank
    • 3
  • P. Ekman
    • 4
  1. 1.Department of StatisticsAthens University of Economics and BusinessAthensGreece
  2. 2.Department of Computer ScienceUniversity of HoustonHouston
  3. 3.School of Informatics/Department of CommunicationUniversity of Buffalo, State University of New YorkBuffalo
  4. 4.Department of PsychiatryUC San FranciscoSan Francisco

Personalised recommendations