Skip to main content
SpringerLink
Log in

Facial Expression Recognition from Infrared Thermal Videos

  • Conference paper
Book cover Intelligent Autonomous Systems 12

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 194))

Abstract

In this paper, a spontaneous facial expression recognition method using infrared thermal videos is proposed. Firstly, the sequence features are extracted from the infrared thermal horizontal and vertical temperature difference sequences of different facial sub-regions. Secondly, a feature subset is selected according to their F-values. Thirdly, the Adaboost algorithm, with the weak classifiers of k-Nearest Neighbor, is used to classify facial expressions in arousal and valence dimensions. Finally, experiments on the Natural Visible and Infrared facial Expression (USTC-NVIE) database demonstrates the effectiveness of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(1), 39–58 (2009)

    Article  Google Scholar 

  2. Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recognition 36(1), 259–275 (2003)

    Article  MATH  Google Scholar 

  3. Khan, M.M., Ward, R.D., Ingleby, M.: Classifying pretended and evoked facial expressions of positive and negative affective states using infrared measurement of skin temperature. ACM Transactions on Applied Perception (TAP) 6(1), 1–22 (2009)

    Article  Google Scholar 

  4. Jenkins, S., Brown, R., Rutterford, N.: Comparing thermographic, eeg, and subjective measures of affective experience during simulated product interactions. International Journal of Design 3(2), 53–65 (2009)

    Google Scholar 

  5. Levine, J.A., Pavlidis, I., Cooper, M.: The face of fear. The Lancet 357(9270), 1757 (2001)

    Article  Google Scholar 

  6. Nozawa, A., Tacano, M.: Correlation analysis on alpha attenuation and nasal skin temperature. Journal of Statistical Mechanics: Theory and Experiment, P01007 (2009)

    Google Scholar 

  7. Nakanishi, R., Imai-Matsumura, K.: Facial skin temperature decreases in infants with joyful expression. Infant Behavior and Development 31(1), 137–144 (2008)

    Article  Google Scholar 

  8. Pavlidis, I., Levine, J.: Thermal image analysis for polygraph testing. IEEE Engineering in Medicine and Biology Magazine 21(6), 56–64 (2002)

    Article  Google Scholar 

  9. Puri, C., Olson, L., Pavlidis, I., Levine, J., Starren, J.: Stresscam: non-contact measurement of users’ emotional states through thermal imaging. In: Extended Abstracts on Human Factors in Computing Systems, CHI 2005, pp. 1725–1728. ACM (2005)

    Google Scholar 

  10. Khan, M.M., Ward, R.D., Ingleby, M.: Infrared thermal sensing of positive and negative affective states. In: IEEE Conference on Robotics, Automation and Mechatronics, pp. 1–6. IEEE (2006)

    Google Scholar 

  11. Nhan, B.R., Chau, T.: Classifying affective states using thermal infrared imaging of the human face. IEEE Transactions on Biomedical Engineering 57(4), 979–987 (2010)

    Article  Google Scholar 

  12. Jarlier, S., Grandjean, D., Delplanque, S., N’Diaye, K., Cayeux, I., Velazco, M., Sander, D., Vuilleumier, P., Scherer, K.: Thermal analysis of facial muscles contractions. IEEE Transactions on Affective Computing (99), 1 (2011)

    Google Scholar 

  13. Yoshitomi, Y.: Facial expression recognition for speaker using thermal image processing and speech recognition system. In: Proceedings of the 10th WSEAS International Conference on Applied Computer Science, pp. 182–186. World Scientific and Engineering Academy and Society, WSEAS (2010)

    Google Scholar 

  14. Krzywicki, A.T., He, G., O’Kane, B.L.: Analysis of facial thermal variations in response to emotion: eliciting film clips. Proceedings of SPIE 7343, 734312 (2009)

    Article  Google Scholar 

  15. Koda, Y., Yoshitomi, Y., Nakano, M., Tabuse, M.: A facial expression recognition for a speaker of a phoneme of vowel using thermal image processing and a speech recognition system. In: The 18th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN 2009, pp. 955–960. IEEE (2009)

    Google Scholar 

  16. Hernández, B., Olague, G., Hammoud, R., Trujillo, L., Romero, E.: Visual learning of texture descriptors for facial expression recognition in thermal imagery. Computer Vision and Image Understanding 106(2-3), 258–269 (2007)

    Article  Google Scholar 

  17. Khan, M.M., Ingleby, M., Ward, R.D.: Automated facial expression classification and affect interpretation using infrared measurement of facial skin temperature variations. ACM Transactions on Autonomous and Adaptive Systems (TAAS) 1(1), 91–113 (2006)

    Article  Google Scholar 

  18. Trujillo, L., Olague, G., Hammoud, R., Hernandez, B.: Automatic feature localization in thermal images for facial expression recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, CVPR Workshops, p. 14. IEEE (2005)

    Google Scholar 

  19. Wang, P., Ji, Q.: Multi-view face and eye detection using discriminant features. Computer Vision and Image Understanding 105(2), 99–111 (2007)

    Article  Google Scholar 

  20. Keys, R.: Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech and Signal Processing 29(6), 1153–1160 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  21. Wang, Y., Ai, H., Wu, B., Huang, C.: Real time facial expression recognition with adaboost. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 3, pp. 926–929. IEEE (2004)

    Google Scholar 

  22. Bartlett, M.S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Recognizing facial expression: machine learning and application to spontaneous behavior. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 568–573. IEEE (2005)

    Google Scholar 

  23. Wang, S., Liu, Z., Lv, S., Lv, Y., Wu, G., Peng, P., Chen, F., Wang, X.: A natural visible and infrared facial expression database for expression recognition and emotion inference. IEEE Transactions on Multimedia 12(7), 682–691 (2010)

    Article  Google Scholar 

  24. Khan, M.M., Ward, R.D., Ingleby, M.: Automated classification and recognition of facial expressions using infrared thermal imaging. In: IEEE Conference on Cybernetics and Intelligent Systems, vol. 1, pp. 202–206. IEEE (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Key Lab of Computing and Communicating Software of Anhui Province School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, P.R. China, 230027

    P. Shen, Shangfei Wang & Zhilei Liu

Authors
  1. P. Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shangfei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhilei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to P. Shen .

Editor information

Editors and Affiliations

  1. , School of Information and Communication, Sungkyunkwan University, 300 Chunchun-Dong Jangan-Ku, Kyunggi-Do, 440-746, Korea, Republic of (South Korea)

    Sukhan Lee

  2. Science and Technology, Daegu Gyeongbuk Institute of, 50-1, Sang-Ri, Hyeonpung-Myeon, Dalseong-Gun, Daegu, 711-873, Korea, Republic of (South Korea)

    Hyungsuck Cho

  3. Konkuk University, Neungdong-ro, Gwangjin-gu 120, Seoul, 143-701, Korea, Republic of (South Korea)

    Kwang-Joon Yoon

  4. , Department of Electronics Engineering, Pusan National University, Geum Jeong-ku, Pusan, 609-735, Korea, Republic of (South Korea)

    Jangmyung Lee

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Shen, P., Wang, S., Liu, Z. (2013). Facial Expression Recognition from Infrared Thermal Videos. In: Lee, S., Cho, H., Yoon, KJ., Lee, J. (eds) Intelligent Autonomous Systems 12. Advances in Intelligent Systems and Computing, vol 194. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33932-5_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-33932-5_31

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-33931-8

  • Online ISBN: 978-3-642-33932-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation