Abstract
This chapter presents two novel approaches that allow computer vision applications to perform human facial expression recognition (FER). From a prob lem standpoint, we focus on FER beyond the human visual spectrum, in long-wave infrared imagery, thus allowing us to offer illumination-independent solutions to this important human-computer interaction problem. From a methodological stand point, we introduce two different feature extraction techniques: a principal com ponent analysis-based approach with automatic feature selection and one based on texture information selected by an evolutionary algorithm. In the former, facial fea tures are selected based on interest point clusters, and classification is carried out us ing eigenfeature information; in the latter, an evolutionary-based learning algorithm searches for optimal regions of interest and texture features based on classification accuracy. Both of these approaches use a support vector machine-committee for classification. Results show effective performance for both techniques, from which we can conclude that thermal imagery contains worthwhile information for the FER problem beyond the human visual spectrum.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Chapter's References
Maja Pantic and Leon J. M. Rothkrantz. Automatic analysis of facial expressions: The state of the art.IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12):1424–1445, 2000
B. Fasel and J. Luettin. Automatic facial expression analysis: A survey.Pattern Recognition, 36(1):259–275, 2003
Fabrice Bourel, Claude C. Chibelushi, and Adrian A. Low. Robust facial expression recog nition using a state-based model of spatially-localised facial dynamics. InProceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (FGR.02), pages 113–118, 2002
Maja Pantic and Leon J. M. Rothkrantz. An expert system for recognition of facial actions and their intensity.Image and Vision Computing, 18:881–905, 2000
Ying-Li Tian, Takeo Kanade, and Jeffrey Cohn. Recognizing action units for facial expression analysis.IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2):97–115, February 2001
Severine Dubuisson, Franck Davoine, and Jean Pierre Cocquerez. Automatic facial feature extraction and facial expression recognition. InAVBPA '01: Proceedings of the Third Interna tional Conference on Audio- and Video-Based Biometric Person Authentication, pages 121– 126, Springer-Verlag, London, 2001
C. Padgett and G. Cottrell. Representing face images for emotion classification.Advances in Neural Information Processing Systems, 9, 1997
Gianluca Donato, Marian Stewart Bartlett, Joseph C. Hager, Paul Ekman, and Terrence J. Sejnowski. Classifying facial actions.IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(10):974–989, 1999
Michael J. Lyons, Julien Budynek, and Shigeru Akamatsu. Automatic classification of single facial images.IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(12):1357– 1362, 1999
Matthew Turk and Alex Paul Pentland. Eigenfaces for recognition.Journal of Cognitive Neu-roscience, 3(1):71–86, 1991
Joseph Wilder, P. Jonathon Phillips, Cunhong Jiang, and Stephen Wiener. Comparison of visi ble and infra-red imagery for face recognition. In2nd International Conference on Automatic Face and Gesture Recognition (FG '96), October 14–16, 1996, Killington, VT, pages 182– 191, 1996
F. Prokoski. History, current status, and future of infrared identification. InCVBVS '00: Pro ceedings of the IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications (CVBVS 2000), page 5, IEEE Computer Society, Washington, DC, 2000
Leonardo Trujillo, Gustavo Olague, Riad Hammoud, and Benjamín Hernández. Automatic feature localization in thermal images for facial expression recognition. InCVPR '05: Pro ceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)– Workshops, page 14, IEEE Computer Society, Washington, DC, 2005
I. Pavlidis, J. Levine, and P. Baukol. Thermal image analysis for anxiety detection. InInterna tional Conference on Image Processing, volume 2, pages 315–318, 2001
Y. Sugimoto, Y. Yoshitomi, and S. Tomita. A method for detecting transitions of emotional states using a thermal facial image based on a synthesis of facial expressions.Journal of Robotics and Autonomous Systems, 31:147–160, 2000
Y. Yoshitomi, S. Kim, T. Kawano, and T. Kitazoe. Effect of sensor fusion for recognition of emotional states using voice, face image and thermal image of face. InProceeding of the 2000 IEEE International Workshop on Robot and Human Interactive Communication, Osaka. Japan – September 27–29 2000, pages 178–183
Seong G. Kong, Jingu Heo, Besma R. Abidi, Joonki Paik, and Mongi A. Abidi. Recent advances in visual and infrared face recognition: a review.Computer Vision and Image Under standing, 97(1):103–135, 2005
Séverine Dubuisson, Franck Davoine, and Jean Pierre Cocquerez. Automatic facial feature extraction and facial expression recognition. In Josef Bigün and Fabrizio Smeraldi, editors,AVBPA, volume 2091,Lecture Notes in Computer Science, pages 121–126, Springer, New York, 2001
Leonardo Trujillo and Gustavo Olague. Using evolution to learn how to perform interest point detection. InProceedings from the 18th International Conference on Pattern Recognition, volume 1, pages 211–214, IEEE Computer Society, Washington, DC, 2006
Xin Chen, Patrick J. Flynn, and Kevin W. Bowyer. PCA-based face recognition in infrared imagery: Baseline and comparative studies. InAMFG '03: Proceedings of the IEEE Interna tional Workshop on Analysis and Modeling of Faces and Gestures, page 127, IEEE Computer Society, Washington, DC, 2003
Christopher J.C. Burges. A tutorial on support vector machines for pattern recognition.Data Mining and Knowledge Discovery, 2(2):121–167, 1998
Zhong-Qiu Zhao, De-Shuang Huang, and Bing-Yu Sun. Human face recognition based on multi-features using neural networks committee.Pattern Recognition Letters, 25(12):1351– 1358, 2004
J.H. Holland.Adaptation in Natural and Artificial Systems.University of Michigan Press, Ann Arbor, MI, 1975
David E. Goldberg.Genetic Algorithms in Search, Optimization, and Machine Learning.Addison-Wesley Professional, Reading, MA, January 1989
J. Malik and P. Perona. Preattentive texture discrimination with early vision mechanisms.Op tical Society of America, 7:923–932, May 1990
Jianchang Mao and Anil K. Jain. Texture classification and segmentation using multiresolution simultaneous autoregressive models.Pattern Recognition, 25(2):173–188, 1992
B. Julesz and J.R. Bergen.Textons, the Fundamental Elements in Preattentive Vision and Perception of Textures. Kaufmann, Los Altos, CA, 1987
Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce. A sparse texture representation us ing local affine regions.IEEE Transactions on Pattern Analysis ' Machine Intelligence, 27(8):1265–1278, 2005
R.M. Haralick. Statistical and structural approaches to texture.Proceedings of the IEEE, 67:786–804, 1979
J. Kjell. Comparative study of noise-tolerant texture classification.1994 IEEE International Conference on Systems, Man, and Cybernetics. “Humans, Information and Technology”, 3:2431–2436, October 1994
Peter Howarth and Stefan M. Rüger. Evaluation of texture features for content-based image retrieval. InACM International Conference on Image and Video Retrieval (CIVR), pages 326– 334, 2004
Philippe P. Ohanian and Richard C. Dubes. Performance evaluation for four classes of textural features.Pattern Recognition, 25(8):819–833, 1992
Zehang Sun, George Bebis, and Ronald Miller. Object detection using feature subset selection.Pattern Recognition, 37(11):2165–2176, 2004
Chih-Chung Chang and Chih-Jen Lin.LIBSVM: A Library for Support Vector Machines, MIT Press Journals-Neural Computation, 2001
Cyril Goutte. Note on free lunches and cross-validation.Neural Computation, 9(6):1245– 1249, 1997
IEEE OTCBVS WS Series Bench; DOE University Research Program in Robotics under grant DOE-DE-FG02-86NE37968; DOD/TACOM/NAC/ARC Program under grant R01-1344-18; FAA/NSSA grant R01-1344-48/49; Office of Naval Research under grant N000143010022.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Olague, G., Hammoud, R., Trujillo, L., Hernández, B., Romero, E. (2009). Facial Expression Recognition in Nonvisual Imagery. In: Hammoud, R.I. (eds) Augmented Vision Perception in Infrared. Advances in Pattern Recognition. Springer, London. https://doi.org/10.1007/978-1-84800-277-7_10
Download citation
DOI: https://doi.org/10.1007/978-1-84800-277-7_10
Publisher Name: Springer, London
Print ISBN: 978-1-84800-276-0
Online ISBN: 978-1-84800-277-7
eBook Packages: Computer ScienceComputer Science (R0)