Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Facial Expression Analysis

  • Chapter
Handbook of Face Recognition

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. J. Ahlberg and R. Forchheimer. Face tracking for model-based coding and face animation. International Journal of Imaging Systems and Technology, pages 8–22, Vol. 13,Issue 1, 2003.

    Article  Google Scholar 

  2. S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9):1167–1183, 2002.

    Article  Google Scholar 

  3. M. Bartlett, B. Braathen, G. Littlewort-Ford, J. Hershey, I. Fasel, T. Marks, E. Smith, T. Sejnowski, and J.R. Movellan. Automatic analysis of spontaneous facial behavior: A final project report. Technical Report INC-MPLab-TR-2001.08, Machine Perception Lab, Institute for Neural Computation, University of California, San Diego, 2001.

    Google Scholar 

  4. M. Bartlett, J. Hager, P. Ekman, and T. Sejnowski. Measuring facial expressions by computer image analysis. Psychophysiology, 36:253–264, 1999.

    Article  Google Scholar 

  5. M. Black. Robust incremental optical flow. In PhD thesis, Yale University, 1992.

    Google Scholar 

  6. M. Black and Y. Yacoob. Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion. In Proc. of International conference on Computer Vision, pages 374–381, 1995.

    Google Scholar 

  7. M. Black and Y. Yacoob. Recognizing facial expressions in image sequences using local parameterized models of image motion. International Journal of Computer Vision, 25(1):23–48, October 1997.

    Article  Google Scholar 

  8. L. Brown and Y.-L. Tian. Comparative study of coarse head pose estimation. In IEEE Workshop on Motion and Video Computing, Orlando, 2002.

    Google Scholar 

  9. L. Camras, L. Lambrecht, and G. Michel. Infant surprise expressions as coordinative motor structures. Journal of Nonverbal Behavior, 20:183–195, 1966.

    Article  Google Scholar 

  10. J. Carroll and J. Russell. Do facial expression signal specific emotions? Journal of Personality and Social Psychology., 70:205–218, 1996.

    Article  Google Scholar 

  11. J. Carroll and J. Russell. Facial expression in hollywood’s portrayal of emotion. Journal of Personality and Social Psychology., 72:164–176, 1997.

    Article  Google Scholar 

  12. L. Chen. Joint processing of ausio-visual information for the recognition of emotional expressions in human-computer interaction. In PhD thesis, University of Illinois at Urbana-Champaign, Department of Electrical Engineering, 2000.

    Google Scholar 

  13. I. Cohen, N. Sebe, F. Cozman, M. Cirelo, and T. Huang. Coding, analysis, interpretation, and recognition of facial expressions. Journal of Computer Vision and Image Understanding Special Issue on Face Recognition, 2003.

    Google Scholar 

  14. J. Cohn, T. Kanade, T. Moriyama, Z. Ambadar, J. Xiao, J. Gao, and H. Imamura. A comparative study of alternative facs coding algorithms. Technical Report CMU-RI-TR-02-06, Robotics Institute, Carnegie Mellon University, Pittsburgh, November 2001.

    Google Scholar 

  15. J. Cohn and G. Katz. Bimodal expression of emotion by face and voice. In ACM and ATR Workshop on Face/Gesture Recognition and Their Applications, pages 41–44, 1998.

    Google Scholar 

  16. J. Cohn, K. Schmidt, R. Gross, and P. Ekman. Individual differences in facial expression: stability over time, relation to self-reported emotion, and ability to inform person identification. In Proceedings of the International Conference on Multimodal User Interfaces (ICMI 2002), pages 491–496, 2002.

    Google Scholar 

  17. J. Cohn, A. Zlochower, J. Lien, and T. Kanade. Automated face analysis by feature point tracking has high concurrent validity with manual facs coding. Psychophysiology, 36:35–43, 1999.

    Article  Google Scholar 

  18. C. Darwin. The Expression of Emotions in Man and Animals. John Murray, reprinted by University of Chicago Press, 1965, 1872.

    Google Scholar 

  19. J. Daugmen. Complete discrete 2d gabor transforms by neutral networks for image analysis and compression. IEEE Transaction on Acoustic, Speech and Signal Processing, 36(7):1169–1179, 1988.

    Article  Google Scholar 

  20. G. Donato, M. Bartlett, J. Hager, P. Ekman, and T. Sejnowski. Classifying facial actions. IEEE Transaction on Pattern Analysis and Machine Intelligence, 21(10):974–989, 1999.

    Article  Google Scholar 

  21. Eihl-Eihesfeldt. Human Ethology. Aldine de Gruvter, New York, 1989.

    Google Scholar 

  22. P. Ekman. The Argument and Evidence about Universals in Facial Expressions of Emotion, pages 143–164. Wiley, New York, 1989.

    Google Scholar 

  23. P. Ekman. Facial expression and emotion. American Psychologist, 48:384–392, 1993.

    Article  Google Scholar 

  24. P. Ekman and W. Friesen. Pictures of Facial Affect. Palo Alto, CA: Consulting Psychologist., 1976.

    Google Scholar 

  25. P. Ekman and W. Friesen. The Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, San Francisco, 1978.

    Google Scholar 

  26. P. Ekman, J. Hager, C. Methvin, and W. Irwin. Ekman-Hager Facial Action Exemplars. Human Interaction Laboratory, University of California, San Francisco.

    Google Scholar 

  27. P. Ekman and E. e. Rosenberg. What the Face Reveals. Oxford University, New York, 1997.

    Google Scholar 

  28. I. Essa and A. Pentland. Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. on Pattern Analysis and Machine Intell., 19(7):757–763, 1997.

    Article  Google Scholar 

  29. L. Farkas and I. Munro. Anthropometric Facial Proportions in Medicine. Charles C Thomas, Springfield, IL, 1987.

    Google Scholar 

  30. B. Fasel and J. Luttin. Recognition of asymmetric facial action unit activities and intensities. In Proceedings of International Conference of Pattern Recognition, 2000.

    Google Scholar 

  31. B. Fasel and J. Luttin. Automatic facial expression analysis: Survey. Pattern Recognition, 36(1):259–275, 2003.

    Article  MATH  Google Scholar 

  32. J. Fleiss. Statistical Methods for Rates and Proportions. Wiley, New York, 1981.

    MATH  Google Scholar 

  33. G. Ford. Fully automatic coding of basic expressions from video. Technical Report INC-MPLab-TR-2002.03, Machine Perception Lab, Institute for Neural Computation, University of California, San Diego, 2002.

    Google Scholar 

  34. N. Fox, R. Gross, P. de Chazal, J. Cohn, and R. Reilly. Person identification using multi-modal features: speech, lip, and face. In Proc. of ACM Multimedia Workshop in Biometrics Methods and Applications (WBMA 2003), CA, 2003.

    Google Scholar 

  35. N. Fox and R. Reilly. Audio-visual speaker identification. In Proc. of the 4th International Conference on Audio-and Video-Based Biometric Person Authentication, 2003.

    Google Scholar 

  36. M. Frank and P. Ekman. The ability to detect deceit generalizes across different types of high-stake lies. Personality and Social Psychology, 72:1429–1439, 1997.

    Article  Google Scholar 

  37. W. Friesen and P. Ekman. Emfacs-7: emotional facial action coding system. Unpublished manuscript, University of California at San Francisco, 1983.

    Google Scholar 

  38. H. Fujiyoshi and A. Lipton. Real-time human motion analysis by image skeletonization. In Proc. of the Workshop on Application of Computer Vision, 1998.

    Google Scholar 

  39. K. Fukui and O. Yamaguchi. Facial feature point extraction method based on combination of shape extraction and pattern matching. Systems and Computers in Japan, 29(6):49–58, 1998.

    Article  Google Scholar 

  40. A. Hampapur, S. Pankanti, A. Senior, Y. Tian, L. Brown, and R. Bolle. Face cataloger: multi-scale imaging for relating identity to location. In Proceedings of IEEE Conference on Advanced Video and Signal Based Surveillance, 2003.

    Google Scholar 

  41. B. Heisele, T. Serre, M. Pontil, and T. Poggio. Component-based face detection. In Proc. IEEE Conf. on Computer Vision and Pattern Recogn. (CVPR), 2001.

    Google Scholar 

  42. C. Izard, L. Dougherty, and E. A. Hembree. A system for identifying affect expressions by holistic judgments. In Unpublished Manuscript, University of Delaware, 1983.

    Google Scholar 

  43. T. Kanade, J. Cohn, and Y.-L. Tian. Comprehensive database for facial expression analysis. In Proceedings of International Conference on Face and Gesture Recognition, pages 46–53, 2000.

    Google Scholar 

  44. S. Kimura and M. Yachida. Facial expression recognition and its degree estimation. In Proc. Of the International Conference on Computer Vision and Pattern Recognition, pages 295–300, 1997.

    Google Scholar 

  45. R. Kleck and M. Mendolia. Decoding of profile versus full-face expressions of affect. Journal of Nonverbal Behavior, 14(1):35–49, 1990.

    Article  Google Scholar 

  46. H. Kobayashi, K. Tange, and F. Hara. Dynamic recognition of six basic facial expressions by discretetime recurrent neural network. In Proc. of the International Joint Conference on Neural Networks, pages 155–158, 1993.

    Google Scholar 

  47. H. Kobayashi, K. Tange, and F. Hara. Real-time recognition of six basic facial expressions. In Proc. IEEE Workshop on Robot and Human Communication, pages 179–186, 1995.

    Google Scholar 

  48. R. Kraut and R. Johnson. Social and emotional messages of smiling: an ethological approach. Journal of Personality and Social Psychology, 37:1539–1523, 1979.

    Article  Google Scholar 

  49. S. Li and L. Gu. Real-time multi-view face detection, tracking, pose estimation, alignment, and recognition. In IEEE Conf. on Computer Visioin and Pattern Recognition Demo Summary, 2001.

    Google Scholar 

  50. J.-J. Lien, T. Kanade, J. Cohn, and C. Li. Subtly different facial expression recognition and expression intensity estimation. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pages 853–859, 1998.

    Google Scholar 

  51. J.-J. Lien, T. Kanade, J. Cohn, and C. Li. Detection, tracking, and classification of action units in facial expression. Journal of Robotics and Autonomous System, 31:131–146, 2000.

    Article  Google Scholar 

  52. M. Lyons, S. Akamasku, M. Kamachi, and J. Gyoba. Coding facial expressions with gabor wavelets. In Proceedings of International Conference on Face and Gesture Recognition, 1998.

    Google Scholar 

  53. A. Manstead. Expressiveness as an individual difference, pages 285–328. Cambridge University Press, Cambridge, UK, 1991.

    Google Scholar 

  54. P. Martin and P. Bateson. Measuring behavior: An Introductory Guide. Cambridge University Press, Cambridge, UK, 1986.

    Google Scholar 

  55. K. Mase. Recognition of facial expression from optical flow. IEICE Transactions, E. 74(10):3474–3483, 1991.

    Google Scholar 

  56. R. Matias, J. Cohn, and S. Ross. A comparison of two systems to code infants’ affective expression. Developmental Psychology, 25:483–489, 1989.

    Article  Google Scholar 

  57. T. Moriyama, T. Kanade, J. Cohn, J. Xiao, Z. Ambadar, J. Gao, and M. Imanura. Automatic recognition of eye blinking in spontaneously occurring behavior. In Proceedings of the 16th International Conference on Pattern Recognition (ICPR’ 2002), volume 4, pages 78–81, 2002.

    Google Scholar 

  58. Y. Moses, D. Reynard, and A. Blake. Determining facial expressions in real time. In Proc. of Int. Conf. On Automatic Face and Gesture Recognition, pages 332–337, 1995.

    Google Scholar 

  59. M. Pantic and L. Rothkrantz. Automatic analysis of facial expressions: the state of the art. IEEE Transaction on Pattern Analysis and Machine Intelligence, 22(12):1424–1445, 2000.

    Article  Google Scholar 

  60. M. Pantic and L. Rothkrantz. Expert system for automatic analysis of facial expression. Image and Vision Computing, 18(11):881–905, 2000.

    Article  Google Scholar 

  61. A. Pentland, B. Moghaddam, and T. Starner. View-based and modular eigenspaces for face recognition. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, pages 84–91, 1994.

    Google Scholar 

  62. F. Pighin, H. Szeliski, and D. Salesin. Synthesizing realistic facial expressions from photographs. In Proc of SIGGRAPH, 1998.

    Google Scholar 

  63. W. Rinn. The neuropsychology of facial expression: a review of the neurological and psychological mechanisms for producing facial expressions. Psychological Bulletin, (95):52–77, 1984.

    Google Scholar 

  64. S. Rizvi, P. Phillips, and H. Moon. The feret verification testing protocol for face recognition algorithms. In Proceedings of the Third International Conference on Automatic Face and Gesture Recognition, pages 48–55, 1998.

    Google Scholar 

  65. M. Rosenblum, Y. Yacoob, and L. Davis. Human expression recognition from motion using a radial basis function network architecture. IEEE Transactions On Neural Network, 7(5):1121–1138, 1996.

    Article  Google Scholar 

  66. H. Rowley, S. Baluja, and T. Kanade. Neural network-based face detection. IEEE Transactions On Pattern Analysis and Machine intelligence, 20(1):23–38, 1998.

    Article  Google Scholar 

  67. J. Russell. Culture and the categorization. Psychological Bulletin, 110:426–450, 1991.

    Article  Google Scholar 

  68. J. Russell. Is there universal recognition of emotion from facial expression? a review of the crosscultural studies. Psychological Bulletin, 115:102–141, 1991.

    Article  Google Scholar 

  69. A. Samal and P. Iyengar. Automatic recognition and analysis of human faces and facil expressions: A survey. Pattern Recognition, 25(1):65–77, 1992.

    Article  Google Scholar 

  70. M. Sayette, J. Cohn, J. Wertz, M. Perrott, and D. Parrott. A psychometric evaluation of the facial action coding system for assessing spontaneous expression. Journal of Nonverbal Behavior, 25:167–186, 2001.

    Article  Google Scholar 

  71. K. Scherer and P. Ekman. Handbook of Methods in Nonverbal Behavior Research. Cambridge University Press, Cambridge, UK, 1982.

    Google Scholar 

  72. K. Schmidt and J. Cohn. Human facial expressions as adaptations: Evolutionary questions in facial expression. Yearbook of Physical Anthropology, 44:3–24, 2002.

    Google Scholar 

  73. K. Schmidt, J. F. Cohn, and Y.-L. Tian. Signal characteristics of spontaneous facial expressions: Automatic movement in solitary and social smiles. Biological Psychology, 2003.

    Google Scholar 

  74. H. Schneiderman and T. Kanade. A statistical model for 3d object detection applied to faces and cars. In IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2000.

    Google Scholar 

  75. K. Sung and T. Poggio. Example-based learning for view-based human face detection. IEEE Transactions on Pattern Analysis and Machine intelligence, 20(1):39–51, 1998.

    Article  Google Scholar 

  76. M. Suwa, N. Sugie, and K. Fujimora. A preliminary note on pattern recognition of human emotional expression. In International Joint Conference on Pattern Recognition, pages 408–410, 1978.

    Google Scholar 

  77. H. Tao and T. Huang. Explanation-based facial motion tracking using a piecewise bezier volume deformation model. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.

    Google Scholar 

  78. D. Terzopoulos and K. Waters. Analysis of facial images using physical and anatomical models. In IEEE International Conference on Computer Vision, pages 727–732, 1990

    Google Scholar 

  79. Y.-L. Tian and R. Bolle. Automatic detecting neutral face for face authentication. In Proceedings of AAAI-03 Spring Symposium on Intelligent Multimedia Knowledge Management, CA, 2003.

    Google Scholar 

  80. Y.-L. Tian, L. Brown, A. Hampapur, S. Pankanti, A. Senior, and R. Bolle. Real world real-time automatic recognition of facial expressions. In Proceedings of IEEE Workshop on Performance Evaluation of Tracking and Surveillance, Graz, Austria, 2003.

    Google Scholar 

  81. Y.-L. Tian, T. Kanade, and J. Cohn. Recognizing action units for facial expression analysis. IEEE Trans. on Pattern Analysis and Machine Intell., 23(2):1–19, 2001.

    MATH  Google Scholar 

  82. Y.-L. Tian, T. Kanade, and J. Cohn. Evaluation of gabor-wavelet-based facial action unit recognition in image sequences of increasing complexity. In Proceedings of the 5th IEEE International Conference on Automatic Face and Gesture Recognition (FG’02), Washington, DC, 2002.

    Google Scholar 

  83. Y.-L. Tian, T. Kanade, and J. Cohn. Eye-state action unit detection by gabor wavelets. In Proceedings of International Conference on Multi-modal Interfaces (ICMI 2000), pages 143–150, Sept, 2000.

    Google Scholar 

  84. K. Toyama. look, ma-no hands! hands-free cursor control with real-time 3d face tracking. In Proc. Workshop on Perceptual User Interfaces(PUI’98), 1998.

    Google Scholar 

  85. J. VanSwearingen, J. Cohn, and A. Bajaj-Luthra. Specific impairment of smiling increases severity of depressive symptoms in patients with facial neuromuscular disorders. Journal of Aesthetic Plastic Surgery, 23:416–423, 1999

    Article  Google Scholar 

  86. T. Vetter. Learning novels views to a single face image. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, pages 22–29, 1995.

    Google Scholar 

  87. P. Viola and M. Jones. Robust real-time object detection. In International Workshop on Statistical and Computational Theories of Vision — Modeling, Learning, Computing, and Sampling, 2001.

    Google Scholar 

  88. Z. Wen and T. Huang. Capturing subtle facial motions in 3d face tracking. In Proc. of Int. Conf. On Computer Vision, 2003.

    Google Scholar 

  89. Y. Wu and K. Toyama. Wide-range person and illumination-insensitve head orientation estimation. In Proceedings of International Conference on Automatic Face and Gesture Recognition, pages 183–188, 2000.

    Google Scholar 

  90. J. Xiao, T. Kanade, and J. Cohn. Robust full motion recovery of head by dynamic templates and reregistration techniques. In Proceedings of International Conference on Automatic Face and Gesture Recognition, pages 163–169, 2002.

    Google Scholar 

  91. J. Xiao, T. Moriyama, T. Kanade, and J. Cohn. Robust full-motion recovery of head by dynamic templates and re-registration techniques. International Journal of Imaging Systems and Technology, September, 2003.

    Google Scholar 

  92. Y. Yacoob and M. Black. Parameterized modeling and recognition of activities. In Proc. 6th IEEE Int. Conf. on Computer Vision, pages 120–127, Bombay, 1998.

    Google Scholar 

  93. Y. Yacoob and L. Davis. Recognizing human facial expression from long image sequences using optical flow. IEEE Trans. on Pattern Analysis and Machine Intell., 18(6):636–642, June 1996.

    Article  Google Scholar 

  94. Y. Yacoob, H.-M. Lam, and L. Davis. Recognizing faces showing expressions. In Proc. Int.Workshop on Automatic Face-and Gesture-Recognition, pages 278–283, Zurich, Switserland, 1995.

    Google Scholar 

  95. M. Yang, D. Kriegman, and N. Ahuja. Detecting faces in images: a survey. IEEE Trans. on Pattern Analysis and Machine Intell., 24(1), 2002.

    Google Scholar 

  96. Z. Zhang. Feature-based facial expression recognition: sensitivity analysis and experiments with a multi-layer perceptron. International Journal of Pattern Recognition and Artificial Intelligence, 13(6):893–911, 1999.

    Article  Google Scholar 

  97. Z. Zhang, M. Lyons, M. Schuster, and S. Akamatsu. Comparison between geometry-based and gaborwavelets-based facial expression recognition using multi-layer perceptron. In InternationalWorkshop on Automatic Face and Gesture Recognition, pages 454–459, 1998.

    Google Scholar 

  98. L. Zhao, G. Pingali, and I. Carlbom. Real-time head orientation estimation using neural networks. In Proc of the 6th International Conference on Image Processing, 2002.

    Google Scholar 

  99. A. Zlochower, J. Cohn, J. Lien, and T. Kanade. A computer vision based method of facial expression analysis in parent-infant interaction. In International Conference on Infant Studies, Atlanta, 1998.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IBM T. J. Watson Research Center, Hawthorne, NY, 10532, USA

    Ying-Li Tian

  2. Carnegie Mellon University, Robotics Institute, Pittsburgh, PA, 15213, USA

    Takeo Kanade & Jeffrey F. Cohn

  3. University of Pittsburgh, Department of Psychology, Pittsburgh, PA, 15260, USA

    Jeffrey F. Cohn

Authors
  1. Ying-Li Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Takeo Kanade
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeffrey F. Cohn
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer Science+Business Media, Inc.

About this chapter

Cite this chapter

Tian, YL., Kanade, T., Cohn, J.F. (2005). Facial Expression Analysis. In: Handbook of Face Recognition. Springer, New York, NY. https://doi.org/10.1007/0-387-27257-7_12

Download citation

  • DOI: https://doi.org/10.1007/0-387-27257-7_12

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-0-387-40595-7

  • Online ISBN: 978-0-387-27257-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation