Encyclopedia of Biometrics

2009 Edition
| Editors: Stan Z. Li, Anil Jain

Facial Expression Recognition

  • Maja Pantic
Reference work entry
DOI: https://doi.org/10.1007/978-0-387-73003-5_98



Facial expression recognition is a process performed by humans or computers, which consists of:
  1. 1.

    Locating faces in the scene (e.g., in an image; this step is also referred to as face detection),

  2. 2.

    Extracting facial features from the detected face region (e.g., detecting the shape of facial components or describing the texture of the skin in a facial area; this step is referred to as facial feature extraction),

  3. 3.

    Analyzing the motion of facial features and/or the changes in the appearance of facial features and classifying this information into some facial-expression-interpretative categories such as facial muscle activations like smile or frown, emotion (affect) categories like happiness or anger, attitude categories like (dis)liking or ambivalence, etc. (this step is also referred to as facial expression interpretation).



A widely accepted prediction is that computing will move to the...

This is a preview of subscription content, log in to check access.


  1. 1.
    Pantic, M., Pentland, A., Nijholt, A., Huang, T.S.: Human computing and machine understanding of human behavior: A Survey. Lect. Notes Artif. Intell. 4451, 47–71 (2007)Google Scholar
  2. 2.
    Pantic, M., Rothkrantz, L.J.M.: Toward an affect-sensitive multimodal HCI. Proceedings of the IEEE 91(9), 1370–1390 (2003)CrossRefGoogle Scholar
  3. 3.
    Tian, Y.L., Kanade, T., Cohn, J.F.: Facial expression analysis. In: Li, S.Z., Jain, A.K. (eds.) Handbook of Face Recognition, pp. 247–276. Springer, New York (2005)CrossRefGoogle Scholar
  4. 4.
    Pantic, M., Bartlett, M.S.: Machine analysis of facial expressions. In: Delac, K., Grgic, M. (eds.) 1Face Recognition, pp. 377–416. I-Tech Education and Publishing, Vienna, Austria (2007)Google Scholar
  5. 5.
    Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2009)Google Scholar
  6. 6.
    Ambady, N., Rosenthal, R.: Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. Psychol. Bull. 111(2), 256–274 (1992)CrossRefGoogle Scholar
  7. 7.
    Ekman, P., Rosenberg, E.L. (eds.): What the face reveals: Basic and applied studies of spontaneous expression using the facial action coding system. Oxford University Press, Oxford, UK (2005)Google Scholar
  8. 8.
    Yang, M.H., Kriegman, D.J., Ahuja, N.: Detecting faces in images: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 24(1), 34–58 (2002)CrossRefGoogle Scholar
  9. 9.
    Li, S.Z., Jain, A.K. (eds.): Handbook of face recognition. Springer, New York (2005)MATHGoogle Scholar
  10. 10.
    Viola, P., Jones, M.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)CrossRefGoogle Scholar
  11. 11.
    Pantic, M., Patras, I.: Dynamics of facial expression: Recognition of facial actions and their temporal segments from face profile image sequences. IEEE Trans. Syst. Man Cybern. B Cybern. 36(2), 433–449 (2006)CrossRefGoogle Scholar
  12. 12.
    Cohn, J.F., Ekman, P.: Measuring facial actions. In: Harrigan, J.A., Rosenthal, R., Scherer, K. (eds.) The New Handbook of Methods in Nonverbal Behavior Research, pp. 9–64. Oxford University Press, New York (2005)Google Scholar
  13. 13.
    Keltner, D., Ekman, P.: Facial expression of emotion. In: Lewis, M., Haviland-Jones, J.M. (eds.) Handbook of Emotions, pp. 236–249. Guilford Press, New York (2000)Google Scholar
  14. 14.
    Ekman, P., Friesen, W.V., Hager, J.C.: Facial action coding system. A Human Face, Salt Lake City, USA (2002)Google Scholar
  15. 15.
    Pantic, M., Valstar, M.F, Rademaker, R., Maat, L. Web-based database for facial expression analysis. Proc. IEEE Int’l Conf. Multimedia & Expo (ICME) 317–321 (2005)Google Scholar
  16. 16.
    Ekman, P., Huang, T.S., Sejnowski, T.J., Hager, J.C. (eds.): NSF Understanding the Face. A Human Face eStore, Salt Lake City, USA, (see Library) (1992)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Maja Pantic
    • 1
  1. 1.Department of Computing Imperial College LondonLondonUK