Facial Expressions Recognition from Image Sequences

  • Zahid Riaz
  • Christoph Mayer
  • Michael Beetz
  • Bernd Radig
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5641)


Human machine interaction is one of the emerging fields for the coming years. Interacting with others in our daily life is a face to face interaction. Faces are the natural way of interaction between humans and hence also useful in human machine interaction.

This paper describes a novel technique to recognize the human facial expressions and manipulating this task for human machine interaction. We use 2D model based approach for human facial expression recognition. An active shape model (ASM) is fitted to the face image and texture information is extraced. This shape and texture information is combined with optical flow based temporal information of the image sequences to form a feature vector for the image. We experimented on image sequences of 97 different persons of Cohn-Kanade-Facial Expression Database. A classification rate of 92.4% is obtained using a binary decision tree classifier, whereas a classification rate of 96.4% is obtained using pairwise classifier based on support vector machines. This system is capable to work in realtime.


Face Modeling Active Appearance Models Facial Expressions Recognition Face Recognition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: The state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1424–1445 (2000)CrossRefGoogle Scholar
  2. 2.
    Edwards, G.J., Taylor, C.J., Cootes, T.F.: Interpreting Face Images using Active Appearance Models. In: Proceedings of International Conference on Automatic Face and Gesture Recognition, pp. 300–305 (1998)Google Scholar
  3. 3.
    Wimmer, M., Riaz, Z., Mayer, C., Radig, B.: Recognizing Facial Expressions Using Model-Based Image Interpretation. In: Advances in Human-Computer InteractionGoogle Scholar
  4. 4.
    Wimmer, M., Stulp, F., Tschechne, S., Radig, B.: Learning Robust Objective Functions for Model Fitting in Image Understanding Applications. In: Proceedings of the 17th British Machine Vision Conference, pp. 1159–1168. BMVA, Edinburgh (2006)Google Scholar
  5. 5.
    Cootes, T.F., Taylor, C.J.: Active shape models – smart snakes. In: Proceedings of the 3rd British Machine Vision Conference, pp. 266–275. Springer, Heidelberg (1992)Google Scholar
  6. 6.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active Appearance Models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998)Google Scholar
  7. 7.
    Li, S.Z., Jain, A.K.: Handbook of Face recognition. Springer, Heidelberg (2005)zbMATHGoogle Scholar
  8. 8.
    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of Fourth IEEE International Conference on Automatic Face and Gesture Recognition (FGR 2000), Grenoble, France, pp. 46–53 (2000)Google Scholar
  9. 9.
    Ekman, P., Friesen, W.: The Facial Action Coding System: A Technique for The Measurement of Facial Movement. Consulting Psychologists Press, San Francisco (1978)Google Scholar
  10. 10.
    Michel, P., Kaliouby, R.E.: Real time facial expression recognition in video using support vector machines. In: Fifth International Conference on Multimodal Interfaces, Vancouver, pp. 258–264 (2003)Google Scholar
  11. 11.
    Cohn, J., Zlochower, A., Lien, J.-J.J., Kanade, T.: Feature-point tracking by optical flow discriminates subtle differences in facial expression. In: Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition, April 1998, pp. 396–401 (1998)Google Scholar
  12. 12.
    Kotsia, I., Pitaa, I.: Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Transaction on Image Processing 16(1) (2007)Google Scholar
  13. 13.
    Riaz, Z., et al.: A Model Based Approach for Expression Invariant Face Recognition. In: 3rd International Conference on Biometrics, Italy (June 2009)Google Scholar
  14. 14.
    Ginneken, B., Frangi, A., Staal, J., Haar, B., Viergever, R.: Active shape model segmentation with optimal features. IEEE Transactions on Medical Imaging 21(8), 924–933 (2002)CrossRefGoogle Scholar
  15. 15.
    Romdhani, S.: Face Image Analysis using a Multiple Feature Fitting Strategy. Ph.D thesis, University of Basel, Computer Science Department, Basel, CH (January 2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Zahid Riaz
    • 1
  • Christoph Mayer
    • 1
  • Michael Beetz
    • 1
  • Bernd Radig
    • 1
  1. 1.Department of InformaticsTechnische Universität MünchenGarchingGermany

Personalised recommendations