Optimizing Eigenfaces by Face Masks for Facial Expression Recognition

  • Carmen Frank
  • Elmar Nöth
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2756)


A new direction in improving modern dialogue systems is to make a human-machine dialogue more similar to a human-human dialogue. This can be done by adding more input modalities. One additional modality for automatic dialogue systems is the facial expression of the human user. A common problem in a human-machine dialogue where the angry face may give a clue is the recurrent misunderstanding of the user by the system. Or an helpless face may indicate a naive user who does not know how to utilize the system and should be led through the dialogue step by step.

This paper describes recognizing facial expressions in frontal images using eigenspaces. For the classification of facial expressions, rather than using the face whole image we classify regions which do not differ between subjects and at the same time are meaningful for facial expressions.

Important regions change when projecting the same face to eigenspaces trained with examples of different facial expressions. The average of different faces showing different facial expressions forms a face mask. This face mask fades out unnecessary or mistakable regions and emphasizes regions changing between facial expressions.

Using this face mask for training and classification of neutral and angry expressions of the face, we achieved an improvement of up to 5 % points. The proposed method may improve other classification problems that use eigenspace methods as well.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bel96.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. In: European Conference on Computer Vision 1996, pp. 45–58 (1996)Google Scholar
  2. Cha98.
    Chai, D., Ngan, K.N.: Locating Facial Regions of a Head–and–Shoulders Color Image. In: Proceedings of the International Conference on Automatic Face and Gesture Recognition, pp. 124–129 (1998)Google Scholar
  3. Ekm78.
    Ekman, P., Friesen, W.: The Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)Google Scholar
  4. Ess95.
    Essa, I., Pentland, A.: Facial Expression Recognition Using a Dynamic Model and Motion Energy. In: Proceedings of the Fifth International Conference on Computer Vision, pp. 360–367 (1995)Google Scholar
  5. Hei00.
    Heisele, B., Poggio, T., Pontil, M.: Face Detection in Still Gray Images. In: MIT AI Memo, AIM–1687 (2000)Google Scholar
  6. Jon99.
    Jones, M., Rehg, J.: Statistical Color Models with Application to Skin Detection. In: Proceedings of Computer Vision and Pattern Recognition, pp. I:274–280 (1999)Google Scholar
  7. Kir90.
    Kirby, M., Sirovich, L.: Application of the Karhunen-Loève Procedure for the Characterization of Human Faces. TPAMI 12(1), 103–108 (1990)Google Scholar
  8. Mar98.
    Martinez, A., Benavente, R.: The AR Face Database, Purdue University, West Lafayette, IN 47907-1285 (1998)Google Scholar
  9. Mog94.
    Moghaddam, B., Pentland, A.: Face Recognition Using View–Based and Modular Eigenspaces, in Vismod, TR-301 (1994)Google Scholar
  10. Tia01.
    Tian, Y., Kanade, T., Cohn, J.: Recognizing Action Units for Facial Expression Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 23(2), 97–115 (2001)CrossRefGoogle Scholar
  11. Tur91.
    Turk, M., Pentland, A.: Face Recognition Using Eigenfaces. In: Proceedings of Computer Vision and Pattern Recognition, pp. 586–591 (1991)Google Scholar
  12. Wah01.
    Wahlster, W., Reithinger, N., Blocher, A.: SmartKom: Multimodal Communication with a Life–Like Character. In: Eurospeech 2001, pp. 1547–1550 (2001)Google Scholar
  13. Yam00.
    Yambor, W.S., Draper, B.A., Beveridge, J.R.: Analyzing PCA–based Face Recognition Algorithms: Eigenvector Selection and Distance Measures. In: Second Workshop on Empirical Evaluation Methods in Computer Vision (2000)Google Scholar
  14. Yan99.
    Yang, M., Ahuja, M., Kriegman, D.: Face Detection using a Mixture of Factor Analyzers. In: Proceedings of the International Conference on Image Processing, vol. 3, pp. 612–616 (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Carmen Frank
    • 1
  • Elmar Nöth
    • 1
  1. 1.Lehrstuhl für MustererkennungUniversität Erlangen-NürnbergErlangenGermany

Personalised recommendations