Automatic Facial Expression Recognition Using Boosted Discriminatory Classifiers

  • Stephen Moore
  • Richard Bowden
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4778)


Over the last two decades automatic facial expression recognition has become an active research area. Facial expressions are an important channel of non-verbal communication, and can provide cues to emotions and intentions. This paper introduces a novel method for facial expression recognition, by assembling contour fragments as discriminatory classifiers and boosting them to form a strong accurate classifier. Detection is fast as features are evaluated using an efficient lookup to a chamfer image, which weights the response of the feature. An Ensemble classification technique is presented using a voting scheme based on classifiers responses. The results of this research are a 6-class classifier (6 basic expressions of anger, joy, sadness, surprise, disgust and fear) which demonstrate competitive results achieving rates as high as 96% for some expressions. As classifiers are extremely fast to compute the approach operates at well above frame rate. We also demonstrate how a dedicated classifier can be consrtucted to give optimal automatic parameter selection of the detector, allowing real time operation on unconstrained video.


Facial Expression Local Binary Pattern Face Detection Gesture Recognition Facial Expression Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alessandro, L.F.: A neural network facial expression recognition system using unsupervised local processing. In: ISPA 2001. 2nd international symposium on image and signal processing and analysis, Pula, CROATIE, pp. 628–632 (2001)Google Scholar
  2. 2.
    Barrow, H.G., Tenenbaum, J.M., Bolles, R.C., Wolf, H.C.: Parametric correspondence and chamfer matching: Two new techniques for image matching. In: DARPA 1977, pp. 21–27 (1977)Google Scholar
  3. 3.
    Bartlett, M., Littlewort, G., Fasel, I., Movellan, J.: Real time face detection and facial expression recognition: Development and application to human-computer interaction (2003)Google Scholar
  4. 4.
    Gong, S., McOwan, P., Shan, C.: Conditional mutual information based boosting for facial expression recognition. In: British Machine Vision Conference (2005)Google Scholar
  5. 5.
    Choudhury, T., Pentland, A.: Motion field histograms for robust modeling of facial expressions. In: ICPR 2000. Proceedings of the International Conference on Pattern Recognition (2000)Google Scholar
  6. 6.
    Dailey, M.N., Cottrell, G.W.: Pca = gabor for expression recognition. Technical report, La Jolla, CA, USA (1999)Google Scholar
  7. 7.
    Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  8. 8.
    Ekman, P., Friesen, W.V., Hager, J.C.: Facial action coding system. Palo Alto, CA, USA (1978)Google Scholar
  9. 9.
    Essa, I.A., Pentland, A.: Facial expression recognition using a dynamic model and motion energy. In: ICCV, pp. 360–367 (1995)Google Scholar
  10. 10.
    Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: International Conference on Machine Learning, pp. 148–156 (1996)Google Scholar
  11. 11.
    OpenCV User Group. OpenCV Library Wiki (2006),
  12. 12.
    Kanade, T., Tian, Y., Cohn, J.F.: Comprehensive database for facial expression analysis. In: FG 2000. Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000, p. 46. IEEE Computer Society Press, Washington, DC, USA (2000)Google Scholar
  13. 13.
    Mahamud, S., Hebert, M., Shi, J.: Object recognition using boosted discriminants. In: CVPR 2001. IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, IEEE Computer Society Press, Los Alamitos (December 2001)Google Scholar
  14. 14.
    Bassili, J.N.: Facial motion in the perception of faces and of emotional expressionGoogle Scholar
  15. 15.
    Saatci, Y., Town, C.: Cascaded classification of gender and facial expression using active appearance models. In: FGR 2006. Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, pp. 393–400. IEEE Computer Society Press, Washington, DC, USA (2006)Google Scholar
  16. 16.
    Shotton, J., Blake, A., Cipolla, R.: Contour-based learning for object detection. In: ICCV 2005. Proceedings of the Tenth IEEE International Conference on Computer Vision, vol. 1, pp. 503–510. IEEE Computer Society Press, Washington, DC, USA (2005)Google Scholar
  17. 17.
    Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)CrossRefGoogle Scholar
  18. 18.
    Wang, Y., Ai, H., Wu, B., Huang, C.: Real time facial expression recognition with adaboost. In: ICPR, vol. 03, pp. 926–929 (2004)Google Scholar
  19. 19.
    Whitehill, J., Omlin, C.W.: Haar features for facs au recognition. In: FGR 2006. Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, pp. 97–101. IEEE Computer Society Press, Washington, DC, USA (2006)Google Scholar
  20. 20.
    Yacoob, Y., Davis, L.S.: Recognizing human facial expressions from long image sequences using optical flow. IEEE Trans. Pattern Anal. Mach. Intell. 18(6), 636–642 (1996)CrossRefGoogle Scholar
  21. 21.
    Zhang, Z., Lyons, M., Schuster, M., Akamatsu, S.: Comparison between geometry-based and gabor-wavelets-based facial expression recognition using multi-layer perceptron. In: FG 1998. Proceedings of the 3rd. International Conference on Face & Gesture Recognition, p. 454. IEEE Computer Society Press, Washington, DC, USA (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Stephen Moore
    • 1
  • Richard Bowden
    • 1
  1. 1.Centre for Vision Speech and Signal Processing, University of Surrey, Guildford, GU2 7JWUK

Personalised recommendations