Skip to main content
SpringerLink
Log in

Texture-independent recognition of facial expressions in image snapshots and videos

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper addresses the static and dynamic recognition of basic facial expressions. It has two main contributions. First, we introduce a view- and texture-independent scheme that exploits facial action parameters estimated by an appearance-based 3D face tracker. We represent the learned facial actions associated with different facial expressions by time series. Second, we compare this dynamic scheme with a static one based on analyzing individual snapshots and show that the former performs better than the latter. We provide evaluations of performance using three subspace learning techniques: linear discriminant analysis, non-parametric discriminant analysis and support vector machines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Several parameters relative to K-NN and SVM have been tested, but only the best results are shown.

References

  1. Abboud, B., Davoine, F.: Facial expression recognition and synthesis based on appearance model. Signal Process. Image Commun. 19(8), 723–740 (2004)

    Article  Google Scholar 

  2. Ahlberg, J.: Model-based coding: extraction, coding and evaluation of face model parameters. Ph.D. Thesis, Department of Electrical Engineering, Linköping University, Sweden (2002)

  3. Ambadar, Z., Schooler, J., Cohn, J.: Deciphering the enigmatic face: the importance of facial dynamics to interpreting subtle facial expressions. Psychol. Sci. 16(5), 403–410 (2005)

    Article  Google Scholar 

  4. Bartlett, M., Littlewort, G., Lainscsek, C., Fasel, I., Movellan, J.: Machine learning methods for fully automatic recognition of facial expressions and facial actions. In: Proceedings of IEEE International Conference on Systems, Man and Cybernetics, vol. I, pp. 592–597. The Hague, The Netherlands (2004)

  5. Black, M.J., Yacoob, Y.: Recognizing facial expressions in image sequences using local parameterized models of image motion. Int. J. Comp. Vis. 25(1), 23–48 (1997)

    Article  Google Scholar 

  6. Breazeal, C.: Robot in society: friend or appliance? In: Proceedings of Workshop on Emotion-Based Agent Architectures. Seattle (1999)

  7. Breazeal, C.: Sociable machines: expressive social exchange between humans and robots. PhD Thesis, Department of Electrical Engineering and Computer Science, MIT, Cambridge, USA (2000)

  8. Cañamero, L., Gaussier, P.: Emotion understanding: robots as tools and models. In: Nadel, J., Muir, D. (eds.) Emotional Development: Recent Research Advances, pp. 235–258. Oxford Univesity Press, Oxford (2005)

    Google Scholar 

  9. Cristianini, N., Shawe-Taylor, J.: Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge (2000)

    Book  Google Scholar 

  10. Dornaika, F., Davoine, F.: View and texture-independent facial expression recognition in videos using dynamic programming. In: Proceedings of IEEE International Conference on Image Processing, Genova, Italy (2005)

  11. Dornaika, F., Davoine, F.: On appearance based face and facial action tracking. IEEE Trans. Circuits Syst. Video Technol. 16(9), 1107–1124 (2006)

    Article  Google Scholar 

  12. Dornaika, F., Raducanu, B.: Inferring facial expressions from videos: tool and application. Signal Process. Image Commun. 22(9), 769–784 (2007)

    Article  Google Scholar 

  13. Ekman, P.: Facial expressions of emotions: an old controversy and new findings. Philos. Trans. Royal Soc. Lond. B335, 63–69 (1992)

    Article  Google Scholar 

  14. Ekman, P.: Facial expression and emotion. Am. Psychol. 48(4), 384–392 (1993)

    Article  Google Scholar 

  15. Ekman, P., Davidson, R.: The Nature of Emotion: Fundamental Questions. Oxford University Press, New York (1994)

    Google Scholar 

  16. Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recogn. 36(1), 259–275 (2003)

    Article  MATH  Google Scholar 

  17. Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press, Boston (1990)

    MATH  Google Scholar 

  18. Huang, X., Zhao, G., Pietikäinen, M., Zheng, W.: Expression recognition in videos using a weighted component-based feature descriptor. In: Heyden, A., Kahl, F. (eds.) Image Analysis. Springer, Berlin (2011)

  19. Kanade, T., Cohn, J., Tian, Y.L.: Comprehensive database for facial expression analysis. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53. Grenoble, France (2000)

  20. Kim, Y., Lee, S., Kim, S., Park, G.: A fully automatic system recognizing human facial expressions. In: Negoita, M., Howlett, R.J., Jain, L.C. (eds.) Knowledge-Based Intelligent Information and Engineering Systems. Springer, Berlin (2004)

  21. Meng, H., Romera-Paredes, B., Bianchi-Berthouze, N.: Emotion recognition by two view SVM_2K classifier on dynamic facial expression features. In: Proceedings of IEEE International Conference on Face and Gesture Recognition—Workshop on Facial Expression Recognition and Analysis Challenge, Santa Barbara, 25 March 2011

  22. Moore, S., Bowden, R.: Local binary patterns for multi-view facial expression recognition. Comp. Vis. Image Underst. 115, 541–558 (2011)

    Google Scholar 

  23. Moore, S., Ong, E., Bowden, R.: Facial expression recognition using spatiotemporal boosted discriminator classifiers. In: Campilho A., Kamel, M. (eds.) International Conference on Image Analysis and Recognition, Springer, Berlin (2010)

  24. Müller, M.: Information Retrieval for Music and Motion. Springer, Berlin (2007)

    Book  Google Scholar 

  25. Pantic, M.: Affective Computing. In: Pagani, M. (ed.) Encyclopedia of Multimedia Technology and Networking vol. I, pp. 8–14. Idea Group Publishing, USA (2005)

  26. Picard, R.W., Vyzas, E., Healy, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)

    Google Scholar 

  27. Robin, T., Bierlairey, M., Cruz, J.: Dynamic facial expression recognition with a discrete choice model. J. Choice Model. 2(1), 95–148 (2011)

    Google Scholar 

  28. Shan, C., Gong, S., McOwan, P.W.: Robust facial expression recognition using local binary patterns. In: Proceedings of International Conference on Image Processing, vol. II, pp. 370–373. Genoa, Italy (2005)

  29. Shan, C., Gong, S., McOwan, P.W.: Dynamic facial expression recognition using a bayesian temporal manifold model. In: Proceedings of British Machine Vision Conference, vol. I, pp. 297–306. Edinburgh, UK (2006)

  30. Sung, J., Lee, S., Kim D.: A real-time facial expression recognition using the STAAM. In: Proceedings of International Conference on Pattern Recognition, vol. I, pp. 275–278. Hong Kong, PR China (2006)

  31. Tian, Y.: Evaluation of face resolution for expression analysis. In: Proceedings of IEEE Workshop on Face Processing in Video. Washington, DC, USA (2004)

  32. Tian, Y., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23, 97–115 (2001)

    Article  Google Scholar 

  33. Valstar, M.F., Jiang, B., Mehu, M., Pantic, M., Scherer, K.: The first facial expression recognition and analysis challenge. In: Proceedings of IEEE International Conference on Face and Gesture Recognition—Workshop on Facial Expression Recognition and Analysis Challenge, Santa Barbara, 25 March 2011

  34. Wu, T., Bartlett, M.S., Movellan, J.R.: Facial expression recognition using gabor motion energy filters. In: Computer Vision and Pattern Recognition Workshops (CVPRW). San Francisco, USA (2010)

  35. Xiang, T., Leung, M.K.H., Cho, S.Y.: Expression recognition using fuzzy spatio-temporal modeling. Pattern Recogn. 41(1), 204–216 (2008)

    Article  MATH  Google Scholar 

  36. Yeasin, M., Bullot, B., Sharma, R.: Recognition of facial expressions and measurement of levels of interest from video. IEEE Trans. Multimed. 8(3), 500–508 (2006)

    Article  Google Scholar 

  37. Zhang, Y., Ji, Q.: Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 699–714 (2005)

    Google Scholar 

  38. Zhang, S., Zhao, X., Lei, B.: Robust facial expression recognition via compressive sensing. Sensors 12, 3747–3761 (2012)

    Article  Google Scholar 

Download references

Acknowledgments

B. Raducanu is supported by the projects TIN2009-14404-C02 and CONSOLIDER-INGENIO 2010 (CSD2007-00018), Ministerio de Ciencia e Innovacion, Spain. This work is supported in part by the Spanish Government under the project TIN2010-18856.

Author information

Authors and Affiliations

  1. Computer Vision Center, Edifici “O”, Campus UAB, 08193 , Bellaterra, Barcelona, Spain

    Bogdan Raducanu

  2. University of the Basque Country UPV/EHU, Paseo Manuel de Lardizabal, 1, 20018 , San Sebastian, Spain

    Fadi Dornaika

  3. IKERBASQUE, Basque Foundation for Science, 48011 , Bilbao, Spain

    Fadi Dornaika

Authors
  1. Bogdan Raducanu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fadi Dornaika
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bogdan Raducanu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Raducanu, B., Dornaika, F. Texture-independent recognition of facial expressions in image snapshots and videos. Machine Vision and Applications 24, 811–820 (2013). https://doi.org/10.1007/s00138-012-0447-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-012-0447-z

Keywords

Search

Navigation