Emotion Detection Using Hybrid Structural and Appearance Descriptors

  • David Sanchez-Mendoza
  • David Masip
  • Xavier Baró
  • Àgata Lapedriza
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8234)

Abstract

In computer vision the facial expression recognition descriptors extracted from raw images are categorized as structural or appearance descriptors. A lot of effort has been done in the literature for improving both type of descriptors for making them more robust; in most cases, both types of descriptors have been used separately. In this work we propose a hybrid model that uses both descriptors for emotion inferring. Our model is based in detecting Action Units and uses a probabilistic approach for emotion prediction based on an ensemble of Support Vector Machine classifiers. Fully detailed inner workings of the method are provided for experiment replication as well as detailed results to assess emotion inferring performance.

Keywords

Computer Vision Machine Learning Emotion Analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ekman, P., Friesen, W.V.: Facial action coding system: A technique for the measurement of facial movement. palo alto. Consulting Psychologists Press, CA (1988); Ellsworth, P.C., Smith, C.A.: From appraisal to emotion: Differences among unpleasant feelings. Motivation and Emotion 12, 271–302 (1978)Google Scholar
  2. 2.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 94–101. IEEE (2010)Google Scholar
  3. 3.
    Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2879–2886. IEEE (2012)Google Scholar
  4. 4.
    Valstar, M., Pantic, M.: Fully automatic facial action unit detection and temporal analysis. In: Conference on Computer Vision and Pattern Recognition Workshop, CVPRW 2006, p. 149. IEEE (2006)Google Scholar
  5. 5.
    Valstar, M.F., Patras, I., Pantic, M.: Facial action unit detection using probabilistic actively learned support vector machines on tracked facial point data. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, CVPR Workshops, p. 76. IEEE (2005)Google Scholar
  6. 6.
    Rojas, M.: On the use of geometric and appearance information for facial analysis. PhD thesis, University of Barcelona (2012)Google Scholar
  7. 7.
    Bartlett, M.S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Recognizing facial expression: machine learning and application to spontaneous behavior. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 568–573. IEEE (2005)Google Scholar
  8. 8.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  9. 9.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  10. 10.
    Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(12), 2037–2041 (2006)CrossRefGoogle Scholar
  11. 11.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, pp. I–511. IEEE (2001)Google Scholar
  12. 12.
    Freund, Y., Schapire, R.E.: A desicion-theoretic generalization of on-line learning and an application to boosting. In: Vitányi, P.M.B. (ed.) EuroCOLT 1995. LNCS, vol. 904, pp. 23–37. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  13. 13.
    Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: Binary robust independent elementary features. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 778–792. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  14. 14.
    Kumano, S., Otsuka, K., Yamato, J., Maeda, E., Sato, Y.: Pose-invariant facial expression recognition using variable-intensity templates. International Journal of Computer Vision 83(2), 178–194 (2009)CrossRefGoogle Scholar
  15. 15.
    Gizatdinova, Y., Surakka, V.: Feature-based detection of facial landmarks from neutral and expressive facial images. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(1), 135–139 (2006)CrossRefGoogle Scholar
  16. 16.
    Vapnik, V.: The nature of statistical learning theory. Springer (2000)Google Scholar
  17. 17.
    Chang, C.-C., Lin, C.-J.: Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST) 2(3), 27 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • David Sanchez-Mendoza
    • 1
  • David Masip
    • 1
  • Xavier Baró
    • 1
  • Àgata Lapedriza
    • 1
  1. 1.Scene Understanding and Artificial Intelligence Lab (SUNAI), Internet Interdisciplinary Institute (IN3)Open University of Catalonia (UOC)BarcelonaSpain

Personalised recommendations