Real-World Facial Expression Recognition Using Metric Learning Method

  • Zhiwen Liu
  • Shan Li
  • Weihong Deng
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9967)


Real-world human facial expressions recognition has great value in Human-Computer Interaction. Currently facial expression recognition methods perform quite poor in real-world compared with in traditional laboratory conditions. A key factor is the lack of reliable large real-world facial expression database. In this paper, a large and reliable real-world facial expression database and a Modified Metric Learning Method based on NCM classifier (PR-NCMML) to regress the probability distribution of emotional labels will be introduced. According to experiments, the six-dimension emotion probability vector derived by PR-NCMML is closer to human perception, which leads to better accuracy than the state-of-the-art methods, such as the SVM based algorithms, both dominant emotion prediction and multi-label emotion recognition.


Real-world facial expressions Metric Learning Probability regression 



This work was partially sponsored by supported by the NSFC (National Natural Science Foundation of China) under Grant No. 61375031, No. 61573068, No. 61471048, and No.61273217, the Fundamental Research Funds for the Central Universities under Grant No. 2014ZD03-01, This work was also supported by Beijing Nova Program, CCF-Tencent Open Research Fund, and the Program for New Century Excellent Talents in University.


  1. 1.
    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: 2000 Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53. IEEE (2000)Google Scholar
  2. 2.
    Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression (PIE) database. In: 2002 Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–51. IEEE (2002)Google Scholar
  3. 3.
    Dhall, A., Goecke, R., Lucey, S., Gedeon, T.: Acted facial expressions in the wild database. Australian National University, Canberra, Australia, Technical report TR-CS-11-02 (2011)Google Scholar
  4. 4.
    Ekman, P., Scherer, K.: Expression and the nature of emotion. Approaches to emotion 3, 19–344 (1984)Google Scholar
  5. 5.
    Kotsia, I., Pitas, I.: Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans. Image Process. 16(1), 172–187 (2007)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Anderson, K., McOwan, P.W.: A real-time automated system for the recognition of human facial expressions. IEEE Trans. Syst. Man Cybern. Part B Cybern. 36(1), 96–105 (2006)CrossRefGoogle Scholar
  7. 7.
    Buciu, I., Pitas, I.: Application of non-negative and local non negative matrix factorization to facial expression recognition. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR2004, vol. 1, pp. 288–291. IEEE (2004)Google Scholar
  8. 8.
    Wang, J., Yin, L.: Static topographic modeling for facial expression recognition and analysis. Comput. Vis. Image Underst. 108(1), 19–34 (2007)CrossRefGoogle Scholar
  9. 9.
    Sebe, N., Lew, M.S., Sun, Y., Cohen, I., Gevers, T., Huang, T.S.: Authentic facial expression analysis. Image Vis. Comput. 25(12), 1856–1863 (2007)CrossRefGoogle Scholar
  10. 10.
    Mensink, T., Verbeek, J., Perronnin, F., Csurka, G.: Distance-based image classification: generalizing to new classes at near-zero cost. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2624–2637 (2013)CrossRefGoogle Scholar
  11. 11.
    Moore, S., Bowden, R.: Local binary patterns for multi-view facial expression recognition. Comput. Vis. Image Underst. 115(4), 541–558 (2011)CrossRefGoogle Scholar
  12. 12.
    Dahmane, M., Meunier, J.: Emotion recognition using dynamic grid-based hog features. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 884–888. IEEE (2011)Google Scholar
  13. 13.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1–27:27 (2011). Software available at CrossRefGoogle Scholar
  14. 14.
    Duin, R.: PRtools version 3.0: a matlab toolbox for pattern recognition. In: Proceedings of SPIE, Citeseer (2000)Google Scholar
  15. 15.
    Deng, H.B., Jin, L.W., Zhen, L.X., Huang, J.C.: A new facial expression recognition method based on local Gabor filter bank and PCA plus LDA. Int. J. Inf. Technol. 11(11), 86–96 (2005)Google Scholar
  16. 16.
    Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recogn. 37(9), 1757–1771 (2004)CrossRefGoogle Scholar
  17. 17.
    Zhang, M-L., Zhou, Z-H.: Multi-instance multi-label learning with application to scene classificationGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.School of Information and Communication EngineeringBeijing University of Posts and TelecommunicationsBejingChina

Personalised recommendations