Recognizing Compound Emotional Expression in Real-World Using Metric Learning Method

  • Zhiwen Liu
  • Shan Li
  • Weihong Deng
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9967)


Understanding human facial expressions plays an important role in Human-Computer-Interaction (HCI). Recent achievements on automatically recognizing facial expressions are mostly based on lab-controlled databases, in which facial images are a far cry from those in the real world. The main contribution of this paper is listed in the following three points. First, a large real-world facial expression database (RAF-DB), with nearly 30,000 images collected from Flickr and labeled by 300 volunteers will be introduced. Second, for the reason that human emotions are much more complexed than the six-basic-emotion defined by Ekman et al., we re-categories real-world facial expressions as compound emotional expressions, which can explain human emotions better. Finally, a metric learning method as well as several state-of-the-art facial expression classifying methods including SVM, are used to recognize our compound expression dataset. And we found that metric learning method performed better than other classifications.


Compound facial expressions Large Real-world Database Metric learning 



This work was partially sponsored by supported by the NSFC (National Natural Science Foundation of China) under Grant No. 61375031, No. 61573068, No. 61471048, and No. 61273217, the Fundamental Research Funds for the Central Universities under Grant No. 2014ZD03-01, This work was also supported by Beijing Nova Program, CCF-Tencent Open Research Fund, and the Program for New Century Excellent Talents in University.


  1. 1.
    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53. IEEE (2000)Google Scholar
  2. 2.
    Lyons, M., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding facial expressions with gabor wavelets. In: Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 200–205. IEEE (1998)Google Scholar
  3. 3.
    Ekman, P.: An argument for basic emotions. Cogn. Emot. 6(3–4), 169–200 (1992)CrossRefGoogle Scholar
  4. 4.
    Du, S., Tao, Y., Martinez, A.M.: Compound facial expressions of emotion. Proc. Nat. Acad. Sci. 111(15), E1454–E1462 (2014)CrossRefGoogle Scholar
  5. 5.
    Kotsia, I., Pitas, I.: Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans. Image Process. 16(1), 172–187 (2007)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Luo, Y., Wu, C.M., Zhang, Y.: Facial expression recognition based on fusion feature of PCA and LBP with SVM. Optik - Int. J. Light Electron Opt. 124(17), 2767–2770 (2013)CrossRefGoogle Scholar
  7. 7.
    Deng, H.B., Jin, L.W., Zhen, L.X., Huang, J.C.: A new facial expression recognition method based on local gabor filter bank and PCA plus LDA. Int. J. Inf. Technol. 11(11), 86–96 (2005)Google Scholar
  8. 8.
    George, T., Potty, S.P., Jose, S.: Smile detection from still images using KNN algorithm. In: International Conference on Control, Instrumentation, Communication and Computational Technologies, pp. 461–465 (2014)Google Scholar
  9. 9.
    Mensink, T., Verbeek, J., Perronnin, F., Csurka, G.: Distance-based image classification: generalizing to new classes at near-zero cost. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2624–2637 (2013)CrossRefGoogle Scholar
  10. 10.
    Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1–27:27 (2011). Software available at
  11. 11.
    Buciu, I., Pitas, I.: Application of non-negative and local non negative matrixfactorization to facial expression recognition. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 1, pp. 288–291. IEEE (2004)Google Scholar
  12. 12.
    Sebe, N., Lew, M.S., Sun, Y., Cohen, I., Gevers, T., Huang, T.S.: Authentic facial expression analysis. Image Vis. Comput. 25(12), 1856–1863 (2007)CrossRefGoogle Scholar
  13. 13.
    Yun, W.H., Kim, D.H., Park, C., Kim, J.: Hybrid facial representations for emotion recognition. ETRI J. 35(6), 1021–1028 (2013)CrossRefGoogle Scholar
  14. 14.
    Duin, R.: Prtools version 3.0: a matlab toolbox for pattern recognition. In: Proceedings of SPIE. Citeseer (2000)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.School of Information and Communication EngineeringBeijing University of Posts and TelecommunicationsBeijingChina

Personalised recommendations