3D Facial Expression Recognition Based on Histograms of Surface Differential Quantities

  • Huibin Li
  • Jean-Marie Morvan
  • Liming Chen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6915)


3D face models accurately capture facial surfaces, making it possible for precise description of facial activities. In this paper, we present a novel mesh-based method for 3D facial expression recognition using two local shape descriptors. To characterize shape information of the local neighborhood of facial landmarks, we calculate the weighted statistical distributions of surface differential quantities, including histogram of mesh gradient (HoG) and histogram of shape index (HoS). Normal cycle theory based curvature estimation method is employed on 3D face models along with the common cubic fitting curvature estimation method for the purpose of comparison. Based on the basic fact that different expressions involve different local shape deformations, the SVM classifier with both linear and RBF kernels outperforms the state of the art results on the subset of the BU-3DFE database with the same experimental setting.


3D facial expression recognition normal cycle theory curvature tensor histogram of surface differential quantities SVM classifier 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Otsuka, K., Sawada, H., Yamato, J.: Automatic inference of cross-modal nonverbal interactions in multiparty conversations: ”who responds to whom, when, and how?” from gaze, head gestures, and utterances. In: ICMI, pp. 255–262. ACM Press, New York (2007)CrossRefGoogle Scholar
  2. 2.
    Ekman, P.: Universals and cultural differences in facial expressions of emotion. In: Nebraska Symposium on Motivation, Lincoln, NE, pp. 207–283 (1972)Google Scholar
  3. 3.
    Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. In: Pattern Recognition, pp. 259–275 (2003)Google Scholar
  4. 4.
    Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.: A 3D Facial Expression Database For Facial Behavior Research. In: The 7th International Conference on Automatic Face and Gesture Recognition (FG), pp. 211–216. IEEE Computer Society, Los Alamitos (2006); TC PAMIGoogle Scholar
  5. 5.
    Wang, J., Yin, L., Wei, X., Sun, Y.: 3d facial expression recognition based on primitive surface feature distribution. In: CVPR, pp. 1399–1406 (2006)Google Scholar
  6. 6.
    Soyel, H., Demirel, H.: Facial expression recognition using 3d facial feature distances. In: Int. Conf. on Image Analysis and Recognition, pp. 831–838 (2007)Google Scholar
  7. 7.
    Mpiperis, I., Malassiotis, S., Strintzis, M.G.: Bilinear models for 3-d face and facial expression recognition. IEEE Transactions on Information Forensics and Security 3(3), 498–511 (2008)CrossRefGoogle Scholar
  8. 8.
    Tang, H., Huang, T.S.: 3d facial expression recognition based on automatically selected features. In: Int. Conf. on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  9. 9.
    Gong, B., Wang, Y., Liu, J., Tang, X.: Automatic facial expression recognition on a single 3d face by exploring shape deformation. In: Int. Conf. on Multimedia, pp. 569–572 (2009)Google Scholar
  10. 10.
    Maalej, A., Ben Amor, B., Daoudi, M., Srivastava, A., Berretti, S.: Local 3D Shape Analysis for Facial Expression Recognition. In: ICPR, pp. 4129–4132. IEEE, Los Alamitos (2010)Google Scholar
  11. 11.
    Berretti, S., Bimbo, A.D., Pala, P., Ben Amor, B., Daoudi, M.: A Set of Selected SIFT Features for 3D Facial Expression Recognition. In: ICPR, pp. 4125–4128. IEEE, Los Alamitos (2010)Google Scholar
  12. 12.
    Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A High-Resolution 3D Dynamic Facial Expression Database. In: IEEE Int. Conference on Automatic Face Gesture Recognition, pp. 1–6 (2008)Google Scholar
  13. 13.
    Sun, Y., Yin, L.: Facial Expression Recognition Based on 3D Dynamic Range Model Sequences. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 58–71. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  14. 14.
    Lowe, D.G.: Distinctive image features from scale invariant keypoints. IJCV, 91–110 (2004)Google Scholar
  15. 15.
    Fabry, M.C., et al.: Feature detection on 3d face surfaces for pose normalisation and recognition. In: BTAS (2010)Google Scholar
  16. 16.
    Taubin, G.: Estimating the tensor of curvature of a surface from a polyhedral approximation. In: ICCV, p. 902. IEEE Computer Society, USA (1995)Google Scholar
  17. 17.
    Cohen-Steiner, D., Morvan, J.M.: Restricted delaunay triangulations and normal cycle. In: Proceedings of the Nineteenth Annual Symposium on Computational Geometry, pp. 312–321. ACM, New York (2003)Google Scholar
  18. 18.
    Morvan, J.M.: Generalized Curvatures. Springer, Berlin (2008)CrossRefzbMATHGoogle Scholar
  19. 19.
    Alliez, P., Cohen-Steiner, D., Devillers, O., Lévy, B., Desbrun, M.: Anisotropic polygonal remeshing. ACM Trans. Graph. 22(3), 485–493 (2003)CrossRefGoogle Scholar
  20. 20.
    Goldfeather, J., Interrante, V.: A novel cubic-order algorithm for approximating principal direction vectors. ACM Trans. Graph., 45–63 (2004)Google Scholar
  21. 21.
    Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines (2001),
  22. 22.
    Zhao, X., Szeptycki, P., Dellandrea, E., Chen, L.: Precise 2.5D Facial Landmarking via an Analysis by Synthesis approach. In: 2009 IEEE Workshop on Applications of Computer Vision (WACV 2009), Snowbird, Utah (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Huibin Li
    • 1
    • 2
  • Jean-Marie Morvan
    • 1
    • 3
    • 4
  • Liming Chen
    • 1
    • 2
  1. 1.Université de Lyon, CNRSFrance
  2. 2.Ecole Centrale de Lyon, LIRIS UMR5205LyonFrance
  3. 3.Institut Camille JordanUniversité Lyon 1Villeurbanne - CedexFrance
  4. 4.GMSV Research CenterKing Abdullah University of Science and TechnologyThuwalSaudi Arabia

Personalised recommendations