Facial Expression Recognition Using Nonrigid Motion Parameters and Shape-from-Shading

  • Fang Liu
  • Edwin R. Hancock
  • William A. P. Smith
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6855)


This paper presents a 3D motion based approach to facial expression recognition from video sequences. A non-Lambertian shape-from-shading (SFS) framework is used to recover 3D facial surfaces. The SFS technique avoids heavy computational requirements normally encountered by using a 3D face model. Then, a parametric motion model and optical flow are employed to obtain the nonrigid motion parameters of surface patches. At first, we obtain uniform motion parameters under the assumptions that motion due to change in expressions is temporally consistent. Then we relax the uniform motion constraint, and obtain temporal motion parameters. The two types of motion parameters are used to train and classify using Adaboost and HMM-based classifier. Experimental results show that temporal motion parameters perform much better than uniform motion parameters, and can be used to efficiently recognize facial expression.


Facial Expression Recognition SFS Nonrigid Motion 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fasel, B., Luettin, J.: Automatic facial expression analysis: A survey. Pattern Recognition 36, 259–275 (2003)CrossRefzbMATHGoogle Scholar
  2. 2.
    Bartlett, M.S., Littlewort, G., Frank, M., et al.: Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior. In: CVPR, pp. 568–573 (2005)Google Scholar
  3. 3.
    Lucey, S., Ashraf, A.B., Cohn, J.F.: Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face. In: Delac, K., Grgic, M. (eds.) Face Recognition, Vienna, Austria, pp. 275–286 (2007)Google Scholar
  4. 4.
    Chang, Y., Hu, C., Feris, R., Turk, M.: Manifold based Analysis of Facial Expression. J. Image and Vision Computing 24(6), 605–614 (2006)CrossRefGoogle Scholar
  5. 5.
    Black, M.J., Yacoob, Y.: Recognizing facial expressions in image sequences using local parameterized models of image motion. International Journal of Computer Vision 25, 23–48 (1997)CrossRefGoogle Scholar
  6. 6.
    Zhu, Z., Ji, Q.: Robust pose invariant facial feature detection and tracking in real-time. In: Proceedings of the 18th ICP Recognition, pp. 1092–1095 (2006)Google Scholar
  7. 7.
    Zhou, L., Kambhamettu, C.: Hierarchical structure and nonrigid motion recovery from 2d monocular views. In: CVPR, pp. 752–759 (2000)Google Scholar
  8. 8.
    Otsuka, T., Ohya, J.: Spotting segments displaying facial expression from image sequences using hmm. In: Proceedings of the 2nd IEEE International Conference on Automatic Face and Gesture Recognition, pp. 442–447 (1998)Google Scholar
  9. 9.
    Aleksic, P.S., Katsaggelos, A.K.: Automatic facial expression recognition using facial animation parameters and multistream hmms. IEEE Transactions on Information Forensics and Security (2006)Google Scholar
  10. 10.
    Smith, W.A.P., Hancock, E.R.: A new framework for grayscale and colour non-lambertian shape-from-shading. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007, Part II. LNCS, vol. 4844, pp. 869–880. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  11. 11.
    Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artifical Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  12. 12.
    Kanade, T., Tian, Y., Cohn, J.F.: Comprehensive database for facial expression analysis. In: Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Fang Liu
    • 1
  • Edwin R. Hancock
    • 2
  • William A. P. Smith
    • 2
  1. 1.School of Computer Sci. and Tech.Huazhong University of Sci. and Tech.China
  2. 2.Department of Computer ScienceThe University of YorkUK

Personalised recommendations