Joint Modeling of Facial Expression and Shape from Video

  • T. Tamminen
  • J. Kätsyri
  • M. Frydrych
  • J. Lampinen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3540)


In this paper, we present a novel model for representing facial feature point tracks during an facial expression. The model is composed of a static shape part and a time-dependent expression part. We learn the model by tracking the points of interest in video recordings of trained actors making different facial expressions. Our results indicate that the proposed sum of two linear models – a person-dependent shape model and a person-independent expression model – approximates the true feature point motion well.


Facial Expression Feature Point Video Sequence Joint Modeling Shape Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Adolphs, R.: Recognizing emotion from facial expressions: psychological and neurological mechanisms. Behavioral and Cognitive Neuroscience Reviews 1(1), 21–62 (2002)CrossRefGoogle Scholar
  2. 2.
    Ekman, P., Friesen, W., Hager, J.: Facial Action Coding System. Consulting Psychologists Press, Palo Alto (1978)Google Scholar
  3. 3.
    Essa, I.A., Pentland, A.P.: Coding, analysis, interpretation and recognition of facial expressions. IEEE TPAMI 19(7), 757–763 (1997)Google Scholar
  4. 4.
    Tian, Y.-l., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE TPAMI 23(2), 97–115 (2001)Google Scholar
  5. 5.
    Donato, G., Bartlett, M.S., Hager, J.C., Ekman, P., Sejnowski, T.J.: Classifying facial actions. IEEE TPAMI 21(10), 974–989 (1999)Google Scholar
  6. 6.
    Ekman, P.: Expression and the nature of emotion. In: Scherer, K., Ekman, P. (eds.) Approaches to Emotion. Lawrence Erlbaum, Mahwah (1984)Google Scholar
  7. 7.
    Ekman, P., Friesen, W.: Unmasking the Face. A Guide to Recognizing Emotions from Facial Expressions. Consulting Psychologists Press, Palo Alto (1975)Google Scholar
  8. 8.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proc. Imaging Understanding Workshop, pp. 121–130 (1981)Google Scholar
  9. 9.
    Tomasi, C., Kanade, T.: Detection and tracking of feature points, Carnegie Mellon University Technical Report CMU-CS-91-132 (1991)Google Scholar
  10. 10.
    Bourel, F., Chibelushi, C.C., Low, A.A.: Robust Facial Feature Tracking. In: Proc. BMVC 2000 (2000)Google Scholar
  11. 11.
    Daugman, J.G.: Complete discrete 2-d Gabor transforms by neural networks for image analysis and compression. IEEE TASSP 36(7), 1169–1179 (1988)zbMATHGoogle Scholar
  12. 12.
    Gelman, A., Carlin, J.B., Stern, H.S., Rubin, D.R.: Bayesian Data Analysis, 2nd edn. Chapman & Hall, Boca Raton (2004)zbMATHGoogle Scholar
  13. 13.
    McKenna, S.J., Gong, S., Wurtz, R.P., Tanner, J., Banin, D.: Tracking facial feature points with Gabor wavelets and shape models. In: Proc. 1st International Conference on Audio- and Video-based Biometric Person Authentication (1997)Google Scholar
  14. 14.
    Wiskott, L., Fellous, J.-M., Krüger, N., von der Malsburg, C.: Face Recognition by Elastic Bunch Graph Matching. In: Jain, L.C., Halici, U., Hayashi, I., Lee, S.B. (eds.) Intelligent Biometric Techniques in Fingerprint and Face Recognition, CRC Press, Boca Raton (1999)Google Scholar
  15. 15.
    Tamminen, T., Lampinen, J.: Bayesian object matching with hierarchical priors and Markov chain Monte Carlo. In: Bernardo, J.M., Bayarri, M.J., Berger, J.O., Dawid, A.P., Heckerman, D., Smith, A.F.M., West, M. (eds.) Bayesian Statistics 7. Oxford University Press, Oxford (2003)Google Scholar
  16. 16.
    Abboud, B., Davoine, F.: Appearance factorization for facial expression analysis. In: Proc. BMVC 2004, pp. 507–516 (2004)Google Scholar
  17. 17.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE TPAMI 23(6), 681–685 (2001)Google Scholar
  18. 18.
    Chatfield, C., Collins, A.J.: Introduction to Multivariate Analysis. Chapman & Hall, Boca Raton (1995)Google Scholar
  19. 19.
    Frydrych, M., Kätsyri, J., Dobšík, M., Sams, M.: Toolkit for animation of Finnish talking head. In: Proc. AVSP 2003, pp. 199–204 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • T. Tamminen
    • 1
  • J. Kätsyri
    • 1
  • M. Frydrych
    • 1
  • J. Lampinen
    • 1
  1. 1.Laboratory of Computational EngineeringHelsinki University of TechnologyFinland

Personalised recommendations