International Journal of Computer Vision

, Volume 50, Issue 2, pp 111–125 | Cite as

Visual Estimation and Compression of Facial Motion Parameters—Elements of a 3D Model-Based Video Coding System

  • Hai Tao
  • Thomas S. Huang


The MPEG4 standard supports the transmission and composition of facial animation with natural video by including a facial animation parameter (FAP) set that is defined based on the study of minimal facial actions and is closely related to muscle actions. The FAP set enables model-based representation of natural or synthetic talking head sequences and allows intelligible visual reproduction of facial expressions, emotions, and speech pronunciations at the receiver. This paper describes two key components we have developed for building a model-based video coding system: (1) a method for estimating FAP parameters based on our previously proposed piecewise Bézier volume deformation model (PBVD), and (2) various methods for encoding FAP parameters. PBVD is a linear deformation model suitable for both the synthesis and the analysis of facial images. Each FAP parameter is a basis function in this model. Experimental results on PBVD-based animation, model-based tracking, and spatial-temporal compression of FAP parameters are demonstrated in this paper.

nonrigid motion analysis facial animation model-based coding MPEG4 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Choi, C.S., Aizawa, K., Harashima, H., and Takebe, T. 1994. Analysis and synthesis of facial image sequences in model-based image coding. IEEE Trans. Circuit Sys. Video Technol., 4(3):257–275.Google Scholar
  2. DeCarlo, D. and Metaxas, D. 1996. The integration of optical flow and deformable models with applications to human face shape and motion estimation. In Proc. CVPR'96, pp. 231–238.Google Scholar
  3. Duda, R.O. and Hart, P.E. 1973. Pattern Classification and Scene Analysis. John Wiley: New York.Google Scholar
  4. Essa, I. and Pentland, A. 1997. Coding, analysis, interpretation and recognition of facial expressions. IEEE Trans. Pattern Analysis and Machine Intelligence, 19(7):757–763.Google Scholar
  5. ISO. 1997. Text for CD 14496-2 Video. ISO/IEC JTC1/SC29/WG11 N1902.Google Scholar
  6. Jain, K. 1989. Fundamentals of Digital Image Processing. NJ. Prentice-Hall: Englewood Cliffs.Google Scholar
  7. Kalra, P., Mangili, A., Thalmann, N.M., and Thalmann, D. 1992. Simulation of facial muscle actions based on rational free form deformations. In Proc. EUROGRAPHICS'92, pp. 59–69.Google Scholar
  8. Lee, Y., Terzopoulos, D., and Waters, K. 1995. Realistic modeling for facial animation. In Proc. SIGGRAPH 95, pp. 55–62.Google Scholar
  9. Li, H., Roivainen, P., and Forchheimer, R. 1993. 3-D motion esti-mation in model-based facial image coding. IEEE Trans. Pattern Analysis and Machine Intelligence, 15(6):545–555.Google Scholar
  10. Parke, F.I. 1982. Parameterized models for facial animation. IEEE Comput. Graph. and Appl., 2(9):61–68.Google Scholar
  11. Sederberg, T.W. and Parry, S.R. 1986. Free-form deformation of solid geometric models. In Proc. SIGGRAPH 86, pp. 151–160.Google Scholar
  12. Sirovich, L. and Everson, R. 1992. Management and analysis of large scientific datasets. Int. J. Supercomput. Appl., 6(1):50–68.Google Scholar
  13. Tao, H., Chen, H.H., and Huang, T.S. 1997. FAP Interpolation Table (FIT). ISO/IEC JTC1/SC29/WG11 MPEG97/M2599.Google Scholar
  14. Tao, H., Chen, H.H., Wu, W., and Huang, T.S. 1998. Compression of MPEG-4 facial animation parameters for transmission of talking heads. IEEE Trans. Circuit and Sys. for Video Technol., 9(2):264–276.Google Scholar
  15. Tao, H. and Huang, T.S. 1999. Explanation-based facial motion track-ing using a piecewise Bézier volume deformation model. In Proc. IEEE Comput. Vision and Patt. Recognition. (CVPR99), vol. 1, pp. 611–617.Google Scholar
  16. Williams, L. 1990. Performance-driven facial animation. In Proc. SIGGRAPH 90, pp. 235–242.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Hai Tao
    • 1
  • Thomas S. Huang
    • 2
  1. 1.Department of Computer EngineeringUniversity of CaliforniaSanta CruzUSA
  2. 2.Image Processing and Formation Laboratory, Beckman InstituteUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations