Facial Analysis and Synthesis Scheme

  • Ilse Ravyse
  • Hichem Sahli
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4179)


We developed an algorithmic scheme to extract the semantical description of the face and the face motion from an image sequence, and to re-play this action in a 3-dimensional (3D) virtual world. The presented Facial Analysis and Synthesis Scheme combines new methods for detection and tracking of the face and facial features, for estimating the 3D face movements and the nonrigid facial expressions, and for extracting the MPEG4 facial animation parameters. In the scheme, the face is treated either as a 2D object that has specific color, shape and motion characteristics, either as a 3D model that is calibrated and moved using a natural displacement-based deformation model. A dynamic MPEG4 displacement table takes care of the semantical controls of the animations of the face model. As a result, this virtual face model mimics well the gestures of the person in the video.


Facial Analysis Rigid Motion Face Motion Synthesis Scheme Nonrigid Motion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    ISO/IEC: Jtc 1/sc 29/wg 11 n2501 coding of moving pictures and audio, information technology - generic coding of audio-visual objects, part1: systems, final draft international standard, Atlantic City (1998)Google Scholar
  2. 2.
    ISO/IEC: Jtc 1/sc 29/wg 11 n2502 coding of moving pictures and audio, information technology - generic coding of audio-visual objects, part2: visual, final draft international standard, Atlantic City (1998)Google Scholar
  3. 3.
    Ostermann, J., Weissenfeld, A.: Talking faces - technologies and applications. In: 17th International Conference on Pattern Recognition (ICPR 2004), Cambridge UK, August 23 - 26, 2004, vol. 3, pp. 826–833 (2004)Google Scholar
  4. 4.
    Ravyse, I., Enescu, V., Sahli, H.: Kernel-based head tracker for videophony. In: The IEEE International Conference on Image Processing 2005 (ICIP 2005), Genoa, Italy, September 11-14, 2005, vol. 3, pp. 1068–1071 (2005)Google Scholar
  5. 5.
    Ikeda, O.: Segmentation of faces in video footage using hsv color for face detection and image retrieval. In: The IEEE International Conference on Image Processing (ICIP), Barcelona Spain, September 14-17, 2003, vol. 3, pp. 913–916 (2003)Google Scholar
  6. 6.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Computer Science Conference on Computer Vision and Pattern Recognition 2001, CVPR 2001, Kauai, Hawaii, December 08-14, 2001, vol. 1, pp. 511–518 (2001)Google Scholar
  7. 7.
    Zivkovic, Z., Kröse, B.: An em-like algorithm for color-histogram-based object tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), Washington, D.C., USA, June 27-July 02, 2004, vol. 1, pp. 798–803 (2004)Google Scholar
  8. 8.
    Matthews, I., Bangham, J., Harvey, R., Cox, S.: A comparison of active shape model and scale decomposition based features for visual speech recognition. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 514–528. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  9. 9.
    Ravyse, I., Sahli, H., Reinders, M., Cornelis, J.: Eye activity detection and recognition using morphological scale-space decomposition. In: 15th International Conference on Pattern Recognition, ICPR 2000, Barcelona, Spain, September 3-8, 2000, vol. 1, pp. 1080–1083 (2000)Google Scholar
  10. 10.
    Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., Salesin, D.H.: Synthesizing realistic facial expressions from photographs. In: SIGGRAPH 1998, in Computer Graphics Proceedings, Annual Conference Series, pp. 75–84 (1998)Google Scholar
  11. 11.
    Schaback, R.: Creating surfaces from scattered data using radial basis functions. In: Daehlen, M., Lyche, T., Schumaker, L.L. (eds.) Mathematical Methods for Curves and Surfaces, in Computer Aided Geometric Design III, pp. 477–496. Vanderbilt University Press, Nashville (1995)Google Scholar
  12. 12.
    Ravyse, I.: Facial Analysis and Synthesis. PhD thesis, Vrije Universiteit Brussel, Dept. Electronics and Informatics, Belgium (2006), online:
  13. 13.
    Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 137–154 (2005)CrossRefGoogle Scholar
  14. 14.
    Parke, F.I., Waters, K.: Computer Facial Animation. A K Peters (1996) ISBN 1-56881-014-8Google Scholar
  15. 15.
    Li, H., Lundmark, A., Forchheimer, R.: Image sequence coding at very low bitrates: A review. IEEE Transactions on Image Processing 3, 589–605 (1994)CrossRefGoogle Scholar
  16. 16.
    Eisert, P.: Very Low Bit-Rate Video Coding Using 3-D Models. PhD thesis, Universitat Erlangen, Shaker Verlag, Aachen, Germany (2000) ISBN 3-8265-8308-6Google Scholar
  17. 17.
    Yilmaz, A., Shafique, K., Shah, M.: Estimation of rigid and nonrigid motion using anatomical face model. In: Int. Conference on Pattern Recognition, ICPR 2002, Quebec, Canada (2002)Google Scholar
  18. 18.
    Waters, K.: A muscle model for animating three-dimensional facial expression. Computer Graphics ACM 21, 17–24 (1987)CrossRefGoogle Scholar
  19. 19.
    Lee, Y., Terzopoulos, D., Waters, K.: Constructing physicsbased facial models of individuals. In: Proceedings of the Graphics Interface 1993 Conference, Toronto, ON, Canada, pp. 1–8 (1993)Google Scholar
  20. 20.
    Essa, I., Basu, S., Darrell, T., Pentland, A.: Modeling, tracking, and interactive animation of faces and heads using input from video. In: Computer Animation 1996, Geneva, Switzerland (1996)Google Scholar
  21. 21.
    Spies, H., Jähne, B., Barron, J.L.: Range flow estimation. Computer Vision Image Understanding (CVIU 2002) 85, 209–231 (2002)MATHCrossRefGoogle Scholar
  22. 22.
    Klaus, B., Horn, P.: 12. Motion Field and Optical Flow. In: Robot Vision. The MIT Electrical Engeneering and Computer Science Series. MIT Press, Cambridge (1986)Google Scholar
  23. 23.
    Torresani, L., Yang, D.B., Alexander, E.J., Bregler, C.: Tracking and modeling non-rigid objects with rank constraints. In: IEEE CVPR 2001 (Best Student Paper Award) (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ilse Ravyse
    • 1
  • Hichem Sahli
    • 1
  1. 1.Department ETRO, Audio Visual Signal Processing (AVSP)Vrije Universiteit BrusselBrussel

Personalised recommendations