Abstract
We developed an algorithmic scheme to extract the semantical description of the face and the face motion from an image sequence, and to re-play this action in a 3-dimensional (3D) virtual world. The presented Facial Analysis and Synthesis Scheme combines new methods for detection and tracking of the face and facial features, for estimating the 3D face movements and the nonrigid facial expressions, and for extracting the MPEG4 facial animation parameters. In the scheme, the face is treated either as a 2D object that has specific color, shape and motion characteristics, either as a 3D model that is calibrated and moved using a natural displacement-based deformation model. A dynamic MPEG4 displacement table takes care of the semantical controls of the animations of the face model. As a result, this virtual face model mimics well the gestures of the person in the video.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
ISO/IEC: Jtc 1/sc 29/wg 11 n2501 coding of moving pictures and audio, information technology - generic coding of audio-visual objects, part1: systems, final draft international standard, Atlantic City (1998)
ISO/IEC: Jtc 1/sc 29/wg 11 n2502 coding of moving pictures and audio, information technology - generic coding of audio-visual objects, part2: visual, final draft international standard, Atlantic City (1998)
Ostermann, J., Weissenfeld, A.: Talking faces - technologies and applications. In: 17th International Conference on Pattern Recognition (ICPR 2004), Cambridge UK, August 23 - 26, 2004, vol. 3, pp. 826–833 (2004)
Ravyse, I., Enescu, V., Sahli, H.: Kernel-based head tracker for videophony. In: The IEEE International Conference on Image Processing 2005 (ICIP 2005), Genoa, Italy, September 11-14, 2005, vol. 3, pp. 1068–1071 (2005)
Ikeda, O.: Segmentation of faces in video footage using hsv color for face detection and image retrieval. In: The IEEE International Conference on Image Processing (ICIP), Barcelona Spain, September 14-17, 2003, vol. 3, pp. 913–916 (2003)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Computer Science Conference on Computer Vision and Pattern Recognition 2001, CVPR 2001, Kauai, Hawaii, December 08-14, 2001, vol. 1, pp. 511–518 (2001)
Zivkovic, Z., Kröse, B.: An em-like algorithm for color-histogram-based object tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), Washington, D.C., USA, June 27-July 02, 2004, vol. 1, pp. 798–803 (2004)
Matthews, I., Bangham, J., Harvey, R., Cox, S.: A comparison of active shape model and scale decomposition based features for visual speech recognition. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 514–528. Springer, Heidelberg (1998)
Ravyse, I., Sahli, H., Reinders, M., Cornelis, J.: Eye activity detection and recognition using morphological scale-space decomposition. In: 15th International Conference on Pattern Recognition, ICPR 2000, Barcelona, Spain, September 3-8, 2000, vol. 1, pp. 1080–1083 (2000)
Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., Salesin, D.H.: Synthesizing realistic facial expressions from photographs. In: SIGGRAPH 1998, in Computer Graphics Proceedings, Annual Conference Series, pp. 75–84 (1998)
Schaback, R.: Creating surfaces from scattered data using radial basis functions. In: Daehlen, M., Lyche, T., Schumaker, L.L. (eds.) Mathematical Methods for Curves and Surfaces, in Computer Aided Geometric Design III, pp. 477–496. Vanderbilt University Press, Nashville (1995)
Ravyse, I.: Facial Analysis and Synthesis. PhD thesis, Vrije Universiteit Brussel, Dept. Electronics and Informatics, Belgium (2006), online: www.etro.vub.ac.be/Personal/icravyse/RavysePhDThesis.pdf
Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 137–154 (2005)
Parke, F.I., Waters, K.: Computer Facial Animation. A K Peters (1996) ISBN 1-56881-014-8
Li, H., Lundmark, A., Forchheimer, R.: Image sequence coding at very low bitrates: A review. IEEE Transactions on Image Processing 3, 589–605 (1994)
Eisert, P.: Very Low Bit-Rate Video Coding Using 3-D Models. PhD thesis, Universitat Erlangen, Shaker Verlag, Aachen, Germany (2000) ISBN 3-8265-8308-6
Yilmaz, A., Shafique, K., Shah, M.: Estimation of rigid and nonrigid motion using anatomical face model. In: Int. Conference on Pattern Recognition, ICPR 2002, Quebec, Canada (2002)
Waters, K.: A muscle model for animating three-dimensional facial expression. Computer Graphics ACM 21, 17–24 (1987)
Lee, Y., Terzopoulos, D., Waters, K.: Constructing physicsbased facial models of individuals. In: Proceedings of the Graphics Interface 1993 Conference, Toronto, ON, Canada, pp. 1–8 (1993)
Essa, I., Basu, S., Darrell, T., Pentland, A.: Modeling, tracking, and interactive animation of faces and heads using input from video. In: Computer Animation 1996, Geneva, Switzerland (1996)
Spies, H., Jähne, B., Barron, J.L.: Range flow estimation. Computer Vision Image Understanding (CVIU 2002) 85, 209–231 (2002)
Klaus, B., Horn, P.: 12. Motion Field and Optical Flow. In: Robot Vision. The MIT Electrical Engeneering and Computer Science Series. MIT Press, Cambridge (1986)
Torresani, L., Yang, D.B., Alexander, E.J., Bregler, C.: Tracking and modeling non-rigid objects with rank constraints. In: IEEE CVPR 2001 (Best Student Paper Award) (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ravyse, I., Sahli, H. (2006). Facial Analysis and Synthesis Scheme. In: Blanc-Talon, J., Philips, W., Popescu, D., Scheunders, P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2006. Lecture Notes in Computer Science, vol 4179. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11864349_74
Download citation
DOI: https://doi.org/10.1007/11864349_74
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44630-9
Online ISBN: 978-3-540-44632-3
eBook Packages: Computer ScienceComputer Science (R0)