Synthesis of expressive facial animations: A multimodal caricatural mirror
- 90 Downloads
This paper describes a natural and intuitive way to create expressive facial animations, using a novel approach based on the so-called ‘multimodal caricatural mirror’ (MCM). Taking as an input an audio-visual video sequence of the user’s face, the MCM generates a facial animation, in which the prosody and the facial expressions of emotions can either be reproduced or amplified. The user can thus simulate an emotion and see almost instantly the animation it produced, like with a regular mirror. In addition, the MCM also enables to amplify the emotions of selected parts of the input video sequence, leaving other parts unchanged. It therefore constitutes a novel approach to the design of very expressive facial animation, as the affective content of the animation can be modified by post-processing operations.
KeywordsFace Analysis Prosody Analysis Facial Animation Spoken Language Processing Multimodal Interfaces
Unable to display preview. Download preview PDF.
- M. S. Bartlett, G. Littlewort, I. Fasel, and J. R. Movellan, “Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction”, inProceedings of Conference on Computer Vision and Pattern Recognition Workshop, vol. 5, (Madison, Wisconsin), pp. 53–58, 16–22 June 2003. 21CrossRefGoogle Scholar
- Z. Deng, U. Neumann, J. Lewis, T.-Y. Kim, M. Bulut, and S. Narayanan, “Expressive Facial Animation Synthesis by Learning Speech Coarticulation and Expression Spaces”,IEEE Transactions on Visualization and Computer Graphics, vol. 12, pp. 1523–1534, November/December 2006. 21, 22CrossRefGoogle Scholar
- P. Ekman and W. V. Friesen,Manual for the Facial Action Coding System. Consulting Psychologists Press, 1977. 22Google Scholar
- D. Comaniciu, V. Ramesh, and P.Meer, “Real-Time Tracking of Non-Rigid Objects using Mean-Shift”, inin Proceedinds of IEEE Conference on Computer Vision and Pattern Recognition, (Hilton Head Island, South Carolina), 2000. 23Google Scholar
- J. Allen, R. Xu, and J. Jin, “Object Tracking Using CamShift Algorithm and Multiple Quantized Feature Spaces”, inin Proceedinds of the Pan-Sydney Area Workshop on Visual Information Processing (VIP2003), (Sydney, Australia), 2003. 23Google Scholar
- K. Sobottka and I. Pitas, “Segmentation and Tracking of Faces in Color Images”, inin Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition, (Killington, Vermont, USA), 14–16 October 1996. 24Google Scholar
- J. Ahlberg, “CANDIDE-3 — an updated parameterized face”, Tech. Rep. Technical report no. LiTH-ISY-R-2326, Dept. of Electrical Engineering, Linkoping University, 2001. 24, 28Google Scholar
- T. Kanade, J. F. Cohn, and Y. L. Tian, “Comprehensive database for facial expression analysis”, inin Proceedinds of the 4th IEEE International Conference on Automatic Face and Gesture Recognition (FG’00), 2000. 24, 26Google Scholar
- J. Y. Bouguet, “Pyramidal implementation of the Lucas-Kanade feature tracker”, tech. rep., Intel Corporation, Microprocessor Research Labs, 1999. 24, 25Google Scholar
- P. Ekman and W. V. Friesen,Emotion in the Human Face. New Jersey: Prentice Hall, 1975. 25Google Scholar
- P. Boersma, “Accurate short-term analysis of the fundamental frequency and the harmonics-to-noise ratio of a sampled sound”, inin Proceedings of the the Institute of Phonetic Sciences, pp. 97–110, 1995. 26Google Scholar