Abstract
Speech synchronized facial animation that controls only the movement of the mouth is typically perceived as wooden and unnatural. We propose a method to generate additional facial expressions such as movement of the head, the eyes, and the eyebrows fully automatically from the input speech signal. This is achieved by extracting prosodic parameters such as pitch flow and power spectrum from the speech signal and using them to control facial animation parameters in accordance to results from paralinguistic research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
I. Albrecht, J. Haber, and H.-P. Seidel. Speech Synchronization for Physics-based Facial Animation. In Proc. WSCG 2002 pages 9–16, 2002.
D. Barr. Trouble in mind: Paralinguistic indices of effort and uncertainty in communication. InOralité et Gestualité. Actes du colloque ORAGE 2001 pages 597–600, 2001.
M. Brand. Voice Puppetry. In Proc. SIGGRAPH ‘89 pages 21–28, 1999.
J. Cassell and M. Stone. Living Hand to Mouth: Psychological Theories about Speech and Gesture in Interactive Dialogue Systems. Technical Report FS-99–03, AAAI Fall Symposium on Psychological Models of Communication in Collaborative Systems, 1999.
C. Cavé, I. Guatella, R. Bertrand, S. Santi, F. Harlay, and R. Espesser. About the relationship between eyebrow movements and f0 variations. In Proc. ICSLP ‘86 1996.
N. Chovil. Discourse-Oriented Facial Displays in Conversation. Research on Language and Social Interaction 25:163–194, 1991.
M. M. Cohen and D. W. Massaro. Modeling Coarticulation in Synthetic Visual Speech. In N. M. Magnenat-Thalmann and D. Thalmann, editors, Models and Techniques in Computer Animation pages 139–156. 1993.
J. Cosnier. Les gestes de la question. In Kerbrat-Orecchioni, editor, La Question pages 163–171. Presses Universitaires de Lyon, 1991.
P. Ekman. About brows: emotional and conversational signals. In M. v. Cranach, K. Foppa, W. Lepenies, and D. Ploog, editors, Human Ethology: Claims and limits of a new discipline: contributions to the Colloquium. pages 169–248. 1979.
J. Haber, K. Kähler, I. Albrecht, H. Yamauchi, and H.-P. Seidel. Face to Face: From Real Humans to Realistic Facial Animation. InProc. Israel-Korea Binational Conference on Geometrical Modeling and Computer Graphics pages 73–82, Oct. 2001.
D. R. Hill, A. Pearce, and B. Wyvill. Animating Speech: An Automated Approach using Speech Synthesised by Rules.The Visual Computer3(5):277–289 Mar. 1988.
D. House, J. Beskow, and B. Granström. Timing and Interaction of Visual Cues for Prominence in Audiovisual Speech Perception. In Proc. Eurospeech 2001 2001.
H. H. S. Ip and C. S. Chan. Script-Based Facial Gesture and Speech Animation Using a NURBS Based Face Model. Computers & Graphics 20(6):881–891, Nov. 1996.
T. Johnstone and K. Scherer. The Effects of Emotions on Voice Quality. In Proc. XIVth International Congress of Phonetic Sciences, 2000. in press.
K. Kähler, J. Haber, and H.-P. Seidel. Geometry-based Muscle Modeling for Facial Animation. In Proc. Graphics Interface 2001 pages 37–46, June 2001.
P. Kalra, A. Mangili, N. Magnenat-Thalmann, and D. Thalmann. SMILE: A Multilayered Facial Animation System. In Proc. IFIP WG 5.10, Tokyo, Japan, pages 189–198, 1991.
J. P. Lewis and F. I. Parke. Automated Lip-Synch and Speech Synthesis for Character Animation. In Proc. Graphics Interface ‘87 pages 143–147, Apr. 1987.
M. Lundeberg and J. Beskow. Developing a 3D-agent for the AUGUST dialogue system. In Proc. Audio-Visual Speech Processing (AVSP) ‘89 1999.
A. Paeschke, M. Kienast, and W. Sendlmeier. FO-Contours in Emotional Speech. InProc. International Congress of Phonetic Sciences ‘89pages 929–931, 1999.
F. I. Parke and K. Waters, editors. Computer Facial Animation. A K Peters, Wellesley, MA, 1996.
A. Pearce, B. Wyvill, G. Wyvill, and D. R. Hill. Speech and Expression: A Computer Solution to Face Animation. In Proc. Graphics Interface ‘86 pages 136–140, May 1986.
C. Pelachaud, N. Badler, and M. Steedman. Generating Facial Expressions for Speech. Cognitive Science 20(1):1–46,1996.
K. Sjölander. The Snack Sound Toolkit. http://www.speech.kth.se/snack/, 1997–2001.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag London
About this paper
Cite this paper
Albrecht, I., Haber, J., Seidel, HP. (2002). Automatic Generation of Non-Verbal Facial Expressions from Speech. In: Vince, J., Earnshaw, R. (eds) Advances in Modelling, Animation and Rendering. Springer, London. https://doi.org/10.1007/978-1-4471-0103-1_18
Download citation
DOI: https://doi.org/10.1007/978-1-4471-0103-1_18
Publisher Name: Springer, London
Print ISBN: 978-1-4471-1118-4
Online ISBN: 978-1-4471-0103-1
eBook Packages: Springer Book Archive