When and How to Smile: Emotional Expression for 3D Conversational Agents

  • Thi Duyen Ngo
  • The Duy Bui
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5044)


Conversational agents have become more and more common in the multimedia worlds of films, educative applications, e - business, computer games. Many techniques have been developed to enable these agents to behave in a human-like manner. In order to do so, conversational agents are simulated with emotion and personality as well as communicative channels such as voice, head and eye movement, manipulator and facial expression. Up to now, creating facial expression from emotions has received much attention. However, most of the work concentrates on producing static facial expressions from emotions. In this paper, we propose a scheme for displaying continuous emotional states of a conversational agent on a 3D face. The main idea behind the scheme is that an emotional facial expression happens for a few seconds only when there is a significant change in the emotional states. This makes the emotional facial expressions of the conversational agents more realistic due to the fact that a facial expression only stay on the face for a few seconds.


Human Computer Interaction 3D Conversational Agents Emotional Facial Expressions 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Albrecht, I.: - Faces and Hands- Modeling and animating anatomical and photorealistic models with regard to the communicative competence of virtual humans. PhD thesis, University at des Saarlandes (2005)Google Scholar
  2. 2.
    Albrecht, I., Haber, J., Kähler, K., Schröder, M., Seidel, H.P.: May i talk to you? facial animation from text. In: Proceedings Pacific Graphics 2002, pp. 77–86 (2002)Google Scholar
  3. 3.
    Bui, T.D., Heylen, D., Nijholt, A.: Improvements on a simple muscle-based 3d face for realistic facial expressions. In: 16th Int. Conf. on Computer Animation and Social Agents, pp. 33–40. IEEE Computer Society, Los Alamitos (2003)Google Scholar
  4. 4.
    Bui, T.D., Heylen, D., Nijholt, A.: Building embodied agents that experience and express emotions: A football supporter as an example. In: Proc. CASA 2004, Computer Graphics Society (2004)Google Scholar
  5. 5.
    Bui, T.D., Heylen, D., Nijholt, A.: Combination of facial movements on a 3d talking head. In: Proc. CGI 2004. IEEE Computer Society, Los Alamitos (2004)Google Scholar
  6. 6.
    Bui, T.D., Heylen, D., Poel, M., Nijholt, A.: Generation of facial expressions from emotion using a fuzzy rule based system. In: Australian Joint Conf. on Artificial Intelligence (AI 2001), Berlin. LNCS, pp. 83–95. Springer, Heidelberg (2001)Google Scholar
  7. 7.
    Bui, T.D., Heylen, D., Poel, M., Nijholt, A.: Parlee: An adaptive plan based event appraisal model of emotions. In: Jarke, M., Koehler, J., Lakemeyer, G. (eds.) KI 2002. LNCS, vol. 2479, pp. 129–143. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Cohen, M.M., Massaro, D.W.: Modeling coarticulation in synthetic visual speech. In: Magnenat Thalmann, N., Thalmann, D. (eds.) Models and Techniques in Computer Animation, pp. 139–156. Springer, Tokyo (1993)CrossRefGoogle Scholar
  9. 9.
    Damasio, A.R.: Descartes error: Emotion, reason, and the human brain. G.P. Putnam, New York (1994)Google Scholar
  10. 10.
    Darwin, C.: The expression of the emotions in man and animals. Univerity of Chicago Press, Chicago (1872/1965)CrossRefGoogle Scholar
  11. 11.
    DeCarlo, D.C., Revilla, M.S., Venditti, J.: Making discourse visible: Coding and animating conversational facial displays. In: Computer Animation 2002(2002)Google Scholar
  12. 12.
    Ekman, P., Friesen, W.V.: Unmasking the Face: A Guide To Recognizing Emotions From Facial Clues. Prentice-Hall, Englewood Cliffs (1975)Google Scholar
  13. 13.
    El-Nasr, M.S., Yen, J., Ioerger, T.R.: FLAME-fuzzy logic adaptive model of emotions. Autonomous Agents and Multi-Agent Systems 3(3), 219–257 (2000)CrossRefGoogle Scholar
  14. 14.
    Forgas, J.P., Moylan, S.: After the movies: the effects of transient mood states on social judgments. Personality and Social Psychology Bulletin 13 (1987)Google Scholar
  15. 15.
    Galernter, D.H.: The muse in the machine. Free Press, New York (1994)Google Scholar
  16. 16.
    Hager, J.C., Ekman, P.: Essential behavioral science of the face and gesture that computer scientists need to know (1995),
  17. 17.
    Hayes-Roth, B., van Gent, R.: Story-making with improvisational puppets. In: Johnson, W.L., Hayes-Roth, B. (eds.) Proceedings of the 1st International Conference on Autonomous Agents, pp. 1–7. ACM Press, New York (1997)CrossRefGoogle Scholar
  18. 18.
    King, S.A., Parent, R.E., Olsafsky, B.: An anatomically-based 3d parametric lip model to support facial animation and synchronized speech. In: Proceedings of Deform 2000, pp. 7–19 (2000)Google Scholar
  19. 19.
    Kshirsagar, S., Magnenat-Thalmann, N.: A multilayer personality model. In: Proceedings of 2nd International Symposium on Smart Graphics, pp. 107–115. ACM Press, New York (2002)CrossRefGoogle Scholar
  20. 20.
    Kurlander, D., Skelly, T., Salesin, D.: Comic chat. In: SIGGRAPH 1996: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pp. 225–236 (1996)Google Scholar
  21. 21.
    Latta, C., Alvarado, N., Adams, S.S., Burbeck, S.: An expressive system for animating characters or endowing robots with affective displays. In: Society for Artificial Intelligence and Social Behavior (AISB), 2002 Annual Conference, Symposium on Animating Expressive Characters for Social Interactions (2002)Google Scholar
  22. 22.
    Paiva, A., Dias, J., Sobral, D., Aylett, R., Sobreperez, P., Woods, S., Zoll, C., Hall, L.: Caring for agents and agents that care: Building empathic relations with synthetic agents. In: Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 194–201. IEEE Computer Society, Los Alamitos (1996)Google Scholar
  23. 23.
    Parke, F.I., Waters, K.: Computer Facial Animation. A K Peters (1996) ISBN 1-56881-014-8Google Scholar
  24. 24.
    Perlin, K., Goldberg, A.: Improv: A system for scripting interactive actors in virtual worlds. Computer Graphics 30(Annual Conference Series), 205–216 (1996)Google Scholar
  25. 25.
    Picard, R.: Affective Computing. MIT Press, Cambridge (1997)CrossRefGoogle Scholar
  26. 26.
    Plutchik, R.: Emotions: A general psychoevolutionary theory. In: Scherer, K.R., Ekman, P. (eds.) Approaches to emotion. Lawrence Erlbaum, London (1984)Google Scholar
  27. 27.
    Raouzaiou, A., Karpouzis, K., Kollias, S.D.: Online gaming and emotion representation. In: García, N., Salgado, L., Martínez, J.M. (eds.) VLBV 2003. LNCS, vol. 2849, pp. 298–305. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  28. 28.
    Reilly, W.S.: Believable social and emotional agents. Technical Report Ph.D. Thesis. Technical Report CMU-CS-96-138, Carnegie Mellon University, Pittsburgh, PA, USA (1996)Google Scholar
  29. 29.
    Stern, A., Frank, A., Resner, B.: Virtual petz: A hybrid approach to creating autonomous, lifelike dogz and catz. In: Sycara, K.P., Wooldridge, M. (eds.) Proc. of Agents 1998, pp. 334–335. ACM Press, New York (1998)Google Scholar
  30. 30.
    EAR. Tanguy. Emotions: the Art of Communication Applied to Virtual Actors. PhD thesis, Universit of Bath (2006)Google Scholar
  31. 31.
    Velásquez, J.D.: Modeling emotions and other motivations in synthetic agents. In: Proc. (AAAI 1997/IAAI 1997), pp. 10–15. AAAI Press, Menlo Park (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Thi Duyen Ngo
    • 1
  • The Duy Bui
    • 1
  1. 1.College of TechnologyVietnam National UniversityHanoiVietnam

Personalised recommendations