An NVC Emotional Model for Conversational Virtual Humans in a 3D Chatting Environment

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7378)


This paper proposes a new emotional model for Virtual Humans (VHs) in a conversational environment. As a part of a multi-users emotional 3D-chatting system, this paper focus on how to formulate and visualize the flow of emotional state defined by the Valence-Arousal-Dominance (VAD) parameters. From this flow of emotion over time, we successfully visualized the change of VHs’ emotional state through the proposed emoFaces and emoMotions. The notion of Non-Verbal Communication (NVC) was exploited for driving plausible emotional expressions during conversation. With the help of a proposed interface, where a user can parameterize emotional state and flow, we succeeded to vary the emotional expressions and reactions of VHs in a 3D conversation scene.


Emotional model and dynamics facial expressions 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    GUILE3D: Virtual assistant denise,
  2. 2.
    Boellstorff, T.: Coming of Age in Second Life: An Anthropologist Explores the Virtually Human. Princeton University Press (2008)Google Scholar
  3. 3.
    Cassell, J., Pelachaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., Douville, B., Prevost, S., Stone, M.: Animated conversation: rule-based generation of facial expression, gesture and spoken intonation for multiple conversational agents. In: SIGGRAPH 1994, pp. 413–420. ACM (1994)Google Scholar
  4. 4.
    Becker, C., Kopp, S., Wachsmuth, I.: Simulating the Emotion Dynamics of a Multimodal Conversational Agent. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS (LNAI), vol. 3068, pp. 154–165. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  5. 5.
    Ekman, P.: Emotions revealed. Henry Holt and Company, LLC, New York (2004)Google Scholar
  6. 6.
    Kappas, A.: Smile when you read this, whether you like it or not: Conceptual challenges to affect detection. IEEE Transactions on Affective Computing 1(1), 38–41 (2010)CrossRefGoogle Scholar
  7. 7.
    Bush, L.E.: Individual differences multidimensional scaling of adjectives denoting feelings. Journal of Personality and Social Psychology 25(1), 50–57 (1973)CrossRefGoogle Scholar
  8. 8.
    Russell, J.A.: A circumplex model of affect. Journal of Personality and Social Psychology 39(6), 1161–1178 (1980)CrossRefGoogle Scholar
  9. 9.
    Scherer, K.R.: What are emotions? and how can they be measured? Social Science Information 44(4), 695–729 (2005)CrossRefGoogle Scholar
  10. 10.
    Cassell, J., Vilhjálmsson, H.H., Bickmore, T.: Beat: Behavior expression animation toolkit. In: SIGGRAPH 2001, pp. 477–486 (2001)Google Scholar
  11. 11.
    Kopp, S., Gesellensetter, L., Krämer, N.C., Wachsmuth, I.: A Conversational Agent as Museum Guide – Design and Evaluation of a Real-World Application. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 329–343. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Becker, C., Nakasone, A., Prendinger, H., Ishizuka, M., Wachsmuth, I.: Physiologically interactive gaming with the 3d agent max. In: Intl. Workshop on Conversational Informatics, pp. 37–42 (2005)Google Scholar
  13. 13.
    Gebhard, P., Schröder, M., Charfuelan, M., Endres, C., Kipp, M., Pammi, S., Rumpler, M., Türk, O.: IDEAS4Games: Building Expressive Virtual Characters for Computer Games. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 426–440. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  14. 14.
    Swartout, W., Traum, D., Artstein, R., Noren, D., Debevec, P., Bronnenkant, K., Williams, J., Leuski, A., Narayanan, S., Piepol, D., Lane, C., Morie, J., Aggarwal, P., Liewer, M., Chiang, J.-Y., Gerten, J., Chu, S., White, K.: Ada and Grace: Toward Realistic and Engaging Virtual Museum Guides. In: Safonova, A. (ed.) IVA 2010. LNCS, vol. 6356, pp. 286–300. Springer, Heidelberg (2010)Google Scholar
  15. 15.
    Gobron, S., Ahn, J., Silvestre, Q., Thalmann, D., Rank, S., Skoron, M., Paltoglou, G., Thelwall, M.: An interdisciplinary vr-architecture for 3d chatting with non-verbal communication. In: EG VE 2011: Proceedings of the Joint Virtual Reality Conference of EuroVR. ACM (September 2011)Google Scholar
  16. 16.
    Ahn, J., Gobron, S., Silvestre, Q., Thalmann, D.: Asymmetrical facial expressions based on an advanced interpretation of two-dimensional russell’s emotional model. In: ENGAGE 2010 (2010)Google Scholar
  17. 17.
    Pelachaud, C.: Studies on gesture expressivity for a virtual agent. Speech Commun. 51(7), 630–639 (2009)CrossRefGoogle Scholar
  18. 18.
    Niewiadomski, R., Hyniewska, S.J., Pelachaud, C.: Constraint-based model for synthesis of multimodal sequential expressions of emotions. IEEE Transactions on Affective Computing 2(3), 134–146 (2011)CrossRefGoogle Scholar
  19. 19.
    Ekman, P., Friesen, W.V.: Facial action coding system. Consulting Psychologists Press, CA (1978)Google Scholar
  20. 20.
    Magnenat-Thalmann, N., Thalmann, D.: Handbook of Virtual Humans. Wiley (2004)Google Scholar
  21. 21.
    Ekman, P.: Universals and cultural differences in facial expressions of emotions. In: Nebraska Symposium on Motivation, pp. 207–283. University of Nebraska Press (1971)Google Scholar
  22. 22.
    Ekman, P.: About brows: Emotional and conversational signals. In: von Cranach, M., Foppa, K., Lepenies, W., Ploog, D. (eds.) Human Ethology, pp. 169–248. Cambridge University Press, Cambridge (1979)Google Scholar
  23. 23.
    Ekman, P., Friesen, W.V., Ancoli, S.: Facial signs of emotional experience. Journal of Personality and Social Psychology 39(6), 1125–1134 (1980)CrossRefGoogle Scholar
  24. 24.
    Gobron, S., Ahn, J., Garcia, D., Silvestre, Q., Thalmann, D., Boulic, R.: An Event-Based Architecture to Manage Virtual Human Non-Verbal Communication in 3D Chatting Environment. In: Perales, F.J., Fisher, R.B., Moeslund, T.B. (eds.) AMDO 2012. LNCS, vol. 7378, pp. 58–68. Springer, Heidelberg (2012)Google Scholar
  25. 25.
    Paltoglou, G., Gobron, S., Skowron, M., Thelwall, M., Thalmann, D.: Sentiment analysis of informal textual communication in cyberspace. In: ENGAGE 2010, pp. 13–15 (2010)Google Scholar
  26. 26.
    Bradley, M.M., Lang, P.J.: Affective norms for english words (ANEW): Stimuli, instruction manual and affective ratings. Technical report, The Center for Research in Psychophysiology. University of Florida (1999)Google Scholar
  27. 27.
    Pennebaker, J., Francis, M., Booth, R.: Linguistic Inquiry and Word Count, 2nd edn. Erlbaum Publishers (2001)Google Scholar
  28. 28.
    Schweitzer, F., Garcia, D.: An agent-based model of collective emotions in online communities. The European Physical Journal B - Condensed Matter and Complex Systems 77, 533–545 (2010)CrossRefGoogle Scholar
  29. 29.
    Garcia, D., Schweitzer, F.: Emotions in product reviews empirics and models. In: IEEE SocialCom 2011 and PASSAT 2011, 483–488 (2011)Google Scholar
  30. 30.
    Schweitzer, F.: Brownian Agents and Active Particles. Collective Dynamics in the Natural and Social Sciences, 1st edn. Springer Series in Synergetics. Springer (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Immersive Interaction Group (IIG)EPFLLausanneSwitzerland
  2. 2.Information and Communication Systems (ISIC)HE-ArcSt-ImierSwitzerland
  3. 3.Chair of Systems Design (CSD)ETHZZurichSwitzerland
  4. 4.Institute for Media Innovation (IMI)NTUSingapore

Personalised recommendations