An Event-Based Architecture to Manage Virtual Human Non-Verbal Communication in 3D Chatting Environment

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7378)


Non-verbal communication (NVC) makes up about two-thirds of all communication between two people or between one speaker and a group of listeners. However, this fundamental aspect of communicating is mostly omitted in 3D social forums or virtual world oriented games. This paper proposes an answer by presenting a multi-user 3D-chatting system enriched with NVC relative to motion. This event-based architecture tries to recreate a context by extracting emotional cues from dialogs and derives virtual human potential body expressions from that event triggered context model. We structure the paper by expounding the system architecture enabling the modeling NVC in a multi-user 3D-chatting environment. There, we present the transition from dialog-based emotional cues to body language, and the management of NVC events in the context of a virtual reality client-server system. Finally, we illustrate the results with graphical scenes and a statistical analysis representing the increase of events due to NVC.


Affective architecture Social agents Virtual reality Non-verbal communication 3D-chatting Avatars 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ahn, J., Gobron, S., Garcia, D., Silvestre, Q., Thalmann, D., Boulic, R.: An NVC Emotional Model for Conversational Virtual Humans in a 3D Chatting Environment. In: Perales, F.J., Fisher, R.B., Moeslund, T.B. (eds.) AMDO 2012. LNCS, vol. 7378, pp. 47–57. Springer, Heidelberg (2012)Google Scholar
  2. 2.
    Becheiraz, P., Thalmann, D.: A model of nonverbal communication and interpersonal relationship between virtual actors. In: Proceedings of Computer Animation 1996, pp. 58–67 (June 1996)Google Scholar
  3. 3.
    Becker, C., Kopp, S., Wachsmuth, I.: Simulating the Emotion Dynamics of a Multimodal Conversational Agent. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS (LNAI), vol. 3068, pp. 154–165. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Becker, C., Nakasone, A., Prendinger, H., Ishizuka, M., Wachsmuth, I.: Physiologically interactive gaming with the 3d agent max. In: Intl. Workshop on Conversational Informatics, pp. 37–42 (2005)Google Scholar
  5. 5.
    Boellstorff, T.: Coming of Age in Second Life: An Anthropologist Explores the Virtually Human. Princeton University Press (2008)Google Scholar
  6. 6.
    Cassell, J., Vilhjálmsson, H.H., Bickmore, T.: Beat: Behavior expression animation toolkit. In: SIGGRAPH 2001, pp. 477–486 (2001)Google Scholar
  7. 7.
    de Melo, C.M., Zheng, L., Gratch, J.: Expression of Moral Emotions in Cooperating Agents. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 301–307. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Ekman, P.: Emotions revealed. Henry Holt and Company, LLC, New York (2004)Google Scholar
  9. 9.
    Gobron, S., Ahn, J., Silvestre, Q., Thalmann, D., Rank, S., Skoron, M., Paltoglou, G., Thelwall, M.: An interdisciplinary vr-architecture for 3d chatting with non-verbal communication. In: EG VE 2011: Proceedings of the Joint Virtual Reality Conference of EuroVR. ACM (September 2011)Google Scholar
  10. 10.
    Hatfield, E., Cacioppo, J.T., Rapson, R.L.: Emotional Contagion. Current Directions in Psychological Science 2(3), 96–99 (1993)CrossRefGoogle Scholar
  11. 11.
    Kappas, A., Krämer, N.: Face-to-face communication over the Internet. Cambridge Univ. Press (2011)Google Scholar
  12. 12.
    Kappas, A.: Smile when you read this, whether you like it or not: Conceptual challenges to affect detection. IEEE Transactions on Affective Computing 1(1), 38–41 (2010)CrossRefGoogle Scholar
  13. 13.
    Kopp, S., Gesellensetter, L., Krämer, N.C., Wachsmuth, I.: A Conversational Agent as Museum Guide – Design and Evaluation of a Real-World Application. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 329–343. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  14. 14.
    Krämer, N.C., Simons, N., Kopp, S.: The Effects of an Embodied Conversational Agent’s Nonverbal Behavior on User’s Evaluation and Behavioral Mimicry. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 238–251. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  15. 15.
    Michael, R., Wagner, J.A.: Second Life: The Official Guide, 2nd edn. Wiley Publishing (2008)Google Scholar
  16. 16.
    Niewiadomski, R., Hyniewska, S.J., Pelachaud, C.: Constraint-based model for synthesis of multimodal sequential expressions of emotions. IEEE Transactions on Affective Computing 2(3), 134–146 (2011)CrossRefGoogle Scholar
  17. 17.
    Pelachaud, C.: Studies on gesture expressivity for a virtual agent. Speech Commun. 51(7), 630–639 (2009)CrossRefGoogle Scholar
  18. 18.
    Prestonand Frans, S.D., de Waal, B.M.: Empathy: Its ultimate and proximate bases. The Behavioral and Brain Sciences 25(1), 1–20 (2002)Google Scholar
  19. 19.
    Schweitzer, F., Garcia, D.: Frank Schweitzer and David Garcia. An agent-based model of collective emotions in online communities. The European Physical Journal B - Condensed Matter and Complex Systems 77, 533–545 (2010)CrossRefGoogle Scholar
  20. 20.
    Weiner, M., Devoe, S., Rubinow, S., Geller, J.: Nonverbal behavior and nonverbal communication. Psychological Review 79, 185–214 (1972)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Information and Communication Systems Institute (ISIC)HE-ArcSt-ImierSwitzerland
  2. 2.Immersive Interaction Group (IIG)EPFLLausanneSwitzerland
  3. 3.Chair of Systems Design (CSD)ETHZZurichSwitzerland
  4. 4.Institute for Media Innovation (IMI)NTUSingapore

Personalised recommendations