Fully Embodied Conversational Avatars: Making Communicative Behaviors Autonomous

  • J. Cassell
  • H. Vilhjálmsson


Although avatars may resemble communicative interface agents, they have for the most part not profited from recent research into autonomous embodied conversational systems. In particular, even though avatars function within conversational environments (for example, chat or games), and even though they often resemble humans (with a head, hands, and a body) they are incapable of representing the kinds of knowledge that humans have about how to use the body during communication. Humans, however, do make extensive use of the visual channel for interaction management where many subtle and even involuntary cues are read from stance, gaze, and gesture. We argue that the modeling and animation of such fundamental behavior is crucial for the credibility and effectiveness of the virtual interaction in chat. By treating the avatar as a communicative agent, we propose a method to automate the animation of important communicative behavior, deriving from work in conversation and discourse theory. BodyChat is a system that allows users to communicate via text while their avatars automatically animate attention, salutations, turn taking, back-channel feedback, and facial expression. An evaluation shows that users found an avatar with autonomous conversational behaviors to be more natural than avatars whose behaviors they controlled, and to increase the perceived expressiveness of the conversation. Interestingly, users also felt that avatars with autonomous communicative behaviors provided a greater sense of user control.

Avatars embodied conversational agents lifelike communicative behaviors 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    D.B. Anderson, J.W. Barrus, D. Brogan, M. Casey, S. McKeown, I. Sterns, R. Waters, and W. Yerazunis, “Diamond Park and Spline: A Social Virtual Reality System with 3D Animation, Spoken Interaction, and Runtime Modifiability.” Technical Report at MERL, Cambridge, 1996.Google Scholar
  2. 2.
    M. Argyle, and M. Cook, Gaze and Mutual Gaze. Cambridge University Press, 1976.Google Scholar
  3. 3.
    M. Argyle, R. Ingham, F. Alkema, and M. McCallin, “The Different Functions of Gaze.” Semiotica, 1973.Google Scholar
  4. 4.
    J. Bates, A.B. Loyall, and W.S. Reilley, “Broad Agents.” SIGART Bulletin, vol. 4(2), 1991.Google Scholar
  5. 5.
    S. Benford, J. Bowers, L.E. Fahlen, C. Greenhalgh, and D. Snowdon, “User Embodiment in Collaborative Virtual Environments.” Proceedings of CHI'95, 1995, pp. 242-249.Google Scholar
  6. 6.
    B. M. Blumberg, T. A. Galyean, “Multi-Level Direction of Autonomous Creatures for Real-Time Virtual Environments.” Proceedings of SIGGRAPH '95.Google Scholar
  7. 7.
    M. S. Cary, “The Role of Gaze in the Initiation of Conversation.” Social Psychology, vol. 41(3), 1978.Google Scholar
  8. 8.
    J. Cassell, “Embodied Conversation: Integrating Face and Gesture into Automatic Spoken Dialogue Systems.” In Luperfoy ed. Spoken Dialogue Systems. Cambridge, MA: MIT Press, 1999.Google Scholar
  9. 9.
    J. Cassell, C. Pelachaud, N. Badler, M. Steedman, B. Achorn, T. Becket, B. Douville, S. Prevost, and M. Stone, “Animated Conversation: Rule-based Generation of Facial Expression, Gesture & Spoken Intonation for Multiple Conversational Agents.” Proceedings of SIGGRAPH '94, 1994.Google Scholar
  10. 10.
    J. Cassell, and K. Thórisson, “The Power of a Nod and a Glance: Envelope vs. Emotional Feedback in Animated Conversational Agents.” Journal of Applied Artificial Intelligence, in press.Google Scholar
  11. 11.
    J. Cassell, O. Torres, and S. Prevost. “Turn Taking vs. Discourse Structure: How Best to Model Multimodal Conversation.” In Wilks ed. Machine Conversations. The Hague: Kluwer, 1998.Google Scholar
  12. 12.
    N. Chovil, “Discourse-Oriented Facial Displays in Conversation.” Research on Language and Social Interaction, vol. 25, 163-194, 1992.Google Scholar
  13. 13.
    J. Donath, “The Illustrated Conversation.” Multimedia Tools and Applications, vol. 1, 79-88, 1995.Google Scholar
  14. 14.
    C. Goodwin, “Gestures as a Resource for the Organization of Mutual Orientation.” Semiotica, 62(1/2), 1986.Google Scholar
  15. 15.
    A. Kendon, Conducting Interaction: Patterns of Behavior in Focused Encounters. Cambridge University Press. NY, 1990.Google Scholar
  16. 16.
    A. Kendon, “The Negotiation of Context in Face-to-Face Interaction.” In A. Duranti and C. Goodwin (eds.), Rethinking Context: Language as Interactive Phenomenon. Cambridge University Press. NY, 1990.Google Scholar
  17. 17.
    D. Kurlander, T. Skelly, and D. Salesin, “Comic Chat.” Proceedings of SIGGRAPH '96, 1996.Google Scholar
  18. 18.
    H. Lieberman, and D. Maulsby, “Instructible Agents: Software That Just Keeps Getting Better.” IBM Systems Journal, vol. 35, Nos. 3 & 4, 1996.Google Scholar
  19. 19.
    P. Maes, and B. Schneiderman, “Direct Manipulation vs. Interface Agents: a Debate.” Interactions, vol. 4Number 6, ACM Press, 1997.Google Scholar
  20. 20.
    D. McNeill, Hand and Mind: What Gestures Reveal about Thought. University of Chicago, 1992.Google Scholar
  21. 21.
    Y. Moon, and C. I. Nass, “How ‘Real’ are Computer Personalities? Psychological Responses to Personality Types in Human-Computer Interaction.” Communication Research, vol. 23(6), 651-674, 1996.Google Scholar
  22. 22.
    K. Perlin, and A. Goldberg, “Improv: A System for Scripting Interactive Actors in Virtual Worlds.” SIGGRAPH 1996 Course Notes #25.Google Scholar
  23. 23.
    S. Prevost, “Modeling Contrast in the Generation and Synthesis of Spoken Language.” Proceedings of ICSLP '96.Google Scholar
  24. 24.
    E. Schegloff, “Sequencing in Conversational Openings.” American Anthropologist, vol. 70, 1075-1095, 1968.Google Scholar
  25. 25.
    E. Schegloff, and H. Sacks, “Opening Up Closings.” Semiotica, vol. 8, 289-327, 1973.Google Scholar
  26. 26.
    K. R. Thórisson, “Gandalf: An Embodied Humanoid Capable of Real-Time Multimodal Dialogue with People.” Proceedings of Agents'97, 536-537.Google Scholar
  27. 27.
    P. Wavish, and D. Connah, “Virtual Actors that Can Perform Scripts and Improvise Roles.” Proceedings of Agents'97, 317-322.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • J. Cassell
    • 1
  • H. Vilhjálmsson
    • 1
  1. 1.MIT Media LaboratoryCambridgeUSA

Personalised recommendations