Embodied Conversational Agent Avatars in Virtual Worlds: Making Today’s Immersive Environments More Responsive to Participants

  • Jacquelyn Ford Morie
  • Eric Chance
  • Kip Haynes
  • Dinesh Rajpurohit


Intelligent agents in the form of avatars in networked virtual worlds (VWs) are a new form of embodied conversational agent (ECA). They are still a topic of active research, but promise soon to rival the sophistication of virtual human agents developed on stand-alone platforms over the last decade. Such agents in today’s VWs grew out of two lines of historical research: Virtual Reality and Artificial Intelligence. Their merger forms the basis for today’s persistent 3D worlds occupied by intelligent characters serving a wide range of purposes. We believe ECA avatars will help to enable VWs to achieve a higher level of meaningful interaction by providing increased engagement and responsiveness within environments where people will interact with and even develop relationships with them.


Virtual Reality Virtual World Intelligent Agent Virtual Teammate Virtual Character 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Some of the projects described herein have been sponsored by the U.S. Army Research, Development, and Engineering Command (RDECOM). Statements and opinions expressed do not necessarily reflect the position or the policy of the United States Government, and no official endorsement should be inferred. We would like to thank the many talented members of all the interdisciplinary teams that comprise the Virtual Human research group at the ICT for their dedicated work and contributions that enabled the adaptation of virtual human technology to virtual worlds. More about their work can be found at


  1. 1.
    Becker-Asano, C., Wachsmuth, I.: Affective computing with primary and secondary emotions in a virtual human. Auton. Agents Multi-Agent Syst. 20, 32–49 (2010)Google Scholar
  2. 2.
    Bickmore, T., Cassell, J.: Social dialogue with embodied conversational agents. In: van Kuppevelt, J., Dybkjaer, L., Bernsen, N.O. (eds.) Advances in Natural Multimodal Dialogue Systems, Speech and Language Technology, Chap. 2, vol. 30, pp. 23–54. Springer, Berlin (2005)Google Scholar
  3. 3.
    Bos, J., Oka, T.: Building spoken dialogue systems for believable characters. In: Proceedings of the Seventh Workshop on the Semantics and Pragmatics of Dialogue (DIABRUCK), (2003)Google Scholar
  4. 4.
    Brooks, D.: Oversimulated suburbia. The New York Times Magazine, Nov 2002Google Scholar
  5. 5.
    Cassell, J.: Embodied conversational agents: representation and intelligence in user interfaces. AI Mag. 22(4), 67–84 (2001)Google Scholar
  6. 6.
    Cassell, J., Bickmore, T.W., Campbell, L., Vilhjálmsson, H.H., Yan, H.: More than just a pretty face: conversational protocols and the affordances of embodiment. Knowl.-Based Syst. 14(1–2), 55–64 (2001)Google Scholar
  7. 7.
    Cassell, J., Vilhjálmsson, H.H.: Fully embodied conversational avatars: making communicative behaviors autonomous. Auton. Agents Multi-Agent Syst. 2(1), 45–64 (1999)Google Scholar
  8. 8.
    Catrambone, R., Stasko, J., Xiao, J.: ECA as user interface paradigm. pp. 239–267. Kluwer Academic Publishers, Norwell, USA (2004)Google Scholar
  9. 9.
    AJ Champandard: Top 10 most influential AI games, 2007.
  10. 10.
    Chance, E., Morie, J.F.: Method for custom facial animation and lip-sync in an unsupported environment, Second Life\(^{\rm TM}\). In: Proceedings of the 9th International Conference on Intelligent Virtual Agents, IVA, pp. 556–557. (2009)Google Scholar
  11. 11.
    Churcher, G.E., Atwell, E.S., Souter, C.: Dialogue management systems: a survey and overview. Technical Report, University of Leeds 1997Google Scholar
  12. 12.
    Egges, A., Kshirsagar, S., Magnenat-Thalmann, N.: Generic personality and emotion simulation for conversational agents. J. Vis. Comput. Animation 15(1), 1–13 (2004)Google Scholar
  13. 13.
    Ellis, S.R.: What are virtual environments? IEEE Comput. Graph. Appl. 14, 17–22 (Jan 1994)Google Scholar
  14. 14.
    Gratch, J., Rickel, J., Andre, E., Cassell, J., Petajan, E., Badler, N.I.: Creating interactive virtual humans: some assembly required. IEEE Intell. Syst. 17(4), 54–63 (2002)Google Scholar
  15. 15.
    Hill, M.: Second Life is frontier for AI research: intelligence tests use virtual world controllable environment, 2008.
  16. 16.
    ICT: Coming Home project description, 2010.
  17. 17.
    ICT: Simcoach project description, 2010.
  18. 18.
    Jan, D., Chance, E., Rajpurohit, D., DeVault, D., Leuski, A., Morie, J., Traum, D.: Checkpoint exercise: training with virtual actors in virtual worlds. In: Kopp, S., Marsella, S., Thòrisson, K. (eds.) Hannes Vilhjálmsson, Intelligent Virtual Agents, Lecture Notes in Computer Science, vol. 6895, pp. 453–454. Springer, Berlin (2011)Google Scholar
  19. 19.
    Jan, D., Roque, A., Leuski, A., Morie, J., Traum, D.: A virtual tour guide for virtual worlds. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H. (eds) Intelligent Virtual Agents, Lecture Notes in Computer Science, vol. 5773, pp. 372–378. Springer, Berlin (2009)Google Scholar
  20. 20.
    Hill, R.W. Jr., Gratch, J., Marsella, S., Rickel, J., Swartout, W.R., Traum, D.R.: Virtual humans in the mission rehearsal exercise system. KI 17(4), 5 (2003)Google Scholar
  21. 21.
    Kenny, P.G., Hartholt, A., Gratch, J., Traum, D.R., Marsella, S., Swartout, W.R.: The more the merrier: multi-party negotiation with virtual humans. In: AAAI, pp. 1970–1971 (2007)Google Scholar
  22. 22.
    Laird, J.E.: Using a computer game to develop advanced AI. IEEE Comput. 34(7), 70–75 (2001)CrossRefGoogle Scholar
  23. 23.
    Martin, J.-C., Abrilian, S., Devillers, L., Lamolle, M., Mancini, M., Pelachaud, C.: Levels of representation in the annotation of emotion for the specification of expressivity in ECAs. In: IVA, pp. 405–417 (2005)Google Scholar
  24. 24.
    Mauldin, M.L.: ChatterBots, TinyMUDs, and the Turing Test: entering the Loebner prize competition. In: AAAI, pp. 16–21 (1994)Google Scholar
  25. 25.
    McLellan, H.: Virtual realities. In: Handbook of Research for Educational Communications and Technology. Macmillan, New York (1996)Google Scholar
  26. 26.
    Mims, C.: Whatever happened to virtual reality? MIT Technology Review, 2010.
  27. 27.
    Morie, J.F., Haynes, E., Chance, E.: Warrior's Journey: a path to healing through narrative exploration. In: Proceedings of the International Journal on Disability and Human Development (2011)Google Scholar
  28. 28.
    Nilsson, N.J.: Eye on the prize. AI Mag. 16(2), 9–17 (1995)Google Scholar
  29. 29.
    Papp, R.: Virtual worlds and social networking: reaching the millennials. J. Technol. Res. 1–15 (2010)Google Scholar
  30. 30.
    Rickel, J., Marsella, S., Gratch, J., Hill, R., Traum, D.R., Swartout, W.R.: Toward a new generation of virtual humans for interactive experiences. IEEE Intell. Syst. 17(4), 32–38 (2002)Google Scholar
  31. 31.
    Robertson, W.G.: The Staff Ride. United States Army Center of Military History, 1987.
  32. 32.
    Swartout, W.R.: Virtual humans. In: Twenty-First National Conference on Artificial Intelligence (AAAI-06), (2006)Google Scholar
  33. 33.
    Swartout, W.R.: Lessons learned from virtual humans. AI Mag. 31(1), 9–20 (2010)Google Scholar
  34. 34.
    Swartout, W.R., Gratch, Jonathan., Hill, R.W. Jr., Hovy, E.H., Marsella, S., Rickel, J., Traum, D.R.: Toward virtual humans. AI Mag. 27(2), 96–108 (2006)Google Scholar
  35. 35.
    Nino, T.: Peering inside Second Life user retention, Dec. 2007.
  36. 36.
    Nino, T.: Second Life traffic gaming: a chat with a bot-operator, and dire portents for lucky chairs, June 2009.
  37. 37.
    Thiebaux, M., Marsella, S., Marshall, A.N., Kallmann, M.: Smartbody: behavior realization for embodied conversational agents. In: AAMAS (1), pp. 151–158 (2008)Google Scholar
  38. 38.
    Verhulsdonck, G., Morie, J.: Virtual chironomia: developing standards for non-verbal communication in virtual worlds. J. Virtual Worlds Res. 2(3) (2009)Google Scholar
  39. 39.
    Weizenbaum, J.: Eliza—a computer program for the study of natural language communication between man and machine. Commun. ACM 9, 36–45 (1966)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Jacquelyn Ford Morie
    • 1
  • Eric Chance
    • 1
  • Kip Haynes
    • 1
  • Dinesh Rajpurohit
    • 1
  1. 1.Institute for Creative TechnologiesUniversity of Southern CaliforniaCAUSA

Personalised recommendations