Skip to main content
SpringerLink
Log in

All Together Now

Introducing the Virtual Human Toolkit

  • Conference paper
Intelligent Virtual Agents (IVA 2013)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 8108))

Included in the following conference series:

Abstract

While virtual humans are proven tools for training, education and research, they are far from realizing their full potential. Advances are needed in individual capabilities, such as character animation and speech synthesis, but perhaps more importantly, fundamental questions remain as to how best to integrate these capabilities into a single framework that allows us to efficiently create characters that can engage users in meaningful and realistic social interactions. This integration requires in-depth, inter-disciplinary understanding few individuals, or even teams of individuals, possess. We help address this challenge by introducing the ICT Virtual Human Toolkit, which offers a flexible framework for exploring a variety of different types of virtual human systems, from virtual listeners and question-answering characters to virtual role-players. We show that due to its modularity, the Toolkit allows researchers to mix and match provided capabilities with their own, lowering the barrier of entry to this multi-disciplinary research challenge.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Rossen, B., Lok, B.: A crowdsourcing method to develop virtual human conversational agents. International Journal of HCS, 301–319 (2012)

    Google Scholar 

  2. Bickmore, T., Bukhari, L., Vardoulakis, L.P., Paasche-Orlow, M., Shanahan, C.: Hospital buddy: A persistent emotional support companion agent for hospital patients. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 492–495. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  3. D’Mello, S.K., Graesser, A.C.: AutoTutor and affective AutoTutor: Learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Transactions on Interactive Intelligent Systems 2(4), Article 23 (2012)

    Google Scholar 

  4. Lane, H.C., Noren, D., Auerbach, D., Birch, M., Swartout, W.: Intelligent Tutoring Goes to the Museum in the Big City: A Pedagogical Agent for Informal Science Education. In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) AIED 2011. LNCS, vol. 6738, pp. 155–162. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  5. Johnson, W.L., Valente, A.: Tactical Language and Culture Training Systems: Using AI to Teach Foreign Languages and Cultures, pp. 72–83 (2009)

    Google Scholar 

  6. Campbell, J., Core, M., Artstein, R., Armstrong, L., Hartholt, A., Wilson, C., Georgila, K., Morbini, F., Haynes, E., Gomboc, D., Birch, M., Bobrow, J., Lane, H., Gerten, J., Leuski, A., Traum, D., Trimmer, M., DiNinni, R., Bosack, M., Jones, T., Clark, R., Yates, K.: Developing INOTS to support interpersonal skills practice. In: Proceedings of the Thirty-second Annual IEEE Aerospace Conference, pp. 1–14 (2011)

    Google Scholar 

  7. Pocketsphinx: A Free, Real-Time Continuous Speech Recognition System for hand-Held Devices, vol. 1, pp. 185–188 (2006)

    Google Scholar 

  8. Littlewort, G., Whitehill, J., Wu, T., Fasel, I., Frank, M., Movellan, J., Bartlett, M.: The Computer Expression Recognition Toolbox (CERT). In: Proc. IEEE International Conference on Automatic Face and Gesture Recognition (2011)

    Google Scholar 

  9. Bickmore, T., Schulman, D., Shaw, G.: DTask and LiteBody: Open Source, Standards-based Tools for Building Web-deployed Embodied Conversational Agents. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 425–431. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  10. Stone, M.: Specifying Generation of Referring Expressions by Example. In: AAAI Spring Symposium on NLG in Spoken and Written Dialogue, pp. 133–140 (2003)

    Google Scholar 

  11. Poggi, I., Pelachaud, C., de Rosis, F., Carofiglio, V., De Carolis, B.: GRETA. A Believable Embodied Conversational Agent, Multimodal Intelligent Information Presentation (2005)

    Google Scholar 

  12. van Welbergen, H., Reidsma, D., Ruttkay, Z.M., Zwiers, J.: Elckerlyc - A BML Realizer for continuous, multimodal interaction with a Virtual Human. Journal on Multimodal User Interfaces 3(4), 271–284 (2010)

    Article  Google Scholar 

  13. Shapiro, A.: Building a Character Animation System. In: Allbeck, J.M., Faloutsos, P. (eds.) MIG 2011. LNCS, vol. 7060, pp. 98–109. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  14. Heloir, A., Kipp, M.: EMBR – A Realtime Animation Engine for Interactive Embodied Agents. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 393–404. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  15. Taylor, P., Black, A., Caley, R.: The architecture of the Festival speech synthesis system. In: Third ESCA Workshop in Speech Synthesis, pp. 147–151 (1998)

    Google Scholar 

  16. http://www.mindmakers.org/projects/saiba/wiki

  17. Heylen, D., Kopp, S., Marsella, S.C., Pelachaud, C., Vilhjálmsson, H.: The Next Step towards a Function Markup Language. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 270–280. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  18. Kopp, S., Krenn, B., Marsella, S., Marshall, A.N., Pelachaud, C., Pirker, H., Thórisson, K.R., Vilhjálmsson, H.: Towards a Common Framework for Multimodal Generation: The Behavior Markup Language. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 205–217. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  19. van Welbergen, H., Xu, Y., Thiebaux, M., Feng, W.-W., Fu, J., Reidsma, D., Shapiro, A.: Demonstrating and Testing the BML Compliance of BML Realizers. In: Vilhjálmsson, H.H., Kopp, S., Marsella, S., Thórisson, K.R. (eds.) IVA 2011. LNCS, vol. 6895, pp. 269–281. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  20. Schröder, M.: The SEMAINE API: Towards a standards-based framework for building emotion-oriented systems. In: Advances in Human-Machine Interaction (2010)

    Google Scholar 

  21. Quigley, M., Conley, K., Gerkey, B.P., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: ROS: an open-source Robot Operating System, ICRA Open Source Software (2009)

    Google Scholar 

  22. Leuski, A., Traum, D.: NPCEditor: Creating virtual human dialogue using information retrieval techniques. AI Magazine 32(2), 42–56 (2011)

    Google Scholar 

  23. Leuski, A., Traum, D.: A statistical approach for text processing in virtual humans. In: Proceedings of the 26th Army Science Conference, Orlando, Florida, USA (December 2008)

    Google Scholar 

  24. Lee, J., Marsella, S.: Nonverbal Behavior Generator for Embodied Conversational Agents. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 243–255. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  25. Feng, A.W., Xu, Y., Shapiro, A.: An Example-Based Motion Synthesis Technique for Locomotion and Object Manipulation. In: I3D 2012 Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, pp. 95–102 (2012)

    Google Scholar 

  26. Thiebaux, M., Lance, B., Marsella, S.: Real-time Expressive Gaze for Virtual Humans. In: AAMAS, vol. 1, pp. 321–328 (2008)

    Google Scholar 

  27. Kallmann, M., Marsella, S.: Hierarchical Motion Controllers for Real-time Autonomous Virtual Humans. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 253–265. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  28. Feng, A., Huang, Y., Kallmann, M., Shapiro, A.: An Analysis of Motion Blending Techniques. In: Kallmann, M., Bekris, K. (eds.) MIG 2012. LNCS, vol. 7660, pp. 232–243. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  29. Feng, A., Huang, Y., Xu, Y., Shapiro, A.: Automating the Transfer of a Generic Set of Behaviors onto a Virtual Character. In: Kallmann, M., Bekris, K. (eds.) MIG 2012. LNCS, vol. 7660, pp. 134–145. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  30. Wagner, J., Lingenfelser, F., Bee, N., Andre, E.: Social signal interpretation (ssi). KI - Kuenstliche Intelligenz 25, 251–256 (2011)

    Article  Google Scholar 

  31. Baltrusaitis, T., Robinson, P., Morency, L.-P.: 3D constrained local model for rigid and non-rigid facial tracking. In: IEEE Computer Vision and Pattern Recognition, pp. 2610–2617 (2012)

    Google Scholar 

  32. Morency, L.-P., Whitehill, J., Movellan, J.: Generalized adaptive view-based appearance model: Integrated framework for monocular head pose estimation. In: Automatic Face and Gesture Recognition, pp. 1–8 (2008)

    Google Scholar 

  33. Suma, E., Lange, B., Rizzo, A., Krum, D., Bolas, M.: FAAST: The Flexible Action and Articulated Skeleton Toolkit. In: Proceedings of IEEE Virtual Reality, pp. 247–248 (2011)

    Google Scholar 

  34. Scherer, S., Marsella, S., Stratou, G., Xu, Y., Morbini, F., Egan, A., Rizzo, A(S.), Morency, L.-P.: Perception Markup Language: Towards a Standardized Representation of Perceived Nonverbal Behaviors. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 455–463. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  35. http://unity3d.com/

  36. http://www.ogre3d.org/

  37. http://www.cereproc.com/products/sdk

  38. http://msdn.microsoft.com/en-us/library/ee125663v=vs.85.aspx

  39. http://activemq.apache.org/

  40. Leuski, A., Kennedy, B., Patel, R., Traum, D.R.: Asking questions to limited domain virtual characters: how good does speech recognition have to be? In: 25th Army Science Conference, Orlando, Fl (2006)

    Google Scholar 

  41. Artstein, R., Gandhe, S., Leuski, A., Traum, D.R.: Field Testing of an interactive question-answering character. In: ELRA, LREC (2008)

    Google Scholar 

  42. Swartout, W., et al.: Ada and Grace: Toward Realistic and Engaging Virtual Museum Guides. In: Allbeck, J., Badler, N., Bickmore, T., Pelachaud, C., Safonova, A. (eds.) IVA 2010. LNCS, vol. 6356, pp. 286–300. Springer, Heidelberg (2010)

    Google Scholar 

  43. Gratch, J., Okhmatovskaia, A., Lamothe, F., Marsella, S., Morales, M., van der Werf, R.J., Morency, L.-P.: Virtual Rapport. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 14–27. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  44. Rizzo, A., Forbell, E., Lange, B., Buckwalter, J.G., Williams, J., Sagae, K., Traum, D.: SimCoach: An Online Intelligent Virtual Agent System for Breaking Down Barriers to Care for Service Members and Veterans. In: Scurfield, R.M., Platoni, K.T. (eds.) Healing War Trauma: A Handbook of Creative Approaches, pp. 238–250. Routledge (2012)

    Google Scholar 

  45. Morbini, F., DeVault, D., Sagae, K., Gerten, J., Nazarian, A., Traum, D.: FLoReS: A Forward Looking, Reward Seeking, Dialogue Manager, Spoken Dialog Systems (2012)

    Google Scholar 

  46. Hartholt, A., Gratch, J., Weiss, L., The Gunslinger Team: At the Virtual Frontier: Introducing Gunslinger, a Multi-Character, Mixed-Reality, Story-Driven Experience. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 500–501. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  47. Kenny, P., Parsons, T.D., Gratch, J., Rizzo, A.A.: Evaluation of Justina: A Virtual Patient with PTSD. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 394–408. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  48. Traum, D.R., Marsella, S.C., Gratch, J., Lee, J., Hartholt, A.: Multi-party, Multi-issue, Multi-strategy Negotiation for Multi-modal Virtual Agents. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 117–130. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  49. Gratch, J., Hartholt, A., Dehghani, M., Marsella, S.: Virtual Humans: A New Toolkit for Cognitive Science Research. In: CogSci (2013)

    Google Scholar 

  50. Khooshabeh, P., McCall, C., Gandhe, S., Gratch, J., Blascovich, J.: Does it matter if a computer jokes. In: Extended Abstracts on Human Factors in Computer Systems, pp. 77–86 (2011)

    Google Scholar 

  51. Batrinca, L., Stratou, G., Shapiro, A., Morency, L.-P., Scherer, S.: Cicero - Towards a Multimodal Virtual Audience Platform for Public Speaking Training. In: Aylett, R., Krenn, B., Pelachaud, C., Shimodaira, H. (eds.) IVA 2013. LNCS (LNAI), vol. 8108, pp. 116–128. Springer, Heidelberg (2013)

    Google Scholar 

  52. http://www.microsoft.com/en-us/kinectforwindows/

Download references

Author information

Authors and Affiliations

  1. Institute for Creative Technologies, University of Southern California, 12015 Waterfront Drive, Playa Vista, CA, 90094, USA

    Arno Hartholt, David Traum, Stacy C. Marsella, Ari Shapiro, Giota Stratou, Anton Leuski, Louis-Philippe Morency & Jonathan Gratch

Authors
  1. Arno Hartholt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Traum
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stacy C. Marsella
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ari Shapiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Giota Stratou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Anton Leuski
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Louis-Philippe Morency
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jonathan Gratch
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. MACS, Heriot-Watt University, Riccarton, EH14 4AS, Edinburgh, UK

    Ruth Aylett

  2. Austrian Research Institute for Artificial Intelligence (OFAI), 1010, Vienna, Austria

    Brigitte Krenn

  3. CNRS-LTCI, Telecom-ParisTech, 75014, Paris, France

    Catherine Pelachaud

  4. School of Informatics, The University of Edinburgh, EH8 9LW, Edinburgh, UK

    Hiroshi Shimodaira

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Hartholt, A. et al. (2013). All Together Now. In: Aylett, R., Krenn, B., Pelachaud, C., Shimodaira, H. (eds) Intelligent Virtual Agents. IVA 2013. Lecture Notes in Computer Science(), vol 8108. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40415-3_33

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-40415-3_33

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-40414-6

  • Online ISBN: 978-3-642-40415-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation