Journal on Multimodal User Interfaces

, Volume 3, Issue 4, pp 271–284 | Cite as


A BML Realizer for continuous, multimodal interaction with a Virtual Human
  • Herwin van Welbergen
  • Dennis Reidsma
  • Zsófia M. Ruttkay
  • Job Zwiers
Open Access
Original Paper


“Elckerlyc” is a BML Realizer for generating multimodal verbal and nonverbal behavior for Virtual Humans (VHs). The main characteristics of Elckerlyc are that (1) it is designed specifically for continuous interaction with tight temporal coordination between the behavior of a VH and its interaction partner; (2) it provides a mix between the precise temporal and spatial control offered by procedural animation and the physical realism of physical simulation; and (3) it is designed to be highly modular and extensible, implementing the architecture proposed in SAIBA.


BML Realizer Virtual Humans Embodied conversational agents Multimodal behavior generation Physical simulation Procedural animation Coordinated interaction Mixed dynamics 


  1. 1.
    Arnason BT, Thorsteinsson A (2008) The CADIA BML realizer.
  2. 2.
    Chi DM, Costa M, Zhao L, Badler NI (2000) The EMOTE model for effort and shape. In: SIGGRAPH. ACM Press/Addison-Wesley, New York/Reading, pp 173–182 CrossRefGoogle Scholar
  3. 3.
    Hartmann B, Mancini M, Pelachaud C (2006) Gesture in human-computer interaction and simulation. In: Implementing expressive gesture synthesis for embodied conversational agents. Lecture notes in computer science, vol 3881. Springer, Berlin, pp 188–199 Google Scholar
  4. 4.
    Heloir A, Kipp M (2009) EMBR—a realtime animation engine for interactive embodied agents. In: Intelligent virtual agents. Lecture notes in computer science, vol 5773. Springer, Berlin, pp 393–404 CrossRefGoogle Scholar
  5. 5.
    Isaacs PM, Cohen MF (1987) Controlling dynamic simulation with kinematic constraints. In: SIGGRAPH. ACM, New York, pp 215–224 Google Scholar
  6. 6.
    Kim M, Kim M, Shin SY (1995) A general construction scheme for unit quaternion curves with simple high order derivatives. In: SIGGRAPH. ACM, New York, pp 369–376 Google Scholar
  7. 7.
    Kochanek DHU, Bartels RH (1984) Interpolating splines with local tension, continuity, and bias control. In: SIGGRAPH. ACM, New York, pp 33–41 Google Scholar
  8. 8.
    Kopp S, Wachsmuth I (2004) Synthesizing multimodal utterances for conversational agents. Comput Animat Virtual Worlds 15(1):39–52 CrossRefGoogle Scholar
  9. 9.
    Kopp S, Krenn B, Marsella S, Marshall AN, Pelachaud C, Pirker H, Thórisson KR, Vilhjálmsson HH (2006) Towards a common framework for multimodal generation: the behavior markup language. In: Intelligent virtual agents. Lecture notes in computer science, vol 4133. Springer, Berlin, pp 205–217 CrossRefGoogle Scholar
  10. 10.
    Loyall AB, Reilly WSN, Bates J, Weyhrauch P (2004) System for authoring highly interactive, personality-rich interactive characters. In: Symposium on computer animation. Eurographics Association, Munich, pp 59–68. doi:  10.1145/1028523.1028532 Google Scholar
  11. 11.
    Neff M, Fiume E (2008) From performance theory to character animation tools. In: Human motion—understanding, modelling, capture, and animation. Springer, Dordrecht, pp 597–629 Google Scholar
  12. 12.
    Nijholt A, Reidsma D, van Welbergen H, op den Akker HJA, Ruttkay ZsM (2008) Mutually coordinated anticipatory multimodal interaction. In: Nonverbal features of human-human and human-machine interaction. Lecture notes in computer science, vol 5042. Springer, Berlin, pp 70–89 CrossRefGoogle Scholar
  13. 13.
    Noot H, Ruttkay Zs (2005) Variations in gesturing and speech by gestyle. Int J Hum Comput Stud 62(2):211–229 CrossRefGoogle Scholar
  14. 14.
    Reidsma D, Nijholt A, Bos P (2008) Temporal interaction between an artificial orchestra conductor and human musicians. Comput Entertain 6(4):1–22 CrossRefGoogle Scholar
  15. 15.
    Reilink R, Visser LC, Bennik J, Carloni R, Brouwer DM, Stramigioli S (2009) The Twente humanoid head. In: IEEE international conference on robotics and automation, pp 1593–1594 Google Scholar
  16. 16.
    Thiebaux M, Marshall AN, Marsella S, Kallmann M (2008) Smartbody: behavior realization for embodied conversational agents. In: Autonomous agents and multiagent systems, pp 151–158 Google Scholar
  17. 17.
    Thórisson KR (2002) Natural turn-taking needs no manual: computational theory and model, from perception to action. In: Multimodality in language and speech systems. Kluwer Academic, Dordrecht, pp 173–207 Google Scholar
  18. 18.
    Thórisson KR, Benko H, Abramov D, Arnold A, Maskey S, Vaseekaran A (2004) Constructionist design methodology for interactive intelligences. AI Mag 25(4). doi:  10.1609/aimag.v25i4.1786
  19. 19.
    Tolani D, Goswami A, Badler NI (2000) Real-time inverse kinematics techniques for anthropomorphic limbs. Graph Models Image Process 62(5):353–388 zbMATHGoogle Scholar
  20. 20.
    van Welbergen H, Nijholt A, Reidsma D, Zwiers J (2006) Presenting in virtual worlds: towards an architecture for a 3D presenter explaining 2D-presented information. IEEE Intell Syst 21(5):47–53 CrossRefGoogle Scholar
  21. 21.
    van Welbergen H, van Basten BJH, Egges A, Ruttkay ZsM, Overmars MH (2009) Real time character animation: a trade-off between naturalness and control. In: Pauly M, Greiner G (eds) Eurographics—state of the art reports. Eurographics Association, Munich, pp 45–72. ISSN 1017-4656 Google Scholar
  22. 22.
    van Welbergen H, Zwiers J, Ruttkay ZsM (2009) Real-time animation using a mix of physical simulation and kinematics. J Graph GPU Game Tools 14(4):1–21 Google Scholar
  23. 23.
    Vilhjálmsson HH, Cantelmo N, Cassell J, Chafai NE, Kipp M, Kopp S, Mancini M, Marsella SC, Marshall AN, Pelachaud C, Ruttkay ZM, Thórisson KR, van Welbergen H, van der Werf RJ (2007) The behavior markup language: recent developments and challenges. In: Intelligent virtual agents. Lecture notes in computer science, vol 4722. Springer, Berlin, pp 99–120 CrossRefGoogle Scholar
  24. 24.
    Witkin A, Popovic Z (1995) Motion warping. In: SIGGRAPH. ACM, New York, pp 105–108 Google Scholar

Copyright information

© The Author(s) 2010

Authors and Affiliations

  • Herwin van Welbergen
    • 1
  • Dennis Reidsma
    • 1
  • Zsófia M. Ruttkay
    • 1
  • Job Zwiers
    • 1
  1. 1.Human Media InteractionUniversity of TwenteEnschedeThe Netherlands

Personalised recommendations