Virtual Reflexes

  • Catholijn M. Jonker
  • Joost Broekens
  • Aske Plaat
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8637)


Virtual Reality is used successfully to treat people for regular phobias. A new challenge is to develop Virtual Reality Exposure Training for social skills. Virtual actors in such systems have to show appropriate social behavior including emotions, gaze, and keeping distance. The behavior must be realistic and real-time. Current approaches are usually based on heavy information processing in terms of behavior planning, scripting of behavior and use of predefined animations. We believe this limits the directness of human bodily reflexes and causes unrealistic responses and delay. We propose to investigate virtual reflexes as concurrent sensory-motor processes to control individual parts of the virtual actor’s skeleton with a body integrity model that keeps the effects coherent. We explain how emotion and cognitive modulation could be embedded, and give an example description of the interplay between emotion and a reflex.


Virtual Reality Social Distance Decision Node Virtual Character Virtual Agent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Argyle, M.: Social Interaction. Transaction Publishers Rutgers, New Jersey (2009)Google Scholar
  2. 2.
    Berthoz, A.: The brains sense of movement. Harvard Univ. Press (2002)Google Scholar
  3. 3.
    Broekens, J., Harbers, M., Brinkman, W.-P., Jonker, C.M., Van den Bosch, K., Meyer, J.-J.: Virtual Reality Negotiation Training Increases Negotiation Knowledge and Skill. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 218–230. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  4. 4.
    Brooks, R.A.: Cambrian intelligence: the early history of the new AI. MIT Press (1999)Google Scholar
  5. 5.
    Cafaro, A., Vilhjálmsson, H.H., Bickmore, T., Heylen, D., Jóhannsdóttir, K.R., Valgarðsson, G.S.: First Impressions: Users’ Judgments of Virtual Agents’ Personality and Interpersonal Attitude in First Encounters. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 67–80. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  6. 6.
    Cañamero, L.: Designing Emotional Artifacts for Social Interaction: Challenges and Perspectives. In: Cañamero, L., Aylett, R. (eds.) Animating Expressive Characters for Social Interaction. Adv. in Consciousness Research (2005)Google Scholar
  7. 7.
    Core, M., Traum, D., Lane, H.C., Swartout, W., Gratch, J., van Lent, M.: Teaching Negotiation Skills through Practice and Reflection with Virtual Humans. Simulation 82(11), 685–701 (2006)CrossRefGoogle Scholar
  8. 8.
    Damasio, A.: The Feeling of What Happens: Body and Emotion in the Making of Consciousness, Harcourt (1999)Google Scholar
  9. 9.
    Damasio, A., Carvalho, G.B.: The nature of feelings: evolutionary and neurobiological origins. Nature Reviews Neuroscience 14, 143–152 (2013); PubMedGoogle Scholar
  10. 10.
    D’Mello, S., Picard, R.W., Graesser, A.: Toward an Affect-Sensitive AutoTutor.  22, 53–61 (2007)Google Scholar
  11. 11.
    Emmelkamp, P.M.G., Bruynzeel, M., Drost, L., van der Mast, C.A.P.G.: Virtual reality treatment in acrophobia: a comparison with exposure in vivo. Cyber Psychology & Behavior 4(3), 335–339 (2001)CrossRefGoogle Scholar
  12. 12.
    Faloutsos, P., van de Panne, M., Terzopoulos, D.: Composable controllers for physics-based character animation. Paper Presented at the Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (2001)Google Scholar
  13. 13.
    Ogan, A., Finkelstein, S., Walker, E., Carlson, R., Cassell, J.: Rudeness and rapport: Insults and learning gains in peer tutoring. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) ITS 2012. LNCS, vol. 7315, pp. 11–21. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  14. 14.
    Folkman, S., Lazarus, R.S.: Coping and emotion. In: Stein, N.L., Leventhal, B., Trabasso, T. (eds.) Psych. and Bio Appr. Emo., pp. 313–332. Erlbaum, Hillsdale (1990)Google Scholar
  15. 15.
    Gratch, J., Wang, N., Gerten, J., Fast, E., Duffy, R.: Creating Rapport with Virtual Agents. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 125–138. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  16. 16.
    Hays, M.J., Ogan, A., Lane, H.C.: The Evolution of Assessment: Learning about Culture from a Serious Game. In: Lynch, C., Ashley, K.D., Mitrovic, T., Dimitrova, V., Pinkwart, N., Aleven, V. (eds.) IllDef 2010, pp. 37–44 (2010)Google Scholar
  17. 17.
    Heylen, D., Nijholt, A., op den Akker, R.: Affect in tutoring dialogues. Applied Artificial Intelligence: An International Journal 19(3), 287–311 (2005)CrossRefGoogle Scholar
  18. 18.
    Huang, L., Morency, L.-P., Gratch, J.: Virtual Rapport 2.0. In: Vilhjálmsson, H.H., Kopp, S., Marsella, S., Thórisson, K.R. (eds.) IVA 2011. LNCS, vol. 6895, pp. 68–79. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  19. 19.
    Kahnemann, D.: Thinking, fast and slow. Penguin Books (2011)Google Scholar
  20. 20.
    Kavanagh, L., Bakhtiari, G., Suhler, C., Churchland, P.S., Holland, R.W., Winkielman, P.: Nuanced Social Inferences about Trustworthiness from Observation of Mimicry. In: Knauff, M., Pauen, M., Sebanz, N., Wachsmuth, I. (eds.) Proceedings of the 35th Annual Conference of the Cognitive Science Society, pp. 734–739. Cognitive Science Society, Berlin (2013)Google Scholar
  21. 21.
    Kendon, A.: Conducting interaction. Studies in Interactional Sociolinguistics, vol. 7. Cambridge University Press (1990)Google Scholar
  22. 22.
    Kim, J.M., Hill, J.R.W., Durlach, P.J., Lane, H.C., Forbell, E., Core, M., et al.: BiLAT: A Game-Based Environment for Practicing Negotiation in a Cultural Context. International Journal of Artificial Intelligence in Education 19(3), 289–308 (2009)Google Scholar
  23. 23.
    Krijn, M., Emmelkamp, P.M.G., Olafsson, R.P., Biemond, R.: Virtual reality exposure therapy of anxiety disorders: A review. Clinical Psych Review 24(3), 259–281 (2004)CrossRefGoogle Scholar
  24. 24.
    LeDoux, J.: The Emotional Brain. Simon and Shuster, New York (1996)Google Scholar
  25. 25.
    LeDoux, J.E.: Emotion: Clues from the brain. Ann. Rev. Psy. 46(1), 209–235 (1995)CrossRefGoogle Scholar
  26. 26.
    Magnenat-Thalmann, N., Thalmann, D.: Virtual humans: thirty years of research, what next? The Visual Computer 21, 997–1015 (2005)CrossRefGoogle Scholar
  27. 27.
    Marsella, S., Gratch, J., Ning, W., Stankovic, B.: Assessing the validity of a computational model of emotional coping. In: Affective Computing and Intelligent Interaction and Workshops, ACII 2009 (2009)Google Scholar
  28. 28.
    Mehrabian, A.: Basic Dimensions for a General Psychological Theory. OG&H (1980)Google Scholar
  29. 29.
    Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press (1988)Google Scholar
  30. 30.
    Parsons, S., Mitchell, P.: The potential of virtual reality in social skills training for people with autistic spectrum disorders. J. Intell. Disability Research 46(5), 430–443 (2002)CrossRefGoogle Scholar
  31. 31.
    Popescu, A., Broekens, J., van Someren, M.: GAMYGDALA: An Emotion Engine for Games. IEEE Transactions on Affective Computing 5(1), 32–44 (2014)CrossRefGoogle Scholar
  32. 32.
    Popović, S., Horvat, M., Kukolja, D., Dropuljić, B., Cosić, K.: Stress inoculation training supported by physiology-driven adaptive virtual reality stimulation. Stud. Health Technol. Inform. 144, 50–54 (2009)Google Scholar
  33. 33.
    Powers, M.B., Emmelkamp, P.M.G.: Virtual reality exposure therapy for anxiety disorders: A meta-analysis. Journal of Anxiety Disorders 22(3), 561–569 (2008)CrossRefGoogle Scholar
  34. 34.
    Scherer, K.R.: Appraisal considered as a process of multilevel sequential checking. In: Scherer, K.R., Schorr, A., Johnstone, T. (eds.) Appraisal Processes in Emotion: Theory, Methods, Research, pp. 92–120 (2001)Google Scholar
  35. 35.
    Schroder, M., Bevacqua, E., Cowie, R., Eyben, F., Gunes, H., Heylen, D., et al.: Building Autonomous Sensitive Artificial Listeners. IEEE Aff. Computing 3(2), 165–183 (2012)Google Scholar
  36. 36.
    Serino, S., Triberti, S., Villani, D., Cipresso, P., Gaggioli, A., Riva, G.: Toward a validation of cyber-interventions for stress disorders based on stress inoculation training: a systematic review. Virtual Reality, 1–15 (2013)Google Scholar
  37. 37.
    de Sevin, E., Hyniewska, S.J., Pelachaud, C.: Influence of Personality Traits on Backchannel Selection. In: Allbeck, J., Badler, N., Bickmore, T., Pelachaud, C., Safonova, A. (eds.) IVA 2010. LNCS, vol. 6356, pp. 187–193. Springer, Heidelberg (2010)Google Scholar
  38. 38.
    Van der Spek, E.: Experiments in Serious Game Design. University of Utrecht (2011)Google Scholar
  39. 39.
    Tekofsky, S., Spronck, P., Plaat, A., Van den Herik, J., Broersen, J.: Psyops: Personality assessment through gaming behavior. Paper Presented at the Proceedings of the International Conference on the Foundations of Digital Games (2013)Google Scholar
  40. 40.
    Thiebaux, M., Marsella, S., Marshall, A.N., Kallmann, M.: Smartbody: Behavior realization for embodied conversational agents. In: Proceedings of the 7th Conf. Autonomous Agents and Multiagent Systems, vol. 1, pp. 151–158. IFAAMS (2008)Google Scholar
  41. 41.
    Wakefield, J.C., Dreyfus, H.L.: Intentionality and the phenomenology of action. In: Lepore, Gulick (eds.) John Searle and His Critics. Blackwell, Cambridge (1991)Google Scholar
  42. 42.
    Wilson, M.: Six views of embodied cognition. Psych. Bull. & Rev. 9(4), 625–636 (2002)CrossRefGoogle Scholar
  43. 43.
    Ziemke, T.: What’s that thing called embodiment. In: Proceedings of the 25th Annual Conference of the Cognitive Science Society. Erlbaum, Mahwah (2003)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Catholijn M. Jonker
    • 1
  • Joost Broekens
    • 1
  • Aske Plaat
    • 2
  1. 1.Interactive IntelligenceDelft University of TechnologyMekelweg 4The Netherlands
  2. 2.Tilburg School of HumanitiesTilburg UniversityWarandelaan 2The Netherlands

Personalised recommendations