Embodied Interaction in Play: Body-Based and Natural Interaction in Games

  • Bernhard MaurerEmail author
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9970)


This chapter describes embodied interaction as a stance towards interaction design for games. It aims at informing game developers with bridging concepts between gameplay and different interaction paradigms that incorporate and focus on the human body (i.e., body-based, natural and tangible interaction). It highlights challenges, potentials and pitfalls of physical interactions and discusses the role of the human mind-body relation as a fundamental concept towards serious game interaction design. This chapter underlines embodied interaction in play as a promising perspective for game developers, that puts emphasize on the notion of humans as social and physical creatures with sentient bodies and highlights the relevance of this perspective for serious game interaction design as well as research in the field. One of the main challenges for researchers in this field will be to create meaningful interactions that internalize these concepts as integral parts of how serious games achieve their impact.


Body-based interaction Natural interaction Embodied interaction Interaction design Game design Tangible interaction Serious games 


  1. 1.
  2. 2.
    Arduino microcontroller.
  3. 3.
    Fitbit - activity tracker.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
    Microsoft research: stroke recovery with kinect.
  8. 8.
    Phidgets - products for usb sensing and control.
  9. 9.
    Tobii eye tracking enabled games.
  10. 10.
    Aymerich-Franch, L., Kizilcec, R.F., Bailenson, J.N.: The relationship between virtual self similarity and social anxiety. Front. Hum. Neurosci. 8, 944 (2014)CrossRefGoogle Scholar
  11. 11.
    Bakker, S., Antle, A.N., Van Den Hoven, E.: Embodied metaphors in tangible interaction design. Pers. Ubiquit. Comput. 16(4), 433–449 (2012)CrossRefGoogle Scholar
  12. 12.
    Bakker, S., Vorstenbosch, D., van den Hoven, E., Hollemans, G., Bergman, T.: Weathergods: tangible interaction in a digital tabletop game. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI 2007, New York, NY, USA, pp. 151–152. ACM (2007).
  13. 13.
    Bente, G., Eschenburg, F., Aelker, L.: Effects of simulated gaze on social presence, person perception and personality attribution in avatar-mediated communication. In: Presence 2007: Proceedings of the 10th Annual International Workshop on Presence, 25–27 October 2007, Barcelona, Spain, pp. 207–214 (2007)Google Scholar
  14. 14.
    Bergen, B.K.: Louder Than Words: The New Science of How the Mind Makes Meaning. Basic Books, New York (2012)Google Scholar
  15. 15.
    Chang, Y.J., Chen, S.F., Huang, J.D.: A kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. Res. Dev. Disabil. 32(6), 2566–2570 (2011)CrossRefGoogle Scholar
  16. 16.
    Clark, A.: Being There: Putting Brain, Body, and World Together Again. MIT press, Cambridge (1998)Google Scholar
  17. 17.
    Cummings, J.J., Bailenson, J.N.: How immersive is enough? a meta-analysis of the effect of immersive technology on user presence. Media Psychol. (ahead-of-print) 19, 1–38 (2015)Google Scholar
  18. 18.
    van Dijk, J., van der Lugt, R., Hummels, C.: Beyond distributed representation: embodied cognition design supporting socio-sensorimotor couplings. In: Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction, pp. 181–188. ACM (2014)Google Scholar
  19. 19.
    Dourish, P.: Where the Action Is: The Foundations of Embodied Interaction. MIT press, Cambridge (2004)Google Scholar
  20. 20.
    Downs, J., Vetere, F., Howard, S., Loughnan, S., Smith, W.: Audience experience in social videogaming: effects of turn expectation and game physicality. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, pp. 3473–3482. ACM (2014)Google Scholar
  21. 21.
    Gajadhar, B.J., De Kort, Y., IJsselsteijn, W.A.: Rules of engagement: influence of co-player presence on player involvement in digital games. Int. J. Gaming Computer-Mediated Simul. (IJGCMS) 1(3), 14–27 (2009)CrossRefGoogle Scholar
  22. 22.
    Gerling, K., Livingston, I., Nacke, L., Mandryk, R.: Full-body motion-based game interaction for older adults. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2012, New York, NY, USA, pp. 1873–1882. ACM (2012).
  23. 23.
    Goldin-Meadow, S., Nusbaum, H., Kelly, S.D., Wagner, S.: Explaining math: gesturing lightens the load. Psychol. Sci. 12(6), 516–522 (2001)CrossRefGoogle Scholar
  24. 24.
    He, G.F., Park, J.W., Kang, S.K., Jung, S.T.: Development of gesture recognition-based serious games. In: 2012 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), pp. 922–925. IEEE (2012)Google Scholar
  25. 25.
    Höök, K., Ståhl, A., Jonsson, M., Mercurio, J., Karlsson, A., Banka Johnson, E.C.: Somaethetic design. Interactions 22, 26–33 (2015). CrossRefGoogle Scholar
  26. 26.
    Hornecker, E., Buur, J.: Getting a grip on tangible interaction: a framework on physical space and social interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 437–446. ACM (2006)Google Scholar
  27. 27.
    Ishii, H., Ullmer, B.: Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 234–241. ACM (1997)Google Scholar
  28. 28.
    Istance, H., Vickers, S., Hyrskykari, A.: Gaze-based interaction with massively multiplayer on-line games. In: CHI 2009 Extended Abstracts on Human Factors in Computing Systems, pp. 4381–4386. ACM (2009)Google Scholar
  29. 29.
    Jack, D., Boian, R., Merians, A., Adamovich, S.V., Tremaine, M., Recce, M., Burdea, G.C., Poizner, H.: A virtual reality-based exercise program for stroke rehabilitation. In: Proceedings of the Fourth International ACM Conference on Assistive Technologies, pp. 56–63. ACM (2000)Google Scholar
  30. 30.
    Johnson, M.: The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. University of Chicago Press, Chicago (2013)Google Scholar
  31. 31.
    Jordà, S., Kaltenbrunner, M., Geiger, G., Alonso, M.: The reactable: a tangible tabletop musical instrument and collaborative workbench. In: ACM SIGGRAPH 2006 Sketches, SIGGRAPH 2006, New York, NY, USA. ACM (2006).
  32. 32.
    Kipp, M., Gebhard, P.: IGaze: studying reactive gaze behavior in semi-immersive human-avatar interactions. In: Prendinger, H., Lester, J., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 191–199. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-85483-8_19 CrossRefGoogle Scholar
  33. 33.
    Kjeldsen, R., Hartman, J.: Design issues for vision-based computer interaction systems. In: Proceedings of the 2001 Workshop on Perceptive User Interfaces, pp. 1–8. ACM (2001)Google Scholar
  34. 34.
    Klemmer, S.R., Hartmann, B., Takayama, L.: How bodies matter: five themes for interaction design. In: Proceedings of the 6th Conference on Designing Interactive Systems, pp. 140–149. ACM (2006)Google Scholar
  35. 35.
    Lankes, M., Mirlacher, T., Wagner, S., Hochleitner, W.: Whom are you looking for?: the effects of different player representation relations on the presence in gaze-based games. In: Proceedings of the First ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play, pp. 171–179. ACM (2014)Google Scholar
  36. 36.
    Lee, J.E.R., Nass, C.I., Bailenson, J.N.: Does the mask govern the mind?: effects of arbitrary gender representation on quantitative task performance in avatar-represented virtual groups. Cyberpsychology Behav. Soc. Netw. 17(4), 248–254 (2014)CrossRefGoogle Scholar
  37. 37.
    Li, Y., Fontijn, W., Markopoulos, P.: A tangible tabletop game supporting therapy of children with cerebral palsy. In: Markopoulos, P., Ruyter, B., IJsselsteijn, W., Rowland, D. (eds.) Fun and Games 2008. LNCS, pp. 182–193. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-88322-7_18 CrossRefGoogle Scholar
  38. 38.
    Malinverni, L., Burguès, N.P.: The medium matters: the impact of full-body interaction on the socio-affective aspects of collaboration. In: Proceedings of the 14th International Conference on Interaction Design and Children, IDC 2015, New York, NY, USA, pp. 89–98. ACM (2015).
  39. 39.
    Maurer, B., Aslan, I., Wuchse, M., Neureiter, K., Tscheligi, M.: Gaze-based onlooker integration: exploring the in-between of active player and passive spectator in co-located gaming. In: Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play, CHI PLAY 2015, NewYork, NY, USA, pp. 163–173. ACM (2015).
  40. 40.
    Maurer, B., Bergner, F., Kober, P., Baumgartner, R.: Improving rehabilitation process after total knee replacement surgery through visual feedback and enhanced communication in a serious game. In: Proceedings of the 30th ACM International Conference on Design of Communication, pp. 355–356. ACM (2012)Google Scholar
  41. 41.
    Maurer, B., Gärtner, M., Wuchse, M., Meschtscherjakov, A., Tscheligi, M.: Utilizing a digital game as a mediatory artifact for social persuasion to prevent speeding. In: Meschtscherjakov, A., Ruyter, B., Fuchsberger, V., Murer, M., Tscheligi, M. (eds.) PERSUASIVE 2016. LNCS, pp. 199–210. Springer, Heidelberg (2016). doi: 10.1007/978-3-319-31510-2_17 Google Scholar
  42. 42.
    McCallum, S., Boletsis, C.: Dementia games: a literature review of dementia-related serious games. In: Ma, M., Oliveira, M.F., Petersen, S., Hauge, J.B. (eds.) SGDA 2013. LNCS, vol. 8101, pp. 15–27. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40790-1_2 CrossRefGoogle Scholar
  43. 43.
    McCallum, S., Boletsis, C.: A taxonomy of serious games for dementia. In: Schouten, B., Fedtke, S., Bekker, T., Schijven, M., Gekker, A. (eds.) Games for Health, pp. 219–232. Springer Fachmedien Wiesbaden, Wiesbaden (2013)CrossRefGoogle Scholar
  44. 44.
    Morawe, V., Reiff, T.: PainStation. Ars Electronica (2002)Google Scholar
  45. 45.
    Mizobata, R., Silpasuwanchai, C., Ren, X.: Only for casual players?: investigating player differences in full-body game interaction. In: Proceedings of the Second International Symposium of Chinese CHI, Chinese CHI 2014, New York, NY, USA, pp. 57–65. ACM (2014).
  46. 46.
    Mueller, F., Isbister, K., Mueller, F., Isbister, K.: Movement-based game guidelines. ACM, New York (2014)Google Scholar
  47. 47.
    Nacke, L.E., Stellmach, S., Sasse, D., Lindley, C.A.: Gameplay experience in a gaze interaction game. In: Proceedings of 5th Conference on Communication by Gaze Interaction - COGAIN 2009, pp. 49–54. The COGAIN Association (2009)Google Scholar
  48. 48.
    Nowak, K.L., Rauh, C.: The influence of the avatar on online perceptions of anthropomorphism, androgyny, credibility, homophily, and attraction. J. Computer-Mediated Commun. 11(1), 153–178 (2005)CrossRefGoogle Scholar
  49. 49.
    Peitz, J., Eriksson, D., Björk, S.: Augmented board games: enhancing board games with electronics. In: Proceedings of DiGRA 2005 Conference: Changing Views-Worlds in Play (2005)Google Scholar
  50. 50.
    Pfeifer, R., Bongard, J.: How the Body Shapes the Way We Think: A New View of Intelligence. MIT press, Cambridge (2006)Google Scholar
  51. 51.
    Purvis, C.K., Jones, M., Bailey, J.O., Bailenson, J., Taylor, C.B.: Developing a novel measure of body satisfaction using virtual reality. PloS one 10(10), e0140158 (2015)CrossRefGoogle Scholar
  52. 52.
    Schönauer, C., Pintaric, T., Kaufmann, H.: Full body interaction for serious games in motor rehabilitation. In: Proceedings of the 2nd Augmented Human International Conference, AH 2011, pp. 4:1–4:8, New York, NY, USA. ACM (2011).
  53. 53.
    Schwarz, J., Marais, C.C., Leyvand, T., Hudson, S.E., Mankoff, J.: Combining body pose, gaze, and gesture to determine intention to interact in vision-based interfaces. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, CHI 2014, New York, NY, USA, pp. 3443–3452. ACM (2014).
  54. 54.
    Shusterman, R.: Body consciousness: a philosophy of mindfulness and somaesthetics. Cambridge University Press, Cambridge (2008)CrossRefGoogle Scholar
  55. 55.
    Sjöblom, B.: Gaming as a situated collaborative practice. Human IT 9(3), 128–165 (2008)Google Scholar
  56. 56.
    Steptoe, W., Wolff, R., Murgia, A., Guimaraes, E., Rae, J., Sharkey, P., Roberts, D., Steed, A.: Eye-tracking for avatar eye-gaze and interactional analysis in immersive collaborative virtual environments. In: Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work, pp. 197–200. ACM (2008)Google Scholar
  57. 57.
    Valli, A.: The design of natural interaction. Multimedia Tools Appl. 38(3), 295–305 (2008)CrossRefGoogle Scholar
  58. 58.
    Vickers, S., Istance, H., Hyrskykari, A., Ali, N., Bates, R.: Keeping an eye on the game: eye gaze interaction with massively multiplayer online games and virtual communities for motor impaired users (2008)Google Scholar
  59. 59.
    Weiser, M., Brown, J.S.: The coming age of calm technology. In: Beyond Calculation, pp. 75–85. Springer (1997)Google Scholar
  60. 60.
    Won, A.S., Tataru, C.A., Cojocaru, C.M., Krane, E.J., Bailenson, J.N., Niswonger, S., Golianu, B.: Two virtual reality pilot studies for the treatment of pediatric crps. Pain Med. 16, 1644–1647 (2015)CrossRefGoogle Scholar
  61. 61.
    Yee, N., Bailenson, J.: The proteus effect: the effect of transformed self-representation on behavior. Hum. Commun. Res. 33(3), 271–290 (2007). CrossRefGoogle Scholar
  62. 62.
    Yin, Y., Davis, R.: Toward natural interaction in the real world: real-time gesture recognition. In: International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, ICMI-MLMI 2010, New York, NY, USA, pp. 15:1–15:8. ACM (2010).
  63. 63.
    Zain, N.H.B.M., Jaafar, A.: Integrating digital games based learning environments with eye gaze-based interaction. In: 2011 International Conference on Pattern Analysis and Intelligent Robotics (ICPAIR), vol. 2, pp. 222–227. IEEE (2011)Google Scholar
  64. 64.
    Zalapa, R., Tentori, M.: Movement-based and tangible interactions to offer body awareness to children with autism. In: Urzaiz, G., Ochoa, S.F., Bravo, J., Chen, L.L., Oliveira, J. (eds.) UCAmI 2013. LNCS, vol. 8276, pp. 127–134. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  65. 65.
    Zhang, H., Fricker, D., Smith, T.G., Yu, C.: Real-time adaptive behaviors in multimodal human-avatar interactions. In: International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, ICMI-MLMI 2010, New York, NY, USA, pp. 4:1–4:8. ACM (2010).

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Center for Human-Computer InteractionUniversity of SalzburgSalzburgAustria

Personalised recommendations