Cognitive Processing

, Volume 18, Issue 3, pp 211–228 | Cite as

Lost in space: multisensory conflict yields adaptation in spatial representations across frames of reference

  • Johannes LohmannEmail author
  • Martin V. Butz
Research Report


According to embodied cognition, bodily interactions with our environment shape the perception and representation of our body and the surrounding space, that is, peripersonal space. To investigate the adaptive nature of these spatial representations, we introduced a multisensory conflict between vision and proprioception in an immersive virtual reality. During individual bimanual interaction trials, we gradually shifted the visual hand representation. As a result, participants unknowingly shifted their actual hands to compensate for the visual shift. We then measured the adaptation to the invoked multisensory conflict by means of a self-localization and an external localization task. While effects of the conflict were observed in both tasks, the effects systematically interacted with the type of localization task and the available visual information while performing the localization task (i.e., the visibility of the virtual hands). The results imply that the localization of one’s own hands is based on a multisensory integration process, which is modulated by the saliency of the currently most relevant sensory modality and the involved frame of reference. Moreover, the results suggest that our brain strives for consistency between its body and spatial estimates, thereby adapting multiple, related frames of reference, and the spatial estimates within, due to a sensory conflict in one of them.


Spatial perception Peripersonal space Multisensory integration Multisensory conflict Virtual reality 


  1. Barr DJ, Levy R, Scheepers C, Tily HJ (2013) Random effects structure for confirmatory hypothesis testing: keep it maximal. J Mem Lang 68(3):255–278CrossRefGoogle Scholar
  2. Barsalou LW (2008) Grounded cognition. Annu Rev Psychol 59:617–645CrossRefPubMedGoogle Scholar
  3. Bates D, Mächler M, Bolker B, Walker S (2015) Fitting linear mixed-effects models using lme4. J Stat Softw 67(1):1–48CrossRefGoogle Scholar
  4. Botvinick M, Cohen J (1998) Rubber hands’feel’ touch that eyes see. Nature 391(6669):756CrossRefPubMedGoogle Scholar
  5. Butz MV, Herbort O, Hoffmann J (2007) Exploiting redundancy for flexible behavior: unsupervised learning in a modular sensorimotor control architecture. Psychol Rev 114:1015–1046CrossRefPubMedGoogle Scholar
  6. Butz MV, Kutter EF, Lorenz C (2014) Rubber hand illusion affects joint angle perception. PLoS ONE 9(3):e92854CrossRefPubMedPubMedCentralGoogle Scholar
  7. Canzoneri E, Ubaldi S, Rastelli V, Finisguerra A, Bassolino M, Serino A (2013) Tool-use reshapes the boundaries of body and peripersonal space representations. Exp Brain Res 228(1):25–42CrossRefPubMedGoogle Scholar
  8. Coello Y, Bartolo A, Amiri B, Devanne H, Houdayer E, Derambure P (2008) Perceiving what is reachable depends on motor representations: evidence from a transcranial magnetic stimulation study. PLoS ONE 3(8):e2862CrossRefPubMedPubMedCentralGoogle Scholar
  9. Cohen YE, Andersen RA (2002) A common reference frame for movement plans in the posterior parietal cortex. Nat Rev Neurosci 3(7):553–562CrossRefPubMedGoogle Scholar
  10. Ehrenfeld S, Butz MV (2013) The modular modality frame model: continuous body state estimation and plausibility-weighted information fusion. Biol Cybern 107(1):61–82CrossRefPubMedGoogle Scholar
  11. Ehrenfeld S, Herbort O, Butz MV (2013) Modular neuron-based body estimation: maintaining consistency over different limbs, modalities, and frames of reference. Front Comput Neurosci 7 (Article UNSP 148)Google Scholar
  12. Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870):429–433CrossRefPubMedGoogle Scholar
  13. Ernst MO, Bülthoff HH (2004) Merging the senses into a robust percept. Trends Cogn Sci 8(4):162–169CrossRefPubMedGoogle Scholar
  14. Farnè A, Làdavas E (2000) Dynamic size-change of hand peripersonal space following tool use. NeuroReport 11(8):1645–1649CrossRefPubMedGoogle Scholar
  15. Fogassi L, Gallese V, Fadiga L, Luppino G, Matelli M, Rizzolatti G (1996) Coding of peripersonal space in inferior premotor cortex (area f4). J Neurophysiol 76(1):141–157PubMedGoogle Scholar
  16. Gamberini L, Seraglia B, Priftis K (2008) Processing of peripersonal and extrapersonal space using tools: evidence from visual line bisection in real and virtual environments. Neuropsychologia 46(5):1298–1304CrossRefPubMedGoogle Scholar
  17. Gentilucci M, Jeannerod M, Tadary B, Decety J (1994) Dissociating visual and kinesthetic coordinates during pointing movements. Exp Brain Res 102(2):359–366CrossRefPubMedGoogle Scholar
  18. Glenberg AM, Witt JK, Metcalfe J (2013) From the revolution to embodiment 25 years of cognitive psychology. Perspect Psychol Sci 8(5):573–585CrossRefPubMedGoogle Scholar
  19. Guna J, Jakus G, Pogačnik M, Tomažič S, Sodnik J (2014) An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking. Sensors 14(2):3702–3720CrossRefPubMedPubMedCentralGoogle Scholar
  20. Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6(2):65–70Google Scholar
  21. Holmes NP, Spence C (2004) The body schema and multisensory representation(s) of peripersonal space. Cogn Process 5(2):94–105CrossRefPubMedPubMedCentralGoogle Scholar
  22. Holmes NP, Spence C (2005) Visual bias of unseen hand position with a mirror: spatial and temporal factors. Exp Brain Res 166(3–4):489–497CrossRefPubMedPubMedCentralGoogle Scholar
  23. Holmes NP, Snijders HJ, Spence C (2006) Reaching with alien limbs: visual exposure to prosthetic hands in a mirror biases proprioception without accompanying illusions of ownership. Percept Psychophys 68(4):685–701CrossRefPubMedPubMedCentralGoogle Scholar
  24. Iriki A, Tanaka M, Iwamura Y (1996) Coding of modified body schema during tool use by macaque postcentral neurones. NeuroReport 7(14):2325–2330CrossRefPubMedGoogle Scholar
  25. Jewell G, McCourt ME (2000) Pseudoneglect: a review and meta-analysis of performance factors in line bisection tasks. Neuropsychologia 38(1):93–110CrossRefPubMedGoogle Scholar
  26. Kirsch W, Kunde W (2013) Visual near space is scaled to parameters of current action plans. J Exp Psychol Hum Percept Perform 39(5):1313–1325CrossRefPubMedGoogle Scholar
  27. Kirsch W, Herbort O, Butz MV, Kunde W (2012) Influence of motor planning on distance perception within the peripersonal space. PLoS ONE 7(4):e34880CrossRefPubMedPubMedCentralGoogle Scholar
  28. Kuznetsova A, Bruun Brockhoff P, Haubo Bojesen Christensen R (2016) lmerTest: tests in linear mixed effects models., r package version 2.0-32
  29. Lawrence MA (2015) ez: easy analysis and visualization of factorial experiments., r package version 4.3
  30. Linkenauger SA, Witt JK, Proffitt DR (2011) Taking a hands-on approach: apparent grasping ability scales the perception of object size. J Exp Psychol Hum Percept Perform 37(5):1432–1441CrossRefPubMedGoogle Scholar
  31. Linkenauger SA, Bülthoff HH, Mohler BJ (2015) Virtual arm’s reach influences perceived distances but only after experience reaching. Neuropsychologia 70:393–401CrossRefPubMedGoogle Scholar
  32. Longo MR, Lourenco SF (2006) On the nature of near space: effects of tool use and the transition to far space. Neuropsychologia 44(6):977–981CrossRefPubMedGoogle Scholar
  33. Longo MR, Lourenco SF (2007) Space perception and body morphology: extent of near space scales with arm length. Exp Brain Res 177(2):285–290CrossRefPubMedGoogle Scholar
  34. Lourenco SF, Longo MR (2009) The plasticity of near space: evidence for contraction. Cognition 112(3):451–456CrossRefPubMedGoogle Scholar
  35. Macaluso E, Maravita A (2010) The representation of space near the body through touch and vision. Neuropsychologia 48(3):782–795CrossRefPubMedGoogle Scholar
  36. McGuire LM, Sabes PN (2009) Sensory transformations and the use of multiple reference frames for reach planning. Nat Neurosci 12(8):1056–1061CrossRefPubMedPubMedCentralGoogle Scholar
  37. Mohler BJ, Bu¨lthoff HH, Thompson WB, Creem-Regehr SH (2008) A full-body avatar improves egocentric distance judgments in an immersive virtual environment. In: Proceedings of the 5th symposium on applied perception in graphics and visualization, ACMGoogle Scholar
  38. R Core Team (2016) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna.
  39. Regenbrecht H, Schubert T (2002) Real and illusory interactions enhance presence in virtual environments. Presence Teleoper Virt 11(4):425–434CrossRefGoogle Scholar
  40. Rossetti Y, Desmurget M, Prablanc C (1995) Vectorial coding of movement: vision, proprioception, or both? J Neurophysiol 74(1):457–463PubMedGoogle Scholar
  41. Schroeder PA, Lohmann J, Butz MV, Christian P (2016) Behavioral bias for food reflected in hand movements: a preliminary study with healthy subjects. Cyberpsychol Behav Soc Netw 19:120–126. doi: 10.1089/cyber.2015.0311 CrossRefPubMedGoogle Scholar
  42. Schubert T, Friedmann F, Regenbrecht H (2001) The experience of presence: factor analytic insights. Presence 10(3):266–281CrossRefGoogle Scholar
  43. Simons DJ, Chabris CF (1999) Gorillas in our midst: sustained inattentional blindness for dynamic events. Perception 28(9):1059–1074CrossRefPubMedGoogle Scholar
  44. Simons DJ, Levin DT (1997) Change blindness. Trends Cogn Sci 1(7):261–267CrossRefPubMedGoogle Scholar
  45. Wood G, Willmes K, Nuerk HC, Fischer MH (2008) On the cognitive link between space and number: a meta-analysis of the SNARC effect. Psychol Sci Q 50(4):489–525Google Scholar

Copyright information

© Marta Olivetti Belardinelli and Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  1. 1.Cognitive ModelingDepartment of Computer Science, University of TübingenTübingenGermany

Personalised recommendations