Lost in space: multisensory conflict yields adaptation in spatial representations across frames of reference

Johannes Lohmann; Martin V. Butz

doi:10.1007/s10339-017-0798-5

Cognitive Processing

pp 1–18

Lost in space: multisensory conflict yields adaptation in spatial representations across frames of reference

Authors
Authors and affiliations

Johannes LohmannEmail author
Martin V. Butz

Research Report

First Online:: 27 March 2017

Received:: 15 May 2016
Accepted:: 09 March 2017

4 Shares
132 Downloads

Abstract

According to embodied cognition, bodily interactions with our environment shape the perception and representation of our body and the surrounding space, that is, peripersonal space. To investigate the adaptive nature of these spatial representations, we introduced a multisensory conflict between vision and proprioception in an immersive virtual reality. During individual bimanual interaction trials, we gradually shifted the visual hand representation. As a result, participants unknowingly shifted their actual hands to compensate for the visual shift. We then measured the adaptation to the invoked multisensory conflict by means of a self-localization and an external localization task. While effects of the conflict were observed in both tasks, the effects systematically interacted with the type of localization task and the available visual information while performing the localization task (i.e., the visibility of the virtual hands). The results imply that the localization of one’s own hands is based on a multisensory integration process, which is modulated by the saliency of the currently most relevant sensory modality and the involved frame of reference. Moreover, the results suggest that our brain strives for consistency between its body and spatial estimates, thereby adapting multiple, related frames of reference, and the spatial estimates within, due to a sensory conflict in one of them.

Keywords

Spatial perception Peripersonal space Multisensory integration Multisensory conflict Virtual reality

References

Barr DJ, Levy R, Scheepers C, Tily HJ (2013) Random effects structure for confirmatory hypothesis testing: keep it maximal. J Mem Lang 68(3):255–278CrossRefGoogle Scholar
Barsalou LW (2008) Grounded cognition. Annu Rev Psychol 59:617–645CrossRefPubMedGoogle Scholar
Bates D, Mächler M, Bolker B, Walker S (2015) Fitting linear mixed-effects models using lme4. J Stat Softw 67(1):1–48CrossRefGoogle Scholar
Botvinick M, Cohen J (1998) Rubber hands’feel’ touch that eyes see. Nature 391(6669):756CrossRefPubMedGoogle Scholar
Butz MV, Herbort O, Hoffmann J (2007) Exploiting redundancy for flexible behavior: unsupervised learning in a modular sensorimotor control architecture. Psychol Rev 114:1015–1046CrossRefPubMedGoogle Scholar
Butz MV, Kutter EF, Lorenz C (2014) Rubber hand illusion affects joint angle perception. PLoS ONE 9(3):e92854CrossRefPubMedPubMedCentralGoogle Scholar
Canzoneri E, Ubaldi S, Rastelli V, Finisguerra A, Bassolino M, Serino A (2013) Tool-use reshapes the boundaries of body and peripersonal space representations. Exp Brain Res 228(1):25–42CrossRefPubMedGoogle Scholar
Coello Y, Bartolo A, Amiri B, Devanne H, Houdayer E, Derambure P (2008) Perceiving what is reachable depends on motor representations: evidence from a transcranial magnetic stimulation study. PLoS ONE 3(8):e2862CrossRefPubMedPubMedCentralGoogle Scholar
Cohen YE, Andersen RA (2002) A common reference frame for movement plans in the posterior parietal cortex. Nat Rev Neurosci 3(7):553–562CrossRefPubMedGoogle Scholar
Ehrenfeld S, Butz MV (2013) The modular modality frame model: continuous body state estimation and plausibility-weighted information fusion. Biol Cybern 107(1):61–82CrossRefPubMedGoogle Scholar
Ehrenfeld S, Herbort O, Butz MV (2013) Modular neuron-based body estimation: maintaining consistency over different limbs, modalities, and frames of reference. Front Comput Neurosci 7 (Article UNSP 148)Google Scholar
Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870):429–433CrossRefPubMedGoogle Scholar
Ernst MO, Bülthoff HH (2004) Merging the senses into a robust percept. Trends Cogn Sci 8(4):162–169CrossRefPubMedGoogle Scholar
Farnè A, Làdavas E (2000) Dynamic size-change of hand peripersonal space following tool use. NeuroReport 11(8):1645–1649CrossRefPubMedGoogle Scholar
Fogassi L, Gallese V, Fadiga L, Luppino G, Matelli M, Rizzolatti G (1996) Coding of peripersonal space in inferior premotor cortex (area f4). J Neurophysiol 76(1):141–157PubMedGoogle Scholar
Gamberini L, Seraglia B, Priftis K (2008) Processing of peripersonal and extrapersonal space using tools: evidence from visual line bisection in real and virtual environments. Neuropsychologia 46(5):1298–1304CrossRefPubMedGoogle Scholar
Gentilucci M, Jeannerod M, Tadary B, Decety J (1994) Dissociating visual and kinesthetic coordinates during pointing movements. Exp Brain Res 102(2):359–366CrossRefPubMedGoogle Scholar
Glenberg AM, Witt JK, Metcalfe J (2013) From the revolution to embodiment 25 years of cognitive psychology. Perspect Psychol Sci 8(5):573–585CrossRefPubMedGoogle Scholar
Guna J, Jakus G, Pogačnik M, Tomažič S, Sodnik J (2014) An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking. Sensors 14(2):3702–3720CrossRefPubMedPubMedCentralGoogle Scholar
Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6(2):65–70Google Scholar
Holmes NP, Spence C (2004) The body schema and multisensory representation(s) of peripersonal space. Cogn Process 5(2):94–105CrossRefPubMedPubMedCentralGoogle Scholar
Holmes NP, Spence C (2005) Visual bias of unseen hand position with a mirror: spatial and temporal factors. Exp Brain Res 166(3–4):489–497CrossRefPubMedPubMedCentralGoogle Scholar
Holmes NP, Snijders HJ, Spence C (2006) Reaching with alien limbs: visual exposure to prosthetic hands in a mirror biases proprioception without accompanying illusions of ownership. Percept Psychophys 68(4):685–701CrossRefPubMedPubMedCentralGoogle Scholar
Iriki A, Tanaka M, Iwamura Y (1996) Coding of modified body schema during tool use by macaque postcentral neurones. NeuroReport 7(14):2325–2330CrossRefPubMedGoogle Scholar
Jewell G, McCourt ME (2000) Pseudoneglect: a review and meta-analysis of performance factors in line bisection tasks. Neuropsychologia 38(1):93–110CrossRefPubMedGoogle Scholar
Kirsch W, Kunde W (2013) Visual near space is scaled to parameters of current action plans. J Exp Psychol Hum Percept Perform 39(5):1313–1325CrossRefPubMedGoogle Scholar
Kirsch W, Herbort O, Butz MV, Kunde W (2012) Influence of motor planning on distance perception within the peripersonal space. PLoS ONE 7(4):e34880CrossRefPubMedPubMedCentralGoogle Scholar
Kuznetsova A, Bruun Brockhoff P, Haubo Bojesen Christensen R (2016) lmerTest: tests in linear mixed effects models. https://CRAN.R-project.org/package=lmerTest, r package version 2.0-32
Lawrence MA (2015) ez: easy analysis and visualization of factorial experiments. https://CRAN.R-project.org/package=ez, r package version 4.3
Linkenauger SA, Witt JK, Proffitt DR (2011) Taking a hands-on approach: apparent grasping ability scales the perception of object size. J Exp Psychol Hum Percept Perform 37(5):1432–1441CrossRefPubMedGoogle Scholar
Linkenauger SA, Bülthoff HH, Mohler BJ (2015) Virtual arm’s reach influences perceived distances but only after experience reaching. Neuropsychologia 70:393–401CrossRefPubMedGoogle Scholar
Longo MR, Lourenco SF (2006) On the nature of near space: effects of tool use and the transition to far space. Neuropsychologia 44(6):977–981CrossRefPubMedGoogle Scholar
Longo MR, Lourenco SF (2007) Space perception and body morphology: extent of near space scales with arm length. Exp Brain Res 177(2):285–290CrossRefPubMedGoogle Scholar
Lourenco SF, Longo MR (2009) The plasticity of near space: evidence for contraction. Cognition 112(3):451–456CrossRefPubMedGoogle Scholar
Macaluso E, Maravita A (2010) The representation of space near the body through touch and vision. Neuropsychologia 48(3):782–795CrossRefPubMedGoogle Scholar
McGuire LM, Sabes PN (2009) Sensory transformations and the use of multiple reference frames for reach planning. Nat Neurosci 12(8):1056–1061CrossRefPubMedPubMedCentralGoogle Scholar
Mohler BJ, Bu¨lthoff HH, Thompson WB, Creem-Regehr SH (2008) A full-body avatar improves egocentric distance judgments in an immersive virtual environment. In: Proceedings of the 5th symposium on applied perception in graphics and visualization, ACMGoogle Scholar
R Core Team (2016) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. https://www.R-project.org/
Regenbrecht H, Schubert T (2002) Real and illusory interactions enhance presence in virtual environments. Presence Teleoper Virt 11(4):425–434CrossRefGoogle Scholar
Rossetti Y, Desmurget M, Prablanc C (1995) Vectorial coding of movement: vision, proprioception, or both? J Neurophysiol 74(1):457–463PubMedGoogle Scholar
Schroeder PA, Lohmann J, Butz MV, Christian P (2016) Behavioral bias for food reflected in hand movements: a preliminary study with healthy subjects. Cyberpsychol Behav Soc Netw 19:120–126. doi:10.1089/cyber.2015.0311 CrossRefPubMedGoogle Scholar
Schubert T, Friedmann F, Regenbrecht H (2001) The experience of presence: factor analytic insights. Presence 10(3):266–281CrossRefGoogle Scholar
Simons DJ, Chabris CF (1999) Gorillas in our midst: sustained inattentional blindness for dynamic events. Perception 28(9):1059–1074CrossRefPubMedGoogle Scholar
Simons DJ, Levin DT (1997) Change blindness. Trends Cogn Sci 1(7):261–267CrossRefPubMedGoogle Scholar
Wood G, Willmes K, Nuerk HC, Fischer MH (2008) On the cognitive link between space and number: a meta-analysis of the SNARC effect. Psychol Sci Q 50(4):489–525Google Scholar

Copyright information

Authors and Affiliations

Johannes Lohmann
- 1
Email author
Martin V. Butz
- 1

1.Cognitive ModelingDepartment of Computer Science, University of TübingenTübingenGermany

Personalised recommendations

Actions

Unlimited access to the full article
Instant download
Include local sales tax if applicable

Subscribe to Journal

Get Access to

Cognitive Processing

for the whole of 2017

Lost in space: multisensory conflict yields adaptation in spatial representations across frames of reference

Abstract

Keywords

Copyright information

Authors and Affiliations

Personalised recommendations

Cookies