On perceptual biases in virtual object manipulation: Signal reliability and action relevance matter

  • Wladimir KirschEmail author
  • Wilfried Kunde


This study examined the role of visual reliability and action relevance in mutual visual-proprioceptive attraction in a virtual grasping task. Participants initially enclosed either the width or the height of a visual rectangular object with two cursors controlled by the movements of the index finger and thumb. Then, either the height or the width of this object or the distance between the fingers was judged. The judgments of object’s size were attracted by the felt finger distance, and, vice versa, the judged finger distance was attracted by the size of the grasped object. The impact of the proprioceptive information on object judgments increased, whereas the impact of visual object information on finger judgments decreased when the reliability of the visual stimulus was reduced. Moreover, the proprioceptive bias decreased for the action-relevant stimulus dimension as compared with the action-irrelevant stimulus dimension. These results indicate sensory integration of spatially separated sensory signals in the absence of any direct spatial or kinematic relation between them. We therefore suggest that the basic principles of sensory integration apply to the broad research field on perceptual-motor interactions as well as to many virtual interactions with external objects.


Perception and action Multisensory processing 



  1. Bhalla, M., & Proffitt, D. R. (1999). Visual–motor recalibration in geographical slant perception. Journal of Experimental Psychology: Human Perception and Performance, 25(4), 1076–1096.Google Scholar
  2. Bresciani, J. P., Ernst, M. O., Drewing, K., Bouyer, G., Maury, V., & Kheddar, A. (2005). Feeling what you hear: Auditory signals can modulate tactile tap perception. Experimental Brain Research, 162, 172–180.CrossRefGoogle Scholar
  3. Collier, E. S., & Lawson, R. (2016). Defining filled and empty space: reassessing the filled space illusion for active touch and vision. Experimental Brain Research, 234, 2697–2708.CrossRefGoogle Scholar
  4. Debats, N. B., Ernst, M. O., & Heuer, H. (2017a). Kinematic cross-correlation induces sensory integration across separate objects. European Journal of Neuroscience, 46, 2826–2834.CrossRefGoogle Scholar
  5. Debats, N. B., Ernst, M. O., & Heuer, H. (2017b). Perceptual attraction in tool-use: Evidence for a reliability-based weighting mechanism. Journal of Neurophysiology, 117, 1569–1580.CrossRefGoogle Scholar
  6. Deroy, O., Spence, C., & Noppeney, U. (2016). Metacognition in multisensory perception. Trends in Cognitive Sciences, 20(10), 736–747.CrossRefGoogle Scholar
  7. Durgin, F. H., Klein, B., Spiegel, A., Strawser, C. J., & Williams, M. (2012). The social psychology of perception experiments: Hills, backpacks, glucose, and the problem of generalizability. Journal of Experimental Psychology: Human Perception and Performance, 38, 1582–1595.Google Scholar
  8. Ernst, M. O. (2006). A Bayesian view on multimodal cue integration. In G. Knoblich, I. M. Thornton, M. Grosjean, & M. Shiffrar (Eds.). Human body perception from the inside out (pp. 105–131). New York, NY: Oxford University Press.Google Scholar
  9. Ernst, M. O. (2007). Learning to integrate arbitrary signals from vision and touch. Journal of Vision, 7, 1–14.CrossRefGoogle Scholar
  10. Ernst, M. O., & Banks, M. S. (2002). Human integrate visual and haptic information in a statistically optimal fashion. Nature, 415(24), 429–433.CrossRefGoogle Scholar
  11. Ernst, M. O., & Bülthoff, H. H. (2004). Merging the senses into a robust percept. Trends in Cognitive Science 8(4), 162–169.CrossRefGoogle Scholar
  12. Firestone, C., & Scholl, B. J. (2015). Cognition does not affect perception: Evaluating the evidence for “top-down” effects. Behavioral and Brain Sciences, 1–72. Advance online publication. doi:
  13. Gepshtein, S., Burge, J., Ernst, M. O., & Banks, M. S. (2005). The combination of vision and touch depends on spatial proximity. Journal of Vision, 5, 1013–1023.CrossRefGoogle Scholar
  14. Grosjean, M., Zwickel, J., & Prinz, W. (2009). Acting while perceiving: Assimilation precedes contrast. Psychological Research, 73(1), 3–13.CrossRefGoogle Scholar
  15. Harris, L. R., Carnevale, M. J., D’Amour, S., Fraser, L. E., Harrar, V., Hoover, A. E. N., Mander, C., & Pritchett, L. M. (2015). How our body influences our perception of the world. Frontiers in Psychology, 6, 819. doi: Google Scholar
  16. Helbig, H. B., & Ernst, M. O. (2007). Knowledge about a common source can promote visual–haptic integration. Perception, 36, 1523–1533.CrossRefGoogle Scholar
  17. Hommel, B. (2004). Event files: Feature binding in and across perception and action. Trends in Cognitive Sciences, 8(11), 494–500.CrossRefGoogle Scholar
  18. Hommel, B., Müsseler, J., Aschersleben, G., & Prinz, W. (2001). The theory of event coding (TEC): A framework for perception and action planning. Behavioral and Brain Sciences, 24, 849–937.CrossRefGoogle Scholar
  19. Kirsch, W., Herbort, O., Ullrich, B., & Kunde, W. (2017). On the origin of body-related influences on visual perception. Journal of Experimental Psychology: Human Perception and Performance, 43(6), 1222–1237.Google Scholar
  20. Kirsch, W., & Kunde, W. (2013). Visual near space is scaled to parameters of current action plans. Journal of Experimental Psychology: Human Perception and Performance,39(5), 1313–1325.Google Scholar
  21. Kirsch, W., & Kunde, W. (2014). Impact of planned movement direction on judgments of visual locations. Psychological Research, 78, 705–720.CrossRefGoogle Scholar
  22. Kirsch, W., & Kunde, W. (2018). The paddle effect in the Pong task is not due to blocking ability of the observer. Journal of Experimental Psychology: Human Perception and Performance. 44(11), 1799–1804.Google Scholar
  23. Körding, K. P., Beierholm, U., Ma, W. J., Quartz, S., Tenenbaum, J. B., & Shams, L. (2007). Causal inference in multisensory perception. PLOS ONE, 2, e943. doi: CrossRefGoogle Scholar
  24. Lindemann, O., & Bekkering, H. (2009). Object manipulation and motion perception: Evidence of an influence of action planning on visual processing. Journal of Experimental Psychology: Human Perception and Performance, 35, 1062–1071.Google Scholar
  25. Philbeck, J. W., & Witt, J. K. (2015). Action-specific influences on perception and postperceptual processes: Present controversies and future directions. Psychological Bulletin, 141, 1120–1144.CrossRefGoogle Scholar
  26. Proffitt, D. R., & Linkenauger, S. A. (2013). Perception viewed as a phenotypic expression. In W. Prinz (Ed.), Tutorials in action science (pp. 171–197). Cambridge, MA: MIT Press.CrossRefGoogle Scholar
  27. Rand, M. K., & Heuer, H. (2013). Implicit and explicit representations of hand position in tool use. PLOS ONE, 8(7), e68471. doi: CrossRefGoogle Scholar
  28. Rand, M.K., & Heuer, H. (2016). Effects of reliability and global context on explicit and implicit measures of sensed hand position in cursor control tasks. Frontiers in Psychology, 6, 2056. doi: CrossRefGoogle Scholar
  29. Rock, I., & Victor, J. (1964). Vision and touch: An experimentally created conflict between the two senses. Science, 143, 594–596.CrossRefGoogle Scholar
  30. Roach, N. W., Heron, J., & McGraw, P. V. (2006). Resolving multisensory conflict: A strategy for balancing the costs and benefits of audio-visual integration. Proceedings Biological Sciences, 273, 2159–2168.CrossRefGoogle Scholar
  31. Schubö, A., Prinz, W., & Aschersleben, G. (2004). Perceiving while acting: Action affects perception. Psychological Research, 68, 208–215.CrossRefGoogle Scholar
  32. Shams, L., & Beierholm, U. R. (2010). Causal inference in perception. Trends in Cognitive Sciences, 14(9), 425–432.CrossRefGoogle Scholar
  33. Takahashi, C., Diedrichsen, J., & Watt, S. J. (2009). Integration of vision and haptics during tool use. Journal of Vision, 9, 3–13.CrossRefGoogle Scholar
  34. Takahashi, C., & Watt, S. J. (2014). Visual-haptic integration with pliers and tongs: Signal “weights” take account of changes in haptic sensitivity caused by different tools. Frontiers in Psychology, 5, 109. doi: CrossRefGoogle Scholar
  35. Takahashi, C., & Watt, S. J. (2017). Optimal visual-haptic integration with articulated tools. Experimental Brain Research, 235, 1361–1373.CrossRefGoogle Scholar
  36. Thomaschke, R., Hopkins, B., & Miall, R. C. (2012). The planning and control model (PCM) of motorvisual priming: Reconciling motorvisual impairment and facilitation effects. Psychological Review, 119(2), 388–407.CrossRefGoogle Scholar
  37. Welch, R. B., & Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88, 638–667.CrossRefGoogle Scholar
  38. Wann, J. P., & Ibrahim, S. F. (1992). Does limb proprioception drift? Experimental Brain Research, 91, 162–166.CrossRefGoogle Scholar
  39. Witt, J. K. (2011). Action’s effect on perception. Current Directions in Psychological Science, 20, 201–206.CrossRefGoogle Scholar
  40. Witt, J. K., & Sugovic, M. (2012). Does ease to block a ball affect perceived ball speed? Examination of alternative hypotheses. Journal of Experimental Psychology: Human Perception and Performance, 38, 1202–1214.Google Scholar
  41. Zwickel, J., Grosjean, M., & Prinz, W. (2010a). On interference effects in concurrent perception and action. Psychological Research, 74, 152–171.CrossRefGoogle Scholar
  42. Zwickel, J., Grosjean, M., & Prinz, W. (2010b). What part of an action interferes with ongoing perception? Acta Psychologica, 134, 403–409.CrossRefGoogle Scholar
  43. Zwickel, J., & Prinz, W. (2012). Assimilation and contrast: the two sides of specific interference between action and perception. Psychological Research, 72(2), 171–182.CrossRefGoogle Scholar

Copyright information

© The Psychonomic Society, Inc. 2019

Authors and Affiliations

  1. 1.Institut für Psychologie III der Universität WürzburgWürzburgGermany

Personalised recommendations