Action compatibility in spatial knowledge developed through virtual navigation

  • Qi WangEmail author
  • Holly A. Taylor
  • Tad T. Brunyé
Original Article


Action-compatibility effects (ACEs) arise due to incongruity between perceptuo-motor traces stored in memory and the perceptuo-motor demands of a retrieval task. Recent research has suggested that ACEs arising during spatial memory retrieval are additionally modulated by individual differences in how experienced participants are with a college campus environment. However, the extent and nature of experience with a real-world environment is difficult to assess and control, and characteristics of the retrieval task itself might modulate ACEs during spatial memory retrieval. The present study provides a more controlled and in-depth examination of how individual differences and task-based factors interact to shape ACEs when participants retrieve spatial memories. In two experiments, participants with varied video game experience learned a virtual environment and then used the computer mouse to verify spatial relationships from different perspectives. Mouse trajectories demonstrated ACEs, differing by retrieval perspective and video game experience. Videogame experts demonstrated the ACE based on learned spatial relationships during egocentric retrieval only, whereas videogame novices showed the ACE based on semantic processing of directional terms only. Specifically, gaming experts invoke perspective-specific perceptuo-motor associations to retrieve spatial knowledge, whereas non-experts are influenced by semantically based associations specific to the retrieval task. Results are discussed in the context of action-compatibility effects, the intentional weighting hypothesis, and the flexible encoding and retrieval of spatial information.



This research was funded by the National Natural Science Foundation of China (31,400,865).

Compliance with ethical standards

Ethical approval

All procedures performed in present studies involving human participants were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

All participants gave their informed consent prior to their inclusion in the study.

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Anguera, J. A., Boccanfuso, J., Rintoul, J. L., Faraji, F., Janowich, J., Kong, E., … Al-Hashimi, O. (2013). Video game training enhances cognitive control in older adults. Nature, 501(7465), 97–101. Scholar
  2. Basak, C., Boot, W. R., Voss, M. W., & Kramer, A. F. (2008). Can training in a real-time strategy video game attenuate cognitive decline in older adults? Psychology and Aging, 23(4), 765–777. Scholar
  3. Boot, W. R., Blakely, D. P., & Simons, D. J. (2011). Do action video games improve perception and cognition? Frontiers in Psychology, 2(SEP), 1–6. Scholar
  4. Boot, W. R., Kramer, A. F., Simons, D. J., Fabiani, M., & Gratton, G. (2008). The effects of video game playing on attention, memory, and executive control. Acta Psychologica, 129(3), 387–398. Scholar
  5. Boumenir, Y., Rebillard, G., & Dresp-Langley, B. Short-Term Memory Representation Of A Complex And Non-Familiar Environment After Brief Exposure. WebmedCentral BRAIN 2011;2(1):WMC001523. Scholar
  6. Brunyé, T. T., Burte, H., Houck, L. A., & Taylor, H. A. (2015). The map in our head is not oriented north: evidence from a real-world environment. PloS one, 10(9), e0135803. Scholar
  7. Brunyé, T. T., Gardony, A., Mahoney, C. R., & Taylor, H. A. (2012). Going to town: Visualized perspectives and navigation through virtual environments. Computers in Human Behavior, 28(1), 257–266. Scholar
  8. Brunye, T. T., Mahoney, C. R., & Taylor, H. A. (2010). Moving through imagined space: Mentally simulating locomotion during spatial description reading. Acta Psychologica, 134(1), 110–124. Scholar
  9. Brunye, T.,T., Ditman, T., Mahoney, C. R., Augustyn, J. S., & Taylor, H. A. (2009). When you and I share perspectives: Pronouns modulate perspective taking during narrative comprehension. Psychological Science, 20(1), 27–32. Scholar
  10. Carlson, L. A., & Kenny, R. (2006). Interpreting spatial terms involves simulating interactions. Psychonomic Bulletin & Review, 13(4), 682–688. Scholar
  11. Cobb, S. V. G., Nichols, S., Ramsey, A., & Wilson, J. R. (1999). Virtual Reality-Induced Symptoms and Effects (VRISE). Presence, 8(2), 169–186. Scholar
  12. Franklin, N., & Tversky, B. (1990). Searching imagined environments. Journal of Experimental Psychology. General, 119(1), 63–76. Scholar
  13. Freeman, J. B., & Ambady, N. (2010). MouseTracker: Software for studying real-time mental processing using a computer mouse-tracking method. Behavior Research Methods, 42(1), 226–241. Scholar
  14. Friedman, A., & Montello, D. R. (2006). Global-scale location and distance estimates: Common representations and strategies in absolute and relative judgments. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32(2), 333–346. Scholar
  15. Gaunet, F., Vidal, M., Kemeny, A., & Berthoz, A. (2001). Active, passive and snapshot exploration in a virtual environment: influence on scene memory, reorientation and path memory. Cognitive Brain Research, 11(3), 409–420. Scholar
  16. Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action.. Psychonomic Bulletin & Review, 9(3), 558–565. Scholar
  17. Glenberg, A. M., Lopez-Mobilia, G., McBeath, M., Toma, M., Sato, M., & Cattaneo, L. (2010). Knowing beans: Human mirror mechanisms revealed through motor adaptation. Frontiers in Human Neuroscience, 4
  18. Hommel, B. (2007). Feature integration across perception and action: Event files affect response choice. Psychological research, 71(1), 42–63. Scholar
  19. Hommel, B. (2011). The Simon effect as tool and heuristic. Acta psychologica, 136(2), 189–202. Scholar
  20. Hommel, B. (2017). Goal-directed actions. The Oxford Handbook of Causal Reasoning, p. 265.Google Scholar
  21. Hommel, B., Gehrke, J., & Knuf, L. (2000). Hierarchical coding in the perception and memory of spatial layouts. Psychological Research/Psychologische Forschung, 64(1), 1–10. Scholar
  22. Hommel, B., Müsseler, J., Aschersleben, G., & Prinz, W. (2001). The theory of event coding (TEC): A framework for perception and action planning. Behavioral and brain sciences, 24(5), 910–926. Scholar
  23. Jansen-Osmann, P., & Berendt, B. (2002). Investigating distance knowledge using virtual environments. Environment and Behavior, 34(2), 178–193. Scholar
  24. Klatzky, R. L., Loomis, J. M., Beall, A. C., Chance, S. S., & Golledge, R. G. (1998). Spatial updating of self-position and orientation during real, imagined, and virtual locomotion. Psychological Science, 9(4), 293–298. Scholar
  25. Knoblich, G., & Flach, R. (2001). Predicting the effects of actions: Interactions of perception and action.. Psychological Science, 12(6), 467–467-472. Scholar
  26. Levinson, S. C. (1996). Frames of reference and molyneux’s question: Crosslinguistic evidence. Cambridge, MA: The MIT Press.Google Scholar
  27. Li, H., & Giudice, N. A. (2013). The Effects of Immersion and Body-based Rotation on Learning Multi-level Indoor Virtual Environments. Proceedings of the Fifth ACM SIGSPATIAL International Workshop on Indoor Spatial Awareness - ISA13, 8–15.
  28. Li, L., & Warren, W. H. (2002). Retinal flow is sufficient for steering during observer rotation. Psychological Science, 13(5), 485–490. Scholar
  29. Longo, M. R., & Lourenco, S. F. (2007). Spatial attention and the mental number line: Evidence for characteristic biases and compression. Neuropsychologia,. 45(7), 1400–1407.
  30. Loomis, J. M., Blascovich, J. J., & Beall, A. C. (1999). Immersive virtual environment technology as a basic research tool in psychology. Behavior Research Methods, Instruments, & Computers, 31(4), 557–564. Scholar
  31. Majid, A., Bowerman, M., Kita, S., Haun, D. B. M., & Levinson, S. C. (2004). Can language restructure cognition? the case for space. Trends in Cognitive Sciences, 8(3), 108–114. Scholar
  32. Memelink, J., & Hommel, B. (2013). Intentional weighting: a basic principle in cognitive control. Psychological Research Psychologische Forschung, 77(3), 249–259. Scholar
  33. Montello, D. R., Waller, D., Hegarty, M., & Richardson, A. E. (2004). Spatial memory of real environments, virtual environments, and maps. Human spatial memory: Remembering where, 251–285.Google Scholar
  34. Munafo, J., Diedrick, M., & Stoffregen, T. A. (2016). The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects. Experimental Brain Research, 1–13.
  35. Murias, K., Kwok, K., Castillejo, A. G., Liu, I., & Iaria, G. (2016). The effects of video game use on performance in a virtual navigation task. Computers in Human Behavior, 58, 398–406. Scholar
  36. Parsons, L. M. (1994). Temporal and kinematic properties of motor behavior reflected in mentally simulated action. Journal of Experimental Psychology: Human Perception and Performance, 20(4), 709–730. Scholar
  37. Prinz, W. (1997). Perception and action planning. European Journal of Cognitive Psychology, 9(2), 129–154. Scholar
  38. Regier, T., & Carlson, L. A. (2001). Grounding spatial language in perception: An empirical and computational investigation. Journal of Experimental Psychology: General, 130(2), 273–298. Scholar
  39. Repp, B. H., & Knoblich, G. (2004). Perceiving action identity: How pianists recognize their own performances.. Psychological Science, 15(9), 604–604-609. Scholar
  40. Richardson, A. E., & Collaer, M. L. (2011). Virtual navigation performance: the relationship to field of view and prior video gaming experience. Perceptual and motor skills, 112(2), 477–498. Scholar
  41. Richardson, A. E., Montello, D. R., & Hegarty, M. (1999). Spatial knowledge acquisition from maps and from navigation in real and virtual environments.. Memory & Cognition, 27(4), 741 – 50.
  42. Richardson, A. E., Powers, M. E., & Bousquet, L. G. (2011). Video game experience predicts virtual, but not real navigation performance. Computers in Human Behavior, 27(1), 552–560. Scholar
  43. Richardson, D. C., Dale, R., & Spivey, M. J. (2007). Eye movements in language and cognition: A brief introduction. In M. Gonzalez-Marquez, I. Mittelberg, S. Coulson & M. J. Spivey (Eds.), Methods in cognitive linguistics (pp. 323–344). Amsterdam: John Benjamins Publishing Company.CrossRefGoogle Scholar
  44. Riecke, B. E., Cunningham, D. W., & Bülthoff, H. H. (2007). Spatial updating in virtual reality: the sufficiency of visual information. Psychological research, 71(3), 298–313. Scholar
  45. Rizzolatti, G., & Craighero, L. (2004). The mirror-neuron system. Annual Review of Neuroscience, 27, 169–192. Scholar
  46. Ruddle, R. A., & Lessels, S. (2006). For efficient navigational search, humans require full physical movement, but not a rich visual scene. Psychological Science, 17(6), 460–465. Scholar
  47. Ruddle, R. A., Payne, S. J., & Jones, D. M. (1999). Navigating Large-Scale Virtual Environments: What Differences Occur Between Helmet-Mounted and Desk-Top Displays? Abstract. Presence, 8(2), 157–168.CrossRefGoogle Scholar
  48. Ruddle, R. A., Volkova, E., Mohler, B., & Bülthoff, H. H. (2011). The effect of landmark and body-based sensory information on route knowledge. Memory & Cognition, 39(4), 686–699. Scholar
  49. Santos, S. B., Dias, P., Pimentel, A., Baggerman, J. W., Ferreira, C., Silva, S., & Madeira, J. (2009). Head-mounted display versus desktop for 3D navigation in virtual reality: A user study. Multimedia Tools and Applications, 41(1), 161–181. Scholar
  50. Sharples, S., Cobb, S., Moody, A., & Wilson, J. R. (2008). Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays, 29(2), 58–69. Scholar
  51. Shelton, A. L., & McNamara, T. P. (2004). Orientation and perspective dependence in route and survey learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30(1), 158–170. Scholar
  52. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. Scholar
  53. Sparrow, B., Liu, J., & Wegner, D. M. (2011). Google effects on memory: Cognitive consequences of having information at our fingertips. Science, 333(6043), 776–778. Scholar
  54. Spence, I., & Feng, J. (2010). Video Games and Spatial Cognition. Review of General Psychology, 14(2), 92–104. Scholar
  55. Spivey, M., Richardson, D., & Dale, R. (2009). The movement of eye and hand as a window into language and cognition. In E. Morsella, J. A. Bargh & P. M. Gollwitzer (Eds.), (pp. 225–249). New York, NY: Oxford University Press.Google Scholar
  56. Stanney, K. M., Hale, K. S., Nahmens, I., & Kennedy, R. S. (2003). What to Expect from Immersive Virtual Environment Exposure: Influences of Gender, Body Mass Index, and Past Experience. Human Factors, 45(3), 504–520. Scholar
  57. Taylor, H. A., & Tversky, B. (1992). Spatial mental models derived from survey and route descriptions. Journal of Memory and Language, 31(2), 261–292. Scholar
  58. Thorndyke, P. W., & Hayes-Roth, B. (1982). Differences in spatial knowledge acquired from maps and navigation.. Cognitive Psychology, 14(4), 560–589. Scholar
  59. Tversky, B. (2005). How to get around by mind and body: Spatial thought, spatial action. In A. Zilhao (Ed.), Cognition, evolution, and rationality: A cognitive science for the XXIst century. London: Routledge.Google Scholar
  60. Waller, D. (2000). Individual Differences in Spatial Learning From Computer-Simulated Environments. Journal of Experimental Psychology. Applied, 6(4), 307–321. Scholar
  61. Wang, Q., Taylor, H. A., & Brunyé, T. T. (2012). When going the right way is hard to do: Distinct phases of action compatibility in spatial knowledge development. Acta Psychologica, 139(3), 449–457. Scholar
  62. Willemsen, P., Colton, M. B., Creem-Regehr, S., & Thompson, W. B. (2009). The effects of head-mounted display mechanical properties and field of view on distance judgments in virtual environments. ACM Transactions on Applied Perception, 6(2)
  63. Wilson, M., & Knoblich, G. (2005). The case for motor involvement in perceiving conspecifics. Psychological Bulletin, 131(3), 460–473. Scholar
  64. Witt, J. K., Proffitt, D. R., & Epstein, W. (2004). Perceiving distance: A role of effort and intent. Perception, 33(5), 577–590. Scholar
  65. Zetzsche, C., Wolter, J., Galbraith, C., & Schill, K. (2009). Representation of space: Image-like or sensorimotor? Spatial Vision, 22(5), 409–424. Scholar
  66. Zwaan, R. A., & Taylor, L. J. (2006). Seeing, acting, understanding: Motor resonance in language comprehension. Journal of Experimental Psychology: General, 135(1), 1–11. Scholar
  67. Mizell, D. W., Jones, S. P., Slater, M., & Spanlang, B. (2002). Comparing immersive virtual reality with other display modes for visualizing complex 3D geometry. University College London, technical report. doi: Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of PsychologySun Yat-Sen UniversityGuangzhouChina
  2. 2.Department of PsychologyTufts UniversityMedfordUSA
  3. 3.Center for Applied Brain & Cognitive SciencesMedfordUSA
  4. 4.U.S. Army Natick Soldier Research, Development and Engineering Center, RDNS-SEW-THCNatickUSA

Personalised recommendations