Predictive remapping leaves a behaviorally measurable attentional trace on eye-centered brain maps

Abstract

How does the brain maintain spatial attention despite the retinal displacement of objects by saccades? A possible solution is to use the vector of an upcoming saccade to compensate for the shift of objects on eye-centered (retinotopic) brain maps. In support of this hypothesis, previous studies have revealed attentional effects at the future retinal locus of an attended object, just before the onset of saccades. A critical yet unresolved theoretical issue is whether predictively remapped attentional effects would persist long enough on eye-centered brain maps, so no external input (goal, expectation, reward, memory, etc.) is needed to maintain spatial attention immediately following saccades. The present study examined this issue with inhibition of return (IOR), an attentional effect that reveals itself in both world-centered and eye-centered coordinates, and predictively remaps before saccades. In the first task, a saccade was introduced to a cueing task (“nonreturn-saccade task”) to show that IOR is coded in world-centered coordinates following saccades. In a second cueing task, two consecutive saccades were executed to trigger remapping and to dissociate the retinal locus relevant to remapping from the cued retinal locus (“return-saccade” task). IOR was observed at the remapped retinal locus 430-ms following the (first) saccade that triggered remapping. A third cueing task (“no-remapping” task) further revealed that the lingering IOR effect left by remapping was not confounded by the attention spillover. These results together show that predictive remapping leaves a robust attentional trace on eye-centered brain maps. This retinotopic trace is sufficient to sustain spatial attention for a few hundred milliseconds following saccades.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

Notes

  1. 1.

    Here, we focus our discussion on findings relevant to attention, but behavioral evidence for predictive remapping is also seen in visual crowding (Harrison et al., 2013) and tilt aftereffect (e.g., He et al., 2015; He et al., 2018).

  2. 2.

    Rolfs et al. (2011) suggested that attention (to a future saccade target) remaps in the opposite direction relative to the saccade vector and previous studies (Mathôt & Theeuwes, 2010; Melcher, 2007) had probed the wrong location. A recent empirical study and a reanalysis of the data from Rolfs et al. (2011) showed that the findings of Rolfs et al. (2011) may have been confounded by the spillover of attention (Arkesteijn et al., 2019), but Szinte et al. (2018) revealed clear evidence for remapping when the spillover of attention (evoked by cueing) is not a concern.

  3. 3.

    Golomb et al. (2008) probed a retinal locus relevant to remapping following two saccades and revealed a trace of attentional benefit (see Fig. S1 in the Supplemental Results accompanying this paper). They referred to this probe location as an “updated retinotopic location” and noted that the attentional benefit there “should only occur if subjects reencoded the spatiotopic location of the cue into new retinotopic coordinates during the second fixation, and this updated retinotopic memory trace persisted after the return saccade.” This study was not designed to reveal the retinotopic trace left by remapping; the fact that the probe was more likely to appear on the same side as the cued location (relative to the saccade vector) makes the results difficult to interpret.

References

  1. Abrams, R. A., & Pratt, J. (2000). Oculocentric coding of inhibited eye movements to recently attended locations. Journal of Experimental Psychology: Human Perception and Performance, 26(2), 776–788. https://doi.org/10.1037/0096-1523.26.2.776

    Article  PubMed  Google Scholar 

  2. Arkesteijn, K., Belopolsky, A. V., Smeets, J. B. J., & Donk, M. (2019). The limits of predictive remapping of attention across eye movements. Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.01146

  3. Awh, E., Armstrong, K. M., & Moore, T. (2006). Visual and oculomotor selection: Links, causes and implications for spatial attention. Trends in Cognitive Sciences, 10(3), 124–130. https://doi.org/10.1016/j.tics.2006.01.001

    Article  PubMed  Google Scholar 

  4. Awh, E., Belopolsky, A. V., & Theeuwes, J. (2012). Top-down versus bottom-up attentional control: A failed theoretical dichotomy. Trends in Cognitive Sciences, 16(8), 437–443. https://doi.org/10.1016/j.tics.2012.06.010

    Article  PubMed  PubMed Central  Google Scholar 

  5. Bennett, P., & Pratt, J. (2001). The spatial distribution of inhibition of return. Psychological Science, 12(1), 76–80. https://doi.org/10.1111/1467-9280.00313

    Article  PubMed  Google Scholar 

  6. Cavanagh, P., Hunt, A. R., Afraz, A., & Rolfs, M. (2010). Visual stability based on remapping of attention pointers. Trends in Cognitive Sciences, 14(4), 147–153. https://doi.org/10.1016/j.tics.2010.01.007

    Article  PubMed  PubMed Central  Google Scholar 

  7. Cohen, Y. A. (1981). Internal and External Control of Visual Orienting. Unpublished doctoral dissertation, University of Oregon.

  8. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Routledge Academic.

  9. d’Avossa, G., Tosetti, M., Crespi, S., Biagi, L., Burr, D. C., & Morrone, M. C. (2007). Spatiotopic selectivity of BOLD responses to visual motion in human area MT. Nature Neuroscience, 10(2), 249–255. https://doi.org/10.1038/nn1824

    Article  PubMed  Google Scholar 

  10. Danziger, S., Kingstone, A., & Snyder, J. J. (1998). Inhibition of return to successively stimulated locations in a sequential visual search paradigm. Journal of Experimental Psychology: Human Perception and Performance, 24(5), 1467–1475. https://doi.org/10.1037//0096-1523.24.5.1467

    Article  PubMed  Google Scholar 

  11. Dowd, E. W., & Golomb, J. D. (2019). Object-feature binding survives dynamic shifts of spatial attention. Psychological Science, 30(3), 343–361. https://doi.org/10.1177/0956797618818481

    Article  PubMed  PubMed Central  Google Scholar 

  12. Duhamel, Colby, C., & Goldberg, M. (1992). The updating of the representation of visual space in parietal cortex by intended eye movements. Science, 255(5040), 90–92. https://doi.org/10.1126/science.1553535

    Article  PubMed  Google Scholar 

  13. Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. https://doi.org/10.3758/BF03193146

    Article  PubMed  Google Scholar 

  14. Godijn, R., & Theeuwes, J. (2003). Parallel allocation of attention prior to the execution of saccade sequences. Journal of Experimental Psychology: Human Perception and Performance, 29(5), 882–896. https://doi.org/10.1037/0096-1523.29.5.882.

  15. Golomb, J. D. (2010). Robustness of the retinotopic attentional trace after eye movements. Journal of Vision, 10(3), 1–12. https://doi.org/10.1167/10.3.19.

  16. Golomb, J. D. (2019). Remapping locations and features across saccades: A dual-spotlight theory of attentional updating. Current Opinion in Psychology, 29, 211–218. https://doi.org/10.1016/j.copsyc.2019.03.018

    Article  PubMed  PubMed Central  Google Scholar 

  17. Golomb, J. D., Nguyen-Phuc, A. Y., Mazer, J. A., McCarthy, G., & Chun, M. M. (2010). Attentional facilitation throughout human visual cortex lingers in retinotopic coordinates after eye movements. Journal of Neuroscience, 30(31), 10493–10506. https://doi.org/10.1523/JNEUROSCI.1546-10.2010

    Article  PubMed  Google Scholar 

  18. Golomb, J. D., Chun, M. M., & Mazer, J. A. (2008). The native coordinate system of spatial attention is retinotopic. Journal of Neuroscience, 28(42), 10654–10662. https://doi.org/10.1523/JNEUROSCI.2525-08.2008

    Article  PubMed  Google Scholar 

  19. Golomb, J. D., & Kanwisher, N. (2012). Higher level visual cortex represents retinotopic, not spatiotopic, object location. Cerebral Cortex, 22(12), 2794–2810. https://doi.org/10.1093/cercor/bhr357

    Article  PubMed  Google Scholar 

  20. Gottlieb, J. P., Kusunoki, M., & Goldberg, M. E. (1998). The representation of visual salience in monkey parietal cortex. Nature, 391(6666), 481–484. https://doi.org/10.1038/35135

    Article  PubMed  Google Scholar 

  21. Harrison, W. J., Retell, J. D., Remington, R. W., & Mattingley, J. B. (2013). Visual crowding at a distance during predictive remapping. Current Biology, 23(9), 793–798. https://doi.org/10.1016/j.cub.2013.03.050

    Article  PubMed  Google Scholar 

  22. He, T., Ding, Y., & Wang, Z. (2015). Environment- and eye-centered inhibitory cueing effects are both observed after a methodological confound is eliminated. Scientific Reports, 5, Article 16586. https://doi.org/10.1038/srep16586

    Article  PubMed  Google Scholar 

  23. He, T., Fritsche, M., & de Lange, F. P. (2018). Predictive remapping of visual features beyond saccadic targets. Journal of Vision, 18(13), Article 20. https://doi.org/10.1167/18.13.20

    Article  Google Scholar 

  24. Hilchey, M. D., Hashish, M., Maclean, G. H., Satel, J., Ivanoff, J., & Klein, R. M. (2014). On the role of eye movement monitoring and discouragement on inhibition of return in a go/no-go task. Vision Research, 96, 133–139. https://doi.org/10.1016/j.visres.2013.11.008

    Article  PubMed  Google Scholar 

  25. Hilchey, M. D., Klein, R. M., Satel, J., & Wang, Z. (2012). Oculomotor inhibition of return: How soon is it “recoded” into spatiotopic coordinates? Attention, Perception, & Psychophysics, 74(6), 1145–1153. https://doi.org/10.3758/s13414-012-0312-1

    Article  Google Scholar 

  26. Jonikaitis, D., Szinte, M., Rolfs, M., & Cavanagh, P. (2013). Allocation of attention across saccades. Journal of Neurophysiology, 109(5), 1425–1434. https://doi.org/10.1152/jn.00656.2012

    Article  PubMed  Google Scholar 

  27. Kastner, S., Pinsk, M. A., De Weerd, P., Desimone, R., & Ungerleider, L. G. (1999). Increased activity in human visual cortex during directed attention in the absence of visual stimulation. Neuron, 22(4), 751–761. https://doi.org/10.1016/S0896-6273(00)80734-5

    Article  PubMed  Google Scholar 

  28. Klein, R. M., & MacInnes, W. J. (1999). Inhibition of return is a foraging facilitator in visual search. Psychological Science, 10(4), 346–352. https://doi.org/10.1111/1467-9280.00166

    Article  Google Scholar 

  29. Krüger, H. M., & Hunt, A. R. (2013). Inhibition of return across eye and object movements: The role of prediction. Journal of Experimental Psychology: Human Perception and Performance, 39(3), 735–744. https://doi.org/10.1037/a0030092

    Article  PubMed  Google Scholar 

  30. Malevich, T., Rybina, E., Ivtushok, E., Ardasheva, L., & MacInnes, W. J. (2020). No evidence for an independent retinotopic reference frame for inhibition of return. Acta Psychologica, 208, Article 103107. https://doi.org/10.1016/j.actpsy.2020.103107

    Article  PubMed  Google Scholar 

  31. Mathôt, S., & Theeuwes, J. (2010). Gradual Remapping Results in Early Retinotopic and Late Spatiotopic Inhibition of Return. Psychological Science, 21(12), 1793–1798. https://doi.org/10.1177/0956797610388813

    Article  PubMed  Google Scholar 

  32. Maylor, E. A., & Hockey, R. (1985). Inhibitory component of externally controlled covert orienting in visual space. Journal of Experimental Psychology: Human Perception and Performance, 11(6), 777–787. https://doi.org/10.1037/0096-1523.11.6.777

    Article  PubMed  Google Scholar 

  33. Melcher, D. (2007). Predictive remapping of visual features precedes saccadic eye movements. Nature Neuroscience, 10(7), 903–907. https://doi.org/10.1038/nn1917

    Article  PubMed  Google Scholar 

  34. Melcher, D., & Morrone, M. C. (2003). Spatiotopic temporal integration of visual motion across saccadic eye movements. Nature Neuroscience, 6(8), 877–881. https://doi.org/10.1038/nn1098

    Article  PubMed  Google Scholar 

  35. Nakamura, K., & Colby, C. L. (2002). Updating of the visual representation in monkey striate and extrastriate cortex during saccades. Proceedings of the National Academy of Sciences of the United States of America, 99(6), 4026–4031. https://doi.org/10.1073/pnas.052379899

    Article  PubMed  PubMed Central  Google Scholar 

  36. O’Regan, J. K., & Noë, A. (2001). A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences, 24(5), 939–973. https://doi.org/10.1017/S0140525X01000115

    Article  Google Scholar 

  37. Pertzov, Y., Zohary, E., & Avidan, G. (2010). Rapid formation of spatiotopic representations as revealed by inhibition of return. The Journal of Neuroscience, 30(26), 8882–8887. https://doi.org/10.1523/JNEUROSCI.3986-09.2010

    Article  PubMed  PubMed Central  Google Scholar 

  38. Posner, M. I. (1980). Orienting of attention. The Quarterly Journal of Experimental Psychology, 32(1), 3–25. https://doi.org/10.1080/00335558008248231

    Article  PubMed  Google Scholar 

  39. Posner, M. I., & Cohen, Y. (1984). Components of visual orienting. In H. Bouma & D. Bowhuis (Eds.), Attention and performance X: Control of language processes (pp. 531–556). Erlbaum.

  40. Posner, M. I., Rafal, R. D., Choate, L. S., & Vaughan, J. (1985). Inhibition of return: Neural basis and function. Cognitive Neuropsychology, 2(3), 211–228. https://doi.org/10.1080/02643298508252866

    Article  Google Scholar 

  41. Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological bulletin, 124(3), 372–422. https://doi.org/10.1037/0033-2909.124.3.372

    Article  PubMed  Google Scholar 

  42. Rolfs, M., Jonikaitis, D., Deubel, H., & Cavanagh, P. (2011). Predictive remapping of attention across eye movements. Nature Neuroscience, 14(2), 252–256. https://doi.org/10.1038/nn.2711

    Article  PubMed  Google Scholar 

  43. Sapir, A., Hayes, A., Henik, A., Danziger, S., & Rafal, R. (2004). Parietal lobe lesions disrupt saccadic remapping of inhibitory location tagging. Journal of Cognitive Neuroscience, 16(4), 503–509. https://doi.org/10.1162/089892904323057245.

  44. Satel, J., Wilson, N. R., & Klein, R. M. (2019). What neuroscientific studies tell us about inhibition of return. Vision, 3(4). https://doi.org/10.3390/vision3040058

  45. Schall, J. D. (2004). On the role of frontal eye field in guiding attention and saccades. Vision Research, 44(12), 1453–1467. https://doi.org/10.1016/j.visres.2003.10.025

    Article  PubMed  Google Scholar 

  46. Simons, D. J., Franconeri, S. L., & Reimer, R. L. (2000). Change blindness in the absence of a visual disruption. Perception, 29(10), 1143–1154. https://doi.org/10.1068/p3104

    Article  PubMed  Google Scholar 

  47. Simons, D. J., & Rensink, R. A. (2005). Change blindness: Past, present, and future. Trends in Cognitive Sciences, 9(1), 16–20. https://doi.org/10.1016/j.tics.2004.11.006

    Article  PubMed  Google Scholar 

  48. Sommer, M. A., & Wurtz, R. H. (2006). Influence of the thalamus on spatial visual processing in frontal cortex. Nature, 444(7117), 374–377. https://doi.org/10.1038/nature05279

    Article  PubMed  Google Scholar 

  49. Szinte, M., Jonikaitis, D., Rangelov, D., & Deubel, H. (2018). Pre-saccadic remapping relies on dynamics of spatial attention. ELife, 7, e37598. https://doi.org/10.7554/eLife.37598

    Article  PubMed  PubMed Central  Google Scholar 

  50. Szinte, M., Jonikaitis, D., Rolfs, M., Cavanagh, P., & Deubel, H. (2016). Presaccadic motion integration between current and future retinotopic locations of attended objects. Journal of Neurophysiology, 116(4), 1592–1602. https://doi.org/10.1152/jn.00171.2016

    Article  PubMed  PubMed Central  Google Scholar 

  51. Theeuwes, J. (2018). Visual selection: Usually fast and automatic; seldom slow and volitional. Journal of Cognition, 1(1), Article 29. https://doi.org/10.5334/joc.13

    Article  Google Scholar 

  52. Thomas, L. E., Ambinder, M. S., Hsieh, B., Levinthal, B., Crowell, J. A., Irwin, D. E., Kramer, A. F., Lleras, A., Simons, D. J., & Wang, R. F. (2006). Fruitful visual search: Inhibition of return in a virtual foraging task. Psychonomic Bulletin & Review, 13(5), 891–895. https://doi.org/10.3758/BF03194015

    Article  Google Scholar 

  53. Turi, M., & Burr, D. (2012). Spatiotopic perceptual maps in humans: Evidence from motion adaptation. Proceedings of the Royal Society B: Biological Sciences, 279(1740), 3091–3097. https://doi.org/10.1098/rspb.2012.0637

    Article  PubMed  Google Scholar 

  54. Umeno, M. M., & Goldberg, M. E. (1997). Spatial processing in the monkey frontal eye field. I. Predictive visual responses. Journal of Neurophysiology, 78(3), 1373–1383. https://doi.org/10.1152/jn.1997.78.3.1373

    Article  PubMed  Google Scholar 

  55. Walker, M. F., Fitzgibbon, E. J., & Goldberg, M. E. (1995). Neurons in the monkey superior colliculus predict the visual result of impending saccadic eye movements. Journal of Neurophysiology, 73(5), 1988–2003. https://doi.org/10.1152/jn.1995.73.5.1988

    Article  PubMed  Google Scholar 

  56. Wang, B., Yan, C., Klein, R. M., & Wang, Z. (2018). Inhibition of return revisited: Localized inhibition on top of a pervasive bias. Psychonomic Bulletin & Review, 25(5), 1861–1867. https://doi.org/10.3758/s13423-017-1410-9

    Article  Google Scholar 

  57. Wurtz, R. H. (2008). Neuronal mechanisms of visual stability. Vision research, 48(20), 2070–2089. https://doi.org/10.1016/j.visres.2008.03.021

    Article  PubMed  PubMed Central  Google Scholar 

  58. Yan, C., He, T., Klein, R. M., & Wang, Z. (2016). Predictive remapping gives rise to environmental inhibition of return. Psychonomic Bulletin & Review, 23, 1860–1866. https://doi.org/10.3758/s13423-016-1066-x

    Article  Google Scholar 

  59. Zhang, S., Xu, M., Kamigaki, T., Hoang Do, J. P., Chang, W.-C., Jenvay, S., Miyamichi, K., Luo, L., & Dan, Y. (2014). Long-range and local circuits for top-down modulation of visual cortex processing. Science, 345(6197), 660–665. https://doi.org/10.1126/science.1254126

    Article  PubMed  PubMed Central  Google Scholar 

  60. Zimmermann, E., Burr, D., & Morrone, M. C. (2011). Spatiotopic visual maps revealed by saccadic adaptation in humans. Current Biology, 21(16), 1380–1384. https://doi.org/10.1016/j.cub.2011.06.014

    Article  PubMed  Google Scholar 

Download references

Author notes

This project was supported by a grant (No. 31371133) from the National Natural Science Foundation of China to Z. Wang.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Zhiguo Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Figure S1
figure3

(A) Saccade latencies at the cued spatial location and the distance-matched control location in the nonreturn-saccade task. (B) Saccade latencies at the cued and control locations following a return saccade; note that the cued spatial location overlaps with the cued retinal locus (Spatial + Retinal), whereas the Remapped retinal locus was never stimulated by the cue. (C) Saccade latencies at the cued location and three control locations in the no-remapping task. Same as in Fig. 2, the light grey dots connected by lines were saccade latencies from individual participants. Error bars represent ±1 SEM. ** p < 0.01, *** p < 0.001. (PNG 400 kb)

ESM 1

(DOCX 18 kb)

High resolution image (TIFF 45093 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yan, C., He, T. & Wang, Z. Predictive remapping leaves a behaviorally measurable attentional trace on eye-centered brain maps. Psychon Bull Rev (2021). https://doi.org/10.3758/s13423-021-01893-1

Download citation

Keywords

  • Eye movements
  • Inhibition of return
  • Predictive remapping
  • Spatial attention
  • Visual stability