Each time we make an eye movement, the retinal projection of the world shifts dramatically. This stands in contrast with our conscious and stable perception of the world. It is believed that one way in which this perceptual stability is achieved is through a mechanism known as predictive remapping. Predictive remapping refers to the fact that in the interval in which an eye movement has been programmed but not yet executed, many visual neurons shift their receptive fields (RFs) from their current, pre-saccadic location to their post-saccadic location.

Predictive remapping was first described by Duhamel et al. (1992) in a classic study in which neural activity was recorded from the monkey lateral intraparietal area (LIP). They showed that just before the execution of a saccade some LIP neurons become responsive to stimuli presented in their post-saccadic RF. They did this by presenting a stimulus outside of the neuron’s RF and having the monkey make an eye movement to bring the stimulus into the neuron’s RF. Even though no stimulus was present in their current (pre-saccadic) RF, just before the eye movement a subset of LIP neurons became active “as if” the saccade had already been executed and had brought the stimulus into their (post-saccadic) RF. The effects of predictive remapping appear to be particularly strong in the PPC, but have been reported in a number of other oculomotor and visual areas as well, such as the extrastriate cortex, the frontal eye fields and the superior colliculus (Nakamura and Colby 2002; Umeno and Goldberg 1997; Walker et al. 1995).

Recently, in a compelling psychophysical study, Melcher (2007) used the tilt adaptation after effect (TAE) to demonstrate predictive remapping in human participants. TAE is a systematic bias in reporting the orientation of a tester stimulus, typically a slightly tilted grating, after being exposed for some time to a tilted adapter stimulus. After the presentation of the adapter stimulus, participants had to make an eye movement. The tester stimulus was presented at the location which after saccade execution would retinotopically match the adapter location. Crucially, TAE was found at this location even if the eye movement had not yet occurred, and therefore the tester and the adapter were not yet retinotopically matched. Predictive remapping in humans was also demonstrated in a recent ERP study. Parks and Corballis (2008) showed that if a saccade carries a stimulus from one visual hemifield into the other, the corresponding interhemispheric shift of the stimulus’ neurophysiological correlate occurs well before the saccade.

Remapping is considered to be a strong candidate mechanism for explaining why visual perception, or at least conscious experience thereof, is left largely undisturbed by eye movements. The intuitive notion that the brain contains a representation of the world in purely spatiotopic (world centered) coordinates has received some support (e.g., d’Avossa et al. 2007; Galletti et al. 1995), but is no longer favored as the complete solution to the problem of visual stability (Colby and Goldberg 1999; Wurtz 2008). Instead, it is believed that visual information is represented largely in retinotopic (eye centered) maps and is constantly remapped within those maps to compensate for saccade-induced retinal changes. Frequently, remapping occurs before the onset of a saccade in which case it is called predictive. One way of describing this process is that neurons receive a “status report” on the location that will be brought into their RF by an impending saccade. This “status report” allows visual information to be preserved across saccades. The important role that remapping plays in visual stability is illustrated by a number of studies investigating remapping in human participants using functional magnetic resonance imaging (Medendorp et al. 2003; Merriam et al. 2003; 2007). In these studies, a visual stimulus was presented and subsequently extinguished, leaving a memory trace (residual neural activity). Next, a saccade was executed such that the former stimulus location was carried across the vertical meridian into the opposite visual field. The key finding in these studies is that, with the eye movement, the neural correlate of the memory trace shifts to the opposite hemisphere as well. Therefore, although the memory trace is represented in retinotopic coordinates, an eye movement does not cause the memory trace to be misaligned with the world. Rather, the memory trace is remapped onto a different set of neurons, such that the memory trace remains tied to the correct spatial location. Without the need for a spatiotopic representation of the world, this explains why we are able to integrate information presented at the same location before and after an eye movement (Ezzati et al. 2008; Melcher and Morrone 2003; Wittenberg et al. 2008). Possibly, also the finding that, in some cases, people are very limited in their ability to integrate visual information across saccades (e.g., Irwin 1991) can be explained in terms of remapping, or a lack thereof.

In a recent study, Golomb et al. (2008) investigated how the locus of attention is affected by an eye movement. They aimed to answer the question what happens when attention is endogenously directed to a location, and subsequently the eyes move elsewhere while the attended location remains relevant to the task. While fixating, participants received a spatial cue and were instructed to hold the cued location in memory. It was assumed that keeping a location in memory requires the deployment of spatial attention to the memorized location (Awh and Jonides 2001). After saccade execution, Golomb et al. measured reaction times (RTs) to probes presented at the memorized (spatiotopic) location and the location which retinotopically matched the memorized location. RT facilitation was found at both locations. This suggests that the locus of attention is partly, but not entirely, independent of eye position. More specifically, this suggests that the neural correlate of visual attention was partly remapped to compensate for the eye movement.

The present study investigated remapping of visual attention. We used the presentation of an irrelevant onset to manipulate attention exogenously (see, e.g., Theeuwes 1991; Yantis and Jonides 1984). Experiment 1 was designed to determine whether exogenous attention is partly, but not completely, remapped as Golomb et al. reported in the case of endogenous attention. If so, we would expect attentional facilitation at the attended (spatiotopic) location as well as the location matching the attended location retinotopically. In Experiments 2 and 3, we tested whether remapping was predictive such that it would affect the allocation of attention prior to saccade execution. If so, we would expect attentional facilitation at the retinotopic location (now actually the “future retinotopic” location) after the eye movement has been cued, but before it has been executed.

Experiment 1

The paradigm was modeled after Golomb et al. (2008). Participants had to execute a saccade. Simultaneously with the presentation of the saccade goal an irrelevant abrupt onset was presented, which is known to summon attention. After the execution of the saccade, a probe stimulus (a titled line segment) appeared at one of four locations (the spatiotopic, retinotopic and two control locations). Participants made a speeded keypress response to indicate the orientation of the probe.

Method

Eighteen naive observers participated in the experiment. Eye movements were recorded using an Eyelink II (SR research). Each trial started with the presentation of a gray fixation dot on a black display at one of four possible locations (Fig. 1a and b). After 500 ms, three additional and identical dots were presented, forming the corners of a 9.0° × 9.0° square. After another 500 ms, the fixation dot reduced in size and one of the adjacent dots turned green, indicating that a saccade had to be made to that location. Participants did not know in advance to which location they had to execute a saccade. At the same time the dot turned green (the saccade cue), an onset stimulus (a 1.8° × 1.8° square) was presented for 100 ms at one of two (given a certain fixation point and saccade cue) possible locations 6.4° from the initial fixation dot and the saccade cue. Participants were instructed to make a saccade to the green dot as quickly as possible. The saccade cue and the onset were presented simultaneously, because a delay between the onset and the saccade cue may lead to inhibition of the onset. Thirty milliseconds after the initiation of the saccade while the eyes were in motion a tilted gray line segment (the probe) was presented for 100 ms. We choose to present the probe in mid-flight (during saccadic suppression) rather than after the saccade to prevent the probe from capturing attention exogenously. The probe was presented sufficiently long for participants to observe it after they had re-fixated. The probe was presented equi-probable at one of four locations. The probe could be presented at the location that previously contained the onset (the actual spatiotopic location), at a location which retinotopically matched the onset location (the actual retinotopic location) or at one of two “Mirror” control locations. Participants made a speeded report of the probe orientation by pressing the “z”-key on a leftwards tilted line segment (\) and the “/”-key on a rightwards tilted line segment (/). The experiment consisted of 48 practice trials, followed by 256 experimental trials.

Fig. 1
figure 1

a, b Two schematic example trials of Experiment 1. The saccade goal is denoted by the open circle. a An example of an actual retinotopic trial. b An example of an actual spatiotopic trial. The gray box contains examples of probe positions in different conditions, in trials in which the onset was presented at the center location. c A schematic example trial of Experiment 3 in the actual future retinotopic condition. In this example, the probe and the onset are presented in different visual quadrants, in this case on opposite sides of the horizontal meridian, but it could be on opposite sides of the vertical meridian as well. The gray box contains example stimulus configurations for actual and mirror future retinotopic trials. The probe and the onset could be presented in the same or in different visual quadrants; analysis revealed that there was no effect of visual quadrant (see “Results” of Experiment 3). Visual quadrants are marked by shades of gray for convenience. Actual and mirror spatiotopic trials were included in Experiment 3 as well, but they are not depicted here

Full size image

Results

Trials were discarded using the following criteria: gaze deviated more than 2° from the fixation point prior to the saccade cue (8.2%); the direction of the saccade deviated more than 22.5° from the straight line between the initial fixation point and the saccade goal (8.4%); saccade latency was either below 100 ms or above 600 ms (1.4%); RT was below 200 ms or above 1,000 ms (2.3%). One participant was replaced due to loss of fixation (50% of the trials), one participant due to overly high saccade latencies (M = 449 ms) and two participants due to a high proportion of misdirected saccade (25 and 34%, respectively). In total, 79.7% of the trials were included in the analysis.

A repeated measure analysis of variance (ANOVA), with condition (spatiotopic or retinotopic) and Location (actual or mirror) as within-subject factors and mean correct RT as a dependent variable, revealed a main effect of Location (F = 11.5, p < .01; see Fig. 2). No other effects were found (all F < 1). Planned comparisons revealed facilitation at the actual, compared to the mirror spatiotopic location (t = 3.1, p < 0.01), and facilitation at the actual, compared to the mirror retinotopic location (t = 2.2, p < 0.05). The same analysis with accuracy as dependent variable revealed a main effect of Location (F = 5.2, p < 0.05; actual more accurate than mirror), a marginally significant effect of Condition (F = 4.5, p < 0.1; spatiotopic more accurate than retinotopic) and no interaction between Location and Condition (F < 1). The average saccade latency was 270 ms (SE = 11.6).

Fig. 2
figure 2

Results of Experiments 1, 2 and 3. RTs are faster at the actual than at the mirror locations, indicating facilitation in both the (future) retinotopic and the spatiotopic condition. Error bars denote the 95% within-subject confidence interval (Cousineau 2005)

Full size image

Discussion

The results indicate that, if attention is captured by an abrupt onset before a saccade is executed, immediately following the saccade attention resides at two locations: the original attended location and a second location which corresponds retinotopically to this location. These results resemble those of Golomb et al. (2008), showing partial remapping of visual attention, and extend these findings to exogenous attention.

Experiment 2

The aim of the second experiment was to investigate whether remapping of visual attention is predictive. In other words, we wanted to determine whether the focus of attention would shift slightly before the saccade was executed. The critical difference with Experiment 1 was that we presented the probe just before the eye movement.

Method

Twenty naive observers participated in the experiment. The method was the same as that of Experiment 1 except for the following differences. Eye movements were recorded using an Eyelink 1000. The probe was presented at a fixed interval of 125 ms after the presentation of the saccade cue. The onset was presented for 50 ms and the probe was presented for 75 ms, to assure that on most trials all stimuli were presented before the saccade was initiated. The retinotopic location is now referred to as the “future retinotopic location”.

Results

Trials were filtered using the same criteria as in Experiment 1: gaze deviation (11.0%), saccade direction (14.0%), saccade latency (1.3%) and RT (2.0%). In addition, trials in which the eyes arrived at the saccade target before the offset of the line segment were discarded (7.9%). One participant was replaced due to anticipatory eye movements on 28% of the trials. In total, 63.9% of the trials were included in the analysis.

A repeated measures ANOVA, using Condition (spatiotopic or future retinotopic) and Location (actual or mirror) as within-subject factors and mean correct RT as a dependent variable, revealed a main effect of Location (F = 8.0, p < 0.05; see Fig. 2). No other effects were found (all F < 1). Planned comparisons revealed facilitation at the actual, compared to the mirror spatiotopic location (t = 2.2, p < 0.05), and facilitation at the actual, compared to the mirror future retinotopic location (t = 2.5, p < 0.05). The same analysis with accuracy as dependent variable revealed a main effect of condition (F = 10.5, p < 0.01; spatiotopic more accurate than future retinotopic). No other main effects were found. The average saccade latency was 269 ms (SE = 13.7).

Discussion

The results indicate that predictive remapping affects the locus of attention in the interval preceding a saccade. Before the saccade was executed, attentional facilitation was observed at the “future retinotopic location”: the location which retinotopically matched the attended location after the eye movement.

Experiment 3

Even though Experiment 2 provides clear evidence for predictive remapping, there is one caveat. In Experiment 2, the Actual and Mirror locations were always presented in opposite visual fields, separated by the horizontal or the vertical meridian. Therefore, the future retinotopic facilitation could have been due to a spreading of attention from the onset location to other areas within the same visual field quadrant (e.g., Rizzolatti et al. 1987). Since spreading of attention across the horizontal or vertical meridian is less pronounced, one may obtain RT differences that have nothing to do with predictive remapping. To address this issue, we adapted the paradigm to allow saccades in every direction. Therefore, the future retinotopic location did not always fall in the same visual quadrant as the onset (see Fig. 1c).

Method

Nine naive observers participated in the experiment. The method was similar to that of Experiment 2 except for the following differences. The initial fixation dot was always presented at the center of the display. There were two potential saccade targets, 7.2° from and on opposite sides of the fixation point (Fig. 1c). Given these constraints, the position of the potential saccade targets was random. All stimuli were scaled to 80% of their original size. The presentation duration of the probe was reduced to 50 ms, to reduce the number of trials in which it was still visible upon arrival at the saccade target. There were 384 experimental trials.

Results

Using the same criteria as in Experiment 2, trials were filtered on gaze deviation (9.7%), saccade direction (13.3%), saccade latency (1.9%), RT (3.2%) and premature arrival at the saccade target (0%). In total, 71.8% of the trials were included in the analysis.

We performed the same analysis as in Experiment 2 and obtained qualitatively identical results: RT data revealed a main effect of location (F = 13.3, p < 0.01; see Fig. 2) and facilitation at both the spatiotopic (t = 4.0, p < 0.01) and the future retinotopic location (t = 2.5, p < 0.05). Accuracy data revealed a marginally significant effect of Condition (F = 4.8, p < 0.1). The average saccade latency was 250 ms (SE = 17.1).

We performed an additional analysis on trials in the future retinotopic condition and included as a factor whether the onset and the probe were presented in the same or in different visual quadrants (see Fig. 1c). A repeated measures ANOVA using location and quadrant as within-subject factors and mean correct RT as dependent variable revealed a main effect of Location (F = 5.8, p < 0.05), but no other effects (all F < 1), indicating that there was no meridian effect (same quadrant actual, M = 588 ms; mirror, M = 609 ms; different quadrant actual, M = 591 ms; mirror M = 603 ms).

Discussion

In Experiment 3, we confirmed that predictive remapping affects the locus of attention in the interval preceding a saccade. In addition, we ruled out a meridian effect as an alternative explanation.

General discussion

The present study clearly shows that predictive remapping affects the allocation of attention prior to an eye movement The presentation of a brief onset attracted attention, resulting in a temporary increase in baseline activity of neurons whose RFs overlap the attended location (for a review, see Reynolds and Chelazzi 2004). When during this short interval a probe is presented at the attended location, this allows for a faster and more accurate report of the probe identity. Most of the visual system is retinotopically organized. In Experiment 1, this resulted in attentional facilitation at the location which retinotopically matched the onset. However, neural responses are often remapped to compensate for eye movements. Similar to the remapping of a memory trace (e.g., Merriam et al. 2003), the activation elicited by the onset was transferred to a different population of neurons, which after the saccade had RFs encompassing the original onset location. This resulted in attentional facilitation at the original onset location. Similar findings showing both retinotopic and spatiotopic integration across saccades have been reported for a variety of phenomena (Ezzati et al. 2008; Golomb et al. 2008; Melcher and Morrone 2003). Crucially, in Experiments 2 and 3, we showed that the locus of attention partly shifts in the direction of the saccade, prior to the eye movement. This can be explained by assuming that the neurons which have been activated by the presentation of the onset remap predictively: they exhibit an anticipatory RF shift in the direction of the saccade. Therefore, they will respond to the presentation of the probe at the future retinotopic location, allowing the probe to “ride the wave” of the onset.

Remapping is believed to be crucial in maintaining visual stability (Wurtz 2008). Most neurons have RFs which are anchored to the retina. Therefore, after an eye movement, they are exposed to a different part of the visual scene. However, rather than perception starting anew after every eye movement, it seems that neurons receive a “status report” on the location that will be brought into their RF by the eye movement. Frequently, this process starts in the interval preceding saccade execution, in which case it is referred to as predictive (Duhamel et al. 1992; Melcher 2007). Here, we report that exogenous visual attention is remapped, but only partly. Therefore, after an eye movement attention is allocated at two locations: the original locus of attention and the location which retinotopically matches the original locus of attention. Importantly, we also show that predictive remapping causes the locus of attention to partly shift in the direction of an eye movement prior to saccade execution.