Introduction

Immediately following an uninformative peripheral cue, responses to targets at the cued location were facilitated. When the target appeared approximately 300 ms after the cue, this facilitatory effect turned into an inhibitory effect, which delayed responses to targets at the cued location (Posner & Cohen, 1984). This latter inhibitory effect is believed to discourage attention from returning and thus was given the name “inhibition of return” [IOR] (Posner, Rafal, Choate, & Vaughan, 1985). Extending this functional explanation of IOR, Klein (1988; Klein & Macinnes, 1999) suggested that IOR discourages orienting toward previously inspected objects and locations, and thus facilitates visual foraging (Wang & Klein, 2010, for review).

To facilitate real-world visual foraging, which usually involves saccadic eye movements, IOR would need to operate in environment-centered coordinates (Posner & Cohen, 1984). The model task for exploring this possibility, devised by Maylor and Hockey (1985), was similar to that illustrated in Fig. 1a. The participants shifted their gaze following the offset of a peripheral cue. This gaze shift dissociated the environmental coordinates and the gaze-centered, or retinotopic, coordinates. IOR effect was “found only for targets that share environmental location with the cue” (p. 784), suggesting that IOR is mapped in environmental coordinates. With similar tasks, more recent studies have observed IOR effects in both environmental and retinotopic coordinates.Footnote 1 These IOR effects were obtained with either manual button presses (Sapir, Hayes, Henik, Danziger, & Rafal, 2004; Satel, Wang, Hilchey, & Klein, 2012) or saccadic responses (Hilchey, Klein, Satel, & Wang, 2012; Krüger & Hunt, 2013; Pertzov, Zohary, & Avidan, 2010) and were not artifacts created by the spatial gradient of IOR (He, Ding, & Wang, 2015).

Fig. 1
figure 1

Experimental methods and results. a Sequence of events in a sample trial of the “saccade” task. The remapped location will be probed in this illustration, if the bar is presented before the saccade instructed by the central arrow. b Neurons on oculocentric brain maps predictively respond to objects that will be brought to their response field by an upcoming saccade. The proper location to probe this predictive activation before saccade, i.e., the remapped location, is marked with a grey disk. c IOR effects at the remapped location, the cued location, and the future retinal location in the saccade task. In the upper panels, the cued location is marked by a dashed circle. In the lower panels, IOR effects at corresponding locations in the no-saccade task are marked as dashed horizontal lines. Error bars represent ± 1 SEM. **p < 0.01

Full size image

The early visual cortices are largely organized as retinotopic maps (e.g., Gardner, Merriam, Movshon, & Heeger, 2008). The retinotopic IOR effect is likely the result of short-term depression of the early visual pathway caused by the cue (e.g., Satel, Wang, Trappenberg, & Klein, 2011). As for the environmental IOR, because it is observed immediately after saccades (He et al., 2015; Hilchey et al., 2012; Pertzov et al., 2010), predictive remapping might be the mechanism underlying it. Predictive remapping is a neural mechanism discovered by Duhamel, Colby, and Goldberg (1992). They showed that some neurons in the lateral intraparietal cortex (LIP) were activated by stimuli that would be brought to their receptive fields (RFs) by an impending saccade. Similar findings have been observed in the frontal eye fields [FEF] (Sommer & Wurtz, 2006; Umeno & Goldberg, 1997), the superior colliculus [SC] (Walker, Fitzgibbon, & Goldberg, 1995) and early visual cortices (Nakamura & Colby, 2002). These observations suggest that stable visual representations can be realized on retinotopic brain maps by transferring neuronal activation to compensate for the retinal shifts caused by saccades (e.g., Hall & Colby, 2011). Predictive remapping also has been demonstrated with well-established behavioral effects, such as visual masking (Hunt & Cavanagh, 2011) and crowding (Harrison, Retell, Remington, & Mattingley, 2013). Recent work even showed that it could be a mechanism underlying spatial attention (Jonikaitis, Szinte, Rolfs, & Cavanagh, 2013; Rolfs, Jonikaitis, Deubel, & Cavanagh, 2011).

The present experiment aims to clarify if predictive remapping could be the mechanism underlying environmental IOR. The task devised by Maylor and Hockey (1985) was adopted, with three major modifications. First, the target was presented before the onset of the required saccades on most trials. This manipulation is necessary to reveal the temporal dynamics of IOR before saccade onset. Second, in addition to the cued location and the location corresponding to the cued retinal locus following the required saccade, this experiment also probed a location relevant to predictive remapping (Rolfs et al., 2011). This remapped location was a shift of the cued location in the opposite direction of the required saccade (grey disk in Fig. 1b). If IOR is predictively remapped, IOR effects at the remapped location would increase as time approaches the onset of the required saccade. Third, the target was a briefly presented tilted bar that required a 2-alternative forced choice (2-AFC) discrimination response as in Harrison, Mattingley, & Remington (2012). We also included a control condition with exactly the same display setup but requiring no saccade to assess the possibility that any IOR effect at the remapped location was simply the spillover of environmental IOR.

Method

The research protocol reported here was approved by the institutional review board of Center for Cognition and Brain Disorders at Hangzhou Normal University. Informed consent was obtained from all participants.

Participants

Fourteen volunteers (10 females; mean age: 21.5 years), including the first author (CY), participated in this experiment. Except for CY, all participants were compensated for 40 yuan per hour. All participants had no psychological or neurological disorders, and reported normal or corrected-to-normal vision.

Apparatus

Participants were tested in a quiet and dimly lit laboratory. The experiment was programmed in Python and tested on a Windows PC. Visual stimuli were presented against a black ground (1.95 cd/m2) on a 21-inch CRT monitor. The visible area of the display measured 33° × 24.7° at a viewing distance of about 64 cm (maintained with a chin rest). Eye movements were recorded with an Eyelink 1000 (SR Research) eye-tracker, sampling at 500 Hz. The spatial resolution of the eye tracker was 0.2° or better. Saccade initiation was detected online, with a velocity threshold of 30°/s and an acceleration threshold of 8000°/s2.

Procedure and design

All participants completed two cueing tasks. They made a saccade following the presentation of the cue in one task, whereas they maintained fixation throughout a trial in the other. For convenience, we will refer to these two tasks as “saccade” and “no-saccade” tasks, respectively. During testing, the no-saccade task always followed the saccade task.

Saccade task

The sequence of events in a sample trial is illustrated in Fig. 1a. Drift correction was performed at the beginning of each trial, followed by the presentation of three grey dots (diameter: 0.5°; Weber contrast: 8.69). The side dots were potential saccade targets that were 8° away from the central dot. Participants fixated at the central dot for a random duration of 600 to 1000 ms. Then, a green disk (diameter: 0.5°; Weber contrast: 59.51), i.e., the cue, was presented for 100 ms, at a location 8° above or below the central dot. After an interstimulus interval (ISI) of 200-800 ms, the central dot turned into an arrow and the participants quickly saccaded to the peripheral dot indicated by this arrow. Regardless of the occurrence of this saccade, 1000 ms after the onset of the cue, a gray bar (0.2° × 1°; Weber contrast: 8.69) tilted 45° from the vertical direction was presented for 50 ms. Participants were instructed to report the orientation of bar by quickly pressing the ‘z’ (tilted left) or ‘/’ (tilted right) key on a standard QWERTY keyboard. The tilted bar (i.e., the target) could appear at the cued location, a location corresponds to the cued retinal locus after the required saccade (i.e., the future retinal location), a location on the same side as the cued location but was 8° away in the opposite direction of the required saccade (the remapped location), or one of three control locations in the field opposite to the cue (Fig. 1c, upper panels). Both speed and accuracy were explicitly emphasized in this task. After a random delay of 1000 to 1500 ms, the next trial began.

By design, each participant needed to complete 18 blocks of 48 trials, in a total of 3 sessions. A practice session of up to 30 minutes was provided to each participant until he could perform the task with little difficulty. To get as many useful trials as possible, a trial was recycled if: a) the gaze deviated more than 2° from the central dot before the saccade signal (arrow at fixation), b) no saccade was made to the saccade signal, c) the saccade missed the peripheral dot by more than 2°, d) button responses were detected before saccade onset, or e) the participant failed to response to the target within 1200 ms or they pressed the wrong key. As a result, the total number of trials tested varied from session to session, from participant to participant.

No-saccade task

The no-saccade task was carried out to assess the spillover effect of IOR in the absence of eye movements. The setup of this task was the same as the saccade task, except that the saccade signal (arrow at fixation) was never presented and participants maintained fixation for the full duration of the trial. The cue-target onset asynchrony (CTOA) was also 1000 ms, and both speed and accuracy were emphasized for the target response. Each participant completed 3 blocks of 72 trials in a single session. A practice block of 15 trials was provided before testing. During testing, trials were recycled if the gaze deviated more than 2° from the central dot, or the participant failed to respond to the target within 1200 ms or responded with the wrong key.

Results

Saccade task

A total of 22,381 trials were obtained for the saccade task. Those on which the fixation was broken before the saccade signal (2.07 %) or no saccade was made in response to the saccade signal (12.09 %) were first excluded. The RTs and error rates reported here were based on the remaining trials. An analysis of the saccade latencies is provided in Supplemental Material; we focus our analysis on the RTs to the target.

RTs

Predictive remapping occurs in a small time window before saccade onset. If predictive remapping is the mechanism behind environmental IOR, a reliable IOR effect should emerge at the remapped location shortly before the required saccade. Thus, our analysis will be focused on trials in which the target was presented before the onset of the required saccade (14,839 trials in total). To reveal the temporal dynamics of IOR, target onset times relative to the onset of the required saccade across all trials of all participants were first divided into three bins of equal number of trials. This binning method was adopted, because the resulting time bins roughly reflect the “absolute” time relative to saccade onset, as in previous work (e.g., Rolfs et al., 2011), while it also maintained roughly equivalent statistical power across bins. The time bins had on average 35 trials/subject in each experimental cell. Mean target onset times in these bins were −167 ms (SD = 25.79), −69 ms (SD = 3.23) and −24 ms (SD = 1.62).

For the RT analysis, those aborted during testing (33.4 %) or had RTs longer than 1000 ms (0.7 %) were discarded. Mean RTs of all time bins are presented in Table 1. As illustrated in Fig. 1c (top panels), IOR at the cued, remapped, and future retinal locations was revealed by comparing to control locations in the field opposite to the cue. A repeated measures ANOVA was performed on the RTs, with variables cueing (target on the same or opposite side as the cue), condition (cued, remapped, vs. future retinal), and target onset time relative to saccade onset (3 bins). The results revealed significant main effects for cueing, F(1, 13) = 7.18, p = 0.018, η G 2 = 0.001, condition, F(2, 26) = 16.43, p < 0.001, η G 2 = 0.008, and target onset time, F(2, 26) = 129.23, p < 0.001, η G 2 = 0.168. The main effect of cueing occurred, because RTs were slightly longer for targets appeared on the same side as the cue, suggesting an overall IOR effect. The main effect of condition occurred because the longest RTs were observed for locations assessing IOR at the future retinal condition (485 ms), whereas slightly shorter RTs were observed for locations assessing IOR at the cued (477 ms) and remapped (474 ms) locations. The main effect of onset time occurred because RTs generally decreased as time approached saccade onset. The 2-way interactions were not significant, all Fs < 1.46, all p > 0.226; however, the 3-way interaction was statistically reliable, F(4, 52) = 2.54, p = 0.051, η G 2 = 0.002.

Table 1 Mean RTs and error rates at the cued location, the remapped location, the future retinal location, and corresponding control locations
Full size table

IOR effects, i.e., the RT differences between targets on the same and opposite side of the cue, are presented in Fig. 1c (lower panels). As is clear from this figure, the 3-way interaction emerged because IOR increased with time at the remapped location, whereas decreased with time at the cued location and the future retinal location. For the remapped location, a reliable IOR effect was observed in the −24 ms time bin, t(13) = 3.70, p = 0.003, but not in the other two time bins, all t < 1, all p > 0.411. This observation provides direct empirical evidence that predictive remapping gives rise to environmental IOR. For the cued location and future retinal location, IOR was not significant in any time bin, all t < 1.65, all p > 0.123.

The lack of IOR at the cued location was unexpected. Several recent studies have observed retinotopic IOR in tasks similar to that of Maylor and Hockey (1985). Thus, some (retinotopic) IOR effect should be observed at the cued location after environmental IOR had been remapped before saccade onset. Also, only 739 trials presented the target after the required saccades. This precluded performing any meaningful analysis to check whether there was environmental IOR following the required saccade. To address these two issues, another 12 participants (6 males; mean age: 21.83 years) were tested in the same experimental setup but responded to targets presented immediately after the termination of the required saccades (see Supplemental Material for a detailed presentation of this experiment). This experiment revealed 20 ms of environmental IOR at the cued location, t(11) = 2.44, p = 0.03, and no IOR at the cued retinal locus (1 ms), t(11) = 0.23, p = 0.82. These results are consistent with previous findings that retinotopic IOR is generally weaker than environmental IOR (He et al., 2015; Hilchey et al., 2012; Pertzov et al., 2010; Satel et al., 2012). The IOR at the far (remapped in the main experiment) location (−5 ms) was not significant, t(11) = 1.53, p = 0.15.

Errors

Trials with response errors, i.e., those on which the participant failed to respond to the target or made incorrect button responses, accounted for about 8 % of the 14,839 trials considered in the RT analysis. The error rates are presented in Table 2. An ANOVA on the error rates, with the same factors and target onset time bins used in the RT analysis, revealed significant main effects for condition, F(2, 26) = 12.41, p < 0.001, η G 2 = 0.048, and target onset time, F(2, 26) = 11.11, p < 0.001, η G 2 = 0.041. The main effect of cueing was not significant, F(1, 13) = 2.78, p = 0.119, η G 2 = 0.001, nor did its interactions with other factors, all Fs < 1.20, all p > 0.316.

Table 2 Mean RTs and error rates in the no-sacccade task
Full size table

No-saccade task

An ANOVA of the RTs, with variables cueing (same vs. opposite) and condition (cued and opposite locations vs. side locations) revealed significant main effects for cueing, F(1, 13) = 6.37, p = 0.025, η G 2 = 0.004, and condition, F(1, 13) = 6.48, p = 0.024, η G 2 = 0.003. The 2-way interaction did not reach significance, F < 1, p > 0.250. Planned contrasts revealed an IOR effect (9 ms) at the cued location, t(13) = 2.20, p = 0.047, whereas the IOR effect at side locations (4 ms) was not significant, t(13) = 1.29, p = 0.219. These results clearly showed that the IOR effect at the remapped location in the saccade task, i.e. side locations in the no-saccade task, was not the spillover of IOR at the cued location.

Analysis of the error rates revealed only a main effect for cueing, F(1, 13) = 6.12, p = 0.028, η G 2 = 0.086, with more errors for targets appeared on the same side as the cue. The main effect of target location and the 2-way interaction was not significant, all Fs < 1.

Discussion

Because environmental IOR effects are observed immediately following saccades (e.g., Hilchey et al., 2012), researchers have argued that environment-centered IOR effects are predictively remapped before eye movements (He et al., 2015). With a cueing task, we have revealed an IOR effect at a retinal locus that will be brought to the cued location by an impending saccade. This discovery, for the first time, establishes direct empirical evidence for this theoretical proposition.

An important characteristic of the predictive remapping of IOR we have discovered is that it occurs in a fairly small time window before saccade onset. This is consistent with previous neurophysiological and behavioral findings. Predictive neuronal discharges that have been observed in multiple brain regions generally occur within about 100 ms before saccade onset (e.g., Duhamel et al., 1992; Umeno & Goldberg, 1997; Walker et al., 1995). In the non-IOR behavioral studies mentioned in the introduction, robust effects also are consistently observed at the remapped location only within 100 ms before saccade onset (e.g., Harrison et al., 2013; Hunt & Cavanagh, 2011; Jonikaitis et al., 2013; Rolfs et al., 2011).

Although remapped neuronal responses have been shown in all regions that have been linked to IOR, available evidence seems to suggest that environmental IOR is most likely mapped in parietal regions. With a task similar to Maylor and Hockey (1985), Sapir et al. (2004) found that patients with right parietal lesions produced no environmental IOR. This finding was later confirmed by a virtual lesion study which abolished environmental IOR by applying TMS to the right intraparietal sulcus (van Koningsbruggen, Gabay, Sapir, Henik, & Rafal, 2009). In visual foraging tasks, IOR is manifested as delayed behavioral responses (e.g., Klein & Macinnes, 1999; Shariat Torbaghan, Yazdi, Mirpour, & Bisley, 2012) or reduced neuronal responses (Mirpour, Arcizet, Ong, & Bisley, 2009) to previous fixated objects (or locations). Mirpour and Bisley (2012) examined LIP neurons that had remapping responses in a foraging task and observed weaker remapping responses for previously fixated items.

In addition to space- and gaze-centered reference frames, IOR can also be mapped in object-based or scene-based representations (e.g., Gabay, Pertzov, Cohen, Avidan, & Henik, 2013; Tipper, Driver, & Weaver, 1991; but see Krüger & Hunt, 2013). While the present study demonstrates that predictive remapping underlies space-centered IOR effects, our findings do not rule out the possibility that IOR in object- or scene-based coordinates is supported by other mechanisms. For instance, Gabay et al. (2013) have recently shown that, in a cueing task, IOR moved with the background image. Because the participant maintained fixation all time, this scene-based IOR is likely realized through mechanisms other than predictive remapping.