Introduction

Saccades are rapid eye movements that shift the center of gaze to a new location in the visual field. Changes in visual perception occur around the time of saccades1,2. For example, our subjective experience of the visual scene remains stable across the abrupt changes of the retinal image during saccades. This phenomenon is called visual stability, and many studies have attempted to explain the mechanism behind it3. Several other perceptual phenomena which occur around the time of saccades have also been studied psychophysically. For example, there is a general reduction in visual sensitivity during saccades, a phenomenon called saccadic suppression or saccadic omission, that has been reported in both macaques and humans4,5,6,7. Saccadic eye movements also alter our perception of time8. Another phenomenon is perisaccadic mislocalization, in which the perceived location of a visual stimulus appearing near the time of a saccade is biased. Perisaccadic mislocalization was first discovered as a perisaccadic shift, a unidirectional mislocalization parallel to the saccade, when the experiments were done in darkness9,10,11,12. Later studies have demonstrated perisaccadic compression13,14, which is mislocalization towards the saccade target (ST), when the subjects make saccades with background illumination and visual references15,16,17,18.

Perisaccadic visual perception in macaques is qualitatively similar to humans4, and many studies have investigated the neurophysiology of perisaccadic visual perception in nonhuman primates19,20,21,22,23,24,25,26. Some neurons in the extrastriate visual areas and prefrontal cortex show a sensitivity shift to the postsaccadic receptive field (RF) even before the saccade, a phenomenon often referred to as future field remapping or forward remapping27,28. There is also another phenomenon, called ST remapping or convergent remapping, in which neural RFs shift towards the ST around the time of saccade29,30,31,32,33,34,35,36,37,38. Both future field and ST remapping can be observed in the same experiments in the same group of neurons39,40,41,42. It has been suggested that RF remapping is associated with perisaccadic mislocalization19,43,44, and some studies have used computational approaches to predict perisaccadic perception of space based on neural responses44,45,46,47. Although these studies have generated insightful experiments, theories, and hypotheses, they usually start with assumptions about the function of visual areas or have a limited precision in accounting for the time-varying relationship between neural modulations and perceptual alterations on the millisecond timescale of saccades. By quantifying the statistical dependencies of spiking responses on several behavioral (e.g., eye movement) or external (e.g., visual stimuli) variables, point process statistical models provide a powerful means to capture the encoding and decoding of visual information as continuously varying with eye movements, with no assumption about the function of neurons. To investigate the neural basis of perisaccadic mislocalization, we used a time-varying generalized linear model (GLM) framework capable of capturing the fast spatiotemporal dynamics of neural sensitivity around the time of saccades42,48, and examined the link between perisaccadic visual responses and visuospatial perception.

In this study, we first measured perisaccadic mislocalization behaviorally and found significant mislocalization against the saccade direction for probes appearing near the ST perisaccadically. To investigate the possible neural correlates of the observed mislocalization, we used a combined experimental and computational approach built upon neuronal responses in the middle temporal (MT) cortex and area V4 of rhesus macaque monkeys. We assessed each neuron’s sensitivity to each location of visual space across time relative to the saccade (the neuron’s kernels) using a statistical model fitted on the recorded spiking data during a visually guided saccade task with visual stimulation. We quantified the representational spatial bias using the spatiotemporal kernels of populations of neurons, based on the similarity in neural sensitivity to neighboring probe locations, without assumptions about the downstream readout mechanisms. We then used this measure of spatial bias to identify the perisaccadic changes in sensitivity that drive it and linked them to neural responses.

We found that neurons with RFs close to the ST do not contribute to spatial bias. In contrast, perisaccadic spatial bias in the direction opposite to the saccade vector can be accounted for by neurons with RFs farther from the ST. These neurons show perisaccadic and postsaccadic sensitivity changes near the ST that contribute to spatial bias. Unexpectedly, we found that the time course and response components of the spatial bias match that of another perisaccadic perceptual phenomenon, namely the enhancement of neural sensitivity around the ST. This representational ST enhancement could be related to the presaccadically enhanced ST perception reported in psychophysical studies35,37,49 and to the presaccadic increase in stimulus selectivity24,26,50,51 or ST remapping38,39 observed in neurophysiological studies. The shared neural response components underlying the ST representational enhancement and spatial bias suggest that the brain potentially prioritizes ST representation with consequent biases in location perception.

Taken together, our findings highlight a potential neural basis for perisaccadic mislocalization, supporting a role for extrastriate neurons in the perception of stimulus location and linking enhanced sensitivity near the ST to perisaccadic spatial bias.

Results

We designed a behavioral paradigm called the Perisaccadic Localization Task to measure perisaccadic mislocalization. Two monkeys performed the task while their eye movements were monitored with a high-resolution eye-tracking system (see “Methods” section). During the task, the monkey made a first saccade from the fixation point (FP) to the ST. A probe stimulus appeared for 50 ms at one out of nine possible locations at a random time during fixation or around the time of the first saccade (Fig. 1a). After landing on the ST, the monkey then made a saccade to the probe location, and the endpoint of that saccade was taken as the reported location. Mislocalization was defined as the horizontal distance between the reported location of each stimulus when presented around the time of the first saccade compared to when presented during fixation (150:250 ms before the onset of the first saccade). The average time course of mislocalization for probes near the ST shows mislocalization against the saccade direction for probes presented perisaccadically (Fig. 1b). We also measured the localization error: the distance between the reported location of each stimulus from the actual location of the probe presented. The horizontal localization error for probes presented perisaccadically (−100:0 ms) was significantly more negative—i.e., larger error opposite the saccade direction—compared to those presented during fixation for both monkeys (Monkey 1: p = 2.57e−14, Monkey 2: p = 0.01) (Fig. 1c). We also divided the trials by the median reaction time of the monkey in each session and found no difference between the horizontal localization errors for perisaccadic probes based on reaction time (p = 0.10; Supplementary Fig. 1a); thus, mislocalization appears to be related to saccade execution rather than factors such as arousal or motor planning. Averaging across the localization errors for probes presented around the ST in each session, we observed significant horizontal localization error for perisaccadic probes compared to fixation (p = 8.22e−05; Supplementary Fig. 1b), but no significant vertical localization error (p = 0.48; Supplementary Fig. 1c).

Fig. 1: Behavioral paradigm and results for perisaccadic mislocalization.
figure 1

a Perisaccadic Localization Task. In each trial, the monkey makes a first saccade from an FP to a peripheral ST, either 10 dva left or 10 dva right from the FP horizontally. At a random time during fixation and saccade execution, a 50-ms visual probe stimulus is presented in one of nine possible locations in a 3×3 grid. When the ST disappears, the monkey makes a second saccade to the remembered location of the probe stimulus. If the monkey reports within a window of the correct location, feedback will be given by a reappearing flash of the stimulus. b Mislocalization is measured as the horizontal distance between the reported location of each stimulus when presented perisaccadically compared to when presented during fixation (−250:−150 ms). Plot shows mean mislocalization and the shaded area represents the standard error of the mean (SEM) over the probes within 5 dva from the ST, for Monkey 1 (top) and Monkey 2 (bottom). Each point shows the mean for probes appearing within ±25 ms of the plotted time value; the number of probes included varies for different timepoints. Positive values indicate mislocalization in the saccade direction, and negative values reflect mislocalization opposite to the saccade direction. Gray bar (−100:0 ms) shows time window used for c. c Horizontal localization error for probes presented in fixation vs. perisaccadic windows. The horizontal localization error is measured as the horizontal distance between the reported location of each stimulus from the actual location of the probe presented. Plots show horizontal localization error for probes appearing perisaccadically (y-axis; −100:0 ms) vs. during fixation (x-axis), for Monkey 1 (top; p = 2.57e−14) and Monkey 2 (bottom; p = 0.01). p-values were computed by one-sided Wilcoxon signed-rank test. Histograms in upper right show the distribution of differences. Source data are provided as a Source Data file.

Full size image

To examine the neural basis of perisaccadic spatial biases in perception, we recorded the responses of extrastriate neurons (see ”Methods” section). We analyzed the activity of 300 neurons from MT and 147 neurons from area V4, recorded while monkeys performed a visually-guided saccade task (Fig. 2a). In this task, the trial began when the monkey fixated on a central FP, upon which an ST appeared 13 degrees of visual angle (dva) away either to the left or the right horizontally, while the monkey held the fixation. When the FP disappeared, the monkey made a saccade to the ST and maintained fixation on the ST during the second fixation period. A series of probe stimuli were presented throughout the task while the monkey fixated and made a saccade. Only one stimulus was presented at a time, selected from a 9×9 grid of possible locations, and each stimulus appeared for 7 ms. The probe grid position and spacing were adjusted to cover the FP, ST, and estimated RF of the neuron. In order to computationally investigate the mechanism of spatial bias, we developed an encoding model that quantitatively characterizes the neuron’s input-output relationship and captures the neuron’s sensitivity map with high temporal precision throughout the eye movement task (see “Methods” section and ref. 48 for details). The model traces the time-varying dynamics of a neuron’s sensitivity across saccades with high-dimensional spatiotemporal kernels. First, we discretized the space into 81 locations in a two-dimensional plane (representing the 81 probe locations) and discretized all times of the neural response relative to saccade onset and delays (times of the stimulus relative to each response time) into 7-ms bins, to form spatiotemporal units (STUs) in a 4-dimensional space (Fig. 2b). These STUs were used to construct the spatiotemporal kernels of each neuron, which represent the neuron’s sensitivity across locations, times, and delays. The neuron’s spatiotemporal kernels were constituted by a weighted linear combination of STUs, where the weight by which each STU contributed to the kernel was estimated using a model that could capture the neuron’s spiking response. Figure 2c shows the STU weights at an example probe location around the RF of an example neuron across time and delay. When responding to a probe stimulus at that location, the STUs form kernels that represent how a neuron’s sensitivity changes across time from saccade onset and across delays. We used the Sparse Variable Generalized Linear Model (SVGLM)48 to estimate the STU weights and the resulting kernels by fitting the model to the neuron’s spiking responses (see “Methods” section and Supplementary Fig. 2). A signal representing the stimulus time and location passes through spatiotemporal kernels (representative of the neuron’s time-varying spatiotemporal sensitivity) and is added to the time-varying baseline neural activity relative to saccade onset (captured by an offset kernel) and the feedback signal generated using a post-spike filter (representing the effects of spiking history). The combined signal is then passed through a sigmoidal nonlinearity capturing the spike generation. The resulting firing rates are used to generate spikes with the Poisson spike generator. These spikes are then fed back to the circuit through the abovementioned post-spike history (Supplementary Fig. 2a). All the model components are learned via an optimization process to directly estimate the recorded spiking activity (see ref. 48 for details). The kernels estimated from the model reflect the time-varying temporal sensitivity of the neuron across space (Fig. 2c).

Fig. 2: Experimental and computational paradigm for measuring spatial bias.
figure 2

a Schematic of the visually-guided saccade task with probes. Monkeys fixate on a central FP, then a ST point appears in either horizontal direction. After a randomized time-interval (700:1100 ms), the FP disappears, cueing the monkeys to saccade to the ST. Throughout the task, a series of pseudorandomly located probes appear in a 9×9 grid of possible locations (white squares). Only one probe is on the screen at each time, for 7 ms. Neurons were recorded from the middle temporal (MT) area or area V4. b Composition of the neuron’s sensitivity map using spatiotemporal units (STUs) defined across locations, and time and delay bins of 7-ms. The inset provides a visualization of an STU, which represents one combination of location (x and y value), delay bin, and time bin relative to saccade onset. c Top image shows the model-estimated weights of STUs corresponding to an example location across times and delays for a sample neuron. Time refers to the time of response relative to saccade onset (−540:540 ms), and delay refers to the time of stimulus relative to particular response time (0:200 ms before), discretized in bins of 7 ms. Bottom cubes at specified time points denote the spatiotemporal kernels obtained by the weighted combination of STUs. The line plot within each cube shows a kernel across its delay dimension only. d Each layer represents the kernel map of one neuron along the time and delay dimensions for 9 probe locations around the neuron’s RF during the initial fixation. e Scatter plots show the kernel weights for the center probe closest to the ST vs. those for the eight surrounding locations, for each neuron in an example ensemble (n = 53 neurons), for a particular time and delay combination (time = 100 ms and delay = 110 ms). Eight correlation vectors can be computed, using correlation strengths as magnitudes and the relative probe positions as directions (brown arrows in center panel). The eight vectors are averaged to a single vector (purple arrow in center panel).

Full size image

Next, we developed a procedure to measure spatial bias based on the neurons’ estimated kernels. We made the assumption that similar neural sensitivity to probes appearing at neighboring locations could create uncertainty in a readout of the stimulus location by a downstream area, which would lead to a bias in spatial perception. In other words, if during a saccade the population response to one probe becomes similar to that of a neighboring probe, we can assume a representational bias toward that neighboring location, without specifically modeling downstream readout mechanisms. To examine the neural basis of spatial bias, we analyzed the similarity between the spatiotemporal kernels at pairs of probe locations in a population of neurons. For the sensitivity analysis at the population level, we divided the neurons into ensembles based on their RF locations. Neurons recorded with the same RF, ST, and grid arrangements were grouped as an ensemble, and each ensemble had a minimum of 10 neurons. Figure 2d shows the kernel maps at 9 probe locations around the RF for an example ensemble of neurons. For each ensemble, we measured the similarity between the neural sensitivity at neighboring locations by assessing correlations between the kernel weights at the center probe and the kernel weights at each of its eight neighboring probes using a cosine similarity measure. The cosine similarity was measured at each time and delay using the corresponding kernel weights. For the rest of the paper, correlation always refers to cosine similarity between the kernel weights for neighboring probes across neurons in an ensemble. In this study, we focused on examining the spatial bias around the ST because prior psychophysics often reported perisaccadic mislocalization close to the ST15,16. We defined a spatial bias measure based on relative ensemble sensitivity to probes near the ST. Figure 2e shows the correlations of kernel weights between an example probe close to the ST and its 8 neighboring probes, for an example ensemble of 53 neurons at time 100 ms and delay 110 ms. The correlation coefficient between the central probe and its neighboring probes (central polar plot) indicates the similarity of neural sensitivity in each direction at that central probe location (Fig. 2e brown arrows). We then averaged over the eight vectors at each probe location to get one vector (Fig. 2e purple arrow). Since the saccade direction was either to the left or the right horizontally, we focused on the horizontal projections of the average vectors, which we defined as the spatial bias. In order to combine data from sessions with leftward and rightward saccades, values were normalized according to saccade direction so that positive always means the same direction as the saccade, and negative means the opposite direction from the saccade. This spatial bias measurement allowed us to predict potential mislocalization of stimuli based on the kernels of the SVGLM fit to neural data.

Next, we examined how spatial bias changed over time relative to the saccade and the stimulus onset. Each kernel map has its own time and delay dimensions (e.g. Fig. 2d), so we measured spatial bias maps across time and delay for each of the 7×7 non-edge probe locations for each ensemble. Figure 3a shows the spatial bias over delay, at response time 100 ms, at a probe location close to the ST for an example ensemble, and Fig. 3b shows the spatial bias over time at delay 110 ms for the same ensemble and probe location. In this study, we focused on the probe locations around the ST. For each ensemble, we selected 6 locations around the ST and averaged their bias maps. We then averaged the bias maps for 15 ensembles (447 neurons) (Fig. 3c), where the individual maps were first normalized so that each ensemble had bias values ranging from −1 to 1 (see “Methods” section). Taking the mean over delays of 50:100 ms, we observed a significant negative bias (which means a representational shift in the direction opposite to the saccade direction) of −0.14 ± 0.06 around the ST for responses ~50:150 ms after saccade onset compared to during fixation (Fig. 3d and Supplementary Table 1). We also measured the vertical projection of spatial bias and found no difference between fixation and perisaccadic windows (Supplementary Fig. 3a); all subsequent analysis is therefore restricted to horizontal bias. A second method of measuring spatial bias based on optimal decoding also predicted perisaccadic mislocalization in the direction opposite the saccade (Supplementary Fig. 4). Altogether, the neural results were consistent with the behavioral results (Fig. 1). To find out which population of neurons contributes to perisaccadic mislocalization, we grouped ensembles of neurons based on d, the distance between the RF center and the ST (Fig. 3e). Ensembles with d < 11 dva show very little bias (0.06 ± 0.12). To examine the variability of neurons within each ensemble and its possible effect on the amount of bias, we resampled 90% of the neurons in each ensemble to compute 100 samples of spatial bias for each ensemble. The mean spatial bias in the perisaccadic response window of 50:150 ms demonstrates that most of the ensembles with RFs closer to the ST show less spatial bias, and the standard error of the mean shows that the phenomenon within each ensemble is consistent (Fig. 3f). Thus, our population of neurons demonstrate perisaccadic spatial bias opposite to the saccade direction, primarily driven by neurons with RFs far from the ST.

Fig. 3: Quantifying the spatial bias and its dynamics over time, delay, and ensembles.
figure 3

a The spatial bias as a function of stimulus delay values, for a probe that appears at a location close to the ST, measured using the neurons’ sensitivities at time 100 ms after saccade onset in an example ensemble. b The spatial bias as a function of time relative to saccade onset, for the same probe and ensemble in a, measured using the neurons’ sensitivities at delay 110 ms relative to each timepoint (x-axis). c Mean spatial bias across time and delay, for 6 probe locations around the ST, averaged across all 15 ensembles (n = 447 neurons). Dashed lines indicate the range of delay values used in d. d Mean spatial bias over time, for delay 50:100 ms, for 6 probe locations around the ST, for all 15 ensembles. Shaded area represents the standard error of the mean (SEM) across ensembles. e Spatial bias over time from saccade onset, for ensembles with various distances between their neurons’ RF center and the ST (d). There are 4 ensembles with d < 11, 4 ensembles with 11 ≤ d < 14, 3 ensembles with 14 ≤ d < 17, and 4 ensembles with d ≥ 17. Shaded area represents SEM across ensembles. f Spatial bias for the 15 ensembles during the perisaccadic window (50:150 ms from saccade onset, gray bar in e), plotted against the distance between RF center and the ST for each ensemble. Error bars indicate the SEM of the bias estimate over resampling the neuronal population in each ensemble (n = 100 samples, 90% of neurons in each sample). Source data are provided as a Source Data file.

Full size image

The above results show that the perisaccadic changes in the spatiotemporal sensitivity of MT and V4 neurons could account for changes in spatial perception during eye movements, but so far, we have focused on the representation at the population level and model-based neural sensitivity measurements. In order to find out which components of the neuronal response of which neurons account for the perisaccadic alteration in the readout of location, we used an unsupervised approach to search for response components that are specifically related to spatial bias. In this study, spatial bias was defined based on similarity in the population representation of neighboring probe stimuli, as captured by the neurons’ spatiotemporal kernels; since the kernels are comprised of STUs, manipulation of certain STU weights can change the kernels and thereby affect the similarity between the population sensitivity at neighboring locations. This assumption-free alteration in the model enables us to determine which STUs are necessary for creating perisaccadic spatial bias. Based on this rationale, we defined a bias index according to the difference between the center kernel and each neighboring kernel across times and delays. Nulling each modulated STU one by one, we quantified their effect on the kernel similarity using this bias index, and systematically identified the bias-relevant STUs (see “Methods” section; Supplementary Fig. 5). Using this unbiased search in the space of STUs, we found different phenomena for ensembles with different distances between the RF center and the ST (d), so we divided the ensembles into two groups (d < 11 dva and d ≥ 11 dva) to examine their bias-relevant STUs separately (Fig. 4a). For ensembles with d < 11 dva, there was a set of bias-relevant STUs around response time 60:90 ms and delay 80:110 ms. For ensembles with d ≥ 11, there are two areas of bias-relevant STUs—one around response time 60:100 ms and delay 60:110 ms, and the other one around response time 110:280 ms and delay 50:100 ms. After removing all the identified bias-relevant STUs, we recomputed the spatial bias over time, and the previously observed bias around 50:150 ms after saccade onset is significantly reduced (Fig. 4b; −0.03 ± 0.04; p = 0.04), confirming that the identified set of STUs drive this bias. Thus, by leveraging the capabilities of the model to decompose the spatiotemporal sensitivity of individual neurons, we were able to identify the specific changes in neural sensitivity that contribute to perisaccadic spatial bias.

Fig. 4: Identifying and validating bias-relevant sensitivity components.
figure 4

a Prevalence of bias-relevant STUs, over delay and time from saccade, for ensembles with d < 11 dva (top, n = 4 ensembles) and the other with d ≥ 11 dva (bottom, n = 11 ensembles). The black contours show the outline of STUs above 60% of the maximum prevalence. b Mean spatial bias over time from saccade onset, in the full model (purple), and in the reduced model (pink) where all the bias-relevant STUs associated with the ST probes and their neighbor probes are removed. Plots show mean ± SEM across 15 ensembles confirming that the spatial bias is significantly reduced for the 50:150 ms time window after saccade onset (gray bar) (p = 0.04).

Full size image

To interpret how the saccade-related changes in STUs link neurophysiological activity to a biased readout of location information, we wanted to relate them back to the neural responses. The model allows us to generate responses to synthetic stimuli and compare the predicted neural response during fixation and perisaccadic windows. We first examined the model-predicted response for ensembles with d < 11 and transformed the time and delay of the bias-relevant STUs to a stimulus-aligned response (Fig. 5a). To investigate the neural response underlying the perisaccadic change in spatial bias, we looked at how neurons responded to probes on different sides of the ST. Data from ensembles recorded with leftward saccades have been flipped to be combined with those recorded with rightward saccades. Out of the six probes around the ST, we called the three probes closer to the FP the “near” probes and the other three probes the “far” probes (Fig. 5b). RFs of neurons in ensembles with d < 11 mostly cluster between FP and ST (see prevalence in Fig. 5b), resulting in the near probes falling close to the RFs. Based on Fig. 5a, we averaged the model response for near vs. far probes over fixation (−500:−100 ms) and perisaccadic (−20:10 ms) windows, and used the neurons’ responses from experimental recordings as validation (Fig. 5c). We specifically compared the responses in 60-ms windows of time from stimulus onset, around the peak of the fixation and perisaccadic responses (fixation: 50:110 ms, perisaccadic: 70:130 ms). During fixation, there is a greater model-predicted response to near probes vs. far probes (near = 1.05 ± 0.01, far = 1.01 ± 0.01; p = 1.25e−09; n = 94 neurons), consistent with the near probes being closer to the RF centers. There is an increase of model-predicted response perisaccadically for both near and far probes, but more of an increase for far probes, such that the perisaccadic responses end up being similar for near and far probes (near = 1.05 ± 0.02, far = 1.07 ± 0.02; p = 0.13; n = 94). The model closely predicts the response from actual neurons during both the fixation window (experimental values: near = 1.04 ± 0.01, far = 0.94 ± 0.01) and the perisaccadic window (experimental values: near = 1.20 ± 0.04, far = 1.20 ± 0.04). We measured the statistical difference between the actual firing rates of neurons in response to near vs. far probes (Fig. 5d), in 60-ms windows matched to their evoked responses. During the fixation period, from 50 to 110 ms after stimulus onset, the firing rate evoked by near probes is significantly higher than that for far probes (near = 38.57 ± 2.30 Hz, far = 35.21 ± 2.18 Hz; p = 5.38e−12). During the perisaccadic period, from 70 to 130 ms after stimulus onset, there is no statistically significant difference between the firing rates in response to near vs. far probes (near = 43.39 ± 2.63 Hz, far = 43.13 ± 2.57 Hz; p = 0.50). These neural responses show that neurons with RFs close to the ST responded more to near probes during fixation, but respond equally to both near and far probes around the time of saccades. The lack of difference in response indicates that there is no neural bias towards either side of the ST around the time of eye movements, which explains the absence of spatial bias for ensembles with d < 11.

Fig. 5: Identifying the neural correlates of spatial bias around the ST area from neurons whose RFs are located near the ST.
figure 5

a Bias-relevant STUs and model-predicted response, plotted as a function of time of stimulus from saccade onset (y-axis) and time of response from stimulus onset (x-axis). Shown for ensembles with d < 11 dva (n = 4 ensembles, 94 neurons). b Map of RF centers relative to the FP and ST using all ensembles used in a, and the probe locations around the ST. Prevalence of RF center (colorbar) indicates the percentage of neurons with RF centers in the corresponding location. “Near” probes are the three probes on the side of the ST towards the FP (orange), while the “far” probes are on the other side of the ST (green). c Mean of normalized model-predicted responses (left) and actual neural responses (right) over time from probe onset, for near (orange) and far (green) probes, during fixation (top; −500:−100 ms) and perisaccadic (bottom; −20:10 ms) windows. Mean±SEM across models or neurons in the ensembles; gray bars show analysis windows used in d. d Comparison of actual neural responses to near vs. far probes (n = 94 neurons), for the fixation period (top; p = 5.38e−12) and perisaccadic period (bottom; p = 0.50). p-values were computed by one-sided Wilcoxon signed-rank test. Histograms in upper right show the distribution of differences. Source data are provided as a Source Data file.

Full size image

Next, we examined the model-predicted and actual fixation and perisaccadic neural responses for ensembles with RFs far from the ST. Similar to Fig. 5a, we transformed the axes to examine the relationship between bias-relevant STUs and the model response for ensembles with d ≥ 11 (Fig. 6a). Results look similar for MT and V4 neurons (Supplementary Fig. 6). Since there are two regions of bias-relevant STUs for this group of neuronal ensembles (Fig. 4a bottom), the contours in Fig. 6a also illustrate two temporal regions of the model-predicted response that might contribute to spatial bias. Near and far probes were defined as in Fig. 5b; however, for these ensembles most of the neurons have RFs on the other side of the FP from the ST (see prevalence in Fig. 6b). Since there are two regions of the model-predicted response that could potentially contribute to spatial bias, we compared the model’s response at near vs. far probes during fixation (−500:−100 ms), perisaccadic (0:40 ms), and postsaccadic (70:200 ms) windows (Fig. 6c from top to bottom). We quantified responses in a 60 ms window covering the peak of each response (shown by the gray bars, fixation: 30:90 ms, perisaccadic: 40:100 ms, postsaccadic: 30:90 ms). During fixation, there is no response to either near or far probes (near = 0.99 ± 0.00, far = 0.99 ± 0.00; p = 0.86). Perisaccadically, responses were observed for both near and far probes, with a larger increase of response for near probes (near = 1.05 ± 0.01, far = 1.01 ± 0.01; p = 1.05e−12). Postsaccadically, there is a continued increase in response at near probes, but the response at far probes decreases back to the fixation level (near = 1.06 ± 0.01, far = 0.99 ± 0.00; p = 5.82e−22). Neuron’s responses mirror the model’s predictions in the fixation window (near = 0.98 ± 0.01, far = 0.10 ± 0.01), perisaccadic window (near = 1.07 ± 0.02, far = 1.03 ± 0.02), and postsaccadic windows (near = 1.12 ± 0.01, far = 1.05 ± 0.01) (Fig. 6c from top to bottom). Figure 6d demonstrates that there is no significant difference between firing rates at near vs. far probes during fixation (near = 26.27 ± 1.01 Hz, far = 26.33 ± 1.02 Hz; p = 0.31), but during the perisaccadic response window the neural firing rate for near probes is significantly higher than the firing rate for far probes (near = 28.51 ± 1.08 Hz, far = 27.21 ± 1.07 Hz; p = 9.44e−10) and continues during the postsaccadic response window (near = 29.27 ± 1.02 Hz, far = 27.27 ± 1.01 Hz; p = 1.23e−13). Neurons with RFs far from the ST do not respond to either near or far probes during fixation, but respond more to near probes perisaccadically and postsaccadically. Neurons respond more strongly to near-ST stimuli closer to the FP, reflecting the spatial bias opposite to the saccade direction in ensembles with d ≥ 11. These findings demonstrate how this systematic and unbiased search in the space of spatiotemporal sensitivity components can identify the neural basis for a biased representation of visual space during eye movements.

Fig. 6: Identifying the neural correlates of spatial bias around the ST area from neurons whose RFs are located far from the ST.
figure 6

a Bias-relevant STUs and model-predicted response, plotted as a function of time from stimulus to saccade onset (y-axis) and time of response from stimulus onset (x-axis). Shown for ensembles with d ≥ 11 dva (n = 11 ensembles, 353 neurons). b The map of RF centers and probe locations. Similar to Fig. 4b, but for ensembles with RF centers primarily on the opposite side of the FP from the ST (defined in a). c Mean of normalized model-predicted responses (left) and actual neural responses (right) over time from stimulus onset, for near (orange) and far (green) probes, during fixation (top; −500:−100 ms), perisaccadic (middle; −20:10 ms), and postsaccadic (60:230 ms) windows. Mean ± SEM across models or neurons in the ensembles; gray bars show analysis windows used in d. d Comparison of actual neural responses to near vs. far probes (n = 94 neurons), for the fixation (top; p = 0.31), perisaccadic (middle; p = 9.44e−10), and postsaccadic (bottom; p = 1.23e−13) periods. p-values were computed by one-sided Wilcoxon signed-rank test. Histograms in upper right show the distribution of differences. Source data are provided as a Source Data file.

Full size image

To gain a deeper understanding of the nature of perisaccadic mislocalization, we wanted to investigate how perisaccadic neural modulations are associated with the representation around the ST and how it might be related to the observed spatial bias. The neural firing rate for probes in the initial RF is higher during fixation (−400:−200 ms) than perisaccadically (50:150 ms), but higher perisaccadically than during fixation for probes near the ST, demonstrated by sample neurons (Fig. 7a) and the distributions of firing rate differences over the population (Fig. 7b; top: p = 1.32e−45, bottom: p = 2.73e−15). To assess the change of neuronal sensitivity around the ST in the corresponding time and delay windows as the spatial bias, we defined a ST sensitivity index using kernels averaged over delays of 50:100 ms. This sensitivity index quantifies the average changes in sensitivity for near-ST probes and can theoretically vary independently of the spatial bias measure. Out of the 6 ST probes, we divided the range of kernel weights by the mean kernel weight over all times relative to saccade onset to quantify the difference in sensitivity of a neuron to all probes presented around the ST area across time from saccade onset (Fig. 7c). We excluded 95 neurons with high kernel weights during the second fixation period (240:440 ms from saccade onset) comparing to the first fixation period (−441:−241 ms) (i.e., neurons whose postsaccadic RF included the near-ST probe locations) to reduce the interference of future field activity. In the same perisaccadic time window that we observed the spatial bias (50:150 ms shown by gray bar in Fig. 7c), there is an increase in the ST sensitivity index compared with the fixation window (−300:−150 ms; Fig. 7d; fixation = 3.27 ± 0.07, perisaccadic = 3.67 ± 0.09; p = 0.04). The increase indicates that the modulation of neurons’ spatiotemporal sensitivity around the time of saccades enhances the representation of the ST area. To examine the relationship between the spatial bias and enhanced ST representation, we measured the ST sensitivity index again with the reduced model in which bias-relevant STUs were nullified (Fig. 4b). In the same perisaccadic window, the ST sensitivity index in the reduced model is significantly smaller than in the full model (Fig. 7e; 3.31 ± 0.12; p = 3.18e−09). Thus, the enhanced ST sensitivity index around the ST relies on the bias-relevant STUs, and a computational manipulation that removes spatial bias leads to decreased sensitivity around the ST. This reveals that the perisaccadic spatial bias could be a result of the same changes in sensitivity, which enhance the ST representation around the time of saccades.

Fig. 7: Perisaccadic enhancement of the ST representation and its relationship to the perisaccadic spatial bias.
figure 7

a Neural responses for three sample neurons to probes appearing at the RF center (top) and near the ST (bottom) over time from stimulus onset, for probes presented during the fixation (green; −400:−200 ms) and perisaccadic (orange; 50:150 ms) windows. Mean ± SEM across trials for each neuron. b Distribution of firing rate differences (perisaccadic-fixation) across 447 neurons for probes at the RF center (top; p = 1.32e−45) and near the ST (bottom; p = 2.73e−15). c Average ST sensitivity index of 352 neurons over time from saccade onset for stimulus delay of 50:100 ms relative to each timepoint. Mean ± SEM across neurons; gray bar shows the analysis window used in d and e. d Comparison of ST sensitivity index in the perisaccadic window (50:150 ms) and fixation window (−300:−150 ms) (p = 0.04). Histogram in upper right shows the distribution of differences. e Comparison of ST sensitivity index in the perisaccadic window using the kernels from the full model vs. those from the reduced model (bias-relevant STUs removed) (p = 3.18e−09). p-values were computed by one-sided Wilcoxon signed-rank test. Histogram in upper right shows the distribution of differences. Source data are provided as a Source Data file.

Full size image

Discussion

How the location of visual stimuli is represented in the brain is not well understood. Imaging studies have suggested that the perceived location could be encoded in extrastriate visual areas along with other visual features52,53. Our perception of location changes around the times of saccades2,4, as do extrastriate responses27,41,54. We measured monkeys’ perception of stimulus location behaviorally and found perisaccadic mislocalization opposite to the saccade direction for stimuli near the ST. We then used a combined experimental and computational approach to examine how changes of sensitivity in MT and V4 could explain perisaccadic mislocalization. We quantified perisaccadic spatial bias around the ST and identified the STUs relevant for the observed bias, which reveals that neurons with RFs far from the ST contribute more to the perisaccadic spatial bias. We found perisaccadic changes in extrastriate sensitivity in the identified bias-relevant time and delay windows, supporting the hypothesis that location representation occurs in extrastriate visual areas. In addition, we demonstrated that the spatial bias is accompanied by the perisaccadic enhancement of neural sensitivity around the ST, with a matching time course and underlying neural response components. This enhancement of ST representation is consistent with previous behavioral and neural findings24,26,35,37,49,50,51, suggesting that the brain prioritizes ST representation at the expense of biases in location perception.

The spatial bias measure predicts perisaccadic mislocalization opposite the saccade direction; importantly, this matches the direction of mislocalization we measured behaviorally in monkeys under conditions similar to the neurophysiological recordings (Fig. 1). The previous psychophysics results have been mixed, but our behavioral and neurophysiological results are consistent with many aspects of the previous literature. In total darkness, Honda reported that mislocalization in human subjects starts in the same direction as saccade and then is reversed to the opposite direction, with the greatest mislocalization occurring ~50 ms after saccade onset9,10. In a double-saccade task, Jeffries et al. found that mislocalization in rhesus monkeys is in the direction opposite to the first saccade, with the maximum mislocalization around 100 ms after saccade onset11. Klingenhoefer and Krekelberg similarly reported that monkeys mislocalized stimuli between the FP and ST in the direction opposite to the saccade4. The magnitude of mislocalization we observed is smaller than that reported in the previous literature, which might be due to various differences in the paradigm, including a shorter saccade vector, smaller offset between probes, smaller range of possible probe locations, and limiting the probes to within 5 dva from the ST. Based on the model’s kernels, we observed spatial bias in the direction opposite the saccade, at a timing consistent with both the human and nonhuman primate studies (Fig. 3d); however, we cannot rule out the possibility that examining different RF or probe positions could reveal cases of spatial bias in the saccade direction. In addition to mislocalization parallel or opposite to the saccade direction, many studies have reported compression when conducting the experiments in a dimly lit room15,16,17, meaning that stimuli are perceived as closer to the ST (i.e., mislocalization opposite the saccade direction for stimuli past the ST, and in the saccade direction for others). In a computational study, Krekelberg et al. also predicted mislocalization in the direction of the saccade at a location close to the FP, and mislocalization in the opposite direction at locations near and past the ST19. They implemented a decoder using nonhuman primate neural data recorded from MT, the medial superior temporal area (MST), the ventral intraparietal area (VIP), and the lateral intraparietal area (LIP). Decoder-based predictions of how perceived location will be altered by various types of remapping vary qualitatively based on assumptions about the decoder; specifically, whether the same decoding algorithm is applied during the perisaccadic and fixation periods (‘unaware’ of the RF shift), or with knowledge of the perisaccadic responses changes (‘aware’). In a theoretical study done by Qian and colleagues, a decoder predicted that future field remapping would result in mislocalization opposite to the saccade direction when the decoder is unaware of the RF shift55. When there is ST remapping, the unaware decoder predicted divergent mislocalization, and the aware decoder predicted convergent mislocalization. Like most previous studies attempting to understand this connection, our approach for measuring spatial bias assumes the same decoding algorithm is used during fixation and around the time of saccades, with altered visual responses driving the perisaccadic perceptual changes. We only found spatial bias opposite to the saccade direction for stimuli around the ST; however, we are not ruling out the possibility of a compression phenomenon or convergent mislocalization, because in this study we did not measure spatial bias for stimuli at locations other than near the ST (nor were our probe positions optimized to make such systematic measurements across the rest of the visual field).

Our results substantiate the association between perisaccadic mislocalization and changes in perisaccadic sensitivity (i.e., RF remapping)19,43,44. Many studies have interpreted perisaccadic mislocalization as a flaw in the visual system while shifting the coordinate systems across saccades10,13,56, but it is not clear what the reason for this flaw is, or if it is the byproduct of another, beneficial, set of changes. Previous psychophysical studies have reported enhanced discrimination performance at the ST35,37 and neurophysiological studies have shown a perisaccadic increase in neural sensitivity at the ST24,26,38,39,50,51. The exact spatial extent of this improved perception near the saccade target is unknown, and locations farther from the saccade endpoint instead display an overall decrease in visual sensitivity around the time of saccades, known as saccadic suppression4,5,6,7,35. Our model-based measure shows increased sensitivity to perisaccadic stimuli within 5 dva of the ST. The saccade target theory has hypothesized that the brain biases toward representing the ST in order to maintain visual stability, and the representation of non-target locations is consequently reduced57,58. Our results demonstrate that removal of bias-relevant neural components is correlated with a reduction of perisaccadic sensitivity around ST (Fig. 7c). Based on our results, we suggest that spatial mislocalization could be a result of allocating more neural resources toward the ST representation. The spatial bias could therefore be interpreted as a tradeoff the brain makes to amplify the ST representation perisaccadically, consistent with the saccade target theory and ST remapping. It should be noted that future field remapping could also contribute to perisaccadic spatial bias, and our dataset was not optimized to definitively differentiate between these two forms of remapping. Figure 6c shows increased perisaccadic response around the ST that might be induced by ST remapping, and the increased postsaccadic response could reflect future field remapping. This possible link between future field remapping and mislocalization will require further investigation. We also cannot definitively state whether these spatial biases in responses arise first in MT and V4 or are inherited from upstream areas.

Our approach in this study also reinforces the feasibility of using a GLM framework to model higher visual areas. The classical GLM has been widely used for encoding and decoding neural responses in low-level visual areas (such as the retina, LGN, and V1)59,60, but they fall short in capturing the time-varying characteristics of higher-level visual areas. To model responses in these areas, nonstationary model frameworks that enable a time-varying extension of a GLM have been developed, which succeeded in characterizing the perisaccadic spatiotemporal changes of neural response and reading out perisaccadic stimulus information on the same timescale of saccadic eye movements42,48,61,62,63. In the present study, we took advantage of this GLM framework (SVGLM) and developed a procedure to measure spatial bias based on instantaneous neural sensitivity at various locations to identify the neural components contributing to spatial bias. Our results provide a potential explanation of the neural basis of mislocalization, which could be tested most definitively through experiments combining psychophysical measurements in macaques with causal manipulations of neural activity. These applications of the SVGLM framework demonstrate that a GLM-based approach is a viable way of studying the complex dynamics in higher-level visual areas, and could also be used to link specific aspects of neural sensitivity to different perceptual phenomena.

Methods

All experimental procedures complied with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and the Society for Neuroscience Guidelines and Policies. The protocols for all experimental, surgical, and behavioral procedures were approved by Institutional Animal Care and Use Committee of the University of Utah.

Behavioral paradigm for Perisaccadic Localization Task

Two adult male rhesus macaques (Macaca mulatta; both 9 years old) performed the Perisaccadic Localization Task. To start a trial, the monkey held fixation on a central FP. After the monkey had held fixation for 500 ms, a ST appeared 10 dva away from the FP horizontally. In each recording session, there was only one saccade direction (leftward or rightward). At a random time during fixation or saccade execution, a 50-ms visual probe stimulus was presented in one of nine possible locations in a 3×3 grid. The probes in the grid were 4 dva apart from each other. After the monkey held fixation for 1000 ms, the FP disappeared, which was the go cue to saccade to the ST. The monkey then made a second saccade to report his perceived location of the probe stimulus. If the monkey reported within 4 dva radius of the correct location, feedback was given by a reappearing flash of the stimulus, and the monkey received a juice reward. The FP and ST were white circles (full contrast) with a radius of 0.25 dva, and the probe stimuli were red circles with a radius of 0.4 dva, against a black background. We collected behavioral data during 89 sessions for Monkey 1 and 40 sessions for Monkey 2. The Perisaccadic Localization Task is based on tasks previously used to characterize perisaccadic mislocalization in monkeys4,11,19; it was designed to be similar to the paradigm used for separate neurophysiological recordings in terms of ambient lighting conditions, probe size, background color, and saccade amplitude.

Combined behavioral and electrophysiological recording

We trained and recorded from four adult male rhesus macaques (Macaca mulatta; 6, 6, 11, and 13 years old). The behavioral task used in this study was a visually guided saccade task, with probe stimuli appearing at pseudorandom locations before, during, and after the saccade. To start a trial, the monkey held fixation on a central FP. While the monkey was holding fixation, a ST appeared 13 dva away from the FP horizontally. In each recording session, there was only one saccade direction (leftward or rightward). After a randomized time-interval (uniform distribution between 700 and 1100 ms), the FP disappeared, which was the go cue for the monkey to saccade to the ST. The monkey then held fixation on the ST for 560:750 ms to receive a juice reward. Throughout the length of each trial, a complete sequence of 81 probe stimuli flashed on the screen in pseudorandom order, one at a time for 7 ms each. The probe locations were selected pseudorandomly from a 9×9 grid of possible locations. Each probe stimulus was a white square (full contrast), 0.5×0.5 dva, against a black background. Each probe stimulus occurred at each time in the sequence with equal frequency across trials.

During each neurophysiological recording session, the grid of possible locations of the probes was placed and scaled to cover the estimated presaccadic and postsaccadic RF centers of the neurons recorded, the FP, and the ST. The probe grids varied in size horizontally from 24 to 48.79 (40.63 ± 5.93) dva, and vertically from 16 to 48.79 (39.78 ± 7.81) dva. The distance between two adjacent probe locations varied horizontally from 3 to 6.1 (5.07 ± 0.74) dva, and vertically from 2 to 6.1 (4.97 ± 0.97) dva.

While the monkey was performing the task, we monitored their eye movements with an infrared optical eye-tracking system (EyeLink 1000 Plus Eye Tracker, SR Research Ltd., Ottawa, CA) with a resolution of <0.01 dva (based on the manufacturer’s technical specifications), and a sampling frequency of 2 kHz. Presentation of the visual stimuli on the screen was controlled using the MonkeyLogic toolbox. In total, 332 neurons in the middle temporal (MT) cortex and 291 neurons in area V4 were recorded in 108 sessions, but only 300 MT and 147 V4 neurons were used in order to make ensembles of neurons with at least 10 neurons with a similar RF, ST, and grid position during recording. We recorded both spiking activity and the local field potential (LFP) from either MT or V4 using 16-channel linear array electrodes (V-probe, Plexon Inc., Dallas, TX; Central software v7.0.6 in Blackrock acquisition system and Cheetah v5.7.4 in Neuralynx acquisition systems) at a sampling rate of 32 KHz, and sorted neural waveforms offline using the Plexon offline spike sorter and Blackrock Offline Spike Sorter (BOSS) software.

Encoding model framework

The Sparse Variable Generalized Linear Model (SVGLM) used in this study was previously developed by Niknam et al.48, see this paper for more details of the model fitting. The SVGLM is a variant of the widely used GLM framework59,60,64 that tracks the fast dynamics of sparse spiking activity with high temporal precision and accuracy. The SVGLM can capture the neurons’ sensitivity varying over space and time with high temporal resolution by providing a parameterized representation of the neuron’s spatiotemporal kernels using discrete STU components. Through a dimensionality reduction process, the model selects STUs which make a statistically significant contribution to the neuron’s response to achieve sparsity for fitting the parameters to the data (see Supplementary Information). The fitted model also captures how much these STUs contribute quantitatively to generating spikes on a precise millisecond timescale during a saccade. The weighted combination of these STUs constitutes the spatiotemporal stimulus kernels. The SVGLM defines a conditional intensity function according to the equation,

$${\lambda }^{\left(l\right)}\left(t\right)=f\left({\sum}_{x,y,\tau }{k}_{x,y}\left(t,\tau \right){s}_{x,y}^{\left(l\right)}\left(t-\tau \right)+{\sum}_{\tau }h\left(\tau \right){r}^{\left(l\right)}\left(t-\tau \right)+b\left(t\right)+{b}_{0}\right)$$
(1)

where \(\lambda\) is the instantaneous firing rate of the neuron at time \(t\) in trial \(l\), \({s}_{x,y}^{\left(l\right)}\) is either 0 or 1 representing respectively the off or on condition in a sequence of probe stimuli presented on the screen at probe location (\(x,y\)) in trial \(l\). \({r}^{\left(l\right)}\left(t\right)\) denotes the spiking response of the neuron for trial \(l\) and time \(t\), \({k}_{x,y}\left(t,\tau \right)\) represent the stimulus kernels, \(h\left(\tau \right)\) indicates the post-spike kernel applied to the spike history which captures the refractory effect, \(b\left(t\right)\) is the offset kernel that reflects the change of baseline activity induced by saccades, the constant \({b}_{0}={f}^{-1}\left({r}_{0}\right)\) with \({r}_{0}\) as the measured mean firing rate (Hz) across all trials in the experimental session, and \(f\left(u\right)=\frac{{r}_{\max }}{1+{e}^{-u}}\) is a static sigmoidal function that describes the nonlinear properties of spike generation with \({r}_{\max }\) indicating the maximum firing rate of the neuron obtained empirically from the experimental data. The model was fitted using an optimization procedure in the point process maximum likelihood estimation framework65 at the level of single trials. The evaluation for model performance is described in supplementary information (Supplementary Fig. 2b–d). More details about the model structure, estimation, and evaluation can be found in ref. 48.

Measuring spatial bias

Neurons recorded with the same ST position and probe arrangements (grid positioning and spacing), and with similar RF locations were grouped as an ensemble. Fifteen ensembles were formed, each with a minimum of 10 neurons. Before any analysis, kernels of all neurons were smoothed by moving average using boxcar windows of length 50 ms across time \(t\) and 20 ms across delay \(\tau\) to reduce noise. Figure 2c shows 9 kernels for a sample probe at the center of a neuron’s RF and its 8 neighboring probes, stacked over neurons in an example ensemble. For each particular time and delay, we constructed two population kernel vectors consisting of the kernel values of center probe and a neighbor probe at that time and delay with all neurons in an ensemble. To measure the similarity between kernels at neighboring probe locations, we computed the correlation between the kernels of center probe and a neighbor probe with all neurons in an ensemble, and subtracted baseline correlation values during fixation (−441:−141 ms from saccade onset). The correlation was measured for each of the 8 neighboring probes and repeated for 7×7 probe locations (after excluding probes on the edges). Using correlation values as magnitudes, and the probe position relative to the center probe as directions, we formed 8 vectors at each probe location across time and delay and took the average of these 8 vectors at each of the 7×7 probe locations. The polar plot in Fig. 2e shows these vectors between a sample probe around ST and its 8 neighboring probes. Spatial bias at each location was defined as the horizontal projection of the average vector at that location, and it was computed for all 15 ensembles. These spatial bias values were used to construct spatial bias maps across time and delay for each of 7×7 probe locations. Figure 3a, b shows two cross-sections of an example bias map, associated with a sample probe location around ST, over particular time and delay windows. For each ensemble, we averaged the bias maps at the 6 probe locations closest to the ST, excluding probes that were within 2 dva from either the presaccadic or postsaccadic RF (Fig. 3c). Before averaging the bias maps of all 15 ensembles, the original spatial bias of each ensemble was near-symmetric around zero and normalized to range from −1 to 1 using the following formula: 2×(bias−min(bias))/(max(bias)−min(bias))−1. We used one-sided Wilcoxon signed-rank test to report p-values for all our statistical comparison analysis, if not mentioned specifically.

Identifying modulated STUs

To identify which components of the neurons’ spatiotemporal sensitivity drive the neuron’s response changes around the time of saccades, we first quantified the contribution of each STU. It was expected that out of all STUs, only some of them at specific times and delays contribute to the generation of the neural response (referred to as ‘contributing STUs’). These contributing STUs were identified using the dimensionality reduction process during the model fitting, based on a statistical significance criterion, which measures the contribution of individual STUs to capturing the stimulus-response relationship (see Supplementary Information, and Niknam et al48. for details).

We then defined the modulated STUs as those for which the fraction of contributing STUs in a 3×3 window around that STU’s time and delay for each STU’s spatial location is significantly different during perisaccadic period compared to fixation period. Mathematically speaking, the fraction of contributing STUs needs to fulfill the following condition:

$$\sqrt{\left|p\left({\tau }_{n},{t}_{m}\right)-{p}_{1}\left({\tau }_{n}\right)\right|\cdot \left|p\left({\tau }_{n},{t}_{m}\right)-{p}_{2}\left({\tau }_{n}\right)\right|} > h$$
(2)

with \(p\left({\tau }_{n},{t}_{m}\right)\) as the fraction of contributing STUs in a 3×3 window around the \(n\) th bin of delay and \(m\) th bin in time 1 < \(n\) < 30,1 < \(m\) < 156. \({p}_{1}\left({\tau }_{n}\right)\) is the fraction of contributing STUs during the first fixation period 540 to 120 ms before saccade in time bin 1 to 60 at \(n\) th bin of delay (1 < \(n\) < 30), and \({p}_{2}\left({\tau }_{n}\right)\) is the fraction of contributing STUs during the second fixation period 280 to 540 ms after saccade in time bin 120 to 156 at \(n\) th bin of delay (1 < \(n\) < 30). \(h\) is a significance threshold between 0 and 1, and was set to 0.3 for the analysis.

Identifying bias-relevant STUs

From the list of modulated STUs, we identified the ones that contribute to spatial bias specifically (termed bias-relevant STUs). The contribution of each modulated STU to the spatial bias was quantified by evaluating its impact on the difference between kernels at neighbor probes across a saccade, by removing each modulated STU one at a time and testing if the change in kernels difference is significant based on a bias index. Because spatial bias was measured from the correlations between kernels at neighbor probes for an ensemble of neurons, the difference between the stimulus kernels of two neighboring probes for individual neurons, may impact the resulting spatial bias read out from that ensemble. The absolute difference between each pair of stimulus kernels of the fitted models at two neighboring probe locations (\({x}_{0},{y}_{0}\)) and (\({x}_{i},{y}_{i}\)) and at each delay (\(\tau\)) across different times to the saccade (\(t\)), was quantified as

$${{AUC}}_{i}={\sum}_{t,\tau }{{{\rm{|}}}}{k}_{{x}_{0},{y}_{0}}\left(t,\tau \right)-{k}_{{x}_{i},{y}_{i}}\left(t,\tau \right){{{\rm{|}}}}$$
(3)

where \({{AUC}}_{i}\), represents the area under curve of the difference of kernels \({k}_{{x}_{0},{y}_{0}}\left(t,\tau \right)\) and \({k}_{{x}_{i},{y}_{i}}\left(t,\tau \right)\) over time and delay, between each center probe at (\({x}_{0},{y}_{0}\)) and each of the eight neighbor probes (\(i\in \{1,\ldots,8\}\)) at (\({x}_{i},{y}_{i}\)) (Supplementary Fig. 5a). Since the spatial bias was measured for 6 probe locations around ST, we also measured the difference AUCs at those same 6 center probes for the neurons in each ensemble. The average of difference AUCs over 8 center-surround probe pairs was used to compute the bias index associated with individual center probe in the following. For each neuron in each ensemble, we first measured the difference AUCs with the full model (no perturbation in the model estimated STUs). Next, we removed each of the modulated STUs one at a time from the full model by replacing that STU with zero and repeat the above steps so that we have a list of difference AUCs measured without each of the modulated STUs. We then defined the bias index for each modulated STU as the absolute difference between the AUC for full model and the AUC corresponding to removing each of the modulated STU from the model (Supplementary Fig. 5b). To identify bias-relevant STUs, we defined a threshold for this bias index as the 90th percentile of the cumulative distribution function of all the nonzero bias indices (bias index of 2.56) as the threshold (Supplementary Fig. 5c). A larger bias index means that nulling the weight of a particular STU results in a stronger change in kernel differences for the probes around the ST, so the STUs with a bias index above the threshold were classified as bias-relevant. The bias indices were specific to each of the 6 ST probes and for each neuron. Figure 4a shows the map of bias-relevant STUs which was generated based on the mean bias index across probes and neurons. To validate if the identified bias-relevant STUs using this procedure would actually contribute to the readout spatial bias from each ensemble, we removed the identified bias-relevant STUs from the model for each neuron, and recomputed the spatial bias (Fig. 4b).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.