Constructing useful representations of our visual environment requires the ability to selectively pay attention to particular locations at specific moments. Whilst there has been much investigation on the influence of selective attention on spatial discrimination, less is known about its influence on temporal discrimination. In particular, little is known about how endogenous attention influences two fundamental and opposing temporal processes: segregation – the parsing of the visual scene over time into separate features, and integration – the binding together of related elements. In four experiments, we tested how endogenous cueing to a location influences each of these opposing processes. Results demonstrate a strong cueing effect on both segregation and integration. These results are consistent with the hypothesis that endogenous attention can influence both of these opposing processes in a flexible manner. The finding has implications for arbitrating between accounts of the multiple modulatory mechanisms comprising selective attention.
Goal-directed behavior requires that visual input be integrated into representations that emphasize task-relevant information. A wealth of evidence demonstrates that this is achieved in part through the deployment of selective attention in space. The spatial deployment of attention strongly impacts visual representations, leading to faster and more accurate target processing (Posner, 1980) and improved perceptual representations at attended locations (Carrasco, Loula, & Ho, 2006; Yeshurun & Carrasco, 1999). However, our understanding of the effect of spatial attention on temporal processing is more limited. Temporal processing has two apparently opposing requirements: we often need to segregate visual input over time so that the visual scene can be parsed into separate features and events, but another common necessity is to integrate strongly-related stimuli so that they are perceived as unitary phenomena (Deco et al., 2015; Pöppel et al., 2009; Stroud, 1955).
Here, we investigate the possibility that the strategic deployment of spatial attention has a flexible impact on both these temporal processes, depending on task confines. This possibility has not been addressed in extant literature, in large part because existing work has focused on the effect of attentional capture on temporal segregation. This literature has thus employed exogenous cues that involuntarily draw attention to a location, either by virtue of their raw physical salience or through their correspondence to current attentional control settings. Rather than exhaustively discussing this literature in the body of the manuscript, we summarize representative findings in Table 1. Clearly demonstrated here is the predominant use of exogenous cues and temporal segregation tasks in this literature.
Results from this work broadly show that exogenous cues degrade the ability to temporally segregate stimuli. In a seminal example, Yeshurun and colleagues conducted a series of experiments using a version of the two-flash fusion paradigm with exogenous cues and forward-masked stimuli (Yeshurun & Levy, 2003; Yeshurun, 2004). In this paradigm, a small disk is presented either slightly to the left or right of fixation after the presentation of a preceding cue. In many trials, a second disk is presented after a brief delay at the same location, and participants are asked to report the number of disks they perceive. Participants’ ability to accurately report the number of disks is better following the neutral cue condition than the valid cue, suggesting that the valid cue degrades the ability to parse two percepts. This idea is consistent with findings from other experiments using various common temporal processing paradigms (Akyürek & van Asselt, 2015; Hein, Rolke & Ulrich, 2006; Rolke et al., 2008; Yeshurun & Hein, 2011).
One account of this finding is that the degradation stems from the differentiation of magnocellular and parvocellular pathways in visual processing. Spatial attention is thought to act in large part through emphasis and modulation of input from parvocellular neurons, which have small receptive fields (e.g., Schiller & Logothetis, 1990). However, because magnocellular neurons have better temporal resolution, a relative down-weighting of magnocellular input could come at the cost of temporal sensitivity. Evidence for this explanation is mixed, with some support from work in which the relative luminance of stimuli was matched (Yeshurun, 2004), but conflicting results showing survival of the cueing effect under conditions that should have mitigated the magno-parvo difference (Akyürek & van Asselt, 2015; Yeshurun & Levy, 2003).
Studies of attentional effects on temporal processing using manipulations other than exogenous cues are rarer, and have commonly manipulated aspects of attention other than discrete location. For example, Goodhew, Shen, and Edwards (2016) used a paradigm that implicitly biased participants to use an “attentional spotlight” that was either spatially focused or spatially diffuse, before subsequently testing spatial and temporal acuity. When biased to employ a focused spotlight, spatial acuity was enhanced, but temporal precision was unaffected. Other results show degraded integration when the availability of attentional resources was reduced by a difficult pre-trial task (Visser & Enns, 2001) and one unique study shows that accurate temporal processing depends on whether or not participants could see their own hands (Bush & Vecera, 2014).
In the few studies that have directly investigated endogenous cueing effects on temporal processing, results tend to show an improvement of resolution that is in contrast to the degradation caused by exogenous cues, as presented in summary in Table 1. For example, Hein, Rolke, and Ulrich (2006) used a temporal order judgement paradigm in which participants were endogenously cued to a hemifield where they would see two rapid sequential stimuli, finding that valid endogenous cues improved performance at identifying the order in which stimuli had arrived.
A further pattern in this small literature is that the experimental task has nearly exclusively required the segregation of temporal events. The general paucity of investigation into the effect of endogenous cueing on segregation leads to an ambiguity in interpretation: does the cueing effect reflect a reconfiguration of the system that biases temporal acuity specifically? Or is it rather that endogenous attention benefits temporal processing broadly? If the latter, this should become evident in a benefit for temporal integration when task confines require this process.
The current study was thus designed to directly test the possibility that endogenous spatial attention might flexibly benefit both temporal integration and temporal segregation. Importantly, we tested the impact of endogenous spatial cueing on both integration and segregation within a single experimental paradigm, with visual stimulation kept constant across conditions. To this end, we adapted the missing dot paradigm of Hogben and Di Lollo (1974). In the standard version of this task, two grids of dots are presented in quick succession. No location contains a dot in both of the sequential displays, and, importantly, one location contains no dot in either display. Successful temporal integration of the two displays therefore allows the viewer to perceive the location missing a dot. We adapted the design to allow for additional insight on temporal segregation. In our experiment, circles comprising two separated halves were presented in a 4 x 4 grid. In each trial one location in the grid was unfilled, while in another location the complementary halves of the circle were presented across two temporally-separated frames (Wutz et al., 2016). As in the canonical missing dot paradigm, locating the missing circle required integration of the two displays. Locating the “half circle,” in contrast, required that the two frames be temporally segregated. In all experiments, we cued target location with a 75 % valid cue while participants completed separate integration and segregation versions tasks across blocks.
We approached results with two exclusive hypotheses. If endogenous spatial cueing acts to increase temporal resolution, we expected to find a benefit of cueing when the task required temporal segregation, but a cost when it required integration. If endogenous attention instead is able to flexibly reconfigure temporal processing, cueing benefits should emerge in both integration and segregation conditions.
Six healthy participants (two male; age 22.67 ± 3.39 years, mean ± SD) gave informed consent before completing the experiment. One participant was an author (P.S.), with the five remaining participants naïve to the experimental hypotheses. All participants had normal or corrected-to-normal vision and reported right-handedness.
Experimental design and stimuli
The stimuli and task were generated with Psychophysics Toolbox (Brainard, 1997) in MATLAB (MathWorks). Stimuli were presented on an LED backlit monitor specialized for psychophysics experiments (VPixx Technologies, gray-to-gray response time of 7 ms) with a refresh rate of 100 Hz.
The basic trial structure is shown in Fig. 1. A small fixation cross in the form of a red “X” was present throughout the presentation of stimuli. This cross measured approximately 0.2° (visual angle). At the beginning of each trial, one of the arms of the cross changed from red to green to provide the cue that the target for that trial would likely appear in the indicated quadrant. Participants were instructed that the cue usually indicated the correct quadrant and would help them with the task. In approximately 22 % of trials the cue was invalid, in 53 % of trials the cue was valid, and in 25 % of trials a neutral cue was shown. The neutral cue took the form of a color change of the tips of all four arms of the cross, such that a similar number of pixels changed from red to green as in the valid/invalid cue condition.
After a fixed interval of 500 ms following the cue, the fixation cross became entirely red again, and the first display appeared on screen for 10 ms. This display comprised a circle at seven locations out of a possible 16 (on a 4 × 4 grid of locations), and each circle was formed from two arc elements. At one position in the display there was a half circle (i.e., a single arc). Each circle was 1.2° (visual angle) in diameter, the grid of possible locations measured 8.4° × 8.4°. Each circle within the display had an irrelevant random orientation selected from the possible orientations of 45–315°.
This was followed by a fixed inter-stimulus interval of 10 ms, 30 ms, 50 ms, 70 ms, or 100 ms. A second display subsequently appeared for 10 ms. This display also comprised a circle at seven locations and a half circle at one location. Crucially, the half circle in the second display was the corresponding arc element at the same location as the half circle location from the first display, such that if the two displays were superimposed the two arc elements formed a standard circle stimulus. As illustrated in Fig. 1, the locations of the seven circles for each display never overlapped, such that if the two displays were superimposed, one of the 16 possible locations remained empty.
After 500 ms, a response probe screen appeared. This comprised a grid of squares where each square identified one of the 16 possible target locations. Participants indicated, by mouse click, at which location they had perceived the target.
There were two versions of the task: stimulus presentation was the same in both but task instructions were varied across blocks. In one version of the task, the target was the half circle. Successful identification of the half circle requires parsing of the two displays, therefore this is referred to as the segregation task. In the other version, the target was the location in which there was no circle in either display. Successful identification of this location requires a combined percept of the two displays, therefore this is referred to as the integration task. Participants completed 1,800 trials total (30 blocks of 60 trials), over two sessions each lasting approximately 90 min. Participants were explicitly instructed to fixate the cross in the center of the screen throughout stimulus presentation.
Behavioral data analysis
Behavioral data were analyzed using Matlab2013a (Mathworks) and R (R Foundation for Statistical Computing). Where assumptions of sphericity were not met, reported statistics are Greenhouse-Geisser corrected. As recommended by Lakens (2013), two estimates of effect sizes are provided: partial eta squared and generalized eta squared. The former is better suited for power analyses and comparisons within a study, whereas the latter is a metric more suited to comparisons across experiments (Bakeman, 2005). Post hoc testing was conducted by Bonferroni-corrected pairwise comparisons.
As illustrated in Fig. 2A, mean accuracy was better for longer ISIs in the segregation task but better for shorter ISIs in the integration task. We computed the effect of valid and invalid cues on performance in each of the segregation and integration tasks in reference to the neutral condition; these are plotted for each of the ISIs in Fig. 2B. Evident here is a performance benefit of the valid cue and performance cost of the invalid cue in both tasks. In addition to this, the results suggest that cues are more effective as a function of task difficulty: valid cues have the greatest benefit on performance when the task is hard and invalid cues have the greatest cost when the task is easy. Thus, when integration is required, the valid cue benefits performance at long ISIs and the invalid cue degrades performance at short ISIs, whereas when segregation is required, the valid cue benefits performance at short ISIs and the invalid cue degrades performance at long ISIs.
Results for each of the participants are illustrated in Fig. 2C and 2D. Generally, the pattern of cue effect for a participant appears comparable across task versions. For example, one participant (P04) showed a disproportionately large effect of the invalid as compared to the valid and neutral cue condition for both the integration and segregation task versions. Inspection of panels C and D indicates that there is some degree of individual differences in magnitude of the invalid cueing effect as compared to the valid cueing effect.
Statistical analysis began with a repeated measures ANOVA (RANOVA) with factors for cue (valid, neutral, invalid), task (segregation, integration), and ISI (10 ms, 30 ms, 50 ms, 70 ms, and 100 ms). Importantly, this identified a main effect of cue (F(2,10) = 13.30, p = .011, ηP2 = .727, = ηG2 = .478), reflecting the consistent benefit of valid cues and cost of invalid cues in both tasks. The effect of the cue did not reliably differ across the tasks, as reflected in a non-significant task by cue interaction (F(2, 10) = 3.51, p = .070, ηP2 = .413, = ηG2 = .019). Also important is the emergence of an interaction between task and ISI (F(4,20) = 40.02, p < .001, ηP2 = .889, = ηG2 = .428), reflecting a pattern in which valid cues most benefited performance when the task was difficult and invalid cues degraded it when the task was easy. Visually, this pattern appears accentuated in the valid cue condition, but the three-way interaction did not reach significance (F(8,40) = 2.11, p = .058, ηP2 = .297, = ηG2 = .031). An effect of ISI (F(4,20) = 5.33, p = .025, ηP2 = .516, = ηG2 = .044) also emerged, as did the interaction between cue and ISI (F(8,40) = 3.23, p = .04, ηP2 = .393, = ηG2 = .015), but there was no effect of task (F(1,5) = 1.95, p = .226, ηP2 = .280, = ηG2 = .066).
Pairwise comparisons between means provided further insight into these results. For the integration task, cue validity (valid, invalid) had an effect at longer ISIs (with corrected alpha of .01; 10 ms: t(5) = 7.79, p <.001; 50 ms: t(5) = 3.68, p = .007; 70 ms: t(5) = 4.53, p = .003; 100 ms: t(5) = 5.17, p = .002), whereas for the segregation task, cue validity had an effect at shorter ISIs (t(5) = 7.79, p <.001; 30 ms: t(5) = 6.52, p = .001; 50 ms: t(5) = 3.85, p = .006). For completeness, comparisons were also made for valid and invalid cueing as compared to the neutral cueing at each ISI, for each task version. Mean performance in the valid cue condition was better than in the neutral cue condition for the 10-ms ISI in the segregation version (t(5) = 3.72, p = .007). Effects at the longer ISIs did not reach significance (all other p values > .01). The valid cueing effect for the integration task was significant at longer ISIs (50 ms: t(5) = 4.98, p= .002; 70 ms: t(5) = 4.98, p= .002; 100 ms: t(5) = 6.25, p= .001; all other ps > .01). Similarly, the cost of the invalid cue was significant at shorter ISIs in the segregations version (10 ms: t(5) = 5.81, p= .001; 30 ms: t(5) = 4.74, p= .003). Effects at the longer ISIs did not reach significance (all other ps > .01). For the integration task, differences between the invalid and neutral results were not significant at corrected alpha.
Results from Experiment 1 demonstrate a benefit of the endogenous deployment of spatial attention on both integration and segregation tasks, though these required opposing mechanisms of temporal processing. In addition, Experiment 1 identified a pattern wherein spatial attention has the greatest impact on temporal perception when the task is more difficult, consistent with broader theoretical understanding of the role and function of spatial attention (e.g., Luck, Girelli, McDermott, & Ford, 1997).
Though participants in Experiment 1 were explicitly instructed to maintain fixation, eye movements were not monitored and so it is theoretically possible that participants may have moved their eyes in response to cue stimuli. To address this possibility and to reproduce the pattern of results observed in Experiment 1, we conducted an expanded replication in which gaze position was monitored.
Sample size choice for the experiment was informed by power analyses conducted with G*Power 3 (Faul et al., 2007). Using the effect size (partial eta squared) for the interaction effect of cue by ISI in Experiment 1, we estimated that a minimum sample size of six participants would be required for 95 % power at the 5 % alpha level for this effect. As noted above, we included eye-tracking to exclude any trials containing eye movements. Thus, there were two possible sources of reduced power in this design: losing trials and, potentially, weaker effects for covert compared to overt attention shifts. To ensure sufficient power we recruited more participants than the minimum suggested by this power analysis.
Twelve healthy participants (three female; mean ± SD age 25.17 ± 5.95 years; two left-handed) gave informed consent before completing the experiment. All participants had normal or corrected-to-normal vision and none had taken part in Experiment 1.
Experimental design, stimuli, and task
The design and stimuli were as in Experiment 1. In addition, gaze position was recorded in order to exclude from analysis those trials where participants inappropriately broke fixation. One participant completed 27 rather than 30 blocks due to time constraints.
A tower-mounted Eyelink 2000 eye-tracker (SR Research, Ontario, Canada) was used to record the gaze position of the left eye at a sampling rate of 1,000 Hz.
Behavioral data analysis
Behavioral data were analyzed using Matlab2014a (Mathworks) and R (R Foundation for Statistical Computing). Prior to the main analysis, trials with unwanted eye movements were removed, with the following criteria: saccades larger than 1° (visual angle) during the cue interval, presentation of the displays and ISI, blinks during presentation of the displays. This resulted in a grand average rejection rate for fixation broken of 13.10 % ± 11.22 % (mean ± SD), and for blinks a rejection rate of 1.02 % ± 1.41 % (mean ± SD) of trials.
As illustrated in Fig. 3, Experiment 2 replicated the results from Experiment 1. The valid cue benefited performance in both the segregation and integration tasks and the invalid cue degraded performance in both tasks. The valid cue once again appeared to have the greatest benefit when the task was difficult (i.e., at shorter ISIs for segregation, longer ISIs for integration).
In a RANOVA with factors for cue (valid, neutral, invalid), task (segregation, integration), and ISI (10 ms, 30 ms, 50 ms, 70 ms, and 100 ms) these patterns emerged in a main effect of cue (F(2,22) = 28.53, p < .001, ηP2 =.722, = ηG2 = .274) and an interaction of task and ISI (F(4,44) = 40.71, p < .001, ηP2 =.787, = ηG2 = .201). A three-way interaction emerged reliably in Experiment 2 (F(8,88) = 5.63, p = .007, ηP2 =.338, = ηG2 = .011), driven by a pattern in the segregation task in which the valid cue had a larger effect at short ISIs but the invalid cue had a larger effect at long ISIs. A significant interaction of task and cue (F(2,22) = 3.48, p = .048, ηP2 =.240, = ηG2 = .005) reflected a slightly larger cueing effect in the integration task. An additional main effect of ISI (F(4,44) = 10.38, p < .001, ηP2 =.486, = ηG2 = .024) appeared, as did a significant interaction of cue and ISI (F(8,88) = 3.21, p = .003, ηP2 =.226, = ηG2 = .003), but no effect of task was detected (F<1, ηP2 =.0003, = ηG2 = .0001).
For the integration task, cue validity (valid, invalid) had a significant effect at all ISIs (with corrected alpha of .01, 10 ms: t(11)=4.19, p < .001; 30 ms: t(11)=4.55, p < .001; 50 ms: t(11)=5.92, p < .001; 70 ms: t(11)=7.35, p < .001; 100 ms: t(11)=5.51, p < .001). The invalid cueing effect was also significant for all ISIs (10 ms: t(11)=3.24, p = .004; 30 ms: t(11)=3.95, p = .001; 50 ms: t(11)=3.67, p = .002; 70 ms: t(11)=3.35, p = .003; 100 ms: t(11)=4.55, p < .001). Similarly, for the segregation task, the valid cueing effect was significant for the four shortest of the five ISIs (10 ms: t(11)=5.10, p < .001; 30 ms: t(11)=5.57, p < .001; 50 ms: t(11)=3.05, p = .005; 70 ms: t(11)=2.88, p = .008; 100 ms: t(11) = 2.20, p = .025). Invalid cueing effects were significant for all ISIs (10 ms: t(11)=3.40, p = .003; 30 ms: t(11)=4.25, p = .001; 50 ms: t(11)=4.55, p < .001; 70 ms: t(11)=3.46, p = .003; 100 ms: t(11)=4.75, p < .001).
Experiment 2 confirms the observations drawn from Experiment 1. Firstly, the valid endogenous cue benefited task performance and the invalid cue degraded performance in both segregation and integration tasks, despite these tasks involving opposing temporal operations. Secondly, the benefit of the valid cue was greatest when the task was hard and the cost of the invalid cue was greatest when the task was easy.
As noted in the introduction, studies of exogenous cueing have found that the capture of attention can result in a degraded ability to temporally segregate stimuli at that location. One account is that the deployment of spatial attention acts in part by emphasizing parvocellular visual input, resulting in a relative down-weighting of the temporally-precise input from magnocellular neurons (Yeshurun & Levy, 2003). As a result, visual perception becomes less sensitive to temporal asynchrony.
The current results suggest that endogenous spatial attention must differ fundamentally from exogenous spatial attention because the endogenous deployment of attention benefits temporal segregation, whereas the exogenous deployment of attention degrades it (e.g., Yeshurun & Levy, 2003). However, the idea that effects of spatial attention on temporal processing are instantiated through a bias toward parvocellular or magnocellular input remains compelling. It may be that exogenous spatial attention necessarily biases perception toward parvocellular input, but that endogenous attention is able to flexibly emphasize either stream as a function of task requirements. In order to gain some perspective on this issue, we conducted a third experiment examining the impact of target eccentricity.
The parvocellular and magnocellular visual systems are defined by a split in the visual pathway beginning in the retinal ganglion cells that propagates through the lateral geniculate nucleus to cortex. The two streams subserve distinct functionality: parvocellular neurons are selective for color and have high spatial resolution, whereas magnocellular neurons are fast and have high contrast sensitivity (Livingstone & Hubel, 1987). Importantly for our present purpose, there is evidence that the ratio of parvocellular to magnocellular neurons in a given area increases with retinal eccentricity (e.g., Azzopardi, Jones, & Cowey, 1999; Connolly & Van Essen, 1984). As a result, if endogenous spatial attention is able to flexibly bias perception towards either parvocellular or magnocellular input, this should express in different ways for stimuli located near fovea or at the periphery. On the one hand, when the goal is to integrate and target location near fovea, the predominance of slow parvocellular cells at this retinotopic location may lead to relatively good performance. Sparse parvocellular representation of peripheral stimuli may lead to comparatively worse performance. There would therefore be greater opportunity for endogenous spatial cueing effects at peripheral than at foveal locations. On the other hand, when the goal is to segregate, performance may be comparatively better at periphery than near-foveal locations on account of the proportion of fast magnocellular cells at these retinotopic locations. This would therefore lead to greater opportunity for spatial cueing effects at near foveal locations as compared to peripheral ones.
To test these hypotheses, Experiment 3 was identical to preceding experiments with one exception: sequential stimulus displays were separated by a consistent ISI of 70 ms. This reduced the dimensionality of the design and resulted in an increased number of trials per condition, which allowed us to introduce a new analytic factor representing retinal eccentricity.
Twenty healthy participants (four male; mean ± SD age 21.53 ± 3.64 years) gave informed consent before completing the experiment. All had normal or corrected-to-normal vision and reported right-handedness.
Experimental design, stimuli, and task
Prior to the main experimental blocks of this experiment, participants completed a short version of the paradigm with no cueing and multiple ISIs. In this pre-experiment version, participants completed five practice trials for each of the two versions of the task, followed by two blocks of 80 trials (one block for each task version, each block comprised ten trials for each of the eight ISIs). These data were used to provide context as the relative difficulty of each task version at the fixed ISI used in the main experiment. Participants then completed 15 practice trials with cueing for each task version (30 total) before subsequently completing ten blocks of the main experiment where each block comprised 60 trials (mixed design, except for task version which varied blockwise). Experimental design was otherwise as described for Experiments 1 and 2, with the exception that the ISI was fixed at 70 ms.
As can be seen in Fig. 4A, mean accuracy in the pre-experiment was better for longer ISIs for the segregation task but better for shorter ISIs for the integration task, as expected. The mean intersect was 52 ms ± 36 ms (mean ± SD) – slightly shorter than the fixed ISI of 70 ms used in the primary experiment.
Figure 4C illustrates how Experiment 3 again replicates the core finding from Experiment 1: valid cues benefited performance in both the integration and segregation task and invalid cues degraded it. The results suggest that performance in the integration task was strongly impacted by the eccentricity of the target, but that this was not the case in the segregation task. In the integration task the efficacy of the valid cue appears to be accentuated for targets in the periphery.
In line with these impressions, a RANOVA with factors for task (segregation, integration), cue (valid, neutral, invalid), and target proximity to fixation (near, mid, far: see Fig. 4B) identified a main effect of cue (F(2,38) = 78.33, p < .001, ηP2 = .805, = ηG2 = .485) and a cue by proximity interaction (F(4,76)= 8.47, p < .001, ηP2 = .308, ηG2 = .022). Variance in cue efficacy across levels of proximity was further modulated by task, as reflected in a significant three-way interaction (F(4,76)= 6.176, p < .001, ηP2 = .245, ηG2 = .015). A reliable cue-by-task interaction (F(2,38)= 8.72, p < .001, ηP2 = .315, ηG2 = .028) reflects an accentuated cueing effect in the integration task (alongside a shift in accuracy in the neutral cue condition across the tasks), and a reliable proximity-by-task interaction (F(2,38)= 47.85, p < .001, ηP2 = .716, ηG2 = .104) reflects an increased impact of target eccentricity in the integration task. A main effect of task (F(1,19) = 14.31, p = .001, ηP2 = .430, = ηG2 = .122) was driven by better performance in the segregation task, reflecting our use of a fixed ISI that favored segregation, and a main effect of target proximity (F(2,38) = 24.848, p < .001, ηP2 = .594, = ηG2 = .060) was driven by performance on the integration task, where accuracy was greatest for central targets.
We conducted two additional RANOVAs with factors for cue and proximity on results from the integration and segregation tasks in isolation. Analysis of the integration task revealed significant main effects of cue (F(2,38)= 78.44, p < .001, ηP2 = .805, ηG2 = .543) and proximity (F(2,38)= 55.78, p < .001, ηP2 = .746, ηG2 = .286) and a significant interaction (F(4,76)= 11.15, p < .001, ηP2 = .370, ηG2 = .066). Analysis of the segregation task revealed a robust main effect of cue (F(2,38)= 52.06, p < .001, ηP2 = .733, ηG2 = .446). In the segregation task, the main effect of proximity was not significant at the corrected alpha level of .025 (F(2,38)= 4.83, p = .027, ηP2 = .203, ηG2 = .014), and nor was the interaction (F(4,76)= 2.91, p = .027, ηP2 = .133, ηG2 = .012).
Though participants in Experiment 3 were explicitly instructed to maintain fixation, eye movements were not monitored and so, again, it is possible that participants may have moved their eyes in response to cue stimuli. To address this possibility and to reproduce the pattern of results observed in Experiment 3 we conducted an expanded replication in which gaze position was monitored.
Sample size choice for the experiment was informed by power analyses conducted with G*Power 3 (Faul et al., 2007). Using the effect size (partial eta squared) for the three-way interaction effect in Experiment 3, we estimated that a sample size of ten participants would be required for 95 % power at the 5 % alpha level for this effect.
Ten healthy participants (two male; mean ± SD age 22.40 ± 1.71 years) gave informed consent before completing the experiment. All had normal or corrected-to-normal vision and reported right-handedness.
Experimental design, stimuli, and task
As described for Experiment 3. In addition, gaze position was monitored in order to reject trials in which participants inappropriately broke fixation.
A tower-mounted Eyelink 2000 eye-tracker (SR Research, Ontario, Canada) was used to record the gaze position of the left eye at a sampling rate of 1,000 Hz.
Behavioral data analysis
Behavioral data were analyzed using Matlab2014a (Mathworks) and R (R Foundation for Statistical Computing). Prior to the main analysis, trials with unwanted eye movements were removed, with the following criteria: saccades larger than 1° (visual angle) during the cue interval or presentation of the displays and ISI, blinks during presentation of the displays. This resulted in a grand average rejection rate for fixation broken of 3.90 % ± 4.75 % (mean ± SD), and for blinks a rejection rate of 0.30 % ± 0.48 % (mean ± SD) of trials.
Figure 5A shows that the results of the pre-experiment were comparable to those in Experiment 3, with a similar intersect at the group level: 52 ms ± 18 ms (mean ± SD), again indicating that the fixed ISI of 70 ms in the main blocks was slightly greater than the mean intersect.
As can be seen from Figs. 5C and 5D, Experiment 4 broadly replicated Experiment 3. As in Experiment 3, A RANOVA with factors for task (segregation, integration), cue (valid, neutral, invalid), and target proximity to fixation (near, mid, far: see Fig. 5B) identified a main effect of cue (F(2,18) = 29.26, p < .001, ηP2 = .765, = ηG2 = .452), an interaction between cue and proximity (F(4,36)= 4.22, p = .006, ηP2 = .319, ηG2 = .024), and a three-way interaction between cue, proximity, and task (F(4,36)= 4.71, p = .004, ηP2 = .344, ηG2 = .018). Also commensurate with Experiment 3 was a significant main effect of task (F(1,9) = 48.68, p < .001, ηP2 = .844, = ηG2 = .376), a main effect of target proximity (F(2,18) = 38.70, p < .001, ηP2 = .811, = ηG2 = .180), and an interaction between proximity and task (F(2,18)= 21.37, p < .001, ηP2 = .704, ηG2 = .107). In Experiment 4 the cue-by-task interaction was not significant, though it had similar directionality and effect size estimates (F(2,18)= 2.63, p = .10, ηP2 = .226, ηG2 = .029).
This consistency with results from Experiment 3 continued in further analysis of each task separately. Analysis of the integration task revealed significant main effects of cue (F(2,18)= 58.28, p < .001, ηP2 = .866, ηG2 = .581) and proximity (F(2,18)= 48.38, p < .001, ηP2 = .843, ηG2 = .492) and a significant interaction (F(4,36)= 5.05, p = .002, ηP2 = .360, ηG2 = .104). Analysis of the segregation task revealed a robust main effect of cue (F(2,18)= 12.04, p < .001, ηP2 = .572, ηG2 = .366). In the segregation task, the main effect of proximity was not significant at the corrected alpha level of .025 (F(2,18)= 1.70, p = .21, ηP2 = .203, ηG2 = .013), and nor was the interaction (F(4,36)= 1.84, p = .14, ηP2 = .169, ηG2 = .005).
In all four experiments, spatial cueing impacted temporal processing regardless of whether the current goal was segregation or integration. This supports the notion that strategic attentional allocation engendered by endogenous cueing modulates temporal processing in a flexible manner. When the task requires that discrete events be integrated to allow for the detection of a target stimulus, endogenous spatial cueing aids this process. But when the task requires that discrete events be segregated, endogenous spatial cueing continues to have a positive impact. Critically, our experimental design allows us to demonstrate this consistent benefit on orthogonal temporal processes when visual stimulation was the same and only task goals differed.
Our experiments replicate existing results show that endogenous cues facilitate temporal segregation (Hein, Rolke, & Ulrich, 2006), extending this finding by demonstrating that (i) there is a cost to segregation when attention is invalidly cued and (ii) endogenous cues have a commensurate benefit to temporal integration. The benefit of endogenous cues contrasts with findings from studies of exogenous cueing, where the capture of attention has been found to degrade temporal segregation (e.g., Yeshurun & Levy, 2003). This is in line with a broad notion that endogenous and exogenous attention fundamentally differ in terms of neurophysiological basis, phylogenetic origins, and behavioral impact (see Chica et al., 2013, for discussion), and additionally supports the notion that exogenous attention may operate at a lower level of the visual hierarchy than endogenous attention (Hein, Rolke, & Ulrich, 2006; Klein, 1994).
One account for the negative impact of exogenous attention on temporal processing is based on the idea that exogenous cues accentuate parvocellular input, resulting in a relative down-weighting of magnocellular input (Yeshurun & Levy, 2003). Because magnocellular neurons have better temporal resolution than parvocellular neurons, this could cause a decrease in sensitivity for temporal asynchrony. In line with the general idea that deployment of spatial attention impacts the relative weighting of parvocellular and magnocellular visual input, in Experiments 3 and 4 we find that the eccentricity of a target has an impact on temporal integration that varies as a function of cue validity. Stimuli at central locations are strongly represented in the parvocellular system by default, but stimuli in the periphery are not. Our results show that valid cues have the greatest benefit for targets at peripheral locations, thus where parvocellular representation is sparse and a bias toward this input stream would have the greatest benefit.
However, we do not see a corresponding relationship between cueing and target eccentricity in results from our segregation task. Though cueing strongly impacts performance in the segregation task, there is no reliable impact of target eccentricity on cueing. This suggests that endogenous cueing effects on temporal integration and segregation are unlikely to act solely through a weighting of parvocellular and magnocellular input.
How else might endogenous spatial attention benefit apparently-opposing temporal processes of segregation and integration? One possibility is that endogenous spatial attention might flexibly adapt the size of rapidly sampled perceptual moments. The idea that perception relies on “temporal windows” has a long history in psychological research, supported by studies of psychophysics (Allport, 1968; Shallice, 1964; Stroud, 1955), optical illusions (Simpson, Shahani, & Manahilov, 2005; VanRullen & Koch, 2003), somatosensory processing (Baumgarten, Schnitzler, & Lange, 2015), sensorimotor synchronization (Mates et al., 1994), and human electrophysiology (Samaha & Postle, 2015; VanRullen, 2016). Previous studies using this variant of the missing dot task suggest that integration windows usefully describe the temporal processing in the integration and segregation task. First, performance alternates over time between better integration versus better segregation: when segregation is better than average, integration is worse, and vice-versa (Wutz et al., 2016). Second, participants strategically increase the instantaneous frequency of their alpha oscillations shortly before segregation trials, compared to integration trials (Wutz, Melcher, & Samaha, 2018). Third, the phase of particular neural oscillations, when the stimulus appears, predicts whether two stimuli are integrated or segregated (Ronconi et al., 2017; Wutz et al., 2016). Together, these studies suggest that temporal integration depends on whether or not two sequential stimuli fall within the same temporal window in sensory processing (see also: Cecere, Rees, & Romei, 2015; Samaha & Postle, 2015).
Figure 6 illustrates how the size of temporal windows might be adapted to match task requirements. Two stimuli presented rapidly one after the other can lead to a percept of either one or two unique events. If they fall within the same temporal window (Fig. 6A), a single stimulus is perceived. There is evidence suggesting the result can be a single integrated percept – as opposed to a preferential processing of the first stimulus over the second – even when there is no benefit of integration, for example in reversal of reported target order in rapid serial visual presentations (Akyürek et al., 2012; Akyürek & Wolff, 2016). In other circumstances, a pair of stimuli separated by the same delay are perceived as two distinct percepts if they fall in different temporal windows, which can occur both on account of the phase of sampling of the visual environment (Fig. 6B) or the size of these windows (Fig. 6C). Applying this concept to the present findings, attentional processing may have operated by biasing the system to sample the visual environment with shorter temporal windows when the task required segregation of temporally separate stimuli, but by biasing the system to longer temporal windows when success depended on integration of temporally disparate stimuli (Wutz et al., 2016; Wutz et al., 2018). In this way the temporal window size could influence the perceptual outcome, independent of any effects (such as gain or tuning) on the low-level representation of the stimulus. However, it is important to note that while our results are consistent with this model, they do not demonstrate its necessity: other accounts are equally possible.
In the experimental paradigm employed in the current study, spatial cues provided temporal information as well as spatial information, and this temporal cueing may have impacted task performance independently of the spatial cueing effect. Consistent with this possibility, Correa and colleagues (2006) reported a benefit to temporal resolution from temporal cueing using a version of the temporal order judgement paradigm, which requires participants to segregate rapid sequential stimuli in order to report their order of presentation. One possibility is that valid cues in our design may have caused the “phase” of the temporal windows to be adapted to current task goals (Ronconi et al., 2017a,b; Wutz et al., 2016). For example, when performance required integration, alignment of the onset of a temporal window to the onset of the stimulus could have increased the likelihood of sequential stimuli falling within the same window, boosting the magnitude of the cueing effect through the synergistic allocation of spatiotemporal attention. This concept fits with the wealth of literature indicating that the ongoing phase of neural oscillations – particularly of those at lower frequencies such as the alpha band – is important in many aspects of visual perception, from detection and selection (Milton & Pleydell-Pearce, 2016; Varela et al., 1981), to causality judgments and conscious updating (Chakravarthi & Vanrullen, 2012; Cravo et al., 2015). In future experiments it will be important to separate these influences on temporal processing.
Given that our experiments involve the detection of rapidly presented stimuli with varying duration, it is important that we consider the possible role of luminance-duration reciprocity in generating our results. Bloch’s law of temporal summation suggests that within a range of short stimulus durations, perceived stimulus energy will be defined by the combination of luminance and duration (Block, 1885). This raises the possibility that participants may have completed our integration task by monitoring for circles that were perceived as being completed, but with lower stimulus intensity (because of the temporal gap in presentation of the two circle halves). However, we do not think this is likely. First, each independent circle segment was presented for an equal duration: two segments could appear together for 10 ms, or each circle half could appear in a separate 10-ms interval. For this difference to impact the perceived luminance of the completed circle, the two circle halves would have to be integrated into a single circle object before summated stimulus energy was computed by the visual system. To our knowledge, there is no evidence that this occurs in the visual system, and results suggests that the integration of form across spatially separated stimuli occurs later than the integration of luminance and duration (e.g., Kahneman, Normal, & Kubovy, 1967). Second, the circle segments employed in our work were relatively large, high-contrast stimuli that, in Experiments 3 and 4, were consistently separated by an ISI of 70 ms. This is beyond the critical interval in which luminance-duration reciprocity occurs for supra-threshold stimuli (e.g., Davy, 1952; Gorea & Tyler, 1986).
To conclude, the current study provides evidence that endogenously cued spatial attention has much the same benefit to integration and segregation, two opposing temporal processes. This distinguishes endogenous attention from the exogenous capture of attention, which has been found to degrade temporal processing. Our results constrain theoretical models of selective attention and are broadly in line with the developing idea that attention in time may act through the adaptation of temporal windows.
Akyürek, E. G., Eshuis, S. A. H., Nieuwenstein, M. R., Saija, J. D., Başkent, D., & Hommel, B. (2012). Temporal target integration underlies performance at lag 1 in the attentional blink. Journal of Experimental Psychology: Human Perception and Performance, 38(6), 1448–1464. https://doi.org/10.1037/a0027610.
Akyürek, E. G., & Van Asselt, E. M. (2015). Spatial attention facilitates assembly of the briefest percepts: Electrophysiological evidence from color fusion. Psychophysiology, 52(12), 1646-1663. https://doi.org/10.1111/psyp.12523.
Akyürek, E. G., & Wolff, M. J. (2016). Extended temporal integration in rapid serial visual presentation: Attentional control at Lag 1 and beyond. Acta Psychologica, 168, 50–64. https://doi.org/10.1016/j.actpsy.2016.04.009.
Allport, D. A. (1968). Phenomenal simultaneity and the perceptual moment hypothesis. British Journal of Psychology, 59(4), 395–406. https://doi.org/10.1111/j.2044-8295.1968.tb01154.x.
Azzopardi, P., Jones, K. E., & Cowey, A. (1999). Uneven mapping of magnocellular and parvocellular projections from the lateral geniculate nucleus to the striate cortex in the macaque monkey. Vision research, 39(13), 2179-2189. https://doi.org/10.1016/S0042-6989(98)00319-8.
Bakeman, R. (2005). Recommended effect size statistics for repeated measures designs. Behaviour Research Methods, 37 (3), 379-384. https://doi.org/10.3758/BF03192707.
Baumgarten, T. J., Schnitzler, A., & Lange, J. (2015). Beta oscillations define discrete perceptual cycles in the somatosensory domain. Proceedings of the National Academy of Sciences of the United States of America, 112(39), 12187–92. https://doi.org/10.1073/pnas.1501438112.
Bloch, A. M. Experiences sur la vision. Soc. Biol. Mem. (Paris). 1885, 27, 493–495.
Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10(4), 433–6. https://doi.org/10.1163/156856897X00357.
Bush, W. S., & Vecera, S. P. (2014). Differential effect of one versus two hands on visual processing. Cognition, 133(1), 232-237. https://doi.org/10.1016/j.cognition.2014.06.014.
Carrasco, M., Loula, F., & Ho, Y.-X. (2006). How attention enhances spatial resolution: evidence from selective adaptation to spatial frequency. Perception & Psychophysics, 68(6), 1004–1012. https://doi.org/10.3758/BF03193361.
Cecere, R., Rees, G., & Romei, V. (2015). Individual differences in alpha frequency drive crossmodal illusory perception. Current Biology, 25(2), 231-235. https://doi.org/10.1016/j.cub.2014.11.034.
Chakravarthi, R., & Vanrullen, R. (2012). Conscious updating is a rhythmic process. Proceedings of the National Academy of Sciences of the United States of America, 109(26), 10599–604. https://doi.org/10.1073/pnas.1121622109.
Chica, A. B., Bartolomeo, P., & Lupiáñez, J. (2013). Two cognitive and neural systems for endogenous and exogenous spatial attention. Behavioural brain research, 237, 107-123.
Correa, Á, Sanabria, D., Spence, C., Tudela, P., & Lupiáñez, J. (2006). Selective temporal attention enhances the temporal resolution of visual perception: Evidence from a temporal order judgment task. Brain Research, 1070(1), 202-205. https://doi.org/10.1016/j.brainres.2005.11.094.
Connolly, M., & Van Essen, D. (1984). The representation of the visual field in parvicellular and magnocellular layers of the lateral geniculate nucleus in the macaque monkey. Journal of Comparative Neurology, 226(4), 544-564. https://doi.org/10.1002/cne.902260408.
Cravo, A. M., Santos, K. M., Reyes, M. B., Caetano, M. S., & Claessens, P. M. E. (2015). Visual Causality Judgments Correlate with the Phase of Alpha Oscillations. Journal of Cognitive Neuroscience, 27(10), 1887–1894. https://doi.org/10.1162/jocn_a_00832.
Davy, E. (1952). The intensity-time relation for multiple flashes of light in the peripheral retina. JOSA, 42(12), 937-941. https://doi.org/10.1364/JOSA.42.000937.
Deco, G., Tononi, G., Boly, M., & Kringelbach, M. L. (2015). Rethinking segregation and integration: contributions of whole-brain modelling. Nature Reviews. Neuroscience, 16(7), 430–439. https://doi.org/10.1038/nrn3963.
Faul, F., Erdfelder, E., Lang, A-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioural, and biomedical sciences. Behavior Research Methods, 39, 175-191. https://doi.org/10.3758/BF03193146.
Goodhew, S., Shen, E., & Edwards, M. (2015). Selective spatial enhancement: Attentional spotlight sizes impacts spatial but not temporal perception. Journal of vision, 15(12), 1239-1239. https://doi.org/10.3758/s13423-015-0904-6.
Gorea, A., & Tyler, C. W. (1986). New look at Bloch’s law for contrast. JOSA A, 3(1), 52-61.
Hein, E., Rolke, B., & Ulrich, R. (2006). Visual attention and temporal discrimination: Differential effects of automatic and voluntary cueing. Visual Cognition, 13(1), 29-50. https://doi.org/10.1080/13506280500143524.
Hogben, J. H., & Di Lollo, V. (1974). Perceptual integration and perceptual segregation of brief visual stimuli. Vision Research, 14(11), 1059–1069. https://doi.org/10.1016/0042-6989(74)90202-8.
Kahneman, D., Norman , J., & Kubovy, M. (1967). Critical duration for the resolution of form: Centrally or peripherally determined? Journal of Experimental Psychology, 73 , 323-327. https://doi.org/10.1037/h0024257.
Klein, R. M. (1994). Perceptual-motor expectancies interact with covert visual orienting under conditions of endogenous but not exogenous control. Canadian Journal of Experimental Psychology, 48(2), 167. https://doi.org/10.1037/1196-19126.96.36.199.
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in psychology, 4. https://doi.org/10.3389/fpsyg.2013.00863.
Livingstone, M. S., & Hubel, D. H. (1987). Psychophysical evidence for separate channels for the perception of form, color, movement, and depth. Journal of Neuroscience, 7(11), 3416-3468.
Luck, S. J., Girelli, M., McDermott, M. T., & Ford, M. A. (1997). Bridging the gap between monkey neurophysiology and human perception: An ambiguity resolution theory of visual selective attention. Cognitive psychology, 33(1), 64-87. https://doi.org/10.1006/cogp.1997.0660.
Mates, J., Müller, U., Radil, T., & Pöppel, E. (1994). Temporal Integration in Sensorimotor Synchronization. Journal of Cognitive Neuroscience, 6(4), 332–340. https://doi.org/10.1162/jocn.19188.8.131.522.
Milton, A., & Pleydell-Pearce, C. W. (2016). The phase of pre-stimulus alpha oscillations influences the visual perception of stimulus timing. NeuroImage, 133, 53–61. https://doi.org/10.1016/j.neuroimage.2016.02.065.
Morey, R. D. (2008). Confidence Intervals from Normalized Data: A correction to Cousineau ( 2005 ). Tutorials in Quantitative Methods for Psychology, 4(2), 61–64. https://doi.org/10.3758/s13414-012-0291-2.
Pöppel, E., Aschoff, J., Bao, Y., Pöppel, E., Bao, Y., Zhou, J., … Zhou, B. (2009). Pre-semantically defined temporal windows for cognitive processing. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 364(1525), 1887–96. https://doi.org/10.1098/rstb.2009.0015.
Posner, M. I. (1980). Orienting of attention. The Quarterly Journal of Experimental Psychology, 32(1), 3-25. https://doi.org/10.1080/00335558008248231.
Rolke, B., Dinkelbach, A., Hein, E., & Ulrich, R. (2008). Does attention impair temporal discrimination? Examining non-attentional accounts. Psychological Research, 72(1), 49-60. https://doi.org/10.1007/s00426-006-0092-0.
Ronconi, L., Oosterhof, N. N., Bonmassar, C., & Melcher, D. (2017). Multiple oscillatory rhythms determine the temporal organization of perception. Proceedings of the National Academy of Sciences, 114(51), 13435-13440. https://doi.org/10.1073/pnas.1714522114.
Samaha, J. & Postle, B. R. (2015). The Speed of Alpha-Band Oscillations Predicts the Temporal Resolution of Visual Perception Report The Speed of Alpha-Band Oscillations Predicts the Temporal Resolution of Visual Perception. Current Biology, 25(22), 2985-2990. https://doi.org/10.1016/j.cub.2015.10.007.
Shallice, T. (1964). The detection of change and the perceptual moment hypothesis. British Journal of Statistical Psychology, 17(2), 113–135. https://doi.org/10.1111/j.2044-8317.1964.tb00254.x.
Simpson, W. A., Shahani, U., & Manahilov, V. (2005). Illusory percepts of moving patterns due to discrete temporal sampling. Neuroscience Letters, 375(1), 23-27. https://doi.org/10.1016/j.neulet.2004.10.059.
Spence, C., & Parise, C. (2010). Prior-entry: A review. Consciousness and cognition, 19(1), 364-379. https://doi.org/10.1016/j.concog.2009.12.001.
Stroud, J. M. (1955). The fine structure of psychological time. In H. Quastler (Ed.), Information Theory in Psychology. Glencoe, III: Free Press.
Varela, F. J., Toro, A., Roy John, E., & Schwartz, E. L. (1981). Perceptual framing and cortical alpha rhythm. Neuropsychologia, 19(5), 675–686. https://doi.org/10.1016/0028-3932(81)90005-1.
VanRullen, R. (2016). Perceptual Cycles. Trends in Cognitive Sciences, 20(10), 723-735.https://doi.org/10.1016/j.tics.2016.07.006.
VanRullen, R., & Koch, C. (2003). Is perception discrete or continuous? Trends in Cognitive Sciences, 7(5), 207–213. https://doi.org/10.1016/S1364-6613(03)00095-0.
Visser, T. A. W., & Enns, J. T. (2001). The role of attention in temporal integration. Perception, 30(2), 135–145. https://doi.org/10.1068/p3089.
Wutz, A., Melcher, D., & Samaha, J. (2018). Frequency modulation of neural oscillations according to visual task demands. Proceedings of the National Academy of Sciences, 201713318. https://doi.org/10.1073/pnas.1713318115.
Wutz, A., Muschter, E., van Koningsbruggen, M. G., Weisz, N., & Melcher, D. (2016). Temporal Integration Windows in Neural Processing and Perception Aligned to Saccadic Eye Movements. Current Biology, 26(13), 1659–1668. https://doi.org/10.1016/j.cub.2016.04.070.
Yeshurun, Y. (2004). Isoluminant stimuli and red background attenuate the effects of transient spatial attention on temporal resolution. Vision Research, 44(12), 1375–1387. https://doi.org/10.1016/j.visres.2003.12.016.
Yeshurun, Y., & Carrasco, M. (1999). Spatial attention improves performance in spatial resolution tasks. Vision Research, 39(2), 293–306. http://doi.org/S004269899800114X.
Yeshurun, Y., & Hein, E. (2011). Transient attention degrades perceived apparent motion. Perception, 40(8), 905-918. https://doi.org/10.1068/p7016.
Yeshurun, Y., & Levy, L. (2003). Transient spatial attention degrades temporal resolution. Psychological Science, 14(3), 225–231. https://doi.org/10.1111/1467-9280.02436.
This research was supported by the Autonomous Province of Trento, Call “Grandi Progetti 2012,” project “Characterizing and improving brain mechanisms of attention—ATTEND”. David Melcher was supported by the European Research Council (ERC grant agreement no. 313658). We would like to thank Claudia Bonmassar and Francesca Bonetti for advice on eye-tracking.
About this article
Cite this article
Sharp, P., Melcher, D. & Hickey, C. Endogenous attention modulates the temporal window of integration. Atten Percept Psychophys 80, 1214–1228 (2018). https://doi.org/10.3758/s13414-018-1506-y
- Temporal processing