Different effects of spatial and temporal attention on the integration and segregation of stimuli in time

Abstract

Having expectations about when and where relevant stimuli will appear engenders endogenous temporal and spatial orienting and can provide vital benefits to visual processing. Although more is known about how each of these forms of orienting affects spatial processing, comparatively little is understood about their influences on the temporal integration and segregation of rapid sequential stimuli. A critical question is whether the influence of spatial cueing on temporal processing involves independent spatial and temporal orienting effects or a synergistic spatiotemporal impact. Here we delineated between the temporal and spatial orienting engendered by endogenous cues by using a paradigm with identical visual stimulation when the goal was to integrate or segregate the stimuli, in separate blocks of trials. We found strong effects of spatial orienting on both integration and segregation performance. In contrast, temporal orienting engendered only an invalid cueing cost, and for integration trials only. This clear differentiation between spatial and temporal cueing effects provides constraints to inform arbitration between theories of how attention biases the visual processing stream and influences the organization of visual perception in time.

In complex and dynamic visual environments, processing can be optimized by attending to where and when relevant information will appear. Endogenous cues can prompt spatial or temporal orienting independently, but often they elicit both at the same time. Research has generally focused on how these different types of orienting affect spatial processing. However, an additional, critical challenge for the visual system is to parse the incoming sensory input into coherent objects and events over time, binding disparate sensory signals together (integration) as well as distinguishing between subsequent sensory inputs that reflect two unique objects/events (segregation). Our understanding of the influence of spatial and temporal cues on such temporal processing remains sparse.

In studies of spatial processing, there are similarities between how spatial and temporal orienting affect vision. For example, Coull and Nobre (1998) demonstrated that valid endogenous cues indicating either when or where a target would appear had qualitatively similar benefits on target detection. Spatial orienting to a particular location after a valid cue provided a larger performance benefit than temporal orienting to a particular interval, but both had benefits. Further investigations also using endogenous cues confirmed this pattern of valid cueing effects on spatial discrimination, for both spatial and temporal orienting (Griffin, Miniussi, & Nobre, 2002; Olk, 2014). Similarly, in a oddball detection task using auditory stimuli, both spatial and temporal orienting were found to benefit spatial processing performance (Lange, Krämer, & Röder, 2006). Although there is evidence of dissociations between the effects on performance of spatial and temporal orienting following exogenous cueing (Ahrens, Veniero, Gross, Harvey, & Thut, 2015; Jones, 2014), when attentional shifts are endogenously driven, the evidence points to similarities, and even interacting performance benefits, from spatial and temporal orienting. Rohenkohl, Gould, Pessoa, and Nobre (2014) reported that benefits to spatial discrimination performance are afforded by both spatial and temporal orienting, in addition to characterizing the interaction between these two types of orienting. Whereas spatial orienting benefits were boosted further by valid temporal expectations, in the case of temporal orienting, performance benefits were extinguished if spatial expectations were misleading. There is also evidence that temporal expectations can combine synergistically with other forms of expectations—for example, with event-specific information about stimulus identity (Langner, Steinborn, Eickhoff, & Huestegge, 2018).

Shifting focus from studies of different types of orienting on spatial processing to those pertaining to temporal processing, we note that these studies have typically featured paradigms in which stimuli appear after a fixed interval from the onset of endogenous spatial cues. Because these cues implicitly provide temporal information, this introduces some ambiguity of interpretation: Is performance impacted purely by information about the location of the target, or is it important that the cue also provides information about timing? For example, Hein, Rolke, and Ulrich (2006) demonstrated that a valid cue indicating the spatial location of two rapid sequential stimuli facilitates segregation of the stimuli, such that participants were able to discriminate which of them appeared first. However, the onset of the cue employed in this work also provided information about when the stimuli would appear, opening the possibility that temporal orienting mechanisms might also have been recruited to facilitate performance. Similarly, we have demonstrated that spatial cueing affects both the segregation and integration of rapid sequential stimuli, depending on which of these opposing temporal processes is required for successful target detection. This was achieved using a paradigm with two forms, whereby the visual stimulation remained the same while the task goals were manipulated (Sharp, Melcher, & Hickey, 2018). Again, the spatial cues in that paradigm could have engendered temporal orienting, since the interval between cue and stimulus onset was of a fixed duration. It therefore remains unclear whether the spatial cueing effects on temporal processing are driven by spatial orienting alone or via a combined, synergistic influence of spatial and temporal orienting combined.

Under normal circumstances, spatial and temporal orienting often co-occur, since we often have access to combined spatiotemporal expectations about the visual environment. Indeed, unique objects or events are defined by their spatiotemporal conjunction. Perception of an object in motion is an obvious example of this maxim. Doherty, Rao, Mesulam, and Nobre (2005) tested whether implicit spatiotemporal expectations about the trajectory of a moving object that disappeared behind an occluder influenced participants’ speed in detecting a dot on the object when it reappeared. Both spatial and temporal expectations improved performance, as compared to the control condition. Crucially, combined spatiotemporal expectations had a synergistic effect, leading to faster reaction times than in either the purely spatial or the purely temporal orienting condition. It is important to note that perception of motion is a salient feature that, by definition, relies on combined spatiotemporal information, raising a question about the generality of this finding for other forms of temporal processing, and leaving open the question of whether the facilitatory effects of valid spatial cues on temporal integration and segregation are in part due to the combined influence of spatial and temporal orienting.

A secondary focus of the present study was on the remaining uncertainty regarding how purely temporal orienting impacts segregation and integration processes. A classical finding concerning the effects of temporal orienting on spatial processing was demonstrated by Los and Van Den Heuvel (2001). In this study, a symbolic temporal cue indicated the delay after which an “L” or “R” target would appear, requiring a binary choice-response with the spatially congruent hand. The results from this “foreperiod task” led the authors to conclude that symbolic temporal cues can engender strategic temporal orienting. This added to a literature demonstrating that valid temporal expectations impact not only auditory perception, typically when using reaction time as a metric (Griffin & Nobre, 2005), but also the detection of targets in a rapid stream of visual stimuli (Correa, Lupiáñez, Milliken, & Tudela, 2004; Miller & Schröter, 2002). Looking rather at the effect of temporal orienting on temporal processing, Correa, Sanabria, Spence, Tudela, and Lupiáñez (2006) employed a temporal order judgment paradigm to probe the influence of endogenous temporal cueing on the perception of rapid sequential visual stimuli. Temporal orienting engendered by the cue was found to improve participants’ ability to report which stimulus appeared first. A further consideration in this context is the relationship of temporal cues and hazard rates: stimuli that appear earlier than was indicated by a cue are surprising, whereas those that appear later than was cued are better expected (Correa et al., 2004; Nobre & van Ede, 2018). Temporal orienting paradigms therefore often employ catch trials in which no stimulus appears after the cue, to modulate the function describing this increasing probability that a stimulus will appear (Correa et al., 2006; Steinborn, Rolke, Bratzke, & Ulrich, 2008).

Here we tested whether spatial cues impact temporal integration and segregation independent of any temporal expectations, and whether temporal cues impact temporal integration and segregation when no spatial expectations are provided by the cueing. Our paradigm features manipulation of the task goal—either integration or segregation of visual events—across blocks of experimental trials. Importantly, visual stimulation remains constant, with endogenous cues predicting either where or when the upcoming target will likely arrive. For purely spatial orienting, we expected to find a performance benefit of valid spatial cueing and a performance cost of invalid cueing for both integration and segregation, consistent with our earlier work (Sharp et al., 2018). In the case of temporal orienting the outcome is less clear, and we did not have strong predictions. One possible outcome would be temporal orienting effects similar to those found with spatial orienting. Alternatively, temporal orienting might be particularly useful for rapid temporal segregation, allowing greater precision in time. A third possibility is that any temporal integration might be more demanding in terms of temporal coding, and thus be more influenced by the temporal cue, given the requirement to encode both of the stimuli across time.

Method

Participants

Twenty-one healthy participants (17 female, four male; age = 22.14 ± 2.94 years, mean ± SD) gave informed consent before completing the experiment. All reported normal or corrected-to-normal vision and right-handedness. Participants provided informed consent in accordance with the Declaration of Helsinki, and approval for the study was granted by the ethics committee of the University of Trento.

Experimental design and stimuli

The stimuli and task were generated with Psychophysics Toolbox (Brainard, 1997) in MATLAB (MathWorks). The stimuli were presented on an LED-backlit monitor designed for psychophysics stimulus presentation (VPixx Technologies) with a 100-Hz refresh rate. The experiment used a 3×2×2 fully within-subjects design: three levels of cue validity (valid, neutral, invalid) × two task versions (segregation, integration) × two types of orienting (spatial, temporal). An overview of the design is shown in panel B of Fig. 1. The experimental task was varied by block (alternating between segregation and integration); the cue validity was varied fully randomly by trial, and the type of orienting indicated by the cue varied across the two separate sessions (counterbalanced such that half of the participants completed the temporal orienting session first).

Fig. 1
figure1

Paradigm and design. (A) Trial structure. In the spatial-orienting session, the cue indicated where the target would likely appear. In this example, the spatial cue indicates that the target will likely appear in the top right quadrant of the display. In the temporal orienting session, the cue indicated when the target would likely appear, after either a short or a long delay. In neutral cue trials, all four tips of the cross turned green. If this trial were in a segregation block, the target would be the location second-down on the far right (the half-circle). If this trial were in an integration block, the target would be the location in the near-left column where no stimulus appeared in either display. In the spatial orienting session, the cue interval was jittered between 350 and 850 ms; in the temporal orienting session, this interval was either 350 or 850 ms. In a preexperiment, the ISI varied (see the Method section), whereas in the main experimental blocks it was fixed at 50 ms. (B) Overview of the full within-subjects design (see the Method section). The segregation target was the half circle, and the integration target was the missing circle

The basic trial structure is shown in Fig. 1. A small fixation cross in the form of a red “X” was present throughout presentation of the stimuli; this cross measured approximately 0.2° (of visual angle). For the spatial orienting session, at the beginning of each trial, one of the arms of the cross changed from red to green, to provide the cue that the target for that trial would likely appear in the indicated quadrant. For the temporal orienting session, the cue indicated whether the displays featuring the target would likely appear after a short or a long delay (350 or 850 ms). The form of the cue was a green cross with two sections filled in, resembling an hourglass either upright or on its side (see Fig. 1), with the pairing between cue and delay counterbalanced across participants. Participants were instructed that the cue was valid around 75% of the time; thus, in approximately 22% of total trials the cue was invalid, in 53% of trials the cue was valid, and in 25% of the trials a neutral cue was shown. The neutral cue took the form of a color change of the tips of all four arms of the cross, such that a similar number of pixels changed from red to green in the neutral and in the valid/invalid cueing conditions. In 10% of trials, the cross turned blue after the maximum cue interval of 850 ms, and no target or displays were presented. These catch trials were included so as to mitigate the influence of a rising hazard rate as time elapsed during the cue interval (Janssen & Shadlen, 2005; Steinborn et al., 2008).

In the spatial orienting session, the duration of the cue interval had a jitter from a square distribution between 350 and 850 ms, meaning that the cue did not give any information about the timing of the stimulus onset. In the temporal orienting session, the cue interval was either 350 or 850 ms and gave no information about the spatial location of the target. After the cue interval, the fixation cross became entirely red again, and the first display appeared on screen for 10 ms. The two frames comprised an odd-element (OE) or missing-dot (MD) task, as had been used previously to measure both temporal integration and segregation using the same stimulus (Sharp et al., 2018; Wutz, Muschter, van Koningsbruggen, Weisz, & Melcher, 2016). This display comprised circles at seven locations out of a possible 16 (in a 4×4 grid of locations), and each circle was formed from two arc elements (see Fig. 1: “Display 1”). At one additional position in the display was a half circle (i.e., a single arc). Each circle measured 1.2° (of visual angle) in diameter, and the grid of possible locations measured 8.4° × 8.4°. Each circle within the display had an irrelevant random orientation of the breaks between the arc elements that formed it, selected from the possible orientations of 45°–315°.

Following a fixed interstimulus interval (ISI) of 50 ms, a second display appeared for 10 ms (see Fig. 1: “Display 2”). This display also comprised circles at seven locations and a half circle at one location. Crucially, the half circle in Display 2 was the corresponding arc element at the same location as the half circle location from Display 1, such that if the two displays were superimposed, the two arc elements would form a standard circle stimulus. The locations of the seven circles for each display never overlapped, such that if the two displays were superimposed, only one of the 16 possible locations would remain empty. This hypothetical superimposition is illustrated in Fig. 1 (top right image). After 500 ms, a response probe screen appeared. This comprised a grid of squares in which each square identified one of the 16 possible target locations. Participants indicated, by mouse click, at which location they had perceived the target.

Task

Participants completed two sessions for the experiment (separated by a minimum of one day and a maximum of ten days). On separate days, the cue provided information either about the location of the target, in one session (spatial session), or about when the displays featuring the target would appear, in the other session (temporal session). The order of sessions was counterbalanced across participants. There were two versions of the task goal, such that the stimulus presentation was the same, and only the task instruction differed blockwise. In one version of the task, the target was the half circle. Successful identification of the half circle required temporal parsing of the two displays, and therefore this is referred to as the segregation task. Integration of the two displays would result in a combined percept (see Fig. 1) in which the half circle location was indistinguishable from the other locations.

In the other version, the target was the location that contained no circle in either display. Successful identification of this location required a combined percept of the two displays; therefore, this was referred to as the integration task. Participants were explicitly instructed to fixate the cross in the center of the screen throughout stimulus presentation.

In the first session, prior the main experimental blocks, participants completed a version of the paradigm with no cueing and multiple ISIs, to identify baseline performance. In this screening task, participants completed five practice trials for each of the two versions of the task, followed by two blocks of 80 trials for analysis (one block for each task version, each block comprising ten trials for each of the eight ISIs). The screening results were used to exclude participants whose difference in performance for the different versions of the uncued task constituted an outlier (described below).

Each experimental session began with 25 practice trials with cueing for each task version (50 total), which were repeated if accuracy on either task version was low (< 25%); one participant was excluded for not being able to attain this level of performance for the segregation task after four attempts. For each of the two sessions, participants completed ten blocks of the main experiment, in which each block comprised 60 full trials plus seven catch trials. Practice trials and catch trials were excluded from the main analysis.

Behavioral data analysis

The behavioral data were analyzed using Matlab2013a (MathWorks, Natick, MA) and R (R Core Team, 2017). To identify outliers, we used data from the screening task, which featured multiple ISIs and no cueing. We calculated a least square fit for the results from the screening task for each participant, estimating the intersect between tasks (i.e., the ISI at which performance on the segregation and integration tasks was equal). We excluded participants when this intersection was more than two standard deviations from the group mean. This, along with the exclusion of a participant who could not achieve reasonably performance on the practice for the segregation version, resulted in the exclusion of two participants (10% exclusion rate).

For data from the main blocks of the spatial-orienting session, the trials were binned by duration of the cue interval, and only data from the short-interval (350–500 ms) and long-interval (600–850 ms) bins were used for the main analysis (in the temporal-orienting session, the cue interval was always either 350 or 850 ms).

As was recommended by Lakens (2013), two estimates of effect size are provided: partial eta-squared and generalized eta-squared. The former is better suited to power analyses and comparisons within a study, whereas the latter is a metric more suited to comparisons across experiments (Bakeman, 2005). Post hoc testing was conducted by Bonferroni-corrected pairwise comparisons. For these tests, we report Cohen’s dz, which is a measure of effect size for repeated measures tests.

Results

Figure 2A shows that in the preexperiment with no cueing, participants performed better at integration when the ISI was shorter, and better at segregation when the ISI was longer, as expected (Sharp et al., 2018; Wutz et al., 2016). The mean intersection at which performance was matched for integration and segregation was close to the fixed ISI of 50 ms used for the main experimental blocks.

Fig. 2
figure2

Main findings. (A) Results of the preexperiment task. The dotted vertical line indicates the fixed ISI used for the main blocks. The error bars here and in all panels indicate within-subjects 95% confidence intervals (Morey, 2008). The mean intersect error bars indicate the SEM. (B) Results of the main experiment for spatial orienting: Accuracy as a function of task and validity of the cue. (C) Results of the main experiment for temporal orienting: Accuracy as a function of task and validity of the cue

For the data from the main experiment, a repeated measures analysis of variance (RANOVA) was conducted, with the factors task (segregation, integration), cue validity (valid, invalid), and type of orienting (spatial, temporal). As in previous experiments, significant main effects were found for task [F(1, 18) = 57.12, p < .001, ηp2 = .760, ηG2 = .502] and cue validity [F(2, 36) = 137.25, p < .001, ηp2 = .884, ηG2 = .547], and a significant interaction was found between task and cue validity [F(2, 36) = 61.17, p < .001, ηp2 = .773, ηG2 = .137]. The type of orienting (spatial vs. temporal) had a significant main effect on performance [F(1, 18) = 7.17, p = .015, ηp2 = .285, ηG2 = .018], and both interactions of task and type of orienting [F(1, 18) = 50.67, p < .001, ηp2 = .738, ηG2 = .116] and cue validity and type of orienting [F(2, 36) = 48.23, p < .001, ηp2 = .728, ηG2 = .255] were observed, as well as a significant three-way interaction [F(2, 36) = 11.30, p = .001, ηp2 = .386, ηG2 = .024].

As can be seen in Fig. 2B, strong spatial cueing effects emerged in both tasks, as confirmed by the results of a follow-up RANOVA confined to results from the spatial cueing conditions, which revealed significant main effects of task [F(1, 18) = 18.88, p < .001, ηp2 = .512, ηG2 = .244] and cue validity [F(2, 36) = 97.27, p < .001, ηp2 = .844, ηG2 = .688], as well as a significant interaction [F(2, 36) = 18.84, p < .001, ηp2 = .511, ηG2 = .091]. Post hoc testing revealed that spatial cueing effects were driven by benefits of valid cueing for both segregation [t(18) = 4.17, p < .001, dz = 0.93] and integration [t(18) = 9.06, p < .001, dz = 2.03], as well as costs of invalid cueing for both segregation [t(18) = 5.76, p < .001, dz = 1.29] and integration [t(18) = 12.41, p < .001, dz = 2.78].

The pattern of effects for temporal cueing was remarkably different. A follow-up RANOVA confined to results from the temporal cueing conditions revealed main effects of task [F(1, 18) = 100.32, p < .001, ηp2 = .848, ηG2 = .722] and cue validity [F(2, 36) = 100.40, p < .001, ηp2 = .848, ηG2 = .83], as well as a significant interaction [F(2, 36) = 85.71, p < .001, ηp2 = .826, ηG2 = .251]. However, as can be seen in Fig. 2C, the pattern of results in the temporal cueing condition was quite different from that observed in the spatial cueing condition. Post hoc testing revealed a significant cost of invalid cueing to performance in the integration task only [t(18) = 11.49, p < .001, dz = 2.57], with no effect of invalid cueing for segregation [t(18) = 1.13, p = .14, dz = 0.25], and no valid cueing effect for either segregation [t(18) = .61, p = .73, dz = 0.14] or integration [t(18) = 1.35, p = .10, dz = 0.30].

Given that the primary research question of the study was whether the effects of spatial cueing on temporal processing would be found not only when spatiotemporal cues were used (as in previous work), but also when purely spatial cues were used (as in the present study), we compared the present results with previous findings. Following Verhagen and Wagenmakers (2014), we conducted a Bayes factor replication test, taking as the input t and n values, to quantify the evidence in favor of a successful replication of the spatial cueing effects from our earlier study (Sharp et al., 2018). By comparing cueing effects between two experiments with near-identical methodologies that differed in terms of whether the cues were spatiotemporal or purely spatial in nature, we found extreme evidence in favor of a successful replication of cueing effects on temporal processing for all comparisons: valid cueing on segregation (BF > 100), invalid cueing on segregation (BF > 100), valid cueing on integration (BF > 100), and invalid cueing on segregation (BF > 100).

To investigate more closely the nature of the temporal cueing effect, we analyzed the data as a function of the delay between cue and stimulus onset (Fig. 3). To test whether the main findings showed any interaction with the delay between cue and stimulus onset, an additional ANOVA was conducted with the factors task (segregation/integration), cue (valid/neutral/invalid), orienting (spatial/temporal), and delay between cue and stimulus onset (short/long). In addition to the effects identified above, this analysis revealed a main effect of delay [F(1, 18) = 19.90, p < .001, ηp2 = .525, ηG2 = .006], with a small but reliable benefit to performance at longer cue intervals. An interaction between task and delay [F(1, 18) = 28.98, p < .001, ηp2 = .726, ηG2 = .013] was driven by better integration performance at the long delay, and an interaction between cue validity and delay [F(2, 36) = 6.64, p = .003, ηp2 = .014, ηG2 = .005] reflected slightly stronger cueing effects at a short delay (see Fig. 3). Importantly, however, the four-way interaction was not significant [F(2, 36) = 1.00, p = .3, ηp2 < .0001, ηG2 = .0006], indicating that the critical three-way interaction observed in the main analysis was not influenced by delay. No other effects emerged [task by orienting by delay: F(1, 18) = 3.12, p = .1, ηp2 = .020, ηG2 = .0007; all other F values < 1].

Fig. 3
figure3

Results of the main experiment, split by the delay between cue and stimulus onset. (A) Spatial orienting: left, short delay; right, long delay. Error bars here and in panel B indicate within-subjects 95% confidence intervals (Morey, 2008). (B) Temporal orienting: left, short delay; right, long delay

Discussion

The present results provide evidence for a strong dissociation between spatial and temporal cueing effects in a task that involves combining or segmenting stimuli over time. Consistent with our prior work, endogenous cues providing spatial information benefited performance when they were valid, and created a cost when invalid, for both integration and segregation tasks (Sharp et al., 2018). In sharp contrast, temporal cueing provided a quite different and specific effect. Invalid temporal cues created a performance cost in the integration task, but not in segregation, and valid temporal cueing had no impact on either task (relative to neutral cues).

This study provides support for the idea that performance benefits following valid spatial cueing are driven by the action of spatial orienting mechanisms, rather than by confounded effects of temporal cueing or synergistic effects of combined spatiotemporal cues (Hein et al., 2006; Sharp et al., 2018). This finding provides an important clarification, since in previous work on the effect of spatial orienting on temporal processing, spatial cues had often also implicitly provided temporal expectations about stimulus onset. Even in the absence of temporal expectations about stimulus onset, the spatial cueing benefits to both temporal integration and segregation performance were quite large and of effect sizes similar to those in our previous work (Sharp et al., 2018). These results provide further evidence that allocation of endogenous attention is able to act in a flexible, spatially specific manner to aid both of these opposing temporal processes.

Striking differences emerged here between the influences of spatial and temporal orienting on temporal processing, ostensibly in contrast to evidence of cueing benefits for both spatial and temporal orienting in the case of spatial processing (Coull & Nobre, 1998; Griffin et al., 2002). Whereas spatial and temporal orienting have both been shown to aid spatial processing, leading to better and faster target detection (Griffin et al., 2002; Lange et al., 2006), here performance benefits were found only for spatial orienting. One possibility is that spatially specific gain modulations facilitate target detection—for example, whereas evidence suggests that spatial attention can enhance the spiking responses of cells with relevant receptive fields (Bosman & Womelsdorf, 2009), attending to a point in time would not allow this selective gain modulation of a specific group of cells. Another interpretation is that any benefits of temporal orienting may principally act on decision- and motor-related processing (Vangkilde, Coull, & Bundesen, 2012; Volberg & Thomaschke, 2017), particularly benefiting speeded responses through a temporal alignment of readiness, allowing efficient and attentive information processing at the expected time point (Steinborn, Langner, & Huestegge, 2017).

The pattern of behavioral results for temporal orienting demonstrated here is novel, and initially it appears inconsistent with previous findings on temporal cueing. Correa et al. (2006) showed a small cueing benefit to performance when participants reported the order in which two rapid sequential stimuli were presented. Here we found no corresponding benefit of valid temporal cues. One interpretation is that this apparent disparity arises due to task differences: Making a judgment about the temporal order of stimuli arguably relies more heavily on higher-level decision processes than did the target-search task employed here. Since temporal-orienting effects usually emerge when the primary measure is a metric, like reaction times, that reflects differences in processing stages further up the hierarchy, such as decision or motor processing (Langner et al., 2018; Thomaschke & Dreisbach, 2015; Vangkilde et al., 2012; Volberg & Thomaschke, 2017), this is a pertinent difference between the present study and previous work. Reaction time data were unsuitable for the analysis in the present study, due to the response collection method: unspeeded mouse clicks on a probe grid of 16 locations after a fixed delay from stimulus offset.

Consideration of details of the paradigm may also provide an explanation for the cost to integration performance from an invalid cue in the temporal orienting condition. Missing one of the rapidly presented displays is more detrimental to performance when the goal is integration than when the goal is segregation, since locating the integration target (e.g., a missing circle) requires perception of both displays. Invalid temporal cues may sometimes have caused participants to miss one of the displays. Indeed, when comparing the types of errors participants made, we saw that in the segregation task, participants clicked roughly equally often on an element of the first as of the second display, whereas in the integration task, their responses indicated a bias to click on missing elements from one display. Crucially, the pattern of whether their bias was toward the first or the second display was reversed for invalid as compared to the neutral and valid temporal cueing conditions.

Despite the inclusion of catch trials in an attempt to mitigate the influence of a growing urgency rate or hazard signal to act (Janssen & Shadlen, 2005), performance was found to improve at longer delays between cue and stimulus onset. This is consistent with work indicating that the inclusion of catch trials does not obviate the need to consider results in terms of a rising urgency to act (Steinborn et al., 2008). The small but reliable effect of delay was slightly more evident for integration than for segregation, indicating that the readiness of the system is of particular importance for temporal integration. This delay effect was slightly more pronounced in neutral than in valid cue trials, indicating that the allocation of selective attention likely compensated for the delay effect at shorter cue intervals. Critically, the three-way interaction between orienting, cue validity, and task was not influenced by these delay effects.

The present findings were not consistent with our hypothesis that temporal cueing would be particularly helpful for the ability to rapidly segment the two stimuli in time. The odd-element task (finding the half circle) required the ability to isolate the two sequential displays, such that a lack of temporal precision might be expected to reduce performance. Instead, performance in that task was particularly good for all temporal cueing conditions. One possibility is that the mere presence of a temporal cue helped participants to be diligent. Alternatively, the fact that the targets appeared at only two discrete times may have provided the opportunity for participants to effectively focus their temporal attention to both of these times. Consistent with this idea, the previous studies demonstrating an effect of temporal cueing have also tended to employ two temporal delays (Coull & Nobre, 1998; Griffin et al., 2002). However, our results demonstrated an invalid cue cost for integration trials. This is difficult to reconcile with a general diligence or split-attention interpretation. Thus, an ability of participants to focus attention at both temporal delays seems unlikely, but it cannot be completely ruled out. Future work will be needed to differentiate between these possibilities—for example, by using an adapted version of the paradigm with speeded responses (e.g., whether the target was on the left or the right), so as to be better suited to detecting temporal orienting effects.

To conclude, the pattern of results from the present study identifies a clear distinction between the effects of spatial and temporal orienting in a task involving the parsing of rapidly presented stimuli over time. Parsing the incoming sensory flow into meaningful objects and events is a key challenge for visual perception. The present findings suggest that spatial attention may increase the efficiency of this process in a task-relevant way, whereas temporal attention has, surprisingly, a comparably weaker influence on the temporal organization of visual processing.

Author note

This research was supported by the Autonomous Province of Trento, Call “Grandi Progetti 2012,” project “Characterizing and Improving Brain Mechanisms of Attention—ATTEND.” D.M. was supported by the European Research Council (ERC Grant Agreement No. 313658).

References

  1. Ahrens, M.-M., Veniero, D., Gross, J., Harvey, M., & Thut, G. (2015). Visual benefits in apparent motion displays: Automatically driven spatial and temporal anticipation are partially dissociated. PLoS ONE, 10, e0144082. https://doi.org/10.1371/journal.pone.0144082

    Article  PubMed  PubMed Central  Google Scholar 

  2. Bakeman, R. (2005). Recommended effect size statistics for repeated measures designs. Behavior Research Methods, 37, 379–384. https://doi.org/10.3758/BF03192707

    Article  PubMed  Google Scholar 

  3. Bosman, C., & Womelsdorf, T. (2009). Neuronal signatures of selective attention—Synchronization and gain modulation as mechanisms for selective sensory information processing. In F. Aboitiz & D. Cosmelli (Eds.), From attention to goal-directed behavior: Neurodynamical, methodological and clinical trends (pp. 3–28). New York, NY: Springer. https://doi.org/10.1007/978-3-540-70573-4_1

    Google Scholar 

  4. Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 433–436. https://doi.org/10.1163/156856897X00357

    Article  PubMed  PubMed Central  Google Scholar 

  5. Correa, Á., Lupiáñez, J., Milliken, B., & Tudela, P. (2004). Endogenous temporal orienting of attention in detection and discrimination tasks. Perception & Psychophysics, 66, 264–278. https://doi.org/10.3758/BF03194878

    Article  Google Scholar 

  6. Correa, Á., Sanabria, D., Spence, C., Tudela, P., & Lupiáñez, J. (2006). Selective temporal attention enhances the temporal resolution of visual perception: Evidence from a temporal order judgment task. Brain Research, 1070, 202–205. https://doi.org/10.1016/j.brainres.2005.11.094

    Article  PubMed  Google Scholar 

  7. Coull, J. T., & Nobre, A. C. (1998). Where and when to pay attention: The neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. Journal of Neuroscience, 18, 7426–7435.

    Article  PubMed  Google Scholar 

  8. Doherty, J. R., Rao, A., Mesulam, M. M., & Nobre, A. C. (2005). Synergistic effect of combined temporal and spatial expectations on visual attention. Journal of Neuroscience, 25, 8259–8266.

    Article  PubMed  Google Scholar 

  9. Griffin, I. C., Miniussi, C., & Nobre, A. C. (2002). Multiple mechanisms of selective attention: Differential modulation of stimulus processing by attention to space or time. Neuropsychologia, 40, 2325–2340.

    Article  PubMed  Google Scholar 

  10. Griffin, I. C., & Nobre, A. C. (2005). Temporal orienting of attention. In L. Itti, G. Rees, J. K. Tsotsos (Eds.), Neurobiology of attention (pp. 257–263). Amsterdam, The Netherlands: Elsevier Academic Press. https://doi.org/10.1016/B978-012375731-9/50046-X

    Google Scholar 

  11. Hein, E., Rolke, B., & Ulrich, R. (2006). Visual attention and temporal discrimination: Differential effects of automatic and voluntary cueing. Visual Cognition, 13, 29–50.

    Article  Google Scholar 

  12. Janssen, P., & Shadlen, M. N. (2005). A representation of the hazard rate of elapsed time in macaque area LIP. Nature Neuroscience, 8, 234–241.

    Article  PubMed  Google Scholar 

  13. Jones, A. (2014). Independent effects of bottom-up temporal expectancy and top-down spatial attention. an audiovisual study using rhythmic cueing. Frontiers in Integrative Neuroscience, 8, 96. https://doi.org/10.3389/fnint.2014.00096

    Article  PubMed  Google Scholar 

  14. Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863. https://doi.org/10.3389/fpsyg.2013.00863

    Article  PubMed  PubMed Central  Google Scholar 

  15. Lange, K., Krämer, U. M., & Röder, B. (2006). Attending points in time and space. Experimental Brain Research, 173, 130–140.

    Article  PubMed  Google Scholar 

  16. Langner, R., Steinborn, M. B., Eickhoff, S. B., & Huestegge, L. (2018). When specific action biases meet nonspecific preparation: Event repetition modulates the variable-foreperiod effect. Journal of Experimental Psychology: Human Perception and Performance, 44, 1313–1323.

    PubMed  Google Scholar 

  17. Los, S. A., & Van Den Heuvel, C. E. (2001). Intentional and unintentional contributions of nonspecific preparation during reaction time foreperiods. Journal of Experimental Psychology: Human Perception and Performance, 27, 370–386. https://doi.org/10.1037/0096-1523.27.2.370

    Article  PubMed  Google Scholar 

  18. Miller, J., & Schröter, H. (2002). Online response preparation in a rapid serial visual search task. Journal of Experimental Psychology: Human Perception and Performance, 28, 1364–1390. https://doi.org/10.1037/0096-1523.28.6.1364

    Article  PubMed  Google Scholar 

  19. Morey, R. D. (2008). Confidence intervals from normalized data: A correction to Cousineau (2005). Tutorials in Quantitative Methods for Psychology, 4, 61–64. https://doi.org/10.20982/tqmp.04.2.p061

    Article  Google Scholar 

  20. Nobre, A. C., & van Ede, F. (2018). Anticipated moments: Temporal structure in attention. Nature Reviews Neuroscience, 19, 34–48.

    Article  PubMed  Google Scholar 

  21. Olk, B. (2014). Effects of spatial, temporal and spatiotemporal cueing are alike when attention is directed voluntarily. Experimental Brain Research, 232, 3623–3633.

    Article  PubMed  Google Scholar 

  22. R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from www.R-project.org

  23. Rohenkohl, G., Gould, I. C., Pessoa, J., & Nobre, A. C. (2014). Combining spatial and temporal expectations to improve visual perception. Journal of Vision, 14(4), 8. https://doi.org/10.1167/14.4.8

    Article  PubMed  PubMed Central  Google Scholar 

  24. Sharp, P., Melcher, D., & Hickey, C. (2018). Endogenous attention modulates the temporal window of integration. Attention, Perception, & Psychophysics, 80, 1214–1228. https://doi.org/10.3758/s13414-018-1506-y

    Article  Google Scholar 

  25. Steinborn, M. B., Langner, R., & Huestegge, L. (2017). Mobilizing cognition for speeded action: Try-harder instructions promote motivated readiness in the constant-foreperiod paradigm. Psychological Research, 81, 1135–1151.

    Article  PubMed  Google Scholar 

  26. Steinborn, M. B., Rolke, B., Bratzke, D., & Ulrich, R. (2008). Sequential effects within a short foreperiod context: Evidence for the conditioning account of temporal preparation. Acta Psychologica, 129, 297–307. https://doi.org/10.1016/j.actpsy.2008.08.005

    Article  PubMed  Google Scholar 

  27. Thomaschke, R., & Dreisbach, G. (2015). The time-event correlation effect is due to temporal expectancy, not to partial transition costs. Journal of Experimental Psychology: Human Perception and Performance, 41, 196–218.

    PubMed  Google Scholar 

  28. Vangkilde, S., Coull, J. T., & Bundesen, C. (2012). Great expectations: Temporal expectation modulates perceptual processing speed. Journal of Experimental Psychology: Human Perception and Performance, 38, 1183–1191. https://doi.org/10.1037/a0026343

    Article  PubMed  Google Scholar 

  29. Verhagen, J., & Wagenmakers, E.-J. (2014). “Bayesian tests to quantify the result of a replication attempt”: Correction to Verhagen and Wagenmakers (2014). Journal of Experimental Psychology. General, 143, 2073.

    Article  Google Scholar 

  30. Volberg, G., & Thomaschke, R. (2017). Time-based expectations entail preparatory motor activity. Cortex, 92, 261–270.

    Article  PubMed  Google Scholar 

  31. Wutz, A., Muschter, E., van Koningsbruggen, M. G., Weisz, N., & Melcher, D. (2016). Temporal integration windows in neural processing and perception aligned to saccadic eye movements. Current Biology, 26, 1659–1668.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Poppy Sharp.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sharp, P., Melcher, D. & Hickey, C. Different effects of spatial and temporal attention on the integration and segregation of stimuli in time. Atten Percept Psychophys 81, 433–441 (2019). https://doi.org/10.3758/s13414-018-1623-7

Download citation

Keywords

  • Selective attention
  • Temporal processing
  • Precuing