There is more information present in any given visual scene than the human cognitive system is capable of fully processing at any one point in time. A challenge for the human visual system then is to construct a stable and functional percept of the world from only a select subset of the available visual input. Mechanisms of selective attention allow us to prioritize the processing of certain visual input so that our conscious percept is one constructed from information in the environment that is functionally pertinent. How these selection mechanisms are controlled and the information they are sensitive to has important consequences for how we interact with our environment. In everyday life, the visual system is frequently challenged to decide whether to attend to information that is relevant to our immediate goals or to prioritize signals that might be unexpected and signal a threat.

Much of the debate in the literature over attentional control has focused on the nature of control of the exogenous attentional system – a system that reflexively shifts the focus of attention to signals of potential importance in the environment. At issue is the degree to which bottom-up factors, namely stimulus saliency, automatically capture attention and to what degree top-down, namely task goals, modulate the effect of salient stimuli (Jonides & Yantis, 1988; Lamy & Kristjánsson, 2013; Leber & Egeth, 2006; Müller, Geyer, Zehetleitner, & Krummenacher, 2009; Posner, 1980; Remington, Johnston, & Yantis, 1992; Shomstein, 2012; Theeuwes, 1991, 2010; Yantis & Jonides, 1984). Previous studies investigating orienting, especially in natural scenes, suggest that salience may in fact be a weak determinant of attentional selection: Although stimuli may need to be salient in order to attract attention, saliency in and of itself is not always sufficient to guide attention. In contrast, top-down models of attentional guidance propose that shifts of attention are contingent on the configuration of task-driven selection mechanisms, that select stimuli according to the current task (e.g., Einhäuser, Rutishauser, & Koch, 2008; Einhäuser, Spain, & Perona, 2008; Foulsham & Underwood, 2008; Hwang, Higgins, & Pomplun, 2009; Stirk & Underwood, 2007; Tatler, Hayhoe, Land, & Ballard, 2011; Valuch, Becker, & Ansorge, 2013). Whether a stimulus will attract attention has been shown to depend on whether the stimulus matches the “attentional set” of the observer, which describes the set of relevant stimuli or features that we need to attend to in order to successfully complete the current task (Eimer & Kiss, 2008; Eimer, Kiss, Press, & Sauter, 2009; Folk, Remington, & Johnston, 1992; Folk, Remington, & Wright, 1994; Folk & Remington, 1998; Ludwig & Gilchrist, 2003). Indeed there are examples in the literature where task-driven processes have been shown to modulate attentional capture even when highly salient stimuli are presented. Folk et al. (1992) showed that a salient distractor presented immediately prior to the target in a spatial cueing paradigm produced differential effects on performance according to its relationship with the target. When participants’ task was to search for a color singleton in the display, only matching-color distractors and not abrupt-onset distractors had an effect on search performance. The opposite was true when the target of search was of abrupt onset (though see Belopolsky, Schreij, & Theeuwes, 2010; van Zoest, Donk, & Theeuwes, 2004). The equivocal support in the literature for purely bottom-up capture leaves open the question of how we become aware of stimuli and events in the world that are not directly relevant to our immediate tasks and or goals.

A commonality across much of the research that informs the debate over attentional control is that stimuli in these paradigms are presented repeatedly, frequently, and often with a degree of predictability. Yet outside of the laboratory the environments we encounter are rarely static and predictable in this way; rather, they are typically dynamic and can be unpredictable. Critical to the function and ultimately to the survival of any organism is the ability to effectively respond to and adapt to changes in its environment signalled by these unexpected events. This ability requires a mechanism for detecting new information (novelty) in the environment that operates largely independently of task-driven control and an observer’s attentional set.

An early theoretical account by Sokolov (1963) proposes that our tendency to reflexively orient towards stimuli in the environment is dependent on a mismatch between stimulus input and a set of contextual expectations that he labelled the “neuronal model.” Over repeated exposure to a novel stimulus the neuronal model is updated to incorporate the novel stimulus, consequently dampening the orienting response to subsequent presentations of the stimulus (“habituation”). A similar mechanism for responding to novelty in the environment is derived from schema theories of perception and cognition where a schema is conceptualized as an organized knowledge structure used to generate predictions about the nature of objects and events in the environment (Rumelhart, 1984; Rumelhart, Smolensky, McClelland, & Hinton, 1986; Meyer, Niepel, Rudolph, & Schűtzwohl, 1991). Unexpected events in the environment generate prediction errors due to the mismatch that arises between the stimulus input and a set of expectations generated by the schema. Prediction errors signal that there is new information in the environment and the system responds by interrupting ongoing processes and devoting attentional resources to process this new information. This process is presumed to underlie the subjective impression of surprise (Meyer et al., 1991; Niepel, Rudolph, Schűtzwohl, & Meyer, 1994; Meyer, Reisenzein, & Schűtzwohl, 1997). Once an unexpected stimulus has been reconciled and incorporated into a schema it will no longer elicit surprise as it is no longer expectation discrepant (Niepel et al., 1994). It is through this mechanism that schemas are thought to be updated and maintained as accurate accounts of one’s environment. Indeed, a novelty selection mechanism has been proposed as the necessary complement to dynamic and efficient goal-driven selection mechanisms to ensure adaptive action in natural environments (Horstmann, 2006).

The role of task-expectancies in determining the allocation of cognitive resources is supported by experimental evidence demonstrating numerous physiological and behavioral changes in response to unexpected events (Asplund, Todd, Snyder, Gilbert, & Marois, 2010; Becker & Horstmann, 2011; Czigler, Weisz, & Winkler, 2006; Forster & Lavie, 2011; Horstmann, 2002, 2005, 2006; Horstmann & Becker, 2008; Kazmerski & Friedman, 1995; Meyer et al., 1991; Neo & Chua, 2006; Niepel et al., 1994; Retell, Becker, & Remington, 2015; Schützwohl, 1998). For example, Meyer et al. (1991) had participants respond to the location of a dot that appeared briefly (0.1 s) either above or below two vertically arranged words. For the first 29 pre-critical trials of the experiment the words appeared as black against a white background. On the 30th “surprise” trial, the color of one of the words and its background was inverted (white letters on a black background). Recall for the inverted word on the surprise trial was significantly better relative to a control condition in which the word was presented in the same way as in the pre-critical trials, suggesting that the expectation-discrepant word had attracted attention. Furthermore, response times (RTs) to the dot on the surprise trial were significantly elevated relative to a control condition, suggesting that additional processing resources were recruited to process and integrate the expectation-discrepant event.

Subsequent work by Schűtzwohl (1998) and more recently by Horstmann (2005) has demonstrated that the magnitude of the surprise response to a new and unannounced stimulus indeed varies with varying task-expectancies. Using the same paradigm as Meyer et al. (1991), Schűtzwohl (1998) varied the number of pre-critical trials between 3, 13, 23, and 33 to modulate the strength of stimulus expectations prior to exposure to the surprising stimulus. In line with the predictions, the effect of the “surprise” stimulus varied across the four conditions. RTs on the surprise trial were significantly longer in the 13-trials condition than the three-trials condition and significantly longer again in the 23-trials condition relative to the 13-trials condition. No difference in RTs was found between the 23- and 33-trials conditions. These results indicate that more practice leads to the formation of stronger expectancies, which, when violated, produce a heightened surprise response.

In a separate experiment, Schűtzwohl (1998) established that the variability of the stimulus array presented in the pre-critical trials can also modulate the surprise response (see also Horstmann, 2005). In one condition the word stimuli in the pre-surprise trials were presented in a uniform font (homogenous stimulus array) while in a second condition the font of one of the two words was varied in each of the pre-critical trials (heterogeneous stimulus array). Presenting a novel combination of font colors on the surprise trial led to RTs that were significantly shorter in the heterogeneous condition relative to the homogenous condition. These results show that the formation of task-expectancies are influenced by experience and the distribution of events and objects that occur prior to a novel stimulus. When a set of expectations is weakly established and/or broadly defined, broad expectations are formed and new and unannounced stimuli are less likely to violate them. Conversely, repeated exposure to relatively homogenous stimulus arrays leads to narrowly defined task-expectancies with seemingly less tolerance for new unannounced deviants.

Common across all the experiments investigating surprise is that the surprising stimulus is always new and unannounced. It is clear from the work of Schűtzwohl (1998) and Horstmann (2005) that novelty or “unexpectedness” per se is not sufficient to elicit a strong surprise response, because the context plays an important role in shaping our expectancies, which in turn determine the surprise response. However, it is less clear to what extent prior exposure and top-down expectations shape our task-expectancies and whether either one of them or both are necessary to elicit surprise. That is, if participants have prior exposure to a stimulus and/or are informed about its occurrence, would it still be unexpected in the sense that it still elicits surprise? We know that both the behavioral and neurophysiologic markers of surprise dissipate across successive presentations of an unexpected stimulus within the visual domain (Retell et al., 2015; Horstmann 2002, 2005, 2006; Schűtzwohl, 1998; Asplund et al., 2010; Kazmerski & Friedman, 1995). In certain instances the surprise effect occurs on only the first presentation of a novel stimulus (Horstmann, 2002, 2005, 2006). It is also true that repeated prior exposure within the experimental context attenuates the surprise response (Horstmann, 2005). However, it is an open question whether the surprise response would be attenuated if participants had been exposed to the stimulus prior to performing a specific experimental task. Presumably, any exposure to a stimulus will render it familiar, and will therefore affect, almost by definition, its novelty. Thus, if surprise is related to the novelty of a stimulus in this sense then we should expect an attenuation of the surprise response when participants are pre-exposed to an “unexpected” stimulus. Similarly, if task-expectancies can be shaped by explicit knowledge about the nature of events and objects in an environment then we might expect that knowledge of a forthcoming “unexpected” stimulus should also result in an attenuation of the surprise response. An alternative possibility is that task-expectancies are highly task- or context-specific and formed strictly through a process of implicit learning. If this is the case, pre-exposure to and or explicit knowledge of an “unexpected” stimulus should have no effect on the subsequent response to such a stimulus.

The present study addressed these open questions by examining responses to an otherwise surprising stimulus when participants: (a) were instructed to expect it, (b) had been pre-exposed to it, (c) were exposed to both of these manipulations, or (d) received no information about the surprising stimulus (standard surprise experiment).

Experiment 1

We used a variant of the spatial cueing paradigm used by Folk and Remington (1998) to test surprise with respect to a task-irrelevant motion singleton distractor (cue). Participants had to report the orientation of a specific red target (e.g., red horizontal bar) that was embedded amongst six differently colored and oriented non-target bars (i.e., red, green, blue, horizontal, and vertical bars). Prior to the target display, a cueing display was presented that contained a to-be-ignored red or green singleton cue. The red and green cues were either presented at the same location as the target (valid trial) or at a different location (invalid trial), and were included to provide a baseline against which the effects of the surprising motion cue could be compared. According to the Contingent Orienting Hypothesis, the target-similar red cue should attract attention because it is consistent with the goal of searching for red, which should lead to faster RTs on valid than invalid trials. On the other hand, the green cue should not attract attention, because it does not match the task goals of searching for red (Folk, Remington, & Wright, 1994; Folk & Remington, 1998), so there should be no effect of cue validity. The motion cue was always presented at an invalid location. Hence, if participants are able to ignore the motion cue in any of the conditions, there should be no performance differences between the invalid green cue and the motion cue in the RTs. On the other hand, if the motion cue attracts attention, RTs should be elevated (e.g., see Horstmann, 2005).

If presenting an irrelevant motion cue generates surprise, then the RT elevation should moreover dissipate with repeated presentations of the motion cue. To assess the RT elevation due to surprise, we presented the motion cue infrequently during the experiment (once every ~32 trials) after the first presentation, to a total number of eight motion cue trials. The central question of the present experiment was whether providing participants with prior information and/or exposing them to the “unexpected” stimulus prior to the experiment would modulate their response to the first presentation of an “unexpected” motion singleton. To that aim, we varied the amount of prior information and exposure to the motion singleton between different groups of participants.

In the “expected” condition participants were told immediately prior to the experiment that at some point during the experiment a new and unexpected stimulus would be presented but not what that stimulus would be. Additionally, they were told that the unexpected stimulus would occur at an invalid location and that it was designed to distract them from searching for the target and therefore they should do their best to ignore it. Moreover, participants in the “expected” condition were informed that the fixation dot would turn blue shortly before the unexpected stimulus was about to occur, as a reminder to them that they were about to see a potentially distracting new stimulus and that they should do their best to ignore it. To summarize, participants in the “expected” condition knew that an “unexpected” stimulus would be presented and roughly when, but were naïve to the specific attributes of the stimulus.

In contrast, in the “exposed” condition participants were shown an example of the motion cue immediately prior to the start of the experiment. However, participants were told a cover story to explain the presence of the motion cue, but were given no information regarding the presentation of this stimulus in the experiment. Participants in the “expected and exposed” condition received the “expected” and the “exposed” manipulations. Critically though, the two were linked. That is, participants were shown the motion cue prior to the experiment and told that it would be the “unexpected” stimulus that would occur later in the experiment. Again, participants in this condition knew that the motion distractor would roughly be predicted by the fixation dot turning blue, and were instructed that attending to the motion cue would be detrimental to performance. Finally, in the “standard” condition, participants were not informed about the appearance of the motion cue and not shown any stimulus examples of it. This condition served as a baseline condition to which the other three experimental conditions were compared.

To ensure that RT elevations in response to the motion cue were not due to participants’ gaze shifts instead of attention shifts, we used an eye tracker to monitor central fixation. All eye-movement analyses were conducted online and trials in which participants broke fixation during the cueing frame were coded as an error.

Method

Participants

Two hundred participants (137 female) aged 17–42 years (M = 19.9, SD = 3.1) from an introductory psychology course at the University of Queensland were assigned to one of the four conditions (standard, expected, exposed, expected, and exposed) and participated for course credit. All participants reported normal or corrected-to-normal vision.

Apparatus

All experiments were conducted using the computer software package Matlab (2010a) and the Psychophysics Toolbox extension (Brainard, 1997; Pelli, 1997). Stimuli were presented on a 19-in CRT monitor attached to a (Pentium 4) personal computer. Stimuli were presented with a resolution of 1,280 × 1,024 pixels and a refresh rate of 75 Hz. Responses were recorded using a keyboard. Participants’ eye-movements were measured using a video-based infrared eye-tracking system (Eyelink 1000 Tower Mount, SR Research, Mississauga, Ontario, Canada) with a spatial resolution of 0.1 and a temporal resolution of 500 Hz and the Eyelink Toolbox extension (Cornelissen, Peters, & Palmer, 2002).

Stimuli

Each trial consisted of a fixation display, a cueing display, and a target display. All displays contained a central fixation circle (0.2° × 0.2°) surrounded by six peripheral circular placeholders (2.6 ° × 2.6°) positioned in a circular array around fixation and subtending 5° of visual angle from fixation. The cueing display consisted of a set of four filled circles (0.3 °) in a diamond configuration surrounding all six placeholders (see Fig. 1). With the exception of motion cue trials, one set of four dots around one location was always colored either red or green (four-dot cue). The motion cue was rendered by a 90° rotation of the diamond configuration in eight 11.25 ° clockwise increments at 13.33-ms intervals.

Fig. 1
figure 1

Example trial sequences. Red and green cues were non-predictive of the target location while the motion cue only ever occurred and invalid locations. The target and cues never occurred at the locations above and below fixation. Note that displays that preceded the cue display are absent from this example trial sequence

Full size image

The target display was comprised of the same stimuli as the fixation display with the addition of six bars presented at each of the six placeholder locations. The bars could be oriented horizontally, vertically or 45 ° to either the left or to the right. In the target display three of the bars were oriented either horizontally or vertically and three were oriented either 45 ° to the left or to the right. Two of the bars in the display were colored red (RGB = 255, 0, 0), two were colored green (RGB = 0, 255, 0) and two were colored blue (RGB = 0, 0, 255). The distribution of colors was such that each orientation (horizontal/vertical and 45 ° left/right) appeared in each color. The target was the red bar that could appear either horizontally or vertically and participants had to report its orientation (horizontal vs. vertical) with a button press (left arrow key for horizontal; right arrow key for vertical). All stimuli were presented against a black (RGB = 5, 5, 5) background.

Design

Experiment 1 consisted of 32 practice trials and 288 experimental trials, though this structure was not apparent to participants. Presentation of the red and green cues and the target was randomized across the experiment with each occurring at each location equally often. This design rendered the cues non-predictive of the target location, thus providing no incentive to orient to the cues. Participants were informed of this and instructed not to attend to the cue throughout the experiment. On eight of the experimental trials the color cue was substituted for a motion cue. The motion cue was presented once every 30–35 trials and was only ever presented at an invalid location. The color cues, the motion cue, and the target never appeared at the position directly above or the position directly below fixation (see Fig. 1). That is, these stimuli could only occur at four of the six placeholder locations.Footnote 1 The motion cue was presented twice at each of four possible locations and replaced an equal number of red and green cues – four of each. Four trials prior to the presentation of the first motion cue the fixation dot was presented as blue for the full duration of one trial. This was the case in all conditions; however, the relevance of this signal varied across conditions and is explained in more detail below.

We independently varied prior exposure to the surprising motion cue, with instructions to expect a surprising motion cue. Due to practical reasons, the first 100 participants were randomly assigned to the either the “standard” condition or the “expected” condition and these two conditions were run first. The second 100 participants were randomly assigned to either the “exposed” or “expected and exposed” condition.

Procedure

Prior to the experiment, all participants were informed about the occurrence of the red and green cues and were instructed to ignore them as they were non-predictive of the target. Moreover, all participants were briefed about the eye tracking procedure and instructed to maintain fixation on the central dot, while responding as fast and accurately as possible.

Apart from this, the instructions differed for the four groups (standard, expected, exposed, and expected and exposed). Participants in the standard condition were not informed about the appearance of the motion cue at all nor shown any examples of it. Participants in the expected condition were informed that a novel cue would be presented at an invalid stimulus location, of which they would be warned by the fixation dot turning blue, and they were instructed to ignore it. In addition to verbal instruction, the following instructions were presented to participants in the Expected condition: “At some point during the Experiment something unexpected will appear on the screen. You will be notified ~WHEN this will happen by the fixation circle turning BLUE. Please do you best to IGNORE the stimulus and continue to respond as QUICKLY and as ACCURATELY as possible!”

Participants in the exposed condition were presented with an exposure display that presented the motion cue at the location of one of the placeholders. Participants were told that the motion stimulus in this context was related to the calibration of the eye-tracker and were asked to maintain central fixation during its presentation. Additionally, participants were told that the fixation dot would turn blue on some of the trials but that this was irrelevant for the purposes of the present experiment. In the expected and exposed condition, participants were fully informed about the appearance of the motion cue, shown the same example as the exposed group, and were informed about the significance of the blue fixation dot. Each trial began with a central fixation dot that was presented for 700 ms. This was followed by the fixation display for 500 ms or until the participant met the fixation criteria of the eye-tracker. Following this, the fixation dot offset for 50 ms then the fixation display was presented for a randomly determined 200, 400 or 600 ms, after which the cueing display was presented for 120 ms. Following the cueing display there was an interstimulus interval (ISI) of 53 ms and then the target display was presented for 53 ms (see Fig. 1). At the offset of the target display, the fixation display was presented and remained on the screen until a response was recorded. If the response was correct and made in fewer than 1,200 ms then the next trial started after a 500-ms delay. If the response was correct but made in over 1,200 ms participants received the feedback “Too Slow!” and if the response was incorrect they received the feedback “Wrong!” In both cases the feedback was presented for 1,600 ms and then the next trial commenced.

The RT deadline of 1,200 ms with the specific feedback was employed to deter participants in the “expected” and “exposed and expected” conditions from disengaging with the search task and actively searching for the motion cue. By installing a soft response deadline we aimed to ensure that participants’ primary focus was on the search task.

Results

Mean RTs and accuracies for Experiment 1 are shown in Figs. 3 and 4 and Table 1, respectively. Errors and all RTs faster than 200 ms or exceeding 1,700 ms were excluded from RT analyses.Footnote 2 Eye movement data were analysed online and trials were coded as an error if participants did not remain fixated throughout the trial. Participants were deemed to be fixating if their gaze fell within a region of 1.3° of visual angle from the center of the fixation cross. This criteria lead to a loss of 8.01 % of trials that was uniformly distributed across conditions. Data from four participants (one Standard; two Exposed; one Exposed and Expected) were excluded from all analyses due to unacceptably high errors of 25 % or greater. In reporting the results, we first present the results for the red and green cues, followed by the motion cue data for each condition, followed finally by the results of a between-subjects analysis of the motion cue data.

Table 1 Percent of errors for each condition in Experiment 1 and Experiment 2
Full size table

Color cues

To ascertain whether the red and green cues shows a results pattern consistent with top-down controlled search for the red target, we first conducted four 2 (cue color: red vs. green) × 2 (cue validity: valid vs. invalid) repeated measures ANOVAs and computed planned follow-up comparison. As shown in Fig. 2, all four conditions (standard, expected, exposed, expected and exposed) showed significant validity effects for the red cue with faster RTs on valid than invalid trials for the red cue, and the reverse effect (of faster RTs on invalid than valid trials) for the green cue.

Fig. 2
figure 2

Mean response times (RTs) as a function of cue type and validity for Experiment 1. RTs for the motion cue reflect the average of all eight motion presentations. Error bars depict ± one standard error of the mean

Full size image

  • Standard: A main effect of cue validity, F(2, 47) = 24.47, p < .001, η p 2 = .34, was qualified by a significant interaction between cue color and cue validity, F(2, 44) = 95.58, p < .001, η p 2 = .67. Follow-up pairwise comparisons revealed a significant validity effect associated with the red cue, t(47) = 8.51, p < .001; d = .91, and a small but significant inverse validity effect associated with the green cue, t(47) = -3.80, p = .001; d = .29.

  • Expected: A main effect of cue validity, F(2, 49) = 43.32, p < .001, η p 2 = .47, was qualified by a significant interaction between cue color and cue validity, F(2, 44) = 121.72, p < .001, η p 2 = .71. Follow-up pairwise comparisons revealed a significant validity effect associated with the red cue, t(49) = 11.12, p <.001; d = 1.2, and a small but significant inverse validity effect associated with the green cue, t(49) = -5.04, p < .001; d = .41.

  • Exposed: A main effect of cue color, F(2, 49) = 5.77, p = .02, η p 2 = .11 and a main effect of cue validity, F(2, 44) = 34.30, p < .001, η p 2 = .41, were qualified by a significant interaction between cue color and cue validity, F(2, 49) = 197.33, p < .001, η p 2 = .80. Follow-up pairwise comparisons revealed a significant validity effect associated with the red cue, t(49) = 12.51, p < .001; d = 1.2, and a small but significant inverse validity effect associated with the green cue, t(49) = -3.48, p = .001; d = .19.

  • Expected and exposed: A main effect of cue color, F(2, 48) = 4.07, p = .049, η p 2 = .08 and a main effect of cue validity, F(2, 48) = 24.38, p < .001, η p 2 = .34, were qualified by a significant interaction between cue color and cue validity, F(2, 49) = 169.85, p < .001. Follow-up pairwise comparisons revealed a significant validity effect associated with the red cue, t(48) = 12.04, p < .001; d = .92, and a small but significant inverse validity effect associated with the green cue, t(48) = -6.16, p < .001; d = .41.

Motion cue

For each condition we computed the effect of the motion cue relative to the average of valid and invalid green cues trials. The logic of this is as follows: remembering that participants searched for a red target, both motion and green were task irrelevant. Thus, if the novelty of the motion cue had any effect on RTs it ought to be observable above and beyond any effect of the task-irrelevant green cue. We chose to use the average of the valid and invalid green cues as a baseline rather than invalid green cues due to the inverse validity effect associated with the green cue. This method resulted in a more conservative estimate of the effect of the motion cue.

To determine the interference effect associated with the first presentation of the motion cue we contrasted RTs on the first motion cue trial to the average of the RTs associated with the green cue trials that were presented prior to the first presentation of the motion cue (16 observations per participant) – we reasoned that subsequent trials may be contaminated by the surprise response and consequently dilute our estimates of surprise. In all four conditions we observed a significant interference effect. Four pair-wise comparisons revealed that RTs associated with the first motion cue were significantly slower relative to RTs associated with the green cue trials in the “standard” condition, t(39) = 3.43, p =.001; d = .72, the “expected” condition, t(41) = 5.68, p <.001; d = 1.24, the “exposed” condition, t(41) = 3.66, p = .001; d = .71, and the “expected and exposed” condition, t(41) = 4.87, p <.001; d = .94 (see Fig. 3).

Fig. 3
figure 3

Average response time (RT) for each presentation of the motion cue in each of the four experimental conditions. The solid white line in each plot shows the predicted pattern of RTs for the motion cue when presentation order has no effect, as determined by the permutation analysis (see Results section). The shaded area defines the 95 % confidence interval around the permuted means, where data outside these boundaries indicate a significant change in RTs across presentations of the motion cue

Full size image

In addition to computing the interference associated with the first presentation of the motion distractor, we also computed the interference associated with presentations 2–8 of the motion distractor in each condition. Four pair-wise comparisons confirmed that RTs associated with the motion cue (averaged across presentations 2–8) were significantly slowed relative to RTs associated with the green cue (occurring after the first motion cue presentation) in the “standard” condition, t(47) = 7.36, p = <.001; d = 1.06, the “expected” condition, t(49) = 6.90, p = <.001; d = 1.01, the “exposed” condition, t(49) = 6.95, p = <.001; d = .90, and the “expected and exposed” condition, t(48) = 7.41, p = <.001; d = 1.13.

To assess whether the RT interference associated with the motion cue dissipated with repeated exposure to the motion cue we ran a permutation analysis on the motion cue data for each condition. This involved shuffling the position of the RT data within participants then calculating new permuted group means for each presentation of the motion cue. We performed 10,000 iterations of this step and calculated the 95 % confidence interval (CI) around the permuted means for each presentation of the motion cue (see Fig. 4). This method allowed us to simulate the pattern of data that would result if exposure were having no effect on RTs. As shown in Fig. 3, the empirical data deviated significantly from the permuted pattern of results in all four conditions. That is, in all four conditions we observed an effect of presentation order such that RTs associated with the motion cue dissipated across repeated presentations of the motion cue.

Fig. 4
figure 4

Normalized response times (RTs) for the motion cue for all four conditions from Experiment 1. RTs are normalized by subtracting the average RT for green cue (valid and invalid) trials – calculated from the trials prior to each presentation of the motion cue – from the RT associated with the individual presentations of the motion cue. These difference scores were computed at the participant level and then average across participants within each condition

Full size image

Motion cue: between-subjects analysis

Having found significant RT interference associated with the motion cue in each condition we sought to test whether these effects varied across our four conditions. To avoid the possibility that different result patterns could be due to baseline RT differences between the groups (e.g., with participants in one group responding generally faster and/or more accurately than participants in the other group), the motion cue data were normalized for each participant by subtracting the average RT for (valid and invalid) green cue trials – calculated from the trials prior to each presentation of the motion cue – from the RT associated with the individual presentations of the motion cue. To analyse differences in the surprise response across conditions we compared normalized RTs to only the first motion cue presentation. The normalized data for all motion cue presentations is shown in Fig. 4.

A 2 (expectation: unexpected vs. expected) × 2 (exposure: unexposed vs. exposed) between-subjects ANOVA computed over the first presentation of the of the motion cue revealed only a main effect of expectation, F(1, 164) = 4.90, p = .028, η p 2 = .03. RTs associated with expected motion cues were slowed relative to unexpected motion cues. Though the interaction term was non-significant, p = .15, inspection of Fig. 4 suggests that this effect is being driven by the “expected” condition (see Fig. 4). Correspondingly, a between-subjects contrast revealed significantly elevated RTs in the expected surprise condition relative to the standard surprise condition, t(80) = 2.48, p = .015. No difference was observed between the exposed and expected and exposed conditions, p = .82.

Errors

The pattern of errors was consistent with the RT data (see Table 1). In all four conditions there was an effect of the red cue on error rates such that there were significantly fewer errors associated with valid red cues than invalid red cues, Standard, p < .001; Expected, p < .001; Exposed, p < .001; Expected and Exposed p < .001. No other contrasts were significant in any of the four conditions.

Discussion

In Experiment 1 we varied knowledge of and/or exposure to a motion distractor to modulate the top-down expectations and/or familiarity with the stimulus. In all four conditions we observed elevated RTs associated with the first presentation of the motion cue indicative of surprise. Following the first presentation, the motion cue continued to produce RT interference; however, the magnitude of this interference dissipated across presentations of the motion cue (see Fig. 3). Critically, interference by the irrelevant motion cue was observed despite the fact that participants adopted a feature specific attentional set for the color red – reflected by the significant validity effect for the red cue and the lack of capture by the green cueFootnote 3 (see Fig. 2).

Moreover, the interference associated with the first presentation of the motion cue did not vary across conditions in the predicted direction. There was no attenuation of the RT interference associated with the first presentation of the motion cue in the “exposed” condition, the “expected” condition, or the “exposed and expected” condition relative to the “standard surprise” condition (see Fig. 4). In fact, in the “expected” condition we observed what appears to be an increase in the interference associated with the first motion cue presentation. That is, knowledge about the occurrence of an otherwise unknown event appears to have amplified the surprise response. Interestingly, this pattern appears specific to the case where participants knew to expect the “unexpected” stimulus but not what to expect. However, given that the interaction term did not reach significance we are cautious of inferring a role of exposure in this effect.

To summarize, in all four conditions we found RT interference associated with the first presentation of the motion cue. This interference persisted beyond the first presentation of the motion cue but was attenuated with repeated exposure to the motion cue. Contrary to our predictions, we found no attenuation of the RT interference associated with the first presentation of the motion cue when participants knew to expect something unexpected, were pre-exposed to the unexpected stimulus or were pre-exposed and knew what to expect. One possible, though admittedly unlikely, explanation for this pattern of results is that the novelty signal we attempted to manipulate in Experiment 1 might not have been the source of the interference in Experiment 1. That is, it is possible that our results reflect a property of the motion stimulus itself and are unrelated to the novelty of the stimulus. In Experiment 2 we addressed this possibility by presenting the motion stimulus frequently.

Experiment 2

Experiment 2 was a control experiment to ensure that the effects reported in Experiment 1 were indeed related to the presentation frequency of the motion stimulus and not the motion stimulus itself. To test this, in Experiment 2 we presented the motion stimulus in the same manner as the color cues – frequently and at both valid and invalid locations. Under these conditions the motion stimulus is comparable to the green cue – frequently occurring and task irrelevant – and should not produce a validity effect when participants are searching for a red target (Folk et al., 1992, 1994).

Method

Participants

Fifteen participants (eight female) aged 21–26 years (M = 24.1, SD = 1.9) from an introductory psychology course at the University of Queensland participated for course credit. All reported normal or corrected to normal vision.

Apparatus

The apparatus used in Experiment 2 was identical to that used in Experiment 1.

Stimuli, design, and procedure

The stimuli, design, and procedure of Experiment 2 were identical to the “standard surprise” condition in Experiment 1 with one critical exception: The motion stimulus in Experiment 2 was presented frequently – on one-third of trials – and at both valid and invalid locations. As a result, in Experiment 2 there were 48 practice and 288 experimental trials.

Results

Response times

Mean RTs and error rates for Experiment 2 are shown in Fig. 5 and Table 1, respectively. RTs exceeding 1,700 ms and errors were excluded from RT analyses. This led to a loss of 7.74 % of experimental trials.

Fig. 5
figure 5

Mean response times as a function of cue type and validity for Experiment 2. Error bars depict the standard error of the mean

Full size image

A 3 (cue type: red vs. green vs. motion) × 2 (cue validity: valid vs. invalid) repeated measures ANOVA of color cue RTs revealed a main effect of cue validity, F(1, 9) = 5.33, p = .046, η p 2 = .37. This effect was qualified by a significant interaction between cue color and cue validity, F(2, 9) = 11.68, p = .004, η p 2 = .75. Follow-up pairwise comparisons revealed a significant validity effect associated with the red cue, t(9) = 4.71, p = .001; d = .79. No validity effect was observed for the green cue, t(9) = -1.15, p = .28 or the motion cue, t(9) = 0.02, p = .98.

Errors

The pattern of errors mirrored the RT data. There were significantly fewer errors associated with valid red cues than invalid red cues, p = .019. No other contrasts were significant. Given the pattern of errors across all conditions, the differences in RTs reported above are not attributable to any speed accuracy trade-offs.

Discussion

The results of Experiment 2 demonstrate that when the motion stimulus was presented frequently it had no effect on RTs relative to the task-irrelevant green cue. Consistent with the contingent capture hypothesis we observed a strong validity effect associated with the red cue but no effect of the green and motion cues when participants were searching for a red target (Folk et al., 1992). Additionally, the motion cue did not produce elevated baseline RTs that would be indicative of filtering costs or other forms of spatially non-specific interference (e.g., Becker, 2007; Folk & Remington, 1998). Instead, the results of Experiment 2 indicate that the motion cue produced no discernible evidence of capture or interference. Therefore, the effects of the motion stimulus reported in Experiment 1 relate to the (in)frequency of its presentation and are not the product of an inherent property of the motion stimulus per se.

Experiment 3

A key component of the surprise response is the orienting of attention toward the inducing stimulus (Horstmann, 2005; Meyer et al., 1991). In Experiment 1 we showed across four conditions that the first presentation of an invalid motion distractor produced RT inference. Although elevated RTs can reflect shifts of attention towards a distractor, it is also true that they can arise from more general, non-spatial forms of interference such as filtering costs that are separate from shifts of attention (Folk & Remington, 1998; see also Becker, 2007). That is to say, it is possible that the interference associated with the motion distractor in Experiment 1 does not reflect surprise as it has previously been reported (see Horstmann, 2002, 2005; Horstmann & Becker, 2008; Meyer et al., 1991; Niepel et al., 1994; Schűtzwohl, 1998). This could explain why we found no effect of prior knowledge and or exposure in Experiment 1. This seems unlikely given the results of Experiment 2; however, it is necessary to demonstrate a spatial validity effect associated with the infrequent motion distractor before relating our effects to surprise and drawing any conclusions about distribution of visual attention. In Experiment 3 we varied the validity of the motion distractor between subjects. If the motion distractor indeed induces surprise and captures attention on its first presentation then we should observe a spatial validity effect – faster RTs associated with valid motion relative to invalid motion.

Method

Participants

One hundred and five participants (72 female) aged 21–34 years (M = 21.3, SD = 1.7) from an introductory psychology course at the University of Queensland participated for course credit. All reported normal or corrected to normal vision.

Apparatus

The apparatus used in Experiment 3 was identical to that used in Experiment 1.

Stimuli, design, and procedure

The stimuli, design, and procedure of Experiment 3 were identical to the Standard condition of Experiment 1 with two critical exceptions. Firstly, for half of the participants (53) the unexpected motion singleton appeared at a valid location. The other half of participants received an invalid motion singleton as per Experiment 1. Secondly, we only collected responses to the first presentation of the unexpected motion singleton. Participants completed 32 practice trials followed by 35 experimental trials. The motion singleton was presented randomly between trials 62 and 67. The sample of participants used in Experiment 3 was separate from those used in the previous experiments.

Results

Response times

Mean RTs and error rates for Experiment 3 are shown in Figs. 7 and 8 and Table 1, respectively. Errors and RTs faster than 200 ms and exceeding 1,700 ms were excluded from RT analyses. This led to a loss of 9.75 % of experimental trials. Data from nine participants (six valid, three invalid) were excluded from all analyses due to unacceptably high errors of 30 % or greater. We first present the results for the red and green cues, followed by a between-subjects analysis of the motion cue data.

Color cues

As was done in Experiment 1, we ran two separate 2 (cue color: red vs. green) × 2 (cue validity: valid vs. invalid) repeated measures ANOVAs, one for each between-subjects condition. As shown in Fig. 7, we found a significant validity effects for the red cue with faster RTs on valid than invalid trials for both the valid and invalid motion conditions. There was no effect of the green cue in either condition.

  • Valid motion: There was no main of cue color, p = .60, or cue validity, p = .058. However, there was a significant interaction between cue color and cue validity, F(1, 46) = 19.90, p < .001, η p 2 = .31. Follow-up pairwise comparisons revealed a significant validity effect associated with the red cue, t(46) = 4.66, p <.001. No effect was observed for the green cue, p = .074.

  • Invalid motion: A main effect of cue validity, F(1, 48) = 4.80, p < .033, η p 2 = .091, was qualified by a significant interaction between cue color and cue validity, F(1, 48) = 7.44, p < .01, η p 2 = .13. Follow-up pairwise comparisons revealed a significant validity effect associated with the red cue, t(48) = 3.00, p = <.01. No effect was observed for the green cue, p = .72.

Motion cue

Following the same logic as Experiment 1, the motion cue in each condition was compared to the average of the relevant valid and invalid green cue trials. A pairwise comparison revealed that RTs associated with the valid motion cue were significantly slowed relative to green cue RTs, t(42) = 4.19, p < .001; d = .99. The same effect was observed in the invalid motion condition, t(42) = 5.62, p <.001; d = 1.03 (see Fig. 6).

Fig. 6
figure 6

Mean response times as a function of cue type and validity for Experiment 3. Error bars depict ± one standard error of the mean

Full size image

Motion cue: Between-subjects analysis

Finally, to test for a validity effect associated with the motion cue we compared the normalized valid motion RTs to the normalized invalid motion RTs using an independent-groups t-test. Again, we performed this contrast over the normalized data to control for potential baseline differences in RTs between the groups – these are evident in Fig. 6. Though the contrast did not reach significance, t(84) = 1.68, p = .097; d = .36, there is a clear trend toward a validity effect, with invalid motion cue RTs slowed relative to valid motion cue RTs (see Fig. 7).

Fig. 7
figure 7

Normalized motion cue response times (RTs) for each condition. RTs are normalized by subtracting the average RT on green cue trials (valid and invalid) – calculated from the trials prior to the presentation of the motion cue – from the motion cue RT. Error bars depict ± one standard error of the mean

Full size image

Errors

The pattern of errors in the valid motion condition was consistent with the respective RT data. There were significantly fewer errors associated with valid red cues than invalid red cues, p = .001. No other contrasts were significant. Error rates did not vary as a function of condition in the invalid motion condition - the effect of the red cue approached significance, p = .09. Given the pattern of errors across all conditions, the differences in RTs reported in Experiment 3 are not attributable to any speed accuracy trade-offs.

Discussion

Consistent with surprise, we found large RT costs associated with both valid and invalid presentations of the motion distractor. Critically, these effects were observed despite both groups of participants adopting a clear set for the task-relevant feature red. Though we found no significant difference between RTs associated with valid and invalid motion, the data presented in Fig. 7 show a trend consistent with previous literature (Horstmann, 2002, 2005; Retell et al., 2015) – invalid motion RTs appear slowed relative to valid motion RTs. This is an important trend as it suggests that the RT interference associated with the motion singleton in Experiment 1 and Experiment 3 reflects a redistribution of attention towards the motion stimulus consistent with surprise.

General discussion

The experiments reported here explored the contributions of prior-exposure and explicit knowledge of an “unexpected” event to the surprise response in visual search. Our results suggest that the surprise response may be independent of both of these factors. In Experiment 1 we observed robust RT interference indicative of surprise to the first presentation of a task irrelevant motion cue that was not attenuated when participants were pre-exposed to the motion stimulus and/or when participants had explicit knowledge about the unexpected event. Even when observers had prior experience with the surprising stimulus and knew roughly when to expect it (“exposed and expected” condition), the motion stimulus still elicited a surprise response. Only in the “expected” condition was there any evidence of a modulation of the surprise response; here though we observed a trend toward greater RT interference (see Fig. 4). That is, when participants knew to expect something unexpected but not what is was, we observed a large surprise response.

The results of Experiment 2 demonstrated that stimulus presentation (in)frequency was indeed the source of the interference associated with the motion cue in Experiment 1, rather than some inherent property of the motion stimulus, such as salience or abrupt luminance transients. Consistent with the contingent orienting hypothesis (Folk et. al., 1992, 1994), when the task-irrelevant motion cue was presented frequently in Experiment 2 we observed no effect of the motion cue relative to the green task-irrelevant cue. It could be argued that the RT elevation observed in the “expected” and “exposed and expected” is due to subjects disengaging from the search task and actively searching for the surprising stimulus given they knew it was coming. We suggest this is unlikely for two reasons: first, we employed a response deadline of 1,200 ms throughout the experiment to encourage engagement with the task and fast responding. Secondly, performance on the trials between the surprise prime (blue fixation) and the surprising stimulus did not differ across conditions (see Fig. 8). That is, participants did not change their search behavior in response to information about the forthcoming novel stimulus in either the “expected” or the “exposed and expected” condition. Thus, we suggest that the RT interferences associated with the first presentation of the motion cue in the “expected” and “exposed and expected” conditions reflects a reflexive redistribution of cognitive resources to the “unexpected” stimulus.

Fig. 8
figure 8

The data show the difference in response times (RTs) between the four trials prior to the surprise prime and the four trials prior to the surprising stimulus for each condition referenced to the standard surprise condition. Error bars depict ± one standard error of the mean

Full size image

To account for the main effect of expectation, more specifically, the surprise response in the expected condition, it is possible that rates of inattentional blindness (IB) varied across the conditions. It is well documented that under attentional demand observes can fail to perceive unexpected seemingly highly salient stimuli (Jensen, Yao, Street, & Simons, 2011; Mack & Rock, 1998; Most, Scholl, Clifford, & Simons, 2005; Most et al., 2001). When observers are aware that an unexpected stimulus may , IB rates are attenuated if not abolished (Jensen et al., 2011). This could account for why we observed large RT costs in the expected condition; the expectation manipulation may have attenuated rates of IB and resulted in higher rates of surprise. However, it is not clear why this would not also have been the case in the expected and exposed condition. Here expectation did not appear to modulate behavior to the same extent, even though this latter contrast should have, in principle, isolated the effect of expectation. Possibly here exposure had an effect by satisfying any implicit curiosity about the unexpected event that may otherwise be present in the expected condition. This is speculative though and warrants further investigation.

In addition to finding a robust surprise effect to the first presentation of the motion cue, we also found RT interference associated with presentations 2–8 of the motion cue in all four conditions. These results are consistent with previous findings showing that an infrequently presented salient distractor in visual search continues to interfere with search performance (e.g., Geyer, Müller, & Krummenacher, 2008). Consistent previous empirical work (Asplund et al., 2010; Horstmann, 2002, 2005, 2006; Kazmerski & Friedman, 1995; Retell et al., 2015; Schűtzwohl, 1998) and theoretical accounts of surprise and novelty (Meyer et al., 1997; Sokolov, 1963; Sokolov & Vinogradova 1975), this interference dissipated as a function of exposure to the motion cue in all four conditions (see Fig. 3). Critically, this pattern of results demonstrates that exposure to the motion cue, though infrequent, led to a reduction in the RT interference produced by the motion cue during the experimental task. However, exposure to the motion cue did not have this effect when it occurred prior to commencing the experimental task as a result of exposure in the “exposed” and “expected and exposed” conditions.

Finally, any concern that our effects do not reflect surprise as it has previously been reported, which would account for the lack of effect of exposure and expectation, should be placated by the result of Experiment 3. Though the difference between valid and invalid motion did not reach significance, there is a trend in the data that is strikingly consistent with previous reports of surprise (Becker & Horstmann, 2011; Horstmann, 2002, 2005; Retell et al., 2015); invalid RTs appear slowed relative to valid RTs suggesting a spatial component to the interference induced by the infrequent motion singleton.Footnote 4 Though our specific paradigm deviates somewhat from those previously used to investigate surprise, conceptually it is very similar. Therefore, we have little reason to suspect that our surprise effects are any different, at a phenomenological level, to those that have previously been reported in the visual search literature.

Thus, the results reported here suggest that surprise is highly context-specific. To attenuate the initial surprise response, it is apparently necessary to present an irrelevant stimulus inside the task, or as part of the ongoing task. More strikingly, the failure to obtain a decrease of the surprise response in the expected and expected and exposed conditions suggests that a reduction of the surprise response is in some sense independent of top-down knowledge. If we define the “unexpectedness value” of a stimulus as its propensity to elicit surprise then our manipulations show that being informed about the impending occurrence of the surprising stimulus did nothing to reduce the unexpectedness value of the stimulus. This is not to say that “unexpected” stimuli cannot be ignored, just that prior knowledge of their occurrence appears insufficient to prevent distraction in the context of visual search. Of course, there may be other contexts in which expectation, as it was manipulated here, affects the manifestation of surprise and or interference. Indeed, Niepel (2001) has reported reduced RT interference from a deviant tone in a repetition change paradigm when participants knew exactly when to expect it. Interestingly, the reduction in RTs corresponded with a reduced subjective report of surprise. In our experiments participants were given imprecise information about when the unexpected event would occur. This was a deliberate decision designed to discourage participants from actively searching for the unexpected stimulus. However, the inability to predict in time when an unexpected event will occur may very well be critical to the surprise response. Unfortunately, the differences between our paradigm and Niepel’s (2001) outnumber the similarities making a comparison of the results difficult. The most notable difference being the modality used to deliver the deviant stimulus; there may well be differences in our ability to suppress irrelevant input from separate sensory modalities.

From previous work we know that for surprise to manifest it is not sufficient that a stimulus is new and unannounced (Schűtzwohl, 1998; Horstmann, 2005). Our results indicate that it is also not necessary for a stimulus to be new and/or unannounced to elicit surprise. This perhaps counterintuitive result is not inconsistent with classical theories of attention that assume that salient irrelevant stimuli are inhibited/filtered out through a process that depends on prior exposure to irrelevant stimuli and implicit learning about their relevance (Becker, 2007; Folk & Remington, 1998; Treisman & Sato, 1990). Note that our procedure may have provided participants only with explicit knowledge about the infrequent motion stimulus, whereas implicit learning may be highly context-specific and require that the to-be-inhibited stimulus is presented in the context of the task at hand. The implicit learning explanation of the results is also consistent with the finding that the effects of the motion cue were attenuated over the course of multiple presentations, but that it continued to produce interference. Of note, stimuli that occur infrequently provide few opportunities for the system to learn the necessary characteristics that require inhibiting. Consequently, they are not included in the formation of contextual expectations that describe the characteristics of irrelevant, to-be-inhibited stimuli.

Whether or not the implicit learning account above is correct, the results provide new insights into the factors that determine the neuronal model, schemata, or expectations which in turn determine orienting to surprising stimuli (e.g., Horstmann, 2005; Meyer et al., 1991, Sokolov, 1963): First, the expectations determining the orienting response are apparently highly task specific or context-specific for exposure to show an effect. Secondly, explicit knowledge of an unexpected event or stimulus apparently does not alter the expectancies that ultimately determine the surprise response, indicating that the neuronal model or schemata may be based on implicit knowledge or predictions about upcoming events. We propose that surprise reflects the foreseeability of a stimulus or event according to inductive processes that operate largely automatically in the traditional sense (Posner & Snyder, 1975) and independent of other cognitive processes (Green, 1956).