The effects of saccade-contingent changes on oculomotor capture: salience is important even beyond the first oculomotor response

Abstract

Whenever a novel scene is presented, visual salience merely plays a transient role in oculomotor selection. Unique stimulus properties, such as a distinct and, thereby, salient color, affect the oculomotor response only when observers react relatively quickly. For slower responses, or for consecutive ones, salience-driven effects appear completely absent. To date, however, the circumstances that may reinstate the effects of salience over multiple eye movements are still unclear. Recent research shows that changes to a scene can attract gaze, even when these changes occur without a transient signal (i.e., during an eye movement). The aim of the present study was to investigate whether this capture is mediated through salience-driven or memory-guided processes. In three experiments, we examined how the nature of a change in salience that occurred during an eye movement affected consecutive saccades. The results demonstrate that the oculomotor system is exclusively susceptible to increases in salience from one fixation to the next, but only when these increases result in a uniquely high salience level. This suggests that even in the case of a saccade-contingent change, oculomotor selection behavior can be affected by salience-driven mechanisms, possibly to allow the automatic detection of uniquely distinct objects at any moment. The results and implications will be discussed in relation to current views on visual selection.

Introduction

In the literature on attention and oculomotor behavior, it is a common view that goal-driven and stimulus-driven signals both influence target selection. Not only do the goals of an observer determine where attention will be allocated, but also the properties of stimuli themselves affect selection (e.g., Corbetta & Shulman, 2002; Theeuwes, 1994). However, a number of relatively recent studies suggest that the stimulus-driven effects on oculomotor behavior are remarkably transient (Donk & van Zoest, 2008; Henderson, Weeks, & Hollingworth, 1999; Siebold, van Zoest, & Donk, 2011; van Zoest & Donk, 2005, 2006; van Zoest, Donk, & Theeuwes, 2004; van Zoest, Donk, & Van der Stigchel, 2012). Does that imply that visual salience is merely of minor relevance in the control of our everyday eye movements? In a series of experiments, we investigated how salience changes implemented during an eye movement affect subsequent oculomotor selection behavior. What are the circumstances under which stimulus-driven signals can be given a significant role to play beyond a first eye movement in a visual scene?

In simple search tasks, merely those saccades that are initiated rapidly after the presentation of a search display appear to be stimulus driven, since only then does performance vary as a function of target and distractor salience (Siebold et al., 2011; van Zoest & Donk, 2005, 2006; van Zoest et al., 2004). In fact, an approximate 250 ms after search display onset, participants often fail to move their eyes toward the most salient singleton in a display (Donk & van Zoest, 2008). Furthermore, the influence of salience is not reinstated for a second oculomotor response (Siebold et al., 2011) and so, presumably, neither for the eye movements beyond that. For instance, in Experiment 2 of Siebold et al., participants were asked to make a speeded saccade toward the only right-tilted singleton in a search display consisting of a grid of vertical line segments and two left-tilted singletons. Relative to the orientation of the background and distractor lines, the target could be most, medium, or least salient. The results indicated that, whereas target salience influenced the initial saccade, second saccades were completely unaffected. These results imply that even though the retinal input alters dramatically from one fixation to another, the effects of salience do not reappear. It seems as if salience effects fade away with time, irrespective of oculomotor behavior.

Nevertheless, the findings reported by Siebold et al. (2011) do not completely rule out that oculomotor behavior is susceptible to salience-driven signals beyond an initial response. The singletons presented by Siebold et al. were defined by orientation contrast, a static feature contrast that has been shown to be less effective in capturing attention and gaze than singletons defined by luminance changes or abrupt onsets (Jonides & Yantis, 1988). Accordingly, the singletons in Siebold et al. might just not have been salient enough to capture the eyes beyond an initial eye movement.

Recently, Siebold and Donk (in press) examined this possibility by manipulating salience through transient luminance changes rather than static orientation contrasts. In one experiment (Experiment 2), observers were presented with search displays containing two fixation dots, a target (a right-tilted line), and a distractor (a left-tilted line) embedded in a raster of vertical line segments. Observers initially had to fixate on the location of one of the fixation dots. There were two main conditions: (1) Participants had to make an eye movement directly toward the target when the search display appeared, or (2) participants had to first make an eye movement to the second fixation dot, followed by an eye movement toward the target. Furthermore, on each trial, the salience of either the target or the distractor was enhanced by a luminance flicker, a rapid continuous change from white to dark gray. The results demonstrated that the highly salient luminance flicker affected oculomotor selection only when participants were to move to the target directly. When an eye movement was preceded by another one from the first to the second fixation dot, oculomotor selection was completely unaffected by salience. These results suggest that even though a luminance flicker is highly salient, its effects on oculomotor selection are restricted in time and limited to fast and first eye movements only. These results are in line with a large body of previous work suggesting that the capture effects of sensory transients are short-lived (e.g., Nakayama & Mackeben, 1989; Posner & Cohen, 1984).

Importantly, however, Siebold and Donk (in press) also had a condition in which the onset of the flicker and the onset of the search display did not co-occur. Instead, one of the singletons started to flicker during the initial eye movement. The results in this condition revealed that, even though observers were functionally blind during the onset of the flicker, salience effects reappeared for the second eye movement, since these were biased to move to the flickering singleton. Furthermore, in another experiment (Experiment 3), Siebold and Donk demonstrated that salience effects could be reinstated not only by the introduction of a luminance flicker during the saccade, but also by a plain increase in luminance. When the luminance of one of the singletons was increased during the observer’s first eye movement, the consecutive eye movement was biased toward the singleton with the increased luminance value. However, taken together, the results of Siebold and Donk do not necessarily point out that the gaze can be captured by a change in salience. They merely show that a saccadic-contingent change of an object can disrupt subsequent eye movement. What exactly lies at the roots of this oculomotor capture remains open for discussion.

It is possible that the capture of the gaze is the result of a local identity mismatch between the memory representation of the initial display and the perceptual representation of the changed display. Accordingly, Brockmole and Henderson (2005, 2008; Matsukura, Brockmole, Boot, & Henderson, 2011; Matsukura, Brockmole, & Henderson, 2009) proposed that if changes in a visual scene are not accompanied by a transient signal, the visual system can detect those changes only on the basis of such a comparison process. According to this view, any identity mismatch between a current perception and its corresponding memory representation may attract the eyes. Consequently, any change may alert the visual system and, potentially, attract the eyes regardless of the direction in which the involved salience levels alter.

On the contrary, the capture of the eye in Siebold and Donk (in press) could simply be the end product of salience-driven processes alone. Visual selection might be based on the activity distribution in a salience map (Itti & Koch, 2001; Koch & Ullman, 1985; Li, 2002; Treisman & Gelade, 1980; Treisman & Sato, 1990; Wolfe, 1994; Wolfe, Cave, & Franzel, 1989). Shortly after the onset of a visual scene, this salience representation contains information concerning the relative salience of certain locations, leading to prioritized selection of salient over less salient locations. As was mentioned before, the role of salience fades away rapidly, since only fast eye movements are affected (Siebold et al., 2011; van Zoest & Donk, 2005, 2006; van Zoest et al., 2004). Accordingly, it might well be that after some time has passed, like after a first saccade, the salience map may only include information about where salient objects are in a background, lacking all information concerning how salient those objects are (Donk & Soesman, 2011). This should then imply that the activity levels corresponding to the different locations varying in salience may eventually become equivalent. When, as in Siebold and Donk (in press), the salience of one location is enhanced during an eye movement, the corresponding activity in the map may become larger than the activity at the other locations, possibly leading to a renewed, albeit temporally limited, salience effect. However, when the salience of a location is reduced, the corresponding activity in the map may become smaller than that at the other locations. This may lead either to a deprioritization of the specific location or to no effect at all.

The aim of the present study was to discriminate between these two possibilities. We employed a task and search display similar to those in Siebold and Donk (in press), with the difference that the salience value of one the singletons could remain constant, increase, or decrease during the first eye movement. In Experiment 1, a salience increase and decrease was realized through an increase and decrease in luminance, respectively. In Experiment 2, a luminance decrease resulted in a higher salience value, whereas a luminance increase led to a lower salience value. Since, in both Experiments 1 and 2, the changed display always consisted of two singletons with differing salience values, in Experiment 3, we investigated how a contingent salience change affects oculomotor selection under conditions in which this change does not result in a single maximal salience level; that is, the changed display always consisted of two equally salient singletons.

Experiment 1

In Experiment 1, participants were asked to make a saccadic eye movement toward a target singleton (left tilted line) in a field of vertically orientated line segments and a distractor singleton (right tilted line). Prior to this saccade, participants had to make an eye movement from a first to a second fixation point. During this first eye movement, the salience value of one of the two singletons was increased by a luminance increase, decreased by a luminance decrease, or unchanged. The changed display always consisted of one singleton with a high luminance value and one singleton with a luminance value equal to those of the background line segments. It was equally likely for the target and distractor to be the single most salient singleton in the changed display. If oculomotor capture is mediated by the detection of a mismatch between a memory representation of the presaccadic display and the perceptual representation of the current display, any change, irrespective of its direction, should be sufficient to evoke capture (Brockmole & Henderson, 2005, 2008; Matsukura et al., 2011; Matsukura et al., 2009). Accordingly, the changed item should attract the gaze in both the increase and the decrease conditions. On the contrary, if capture is truly salience driven, it should depend on the direction of the change. According to this view, the changed item should attract the gaze only in the increase condition, whereas in the decrease condition, the change should lead either to a deprioritization of the specific location or to no effect at all.

Method

Participants

Twelve students (18–30 years old; 10 female) were tested at the Vrije Universiteit Amsterdam. All reported having normal or corrected-to-normal vision. All signed an informed consent form to participate in the experiment. The experiment was conducted in accordance with the guidelines of the Helsinki Declaration.

Apparatus

All stimuli were presented using a Pentium IV computer (2.3 GHz). The display, a 21-in. SVGA monitor, had a refresh rate of 100 Hz (resolution, 1,024 × 768 pixels). A chinrest was situated at a distance of 68 cm from the screen.

Monocular movements were tracked using the Eyelink 1000 system (Tower model, infra-red video-based; SR Research Ltd., Canada), with a resolution of 1000 Hz (temporal) and 0.01° (spatial).

Stimuli and design

Each trial started with a drift correction in which participants were required to press the space bar while fixating on a cross (0.80° × 0.80°) in the middle of the screen. After the drift correction, a fixation dot (r = .11°) was presented either above or below the center of the screen for 1,000 ms (see Fig. 1). Participants had to make a preparatory eye movement toward the dot and stay fixated until the search display appeared.

Fig. 1

An overview of a typical trial sequence in the increase condition

The search display consisted of two fixation dots, a target (a line segment tilted 15° toward the left), and a distractor (a line segment lilted 15° toward the right), embedded in a grid of vertically oriented line segments (19 × 19 segments, sized 1.01° × 0.11°, center-to-center distance of 1.06°) and a dark background (3.53 cd/m²). Participants had the task of making a speeded saccade toward the other fixation dot (the first eye movement), followed by a saccade toward the target (the second eye movement).

The display was configured such that the second fixation dot was presented opposite to the first one; that is, when the first dot was presented in the lower half of the display, the second was presented in the uppers half, and vice versa. Target and distractor were always presented to the left and right, 6.38° of the center of the display, and they randomly switched positions from trial to trial.

The experiment consisted of a 2 × 3 within-subjects design: target salience (high and low) and condition (constant, decrease, and increase) (see Fig. 2). Both target salience and condition were varied within blocks of trials. In the constant condition, the luminance of the target and the distractor remained unchanged throughout each trial. That is, either the target or the distractor had a high luminance (46.6 cd/m²), while the luminance of the other elements was low (16.5 cd/m²) from the beginning of the trial to the end. In the other two conditions, the increase and the decrease conditions, the luminance of either the target or the distractor was changed contingent upon the eye movement from the first to the second fixation dot.

Fig. 2

An overview of the three conditions

Luminance changes in the decrease and increase conditions were implemented as soon as the registered eye position left an area of 3° around the first fixation dot. Importantly, the changed display was equal over all conditions and always contained either a high luminance target or a high luminance distractor, resulting in high target salience or low target salience, respectively. The presentation of the changed display was terminated until 250 ms after the eyes landed in an area of 3° around either the target or the distractor. If the eyes directly landed on one of these singletons without hitting the second fixation point first, or whenever participants moved their eyes before the display appeared, an error tone (500 Hz, 500 ms) was presented.

Each participant completed a total of 480 trials, which were divided in four blocks of 120 trials each. Accuracy and speed feedback were provided after each block of trials. Each block included an equal number of trials corresponding to each combination of target salience and condition. In two of the four blocks, the first fixation dot was presented at the top of the display, whereas in the other two blocks, the first fixation dot was presented at the lower part of the display (in counterbalanced order). It took participants approximately 60 min to finish the experiment.

Results

An eye movement was classified as valid if the velocity exceeded 35°/s and the acceleration exceeded 9,500°/s². The saccadic latency of a first eye movement was defined as the time interval between the onset of the search display and the initiation of the first saccade. The saccadic latency of a second eye movement was defined as the time interval between the start of the fixation at the second fixation dot and the initiation of the second eye movement.

Trials were excluded from further analyses when the saccadic latency of either the first or the second eye movement was below 50 ms or above 700 ms (14.6 %), when the first eye movement failed to reach an area of 3° around the second fixation dot in a single saccadic movement (3.0 %) or when the second eye movement failed to reach an area of 3° around either one of the two singletons in a single saccadic movement (1.9 %).

The saccadic latencies of second eye movements were rank ordered from fastest to slowest responses and divided into four bins separately per condition (i.e., separately for the constant, decrease, and increase conditions) and participant. Then, individual proportions correct were calculated for each bin and level of target salience, averaged over participants. The results are depicted in Fig. 3. An ANOVA was performed on the individual proportions correct with the factors condition (constant, increase, decrease), target salience (high, low), and saccade latency bin (1, 2, 3, 4). The results show that there is a main effect of condition, F(2, 22) = 12.557, p < .001, η ²_p =.533, a main effect of target salience, F(1, 11) = 8.727, p < .05, η ²_p =.442, and no effect of saccade latency bin, F(3, 33) = 1.600, p > .2. There was a significant interaction between condition and target salience, F(2, 22) = 7.146, p < .005, η ²_p =.394. There were no further significant two-way interactions [condition × saccade latency bin, F(6, 66) = 1.112, p > .3; target salience × saccade latency bin, F(3, 33) = 1.054, p > .3], but the three-way interaction between condition, target salience, and saccade latency bin was significant, F(6, 66) = 3.300, p < .01, η ²_p =.231, showing that the effect of target salience varied per condition and saccade latency bin (see Fig. 3).

Fig. 3

An overview of the main results of Experiment 1. Illustrated is the proportion of saccades successfully aimed at the target depicted for each level of target salience (high, low), condition (constant, increase, decrease), and saccade latency bin (1, 2, 3, 4). The bars reflect the standard errors of the means

Three ANOVAs were performed to investigate the effects separately within the three conditions. For the increase condition, an ANOVA on individual proportions correct, using the factors target salience (high, low) and saccade latency bin (1, 2, 3, 4) showed an effect of target salience, F(1, 11) = 20.393, p < .005, η ²_p =.650, no effect of saccade latency bin, F(3, 33) = 1.844, p > .1, and a significant target salience × saccade latency bin interaction, F(3, 33) = 5.629, p < .005, η ²_p =.339. A similar ANOVA on the data obtained in the decrease condition showed no effect of target salience, F(1, 11) = 0.079, p > .7, no effect of saccade latency bin, F(3, 33) = 0.661, p > .5, and no target salience × saccade latency bin interaction, F(3, 33) = 2.742, p = .06. Finally, a similar ANOVA on the data obtained in the constant condition showed no effect of target salience, F(1, 11) = 0.009, p > .9, no effect of saccade latency bin, F(3, 33) = 1.911, p > .10, and no significant target salience × saccade latency bin interaction, F(3, 33) = 0.638, p > .6.

To examine whether oculomotor selection performance also varied with the saccadic latencies of the first eye movements, the individual proportions correct were calculated for each saccade latency bin of the first eye movements. Three ANOVAs were performed on individual proportions correct obtained in each of the conditions separately. The Increase condition showed a main effect of target salience, F(1, 11) = 18.836, p < .005, η ²_p =.631, but no effect of first saccade latency bin, F(3, 33) = 0.958, p > .4, and no target salience × first saccade latency bin interaction, F(3, 33) = 0.044, p > .9. Importantly, the other two ANOVAs showed no significant effects (all ps > .3).

To investigate whether the conditions differed with respect to saccadic latency, an ANOVA was performed on the individual latencies of the second eye movements employing the factors condition (constant, increase, decrease) and target salience (high and low). The results show that there is no effect of condition, F(2, 22) = 2.448, p > .1, and no effect of target salience, F(1, 11)=0.269, p > .6, showing that participants were reacting equally fast in each condition.

Discussion

The results of Experiment 1 demonstrate that target salience affects oculomotor selection only when the salience value of one of the singletons increases. When the salience value remains constant or decreases during the first eye movement, target salience does not affect selection. Interestingly, in the case of a salience decrease, neither the changed singleton nor the more salient singleton receives selection priority. This suggests that oculomotor capture cannot be evoked by any change in a scene as could be expected from the account of memory-guided mechanisms (Brockmole & Henderson, 2005, 2008; Matsukura et al., 2011: Matsukura et al., 2009), because that would imply that also the change in the decrease condition should have led to a capture of the gaze. The results show however that capture occurs only when a salience change involves a salience increase. This finding matches the idea that selection can still be influenced by salience-driven processes, even after a first oculomotor response.

It is important to note, however, that a salience increase was realized through a luminance increase. Accordingly, it might well be that the luminance increase accounts for the results of Experiment 1, rather than the concomitant increase in salience. The results of several studies suggest that the effects of luminance increases differ from those of luminance decreases (Boot, Kramer, & Peterson, 2005; Jonides & Yantis, 1988; Miller, 1989; Pratt & McAuliffe, 2001; Theeuwes, 1991; Yantis & Johnson, 1990; Yantis & Jonides, 1984). For instance, Pratt and McAuliffe (2001) showed that onset and offset cues can both induce early facilitation and late inhibition of return effects, but when presented simultaneously, reaction times are shorter when the onset cue, rather than the offset cue, was presented at the target location. A luminance onset had a clear advantage over a luminance offset, even though the change in luminance was of equal size in both conditions. Accordingly, it is possible that the luminance increase in Experiment 1 affected oculomotor selection as a special kind of change, and not necessarily because the salience was changed. In other words, the oculomotor capture that was demonstrated in the increase condition may not be found if another type of change is used to increase salience. Experiment 2 was designed to test this possibility.

Experiment 2

A second experiment was conducted similar to Experiment 1, except that a salience increase was realized by a lowering of the luminance value, whereas a salience decrease was achieved by an enhancement of the luminance value. If a luminance increase is a special case of change, the results of Experiment 2 should be the reversed, relative to those of Experiment 1. However, if a salience increase, rather than a luminance increase, is crucial, the results of Experiment 2 should be similar to those of Experiment 1.

Method

Participants

Twelve students (18–27 years old; 7 female) were tested at the Vrije Universiteit Amsterdam. All reported having normal or corrected-to-normal vision, all signed an informed consent form prior to the study, and all were debriefed afterward.

Apparatus

The apparatus was the same as that in Experiment 1.

Stimuli and design

The stimuli and the design were equal to those of Experiment 1, apart from the luminance values: The search displays consisted of a relatively high background luminance (58.21 cd/m²), as well as a relative high luminance for the background line segments (26.72 cd/m²), whereas a highly salient singleton consisted of a low luminance (0 cd/m²). Importantly, the increase, decrease, and constant conditions again refer to the relative changes in salience, not to the change in luminance.

Results

Trials were excluded from further analyses when the saccadic latency of either the first or the second eye movement was below 50 ms or above 700 ms (8.5 %), when the first eye movement failed to reach an area of 3° around the second fixation dot in a single saccadic movement (5.6 %) or when the second eye movement failed to reach an area of 3° around either one of the two singletons in a single saccadic movement (1.1 %).

As in Experiment 1, an ANOVA was performed on the individual proportions correct using the factors condition (constant, increase, decrease), target salience (high, low), and saccade latency (1, 2, 3, 4). The results show that there is a main effect of condition, F(2, 22) = 5.629, p < .05,  η ²_p =.339, a main effect of target salience, F(1, 11) = 8.892, p < .05, η ²_p =.447, and a main effect of saccade latency bin, F(3, 33) = 10.605, p < .001,  η ²_p =.491. There was one significant interaction: condition × target salience, F(2, 22) = 5.947, p < .05, η ²_p =.351. The other two interactions were merely trending [condition × saccade latency bin, F(6, 66) = 2.196, p = .054, η ²_p =.166; and target salience × saccade latency bin, F(3, 33) = 2.633, p = .066, η ²_p =.193. Furthermore, also the three-way interaction between condition, target salience, and saccade latency bin was not significant, F(6, 66) = 1.566, p > .1 (see Fig. 4).

Fig. 4

An overview of the main results of Experiment 2. Illustrated is the proportion of saccades successfully aimed at the target depicted for each level of target salience (high, low) and condition (constant, increase, decrease). The bars reflect the standard errors of the means

Three separate ANOVAs were performed on the data obtained in the three conditions. For the Increase condition, an ANOVA on individual proportions correct, using the factors target salience (high, low) and saccade latency bin (1, 2, 3, 4) showed an effect of target salience, F(1, 11) = 13.144, p < .005, η ²_p =.544, an effect of saccade latency bin, F(3, 33) = 9.159, p > .001, η ²_p =.454, and a target salience × saccade latency bin interaction that trends toward significance, F(3, 33) = 2.614, p = .06, η ²_p =.192. A similar ANOVA on the data obtained in the decrease condition showed no effect of target salience, F(1, 11) = 1.580, p > .2, no effect of saccade latency bin, F(3, 33) = 1.742, p > .1, and no significant target salience × saccade latency bin interaction, F(3, 33) = 1.148, p > .3. Finally, a similar ANOVA on the data obtained in the constant condition showed no effect of target salience, F(1, 11) = 0.467, p > .5, an effect of saccade latency bin, F(3, 33) = 9.182, p < .001, η ²_p =.455, and no significant target salience × saccade latency bin interaction, F(3, 33) = 1.135, p > .3.

To examine whether oculomotor selection performance also varied with the saccadic latencies of the first eye movements, the individual proportions correct were calculated for each saccade latency bin of the first eye movements. Three ANOVAs were performed on individual proportions correct obtained in each of the conditions separately. The increase condition showed a main effect of target salience, F(1, 11) = 16.184, p < .005, η ²_p =.595, but no effect of first saccade latency bin, F(3, 33) = 0.787, p > .5, and no target salience × first saccade latency bin interaction, F(3, 33) = 0.251, p > .8. Importantly, the other two ANOVAs showed no significant effects (all ps > .3).

Discussion

Experiment 2 shows a similar pattern of results as Experiment 1: Target salience affected selection behavior only in the increase condition. This result was found even though the salience increase now involved a decrease in luminance. This supports the view that an object does not have to become brighter in order to capture the gaze. Instead, an object needs to become more conspicuous, more distinct, to affect oculomotor selection. Together, the results of Experiments 1 and 2 demonstrate that a saccade-contingent change leads to oculomotor capture only when this change results in an increased salience value.

These results are not in line with the assumption that nontransient changes can only lead to memory-guided prioritization (Brockmole & Henderson, 2005). On the basis of the absence of oculomotor capture in the decrease conditions, in both Experiments 1 and 2, one may wonder whether matching or mismatching memory representations are a relevant factor in the present paradigm at all. However, it is possible that oculomotor capture in the decrease conditions was obscured by the lower salience level of the changed singleton. More specifically, it could have been that the gaze was attracted to the changed location on half of the trials and to the highest salience value on the other half. It may be for this reason that no capture occurred in the decrease conditions.

A third experiment was performed to examine this possibility. Experiment 3 was similar to Experiment 1, except that a contingent change now leads to equivalent, rather than different, salience levels in the changed display. If oculomotor capture can be evoked by any change, as predicted by a memory-guided capture account (Brockmole & Henderson, 2005), the gaze should be attracted to the changed singleton in both the Increase and the decrease conditions. Alternatively, if a contingent change results in salience-driven capture, the gaze should no longer be attracted to the changed singleton, because the salience values of the singletons in the changed display were equivalent.

Experiment 3

A third experiment was conducted to examine the effect of contingent salience changes that lead to equivalent, rather than distinct, salience levels in the changed display. That is, in the case of a salience increase, the changed singleton became more salient so as to match the other singleton that already was highly salient from the start of the trial. In the luminance decrease condition, the changed singleton became less salient so as to match the other singleton’s lower salience level. Since the increase and decrease manipulations led to a different changed search display, in this third experiment, we investigated the change type, instead of the salience level of the target in the changed display. The change type condition could be defined as a target change, a distractor change, or no change. These conditions were analyzed for the increase and decrease manipulations separately.

Method

Participants

Twelve students (19–30 years old; 9 female) were tested at the Vrije Universiteit Amsterdam. All reported having normal or corrected-to-normal vision, all signed an informed consent form prior to the study, and all were debriefed afterward.

Apparatus

The apparatus was the same as that in Experiment 1.

Stimuli and design

The aim of Experiment 3 was to investigate the effect of contingent salience changes that lead to equivalent, rather than distinct, salience levels in the changed display. To this end, the salience increase manipulation involved a singleton with a constant high salience (luminance of 46.6 cd/m²) and a second singleton of which the salience was increased (luminance value of 16.5 cd/m² to 46.6 cd/m²) during the participant’s first eye movement. As a result, the display contained two highly salient singletons. A decrease manipulation was applied in a similar way; there was one singleton present with a high luminance value (46.6 cd/m²), a value that was decreased during the participant’s first eye movement such that it matched the luminance value of the other singleton (16.5 cd/m²). Consequently, the changed search display eventually contained two singletons that were both of a low salience.

Importantly, the increase and decrease manipulations lead to different changed displays. Therefore, we analyzed the increase and decrease manipulations separately. An overview of this design is shown in Fig. 5. Interestingly, this design allowed us to test the difference between two different constant conditions, so as to investigate what the absolute effect of a brighter singleton was as opposed to singletons that are equiluminant to the background line segments. Moreover, instead of the target salience factor, the factor change type was used with three levels: target change, distractor change, and no change. Except for these changes in the design, the exact same stimuli were used as those in Experiment 1.

Fig. 5

An overview of Experiment 3’s conditions

Results

Trials were excluded from further analyses when the saccadic latency of either the first or the second eye movement was below 50 ms or above 700 ms (9.4 %), when the first eye movement failed to reach an area of 3.99° around the second fixation dot in a single saccadic movement (3.6 %) or when the second eye movement failed to reach an area of 3.00° around either one of the two singletons in a single saccadic movement (2.1 %).

With regard to the increase manipulation, an ANOVA was performed on the individual proportions correct using the factors change type (target change, distractor change, no change) and saccade latency (1, 2, 3, 4). The results show that there is no effect of change type, F(2, 22) = 1.783, p > .1, and no effect of saccade latency bin, F(3, 33) = 1.258, p > .3. There was no significant change type × saccade latency bin interaction, F(6, 66) = 0.650, p > .7. These results, as depicted in Fig. 6, suggest that there is no effect of the salience increase on proportions correct, regardless of the saccade latency.

Fig. 6

An overview of the main results of Experiment 3. Illustrated is the proportion of saccades successfully aimed at the target depicted for the two manipulations (the increase and decreased manipulations), change type (target change, distractor change, no change), and saccade latency bin (1, 2, 3, 4). The bars reflect the standard errors of the means

Furthermore, concerning the decrease manipulation, an ANOVA was performed on the individual proportions correct using the factors change type (target change, distractor change, no change) and saccade latency (1, 2, 3, 4). The results show that there is no effect of change type, F(2, 22) = 1.154, p > .3, and no effect of saccade latency bin, F(3, 33) = 2.406, p = .09. There was no significant change type × saccade latency bin interaction, F(6, 66) = 0.530, p > .7. These results suggest that there is no effect of the salience increase on proportions correct, regardless of the saccade latency.

Two separate t-tests (paired samples) were performed to test the difference between the two no-change conditions of the increase and decrease manipulation, applied to proportions correct and saccadic latency. The resulting statistics show that there is a significant difference neither in proportions correct, t(11) = 0.447, p > .6, nor in saccadic latency, t(11) = 0.040, p > .9, suggesting that the presence of highly salient singletons from the start of the trial does not necessarily make the task easier.

Finally, as Fig. 6 suggests, a performance difference between target change and distractor change conditions, a (paired samples) t-test was performed on the averaged individual proportions correct directly comparing a target change and a distractor change condition. The results showed no difference between the two, t(11) = 1.749, p > .1.

Discussion

The results of Experiment 3 show that selection behavior was unaffected by change type, whether the increase or the decrease manipulation was applied. The exact same physical changes were employed as the ones of Experiment 1, and yet, no oculomotor capture was found. Apparently, the fact that the salience increase did not lead to a uniquely high salience level completely eliminated the influence of the salience increase.

Furthermore, no difference was found between the two no-change conditions, neither in proportion correct nor in saccade latency. Apparently, performance is not altered by the luminance of the singletons per se.

General discussion

The results demonstrate that an increase in salience during an eye movement affects subsequent oculomotor selection behavior (Experiments 1 and 2). This is not due to a luminance increase per se, because a luminance decrease can lead to a similar result (Experiment 2). Furthermore, the salience increase should lead to a uniquely high salience level, since the results of Experiment 3 show that a contingent salience increase does not capture the gaze when the changed display contains two equally salient singletons. Together, the results show that oculomotor selection is not affected by contingent salience changes in general. Objects actually need to obtain a uniquely high salience level in order to exogenously affect the observer’s oculomotor behavior.

It is interesting to note that target information was readily available from the beginning of the trial in all experiments. In fact, the distance to the target was the same for the first and the second fixation dots. There thus was sufficient time for goal-driven mechanisms to take over control. Nevertheless, participants were still susceptible to the salience increases, even though these increases were presented during an eye movement, while participants were virtually blind. Apparently, the visual system is sensitive to relative salience not only immediately after the presentation of a visual scene, but also after a first oculomotor response, in case the salience value of one object is suddenly uniquely high.

The present results are not in line with the assumption that prioritization of contingently changed locations only involve memory-guided mechanisms, as proposed by Brockmole and Henderson (2005). First, from this identity-mismatch account, one could expect contingent changes (as evident as they were in the present experiments) to affect oculomotor behavior generally. However, no capture was found in the decrease condition. Second, there was no capture in Experiment 3, demonstrating that the resulting salience value, rather than the change itself, is crucial in capturing the gaze. Finally, the time course of the effect (fast and short-lived) fits the conception of stimulus-driven capture. Capture was merely found when latencies were short, so the idea that it was mediated by the mismatch of identity between the memory representation and the perceptual representation seems rather unlikely.

The present results are different from those recently reported by Matsukura et al. (2009). In their study, a color change occurring during a saccade also captured the gaze, but this capture was not related to the salience level of the changed region. It appeared to be the case that the prioritization of the color change was temporally delayed; that is, the probability of fixating the changed object was larger for the second saccade than for the first saccade after the change. This suggests that capture was mediated by memory guidance rather than salience. However, Matsukura et al. (2009) used heterogeneous color pictures of real-world scenes. As a result, it is likely that a contingent change in Matsukura et al. (2009) never resulted in a uniquely high salience value. The present results show that a contingent change should result in a unique high salience level in order to capture the gaze in a truly salience-driven manner.

For similar reasons as those mentioned above, the present findings are not quite in line with models of visual search that assume eye movements to be fully top-down controlled. For instance, the target acquisition model (TAM) of Zelinsky (2008) cannot account for the salience effect observed in the increase conditions or for its typical rapid decline. Instead of solely processing the similarities between a target representation and the search scene, and taking that to reflect visual salience, TAM should also deal with information that is not necessarily related to the search target, information such as the relative distinctiveness or uniqueness of regions in the scene. On the contrary, models assuming selection to be at least partly under bottom-up control appear to be more in line with the present findings (Itti & Koch, 2001; Li, 1998; Wolfe, 1994; Wolfe, Cave, & Franzel, 1989), although these models do not explicitly assume salience effects to decline in time. These models often assume visual salience to have a constant influence on attention and gaze, but the present results, together with findings in prior work (Siebold et al., 2011; van Zoest & Donk, 2005, 2006; van Zoest et al., 2004), suggest that the influence of salience changes over time._ENREF_8_ENREF_11 Moreover, newly introduced salience levels in the scene can still have an effect, but only when these are uniquely high. This temporal dynamic in the role of salience is an important finding that should be incorporated into any new or updated model of visual search.

The present results are congruent with the idea that salience functions as a spatiotopic placeholder system to realize a sustained object location representation that is persistent over saccades (Donk & Soesman, 2011; Zhaoping, 2008). Immediately after the presentation of a visual scene, this representation, or salience map, may contain information concerning the relative salience at different locations in the visual field. As time passes, the map loses the information concerning the relative salience of the different locations but still carries information about the presence of distinct locations. Evidence for the existence of such an object location representation derives from a study by Zhaoping. Participants had to indicate the location of a moderately salient target that was presented among multiple nontargets. On a proportion of trials, the search display was only shown briefly and replaced by a mask display after which participants continued their search. Even though, in these “after search” trials, the target, as well as the nontargets, were masked, participants often fixated the target location, occasionally even after several intermittent saccades. Interestingly, participants more accurately reported target location on those trials on which it took multiple eye movements to hit the target than on trials on which target location was not fixated at all in the masked display. From this, Zhaoping concluded that observers may have used some preattentive memory map that carries information about the locations of distinct items. This representation was demonstrated to be different from iconic memory in the sense that it persisted much longer.

With this in mind, it remains a rather striking result that the increase manipulation in Experiment 1 and the one in Experiment 3 lead to different effects, even though exactly the same luminance change was involved. Since the uniqueness of the resulting salience level made all the difference, one possible explanation is that only new and unique levels of salience automatically capture attention and gaze. If a salience enhancement does not lead to a uniquely high salience level, all distinct elements remain just equivalent to the visual system. Together, the present findings may offer a new perspective on why saccade-contingent changes sometimes do and sometimes do not attract gaze. If no new and uniquely high salience levels arise, nothing will be able to distract the observer. In this sense, the mechanisms behind selection are remarkably efficient. They are predominantly at play when observers need to quickly react to novel visual input, but they also ensure that the observer is in control already shortly after. In addition, beyond initial responses, they keep oculomotor behavior flexible to some extent, since they stay sensitive to changes that lead to new and unique information.