We move our eyes three to four times each second as we investigate our visual surroundings. Each one of these eye movements requires a deceptively simple decision: Where should we look next? Despite recent progress in understanding the factors that determine where an observer will eventually look in a scene, we know very little about what determines how an observer decides where he or she will specifically look next. Some evidence suggests that visual salience (Itti & Koch, 2000), occulomotor biases (Tatler & Vincent, 2009), the locations of prior fixations (Dodd, Van der Stigchel, & Hollingworth, 2009; Klein & MacInnes, 1999), semantic content (Underwood & Foulsham, 2006), and momentary task relevance (Hayhoe, Shrivastava, Mruczek, & Pelz, 2003) likely play roles, but identifying the factors that influence saccade sequencing remains an underdeveloped area of research that requires much greater scrutiny (Henderson, 2011). In this article, we consider the degree to which one’s representation of a fixated object’s spatio-temporal properties influences the direction in which a subsequent eye movement will travel.

When viewing a scene, an observer forms several representations regarding objects’ spatial position and temporal dynamics. For example, we use spatial reference frames to define objects’ fronts, backs, left sides, right sides, and so forth. By doing so, we are able to describe a chair as “facing” a certain direction, being to the left of a table, or being behind a stool. In addition to defining the current spatial position of an object in a scene, it is also important to consider the ways in which an object’s position might be altered over time. For example, some objects may be more or less likely to move. Similarly, considering the potential causes of an object’s motion allows one to predict future events. Some objects may require an external force to act upon them, whereas others may be capable of volitional self-movement. One’s assignment of reference frames (e.g., Carlson-Radvansky & Jiang, 1998; Logan, 1996; Logan & Sadler, 1996), representation of any possible motion (e.g., Gervais, Reed, Beall, & Roberts, 2010), and interpretation of animacy (e.g., Pratt, Radulescu, Guo, & Abrams, 2010) each affect covert aspects of attentional control. However, their impacts on overt attention (i.e., gaze control) have not been investigated.

Experiment 1

The purpose of Experiment 1 was to assess whether a fixated object’s spatial orientation in a display influences subsequent shifts of gaze. Our empirical approach was based on the logic of previous work conducted by Marwan Daar and Jay Pratt (2008) examining response selection processes within the context of the spatial–numerical association of response codes (SNARC) effect. The SNARC effect refers to an interaction between numerical magnitude and spatially mapped responses in number-processing tasks. In its initial demonstration, observers were shown to be faster to respond to low numbers with leftward responses (e.g., a keypress with the left hand) and faster to respond to high numbers with rightward responses (Dehaene, Bossini, & Giraux, 1993). Daar and Pratt further showed that in addition to altering the speed with which a spatially mapped response is made, numerical magnitude biases which of several spatially mapped responses an observer will volitionally select. They presented observers with low digits (1 or 2), high digits (8 or 9), or a neutral symbol (@, #, &, or *) and asked them to make one of two keypresses (left or right). Crucially, on each trial, the observers were given free choice of which key to press. When presented with neutral stimuli, observers chose the right keypress approximately 56% of the time, indicating a slight baseline bias to press the right key. Importantly, when high digits were presented, this rightward bias increased to 63%. However, when low digits were presented, the baseline rightward bias decreased (52%). Because the semantic identity of the digit (or symbol) was irrelevant to the task (which was simply to press a key of one’s own choice), Daar and Pratt concluded that an observer’s processing of numerical magnitude influences subsequent decisions regarding response selection.

We applied Daar and Pratt’s (2008) logic to our investigation into the possible influence of an object’s spatio-temporal properties on saccadic decision making (see Fig. 1a). Participants’ eye movements were monitored as they viewed inanimate objects one by one on a computer screen. Viewing single objects allowed us to isolate and manipulate the effects of spatial orientation independently of other factors that might be present in a full scene. Each object was oriented so that it faced the left or the right side of the monitor. Two small squares flanked each object, such that one was in front of the object and the other was behind it. When presented with a go signal, participants moved their eyes from the object to the square of their choice. This eye movement was analogous to the left/right buttonpress made by Daar and Pratt’s (2008) participants. If interpretations of spatial orientation are used to make gaze control decisions, then gaze should be biased to shift in the same direction the object is facing, given the behavioral and functional importance of the area of space in front of an object.

Fig. 1
figure 1

Panel A: An example stimulus is illustrated that depicts one of the neutral nonobjects used across all five experiments. Panel B: A schematic illustration of the procedure. The position of the observer’s gaze is depicted throughout the trial by the cartoon of an eye. The observer was free to shift gaze to either peripheral square

Full size image

Method

Participants

A group of 36 University of Notre Dame undergraduates participated in exchange for course credit or monetary compensation.

Stimuli and apparatus

Example stimuli are illustrated in Fig. 2. Each display consisted of a line drawing, presented in the center of the screen, that was flanked to its left and right by two solid squares. The drawings consisted of inanimate objects (a clothespin, kettle, trumpet, and whistle) adapted from Snodgrass and Vanderwart (1980) and symmetrical nonobjects adapted from Kroll and Potter (1984). In addition to the originals, mirror-reversed versions were created to produce left- and right-facing versions of each drawing. The drawings were adjusted to be subjectively the same size (approximately 7° horizontally and 6° vertically), and the flanking squares subtended 0.55° horizontally and vertically and were positioned 9° to the left or right of center. All items were drawn in black on a gray background.

Fig. 2
figure 2

Objects used in the experiments. Only right-facing objects are depicted. Mirror-reversed versions were created to face left. Row A: Nonobjects used in all experiments. Row B: Objects used in Experiment 1. Row C: Objects used in Experiment 2. Row D: Objects used in Experiment 3. Row E: Objects used in Experiment 4. Row F: Objects used in Experiment 5

Full size image

Stimuli were presented on a 21-in. CRT display with a refresh rate of 120 Hz. Chin and forehead rests maintained a viewing distance of 81 cm. Eye movements were sampled from each participant’s right eye at a rate of 1000 Hz by an EyeLink 2K eyetracking system (SR Research, Inc.), which provided an average spatial accuracy of 0.15°.

Procedure

The procedure is illustrated in Fig. 1b. Participants began the experiment by completing a calibration procedure that mapped the output of the eyetracker onto the display. Calibration was monitored throughout the experiment and was adjusted if necessary. Participants initiated each trial by pressing a button. A fixation cross was presented for a randomly determined interval between 500 and 1,500 ms, after which a stimulus was presented. Participants were instructed to maintain fixation on the central image. After 50, 500, or 1,000 ms had elapsed, a small white square subtending 0.2° horizontally and vertically appeared at the center of the screen for 150 ms. This onset served as a “go signal” and indicated that the observer was to shift his or her gaze from the central figure to one of the peripheral squares. The go signal did not inform the observer which square he or she was to look toward; observers were told to move their eyes to the saccade target of their choice. The variable timing of the go signal’s onset was used to prevent participants from automatizing their saccades across trials. Upon fixating one of the peripheral squares, the trial was terminated. A trial was also terminated if a participant shifted gaze away from the central figure prior to the go signal (such trials were recycled at the end of the experiment).

In the experiment, we employed an entirely within-subjects design. The eight figure identities (kettle, trumpet, Nonobject 1, Nonobject 2, etc.), and two figure orientations (left- vs. right-facing) were fully crossed with a repetition factor of 12, yielding 192 experimental trials. Trials were presented in different random orders across participants.

Results and discussion

Because leftward and rightward saccades together accounted for 100% of the data in each cell of the experimental design, we arbitrarily focused our analyses on the frequency with which observers looked to the right after viewing each image. These frequencies were contrasted across three conditions: objects that faced to the left, neutral nonobjects, and objects that faced to the right (Fig. 3). The nonobject condition established each observer’s baseline tendency to shift gaze to the right. To avoid floor and ceiling effects, seven participants who looked to the right on less than 5% or more than 95% of these neutral trials were excluded from our analyses.

Fig. 3
figure 3

Results from all five experiments. Percentages of rightward saccades are illustrated as a function of object identity (objects, tilted objects, vehicles, animals looking at the participant, or animals looking in the direction of travel) and object type (leftward-facing, neutral, or rightward-facing)

Full size image

A one-way repeated measures analysis of variance (ANOVA) indicated that the frequency of rightward saccades depended on the nature of the object at fixation, F(2, 56) = 20.2, p < .001, ω 2 = .244. Observers looked to the right on 46% of the neutral nonobject trials. This slight leftward bias increased when objects faced to the left (32% rightward saccades) and decreased when objects faced to the right (60% rightward saccades). In other words, relative to their neutral baseline tendencies, observers were biased to shift their gaze in the direction the object faced.

A subsequent analysis considered the strength of the gaze bias as a function of the stimulus onset asynchrony (SOA) between the onset of the object and the go signal. Although SOA was manipulated to prevent participants from automatizing their saccades across trials, longer SOAs nevertheless provided participants more time to process each object before executing an outbound saccade. This afforded us an opportunity to examine a potential link between viewing time and the degree to which gaze decisions were based on spatial orientation. In this experiment, the time spent viewing the object did not alter gaze control decisions: The main effect of SOA was not reliable, F(2, 56) = 1.65, p = .20, nor did it interact with object orientation, F(4, 112) = 1.09, p = .36. This suggests that the processing time needed for spatial orientation to inform the location of the next saccade was fully completed even in our shortest SOA condition.

To place this timing in context, we determined the average saccade latencies (i.e., the elapsed time between the onset of the go signal and the start of the saccade) in the shortest SOA condition (50 ms). The average saccade latencies were 527 ms, indicating a total object viewing time of 577 ms before the saccade was executed. This is substantially longer than the saccade latencies typically observed under normal scene-viewing conditions, in which the eyes move three or four times per second (Rayner, 1998). It is perhaps not surprising, then, that SOA had no effect on the gaze biases observed in this experiment.

In summary, the results of Experiment 1 suggest that the determination of a fixated object’s front and back through the application of a spatial reference frame can influence subsequent shifts of gaze. Experiment 2 was designed to provide further evidence for this conclusion.

Experiment 2

In Experiment 1, the objects were directly aligned with the saccade target squares. In Experiment 2, we rotated the objects 45° so that they now faced the lower left and lower right corners of the monitor (Fig. 2). We did not, however, alter the locations of the saccade target squares. As a result, the alignment between the facing direction of the objects and the saccade direction choices available to the participants was reduced. If participants are biased to look in the direction an object faces, he or she should do so more often when the experiment permits him or her to do so (Exp. 1) than when it does not (Exp. 2). To take one concrete example as an illustration, an object facing down-and-to-the-left may still bias leftward saccades, but it should do so at a rate smaller than that observed for objects facing directly to the left.

Method

Participants

A group of 34 University of Notre Dame undergraduates participated in exchange for course credit or monetary compensation. None had participated in Experiment 1.

Stimuli, apparatus, and procedure

The stimuli, apparatus, and procedure were the same as in Experiment 1, except that the objects used in Experiment 1 (clothespin, kettle, trumpet, and whistle), were rotated 45°, so that they either faced down-and-to-the-left or down-and-to-the-right. Although the stimuli could have also been rotated to face up-and-to-the-left and up-and-to-the-right, we elected not to include these additional directions, in order to keep the number of possible object orientations (i.e., two) the same across Experiments 1 and 2.

Results and discussion

The initial analyses paralleled those in Experiment 1 (Fig. 3). Five participants were excluded because they looked to the right on less than 5% or more than 95% of the neutral nonobject trials. As in Experiment 1, the frequency of rightward saccades depended on the nature of the fixated objects, F(2, 56) = 6.27, p = .003, ω 2 = .061. Observers looked to the right on 50% of the neutral nonobject trials. This bias decreased when objects faced to the left (42% rightward saccades) and increased when objects faced to the right (53% rightward saccades). As in Experiment 1, the strength of this gaze bias did not vary as a function of the SOA between the onset of the object and the go signal: The main effect of SOA was not reliable, F(2, 56) < 1, nor was the interaction with object orientation, F(4, 112) < 1.

To compare the results of Experiment 2 with those of Experiment 1, we conducted a 2 (Experiment) × 3 (Cue Direction) mixed-model ANOVA. As expected, the main effect of cue direction was reliable, F(2, 112) = 25.3, p < .001. Importantly, the magnitudes of this effect were different across the experiments, as indicated by a reliable interaction term, F(2, 112) = 4.55, p = .013. Specifically, the orientation-based biases on saccade direction were reduced when the objects were misaligned with the available saccade targets. Experiments 1 and 2, therefore, provide strong evidence that in the absence of other pictorial cues, a representation of an object’s spatial orientation can influence an individual’s gaze control decisions. In Experiment 3, we considered whether an object’s potential for motion can augment this bias.

Experiment 3

In Experiments 1 and 2, the objects were stationary and no implication of upcoming or ongoing motion was provided. Some objects, however, are strongly associated with motion (e.g., vehicles). Implied motion affects visual cognition at multiple levels, ranging from perception (e.g., Pavan, Cuturi, Maniglia, Casco, & Campana, 2011; Winawer, Huk, & Boroditsky, 2008), to attention (Shi, Weng, He, & Jiang, 2010), to memory (e.g., Freyd & Finke, 1984). For example, still images of people engaging in activities such as throwing a ball have been shown to direct spatial attention in the direction of the implied motion (Gervais, Reed, Beall, & Roberts, 2010). If representations of possible or implied motion similarly influence gaze, then the biases associated with the direction an object is facing and a potential bias to shift gaze in the direction it is moving should be additive.

Method

Participants

A group of 35 University of Notre Dame undergraduates participated in exchange for course credit or monetary compensation. None had participated in Experiment 1 or 2.

Stimuli, apparatus, and procedure

The stimuli, apparatus, and procedure were the same as in Experiment 1, except that the immobile objects used in Experiment 1 (clothespin, kettle, trumpet, and whistle), were replaced by drawings of vehicles (airplane, bus, car, and truck). These stimuli were also adapted from Snodgrass and Vanderwart (1980), and both left- and right-facing versions were created.

Results and discussion

The initial analyses paralleled those in prior experiments (Fig. 2). Five participants were excluded because they looked to the right on less than 5% or more than 95% of the neutral nonobject trials. As in Experiment 1, the frequency of rightward saccades depended on the nature of the fixated objects, F(2, 58) = 17.7, p < .001, ω 2 = .159. Observers looked to the right on 52% of the neutral nonobject trials. This bias decreased when objects faced to the left (45% rightward saccades) and increased when objects faced to the right (66% rightward saccades). As in previous experiments, the strength of this gaze bias did not vary as a function of the SOA between the onset of the object and the go signal: The main effect of SOA was not reliable, F(2, 62) = 1.24, p = .30, nor was the interaction with object orientation, F(4, 124) < 1.

To consider possible additive effects of spatial orientation and implied motion, the results of Experiment 3 were compared to those of Experiment 1 with a 2 (Experiment) × 3 (Cue Direction) mixed-model ANOVA. As expected, the main effect of cue direction was reliable, F(2, 114) = 37.6, p < .001. Importantly, the magnitudes of this effect were not different across the experiments, as indicated by a nonreliable interaction term, F(2, 114) = 1.03, p = .36. That is, vehicles did not cue gaze in the direction of implied motion more frequently than did stationary objects in the direction of their orientation. A representation of motion, therefore, does not necessarily influence an individual’s gaze control decisions. In Experiment 4, we considered whether one’s representation of the potential cause(s) of an object’s motion can enhance the saccade direction bias observed in Experiment 1.

Experiment 4

In Experiment 3, the objects were associated with motion, but could not move unless acted upon by an external force. Some objects, however, are capable of volitional self-motion (e.g., animals). The visual system seems to be tuned to detect animacy in the environment (Scholl & Tremoulet, 2000), and observers freely anthropomorphize abstract shapes that move in ways reminiscent of chasing or stalking behavior (Gao & Scholl, 2011). Such “animate” motion seems to capture covert attention more efficiently than inanimate motion (Pratt et al., 2010). Thus, if representations of animacy similarly influence gaze, then the biases observed in Experiment 1 should be larger after viewing objects capable of animate motion.

Method

Participants

In all, 30 University of Notre Dame undergraduates participated in exchange for course credit or monetary compensation. None had participated in Experiments 1–3.

Stimuli, apparatus, and procedure

The stimuli, apparatus, and procedure were the same as in Experiment 3, except that the vehicles (airplane, bus, car, and truck), were replaced by drawings of animals (bear, deer, fox, and raccoon). These stimuli were again adapted from Snodgrass and Vanderwart (1980), and both left- and right-facing versions were created. In the original versions of each drawing, each animal was itself looking in the direction of its motion. To avoid confounding each animal’s direction of motion and direction of gaze (cf. Friesen & Kingstone, 1998), an artist redrew the head of each animal so that it was looking at the participant.

Results and discussion

The analyses paralleled those in previous experiments (see Fig. 3). Two participants were excluded because they looked to the right on less than 5% or more than 95% of the neutral nonobject trials. As in Experiments 1 and 2, the frequency of rightward saccades depended on the nature of the fixated objects, F(2, 54) = 21.8, p < .001, ω 2 = .197. Observers looked to the right on 56% of the neutral nonobject trials. This bias decreased when objects faced to the left (42% rightward saccades), and increased when objects faced to the right (67% rightward saccades). As in the previous experiments, the strength of this gaze bias did not vary as a function of the SOA between the onset of the object and the go signal: The main effect of SOA was not reliable, F(2, 54) < 1, nor was the interaction with object orientation, F(4, 108) < 1.

To consider possible additive effects of implied motion and animate motion, the results of Experiment 4 were compared to those of Experiment 1 with a 2 (Experiment) × 3 (Cue Direction) mixed-model ANOVA. As expected, the main effect of cue direction was reliable, F(2, 110) = 41.4, p < .001. Once again, however, the magnitudes of this effect were not different across the experiments, F(2, 110) < 1. Thus, neither vehicles (nonanimate) nor animals (animate) cued gaze in the direction of motion more frequently than did other cues. A representation of animacy, therefore, did not independently influence individuals’ gaze control decisions in this experiment.

Experiment 5

In Experiment 4, we had intentionally decoupled the animals’ implied directions of motion and gaze. Gaze cues, however, can create strong biases of both covert and overt attention (e.g., Friesen & Kingstone, 1998; Mansfield, Farroni, & Johnson, 2003; see Frischen, Bayliss, & Tipper, 2007, for a review). Thus, we repeated Experiment 4 under conditions in which the depicted animals were looking in their own direction of travel, instead of at the participant.

Method

Participants

A group of 32 University of Notre Dame undergraduates participated in exchange for course credit or monetary compensation. None had participated in Experiments 1–4.

Stimuli, apparatus, and procedure

The stimuli, apparatus, and procedure were the same as in Experiment 4, except that the original Snodgrass and Vanderwart (1980) versions of each animal drawing were used, such that each animal was itself looking in the direction of its motion. As in previous experiments, both left- and right-facing versions were created.

Results and discussion

The analyses paralleled those in previous experiments (see Fig. 3). One participant was excluded from the analysis because he or she looked to the right on less than 5% or more than 95% of the neutral nonobject trials. As in all previous experiments, the frequency of rightward saccades depended on the nature of the fixated objects, F(2, 60) = 10.1, p < .001, ω 2 = .123. Observers looked to the right on 55% of the neutral nonobject trials. This bias decreased when objects faced to the left (43% rightward saccades) and increased when objects faced to the right (64% rightward saccades). As in previous experiments, the strength of this gaze bias did not vary as a function of the SOA between the onset of the object and the go signal: The main effect of SOA was not reliable, F(2, 60) < 1, nor was the interaction with object orientation, F(4, 120) < 1.

To consider possible additive effects of implied motion, animacy, and gaze direction, the results of Experiment 5 were compared to those of Experiment 1 with a 2 (Experiment) × 3 (Cue Direction) mixed-model ANOVA. As expected, the main effect of cue direction was reliable, F(2, 116) = 28.7, p < .001. Importantly, the magnitudes of this effect were not different across the experiments, F(2, 116) < 1. Hence, even the combination of implied motion, animacy, and depicted intention (through gaze) did not influence gaze control decisions more than spatial orientation alone. One potential caveat to this final conclusion, however, is our use of animals instead of humans as the stimuli in Experiment 5. Our selection of animals was practical, given our use of the Snodgrass and Vanderwart (1980) stimulus set. It remains possible, and even likely, that gaze cues in illustrations of human faces could produce stronger effects within the present paradigm. Indeed, such human-based gaze cues have been strong predictors of behavior in a variety of perceptual and attentional tasks (see Frischen et al., 2007).

Conclusions

Research over the past several decades has unequivocally shown that gaze control is not the product of random fixation point selection (e.g., Buswell, 1935; Yarbus, 1967). By blending image-based measures of visual salience (e.g., Itti & Koch, 2001), computations of local image statistics (e.g., Parkhurst & Niebur, 2003), episodic memory for specific scene content (e.g., Brockmole & Henderson, 2006), memory for semantic or structural redundancies within scene categories (e.g., Torralba, Oliva, Castelhano, & Henderson, 2006), and task knowledge (e.g., Land, Mennie, & Rusted, 1999), observers are able to efficiently control gaze in the service of completing visually guided tasks. An underdeveloped area of investigation within this program of research is work specifically addressing the sequencing of saccades. In this article, we demonstrated that one’s representations of an object’s spatio-temporal properties contribute to volitional decisions regarding where one will look next.

All else being equal, after fixating an object, observers are biased to shift gaze toward the area in front of objects. We suspect that this bias is adaptive, because it focuses gaze on areas of space that are likely to be important in eventual actions involving the object. For example, functional attributes of objects are often used to determine the fronts of objects (drawers are in the front of the dresser, the seat is in the front of the chair, etc.). Perhaps surprisingly, implied or potential motion does not augment this bias. This suggests that the direction of motion does not, in isolation, provide additional information to the process of fixation point selection beyond orientation alone. For example, the orientation of a car may be sufficient to indicate regions of a scene that are likely to change, without the need to additionally compute its motion capabilities. Furthermore, the lack of an effect of implied motion on gaze control was observed regardless of whether that motion could be expected to arise from external forces or animate self-movement. Together, these results suggest that implied motion and animacy may only affect covert shifts of attention. Understanding how these biases interact with other factors already known to affect saccade sequencing and those yet to be discovered is an important avenue for future research.