Abstract
Attentional selection is driven, in part, by a complex interplay between endogenous and exogenous cues. Recently, one’s interactions with the physical world have also been shown to bias attention. Specifically, the sense of agency that arises when our actions cause predictable outcomes biases our attention toward those things which we control. We investigated how this agency-driven attentional bias interacts with simultaneously presented endogenous (words) and exogenous (color singletons) environmental cues. Participants controlled the movement of one object while others moved independently. In a subsequent search task, targets were either the previously controlled objects or not. Targets were also validly or invalidly cued. Both cue types influenced attention allocation. Endogenous cues and agency-driven attentional selection were independent and additive, indicating they are separable mechanisms of selection. In contrast, exogenous cues eliminated the effects of agency, indicating that perceptually salient environmental cues can override internally derived effects of agency. This is the first demonstration of a boundary condition on agency-driven selection.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Our visual surroundings contain more information than we can process at a given moment. To compensate, we attend to information serially and build composite representations of the world over time. Thus, selecting what to attend to and when to attend to it are critical to our successful understanding of, and behavior within, the world. One major, but by no means exclusive, approach to studying how information is chosen for selection considers the means by which different types of information in the environment are used to direct attention. In this vein, attentional selection is viewed to arise from the interplay of endogenous (top-down) and exogenous (bottom-up) cues that orient attention in observer-driven and stimulus-driven manners, respectively (Desimone & Duncan, 1995; Duncan, 1985; Egeth & Yantis, 1997; Folk et al., 1992; Jonides & Yantis, 1988; Posner, 1980; Theeuwes, 1992).
Endogenous cues direct attention by invoking the recall of information stored in long-term memory. As such, to be effective guides of attention, endogenous cues are reliant on the observer’s prior experience or learning. Common forms of endogenous cues include words and symbols that are meaningful only through their interpretation. A common means to study endogenous attentional orientation is to use words (e.g., “left” or “above”) or symbols (e.g., arrows) to predict a target’s identity or location within detection and search tasks (e.g., Hommel et al., 2001; Langton & Bruce, 2000; Posner & Cohen, 1984). In such cases, a symbolic cue’s ability to drive attention is revealed empirically by contrasting situations where the cue correctly identifies the target (valid), cases where the cue instead identifies a distractor (invalid), and instances where no cue is provided (neutral). A person’s use of a cue predicts fastest search times in valid conditions and slowest search in invalid conditions.
A classic example of an endogenous cueing study was conducted by Posner et al. (1978). The authors presented participants with an arrow that pointed in the likely (80% valid) direction of an eventual detection target. Neutral cues (a plus sign) were also included in the design. That study found faster response times to validly cued compared with invalidly cued targets. Moreover, the response time benefit of valid cues, and costs for invalid cues, increased as the elapsed time between the cue and target onsets increased from 0 to approximately 200 ms, indicating that cue interpretation took time as its meaning was decoded. Such studies and many more like them (e.g., Brisson & Jolicœur, 2007; Burnham, 2018; Folk et al., 1992; Geng & Behrmann, 2002, 2005; Posner et al., 1980; Pratt & Hommel, 2003; Richard et al., 2003; Wright & Richard, 2003) show that by cueing a potential item or location prior to a visual search task, a participants’ performance can be significantly enhanced or inhibited.
In contrast to endogenous cues, exogenous cues direct attention via their physical properties. This allows perceptually novel or salient information to interrupt goal-directed behavior. Common examples include bright lights (e.g., Eriksen & Hoffman, 1972; Jonides, 1980), unique colors (e.g., Theeuwes, 1992), and sudden motion (e.g., Abrams & Christ, 2003). The power of exogenous cues in attracting attention is often demonstrated in experiments where attending to such features harms performance (see Egeth & Yantis, 1997; Most & Simons, 2001; Theeuwes et al., 2010, for a review). For example, Theeuwes (1992) engaged participants in a visual search task in which the target was a green circle. On half of the trials, the distractors were all green squares. On the other half of trials, the distractor set also included one red square. Although the red square shared no features with the target (it was neither green nor a circle), search times were slowed in its presence, even though doing so delayed localization of the target. Results like these suggest that complete goal-driven selectivity toward a particular feature is not always possible—the presence of an irrelevant, but perceptually salient item (in this case a unique color) can affect attentional selection.
Once studies began to explore the behavioral and psychophysical reactions elicited by endogenous and exogenous cues in isolation, interest grew in exploring how multiple cues interact with one another (Berger et al., 2005; Folk et al., 1992; Hommel et al., 2001; Kingstone, 1992; Pfister et al., 2012; Pratt & Hommel, 2003; Richard et al., 2003). For example, Berger et al. (2005) implemented a task in which participants (a) simply detected the appearance of a target in one of two boxes, (b) indicated which box the target appeared in, or (c) executed an eye movement to the appearing target. Prior to target appearance, two cues were presented. The first was an arrow that pointed to the eventual target location 80% of the time (i.e., endogenous cue). The second was the non-predictive abrupt brightening of a target location (i.e., an exogenous cue). In all tasks, when the target appeared immediately after the second cue, responses were faster to valid vs invalid endogenous cues regardless of the validity of the exogenous cue. Furthermore, valid exogenous cues reduced response times regardless of the validity of the endogenous cue. This indicates the two cue types retain their value when placed in competition with each other, suggesting they rely on separable mechanisms. This approach of placing different cues in cooperation and competition with each other is used in the current study to study the possible independence of, or interactions among, various mechanisms of attentional selection.
In addition to environmental cues, several cognitive cues influence moment-to-moment selection. Well established mechanisms include episodic and semantic memory, observer interests and goals, and task-related knowledge (see Brockmole et al., 2012, for review). Recently, agency—the subjective feeling that you have some degree of control over an object—has been identified as an additional factor that biases attentional selection. When you reach out, grasp a doorknob, and rotate your hand, the knob subsequently turns, and the door opens. Because the expected outcome resulted from your action you feel a sense of agency over the object and situation (Gozli, 2019). As well as willfully exerting a force on a foreign object, a sense of agency also arises from the feeling of being in control of one’s own self (Haggard & Chambon, 2012; Leslie, 1995). Hence, agency is tightly associated with self-motion and self-generated outcomes (e.g., Blakemore et al., 1998; Farrer et al., 2008; Haggard & Tsakiris, 2009; Moore & Haggard, 2008; Nahab et al., 2011; Sato, 2009). Although researchers have extensively explored the factors that contribute to a sense of agency (Haggard & Chambon, 2012; Kumar & Srinivasan, 2014; Leslie, 1995; Metcalfe et al., 2013; Metcalfe & Greene, 2007; Potts & Carlson, 2019; Salomon et al., 2013; van der Wel & Knoblich, 2013; Wenke et al., 2010) and its neurological origins (Bednark & Franz, 2014; Chambon et al., 2012; David et al., 2008; Farrer et al., 2008; Haggard, 2005; Haggard & Chambon, 2012; Haggard & Tsakiris, 2009; Kühn et al., 2011; Leotti et al., 2010; Sidarus et al., 2017; Stephenson et al., 2018; Wen & Haggard, 2020), its ability to affect cognitive systems remains relatively unknown.
Wen and Haggard (2018) recently examined the relationship between agency and selective attention. Using a computer mouse, participants manipulated the motion of circles on a computer screen. The participants had various degrees of control over the direction of one preselected circle in the array (the controlled circle was not revealed to participants in advance). On some trials, the controlled circle would always follow the participant’s mouse movements. On other trials, the controlled circle would sometimes follow the mouse (either 40% or 60% of the time). Finally, on the last set of trials, the controlled circle never followed the mouse (so, it was technically uncontrolled). After moving the stimuli for several seconds, gaps appeared in their contours. Participants were to find the circle with exactly one gap. The target was always the circle that the participant controlled. Their principal finding was that search times were inversely related to one’s level of control over the eventual target. The authors argued this pattern of results indicated that we bias our attention towards objects under our control, and as our level of control increases, so too does this bias.
While Wen and Haggard’s (2018) study provided important initial evidence for agency-driven attentional selection, several questions remained. Because the controlled stimulus was always the eventual search target, participants were incentivized to determine which object was under their control. Because participants also reported which circle they controlled at the end of each trial, agency was explicitly task-relevant. Finally, because stimuli followed voluntary mouse movements, participants’ sense of agency only arose from volitional actions. These design choices left open questions as to what extent agency-driven selection depends on task-relevance, awareness, and/or volition.
To answer these questions, Huffman and Brockmole (2020) expanded on Wen and Haggard’s (2018) study. Rather than use a mouse to control objects, participants used directional arrows on a keyboard. In their critical third experiment, they presented a situation in which control was non-predictive of target identity, participants were not informed or queried about their control over objects, and participants were told what keys to press and when to press them in order to make the stimuli move and change directions. Even without task-relevance, awareness, and volitional choice of action, the authors observed significantly faster search times for trials on which the participants previously controlled the target as opposed to a distractor. Hence, they concluded that humans are highly sensitive to the changes their actions cause in the environment, and selective attention is biased towards those changes.
To date, examinations of agency-driven selective attention have isolated the effects of agency on performance to establish its presence, magnitude, and potential limits. No work has yet considered how robust agency-driven selection effects are when other attentional cues are simultaneously available to an observer. Our goal was to study how the biasing effects of cognitively determined agency operate when placed in cooperation or competition with endogenous (Experiment 1) and exogenous (Experiment 2) cues within the environment. In particular, these experiments asked if agency-driven selection augments, or is perhaps negated by, other factors. If a sense of agency influences attention independently of endogenous and/or exogenous environmental cues, we should observe beneficial effects of agency regardless of the validity of other environmental cues available within the search display. That is, the effects of agency and environmental cues should be additive when they both validly predict a search target and agency over an eventual search target should facilitate response times even when environmental cues are invalid. Alternatively, agency-driven selection may be diminished or eliminated if environmental cues are given precedence when directing attention. Hence, this study will reveal the relative independence of agency within the constellation of factors known to bias selection and whether or not agency-driven attentional selection lose informative value when other drivers of selection are simultaneously available.
Experiment 1
Experiment 1 examined agency-driven selection when placed in cooperation or competition with an endogenous cue (Fig. 1). On each trial, participants first completed a control task. Here, four colored (red, blue, green, yellow) circles appeared on the display in random locations. By pressing the directional arrow keys on a keyboard, participants could move the stimuli. One circle always moved in the direction of the participants’ keypresses while the other three circles moved in randomly selected directions. After several seconds of movement, the stimuli stopped, and participants then engaged in a visual search task. Gaps appeared in each circle and participants were to locate the circle with only one gap (the distractors had two) as quickly as possible. As an endogenous cue, we presented a centrally located color cue simultaneously with target onset. This cue indicated, with 81.25% validity, the likely identity (color) of the target circle. We therefore expected search times to be faster on valid trials than on invalid trials. The question of main concern was the extent to which prior control over objects would additionally influence search times. If agency and endogenous cues provide additive benefits, response times on valid trials should be faster when the target was previously controlled compared with situations where participants previously controlled an eventual distractor. If agency retains informative value even when an endogenous cue provides contrary information, then response times on invalid trials should be similarly faster when the target was previously controlled compared with cases where the controlled object becomes a distractor.
The trial procedures used in Experiment 1. Throughout the movement interval, participants used directional arrow keys to control the stimuli. Upon search array onset, a color cue was presented; target identification was indicated by pressing the space bar. Participants used the number pad to identify the target
Methods
Participants
Data collection was conducted in two waves. In the first wave, sample size was based on prior agency-driven attention effects: Wen and Haggard (2018) collected data from 18-20 participants per reported experiment and Huffman and Brockmole (2020) included 24 participants per reported experiment. Because the current experiment included endogenous cuing as a second factor, we established an a priori minimum sample size of 36 participants. In order to compensate for potential participant exclusions or no-shows, we overrecruited by approximately 10% and obtained a first-wave sample of 41 participants. Through the peer-review process, however, we were advised to increase our sample in Experiment 2; we did so in Experiment 1 as well in order to equate the sample sizes in each experiment. We updated our total sample target to 45 participants which would achieve 80% power for effects larger than d = .5 assuming a correlation between factor levels of .3 or greater (Maxwell & Delaney, 1993)Footnote 1 and collected data from an additional 4 participants through a second wave of data collection. In augmenting our sample, we acknowledge the increase to the probability of Type I error and correspondingly adjusted alpha thresholds for null hypothesis significance testing (see Results for details). In the final sample, all participants were undergraduate students at the University of Notre Dame (Mage = 19.28 years, SD = 1.08 years) and were compensated with course credit; 29 participants identified as female and 16 as male. All participants provided informed consent and were screened for colorblindness.
Stimuli and apparatus
Stimuli were displayed on a 22” LCD monitor with a screen resolution of 1,920 × 1,080 pixels and a refresh rate of 60 Hz. Stimuli consisted of four uniquely colored circles that moved across a uniform gray field (RGB: 128, 128, 128). Each circle had a diameter of 96 pixels. The outline of each circle was 3 pixels in thickness and these outlines were either yellow (RGB: 255, 255, 000), red (RGB: 255, 000, 000), blue (RGB: 000, 000, 255), or green (RGB: 000, 255, 000). The interior of each circle matched the gray background. Stimuli moved horizontally or vertically (i.e., Up, Down, Left, or Right) at a speed of 600 pixels/s. When stimuli reached the outer border of the display they would “bounce back” by reversing direction, maintaining a constant velocity. A black banner 67 pixels in thickness ran along the bottom of the display; various response prompts appeared within this lower area in white 30-point font. Responses were collected using a standard QWERTY keyboard. The experiment was programmed using MATLAB by MathWorks (The MathWorks, Natick, MA) with the Psychophysics Toolbox (Brainard, 1997; Kleiner et al., 2007) and The VideoToolbox (Pelli, 1997). Viewing distance was not constrainedFootnote 2.
Design and procedure
The experimental procedures are illustrated in Fig. 1. Each trial began with the presentation of a black (RGB: 0, 0, 0) fixation point presented in the center of the display. Following a 1,000-ms delay, the four colored circles appeared in randomly selected locations. By pressing one of the directional keys (Up, Down, Left, Right) the circles began to move. Whenever the participant would release a key, the motion of all circles would stop. Each time a participant pressed a new arrow key, each circle had the possibility to change direction. Direction changes were probabilistic. Three of the four circles moved in randomly selected directions (i.e., each of these circles’ new direction of motion was determined independently) while the fourth circle always moved in the direction of the arrow keypress. The participants were informed that stimuli would only move while a key was held down; releasing a key caused all stimuli to stop. They were also told that one of the four circles was completely under their control.
After 3,500–4,500 ms of motion (jittered randomly), the stimuli stopped, and participants engaged in a visual search task. Gaps appeared on either the left side, right side, or both sides of each circle. The search target was the one circle with a single gap; the three circles with two gaps were distractors. Simultaneous with the onset of the gaps, a color cue word was presented in the center of the screen in black font. This cue identified the target with 81.25% probability. The participant pressed the spacebar to indicate they found the target. The gaps then disappeared, and the stimuli were numbered 1–4 (the numbers were assigned randomly without replacement on each trial). The participants used the number pad to indicate which circle was the target. Response times were defined as the elapsed time from the onset of the search task to the spacebar press. The number press portion of the test was implemented to evaluate accuracy. Trials on which the participant did not press the spacebar within 5,000 ms of the onset of the search task were terminated and considered errors.
Both the controlled circle and target were randomly selected at the beginning of each trial. The conditions were programmed to ensure each circle had an equal (25%) probability of being controlled, while the target circle was chosen at random. The cue was programmed to have 81.25% validity.Footnote 3 The effect of two within-subject factors on search times were considered: the effect of agency and the effect of cue validity. The experiment consisted of four blocks of 128 trials, totaling 512 trials, interspersed with self-paced breaks.
Results
Trials were excluded from analysis if an incorrect target identification response was made (2.9% of trials) or if correct response time was less than 200 ms (0.25%) or more than 2.5 standard deviations greater than each participant’s mean response time (2.5%). The total trials excluded due to trimming parameters was 5.65%.
Response time and accuracy data were submitted to separate 2 (agency: controlled target vs. uncontrolled target) × 2 (cue validity: valid cue vs. invalid cue) within-subject analyses of variance (ANOVA). Due to the augmentation of our sample following analysis of the first 41 (out of 45) participants, we followed the procedures outlined by Sagarin et al. (2014) we used an alpha level of .0388 as the criterion for statistical significance as it represents the “worst case scenario” to Type I error rate inflation in our study. In addition to null-hypothesis significance testing, a corresponding Bayesian ANOVA was also conductedFootnote 4 to characterize the predictive accuracy of the null and alternative hypotheses (see Appendix for full details).
Response time
Mean response times for each condition are illustrated in Fig. 2a. Within the omnibus ANOVA, a main effect of agency was observed, F(1, 44) = 8.67, p = .005, ηp2 = 0.17. Participants found targets faster if those targets were previously under their control (M = 1,233 ms, SD = 224 ms) than not (M = 1,284 ms, SD = 209 ms). A main effect was also observed for cue validity, F(1, 44) = 131, p < .001, ηp2 = 0.75, with faster response times stemming from validly cued targets (M = 1,092 ms, SD = 177 ms) compared with invalidly cued targets (M =1,424 ms, SD = 274 ms). Agency and cue validity did not interact, F(1, 44) = 1.52, p = .224, ηp2 = 0.03, indicating the facilitatory effect of agency on search times did not vary as a function of cue validity. As reported in Appendix Table 1, the Bayesian ANOVA indicated that the best-fit model likewise included the main effects of agency and cue validity. There was substantial evidence that this model accounted for the data better than one that included the agency by cue validity interaction (BF01 = 4.126; see Jeffreys, 1961; Wetzels et al., 2011). This further suggests that agency and cue validity were independent effects.
Accuracy
Mean accuracy for each condition is illustrated in Fig. 2b. A significant main effect of agency was observed, F(1, 44) = 18.5, p < .001, ηp2 = 0.30, participants were more accurate when the target was previously under their control (M = 0.98, SD = 0.016) than when it was not (M = 0.972, SD = 0.015). Additionally, a significant main effect was observed for cue validity, F(1, 44) = 44.5, p < .001, ηp2 = 0.50, with more accurate performance for validly cued targets (M = 0.984, SD = 0.012) compared with invalidly cued targets (M = 0.946, SD = 0.041). We did observe a significant interaction between agency and cue validity, F(1, 44) = 4.92, p = .032, ηp2 = 0.10, as the benefit of control on accuracy was larger in the invalid condition relative to the valid condition. Regardless, when the cue was valid, targets that were previously under control resulted in better accuracy to those that were not (Mdiff. = 0.007, SD = 0.011, p < .001), and the same was true when the word cue was invalid (Mdiff. = 0.021, SD = 0.052, p = .01). A corresponding Bayesian ANOVA corroborated these findings Appendix (Table 2). That is, the best-fit model included both the main effects of agency and cue validity as well as the agency by cue validity interaction. Of note, there was only anecdotal evidence suggesting that this model was a better fit than one that included only the main effects of agency and validity (BF01 = 1.197). There was, however, substantial evidence that the best-fit model accounted for the data better than all other models (all BF01 > 56.831). Considering the findings for RT and accuracy collectively, accuracy was inversely related to response time for both agency and cue validity, ruling out speed–accuracy trade-offs.
Discussion
Experiment 1 examined the attentional biasing effects of agency when placed in conjunction with an endogenous cue. As expected, cue validity facilitated search time. Critically, so too did agency. Prior control over an object facilitated search regardless of cue validity, and the magnitude of this benefit was equal in the presence of both valid and invalid cues. This indicates that agency and endogenous cues can additively bias attentional selection in situations where they work in cooperation to locate relevant information (i.e., valid trials). Likewise, in situations where contradictory information is present in the environment (i.e., invalid trials), agency continues to guide selection. This suggests that agency and endogenous cues operate independently. Experiment 2 tested whether agency similarly operates independently of exogenous cues.
Experiment 2
Experiment 2 examined agency-driven selection when placed in cooperation or competition with exogenous cues. Methods paralleled those in Experiment 1 with two exceptions (Fig. 3). First, all stimuli were the same color (black). Second, when the search task began one circle turned red which introduced a color singleton in the display. This singleton was nonpredictive of target location. That is, the singleton was the target on 25% of trials. Although observers had no incentive to prioritize the singleton during search, such exogenous cues capture attention automatically (Awh et al., 2012; Belopolsky & Awh, 2016; Desimone & Duncan, 1995; Pfister et al., 2012; Posner, 1980; Theeuwes, 1992). Following the same logic applied in Experiment 1, if agency and exogenous cues provide additive benefits, response times on valid trials should be faster when the target was previously controlled compared with situations where the previously controlled object becomes a distractor. And, if agency retains informative value when exogenous drivers of selection provide contrary information, response times on invalid trials should also be faster when the target was previously controlled. Alternatively, if exogenous cues take precedence (perhaps by virtue of the efficiency with which they attract attention), the effects of agency may be diminished or eliminated.
The trial procedures used in Experiment 2. Throughout the movement interval, participants used the directional arrows on a keyboard to control the movement of the stimuli. Upon search array onset, a singleton cue was presented, and both persisted until the space bar was pressed. Participants used the number pad to signal the target circle
Methods
Participants
As in Experiment 1, data collection occurred in two waves. In the first wave, our minimum a priori target sample size was 36 participants which was obtained. Following analysis of the corresponding data and following the advice of peer-reviewers, we augmented this sample by collecting data from an additional 9 participants in a second wave of data collection to yield a total sample of 45 participants. We again acknowledge the increase to the probability of Type I error and correspondingly adjusted alpha thresholds for null hypothesis significance testing. Participants in the combined sample were all undergraduate students (36 identifying as female and 9 as male, Mage = 18.87 years, SD = 0.99 years) from the University of Notre Dame and were compensated with course credit. All participants provided informed consent.
Stimuli and apparatus
The stimuli and apparatus were the same as in Experiment 1 apart from the circle colors. In Experiment 2, circles were either black (RGB: 0, 0, 0) or red (RGB: 255, 0, 0).
Design and procedure
The trial procedures are illustrated in Fig. 3. As in Experiment 1, trials were divided into a control phase and a search phase. The control phase was the same as in Experiment 1, save for the fact that all four circles were black. The search phase was also the same in Experiment 1 with the following exceptions. First, simultaneous with the onset of the gaps, one of the black circles turned red, thereby introducing a color singleton in the display. This singleton was the search target with 25% probability. Once the participant located the search target, he or she pressed the space bar. The gaps then disappeared; the cued circle reverted to black prior to the circles being numbered 1- 4 so that participants could use the number pad to indicate which circle was the target.
Both the controlled circle and target were randomly selected at the beginning of each trial. The conditions were programmed to ensure each circle has an equal (25%) probability of being controlled as well as an equal (25%) probability of being the target. Analyses again examined search times according to two within-subject factors of interest: the effect of agency and the effect of cue validity. The experiment consisted of four blocks of 128 trials, totaling 512 trials, interspersed with self-paced breaks.
Results
Trials were excluded from analysis if an incorrect target identification response was made (3.03% of trials) or if correct response times were less than 200ms (0.28%) or more than 2.5 standard deviations greater than each participant’s mean response time (2.50%). The total trials excluded due to trimming parameters was 5.81%.
Response time
Mean response times for each condition are illustrated in Figure 4a. As in Experiment 1, data were submitted to a 2 (agency: controlled target vs. uncontrolled target) × 2 (cue validity: valid cue vs. invalid cue) within-subjects ANOVA. Due to the augmentation of our sample following analysis of the first 36 (out of 45) participants, again following the procedures outlined by Sagarin et al. (2014) we used an alpha level of .0347 as the criterion for statistical significance as it represents the “worst case scenario” to Type I error rate inflation in this experiment. We once again conducted corresponding Bayesian ANOVAs to characterize the predictive accuracy of the null and alternative hypotheses. A significant main effect was observed for cue validity, F(1, 44) = 182, p < .001, ηp2 = 0.81, with faster response times (M = 977 ms, SD = 137.17 ms) for validly cued targets than invalidly cued targets (M =1,183 ms, SD = 171.65 ms). Unlike Experiment 1, however, there was no main effect of agency, F(1, 44) = 1.83, p = .183, ηp2 = 0.04, and no agency by cue validity interaction, F(1, 44) = 1.66, p = .204, ηp2 = 0.04. Similarly, the corresponding Bayesian ANOVA (Appendix Table 3) showed that the best-fit model only included the main effect of cue validity. There was only anecdotal evidence suggesting that this model accounted for the data better than one that included both main effects of agency and cue validity (BF01 = 2.214), but there was substantial evidence that the best-fit model accounted for the data better than all other models, including one with the agency by cue validity interaction (all BF01 > 7.686). These collective findings lead us to conclude that agency had no effect on attentional selection when accompanied by an exogenous cue.
Accuracy
Mean accuracy for each condition is illustrated in Figure 4b. In analyses paralleling those for response time, a main effect was observed for cue validity, F(1, 44) = 9.17, p = .004, ηp2 = 0.17, with better performance stemming from validly cued targets (M = 0.98, SD = 0.021) compared with invalidly cued targets (M = 0.966, SD = 0.017). No main effect of agency was observed, F(1, 44) = 0.28, p = .603, ηp2 = 0.01, indicating that performance did not differ when the target was previously under participant control (M = 0.973, SD = 0.02) than not (M = 0.972, SD = 0.015). Agency and cue validity did not interact, F(1, 44) = 0.237, p = .628, ηp2 = 0.005. The best-fit model within the Bayesian ANOVA likewise included only the main effect of cue validity (Appendix Table 4). There was substantial evidence that this model best accounted for the data compared with all other models (all BF01 > 5.30). Thus, accuracy was only affected by cue validity and the inverse relationship between accuracy response time rules out speed–accuracy trade-offs.
Discussion
Experiment 2 examined the attentional biasing effects of agency in conjunction with an attention-capturing exogenous cue. As expected, cue validity facilitated search. Critically, we found no evidence that prior control over an object facilitates search when exogenous cues are present. Thus, in contrast to the additive nature of agency and endogenous cues observed in Experiment 1, agency-driven selection was absent when attention was directed by exogenous cueing mechanisms. This finding suggests that exogenous cues take precedence, and this precedence is strong enough to eliminate the influence of agency on attentional selection.
General discussion
Our visual surroundings contain more information than we can process at a given moment. Hence, we attend to information serially and build composite representations of the world over time. Selecting what to attend to and when to attend to it are critical for successful understanding of, and behavior within, the world. We improve our chances of acquiring the right information at the right time by taking advantage of content available in the world and in memory that predict which object(s) or location(s) in our visual field are the most important given our current task or goal. Recently, our ability to act upon the physical world has been shown to also bias selection. When we exert control over objects, we have agency and our attention is biased toward those things which we control (Wen & Haggard, 2018), even when our actions are task-irrelevant or divorced from volitional decision-making (Huffman & Brockmole, 2020). This agency-driven attentional selection presents new opportunities to consider how our actions impact attention and the means by which we build our understanding of our environments. In the experiments presented here, we specifically considered the relationship between agency-driven attentional selection and other forms of attentional cueing.
Attentional selection is driven by a complex interplay between endogenous cues (e.g., words or symbols) and exogenous cues in the environment. Experiment 1 investigated potential interactions between agency-driven and endogenous attentional selection. In a two-phase procedure, participants first manipulated a dynamic array of uniquely colored circles where they had agency over one of the four items in the display. In the search phase, motion ceased, and a color word, presented in black, appeared at the center of the screen, predicting with 81.25% accuracy the potential target. Agency enhanced search efficiency when coupled with either a valid or invalid endogenous cue. Thus, an internally generated sense of agency can influence attention over and above the influences of endogenous cues in the environment. Moreover, prior control over an object reduced the costs associated with incorrect environmental information regarding relevant objects (i.e., an invalid cue). Thus, agency has independent informational value relative to endogenous environmental cues. The results obtained with endogenous cues also indicate that the effects of agency persist across shifts of attention away from controlled objects. In this experiment the symbolic word cue at the beginning of the search phase was presented in the center of the screen, requiring the participant to shift their gaze away from any of the potential targets. This shows that assessing and maintaining the identity and/or location of what is being controlled may be stored in working memory. How long this representation persists, and to what degree agency may influence the contents and capacity of working memory are therefore interesting questions to explore in future research.
Experiment 2 used a similar control-then-search task, although in place of the endogenous cue, one of the circles became a color singleton in the search phase. In this case, agency did not enhance the benefits of valid cues or reduce response times when paired with invalid cues. On valid trials, exogenous cues likely maximized attentional orienting leaving no room for further enhancement by agency. On invalid trials, agency did not retain its informational value as it did in the presence of invalid endogenous cues. Hence, exogenous cues are given priority over agency when allocating attention. We speculate this priority reduces the cognitive salience of agency and/or disrupts one’s ability to maintain a representation of previously controlled objects that can be used to guide attention when exogenous cues fail.
While we have primarily framed our discussion around issues of agency and its effects within the context of other attentional cues, it is important to note that they can also be viewed from the perspective of endogenous and exogenous cuing per se. Past work has shown that, when put in concert, multiple expectancies established endogenously are jointly considered when deploying attention (e.g., Kingstone, 1992). Furthermore, endogenous cues can enhance search efficiency regardless of the validity of a simultaneously present exogenous cue, and conversely, valid exogenous cues enhanced search regardless of endogenous validity (e.g., Berger et al., 2005). Such results have been taken as evidence that both types of orienting tap into independent mechanisms. Similarly, our results provide the first evidence to suggest that agency operates independently of endogenous environmental cues and provides evidence that it constitutes a behaviorally separable attentional mechanism that operates alongside other forms of orienting. Interestingly, this conclusion is also suggested by the results of Experiment 2; unlike endogenous cues agency-driven attentional selection does not seem influence behavior when exogenous cues are present.
While these experiments collectively contribute to a body of evidence suggesting the existence of an agency-driven attention mechanism, more work is needed to fully articulate the properties of this mechanism. First, with no studies yet employing eye tracking methods, it is unclear if the agency-driven attentional selection is overt or covert in nature. Understanding the means by which agents are “tracked” when grouped with other objects, may reveal important capacity limits or interactions with mechanisms of gaze control. For example, Vercher et al. (1996) explored smooth pursuit tracking of a single object, controlled by the participant’s arm movement, and found that participants were better at tracking self-controlled objects than those which were externally-controlled. This could mean, that when grouped with other non-controlled objects, one might be biased to tracking the object under his or her control. Second, agency-driven attentional selection has only been demonstrated in situations where control is task-relevant (Wen & Haggard, 2018) or task-irrelevant (Huffman & Brockmole, 2020; this report). Does agency bias selection in situations where doing so would harm performance? The experiments reported here indicate that salient reflexive exogenous cues in the environment can cancel agency-driven attentional selection; examining cases where task-goals emphasize uncontrolled aspects of the environment would indicate the extent to which the bias can be overridden by volitional task-sets. Third, it will be important to evaluate the independence of agency-driven selection from other cognitive mechanisms. For example, agency may be a product of prediction (Farrer et al., 2008) or a sensitivity to statistical regularities (Geng & Behrmann, 2005; Zhao et al., 2013) given that control over an object can enable one to know where it will be located at any moment. Fourth, researchers should ask how agency-driven attention operates when actions have multiple outcomes. In the studies conducted so far, participant actions have resulted in one effect—a change in object movement. Studies where actions have multiple outcomes (e.g., a change in direction and color), effects occur in multiple modalities (e.g., a change in direction and sound), and task-relevance of action-effect pairings are varied (e.g., a visual or auditory effect crossed with a visual search or auditory target discrimination) will provide a more robust assessment of the limits of agency-driven selection. Fifth, agency should be examined across multiple perceptual modalities including visual (e.g., direction, speed, luminance, color), audio (e.g., volume, pitch, frequency, etc.), and haptic (e.g., temperature, weight). It is important to understand if agency-driven selection presents across several modalities or if they are limited to a very few or, possibly, vision. Finally, “downstream” effects of agency-driven attentional selection on working memory, long-term memory, and decision making should be investigated to determine the role they may play in a variety of cognitive behaviors.
In addition to the theoretical questions above, work in agency-driven selection may have important relevance to application. Technology is quickly changing how we interact with the physical world. People have gained new means by which to control the world (e.g., augmented reality) and have simultaneously lost control over previously subservient objects (e.g., self-driving cars, home automation). What are the cognitive and performance ramifications of these changes? By understanding the dynamics between action, agency, and attentional selection, we may elucidate broad principles of human thought and behavior that should be incorporated into the design of the technologies that are changing the world. This work may also increase our ability to interact with technology fluently, which is becoming more important to success in one's personal and professional life (e.g., dating apps or telecommuting for work via video conference).
Notes
In Experiment 1, our observed correlations between levels of the control and cuing conditions were .86 and .71, respectively, indicating power was significantly higher. In Experiment 2, which uses the same experimental design and sample size, our observed correlations between levels of the control and cuing conditions were .81 and .80, respectively. In the Wen & Haggard (2018) and Huffman & Brockmole (2020) experiments, effectsizes associated with agency-driven attentional selection were medium-to-large (.51 < d < 2.9).
For illustrative purposes, if we consider a typical viewing distance of 57 cm, the display subtended approximately 47 deg of visual angle horizontally and 30.5 deg vertically; each circle stimulus subtended 2.35 deg, and moved at a rate of 14.7 deg/s.
The cue–target relationship was determined using a series of random number selections. On each trial, target color was first determined by selecting a random integer from the set {1, 2, 3, 4}, with each integer assigned to one of the possible target colors. Next, cue identity was chosen. A second random integer was again selected from the set {1, 2, 3, 4}. If 1, 2, or 3 was returned, the cue’s identity was set to match the previously chosen target. This method determined cue identity on 75% of all trials, with all of these trials being valid. On the remaining 25% of trials where a 4 was returned, a third random integer was selected from the set {1, 2, 3, 4}, with each integer again assigned to a specific color. In this case, 25% of the selections would be valid (i.e., 6.25% of all trials). Hence, taken together, the probability of a valid cue was 81.25% (75% + 6.25%).
All analyses were conducted with the software package JASP (Version 0.16) using default priors.
References
Abrams, Richard A.., & Christ, Shawn E.. (2003). Motion Onset Captures Attention. Psychological Science, 14(5), 427–432.
Awh, E., Belopolsky, A. V., & Theeuwes, J. (2012). Top-down versus bottom-up attentional control: a failed theoretical dichotomy. Trends in Cognitive Sciences, 1–7.
Bednark, J. G., & Franz, E. A. (2014). Agency attribution: Event-related potentials and outcome monitoring. Experimental Brain Research, 232(4), 1117–1126.
Belopolsky, A. V., & Awh, E. (2016). The role of context in volitional control of feature-based attention. Journal of Experimental Psychology: Human Perception and Performance, 42(2), 213–224.
Berger, A., Henik, A., & Rafal, R. (2005). Competition between endogenous and exogenous orienting of visual attention. Journal of Experimental Psychology: General, 134(2), 207.
Blakemore, S. J., Wolpert, D. M., & Frith, C. D. (1998). Central cancellation of self-produced tickle sensation. Nature Neuroscience, 1(7), 635–640.
Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10, 433–436.
Brisson, B., & Jolicœur, P. (2007). A psychological refractory period in access to visual short-term memory and the deployment of visual–spatial attention: Multitasking processing deficits revealed by event-related potentials. Psychophysiology, 44(2), 323–333.
Brockmole, J. R., Davoli, C. C., & Cronin, D. A. (2012). The visual world in sight and mind: How attention and memory interact to determine visual experience. In B. H. Ross (Ed.), The psychology of learning and motivation (Vol. 57, pp. 103–145). Elsevier.
Burnham, B. R. (2018). Selectively ignoring locations does not modulate contingent involuntary orienting, but selectively attending does. Visual Cognition, 26(1), 48–70.
Chambon, V., Wenke, D., Fleming, S. M., Prinz, W., & Haggard, P. (2012). An online neural substrate for sense of agency. Cerebral Cortex, 23(5), 1031–1037.
Cousineau, D. (2005). Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson’s method. Tutorial in Quantitative Methods for Psychology, 1(1), 4–45.
David, N., Newen, A., & Vogeley, K. (2008). The “sense of agency” and its underlying cognitive and neural mechanisms. Consciousness and Cognition, 17, 523–534.
Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18(1), 193–222.
Duncan, J. (1985). Visual search and visual attention. Attention and Performance, 11, 85–106.
Egeth, H. E., & Yantis, S. (1997). Visual attention: Control, representation, and time course. Annual Review of Psychology, 48(1), 269–297.
Eriksen, C. W., & Hoffman, J. E. (1972). Temporal and spatial characteristics ofselective encoding from visual displays. Perception & Psychophysics, 12(2), 201–204.
Farrer, C., Frey, S. H., Van Horn, J. D., Tunik, E., Turk, D., Inati, S., & Grafton, S. T. (2008). The angular gyrus computes action awareness representations. Cerebral Cortex, 18(2), 254–261.
Folk, C. L., Remington, R. W., & Johnston, J. C. (1992). Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 18(4), 1030.
Geng, J. J., & Behrmann, M. (2002). Probability cuing of target location facilitates visual search implicitly in normal participants and patients with hemispatial neglect. Psychological Science, 13(6), 520–525.
Geng, J. J., & Behrmann, M. (2005). Spatial probability as an attentional cue in visual search. Perception & Psychophysics, 67(7), 1252–1268.
Gozli, Davood. (2019). Experimental psychology and human agency. Springer Nature Switzerland.
Haggard, P. (2005). Conscious intention and motor cognition. Trends in Cognitive Sciences, 9(6), 290–295.
Haggard, P., & Chambon, V. (2012). Sense of agency. Current Biology, 22(10), R390–R392.
Haggard, P., & Tsakiris, M. (2009). The experience of agency: Feelings, judgments, and responsibility. Current Directions in Psychological Science, 18(4), 242–246.
Hommel, B., Pratt, J., Colzato, L., & Godijn, R. (2001). Symbolic control of visual attention. Psychological Science, 12(5), 360–365.
Huffman, G., & Brockmole, J. R. (2020). Attentional selection is biased towards controllable stimuli. Attention, Perception, & Psychophysics, 1–12. Advance online publication.
Jeffreys, H. (1961). Theory of probability (3rd ed.). Oxford University Press.
Jonides, J. (1980). Towards a model of the mind's eye's movement. Canadian Journal of Psychology/Revue Canadienne de Psychologie, 34(2), 103.
Jonides, J., & Yantis, S. (1988). Uniqueness of abrupt visual onset in capturing attention. Perception & Psychophysics, 43(4), 346–354.
Kingstone, A. (1992). Combining expectancies. The Quarterly Journal of Experimental Psychology Section A, 44(1), 69–104.
Kleiner, M., Brainard, D., & Pelli, D. (2007). What's new in Psychtoolbox-3?.
Kühn, S., Nenchev, I., Haggard, P., Brass, M., Gallinat, J., & Voss, M. (2011). Whodunnit? Electrophysiological correlates of agency judgements. PLOS ONE, 6(12), e28657.
Kumar, D., & Srinivasan, N. (2014). Naturalizing sense of agency with a hierarchical event-control approach. PLOS ONE, 9(3), e92431.
Langton, S. R. H., & Bruce, V. (2000). You must see the point: Automatic processing of cues to the direction of social attention. Journal of Experimental Psychology: Human Perception and Performance, 26, 747–757.
Leotti, L. A., Iyengar, S. S., & Ochsner, K. N. (2010). Born to choose: The origins and value of the need for control. Trends in Cognitive Sciences, 14(10), 457–463.
Leslie, A. M. (1995). A theory of agency. In D. Sperber, D. Premack, & A. James Premack (Eds.), Causal cognition: A multidisciplinary debate (pp. 121–141). Oxford University Press.
Maxwell, Scott E.., & Delaney, Harold D. (1993). Bivariate median splits and spurious statistical significance. Psychological Bulletin, 113(1), 181.
Metcalfe, J., & Greene, M. J. (2007). Metacognition of agency. Journal of Experimental Psychology: General, 136(2), 184.
Metcalfe, J., Eich, T. S., & Miele, D. B. (2013). Metacognition of agency: proximal action and distal outcome. Experimental Brain Research, 229(3), 485–496.
Moore, J., & Haggard, P. (2008). Awareness of action: Inference and prediction. Consciousness and Cognition, 17(1), 136–144.
Most, S. B., & Simons, D. J. (2001). Attention capture, orienting, and awareness. In C. L. Folk & B. S. Gibson (Eds.), Attraction, distraction and action: Multiple perspectives on attentional capture (pp. 151–173). Elsevier Science. https://doi.org/10.1016/S0166-4115(01)80009-1
Nahab, F. B., Kundu, P., Gallea, C., Kakareka, J., Pursley, R., Pohida, T., Miletta, N., Friedman, J., & Hallett, M. (2011). The neural processes underlying self-agency. Cerebral Cortex, 21(1), 48–55.
Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437–442.
Pfister, R., Heinemann, A., Kiesel, A., Thomaschke, R., & Janczyk, M. (2012). Do endogenous and exogenous action control compete for perception? Journal of Experimental Psychology: Human Perception and Performance, 38(2), 279.
Posner, M. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32(1), 3–25.
Posner, M. I., & Cohen, Y. (1984). Components of visual orienting. In H. Bouma & D. G. Bouwhuis (Eds.), Attention and performance X (pp. 531–556). Erlbaum.
Posner, M. I., Nissen, M. J., & Ogden, W. C. (1978). Attended and unattended processing modes: The role of set for spatial location. Modes of Perceiving and Processing Information, 137(158), 2.
Posner, M. I., Snyder, C. R., & Davidson, B. J. (1980). Attention and the detection of signals. Journal of Experimental Psychology: General, 109(2), 160.
Potts, C. A., & Carlson, R. A. (2019). Control used and control felt: Two sides of the agency coin. Attention, Perception, & Psychophysics, 81(7), 2304–2319.
Pratt, J., & Hommel, B. (2003). Symbolic control of visual attention: The role of working memory and attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 29(5), 835.
Richard, C. M., Wright, R. D., & Ward, L. M. (2003). Goal-driven modulation of stimulus-driven attentional capture in multiple-cue displays. Perception & Psychophysics, 65(6), 939–955.
Sagarin, B. J., Ambler, J. K., & Lee, E. M. (2014). An ethical approach to peeking at data. Perspectives on Psychological Science, 9, 293–304.
Salomon, R., Lim, M., Kannape, O., Llobera, J., & Blanke, O. (2013). “Self pop-out”: Agency enhances self-recognition in visual search. Experimental Brain Research, 228(2), 173–181.
Sato, A. (2009). Both motor prediction and conceptual congruency between preview and action-effect contribute to explicit judgment of agency. Cognition, 110(1), 74–83.
Sidarus, N., Vuorre, M., & Haggard, P. (2017). Integrating prospective and retrospective cues to the sense of agency: A multi-study investigation. Neuroscience of Consciousness, 2017(1), nix012.
Stephenson, L. J., Edwards, S. G., Howard, E. E., & Bayliss, A. P. (2018). Eyes that bind us: Gaze leading induces an implicit sense of agency. Cognition, 172, 124–133.
Theeuwes, J. (1992). Perceptual selectivity for color and form. Perception & Psychophysics, 51(6), 599–606.
Theeuwes, J., Olivers, C. N., & Belopolsky, A. (2010). Stimulus-driven capture and contingent capture. Wiley Interdisciplinary Reviews: Cognitive Science, 1(6), 872–881.
van der Wel, R., & Knoblich, G. (2013). Cues to agency: Time can tell. Agency and Joint Attention, 54, 256.
Vercher, J. L., Gauthier, G. M., Guedon, O., Blouin, J., Cole, J., & Lamarre, Y. (1996). Self-moved target eye tracking in control and deafferented subjects: Roles of arm motor command and proprioception in arm-eye coordination. Journal of Neurophysiology, 76(2), 1133–1144.
Wen, W., & Haggard, P. (2018). Control changes the way we look at the world. Journal of Cognitive Neuroscience, 30(4), 603–619.
Wen, W., & Haggard, P. (2020). Prediction error and regularity detection underlie two dissociable mechanisms for computing the sense of agency. Cognition, 195, 104074.
Wenke, D., Fleming, S. M., & Haggard, P. (2010). Subliminal priming of actions influences sense of control over effects of action. Cognition, 115(1), 26–38.
Wetzels, R., Matzke, D., Lee, M. D., Rouder, J. N., Iverson, G. J., & Wagenmakers, E. J. (2011). Statistical evidence in experimental psychology: An empirical comparison using 855 t tests. Perspectives on Psychological Science, 6, 291–298.
Wright, R. D., & Richard, C. M. (2003). Sensory mediation of stimulus-driven attentional capture in multiple-cue displays. Perception & Psychophysics, 65(6), 925–938.
Zhao, J., Al-Aidroos, N., & Turk-Browne, N. B. (2013). Attention is spontaneously biased toward regularities. Psychological Science, 24(5), 667–677.
Author information
Authors and Affiliations
Corresponding author
Additional information
Open practices statement
The data and or materials used for the current study are available upon request.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Significance statement
The sense of agency that arises when our actions cause predictable outcomes biases our attention toward those things which we control. The present study considered agency-driven selection effects when other forms of attentional orienting are simultaneously available to an observer. Results suggest agency can operate independently of endogenous environmental cues but are eliminated by exogenous cues. Thus, this research illustrates the importance of, and provides a clear path for, the continued exploration of the attentional effects of agency.
Appendix
Appendix
Rights and permissions
About this article
Cite this article
Vilanova-Goldstein, A.C., Huffman, G. & Brockmole, J.R. Interactions among endogenous, exogenous, and agency-driven attentional selection mechanisms in interactive displays. Atten Percept Psychophys 84, 1477–1488 (2022). https://doi.org/10.3758/s13414-022-02507-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13414-022-02507-1