The interaction between perception and action is critically important for many everyday behaviors and continues to be the focus of considerable research. While substantial effort has been put into examining the role of perception in guiding action, how action affects perception is less well understood. Recently, Buttaccio and Hahn (2011) demonstrated that performing a simple and arbitrary action towards an object influences how similar objects are attended in subsequent processing. Their experiments consisted of a priming phase and a search phase. In the priming phase, participants saw a prime object and sometimes responded with a key—pressing only if the prime matched a previously presented pre-cue (e.g., if the shape was green and they had previously seen the word “green”). In the search phase, participants performed an unrelated visual search task, looking for a tilted line embedded within a shape that sometimes matched the prime in shape and color. The results revealed an interaction between action in the priming phase and the target-containing shape of the search phase: after a response to the prime trials (an action trial), participants’ response in the search task was faster when the search target appeared in a shape that matched the prime; after viewing trials (i.e., no action had been performed on the prime), the prime’s shape had no impact on the search task. The authors suggested that the action towards the prime strengthened the “trace” of its properties, which in turn influenced deployment of attention in visual search. Weidler and Abrams (2014) replicated and extended this finding, the action effect, under a variety of different conditions. For example, they showed that detailed processing of the prime’s properties (i.e., deciding about its color) was not necessary to induce the effect—a simple action in the presence of the prime was sufficient. Presumably, the action effect facilitates effective interaction with objects in the environment by biasing attention toward properties of objects that have been recently acted upon.

While the action effect provides valuable insights into the interaction between action and perception, one alternative explanation of the action effect remains a possibility. In all previous studies of the phenomenon, in the search phase participants were asked to make an action response in the same motor modality as that required in the priming phase. More specifically, Buttaccio and Hahn (2011) required two identical keypresses in both the priming and visual search phases whereas other studies required a single keypress in the priming phase followed by a two-choice keypress in the search phase (Suh, Weidler, & Abrams, 2016; Weidler & Abrams, 2014). This leads to the possibility that the action effect might not have originated from the previous action’s influence on subsequent processing at an attentional level, but instead from the interplay between the motoric action components in the priming and search phases.

Numerous studies have shown that participants can form rapidly “stimulus-response bindings” or arbitrary associations between stimuli and responses, which are then automatically retrieved during subsequent processing and affect responses to identical or related stimuli (Dennis & Perfect, 2013; Hommel, 1998, 2004; Horner & Henson, 2009); for a recent review, see: Henson, Eckstein, Waszak, Frings, & Horner, 2014). In Buttaccio and Hahn’s (2011) initial work, during action trials of the priming phase participants might also form bindings between the prime and the action response (e.g., pressing a key to a green circle), and in the subsequent search phase in which an identical action response was required to prime-related stimuli (e.g., pressing a key when the target is in a green circle), such bindings could affect the response as well. In the viewing trials, no prime-action binding is formed and subsequent search performance is unaffected by the attributes of the prime. In later studies (Suh et al., 2016; Weidler & Abrams, 2014), the responses in the priming and the search phases were mapped onto different keys and performed with different hands (i.e., press the space bar with the left hand on the action trials in the priming phase, and press the left or right arrow key with the right hand in the search phase). That procedure rules out the stimulus-response binding interpretation to some extent. However, various bindings could still occur at a more abstract level (Henson et al., 2014; Horner & Henson, 2009), such as the binding between the stimulus and the decision of pressing a key.

One recent study may provide some insight into the issue. Suh et al. (2016) used a typical action-effect paradigm but also monitored participants’ eye movements as they performed the task. They found that acting toward the prime increased the probability that participants would direct their first eye movement to the target during the search phase. However, because a manual response was required in both action and search phases, the eye movements may have been influenced by the manual responses required in the search phase. As a result, this study does not provide strong evidence against the response-binding possibility.

In the current study, we modified the action effect paradigm by replacing the manual action response in the search phase with an oculomotor response (i.e., a saccade) on some trials. Such modification has two benefits. First, previous studies have indicated that changing response modality could effectively reduce stimulus-response binding (Dennis & Perfect, 2013; Horner & Henson, 2009). For example, Horner and Henson (2009, Experiment 6) found that a shift between keypressing and vocal responding resulted in a smaller repetition priming effect in a speeded classification task. Therefore, replacing manual with saccadic responses may help to identify the role of prime-action bindings in the action effect. Secondly, numerous studies have shown that there is a substantial overlap between spatial attention and eye movements at both behavioral and neural levels, and it is likely that covert attention and saccade programming are driven at least partly by overlapping neural mechanisms (Awh, Armstrong, & Moore, 2006). Therefore, saccadic movements can be a reliable indicator of covert attention in visual search (Becker, 2008; McPeek, Maljkovic, & Nakayama, 1999). If the action effect does indeed originate from action’s impact on visual selection, a similar interaction should be observed when participants respond with either a manual or ocular movement in the search phase.

Experiment 1

Method

Participants

Twenty-four individuals from Tsinghua University (16 females, 22.58 ± 2.82 years old, mean ± SD, same below) participated in the experiment. All participants were right-handed and had normal or corrected-to-normal vision. Five participants were removed, because their eye movements could not be reliably tracked, leaving data from 19 participants for the current study (14 females, 22.00 ± 2.54 years old). The experimental procedures were approved by the local ethics committee.

Apparatus

The experiment was administrated in a dimly lighted room. Stimuli were presented on a 19” Dell 2210 LCD screen with a resolution of 1680 × 1050 at 60 Hz with Psychtoolbox 3.0 (Brainard, 1997; Kleiner et al., 2007) running under Matlab 2010a environment (Mathworks, CA). Viewing distance was fixed at 60 cm by a chinrest. Eye movements were recorded with a SensoMotoric Instruments RED250 desktop eye-tracking system (SensoMotoric Instruments GmbH, Germany) at a refresh rate of 120 Hz using in-house Matlab code.

Stimuli and procedure

The paradigm is based on Buttaccio and Hahn’s (2011) Experiment 1. The current experiment employed a 2 (response modality: manual vs. ocular) × 2 (priming task: action vs. viewing) × 2 (prime validity in visual search task: valid vs. invalid) within-subject design. Figure 1 illustrates a typical trial.

Fig. 1
figure 1

Sequence of events on a trial in the experiment. An action, valid trial is shown (not drawn to scale)

Full size image

Each trial began with a 0.64° white fixation cross on a black background. After 1000 ms, a color name randomly chosen from one of five colors (blue, green, purple, orange, red) was presented for 500 ms, followed by a fixation cross for 133 ms. Next, in the priming phase of the trial, a colored shape (~4 deg2) randomly chosen from one of four shapes (circle, diamond, square, triangle) was presented, and participants were to press the space bar as quickly as possible if the color of the shape matched the previously presented color name (50% of trials, action trials). If they mismatched, participants did not make any response (50%, viewing trials). After the response or 750 ms, a fixation cross was presented for 500 ms, followed by the visual search array (the search phase of the trial) in which four colored shapes that were the same as those possible in the priming phase were presented on an imaginary circle with a radius of 6.43° with eight potential positions. Three of the colored shapes contained a vertical line of 0.71° in length, 0.24° in width, whereas the fourth contained a tilted line (at an angle of −20° or +20°) of the same size. The prime (shape-color combination) was always present in the search phase, and the ratio of it containing a vertical (invalid trial) versus a tilted line (valid trial) was 3:1 (i.e., 25% of the trials were valid, so explicitly searching the prime object first was not a useful top-down strategy). The colors of the other shapes were randomly chosen from the same set of colors in the priming phase excluding the one used in the prime. Participants were instructed to search for the tilted line and either press the space bar (manual condition) or make a saccadic movement towards it (ocular condition) as quickly as possible. The search array was removed after either a keypress (manual-response condition) or fixation within a circle in a radius of 2.14° around the target shape (oculomotor-response condition) for at least 200 ms. Participants then indicated the direction of the tilted line by pressing the left or right arrow key.

Participants completed two practice blocks of 32 trials each, one with manual responses and the other with ocular responses. Then, they completed four experimental blocks of 96 trials each (two with manual responses and two with ocular responses), with conditions varying in an A-B-B-A pattern. Half of the subjects started with the manual-response condition and half of the subjects started with the ocular-response condition. At the beginning of each block, the eye tracking system was calibrated to an accuracy of approximately 1° with a 9-point calibration.

Data analyses

Overall, participants were very accurate in both the action task and the visual search task (correct rate: 98.04% and 99.42%, respectively). Therefore, we did not analyze accuracy data further and instead focused on the overall RT of the visual search task (the elapsed time between the presentation of the search array and the manual or ocular response, including the 200 ms fixation time on the target). For both manual-response data and ocular-response data, trials with RT below 150 ms or more than 3 standard deviations above each individual’s overall mean were also discarded. Median RT was computed for each condition and submitted to a 2 (prime type: action/viewing) × 2 (validity of prime: valid/invalid) repeated measures ANOVA, separately for the manual and the ocular conditions.

Results and discussion

Figure 2 presents median RTs for each condition of the visual search task. For the manual condition, a 2 × 2 repeated measures ANOVA yielded a significant two-way interaction between prime type and validity of prime, F (1, 18) = 5.09, p = 0.04, partial η 2 = 0.22, replicating the typical action effect pattern. A simple main effect analysis revealed that after an action response, RT was marginally faster on valid vs. invalid trials (p = 0.06), whereas for the viewing trials the effect of validity was not significant (p = 0.91). The main effects were not significant, Fs < 1.67, ps > 0.21, partial η 2s < 0.09. For the ocular condition, a similar action effect also emerged, F (1, 18) = 6.42, p = 0.02, partial η 2 = 0.26, for the interaction of prime type and prime validity. A simple main effect analysis revealed that after an action response, RT was marginally faster on valid versus invalid trials (p = 0.07), whereas for viewing trials the effect of validity was not significant (p = 0.20). The main effect of type of prime was also significant, F (1, 18) = 19.69, p < 0.001, partial η 2 = 0.52, such that RT was shorter on action vs. viewing trials. The main effect of prime validity was not significant, F (1, 18) = 0.34, p = 0.57, partial η 2 = 0.02. Moreover, a three-way repeated measures ANOVA involving prime type, validity, and response modality revealed a nonsignificant three-way interaction, F (1, 18) = 0.42, p = 0.52, partial η 2 = 0.02, indicating that the magnitude of the action effect did not depend on the response modality of the search task. Thus, the present experiment replicated the action effect with manual responses and also showed that a similar action effect occurs when the response is an eye movement. Because the eye movement was in a different modality than the response to the prime (which was a manual keypress), the results rule out a binding account of the action effect and instead suggest that it is attentional.

Fig. 2
figure 2

Median reaction time in the visual search task by experimental conditions, from Experiment 1 (error bars represent standard errors)

Full size image

Experiment 2

The results of Experiment 1 indicated that the action effect was robust even after the response modality of the visual search task was changed to ocular, ruling out the binding account of the action effect. However, because we manipulated response modality in the search phase within subjects, it is possible that participants prepared, but did not produce, manual responses even in the ocular response condition, thus complicating the interpretation of the results. Furthermore, we did not record individual target locations, leaving us unable to analyze eye movement data in more detail. These two issues were further addressed in Experiment 2, in which participants were asked to make only oculomotor response in the search phase. Under these settings, more repeated trials were acquired and the target location in each trial was recorded. If action upon an object indeed affects one’s attention allocation towards it, we would expect a replication of Experiment 1 and similar action effect patterns be observed on eye movement indices such as the direction of the first saccade (Suh et al., 2016).

Method

Participants

Twenty-four individuals from Tsinghua University (16 females, 23.29 ± 3.04 years old) participated in the experiment. The sample size was predetermined to achieve 95% statistical power for an effect equal to or stronger than the action effect in Experiment 1. All participants were right-handed and had normal or corrected-to-normal vision. Four participants were removed, because their eye movements could not be reliably tracked, leaving data from 20 participants for the current study (14 females, 23.38 ± 3.12 years old). The experimental procedures were approved by the local ethics committee.

Apparatus, stimuli and procedure

Apparatus, stimuli, and procedure were almost identical to Experiment 1 with one exception: participants only completed three blocks with ocular responses in the search phase, with each block containing 128 trials. In addition, target location was recorded, enabling us to analyze the eye-tracking trajectories in more detail.

Data analyses

Participants were again very accurate in both the action task and the visual search task (correct rate: 97.04% and 98.96%, respectively). Trials with incorrect responses in either task or with RT below 150 ms or more than 3 standard deviations above each individual’s overall mean also were discarded. Median RT was computed for each condition and submitted to a 2 (prime type: action/viewing) × 2 (validity of prime: valid/invalid) repeated measures ANOVA.

Blink points in eye-tracking trajectories were identified (physiologically implausible velocity of faster than 1000 degree/sec or pupil diameter = 0) and fixed with linear interpolation. The raw gaze coordinates (xi, yi) were smoothed with a Savitzky-Golay FIR smooth filter (Savitzky & Golay, 1964). The dispersion threshold algorithm for fixation identification (I-DT; Salvucci & Goldberg, 2000) was then applied to the smoothed gaze coordinates to identify fixations and saccades (fixation threshold: 100 ms, dispersion thresholds: 2 degrees; Blignaut, 2009). A 2 (prime type: action/viewing) × 2 (validity of prime: valid/invalid) repeated measures ANOVA was performed on three indices: the latency of the first saccade, the proportion of the first saccades that were directed towards the target location (deviant angle < 22.5 degree), and the total number of saccades produced if the first saccade was directed towards a nontarget location.

Results and discussion

RT results

The ANOVA on RT yielded a significant interaction effect between prime type and validity of prime, F (1, 19) = 4.65, p = 0.04, partial η 2 = 0.20; simple main effect analyses further revealed that on action trials, RT was significantly shorter on valid vs. invalid trials, p = 0.002, but the effect of validity was nonsignificant on viewing trials, p = 0.25 (Fig. 3a). Thus, the action effect was successfully replicated in this experiment. The main effect of prime validity also was significant, F (1,19) = 15.83, p = 0.001, partial η 2 = 0.45, such that RTs were shorter on valid vs. invalid trials. The main effect of prime type was marginally significant, F (1,19) = 3.58, p = .07, partial η 2 = 0.16, with RTs shorter on action compared with viewing trials.

Fig. 3
figure 3

Effects of experimental conditions on (a) reaction time of the ocular response; (b) proportion of first saccades were directed towards the target; (c) total number of saccades if the first saccade was directed to a nontarget location, from Experiment 2 (error bars represent standard errors)

Full size image

Eye movements results

Analyses of eye movements trajectories further revealed a similar action effect on the direction of the first saccade, F (1, 19) = 9.88, p = 0.01, partial η 2 = 0.34, for the interaction (Fig. 3b), and the total number of saccades if the first saccade was directed towards a nontarget location, F (1, 19) = 6.91, p = 0.02, partial η 2 = 0.27, for the interaction (Fig. 3c). For the action trials, participants’ first saccade was more likely to be directed towards the target on valid vs. invalid trials, p < 0.001, and if the first saccade was in wrong direction, it took fewer saccades to correct and complete the search task on valid vs. invalid trials, p = 0.001. For the viewing trials, the differences between valid and invalid trials also were significant, but the effects were smaller than on the action trials (p = 0.04 and 0.01 for the two indices, respectively). The main effects of prime validity also were significant for these two indices (F (1, 19) = 17.36, p = 0.001, partial η 2 = 0.47, for the proportion of first saccades, and F (1, 19) = 21.11, p < 0.01, partial η 2 = 0.53, for the number of saccades on trials where the first saccade missed the target), whereas the main effects of type of prime were not (Fs (1, 19) < 0.88, ps > 0.36, partial η 2s < 0.04). For the latency of the first saccade, none of the interaction effect or the main effects were significant, Fs < 2.31, ps > 0.15, partial η 2s < 0.11. Thus, the present experiment replicated the action effect for ocular response latencies and also shows that a prior action influences the production of eye movements during a search, which are strongly biased to be directed toward the search element that shares the color of the acted-upon prime.

General discussion

By employing eye movement responses, the present study showed that the “action effect” (Buttaccio & Hahn, 2011; Weidler & Abrams, 2014) exists across response modalities. Previous studies investigating the action effect used either identical (i.e., pressing the same key; Buttaccio & Hahn, 2011) or similar (i.e., pressing different keys; Weidler & Abrams, 2014) response methods and only in the manual modality, or required only manual responses while monitoring spontaneous eye movements (Suh et al., 2016). Although researchers have attributed the action effect to the facilitation of attentional processing of the object that had been acted upon, arbitrary associations (e.g., stimulus-response binding) might be automatically formed between the stimuli and responses in the priming phase (Dennis & Perfect, 2013; Horner & Henson, 2009; Henson, Eckstein, Waszak, Frings, & Horner, 2014) and later retrieved in the visual search task, which also would facilitate the response and account for the reduced RT typically observed. The present study showed that even after the response modality of the visual search task was changed from a manual to an eye movement response, a similar action effect still emerged. Furthermore, Experiment 2 showed that the cross-modality action effect was accompanied by changes in eye movement patterns, such that action upon a valid prime made participants’ first saccade more likely to be directed towards the target location, and if the first saccade was misdirected, fewer additional saccades were necessary to reach the target. Taken together, these results support the idea that the action affect is indeed originating in enhanced visual attention allocation.

It should be noted, however, that while our results rule out the possible binding between stimulus and response modality, an even more abstract binding might still be formed. For example, in the ocular condition, both the manual priming task and the ocular visual search contained a component of “response to a stimulus,” and this similarity might still lead to binding at a very abstract level (Henson et al., 2014; Horner & Henson, 2009). This very abstract form of binding may not be easily distinguished from attentional effects. In fact, most binding theories do contain an attentional component (Hommel, 1998). If binding occurs at this level, the attention account and the binding account of the action effect might actually be one and the same.

One interesting difference between the manual and ocular response conditions in our results is that the RT was significantly shorter after an action but only for the ocular response condition. In other words, for the ocular response condition, the major contributor to the validity effect after an action response seems to be a facilitating effect when the prime was the search target, rather than an inhibitory effect when the prime was a distractor. Manual responses, on the other hand, show a small amount of facilitation for validly cued targets after an action, as well as some inhibition for invalidly cued targets. Previous studies on the action effect have yielded either facilitation (Buttaccio & Hahn, 2011, Study 1), inhibition (Weidler & Abrams, 2014, Study 1), or both (Buttaccio & Hahn, 2001, Study 2), and these effect patterns have not been fully explained. The stimuli and conditions of these experiments varied greatly, making it difficult to provide a full account. Future research might address this issue more systematically.

Another potentially fruitful avenue of research is to explore the exact nature of the “action” in the action effect. In all of the existing studies on the effect, the action has been defined as a simple response (i.e., a keypress). It would be interesting to see if the effect can be generalized to other forms of action. For example, if participants hold the key and release it in response to the prime, does the same action effect still occur? Comparing different forms of action would help to identify the critical component of the action. Furthermore, it would be interesting to use neuroimaging methods such as event-related potentials (ERP) and functional magnetic resonance imaging (fMRI) to investigate how the action modulates brain activity during the prime phase as well as during the subsequent search task.

In summary, the action effect reveals a powerful influence of prior action on subsequent attentional prioritization: Features of objects that have been acted upon enjoy preferential processing after the action. The present findings help to rule out a stimulus-response binding account of the phenomenon, and reveal that the changes occur at an attentional level.