The human brain efficiently stores information in such a way that related concepts activate each other (e.g., Meyer & Schvaneveldt, 1971). One demonstration of this can be found in word–picture priming, in which participants are faster to name a picture when it is preceded by a related rather than by an unrelated word (e.g., Glaser & Düngelhoff, 1984; Gordon & Irwin, 2000). In addition, it is known that meaning can influence how people scan a cluttered scene. For example, participants look more quickly to an image that is semantically (e.g., Odekar, Hallowell, Kruse, Moates, & Lee, 2008) or visually (de Groot et al., 2016) related to a previously presented word. Thus, seeing (or hearing) a word can facilitate visual search for, and identification of, semantically related images. Such a result may stem from spreading activation from the word to related concepts, including visual representations of the named object (e.g., Reinitz, Wright, & Loftus, 1989; see also Estes, Verges, & Barsalou, 2008).

One can see how such spreading activation could facilitate the performance of important behaviors. For example, the word APPLE might bring to mind the positive sensory (and metabolic) experiences associated with apples (e.g., Estes et al., 2008), which might lead a person to more readily detect, and seek, an apple. Likewise, the word THORN might bring to mind unpleasant experiences that followed an injurious encounter with a thorn, leading to the avoidance of thorns. Certainly the activation of object-related sensory and perceptual experiences could facilitate identification of and interaction with the objects in question.

Although people are able to identify many thousands of objects by name, some objects are more important than others. For example, PENCIL may activate experiences of using a pencil, but such activation may be less critical for survival than activation of the sensory and perceptual experiences associated with APPLE. How is it that the cognitive system prioritizes connections between words and their associated sensory and perceptual experiences? We explore here, and present support for, the possibility that connections between words and their associated sensory and perceptual experiences are at least temporarily enhanced by action. That is, when a person performs an action, the concepts that are active at the time of the action receive enhanced processing, and are prioritized for a brief period of time subsequently. The benefits of such an effect for survival are clear: Objects that are targets of our actions are, by definition, important to us, and prioritization of such objects could be advantageous.

To investigate the role of action in prioritizing semantic relationships, we utilized a new paradigm that has revealed that a simple action in the presence of an object can facilitate subsequent visual search—in that case, for objects with shared visual features (e.g., Buttaccio & Hahn, 2011; Weidler & Abrams, 2014, 2017a, b). Because we draw on a paradigm used in action–perception research, we detail it briefly here. In a typical experiment (e.g., Weidler & Abrams, 2014), participants are cued in advance on each trial to either act in response to or view a colored object (e.g., a red circle, the prime). The “action,” typically, is a simple keypress. They then search for a noncolor target (e.g., a tilted line) in a subsequent display in which the color of the prime appears along with a different-colored distractor (e.g., a blue circle); the prime’s color is uninformative with respect to the target. The common result is that when the search target happens to match the color of the prime, participants are faster to find it—but predominantly in trials on which they had earlier acted in response to the prime—not in trials on which the prime had been only viewed. In our example, participants would be faster to find the target when it was in a red circle than when it was in a blue circle (and the red circle in the search display contained a distractor). Thus, the action effect occurs when, after an action in response to an object, the visual features of that object are prioritized in an unrelated search task.

Importantly, all of the existing evidence for this effect of action on perception has come from research in which a specific visual feature, such as color (Buttaccio & Hahn, 2011; Huffman & Pratt, 2017; Wang, Sun, Sun, Weidler, & Abrams, 2017; Weidler & Abrams, 2014, 2017b) or shape (Weidler & Abrams, 2017a), that is present during the action reappears in the search display. Therefore, it remains an open question whether the mechanism supporting the facilitation is simply a prioritization of basic perceptual features that can be extracted relatively early in visual processing. (For perceptual features, we adopt Reinitz et al.’s, 1989, definition: information that is available only from the physical stimulus itself.) Here we explored whether simple motor responses can also facilitate semantic relationships. By semantic relationships, we mean those that cannot rely simply on basic perceptual feature similarity (such as the similarity between one red circle and a subsequently presented red circle), and must rely instead on some meaning-based mechanisms. Specifically, in the present research we characterized a word and its picture referent (i.e., the word CAT and a picture of a cat) as having a semantic or conceptual relationship because an image of a cat cannot be acquired from the physical stimulus “CAT” (i.e., they have more than a perceptual relationship). If a simple action were to modulate a semantic relationship, such an occurrence would necessarily require processing beyond what is needed to extract basic visual features (i.e., color or shape) from a display.

There are some reasons to expect visual search to be modulated by previous actions to semantically related objects. As was mentioned earlier, research has revealed that word comprehension involves the activation of visual properties associated with the named object (e.g., Boutonnet & Lupyan, 2015; Estes et al., 2008). Thus, when one sees a word, visual features associated with its meaning become activated. Given that objects that are acted upon are, by nature, prioritized, an action in response to a word might further potentiate typical visual features of the object named by the word and influence perception of its image.

In the present series of experiments, we investigated whether an arbitrary action can potentiate a semantic relationship between a word and a picture when the prime (a word) shares conceptual features but no sensory or perceptual features with the stimuli in the search task (pictures). To anticipate the results, we found that action increased the magnitude of word–picture priming, revealing that action can influence semantic relationships, as well as highlighting the important role of action in guiding perception and cognition to facilitate ongoing behaviors.

Experiment 1

Method

Participants

Thirty-two undergraduates (22 female, 10 male) participated for course credit.

Stimuli, procedure, and design

The stimuli were presented on a CRT monitor with an 85-Hz refresh rate. Each trial began with a centrally presented black fixation cross, 1° in height, for 506 ms (all text stimuli were presented centrally unless otherwise noted), followed by the word “GO” or “NO” (black, 1°) for 506 ms (see Fig. 1 for a depiction of the trial events). After another fixation cross for 129 ms, a word appeared (one of the 24 possible words listed in the Table 1; 2° in height, black), and participants performed the action task: If the word “GO” had previously appeared, participants were instructed to press the space bar as quickly as possible (Action trials), whereas if the word “NO” had appeared, participants were instructed only to view the word (Viewing trials). Thus, no decision was needed about the word itself. The word was presented for 753 ms or until response.

Fig. 1
figure 1

Main events on each trial of Experiment 1 (see the text for further details). Participants hit the space bar when the word (e.g., “APPLE”) appeared if they had previously seen “GO,” or they simply viewed the word if they had previously seen “NO.” Next, participants looked for the sole left (<) or right (>) arrow and indicated which of these arrows was present. On valid trials (depicted at the top), the image representing the word contained the target; on invalid trials (bottom), the image representing the word contained the distractor

Full size image

After a 506-ms blank screen, two pictures appeared (colored photographs of objects from Moreno-Martinez & Montoro, 2012; each approximately 8° × 8° in size, and each depicting a different one of the 24 items listed in the Table 1), positioned randomly at two corners of an invisible 16° × 16° square. The picture representing the previously seen word was always in the search display (e.g., if the word was APPLE, a picture of an apple appeared in the search display), and the second image in the search display was chosen randomly on each trial from the other 23 pictures in the set. After a 200-ms delay, the target (< or >, 1° black) and distractor (> rotated 90 or – 90 deg, to point up or down) placed on a white 1° square appeared superimposed on the images and remained until the participant had responded (or until the display timed out 1,600 ms after onset of the pictures). The target and distractor stimuli (the arrow and its white “background”) each appeared displaced from the center of the image upon which they were superimposed by a random amount between – 2° and + 2° in the x and y dimensions. Both target and distractor identity were chosen randomly on each trial.

During the search task, participants were instructed to indicate whether a left or a right arrow was present as quickly as possible by pressing the left or right arrow key with the index or the middle finger, respectively, of their right hand. Although the image corresponding to the prime word appeared in the search array on every trial, it was not predictive of the target’s location. On valid trials (50%), the target arrow appeared on the image corresponding to the prime, but on invalid trials (50%), the prime-corresponding image contained the distractor (and the target was on the second, randomly chosen image). The offset of the search array was followed by a 1,506-ms intertrial interval that included 506 ms of visual and auditory feedback for responses that were too slow or incorrect.

Participants performed 24 practice trials followed by 192 test trials (2 repetitions of the 96 unique trials—2 Action × 2 Validity × 24 Prime Words) presented in a random order.

Results and discussion

Action task

The average correct response time in action trials was 245 ms (SD = 67). Participants performed the action task correctly on 98.2% of trials and were more accurate on viewing trials (99.3%) than on action trials (97.0%), t(31) = 3.57, p = .001.

Search task

The main analysis involved the reaction times (RTs) from the search task. The average search RTs for trials in which participants performed both the action and search tasks correctly (94.0%, SD = 4.4%) were submitted to a 2 Action (action or viewing) × 2 Validity (valid or invalid) repeated measures ANOVA. The effect of action approached significance, F(1, 31) = 4.02, p = .054, η 2 p = .12, with participants searching more quickly on action trials (691 ms) than on viewing trials (703 ms). In addition, we found a robust effect of validity, F(1, 31) = 32.17, p < .001, η p 2 = .51, with participants responding more quickly on valid (670 ms) than on invalid (724 ms) trials. This might reflect the typical word–picture priming that has been reported previously (e.g., Odekar et al., 2008). Importantly, action and validity interacted, F(1, 31) = 12.12, p = .002, η p 2 = .28: Although reliable in both conditions, the validity effect was larger following an action to the word (69 ms) than following viewing the word (40 ms); see Fig. 2.Footnote 1 Thus, this experiment revealed for the first time that an arbitrary action can influence visual search for items that share only a conceptual (i.e., beyond basic perceptual features) relationship with the acted-on object.

Fig. 2
figure 2

Response times in the search tasks of Experiments 1, 2, and 3 as a function of action and validity. The validity effect (i.e., the word–picture priming effect) was increased following an action in the presence of the word as compared to following only viewing of the word. Error bars represent within-subjects 95% confidence intervals (Loftus & Masson, 1994)

Full size image

Accuracy in the search task for trials in which the action task was correct was not affected by either action, F < 1, or validity, F(1, 31) = 2.06, p = .161, nor did the two factors interact, F < 1.

Experiment 2

Experiment 1 revealed for the first time that an arbitrary action can bias attention toward semantically related objects in a visual search entirely unrelated to meaning. This expands on previous research indicating that action can bias attention toward a repeated visual feature (i.e., a color; e.g., Buttaccio & Hahn, 2011). However, it should be noted that, in addition to the manipulation of a conceptual relationship between the prime and the items in the search display, there was another discrepancy between the procedure of Experiment 1 and that used in typical action effect research. Because of a concern about the arrows obscuring portions of the images, we inserted a brief delay between the onset of the images and the onset of the target and distractor arrows. As a result, participants had time to orient to the images before they were able to begin the formal search. It might be the case that, if the targets appeared concurrently with the pictures, thus permitting participants to immediately employ goal-directed search mechanisms for the target, the facilitation observed in Experiment 1 might not occur. We tested this possibility in Experiment 2.

Method

Participants

Thirty-two additional undergraduates (24 female, eight male) participated.

Stimuli, procedure, and design

The method was as in Experiment 1, with the following exceptions. All of the elements in the search array (i.e., images, target, and distractor) appeared 506 ms after offset of the prime word and remained present for 1,600 ms or until response. The target and distractor symbols were both blue and appeared centrally on the images, which were presented in two of five possible locations around an imaginary circle with a radius of 12 deg.

Results and discussion

Action task

During the action task, participants were again less accurate during action (96.4%) than during viewing (99.3%) trials, t(31) = 4.26, p < .001. Overall, the accuracy in the action task was 97.8% (SD = 1.9%), and the average RT was 267 ms (SD = 65).

Search task

The 2 Action × 2 Validity analysis of search RTs from correct trials (conjoined accuracy: M = 91.8%, SD = 3.8) revealed no main effect of action, F < 1. As in Experiment 1, we found a main effect of validity, F(1, 31) = 14.92, p = .001, η p 2 = .33, and importantly, it was again modulated by action, F(1, 31) = 4.31, p = .046, η p 2 = .12. The interaction occurred because, although reliable in both conditions, the validity effect was larger after acting to the word (50 ms) than after viewing it (26 ms); see Fig. 2. This implies that the results of Experiment 1 were not driven by the brief preview of the images before appearance of the targets, and further establishes the strength of the “conceptual” action effect.

As in Experiment 1, accuracy in the search task for correct action trials did not differ as a function of either action, F(1, 31) = 1.83, p = .186, or validity, F(1, 31) = 1.50, p = .230, nor did the two factors interact, F < 1.

Experiment 3

In the two experiments thus far, we have shown that an arbitrary action can influence visual search for items that share only a conceptual relationship with the acted-on object. The evidence for this conclusion came from greater facilitation in RTs when the target was on an image representing the previously seen word (valid trials), as compared to when that image contained a distractor (invalid trials), following an action than following viewing. However, there might be an alternative explanation for this. Although the times between the offset of the prime word and the onset of the search display were equated in the action and viewing conditions, the time between the onset of the prime word and the onset of the search display were not equated between the two conditions. More specifically, on action trials there was a shorter time between the appearance of the prime word and the appearance of the search display, because the word disappeared when participants produced the required action. Prior research has shown that variations in the interval between prime onset and search display onset cannot account for the “perceptual” action effect (when an exact feature repeats between action and search; Weidler & Abrams, 2014). However, it is certainly plausible that the facilitation from word to picture in an unrelated search task could decrease with increasing time between the word and the pictures in our task (i.e., as in the viewing trials of Exps. 1 and 2). If that is the case, then Experiments 1 and 2 might not reveal a role of action per se. Experiment 3 addressed this concern.

Method

Participants

Thirty-two additional undergraduates (24 female, eight male) participated.

Stimuli, procedure, and design

The method was as in Experiment 1, with the following exception. The times between the onset of the prime word and the onset of the search display were identical on every trial, regardless of when the space bar was pressed. (However, as in Exps. 1 and 2, the word disappeared after participants had pressed the space bar).

Results and discussion

Action task

During the action task participants were again less accurate during action (96.6%) than viewing (99.4%) trials, t(31) = 5.38 p < .001. The overall accuracy in the action task was 98.0% (SD = 2.0%), and the average RT was 256 ms (SD = 75).

Search task

A 2 Action × 2 Validity analysis was conducted on the search RTs from correct trials (conjoined accuracy: M = 93.4%, SD = 3.6). As in the previous experiments, we found a main effect of validity, F(1, 31) = 44.53, p < .001, η p 2 = .59, and no effect of action, F < 1. Additionally, as in the previous experiments, the factors interacted, F(1, 31) = 8.76, p = .006, η p 2 = .22. The interaction occurred because, although reliable in both conditions, the validity effect was larger after acting in response to the word (68 ms) than after viewing it (37 ms); see Fig. 2. This implies that the results of Experiments 1 and 2 were indeed a result of the action in the presence of the prime word and did not occur because the action reduced the time between the prime and the pictures.

Accuracies in the search task for correct action trials did not differ as a function of action, F < 1, but there was an effect of validity, F(1, 31) = 4.93, p = .034, η p 2 = .14, with better accuracy on valid (95.9%) than on invalid (94.7%) trials. The two factors did not interact, F(1, 31) = 2.11, p = .156.

General discussion

In three experiments, a simple action (a keypress) made while viewing a word prioritized images of a picture related to the word in a subsequent search. The results have implications for three distinct aspects of cognition: language processing, the interplay between action and cognition, and visual search.

First, we’ve shown that simple action is a factor that can enhance semantic activation and strengthen the magnitude of word–picture priming. Previous research has indicated that deeper processing of a prime word increases the magnitude of word–picture priming (e.g., Irwin & Lupker, 1983), so one interpretation of the present results is that making an action in the presence of a word facilitates a deeper level of conceptual processing of the word, which leads to attentional prioritization of its related picture during the search task.

Second, the present results are the first to reveal a “conceptual” action effect—or that simple actions can affect the subsequent perception of objects that share only a conceptual relationship with the acted-on stimulus. This extends previous reports that action can influence subsequent attention to repeated perceptual features (i.e., color; e.g., Buttaccio & Hahn, 2011) and fits with recent research indicating that event files (brief episodic bindings typically defined as containing perceptual and action information; e.g., Hommel, 1998) can store conceptual information, as well (e.g., Frings, Moeller, & Rothermund, 2013). Together, these conclusions imply a more elaborate influence of motor behavior on perception and cognition than has previously been believed. The results also fit well with models of thinking that propose a shared basis for cognitive and motor processes (embodied cognition; e.g., Glenberg, Witt, & Metcalfe, 2013; see also Abrams & Weidler, 2016). According to that view, the ways in which humans interact with the world (i.e., the actions made) influence not only the perceptual system but also higher-level cognitive processes that rely on conceptual relationships, such as language comprehension.

Third, the present results reveal a novel way in which selection history (see Awh, Belopolsky, & Theeuwes, 2012; e.g., Anderson, Laurent, & Yantis, 2011) can influence visual search processes. More specifically, our data illustrate that the previously “selected” item (a word) does not need to share basic perceptual features (e.g., color or shape) with any element in the subsequent search. Instead, prior history with an item that shares a higher-level (i.e., meaning-based) relationship with elements in the search display can influence visual search processes.

It is important to note that action in the present experiments modulated an existing priming effect; in each experiment, there was a reliable effect of validity on the viewing trials. This pattern differs from other action effect experiments, in which validity effects were inconsistent, or even reversed, when participants merely viewed a colored stimulus and search among colored shapes (e.g., Buttaccio & Hahn, 2011; Huffman & Pratt, 2017; Weidler & Abrams, 2014, 2017b). However, the pattern of validity effects that we obtained was not unexpected. Past research has shown that exposure to a word can activate visual properties of its referent (e.g., Boutonnet & Lupyan, 2015; Estes, Verges, & Adelman, 2015; Estes et al., 2008; Reinitz et al., 1989). Theoretically, then, even though participants’ visual search goal was unrelated to the images in the present experiments, the visual information acquired from reading the image’s referent word may have led to that item being prioritized in the search. Indeed, when naturally viewing a scene, participants look more quickly toward images related to a previously seen word than toward unrelated images (e.g., Odekar et al., 2008).

Therefore, a validity effect was expected when participants searched a display containing a picture representing a previously read word. How, then, does action enhance this priming effect? Previous researchers have suggested that the “perceptual” action effect (i.e., when color repeats from prime to search) results from biased competition during visual search, as neural mechanisms attempt to select one out of the multiple stimuli present for more detailed processing (Huffman & Pratt, 2017; cf. Desimone & Duncan, 1995). Specifically, acted-on features are thought to become more heavily weighted as a result of the action, which then makes the objects containing those features more likely to win the competition for selection during the subsequent search. Such a bias (caused by an action) will increase the likelihood that the target object will be selected first on valid trials, resulting in an interaction between action and validity such as those reported here and in earlier action effect experiments (e.g., Weidler & Abrams, 2014). We suggest that in the present experiments a similar mechanism was at work. That is, on trials when an action was made in the presence of a word, properties associated with that word—such as an image of the named object—became more heavily weighted. and thus were more likely to win the competition for selection during the subsequent search.

Additionally, it is noteworthy that the action made in the present experiments was both a very simple motor response and completely arbitrary (i.e., not contingent on any features or meaning of the visual stimulus). Furthermore, the conceptual relationship between the word seen during the action task and one of the images in the search display was irrelevant to participants’ goals during the search (because that relationship did not predict the target location). Because of these features of the task in the present experiments, there are many avenues for future research. For example, it is not known whether complex actions such as grasping or pointing might influence subsequent visual search differently from a simple keypress (see, e.g., Wykowska, Schubö, & Hommel, 2009). Additionally, it is unknown how conceptual relationships might influence visual perception when they are relevant to the participants’ task, as may often be the case in real-world interactions with object (see, e.g., Hommel, 1998, who showed how task relevance can influence some action–perception relationships, or Irwin & Lupker, 1983, who showed that the task requirements for the prime and target can affect semantic priming).

Planning and producing actions is known to prioritize features of the target object both before (e.g., Bekkering & Neggers, 2002) and after (e.g., Buttaccio & Hahn, 2011) the action. Such effects facilitate effective interactions with the objects in our environment by making us more sensitive to the features that are important for manipulating them. We have shown here that the advantage also arises when the actions are directed only to stimuli (words) that evoke the concept of an object, without sharing any perceptual features with it. So, if you are searching for misplaced keys, tapping your finger while viewing the word KEYS may make your search easier.