We compared the classical motorvisual dual-task priming paradigm with three different stimulus sets as target stimuli in the visual discrimination task: hand pictures, arrows and words. According to previous findings with motorvisual priming, we expected negative priming for each stimulus set: that is, we expect better performance when response in the motor task and stimulus in the visual discrimination task are incongruent on the element-level than when they are congruent. The left/right representation should be occupied by motor response processing, and should, consequently, be difficult to access for perceptual processing of a congruent visual discrimination target.
Importantly, we hypothesized that the negative priming effect should be stronger for hand pictures than for arrows, than for words. Words overlap with button presses only on the verbal semantic ‘left’/‘right’ dimension. Arrows overlap with button presses additionally on the non-verbal symbolic level. Finally, pictures of hand movements overlap with hand movements above the verbal and semantic dimensions also on a variety of low-level physical anatomical dimensions.
For Experiment 1, half of the participants were students of Lancaster University, the other half were students of Birmingham University. They received £24 or course credit. All participants reported having normal, or corrected-to-normal vision. Eighteen of the 22 participants were female, 16 were right handed. Their mean age was 19.08 (SD 1.70; range 18–26). The sample size of 22 was chosen, because comparable sample sizes did provide robust motorvisual and visuomotor priming effects in previous studies (see, e.g., Thomaschke et al., 2012b).
The cues for the motor task’s response were left and right arrowheads. These arrowheads were the same as the ones used as visual discrimination targets in the “arrow” condition (see “General method”). These cues were the same for all experimental conditions and were compatibly mapped to the motor responses (see e.g., Hommel & Müsseler, 2006; Müsseler, 1999).
The motor responses were prompted by color change of a frame. For a certain interval, in each trial, a rectangular frame was displayed. During this interval, the frame changed its color from white to red, and back to white again. The frame circumscribed the stimulus area for the target stimuli in the visual discrimination task (arrow and word stimuli and the distinction-relevant area for hand stimuli, see “General method”). Hence, the frame’s interior measures were 32 pixels (width) and 64 pixels (height). The frame’s border was 3 pixels thick.
The target stimulus sets for the visual discrimination task were the hand picture, arrow, and word sets described in the “General method”.
The visual discrimination targets were followed by a mask. The mask had the same extension as the white/red frame (38 × 70 pixels). Half of the mask’s pixels was black, the other half had the same brightness as the preceding stimulus (see below). Which pixels were grey and which ones were black was determined randomly before each trial.
The report of the visual discrimination was cued by two white question marks measuring together 78 pixels (width) and 133 pixels (height). All error messages were written in black, surrounded by white boxes on the black background.
Each trial began with the display of the cue for the motor response for 500 ms, followed by a black screen for 500 ms, and the fixation cross for another 500 ms (see Fig. 3). During that period participants should have prepared the cued motor response (left or right button press) and keep it on hold. Then, the white frame was displayed and turned red after 1000 ms for only six screen cycles (ca. 100 ms), and white again for further 400 ms. The motor response had to be executed within these six screen cycles where the frame was red.
At some point during the frames, the visual discrimination target was displayed with the frame. Trials differed in response stimulus onset asynchrony (RSOA). This is the time interval between the go-signal for the motor response (frame turning red) and the onset of the discrimination target display. This variable was manipulated in most previous motorvisual priming studies, with contrasting findings concerning the time course of the effect (e.g., Oriet, Stevanovski, Jolicoeur, & Cowan, 2003; Wühr & Müsseler, 2001). The variable has been included in the present experiment in order to test whether potential differences between the three stimulus sets are specific to a certain temporal relation between the motor task’s response and the discrimination task’s target stimulus. Frequently tested RSOAs are −400, −200, 0, and 200 ms (e.g., Hommel & Müsseler, 2006). These RSOAs have also been applied in the present study. Each of the four intervals separated response and stimulus in one quarter of all trials. A negative RSOA means that the target stimulus onset preceded the go-signal for the response (frame turning red, see above). The target stimulus was displayed for a duration that was individually determined before each block (see below) by evaluating visual discrimination performance in the previous blocks. It was immediately followed by the mask for 100 ms.
The frame displays were followed by a black screen for 1000 ms, and then the two question marks. These stayed on the screen until the report for the visual discrimination task was given. The report was given by left or right button presses. These button presses were, in contrast to the earlier motor response, not time-pressured. The report was immediately followed by a written feedback message for 150 ms, saying “correct” or “incorrect”. Trials were separated by a 200-ms interval between the offset of the feedback message in one trial and the onset of the cue for the next trial.
Trials differed along three dimensions: motor response (left, or right), discrimination target (left, or right), and RSOAs (−400, −200, 0, or 200 ms). Consequently, there were 16 (2 × 2 × 4) different trial types. Each of the trial types appeared 16 times as experimental trial in each experimental session. The 256 experimental trials in a session were randomized.
The experiment was conducted in four separate sessions for each participant. The sessions were conducted on different days with no more than 2 days between two consecutive sessions for each individual. The purpose of the first session was to determine the individually appropriate display times for each participant for each stimulus set. This session will be referred to as the adaptation phase. The three remaining sessions were identical, with the exception that each of them applied a different stimulus set for the visual discrimination targets. The order of the three stimulus sets was counterbalanced across participants. The three latter sessions will be referred to as the experimental phase.
Each session of the experimental phase comprised 18 blocks, the first two of which were practice blocks and were not analyzed. The total duration of a session lasted 65 min. Each block consisted of one practice trial and 16 experimental trials and. The practice trials were not analyzed. Participants paused for 35 s between blocks. An additional break of 3 min was scheduled between the 10th and 11th blocks.
Any invalid trials were repeated at the end of the respective block. When there were more than four invalid trials within one block, the block was not analyzed, and was repeated at the end of the session. The following types of mistakes made a trial invalid:
The motor response was wrong, meaning it differed from the one that had been, by instruction, assigned to the respective cue.
The motor response was executed too early, meaning before the respective go-signal (see above).
The motor response was executed too late, meaning after the go-signal had expired (see above).
Invalid trials or blocks were followed immediately by a specific error message and information that the respective trial or block will be repeated later. Error messages were displayed for 5 s.
The display time for the visual discrimination target was individually adapted throughout the experiment. When a participant judged more than 14 of the 16 targets in a block correctly, the display time was reduced by 1 screen cycle from the consecutive block on. When a participant judged, on the contrary, more than six targets incorrect, the display time was prolonged by one screen cycle. For each experimental session, the initial display time was set to the duration that was determined in the adaptation session for each participant and stimulus set. The initial display time in the adaptation session was three screen cycles for each of the stimulus sets (see below).
In order to make the three stimulus sets comparable with each other also with regard to display time, a second adaptation algorithm, regarding target brightness, was implemented. It was applied only after blocks that did not require an adaptation of display time (i.e., where participants had judged more than 10 and less than 15 stimuli correctly). When this was the case, and when also the display time for the current stimulus set was longer than the display times of both other stimulus sets, then the brightness of the current stimulus set was increased. In the long run, this had the effect that the participant made fewer incorrect judgements for the current stimulus, and that, consequently, its display time was changed, by the primary adaptation algorithm, towards the display times for the other stimulus sets. Likewise, when both other stimulus sets had longer display times, and when the display time of the current stimulus set was not changed after the current block, its brightness was reduced. Brightness increased or reduced in steps of 10% of the full range (0–255), simultaneously in all three Red Green Blue (RGB) channels. Initial brightness for the stimulus sets were 50% for hand and word stimuli, and 30% for arrow stimuli, relative to full brightness (RGB = 255, 255, 255). Pilot studies have shown that these brightness proportions lead to relatively homogeneous display times for the three stimulus sets. Both algorithms were effective throughout the full experiment, including the adaptation session and the practice blocks in the experimental session.
The brightness adaptation was included to prevent the display time adaptation from yielding very different display times, thereby avoiding potential display time effects on cognitive processing.
The adaptation session differed from the experimental sessions in two main ways. First, the motor response task was absent in the adaptation session. Consequently, the cue for the motor response was not displayed in the adaptation session. However, with the exception of this difference, the trial structure exactly resembled the trials in the experimental sessions. Thus, the go-signal for the motor response was displayed, but had no imperative function. Participants had been informed that it will become relevant in the consecutive experimental sessions. Secondly, all three stimulus sets appeared already in the adaptation session in the same order as they later appeared, one per session, in the experimental phase.
The adaptation session was 15 blocks long—5 for each stimulus set. Each block consisted of 16 randomly ordered trials. Participants paused for 30 s between the blocks, and for an additional 3 min between every fifth block. The purpose of the adaptation session was to determine the individual display times in advance of the experimental sessions. The total duration of the adaptation session was approximately 40 min.
One participant did not complete all blocks of the fourth session, and was hence excluded from all analyses.
Display durations and invalid trials
Table 1 shows the average display times for the first block of each experimental session compared with the average display times of all remaining blocks in the respective experimental session. The relatively small differences show that much of the individual display-time-adaptation had been achieved by the adaptation session. Thus, the differences between display times in individual blocks did not add much variance to the motorvisual priming effect.
Participants produced on average 8.4 (SD 4.3) invalid trials for arrow stimuli, 7.6 (SD 3.6) invalid trials for hand stimuli, and 6.1 (SD = 3.9) invalid trials for word stimuli. A χ
2 test of independence between validity and congruency of trials was conducted separately for each stimulus set, but with no significant results.
Mean accuracy scores were calculated separately for congruent and for incongruent response–stimulus pairings, for each stimulus set, and for each RSOA (see Table 2). We conducted a three-way ANOVA with the factors stimulus set (hands, arrows, words), RSOA (−400, −200, 0, 200), and congruency (congruent, incongruent). We found main effects for stimulus set, F(2, 40) = 4.314, p = .020, η
= 0.177, for RSOA, F(3, 60) = 4.970, p = .004, η
= 0.199, but not for congruency, F(1, 20) = 0.012, p = .913, η
= 0.001. RSOA interacted with congruency, F(3, 60) = 4.121, p = .010, η
= 0.171, and, most importantly, stimulus set also interacted with congruency, F(2, 40) = 10.093, p < .001, η
= 0.335. Neither the interaction between RSOA and Stimulus set, F(6, 120) = 1.645, p = .141, η
= 0.076, nor the three-way interaction attained significance, F(6, 120) = 0.809, p = .565, η
The interaction between stimulus set and congruency was due to motorvisual priming effects in different directions for different stimulus sets: for hand stimuli, performance was significantly better in incongruent trials, t(20) = 2.407, p = .026, but with arrow stimuli, performance was significantly better in congruent trials, t(20) = 2.471, p = .023. With word stimuli, the difference between congruent and incongruent trials was not significant, t(20) = 0.073, p = .943. The priming effect (i.e., performance in congruent trials subtracted from the performance in incongruent trials) differed significantly in pairwise comparisons between all three stimulus sets, t(20) = 4.257, p < .001, for hands vs. arrows, t(20) = 2.426, p = .025, for hands vs. words, and t(20) = 2.185, p = .041 (see Fig. 4).
Despite the non-significant three-way interaction, we analyzed the modulation of the priming effect by RSOA separately for the different stimulus sets, because this modulation seems to point in different directions (see Fig. 5). In two-way repeated measures ANOVAs with the factors RSOA and congruency, the factors significantly interacted only for arrows, F(3, 60) = 4.716, p = .005, η
= 0.191, but not for hands, F(3, 60) = 0.583, p = .628, η
= 0.028, or words, F(3, 60) = 0.774, p = .513, η
= 0.037. The interaction with arrows was due to a decrease of the positive priming effect with RSOA, while the priming effect for hands rather increased numerically with RSOA, which was however, not significant.
We hypothesized that all stimulus sets would show negative priming effects, and that the priming effect would get stronger with higher set-level congruency between stimulus and response. That is, the priming effect should have been stronger for hands than for arrows than for words.
Our predictions have been confirmed by the results for hand and word stimuli. First, for both stimulus set, element-incongruent trials lead to better performance than element-congruent trials, though the effect was not significant for words. Second, the magnitude of the priming effect was stronger for hands than for words. However, for arrow stimuli, the results were surprising and not predicted by our hypotheses. Indeed, a significant motorvisual priming effect was observed, but contrary to our expectancy, it was positive. This result stands in stark contrast to previous motorvisual priming studies with arrowheads (see Thomaschke, Hopkins, & Miall, 2012a, for a review).
Yet, there is a testable explanation for this unexpected result, based on the planning and control model (PCM) of motorvisual priming (Thomaschke, 2012; Thomaschke et al., 2012a). According to the PCM, there is a fundamental difference between the processing of scalar and categorical representations in motor cognition. Categorical representations code action features like the identity of a graspable object, the identity of the acting effector, the valence of the action, etc. These representations classify actions into rather coarse-grained classifications. They convey, among others, also symbolic and semantic information about actions. Scalar representations, on the contrary, code the action’s current position as coordinates in a feature space with metric properties, on dimensions like location, orientation, size, and weight. Scalar representations allow, for instance, computing the future path of actions, or its exact spatial relation to objects.
Categorical representations of action features are known to be involved in action planning and selection, whereas scalar representations are primarily involved in action control (Glover, 2004; Glover, Wall, & Smith, 2012). The PCM claims that action planning is primarily responsible for negative motorvisual priming. Selection of an action binds all representations of categorical action features into a compound representation of that action, and shields them against other cognitive processes. Thus, perception of such features is impaired during action (Hommel et al., 2001; Müsseler, Steininger, & Wühr, 2001). As action selection (not action control) is the primary explanatory domain of ideomotor theory, our literature review was focused on studies, where stimuli and responses overlapped on categorical dimensions. Accordingly, we have chosen the stimulus sets in for the present study so that they overlapped with the response on a categorical stimulus dimension (i.e., the binary categories ‘left’/‘right’). In line with all previous motorvisual priming studies (James & Gauthier, 2009; Kunde & Kiesel, 2006; Kunde & Wühr, 2004; Müsseler, Wühr, & Prinz, 2000; see Thomaschke et al., 2012a, for a review), we hypothesized that the priming effect would be negative.
However, PCM also claims that the processing of scalar representation in action control leads to positive motorvisual priming effects. Scalar representations play an important role in fast online action feedback processing during control; consequently, congruent scalar representations are facilitated. Accordingly positive motorvisual priming has been observed for response–stimulus overlap on various scalar dimensions, like size (Fagioli, Ferlazzo, & Hommel, 2007; Fagioli, Hommel, & Schubotz, 2007; Symes, Tucker, Ellis, Vainio, & Ottoboni, 2008; Wykowska, Hommel, & Schubö, 2011, 2012; Wykowska, Schubö, & Hommel, 2009), location (Collins, Schicke, & Röder, 2008; Deubel, Schneider, & Paprotta, 1998; Fischer & Hoellen, 2004; Hommel & Schneider, 2002; Koch, Metin, & Schuch, 2003; Linnell, Humphreys, McIntyre, Laitinen, & Wing, 2005; Müsseler, Koch, & Wühr, 2005), weight (Hamilton, Wolpert, & Frith, 2004), or orientation (Lindemann & Bekkering, 2009).
How does the PCM relate to the present results? Although it is well established in previous literature that arrows are typically processed categorically as symbols denoting the categories ‘left’ and ‘right’ (e.g., Müsseler & Hommel, 1997a), the arrows might have been processed via scalar representations in our study. Instead of representing and processing the arrows as conveying categorical symbolic information, participants might have encoded and processed locational information of the arrows. They might have attended only to the location of the arrows apex, instead of processing its symbolic meaning. Evaluating whether the arrow’s apex appeared on the left or right side of the decision relevant area, would have also allowed to classify its direction correctly. Thus, the left/right information of the arrows was represented scalar in the form of location information. As response–stimulus overlap on scalar dimensions leads to a positive priming effect, this assumption would be in line with the observed results.
We assume that the scalar processing of arrows was caused by the way we constructed the stimuli. Previous studies with arrows usually described the stimuli by the symbols ‘>’ and ‘<’ appearing in the methods sections. Further information about the thickness of the lines, the angle between these lines and so on is not given. Instead of using the standard font symbols, we constructed the stimuli from scratch as geometric triangles, with relatively broad arrowheads. This might have biased participants to scalar locational encoding of the left/right information.
This interpretation is strongly supported by the temporal dynamics of the priming effect. The influence of action planning typically declines over the course of an action, while the influence of action control increases. If the priming effect for hands and words was due to categorical processing in planning, while the priming effect for arrows was due to scalar processing in control, one would expect over the course of the action a decrease in the former two priming effects, but an increase in the latter one. These were exactly the dynamics observed in the present study as we changed the stimulus onset asynchronies.
Furthermore, the scalar processing of our arrowhead stimuli can be independently tested in Experiment 2, because also the Simon effect has been shown to differ in dynamics for scalar and categorical stimulus–response overlap (see below).
To conclude, for hand and for word stimuli, we confirmed our hypothesis: higher set-level congruency leads to a larger motorvisual priming effect. Yet, the arrow stimuli seem to have been processed as conveying scalar locational information. Processing of scalar information is, however, not within the scope of the ideomotor theory. Thus, our initial hypotheses do not apply to the arrow stimuli. We have modified the hypothesis for Experiment 2 accordingly (see below).