Stimulus–response bindings code both abstract and specific representations of stimuli: evidence from a classification priming design that reverses multiple levels of response representation
Repetition priming can be caused by the rapid retrieval of previously encoded stimulus–response (S–R) bindings. S–R bindings have recently been shown to simultaneously code multiple levels of response representation, from specific Motor-actions to more abstract Decisions (“yes”/”no”) and Classifications (e.g., “man-made”/”natural”). Using an experimental design that reverses responses at all of these levels, we assessed whether S–R bindings also code multiple levels of stimulus representation. Across two experiments, we found effects of response reversal on priming when switching between object pictures and object names, consistent with S–R bindings that code stimuli at an abstract level. Nonetheless, the size of this reversal effect was smaller for such across-format (e.g., word–picture) repetition than for within-format (e.g., picture–picture) repetition, suggesting additional coding of format-specific stimulus representations. We conclude that S–R bindings simultaneously represent both stimuli and responses at multiple levels of abstraction.
KeywordsRepetition primingComponent processesPerceptual primingResponse learning
When a stimulus is repeatedly classified, the reaction time (RT) to make that classification often decreases with repetition. Recent research has shown that a large proportion of this repetition priming reflects the retrieval of stimulus–response (S–R) bindings (Denkinger & Koutstaal 2009; Horner & Henson, 2009; Schnyer, Dobbins, Nicholls, Schacter, & Verfaellie, 2006). According to this view, the co-occurrence of a stimulus and response during the initial classification results in the encoding of an “instance” (Logan, 1990), or “event file” (Hommel, 1998). Cuing of this S–R binding when the stimulus is repeated allows for the rapid retrieval of a response, potentially by passing the perceptual and semantic processes that were engaged by its initial classification (Dobbins, Schnyer, Verfaellie, & Schacter, 2004).
These S–R accounts of priming contrast with the common conception that priming reflects the facilitation of one or more of the cognitive processes that were engaged during initial presentation of the stimulus (Blaxton, 1989; Roediger & McDermott, 1993; Roediger, Weldon, & Challis, 1993). For example, faster visual identification of the stimulus (a perceptual process) and/or faster extraction of task-relevant semantic information (a conceptual process) might also contribute to the shorter RTs for primed stimuli. Such component process (CP) theories generally predict that the amount of priming depends on the degree of overlap between the stimulus-specific processes engaged during initial and subsequent presentations (Franks, Bilbrey, Lien & McNamara, 2000; Morris, Bransford, & Franks, 1977). However, evidence for such CP theories cannot be established unless effects of S–R retrieval are controlled. This issue has been appreciated for some time, and clear evidence of priming has been found under conditions where S–R bindings would not seem relevant (e.g., when a word is repeated from a naming task to a lexical decision task; Bowers & Turner, 2003). Nonetheless, the recent evidence for the flexibility of S–R bindings, such as their multiple levels of response representation, means that previous attempts to control for S–R retrieval, particularly in speeded classification paradigms, have not always been sufficient. Here, we provide evidence for yet greater flexibility—namely, evidence that S–R bindings also include multiple levels of stimulus representation.
An “optimized” design for isolating S–R contributions to priming
Several approaches have been used to separate effects of S–R bindings from effects of CP facilitation within binary classification paradigms. Consider a study–test design, in which stimuli are initially classified in a study phase, after which there is a test phase, in which priming is defined as the difference in classification RTs for stimuli repeated from the study phase relative to RTs for novel stimuli (see Fig. 1b). One approach to assessing S–R retrieval is to compare the amount of priming when the same task is repeated at study and test (where both S–R retrieval and CP facilitation can contribute) with the amount of priming when the task is switched, such that the responses at test are unrelated to those at study (where only CP facilitation might be expected to contribute). For example, the stimuli might be pictures of everyday objects, and the classification in the test phase (e.g., “living thing?”) might be designed to be orthogonal to that in the study phase (e.g., “bigger than a shoebox?”), by ensuring that an equal number of living and nonliving things are bigger than a shoebox (Denkinger & Koutstaal, 2009; Horner & Henson, 2008, 2009). In this case, one might expect that facilitatory effects of response repetition for half of the trials should be canceled by interfering effects of response reversal for the other half of trials. A problem with this approach, however, is that the switch in task may also affect the nature of component processes—for example, requiring retrieval of different semantic information and, hence, reducing any conceptual facilitation. Thus, the reduction in priming for orthogonal relative to repeated study–test tasks does not uniquely index S–R retrieval.1
An alternative approach to assessing S–R contributions has been to reverse the task between study and test. An example of this would be to use a “bigger than shoebox?” classification at study, but a “smaller than shoebox?” classification at test (Dobbins et al., 2004; Horner & Henson, 2009; Schnyer et al., 2007; Schnyer et al., 2006). Because both classification tasks now refer to the same semantic information (the everyday size of an object), the same component processes should be engaged at study and test. At the same time, any facilitatory effects of response repetition in the repeated task can be contrasted with any interfering effects in the reversed task in order to reveal effects of S–R retrieval on priming. However, the comparison of priming in repeated versus reversed tasks again fails to index all types of S–R retrieval. This is because, although reversing the polarity of a classification does reverse responses at the level of Decisions and Actions, it does not reverse the response at the Classification level (Horner & Henson, 2009). A lion, for example, is “bigger” than a shoebox, regardless of the direction in which the question is framed.2
Furthermore, a problem with both the orthogonal and reversed task approaches to isolating S–R contributions is that any change in task between study and test may modulate priming, because bindings between stimuli and task sets have also been shown to affect priming (Koch & Allport, 2006; Waszak et al., 2003). Retrieval of a different task set by repeated but not novel stimuli might reduce the amount of priming, relative to when the same task set is repeated. In other words, differences in the amount of priming between repeated tasks and reversed or orthogonal tasks might be attributed to S–task bindings, rather than S–R bindings.
A third, more recent approach is to keep the task constant but change the referent (Denkinger & Koutstaal, 2009). For example, the comparator object at study (e.g., “bigger than a shoebox?”) could be switched to a larger or smaller object at test (e.g., “bigger than a wheelie bin?”3; Horner & Henson, 2009). Switching referent between study and test results in a repetition of response for stimuli that are either smaller than both a shoebox and a wheelie bin or larger than both a shoebox and a wheelie bin (congruent responses). Stimuli that are bigger than a shoebox but smaller than a wheelie bin, however, will require a reversal in response between study and test (incongruent responses; see Fig. 1c). This reversal now occurs at all three of the levels of response representation identified above, including the Classification (given that a monkey, for example, is “bigger” than a shoebox but “smaller” than a wheelie bin). The contribution of S–R bindings to priming can then be isolated by the difference in priming for congruent and incongruent trials.
We argue that this design is “optimized” in the sense that the comparison of congruent and incongruent responses is a relatively pure measure of S–R bindings, because (1) the component processes engaged are unlikely to be affected by the referent change (unlike for orthogonal tasks); (2) the response representations are reversed at all levels on incongruent trials, including the Classification level (unlike for reversed tasks); and (3) either the task set is unchanged across study and test phases, or, if task set is affected by the referent change, then this is true for both congruent and incongruent trials, such that the difference between them is likely to reflect S–R, rather than S–task, bindings.
The level of stimulus representation in S–R bindings
Using the changing-referent paradigm above that isolates S–R retrieval at multiple response levels, we addressed the nature of stimulus representation in S–R bindings, including the possibility that S–R bindings also include multiple levels of stimulus representation. Two prior studies have investigated the level of abstraction of stimulus representations in such bindings—for example, whether S–R learning effects are restricted to a specific picture of an object, or whether they generalize across different pictures of the same object (i.e., different exemplars with the same basic-level name—e.g., “lion”). Schnyer et al. (2007) used a reversed task approach to isolate S–R contributions but failed to find greater priming for task repetition than for task reversal when the exemplar was changed between study and test. The authors therefore concluded that responses are bound to specific visual images. Denkinger and Koutstaal (2009), however, used stimuli similar to those in Schnyer et al. (2007) and did find significant S–R effects across exemplar changes. Importantly, Denkinger and Koutstaal used a referent-change approach, in which the reversals at multiple levels of response representation may have increased the probability of detecting S–R contributions across exemplars (as explained above).
One issue with these studies however is the degree of visual similarity between exemplars. For example, two differing pictures of a lion are likely to be more visually similar than two pictures of different objects (see Chouinard, Morrissey, Köhler, & Goodale, 2008, for further discussion).To date, no experiment has provided evidence for S–R retrieval that cannot be explained by a degree of visual similarity between the stimulus used at test and the stimulus used at study. To address this issue, we presented the names of objects at study and pictures of objects at test or vice versa. There would seem no reason for any greater visual similarity between, say, the word “lion” and a picture of a lion (the repeated case) than between the word “lion” and a picture of a monkey (the novel case).
Present paradigm and predicted results
We report two experiments that conformed to the same basic design (Fig. 1b, c). Words and pictures were classified at study according to whether the objects to which they referred were “bigger than a shoebox” in real life. At test, these primed stimuli were intermixed with experimentally novel stimuli, and the referent for the size comparison task was made either larger or smaller than the referent used at study, allowing separation of congruent and incongruent repeated trials. In Experiment 1, only pictures were presented at test, such that the repeated trials were further split into Within-format (i.e., picture–picture, with the same picture being used at study and test) and Across-format (i.e., word–picture) conditions; in Experiment 2, both pictures and words were presented at test, producing all four word–word, picture–picture, word–picture, and picture–word conditions.
Note, however, that the precise pattern of priming in the Incongruent condition depends on the nature of S–R retrieval. According to the “event files” of Hommel (1998), when responses retrieved from S–R bindings conflict with those produced by component processes, the resulting interference can lengthen RTs and produce negative priming (as assumed in Fig. 2). According to the “instance” theory of Logan (1990), on the other hand, responses retrieved from S–R bindings are generally assumed to facilitate RTs only when congruent with those produced by component processes (see the General Discussion section). In this case, no priming would be expected for Incongruent trials, and indeed, this has been the modal finding in previous studies (Denkinger & Koutstaal, 2009; Horner & Henson, 2009; Race et al., 2009). A further advantage of the present design, which requires switches at multiple levels of response representation, is that we can test for such interference effects of S–R retrieval more thoroughly than before.
Alternatively, priming might be determined purely by facilitation of perceptual component processes. Here, we would expect to see positive priming in the Within-format conditions, but not the Across-format conditions (Fig. 2d). Of course, there may be multiple factors contributing to priming, from both S–R retrieval and facilitation of component processes. A combination of perceptual facilitation (as in Fig. 2d) and multiple stimulus representations in S–R bindings (as in Fig. 2c), for example, would predict a pattern like that shown in Fig. 2e. Indeed, Fig. 2e foreshadows the general pattern that we found in the present experiments.
In summary, the main purpose of the present study was to assess the level of stimulus representation in S–R bindings. When pictures or words were used at study and pictures at test, Experiment 1 showed a reliable congruency effect across different study and test formats, supporting the existence of abstract stimulus codes in S–R bindings. Nonetheless, there was evidence that this congruency effect was greater for Within-format than for Across-format priming, thereby providing additional evidence for simultaneous coding of more specific stimulus representations. To check that this result did not reflect a general advantage in encoding of S–R bindings for pictures, relative to words, Experiment 2 fully crossed pictures and words at study and test and replicated the critical interaction between Congruency and Within-format versus Across-format priming. Furthermore, the precise pattern of priming across both experiments provided not only new evidence for S–R bindings at multiple levels of stimulus abstraction, but also the most robust evidence to date for interference from incongruent S–R retrieval in a long-lag repetition priming paradigm.
Experiment 1 conformed to a 3 × 2 factorial design, with study condition (Within-format, Across-format, Novel) and Congruency (Congruent, Incongruent) as factors (see Fig. 1b). Note that for Novel stimuli, congruency refers to whether the correct response for the “bigger than X” task would be the same or different for the study task referent as for the test task referent even though participants never actually classified novel items according to the study task referent. Therefore the subtraction of novel RTs from repeated RTs for Congruent and Incongruent conditions separately meant that priming effects were not confounded by item differences owing to how “close” in size each object was to the relevant referent.
The experiment was run on two groups of participants: one group participating as part of an fMRI study, and the other group as part of an MEG study (the neuroimaging data will be reported elsewhere).
Participants in all experiments were recruited from the volunteer panel of the MRC Cognition and Brain Sciences Unit or from the student population of Cambridge University. All participants had normal or corrected-to-normal vision and gave informed consent. The experiment was of the type approved by the Cambridge Psychology Research Ethics Committee (reference: 2005.08). By self-report, all participants were right-handed.
Both participant groups contained 18 participants (full counterbalancing [see below], requiring a multiple of 6 participants). The MEG Group contained 10 males (8 females) with a mean age of 25.9 years (SD = 4.3); the MRI Group contained 8 males (10 females) with a mean age of 23.5 years (SD = 4.0).
Stimuli were 384 colored images of everyday objects and their names, previously used by Horner and Henson (2009), split into two groups, relating to the wheelie bin and pencil case referent change (192 stimuli per group). For the wheelie bin referent group, stimuli were classified so that 25% were smaller than both a shoebox and a wheelie bin (Congruent), 50% were bigger than a shoebox but smaller than a wheelie bin (Incongruent), and 25% were bigger than both a shoebox and a wheelie bin (Congruent). For the pencil case referent group, 25% were smaller than a pencil case and a shoebox (Congruent), 50% were bigger than a pencil case but smaller than a shoebox (Incongruent), and 25% were bigger than a pencil case and a shoebox (Congruent). This resulted in 96 stimuli per Congruency condition for each referent group. Stimuli within each of these Congruency groups were randomly assigned to one of three study condition groups, relating to whether they were presented as a picture at study (Within-format), were presented as a word at study (Across-format), or were experimentally novel (Novel). This resulted in 64 stimuli per condition when collapsing across the two referent changes. The assignment of stimuli to the three study condition factors was rotated across participants.
Prior to the experiment, participants performed a practice session using the “bigger-than-shoebox” task, where it was made clear that this comparison referred to the object’s typical size in real life. They responded using a “yes” or “no” key with their right or left index finger, respectively, and were required to respond as quickly as possible without compromising accuracy. Stimuli in the practice session were 10 objects (5 pictures, 5 words) that were not included in the main experiment. Following the practice session, participants were shown example photos of each object referent (i.e., shoebox, wheelie bin, pencil case) and were asked to report the average size of each referent. They were told that the referent might change in the course of the experiment; however, they were not informed as to when this might occur.
The experiment consisted of four alternating study–test cycles (two relating to the wheelie bin referent and two relating to the pencil case referent), with each cycle lasting approximately 15 min. During each study phase, 64 stimuli were shown 3 times, resulting in 192 trials. Thirty-two stimuli were presented as pictures (Within-format), and 32 were presented as words (Across-format). Each set of 32 stimuli consisted of equal numbers of Congruent and Incongruent items. Apart from ensuring that there were no immediate repetitions, the stimulus presentation order was randomized. Participants were always asked “is the object bigger than a shoebox?” at study.
During each test phase, the 64 stimuli from the study phase (Within-format and Across-format) were randomly intermixed with 32 novel stimuli (Novel). All items at test were presented as pictures. Participants were asked either “is the object bigger than a wheelie bin?” or “is the object bigger than a pencil case?” The order of task (i.e., referent change) was counterbalanced across participants in an ABBA/BAAB manner.
For the MEG group, participants were seated with their head underneath the helmet of an MEG machine, and stimuli were projected onto a screen approximately 1 m in front of them, such that images subtended approximately 6o of horizontal and vertical visual angle. Each trial consisted of a 500-ms fixation cross, followed by a stimulus for 1,000 ms, followed by another fixation cross for 500 ms. At the end of each trial, a circle was presented centrally for a variable time between 750 and 1,250 ms, with a mean of 1,000 ms (this jittering was including for methodological reasons relating to the collection of MEG/EEG data). For the MRI group, participants were supine, and stimuli were viewed via a mirror approximately 20 cm above the participants’ eyes, such that stimuli subtended approximately 4o of visual angle. Each trial consisted of a 500-ms fixation cross, followed by a stimulus for 2,000 ms, followed by a blank screen for 500 ms. The total (mean) trial length for both the MEG and MRI studies was 3,000 ms. For both groups, words were presented in black on a white background, with the same pixel dimensions as the pictures.
Trials with RTs less than 400 ms or two or more standard deviations above or below a participant’s mean for a given task were excluded (also rendering the RT distributions more Gaussian). Given that there is some subjectivity in determining whether an object is bigger than a shoebox, wheelie bin, or pencil case, accuracy was defined by the modal response across participants for each object. RTs for correct trials at test constituted the main dependent variable. Given the focus on S–R effects, RTs were further restricted to objects also given a correct judgment on every occurrence at study. These RT data were subjected to repeated measures analyses of variance (ANOVAs). Only ANOVA effects that survived an alpha of .05 were reported, and unless stated otherwise, t-tests were two-tailed.
Two ANOVAs were performed: (1) a two-way mixed ANOVA on novel RTs, with Congruency (Congruent vs. Incongruent) and Participant group (MRI vs. MEG) as factors, and (2) a three-way mixed ANOVA on priming scores, with Congruency, Participant group, and Format match (Within- vs. Across-format) as factors. Because of potential differences in Congruent and Incongruent RTs for Novel stimuli (as tested by the first ANOVA), the three-way ANOVA on priming scores was performed for both subtractive and proportional definitions of priming. Subtractive priming was simply the difference in RTs for Repeated versus Novel stimuli (Novel-Repeated); proportional priming was the difference in RTs for Repeated versus Novel stimuli divided by RTs for Novel stimuli [(Novel-Repeated)/Novel], which controls for linear Novel RT (i.e., baseline) differences across Congruency conditions (Schnyer et al., 2006). Proportional analyses for Experiments 1 and 2 are reported in the Supplementary Material.
Additional planned comparisons on subtractive and proportional priming measures were performed as one-tailed t-tests for (1) greater priming for Congruent than for Incongruent trials in the Across-format condition alone (supporting the existence of abstract stimulus codes), (2) a greater effect of Congruency on priming for the Within- than for the Across-format conditions (supporting the additional existence of format-specific stimulus codes), and (3) negative priming for Incongruent conditions (supporting the interfering effects caused by S–R bindings that code a response that is opposite to that required at test).
Mean percentages of errors and reaction times (RTs), (plus standard deviations) across Within-format, Across-format, and Novel conditions in Experiment 1, plus error priming, RT priming, and proportional priming (prop. priming) as a function of Congruency (Congruent [Con], Incongruent [Incon])
RT prop. priming
RT prop. priming
The 2 × 2 (Congruency × Participant group) ANOVA on Novel conditions showed only a significant main effect of Congruency, F(1, 34) = 69.4, p < .001, which reflected longer RTs in the Incongruent than in the Congruent condition. This was expected, since these items tended to be closer to the referents and, hence, more ambiguous, and illustrates the importance of having separate novel baselines with which to measure priming, as well as of measuring priming by proportional as well as subtractive means.
Inspection of Fig. 3a suggests that our response congruency manipulation had an effect on priming, with more positive priming evident in the Congruent than in the Incongruent conditions. Indeed, there was evidence for negative priming in the Incongruent Across-format condition (error bars in Fig. 3a are two-tailed 95% confidence intervals). Furthermore, the effect of Congruency on priming appeared greater for the Within- than for the Across-format condition. This was confirmed by the 2 × 2 × 2 (Congruency × Format match × Participant group) ANOVA on subtractive priming, which revealed an interaction between Congruency and Format match, F(1, 34) = 5.13, p < .05, as well as main effects of Format match, F(1, 34) = 84.51, p < .001, and Congruency, F(1, 34) = 41.45, p < .001.
Averaging across Participant group, our first planned comparison confirmed greater priming for Congruent than for Incongruent trials in the Across-format condition alone, t(35) = 4.33, p < .001, supporting the existence of abstract stimulus codes. Our second planned comparison confirmed the interaction above, in that Congruency had a greater effect on priming in the Within- than in the Across-format conditions, t(35) = 2.14, p < .05, supporting the additional existence of format-specific stimulus codes. Our third planned comparison confirmed negative priming for Incongruent trials, t(35) = 3.21, p < .01, at least when averaging across Format match, supporting the existence of interference from S–R bindings that code a response that is opposite to that required at test. The latter negative priming effect was reliable, however, only in the Across-format condition (as is apparent in Fig. 3a), an issue we return to in the Discussion section.
In the ANOVA above, the three-way interaction between Congruency, Format match, and Participant group also reached significance, F(1, 34) = 4.97, p < .05, with a greater effect of Congruency on priming in the Within-format than in the Across-format conditions (our second planned comparison) within the MEG group, t(17) = 3.19, p < .01, but not within the MRI group, t(17) = 0.03, p = .98. We return to this point in the Discussion section. Nonetheless, the Congruency effect on priming within the Across-format condition alone (our first planned comparison) was significant within both the MEG group, t(17) = 2.66, p < .01, and the MRI group, t(17) = 3.42, p < .001. Analyses of proportional priming revealed the same pattern of results, with all three planned comparisons reaching significance (see the Supplementary Material).
Controlling for response bias at test
One aspect of our design, noted by a reviewer, is that there was an unequal distribution of bigger/smaller responses across test trials (e.g., 75% of the stimuli were bigger than a pencil case, and 75% of the stimuli were smaller than a wheelie bin), which was likely to induce a response bias toward the more frequent response in a given test phase. Moreover, this distribution differed across Congruent and Incongruent conditions, with 50% of Congruent trials receiving the most frequent response, but 100% of Incongruent trials receiving the most frequent response. As was expected, a bias against the less frequent response meant that RTs were longer overall for non frequent than for frequent responses within the Congruent condition (collapsed across other factors), t(35) = 13.71, p < .001. Although this bias would have increased the average RTs for Congruent trials, this increase was not sufficient to counteract the longer RTs to Incongruent than to Congruent Novel trials reported above.
Given that separate novel baselines were used to measure Congruent and Incongruent priming, this response bias would not necessarily be expected to differentially affect priming. Indeed, there was no evidence for a significant difference in overall priming for non frequent (41 ms) relative to frequent (49 ms) responses, t(35) = 1.24, p = .22. Nonetheless, we repeated the above ANOVA on subtractive priming after excluding the 50% of the trials in the Congruent condition that were associated with the non frequent response. As above, the main effects of Format match, F(1, 34) = 89.2, p < .001, and of Congruency, F(1, 34) = 51.3, p < .001, were significant, although the Format match × Congruency interaction did not quite reach significance, F(1, 34) = 4.06, p = .052 (although it was in the same direction as above). A further two-way interaction between Format match and Participant group also reached significance, F(1, 34) = 8.58, p < .01, revealing a greater difference between Within-format than Across-format priming in the MRI group, as compared with the MEG group.
Finally, our planned comparisons confirmed (1) significantly greater priming for Congruent than for Incongruent stimuli in the Across-format condition alone, t(35) = 4.70, p < .001, and (2) a greater Congruency effect in the Within- than in the Across-format condition, t(35) = 1.98, p < .05 (given our planned comparisons are one-tailed). The third planned comparison concerned only Incongruent conditions and so was unaffected by removal of infrequent responses in the Congruent condition. Thus, we could find no evidence that response bias affected the present pattern of priming effects.
The most important finding from Experiment 1 was that there was a reliable effect of response congruency on priming even when the stimulus appeared in a different format between its initial and final presentations (the Across-format condition). This finding, corresponding to our first planned comparison, was found (1) within both participant groups, (2) using either a subtractive or a proportional measure of priming, and (3) regardless of the response bias that confounded the main effect of Congruency on overall RTs. This finding suggests that the stimulus representations within the S–R bindings that are assumed to underlie this congruency effect are more abstract than a specific letter string or picture. Importantly, unlike previous research that switched between pictures of two exemplars of the same object (Denkinger & Koutstaal, 2009; Schnyer et al., 2007), the present finding cannot be explained simply by visual similarity between the formats used at study and test.
A second important finding was that the effect of Congruency on priming was even greater when the visual format was maintained across study and test (i.e., greater in the Within- than in the Across-format condition). This finding suggests additional coding of more specific stimulus representations within S–R bindings, which add to the Congruency effect when the format is repeated across presentations. The evidence for this finding was not as clear-cut, however, in that this difference between Within- and Across-format conditions was significant only in one of the participant groups. The reason for this is unclear, since the procedure was very similar for both groups. It may reflect individual differences or possible differences between the scanner environments (see Hommel, Fischer, Colzato, van den Wildenberg & Cellini, in press). More important, there is an alternative explanation for the greater Congruency effect in Within- than in Across-format conditions: that the encoding of S–R bindings is stronger or more frequent for pictures than for words.5 Because the Within- versus Across-format comparison of Experiment 1 was confounded by the use of pictures versus words at study, the greater effect of Congruency on priming in the Within- than in the Across-format condition could equally well reflect a greater effect of Congruency on priming for stimuli studied as pictures than for stimuli studied as words. This alternative was tested in Experiment 2.
The last important finding of Experiment 1, in relation to our third planned comparison, was that priming was negative (i.e., longer RTs for Repeated than for Novel stimuli) in at least one of the two Incongruent conditions. This is the first clear evidence from long-lag visual object classification paradigms for interference between S–R bindings and component processes (Horner & Henson, 2009)—that is, when the response retrieved from an S–R binding is the opposite of that generated by an algorithmic route. Interestingly, the negative priming effect was seen only in the Across-format Incongruent condition, with no evidence for positive or negative priming in the Within-format Incongruent condition. The latter finding of no negative priming may reflect such interference being counteracted by a decrease in RTs for Repeated Within-format items, owing to, for example, the facilitation of perceptual component processes that occurs when an item is repeated in the same visual format. This was another issue explored in Experiment 2.
Experiment 1 provided unequivocal evidence that S–R congruency effects on priming can be seen despite switching from words at study to pictures at test (the Across-format condition), suggesting that S–R bindings code abstract representations of stimuli. It was less clear, however, whether S–R bindings code additional format-specific representations. The evidence for greater S–R congruency effects in the Within- than in the Across-format conditions was mixed (differing across participant groups) and was also confounded by possible format differences (because the Within- vs. Across-format factor was confounded by pictures vs. words at study). A second experiment was therefore conducted to address these issues, in which picture and word formats were factorially crossed with Within- and Across-format conditions.
More specifically, Experiment 2 entailed three main changes from Experiment 1. First,we introduced a between-participants test-format factor, whereby either words or pictures were presented at test. Test format was therefore crossed with the Format match factor employed in Experiment 1. Thus, in the Picture Test-format condition, the Within- and Across-format conditions were identical to those in Experiment 1 (i.e., the Picture–picture and Word–picture conditions, respectively). In the Word Test-format condition, however, the Within-format condition entailed words at study (the Word–word condition), while the Across-format condition entailed pictures at study (the Picture–word condition). If S–R bindings form at multiple levels of stimulus representation, we should see a larger congruency effect in the Within-format conditions (Picture–picture and Word–word) than in the Across-format conditions (Word–picture and Picture–word). This would be revealed as an interaction between Congruency and Format match on priming (akin to the interaction seen in Experiment 1), which does not further interact with Test-format. If, on the other hand, the results of Experiment 1 were simply due to encoding of stronger S–R bindings for pictures than for words, we should see a larger congruency effect in the Picture–word than in the Word–picture conditions (and Picture–picture than Word–word conditions). In this case, we would expect a three-way interaction between Congruency, Format match, and Test-format on priming.
A second change from Experiment 1 was the addition of an orthogonal manipulation of prime-level, whereby stimuli at study were presented either once (Low-primed) or three times (High-primed). This prime-level manipulation has been shown to modulate S–R effects in similar paradigms(Dobbins et al., 2004; Horner & Henson, 2009) and so was included as an extra dimension along which to compare Within-format and Across-format congruency effects. Moreover, by having some items that had been seen only once at study, we hoped to reduce the contribution to priming of the facilitation of perceptual/conceptual component processes, relative to their contribution in Experiment 1 (where all items were presented three times at study—i.e., equivalent to “high-primed” stimuli in the present experiment). This reduced contribution to positive priming might, in turn, reveal greater negative priming effects in the Incongruent conditions and, hence, stronger evidence for interference between incompatible S–R bindings and component processes (given that negative priming, although significant for the Across-format condition of Experiment 1, was not significant for the Within-format condition).
The final change from Experiment 1 was the increase in the proportion of Novel items at test, so that they made up 50% of the test trials (as compared with 33% in Experiment 1). By randomly assigning these Novel items to four subsets, with which to compare the High- versus Low-primed and Within- versus Across-format Repeated items, the additions above resulted in a 2 × 2 × 2 × 2 × 2 pseudo factorial design, with Test-format (Picture, Word), Format match (Within-format, Across-format), Congruency (Congruent, Incongruent), Prime-level (Low-primed, High-primed), and Repetition (Repeated, Novel) as factors. This also allowed independent baselines with which to measure priming for Within- and Across-format conditions, unlike in Experiment 1.
All participants were drawn from the student populations of the University of Cambridge or of University College London. Seventy participants (38 male) gave informed consent to participate in the experiment. Five participants were excluded from the main analysis due to abnormally high error rates (>30%). A further participant was excluded due to switching response keys during the experiment. This resulted in a total of 64 participants, of which 32 were assigned to the Picture Test-format condition and 32 were assigned to the Word Test-format condition (full counterbalancing now requiring a multiple of 8 participants). The mean age across participants was 24.0 years (SD = 4.2). By self-report, 3 participants were left-handed; the remainder were right-handed.
The 96 stimuli per Congruency per referent were initially randomly assigned to one of eight groups relating to the eight remaining experimental conditions (formed by crossing the factors of Repeated vs. Novel, Within- vs. Across-format, and High- vs. Low-primed), resulting in 12 stimuli per group. The assignment of these groups to experimental conditions was then rotated across participants. Note that there was no reason to expect performance differences, on average, between the four groups assigned to the Novel condition for a given participant, given that the Format-match and Prime-level manipulations did not apply to Novel items (hence, the “pseudo factorial” design).
The data were acquired while the participant was seated within a behavioral testing room, with the stimuli presented on a computer screen approximately 57 cm in front of the participant, with images subtending approximately 6o of visual angle. During each of the four study phases, 48 stimuli were shown: 12 words presented once (Low-primed), 12 words presented three times (High-primed), 12 pictures presented once (Low-primed), and 12 pictures presented three times (High-primed), resulting in 96 trials. Importantly, for each set of 12 stimuli, half received a Congruent and half an Incongruent response at test. During each test phase, the 48 stimuli from the study phase (Repeated) were randomly intermixed with 48 Novel stimuli. For participants assigned to the Picture Test-format condition, all test stimuli were pictures, and for those assigned to the Word Test-format condition, all test stimuli were words. The timings were identical to those for the MRI group in Experiment 1.
Mean percentages of errors, error priming, reaction times (RTs), RT priming, and proportional priming (plus standard deviations) across Test-format, Format match, Congruency, Prime-level, and Repetition for Experiment 2. LP = Low-primed; HP = High-primed; N = Novel; R = Repeated; P = priming (Novel-Repeated); PP = proportional priming [(Novel-Repeated)/Novel]. Note that the division of Novel stimuli into High- versus Low-primed and Within- versus Across-format is based on an arbitrary, equal split
The 2 × 2 × 2 × 2 ANOVA on subtractive priming showed a significant main effect of Congruency, F(1, 62) = 6.03, p < .05, with decreased errors for congruently primed items, but increased errors for incongruently primed items. This pattern of decreases and increases with Congruency is the opposite of that found on RTs below, suggesting that there was no speed–accuracy trade-off. There was also a main effect of Format match, F(1, 62) = 4.16, p < .05, with more priming-related errors for Across- than for Within-format conditions.
A further 8.5% of Repeated trials were excluded from RT analyses due to incorrect responses given at study (see Footnote 4). Table 2 displays the mean RTs, as well as subtractive and proportional priming measures, while Fig. 3b shows subtractive priming of RTs by Format match and Congruency (averaging across Test-format and Prime-level).
The 2 × 2 × 2 × 2 (Test-format × Format match × Congruency × Prime-level) ANOVA on Novel conditions showed a significant main effect of Congruency, F(1, 62) = 77.1, p < .001, which reflected longer RTs in the Incongruent than in the Congruent conditions (as was seen in Experiment 1), and a significant main effect of Test-format, F(1, 62) = 8.70, p < .05, which reflected longer RTs for Words than for Pictures at test (and paralleling the greater number of errors for Novel trials above). As in Experiment 1, the reliable main effects illustrate the importance of having separate Novel baselines with which to measure priming and of measuring priming by proportional as well as subtractive means below.
Inspection of Fig. 3b suggests that subtractive priming effects were similar to those seen in Experiment 1. First, we saw robust positive priming in both the Within-format and Across-format Congruent conditions. Second, the Congruency effect appeared greater for Within- than for Across-format conditions. Third, negative priming was now seen for Within-format, as well as Across-format, Incongruent conditions.
The 2 × 2 × 2 × 2 (Test-format × Format match × Congruency × Prime-level) ANOVA on subtractive priming replicated the interaction between Congruency and Format match, F(1, 62) = 10.7, p < .05, as well as main effects of Format match, F(1, 62) = 15.7, p < .001, and Congruency, F(1, 62) = 59.1, p < .001. Importantly, this interaction did not depend on Test-format, in that the Congruency × Format-match × Test format interaction was not significant, F(1, 62) < 1. As such, it is unlikely that the Congruency × Format-match interaction in Experiment 1 was driven by stronger S–R encoding for Pictures than for Words at study. The only other effect to reach significance in the ANOVA above was the two-way interaction between Congruency and Prime-level, F(1, 62) = 5.88, p < .05, which reflected a greater Congruency effect for High- than for Low-primed items, as would be expected if multiple study presentations lead to stronger S–R bindings.
Averaging across Prime-level and Test-format, our planned comparisons confirmed (1) greater priming for Congruent than for Incongruent trials in the Across-format condition alone, t(63) = 3.85, p < .001, (2) a greater congruency effect in the Within-format than in the Across-format condition, t(63) = 3.30, p < .01 (as in the interaction above), and (3) negative priming in the Incongruent conditions, t(63) = 3.66, p < .01 (Fig. 3b). Importantly, all three planned comparisons were significant in the Picture Test-format condition [(1) t(31) = 3.91, p < .001; (2) t(31) = 2.44, p < .05; (3) t(31) = 4.08, p < .001], and the first two planned comparisons were significant in the Word Test-format condition [(1) t(31) = 1.83, p < .05; (2) t(31) = 2.29, p < .05], with the third comparison approaching significance, t(31) = 1.59, p = .06. Thus, evidence for (1) abstract stimulus codes, (2) additional specific stimulus codes, and (3) interference effects from the retrieval of an incongruent S–R binding was found regardless of the specific format of the stimuli used at test. The same pattern of significant effects was found when scoring priming proportionally rather than additively (see the Supplementary Material).
Controlling for response bias at test
Finally, as in Experiment 1, we interrogated the data for possible effects of response bias. Due to insufficient numbers of trials per condition (<5), 6 participants from the Word Test-format condition were excluded. Although raw RTs were longer for non frequent than for frequent responses in the Congruent condition when either Pictures, t(31) = 9.71, p < .001, or Words, t(25) = 9.10, p < .001, were used at test, no significant difference was seen between subtractive priming scores for frequent responses (pictures at test, 63 ms; words at test, 65 ms), relative to non frequent responses (pictures at test, 46 ms; words at test, 58 ms), when either Pictures, t(31) = 1.14, p = .26, or Words, t(25) = 0.51, p = .61, were used.
The 2 × 2 × 2 × 2 ANOVA on subtractive priming, when restricted to frequent responses at test, replicated the interaction between Congruency and Format match, F(1, 62) = 10.8, p < .05, as well as the main effect of Format match, F(1, 62) = 16.7, p < .001, the main effect of Congruency, F(1, 62) = 86.2, p < .001, and the two-way interaction between Congruency and Prime-level, F(1, 62) = 16.7, p < .001, plus an additional main effect of Prime-level, F(1, 62) = 4.89, p < .05. Our planned comparisons confirmed (1) significantly greater priming for Congruent than for Incongruent stimuli in the Across-format condition, t(63) = 4.51, p < .001, (2) a congruency effect that was significantly greater in the Within-format than in the Across-format condition (as in the interaction above), t(63) = 3.34, p < .001, and (3) significant negative priming in both the Across-format Incongruent condition, t(63) = 3.65, p < .001, and the Within-format Incongruent condition, t(63) = 2.54, p < .01. Thus, the present results did not seem affected by a response bias at test.
As in Experiment 1, we found a reliable effect of response congruency on priming even when the stimulus appeared in a different format between its initial and final presentations (the Across-format condition). Furthermore, this occurred regardless of whether the stimuli at test were pictures or words. Once again, this support for our first planned comparison suggests that S–R bindings include abstract representations of stimuli. The important new finding in Experiment 2 was that the additional, greater effect of congruency on priming when the visual format was maintained across study and test (i.e., greater in the Within- than in the Across-format conditions) occurred regardless of whether test stimuli were pictures or words. This support for our second planned comparison suggests that S–R bindings additionally code format-specific stimulus representations. Importantly, Experiment 2 ruled out the alternative explanation for this pattern in Experiment 1: that it was due to stronger or more frequent encoding of S–R bindings when pictures were presented, relative to words. Furthermore, we also confirmed that this greater congruency effect in the Within- than in the Across-format conditions was unlikely to have been due to a specific participant group (suggesting that the lack of this effect in the MRI group of Experiment 1 was a type II error). Additional analyses also confirmed that neither of our planned comparisons was likely to have been due to proportional differences in priming as a function of novel (baseline) RTs or to a bias against less frequent responses at test. Finally, these effects were unlikely to be specific to the ratio of repeated versus novel items at test, given that this ratio differed across Experiments 1 and 2.
The only notable difference from Experiment 1 was that the negative priming for Incongruent trials in the Within-format condition now reached significance. This support for our third planned comparison suggests that S–R bindings interfere with component processes, even when those processes include perceptual processes that are likely to be facilitated by repetition of the same format between study and test. One reason why this negative priming reached significance in Experiment 2, but not in Experiment 1, may be the inclusion of Low-primed trials in Experiment 2, whereby one half of the repeated items were presented only once (rather than thrice) at study. This may have decreased any contribution from the facilitation of perceptual processes in the Within-format condition, revealing interference effects in the Incongruent condition. There is no direct support for this hypothesis, however, in that we did not see a significant interaction between Format-match and Prime-level that would be expected if facilitation of perceptual processes were reduced for Low-primed stimuli, relative to High-primed stimuli. This issue is complicated by the fact that we saw a significant decrease in the congruency effect for Low-primed rather than for High-primed items, suggesting that number of stimulus repetitions at study affects the magnitude of congruency effects (as well as possibly affecting perceptual facilitation). An alternative explanation is that the lack of negative priming in Experiment 1 was simply a type II error. In either case, the significant negative priming found in three of the four Incongruent conditions across Experiments 1 and 2 represents novel evidence that S–R bindings can slow down RTs even in long-lag repetition priming paradigms (cf. Logan, 1990). We turn to such theoretical issues in the General Discussion section.
The present experiments were designed to assess the level of stimulus representation in S–R bindings. Using an “optimized” classification priming paradigm that controls for multiple levels of response representation, we found evidence of greater priming for stimuli that received congruent, as opposed to incongruent, responses across study and test presentations. Importantly, across both experiments, we saw such a congruency effect despite switching visual format between study and test—for example, from a word at study (e.g., the word “lion”) to a picture at test (e.g., a picture of a lion).This suggests that the representations of stimuli in S–R bindings can abstract away from the specific visual format. Although previous research (e.g., Denkinger & Koutstaal, 2009) has suggested that such stimulus representations are not specific to a particular picture (exemplar), it was unclear whether such across-exemplar congruency effects were, in fact, driven by residual visual similarity between pictures. The across-format manipulation used here does not suffer from this issue.
Do S–R bindings include multiple levels of stimulus representation?
Interestingly, we also saw evidence for a greater effect of congruency on priming when stimuli were presented as pictures at both study and test than when they were presented as words at study and pictures at test (Experiment 1) or vice versa (Experiment 2). As was outlined in the introduction, this result can be taken as evidence for binding of responses to two distinct levels of stimulus representation—one at a specific perceptual (visual) level and one at an abstract level (e.g., an object identity)—both of which could contribute to the congruency effect in our Within-format condition, but only the latter of which could contribute to the congruency effect in our Across-format condition. This possibility of more than one level of stimulus representation might help explain the apparent discrepancy between the degree of generalization of S–R bindings found in previous studies (cf. Denkinger & Koutstaal, 2009; Schnyer et al., 2007). The existence of multiple levels of stimulus coding within S–R bindings also parallels the recent evidence for multiple levels of response coding (Horner & Henson, 2009), reinforcing the flexibility of S–R learning.
Facilitation and interference as a result of S–R retrieval
We generally saw evidence for a speeding of RTs in the congruent condition (i.e., positive priming) and a slowing of RTs in the incongruent condition (i.e., negative priming). This pattern suggests that the retrieval of S–R bindings can both facilitate and interfere with the responses generated by component processes. Previous research using long-lag, speeded classification paradigms has shown primarily facilitatory effects (i.e., positive priming), with little evidence for significant interference from S–R retrieval(Dobbins et al., 2004; Horner & Henson, 2008, 2009; Race et al., 2009; Schnyer et al., 2007; Schnyer et al., 2006).However, these studies did not manipulate S–R retrieval at all levels of response representation, particularly the Classification level, so they may not have been as sensitive to S–R effects as the present experiments (although see Denkinger & Koutstaal, 2009). Although previous research has reported interference effects across immediate repetitions (Hommel, 1998; Keizer, Colzato, & Hommel, 2008; Kuhn, Keizer, Colzato, Rombouts, & Hommel, 2011) or when participants rapidly changed tasks (Posse et al., 2006; Waszak & Hommel, 2007), the present data constitute the first evidence for both facilitation and interference from S–R bindings during long-lag repetition priming, with no (or at most one) change of task set.
Evidence for interference from S–R bindings has implications for S–R theories, particularly instance theory (Logan, 1988, 1990). According to this theory, the speedup in RTs in repetition priming experiments is explained by a race between an algorithmic (i.e., component process) route and an independent, instance retrieval (i.e., S–R) route. The additional operation of the instance retrieval route for repeated stimuli (but not novel stimuli) means that the average (and variance) of RTs decreases with number of repetitions (note that the algorithmic route is not itself assumed to be affected by priming; i.e., there is no facilitation of component processes within this theory). Instance theory therefore does not predict negative priming: Either the instance retrieval route wins the race, speeding up RTs, or the algorithmic route wins the race, resulting in no change in RTs. In order to explain negative priming, the two routes must interact prior to response generation. That is, the retrieval of a response from an S–R binding must interfere with response production if it conflicts with the response being generated by the algorithmic route. One possibility is that this interference results from competition between opposing response representations (e.g., “bigger” vs. “smaller” at the Classification level) that gradually accumulate evidence from multiple routes (Horner & Henson, 2009). Responses from the multiple routes reinforce each other when they agree, producing positive priming, but interfere with each other when they disagree, producing negative priming.
Facilitation of component processes?
The present experiments also underlined the difficulty in revealing evidence for facilitation of component processes, at least in speeded classification paradigms. Negative priming was seen in three of the four incongruent conditions across Experiments 1 and 2, raising the possibility that any facilitation of component processes was potentially masked by interference owing to the retrieval of an incongruent response. Despite this, we did see evidence for a greater magnitude of positive priming than of negative priming in the Within-format condition (see Fig. 3). One possibility is that the amount of facilitation caused by retrieval of congruent S–R bindings is greater than the amount of interference caused by retrieval of incongruent S–R bindings, although it is unclear why this would not also produce a similar asymmetry in the Across-format condition, for which positive and negative priming for Congruent and Incongruent conditions was more symmetrical. Another possibility is that the large positive priming in the Within-format Congruent condition reflects the facilitation of perceptual component processes that occurred only when the stimulus was repeated in the same visual format (as in Fig. 2e). This would also accord with previous word priming studies that have shown evidence of priming for low-frequency words despite no obvious contribution from the retrieval of S–R bindings (Bowers, 2000; Bowers & Turner, 2003; Coane & Balota, 2010) and with other research suggesting that switches between pictures and words can greatly diminish, if not eliminate, priming (Bowers & Turner, 2003; Rajaram & Roediger, 1993). If so, then the present data raise the interesting possibility that any across-format or across-modality priming that is found (in speeded classification paradigms) may reflect abstract S–R learning, rather than facilitation of higher level (e.g., conceptual) component processes. This ubiquity of S–R bindings, as seen in the present experiments, therefore underlines the importance of S–R learning in priming.
One might isolate S–R retrieval by splitting trials within an orthogonal test task according to whether the yes/no response was repeated (congruent) or reversed (incongruent) between study and test (Horner & Henson, 2009; Race, Shanker, & Wagner, 2009). However, while such a comparison reverses the yes/no Decision and/or a specific Motor-action, it does not reveal effects of S–R retrieval at the level of the Classification (Fig. 1a), since the classifications given at study (e.g., “living”) are irrelevant to those given at test (e.g., “bigger”).
A related proposal is that, because participants are likely to be aware of the task reversal, they could adopt a strategy such that when a response (e.g., “yes” and/or left finger press) is retrieved from an S–R binding, it could be quickly reversed and still produce shorter RTs relative to novel stimuli.
A wheelie bin is a large trashcan in the U.K.
One potential consequence of trimming stimuli on the basis of their accuracy at study is that stimuli that are particularly difficult to classify are excluded from the Repeated conditions, but not from the Novel conditions. Given that these stimuli might be expected to produce longer RTs, relative to easier stimuli, this exclusion may have the consequence of artificially inflating RT priming effects. However, we could find no evidence for this possibility, with overall RT priming in Experiment 1 actually decreasing from 19 to 12 ms (and from 26 to 11 ms in Experiment 2) when stimuli were excluded solely on the basis of accuracy at test (i.e., equating Repeated vs. Novel conditions). Thus, we could find no evidence that trimming in this manner artificially inflated overall RT priming effects.
This is not to imply that S–R learning does not occur for words, since several other experiments have shown S–R effects using words, some of which are, in fact, of similar size to the present effects using pictures (e.g., Horner & Henson, 2009; Logan, 1990; Race et al., 2009).
The four-way interaction also reached significance, F(1,62) = 4.34, p < .05, but since the Format-match and Prime-level factors were arbitrary for Novel items (see the Method section), this effect is likely to be a type I error (especially given the number of ANOVA effects tested) and was not considered further. Note that this four-way interaction is not seen in the analysis of Novel RT data below and, therefore, is unlikely to have affected our RT priming analyses.
We thank Jeff Bowers, David Huber, and an anonymous reviewer for their helpful comments. This work was funded by the U.K. Medical Research Council (MC_US_A060_0046).
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.