What factors influence the effectiveness of a cue to serve as a discriminative stimulus in a behavior chain? This question takes on more importance when one considers that behavior chains in nonhumans are often equivalent to motor skills in humans. The development of motor skills is of central concern in a wide array of training programs in education, the military, sports, and the arts. Skill learning has recently received extensive attention in neuroscience, in which the central question has been to identify the neuronal mechanisms that underlie the ability to sequence a series of motor acts (e.g., Averbeck, Sohn, & Lee, 2006; Calabresi & Di Filippo, 2010; Flagel, Clark, Robinson, Mayo, Czuj, Willuhn and Akil 2011; Hazeltine & Ivry, 2002; Jin & Costa, 2010; Jog, Kubota, Connolly, Hillegaart, & Graybiel, 1999; Ohbayashi, Ohki, & Miyashita, 2003). Such behavioral sequences represent the integration of environmental cues and responses to produce adaptive patterns of behavior that are useful to nonhumans for foraging, avoiding predators, and social interactions, and are useful to humans in everyday skills involving goal-directed behaviors such as driving a car or repairing the car. In the research reported here, we focused on the rules of integration of these cues and responding.

A central principle of Pavlovian and operant conditioning and of behavioral ecology is that organisms are sensitive to environmental cues that predict important events and to behavioral consequences such as reward (McLinn & Stephens, 2006). Organisms are remarkably good at detecting statistical regularities in environmental events, by detecting the temporal and spatial patterns of environmental stimuli (e.g., Fast & Blaisdell, 2011; Fountain, 2008; Fountain & Doyle, 2011; Shettleworth, 2010). It is particularly interesting when organisms appear insensitive to cues that (at face value) should be highly predictive of important events such as reward. Blocking and overshadowing have received much attention, for this reason. We provided another striking example of such insensitivity in Reid, Nill, and Getz (2010), in which we trained rats to complete a sequence of two lever presses guided by panel lights. The experiments that we report below were designed to gain further understanding of this insensitivity to highly predictive cues, so we will describe the earlier study in some detail.

At least two sources of stimulus control are involved in guided skill learning: environmental events (from instructors or panel lights) that aid response selection or timing, and “practice cues” that result from the subject’s own behavior of repeating the same response pattern (Lattal, 1975; Shimp, 1981, 1982). As the skill is acquired, reliance on (or control by) environmental guiding cues decreases until the behavior pattern can be performed even in the absence of those guiding cues—with the skill now being controlled by newly developed practice cues.

Reid et al. (2010) measured these changes in stimulus control during guided skill learning in rats in two experiments. In both experiments, food pellets were provided for the completion of a left–right (L–R) lever-press sequence guided by the illumination of panel lights over the respective levers (lights condition). That is, at the beginning of a trial the light over the left lever was illuminated, whereas the light over the right lever was off. A press of either lever caused the panel light over the left lever to turn off and the light over the right lever to turn on. A second press on either lever caused the trial to end and turned off the panel lights. The L–R sequence produced food delivery, but all other response sequences produced time-outs. Rats were exposed to this lights condition until L–R sequence accuracy reached an asymptote above 80 %. In Experiment 1, rats were shifted from a lights condition to a no-lights condition, which simply eliminated the lights as guiding cues and measured the rats’ ability to complete the L–R sequence without the lights. Sequence accuracy dropped about 50 %, but it remained well above chance, demonstrating an ability, albeit reduced, to complete the behavior chain without the lights serving as cues. Yet, in Experiment 2, when we reversed the order of the panel lights in the second condition rather than eliminating the lights (without changing the response–reinforcement contingency), all rats “followed” the lights by pressing the levers in the incorrect R–L order until responding extinguished. The rats were not able to continue the same L–R sequence that they had produced with such accuracy in the previous lights condition, even though they did when the lights were removed. We described this observation as an operant overshadowing effect (Pearce & Hall, 1978), in which control by the guiding lights appeared to overshadow all cues resulting from performing the L–R sequence hundreds of times. However, we gave no explanation of why this overshadowing would occur. In the experiments reported below, we adopted the same basic procedures to identify what processes were responsible for the insensitivity to cues that should have been perfect predictors of reinforcement. The goal of each experiment was to gain a better understanding of the interactions between environmental guiding cues and developing practice cues as a simple behavioral skill is acquired and maintained.

The research literature has reported two explanations of bias, or insensitivity to cues, in stimulus control or response selection that could be responsible for this overshadowing effect. The first, feature-positive discrimination bias (or the feature-positive effect), is the greater ability to notice the presence or addition of a stimulus in an environment as opposed to noticing the absence or removal of the stimulus (Hearst, 1991; Lotz, Uengoer, Koenig, Pearce & Lachnit 2012; Sainsbury, 1973). Jenkins and Sainsbury (1969, 1970) called an arrangement “feature-positive” if the presence of a distinctive feature signals a reward. A “feature-negative” arrangement is one in which the absence of the feature signals a reward. In Reid et al. (2010), the lights condition could be considered a feature-positive condition because two panel lamps were always present (the common element), but reward occurred only for pressing the lever that had its lamp illuminated (the feature: AB+/B–), which served as the guiding cue for the L–R sequence. Conversely, the reversed-lights condition could be considered a feature-negative condition, because two panel lamps were present (the common element), but reward occurred only for pressing the lever with its panel light turned off (the negative feature: AB–/B+). Because the presence of a feature is often more salient or noticeable than its absence (Beckmann & Young, 2007; Hearst, 1991; Sainsbury, 1971), one might expect feature-positive cues to overshadow weaker cues that are also present, such as the early practice cues generated by properties of the subjects’ own behavior when repeating the same response sequence. However, weaker feature-negative cues would be less likely to overshadow practice cues.

An alternative explanation, often called the spatial S–R compatibility effect, extensively studied in humans by human-factors researchers, focuses on the agreement in spatial location between the guiding cue and the required response (Fitts & Seeger, 1953; Hommel, 1995, 2011; Hommel & Prinz, 1997; Kiernan, Ray, & Welsh, 2012; Proctor & Reeve, 1990). Recall that in each trial of our lights and reversed-lights conditions, one panel light was illuminated and the other was extinguished, and their spatial locations alternated with each response. Spatial S–R compatibility predicts that responding to the two levers would be faster and more accurate when the light to which the response was required (our lights condition) was illuminated, rather than when the light over the other lever was illuminated (reversed-lights condition). Although spatial S–R compatibility has been studied mostly in humans, the speed and accuracy with which some nonhumans respond to a relevant cue are affected by the spatial location of that cue, even when the cue’s location is irrelevant. The Simon effect (Simon, 1969; Simon & Rudell, 1967), a more complex version of spatial S–R compatibility in which stimulus position is completely irrelevant for the task, has been demonstrated with humans (e.g., Hommel, 2011; Kiernan et al., 2012; Proctor & Reeve, 1990), pigeons (Urcuioli, Vu, & Proctor, 2005), and rats (Coutière, Hardouin, Burle, Vidal, & Hasbroucq, 2007). Therefore, it is likely that S–R compatibility would influence response selection during guided skill learning in rats.

Both explanations have been used to explain the reduced stimulus control or failure to respond appropriately in situations requiring a single response on one of two available operanda. In the experiments reported below, we explored the adequacy of these explanations as rats acquired the ability to complete a two-response behavior chain when panel lights were manipulated over both levers.

Experiment 1a: lights → reversed-lights

Experiment 1b: reversed-lights → lights

In Reid et al. (2010), the lights → reversed-lights transition produced an unexpected inability to maintain the same L–R sequence that had been practiced hundreds of times with high accuracy. In the present study, we wanted to know whether reversing the order of exposure to these two conditions would produce the same decrease in sequence accuracy. Therefore, in Experiment 1a we replicated Reid et al. by exposing naive rats to a lights → reversed-lights transition, and then compared the results to those from different rats exposed to a reversed-lights → lights transition in Experiment 1b.

We predicted that reversing the order of exposure of these two conditions to naive rats would produce markedly different results. Feature-positive discrimination bias and spatial S–R compatibility both make the same prediction here, but for different reasons. Because the presence of a feature (a panel light) is often more salient or noticeable than its absence (Hearst, 1991), the feature-positive effect predicts that training on the reversed-lights condition would produce slower acquisition of the L–R sequence, perhaps influencing the strength of the practice cues generated by properties of the subjects’ own behavior when repeating the same response sequence. Weaker feature-negative environmental guiding cues should be less likely to overshadow practice cues. Similarly, S–R compatibility predicts that activating the lights over the “incorrect” levers would decrease accuracy and delay acquisition, whereas activating the lights over the “correct” levers would promote high accuracy. Therefore, transition to the lights condition should not disrupt accuracy so strongly. Therefore, Experiment 1b allowed naive rats to learn the L–R lever-press sequence in the reversed-lights condition until accuracy was high and stable, and then exposed them to the lights condition to measure the amount of change in L–R accuracy.

Method

Subjects

Seven naive Long Evans female rats (Rattus norvegicus) about four months old at the beginning of training were used in Experiment 1a. An additional seven rats (five Long Evans and two older Wistar female rats approximately ten months old) were used in Experiment 1b. All of the rats were housed in individual polycarbonate cages in a facility maintaining constant temperature, humidity, and a 12:12 light:dark cycle. Although the rats were naive to this apparatus and levers, the older rats had had previous experience in a laboratory project (rat basketball; www.wofford.edu/psychology/content.aspx?id=4844) for approximately three months, which involved daily handling by various students, exposure to relatively open areas in a noisy environment, and reinforcement with the same 45-mg Research Diet (Formula A/1) pellets used in this study. Each subject was maintained at approximately 85 % of its free-feeding body weight by providing food (Teklad Rodent Diet) after each session in home cages. Water was freely available in the home cages.

Apparatus

For the experiments, we utilized four standard Med Associates modular test chambers for rats, measuring 30 × 24 × 22 cm. Each chamber was located inside an isolation chamber containing a ventilation fan, a 7-W, 120-V nightlight, and a miniature TV camera on the ceiling. A sound generator produced constant white noise at approximately 65 dB. Each operant chamber contained two retractable levers on the front wall and two nonretractable levers on the rear wall. Each pair of levers was separated by 16.5 cm, center to center, and located 6 cm above the floor. The magazine hopper, 5 × 5 cm, was centered between the two response levers on the front wall, 3 cm above the floor. One round 28-V white stimulus lamp, 2.5 cm in diameter, was located 2.5 cm above each of the four levers, and a 28-V houselight (GE1819) was located at the center top of the rear wall. The pellet dispenser dispensed 45-mg Research Diet (Formula A/1) pellets. All four operant chambers were controlled by a single Dell personal computer (Pentium 4) located in an adjacent room and programmed in MED-PC IV, which controlled all of the experimental conditions and recorded every event and its time of occurrence with 10-ms resolution.

Procedure

The onset and offset of the panel lights were exactly reversed for Experiments 1a and 1b during all training and experimental conditions. In Experiment 1b, we reversed the order of presentation of the panel lights, but pellet delivery remained contingent on the same L–R response sequence. For simplicity, we will describe only Experiment 1a.

Training

The training procedures, MED-PC computer programs, and stability criteria were the same as those used in Reid et al. (2010), except that the panel lights were presented in reversed order for Experiment 1b. Subjects were trained to press the right lever on the front wall (adjacent to the hopper) using a successive approximations procedure. They were exposed to a fixed ratio 1 (FR-1) schedule for pressing the front-right lever while the panel light over that lever was turned on to indicate S+, and the panel light over the (ineffective) left lever was off, indicating S–. Training continued until the subjects had earned 45 pellets of food for two consecutive sessions. Subsequently, each subject was exposed to a single 45-min session of FR-1 for pressing the right lever on the rear wall, while the levers on the front wall were retracted. As with the shaping procedure, the light over the right-rear lever was turned on to indicate S+, and the light over the left lever was off to indicate S–. This session lasted for 45 min or until subjects had received 45 food pellets. The purpose of this session was to ensure that all subjects were given the same amount of exposure to the reinforcement conditions on the right-rear lever before the experiment proper began, given that subjects required various amounts of original lever-press training.

Experimental conditions

In all subsequent conditions, the two front levers remained retracted and only the left and right levers on the rear wall were used. Trials were signaled by the presence of the houselight, whether the preceding trial ended with food delivery or a time-out. The two experimental conditions in this study (reversed lights and lights) were identical to those in Reid et al. (2010), but the order of exposure was reversed for Experiment 1b. The response–reinforcement contingencies in both conditions were identical: Completion of a left (L)–right (R) lever-press sequence resulted in pellet delivery. Pellet deliveries were accompanied by a 0.1-s tone and brief pulse (off) of the houselight. All other response sequences resulted in 10-s time-out during which the panel lights and houselight were off (the nightlight in the isolation chamber continued to provide general illumination) and lever presses had no programmed consequences. There was no intertrial interval. Sessions lasted for 45 min or until 150 trials were completed, whichever came first.

The subjects in Experiment 1a were first exposed to the lights condition, whereas the subjects in Experiment 1b were first exposed to the reversed-lights condition, until L–R lever-press accuracy appeared to asymptote, remaining at 80 % or greater accuracy for five sessions with no increasing or decreasing trend. At the beginning of each trial in the lights condition, the panel light above the left lever was on, while the panel light above the right lever was off. A press to either lever caused the left panel lamp to turn off and the right lamp to be illuminated. A second lever press ended the trial, turning off both panel lamps and delivering a food pellet or beginning the 10-s time-out. Only L–R lever-press sequences produced food. No feedback about response accuracy was provided until two lever presses had been completed. The onset and offset of the panel lights were exactly reversed for the reversed-lights condition.

Once each subject had reached our stability criteria on the training condition (a multiple baseline design across subjects), it was exposed to the alternative condition for five sessions. The purpose of this transition was to reverse the order of presentation of the two light cues without altering the required lever-press sequence.

Results and discussion

The left panel of Fig. 1 shows the changes in percentages of L–R accuracy when the subjects in Experiment 1a were exposed to the lights → reversed-lights transition, replicating the procedure and results of Reid et al. (2010). The right panel shows the results of the reversed-lights → lights transition of Experiment 1b. Rats in Experiment 1b required approximately twice the number of sessions to learn the L–R sequence and to reach our stability criteria under the reversed-lights condition (Exp. 1b: M = 26.4 sessions) than it took for subjects under the lights condition (Exp. 1a: M = 12.7).

Fig. 1
figure 1

Experiment 1: In a replication of Reid et al.’s (2010) results, the left panel shows a persistent decrease in sequence accuracy when the order of the lights over two levers was reversed. The right panel shows the effects of reversing the order of exposure to these two conditions in Experiment 1b. The vertical lines represent the switch from one condition to another. The dashed horizontal lines represent the accuracy predicted if levers were selected at random. Error bars represent SEMs

Full size image

Following Reid et al. (2010), we focused on the immediate decrease in accuracy (from the stable baseline) when cue conditions were changed, before subjects eventually adapted to the new guiding-cue conditions. In Experiment 1a (left panel), the mean decrease in accuracy from the last session of the lights condition to the first session of the reversed-lights condition was a 98.8 % drop in sequence accuracy. This decrease corresponds well with the 91.5 % drop in accuracy observed by Reid et al. This large decrease in accuracy contrasts strongly with the small, 6.8 % decrease observed in Experiment 1b as a result of the reversed-lights → lights transition. Even though the decrease was relatively small, a 2 (condition) × 5 (session) repeated measures analysis of variance (ANOVA) comparing the last five sessions of the reversed-lights condition with the first five sessions of the lights condition demonstrated that the decrease approached significance, F(1, 12) = 4.25, p = .062, but that effect was qualified by a significant Condition × Session interaction, F(1, 12) = 6.09, p = .030. A one-tailed, one-sample t test comparing the mean accuracy in the last five sessions of the reversed-lights condition with accuracy in the first session of the lights condition demonstrated that the expected drop during that first session was not significant, t(6) = −1.66, p = .148.

These results identify an extraordinary asymmetry: Lights → reversed-lights produced very different effects than did reversed-lights → lights. We identified two potential explanations for this asymmetry and carried out experiments designed to test them. First, the high accuracy of our second condition (lights) might have been the result of an increased number of sessions required to reach stability. Acquisition took twice as long in our reversed-lights condition as in the lights condition, as is depicted in Fig. 1. The increased exposure to the reversed-lights condition may have increased stimulus control by the practice cues resulting from repeating the same L–R sequence for many sessions. Thus, the small drop in accuracy when the cues were reversed may have occurred because L–R response selection was accurately controlled by these strong practice cues, and the lights themselves could be ignored as redundant cues.

An alternative explanation for this asymmetry is that the lights condition might have provided a new, more salient guiding cue (light onset over the appropriate lever) that made it easier to select the correct lever in the correct order. In order to evaluate these explanations, we conducted the following experiment, which controlled for the duration of training and eliminated the information provided by the cues.

Experiment 2: reversed-lights → both-lights → reversed-lights

One potential explanation why accuracy did not decrease much in the reversed-lights → lights transition is that increased exposure to the first condition may have increased stimulus control by the practice cues resulting from repeating the same L–R sequence for many sessions. Thus, response selection may have been accurately controlled by these strong practice cues, and the lights themselves could be ignored as redundant cues in the second condition. In the present experiment, we tested this explanation by exposing rats to the same reversed-lights condition for equal numbers of sessions (or longer), and then exposing them to a condition under which the lights were removed as guiding cues. If extended exposure to the reversed-lights condition produced such strong practice cues, then L–R accuracy should be reasonably maintained when the reversed light cues were eliminated.

The two panel lights could be arranged in two ways so as to eliminate their information value as guiding cues: Both lights could remain on or off during every trial in the second condition. We wanted our approach to be consistent with the approach we had used in Experiment 1 of Reid et al. (2010), because this would allow us to compare our present results with those for consistency. In the earlier lights → no-lights transition, illuminated lamps were originally S+, and lights off represented S–. Therefore, both lamps were originally S– in the no-lights condition. Using the same policy in the present study would produce a reversed-lights → both-lights transition, because both illuminated lamps would originally represent S–. We implemented this transition in an ABA design.

Method

Subjects

Seven naive four-month-old Long Evans female rats were housed and maintained at 85 % free-feeding body weight, as in Experiment 1.

Apparatus

The apparatus was the same as in Experiment 1.

Procedure

The training procedure and the reversed-lights condition were the same as in Experiment 1b.The new both-lights condition was identical to the no-lights condition of Reid et al. (2010), except that both panel lights remained illuminated within every trial so they could not signal which lever to press. Both panel lights were off during pellet delivery and time-outs until the next trial began. All subjects were exposed to the reversed-lights condition until the accuracy/stability criteria were met, followed by seven sessions exposed to the both-lights condition, followed by recovery of the baseline reversed-lights condition for five sessions. Food delivery remained contingent on completion of the same L–R lever-press sequence in all conditions.

Results and discussion

The subjects reached our accuracy/stability criteria in 28.2 sessions (range: 22–39), slightly more than the 26.4 sessions that had been required in Experiment 1b. Therefore, these subjects had exposure to the reversed-lights condition that was equivalent to that in Experiment 1b. As a result, the reversed-lights condition should have produced equivalent stimulus control by the practice cues resulting from repeating the same L–R sequence for many sessions. If these strong practice cues were responsible for response selection, as we proposed in Experiment 1b, we would expect sequence accuracy to be equally high when the guiding light cues were removed and when they were reversed. Figure 2 shows the changes in accuracy across the three conditions.

Fig. 2
figure 2

Experiment 2: Decrease in L–R lever-press accuracy when subjects were trained on the reversed-lights condition and shifted to the both-lights condition, which prevented lights from signaling which lever to select. The upper dashed line represents the average accuracy during the reversed-lights baseline. The lower dashed line represents the expected accuracy from selecting levers randomly. Error bars represent SEMs

Full size image

Accuracy levels dropped 19.6 % during the first session of the both-lights condition, and they remained low for the seven days of that condition (M = 57.1 %). Once the reversed-lights condition was reinstated, accuracy immediately recovered, showing strong stimulus control by the reversed-lights condition. We assessed the decrease in accuracy during the both-lights cues by carrying out a 2 (condition) × 5 (sessions) repeated measures ANOVA and found the decrease in accuracy to be statistically significant, F(1, 12) = 10.53, p = .007, with no effect of session nor an interaction. A similar analysis of the recovery of the reversed-lights condition was also significant, F(1, 12) = 14.69, p = .002, with no effect of session nor an interaction.

When discussing Experiment 1b, we argued that if extended exposure to the reversed-lights condition had produced such strong practice cues as to maintain high accuracy when subjects were shifted to the lights condition (right panel of Fig. 1), then high L–R accuracy should be maintained in the present study when the informative cues were eliminated in the both-lights condition. However, we observed a moderate, stable drop in accuracy, implying that the practice cues were not sufficient to maintain the same high level of accuracy, even though subjects received an equivalent amount (or more) of training on the reversed-lights condition. Therefore, the high level of accuracy in Experiment 1b must not have been due solely to practice effects: The high accuracy during the lights condition in that experiment must have also depended on particular properties of those light-on cues (salience, location, or both), not only on the stronger practice cues.

Although quantitative comparisons across studies are difficult, it would be useful to compare the present decrease in accuracy with that obtained in the similar study in Reid et al. (2010), to see whether the results were relatively consistent. The previous study’s lights → no-lights transition produced a greater percentage drop in L–R accuracy (46.4 %) than was produced by the present reversed-lights → both-lights transition (19.6 %). This difference is consistent with expected differences in the strengths of the practice cues: The reversed-lights condition provided about twice the number of training sessions as did the lights condition, giving more time to develop stronger practice cues that could aid response selection when the information from the light cues was eliminated.

We concluded from this study that the extended exposure to the reversed-lights condition created stronger practice cues than were obtained during the easier lights condition. The high accuracy during the lights condition of Experiment 1b was partially due to these stronger practice cues, but they acted in combination with properties of those light-on guiding cues that made them especially effective at controlling response selection. This conclusion is compatible with the predictions based on feature-positive discrimination bias and spatial S–R compatibility that the lights condition should be especially effective at guiding cues. In the experiments that follow, we measured the relative effectiveness of these guiding cue conditions.

Experiment 3: comparing the effectiveness of guiding cues

Experiment 2 demonstrated clear stimulus control of responding in the reversed-lights condition, especially when this condition was reinstated. The decrease in accuracy during the both-lights condition could have been due to a loss of this stimulus control, or to a change in context produced by having two panel lights illuminated simultaneously instead of one. Thus, it is not yet clear from the experiments above whether the reversed lights actually served as guiding cues at all in the behavior chains.

Both feature-positive discrimination bias and spatial S–R compatibility predict that training on the lights condition should produce faster acquisition of the L–R sequence than would the reversed-lights condition, as we observed in Experiment 1. However, feature-positive bias and classic information theory predict that the cues in the reversed-lights condition should still promote faster acquisition than would conditions in which lights provide no information to aid response selection. The predictions from spatial S–R compatibility are not as straightforward, and may depend on the characteristics of the both-lights and no-lights conditions. Responding to the two levers should be faster and more accurate when the light is illuminated to which the response is required, rather than when the light is illuminated over the other lever. S–R compatibility may predict faster and more accurate responding in the both-lights condition than in the no-lights condition because lights are illuminated over the “correct” lever for both responses in each trial, as compared to the no-lights condition, which provides no light for orientation or spatial coding over either lever. However, if the animals are attracted to both locations simultaneously, that attraction might generate interference and slower responding. Such interference would not be predicted in the no-lights condition.

The present experiment was designed to test these predictions by measuring these differences between four conditions. We allowed four groups of rats to acquire the L–R lever-press sequence for 20 sessions under lights, reversed-lights, both-lights, or no-lights conditions, and measured the differences in the rates of sequence acquisition. We predicted that the rates of acquisition would be lights >> reversed lights > no lights = both lights.

Method

Subjects

Sixteen naive four-month-old Long Evans female rats were housed and maintained at 85 % of their free-feeding body weight.

Apparatus

The apparatus was the same as in Experiments 1 and 2.

Procedure

The subjects were randomly assigned to one of four groups, each corresponding to the four conditions described in the previous experiments: lights, reversed-lights, both-lights, and no-lights conditions. Lever-press training occurred as in the previous studies, with lights on or off over the levers, depending on the condition. Once the experiment began, each group was exposed to their respective condition for 20 sessions. Food delivery remained contingent on completion of the same L–R lever-press sequence for all groups.

Results and discussion

The two panels of Fig. 3 show two methods of assessing the effects of stimulus condition on acquisition of the L–R response sequence. The top panel displays the mean numbers of pellets earned per session (with a maximum of 150 trials) for each group. A 4 (groups) × 20 (sessions) mixed ANOVA compared the learning rates for all rats in each of the four groups across the 20 sessions. It showed a significant main effect of session, F(19, 228) = 26.144, p < .001, η p 2 = .685, indicating that accuracy improved across sessions. It also showed a significant main effect of group, F(3, 12) = 6.152, p = .009, η p 2 = .606. Most importantly, it revealed a significant Group × Session interaction, F(57, 228) = 2.54, p < .001, η p 2 = .388, indicating that some groups learned more rapidly than other groups, without identifying which particular groups were different. To discover which groups were responsible for the significant interaction, in paired t tests we examined the distance between pairs of curves. For example, Fig. 3 shows that the lights group produced more pellets per session (higher curves) than did the reversed-lights group. The paired t test worked by subtracting the number of pellets per session for the reversed-lights group (represented by a point with the filled triangles) from the corresponding value for the lights group (i.e., the difference between the two corresponding points) for each session independently, and determining whether the sum of these differences was statistically significant. The paired t test demonstrated that rats exposed to the lights condition learned the L–R response sequence significantly faster than did the reversed-lights group, t(19) = 7.20, p < .001. Similarly, the reversed-lights group learned the sequence significantly faster than did the no-lights group, t(19) = 5.95, p < .001, and the both-lights group, t(19) = 6.39, p < .001. The no-lights group did not differ from the both-lights group, t(19) = 0.455, p = .327.

Fig. 3
figure 3

Experiment 3: Two methods of assessing the effects of stimulus condition on acquisition of the L–R response sequence. Each curve represents the mean of all subjects exposed to a stimulus condition. The top panel displays the number of pellets earned per session for each group. Error bars represent SEMs. The bottom panel depicts the learning rate for each group as a function of cumulative pellets versus time-outs. Greater slopes represent faster learning: that is, more pellets and fewer time-outs. Longer curves represent more trials completed in 20 sessions than do shorter curves

Full size image

Every trial had to end in either pellet delivery or time-out. Therefore, the bottom panel of Fig. 3 shows the dynamics in learning rates for the four groups by examining the changing distributions of pellets and time-outs across the 20 sessions. The ordinate represents the mean cumulative number of pellets, representing correct trials that resulted in pellet delivery. The abscissa represents the mean cumulative number of time-outs, representing incorrect trials that resulted in a time-out. Each curve represents the mean of all four rats in the group, and each point represents one session (thus, 20 points for each curve). The greater the slope of a curve, the faster the acquisition of the L–R sequence. Longer curves represent more trials completed in 20 sessions than do shorter curves.

Interestingly, the both-lights group appeared to produce slower and less responding (approximately 30 % fewer trials per session; bottom panel of Fig. 3; M = 60.9, SD = 35.8) than did the no-lights group (M = 89.4, SD = 41.0). However, within-group variability was high, and with only four subjects per group, the difference was not significant, t(6) = 1.05, p = .167. The average final L–R accuracies (and thus, the obtained probabilities of reinforcement per trial) of the last two sessions were the same for the subjects in each group (both-lights, M = 52.5 %; no-lights, M = 52.6 %).

The results of this study confirmed that the reversed-lights condition actually did facilitate response sequence learning: The reversed lights were beneficial guiding cues. The effectiveness of these guiding cues was weak relative to the lights condition, as was predicted by both hypotheses above. The obtained rates of acquisition were lights >> reversed lights > no lights = both lights. These results confirmed our predictions that the guiding cues provided by the lights condition would be more effective than those provided by the reversed-lights condition, but the latter cues would still be better than no cues at all when learning a behavior chain.

Experiment 4: comparing the front and rear walls

The experiments above provide compelling evidence that the relative locations of the guiding lights have a strong influence on response selection in behavior chains. This influence is particularly evident in the asymmetrical effects of lights → reversed-lights (Fig. 1, left panel) versus reversed-lights → lights (Fig. 1, right panel) transitions. However, a colleague (G. Bachá-Méndez, personal communication, November 2011) described an ongoing pilot study similar to Experiment 1b that seemed to produce effects opposite from ours. Two rats in that study’s reversed-lights → lights transition produced strong decreases in accuracy, whereas our study produced a minor decrease in accuracy. As we worked to identify differences in the methods between the two studies, one difference seemed important: All of our experiments utilized the two levers on the rear wall, distant from the food hopper, whereas their pilot study utilized the two levers on the front wall, just to the left and right of the hopper. Possibly, the position of the levers might produce different stereotyped movement patterns in the apparatus that would contribute to practice cues. Therefore, we decided to replicate the lights → reversed-lights transition and the reversed-lights → lights transition on both the front and rear walls of our apparatus, to see whether the front and back walls affected accuracy differently during transitions.

Extending the procedure to an ABA design allowed us to ask an important question about the effects of these transitions between conditions: Are the changes in sequence accuracy during a transition determined exclusively by the current transition type, or does exposure to earlier conditions also influence these changes? For example, would the strong decrease observed in a lights → reversed-lights transition (Fig. 1, left panel) be observed if the subjects were first exposed to a reversed-lights condition in an ABA design (reversed-lights → lights → reversed-lights)? This question concerns the general issue of path independence versus dependence in the conditioning literature (e.g., Miller, Barnet, & Grahame, 1995): Are the characteristics of acquisition (e.g., the effects of guiding cues) independent of the subject’s earlier behavioral history (the path), or do they depend on prior learning history?

This study consisted of four independent groups of four naive rats, each of which was exposed to an ABA design. As in the previous studies, food delivery remained contingent on completion of the same L–R lever-press sequence in all conditions. Two groups utilized levers on the front wall, and two utilized levers on the rear wall. The order of conditions was also manipulated: lights → reversed-lights → lights (abbreviated LRL) and reversed-lights → lights → reversed-lights (abbreviated RLR). We labeled the four independent groups as LRL–front, LRL–rear, RLR–front, and RLR–rear. The purpose was to determine whether the changes in sequence accuracy during a transition between conditions would be affected by the location of the operative levers and corresponding panel lights in the apparatus, and whether exposure to an earlier condition would affect these changes.

Method

Subjects

Sixteen naive four-month-old Long Evans female rats were housed and maintained at 85 % of free-feeding body weight.

Apparatus

The apparatus was the same as in the earlier experiments.

Procedure

Subjects were randomly assigned to one of the four groups described above: LRL–front, LRL–rear, RLR–front, and RLR–rear. Lever-press training occurred as in the previous studies, with lights on or off over the lever, depending on the first condition of the ABA design. All procedures for the lights and reversed-lights conditions were identical to those in the studies above, including the accuracy/stability criteria. All subjects in each group were exposed to the second and third conditions of the ABA design for five sessions each. Food delivery remained contingent on completion of the same L–R lever-press sequence in all conditions.

Results and discussion

Figure 4 compares the rates of acquiring the L–R response sequence in the first condition for the four groups. The two groups of subjects in the lights condition (filled symbols) reached the minimum 80 % accuracy criterion in fewer sessions than did the groups exposed to the reversed-lights condition (unfilled symbols), confirming our results from Experiment 3 (see Fig. 3). The rates of learning in the lights condition were approximately the same for the front and rear walls. Stimulus condition was a more important determinant of acquisition rate than was the wall containing the operative levers. However, the wall did appear important for the two groups in the reversed-lights condition, because RLR–front subjects reached the 80 % criterion faster than the RLR–rear subjects did (about 15 vs. 23 sessions). This difference implies that RLR–front subjects would have less opportunity to develop strong practice cues than would the RLR–rear subjects.

Fig. 4
figure 4

Experiment 4: Rates of acquiring the L–R response sequence in the four groups. Filled symbols represent sessions of the lights condition, and open symbols represent sessions of the reversed-lights condition. The dashed horizontal line at 80 % represents the required accuracy criterion

Full size image

The top two panels of Fig. 5 depict the influence of front versus rear wall on the drop in L–R accuracy during the lights → reversed-lights transitions. This large drop replicated our results depicted in Fig. 1 (left panel), even though the two groups of subjects pressed levers on different walls. However, wall location did seem to affect acquisition during the reversed-lights condition and during reinstatement of the lights condition. Subjects pressing levers on the rear wall (top right panel) showed no acquisition during the reversed-lights condition, and accuracy immediately recovered when the lights condition was reinstated, replicating the strong stimulus control obtained in Experiment 1b (Fig. 1). But accuracy did improve for the subjects pressing levers on the front wall, systematically increasing over the five sessions. When switched back to the original lights condition, accuracy began low and increased systematically over the condition, indicating that the subjects had to relearn the stimulus–response contingencies of the reinstated lights condition.

Fig. 5
figure 5

Experiment 4: Changes in L–R lever-press accuracy for all four groups when subjects responding on the front or rear wall were shifted to the alternative condition and back. Error bars represent SEMs

Full size image

The bottom panels of Fig. 5 show the effect of wall position on the changes in accuracy for the two reversed-lights → lights → reversed-lights groups. The decrease in L–R accuracy when the lights condition was imposed was small as compared to those in the two LRL groups, depicted in the top panels, generally replicating the results of Experiment 1b (Fig. 1, right panel). Recall from Fig. 4 that RLR–front subjects had fewer sessions to develop strong practice cues than did the RLR–rear subjects. Therefore, we should expect a somewhat greater drop in accuracy for the RLR–front group. Accuracy during this condition did improve over the five sessions, just as it had with the LRL–front group.

Our second question was whether the changes in sequence accuracy during a transition are determined exclusively by characteristics of the current transition, or do previous conditions influence these changes? The answer was clear: Previous conditions strongly influence these changes. Lights → reversed-lights transitions produced strong decreases in accuracy when those two conditions occurred first in the experiments (see Fig. 1, left panel, and Fig. 5, LRL–front and LRL–rear), but accuracy actually improved when the transition represented a reinstatement of baseline conditions (see Fig. 5, RLR–front and RLR–rear). Similarly, reversed-lights → lights transitions produced small decreases in accuracy when they occurred first in the experiments (Fig. 1, right panel, and Fig. 5, RLR–front and RLR–rear), but accuracy actually improved when the transition represented a reinstatement of baseline conditions (Fig. 5, LRL–front and LRL–rear). Therefore, the prior behavioral history with guiding cues strongly affected the asymmetrical changes in accuracy during transitions between cue conditions.

In summary, the decreases in sequence accuracy in the RLR groups were small as compared to the very large decreases in the LRL groups, replicating the results of Experiments 1a and 1b. Therefore, the ways in which these two transition types influenced sequence accuracy were generally the same for the front and rear walls. Second, prior behavioral history with guiding cues strongly affected the asymmetrical results of transitions between cue conditions: The changes in sequence accuracy during a transition were not determined simply by the current transition type. Transitions were so strongly path-dependent that prior exposure completely reversed the effects of both transition types.

General discussion

It may seem appropriate to assume that organisms that forage for food would have evolved sensitivity to all of the cues that are predictive of food availability. However, scores of studies on blocking and overshadowing have demonstrated that not all predictive cues actually control behavior. A common assumption of most descriptions of behavior chains is that stimuli (especially response-produced stimuli) that accurately indicate which response leads to reinforcement will quickly come to control response selection. In the present experiments, either the illumination of a panel light or a dark bulb indicated perfectly which lever to press. Classical information theory would predict that lights on or off would have controlled behavior equally, since there was no difference in the information that they should have provided (Zentall & Stagner, 2012). But, as studies of feature-positive bias and spatial S–R compatibility have repeatedly demonstrated, discrimination and response selection are biased in favor of particular stimulus conditions.

The present experiments demonstrated and explored this bias in simple behavior chains. Following the demonstration of overshadowing in Reid et al. (2010), in Experiment 1 (Fig. 1) we identified a powerful asymmetry in the effectiveness of panel lights to control responding in lights → reversed-lights versus reversed-lights → lights transitions in behavior chains. During Experiment 2 (Fig. 2), we demonstrated that this difference was not simply due to increased control by the practice cues—something was unique about these reversed-light cues that led to slow acquisition and weaker stimulus control. Experiment 3 (Fig. 3) demonstrated that acquisition of the behavior chain was much faster in lights than in reversed-lights conditions, but reversed-lights conditions were still better than no external cues at all. Experiment 4 (Fig. 5) replicated our discovery of asymmetry (including the results of Reid et al., 2010) by showing that the asymmetry occurred whether the levers were located on the front or rear wall of the apparatus. Finally, in Experiment 4 (Fig. 5) we also demonstrated that the asymmetrical effects of stimuli on sequence accuracy during lights → reversed-lights and reversed-lights → lights transitions depended on more than those two conditions—their effects on accuracy depended strongly on the presence or absence of prior exposure to conditions.

We have explored two explanations for bias in stimulus control: feature-positive discrimination bias and spatial S–R compatibility. Both explanations were generally successful at accounting for the effects of isolated binary cue transitions with naive rats, without considering the influence of previous exposure on these transitions. Yet Fox, Reid, and Kyonka (2012) demonstrated that feature-positive discrimination bias is unable to account for similar transitions between stimulus conditions in pigeons. They replicated the lights → reversed-lights and reversed-lights → lights procedures with pigeons pecking red and white keys and observed the same asymmetry between transitions that we observed in Experiment 1. This replication is problematic for explanations based on feature-positive bias, because Fox et al. did not manipulate the presence and absence of stimuli—the red and white key lights were always feature-positive. Other studies have provided support for this explanation. At least two other studies have explored the effects of transitions between feature-positive and feature-negative conditions. Nallan, Miller, McCoy, Taylor, and Serwatka (1984) explored these transitions in pigeons, and Nallan, Brown, Edmonds, Gillham, Kowalewski and Miller (1981) in humans. Although their tasks did not involve behavior chains, both studies were consistent with the conclusion that feature-positive → feature-negative transitions are more disruptive than feature-negative → feature-positive transitions.

Isolated transitions aside, feature-positive bias and spatial S–R compatibility both failed to account for the powerful influence of exposure to previous conditions on subsequent transitions—that is, path dependence. In Experiment 4, we demonstrated that initial exposure to condition A in an ABA design dramatically reversed the effects of the subsequent B → A transition. Lights → reversed-lights transitions presented first or in isolation produced sequence accuracies below 10 %, often resulting in extinction. This same transition following exposure to the reversed-lights condition produced increases in sequence accuracy at 80 % or above. Similarly, reversed-lights → lights transitions presented first or in isolation produced only a small drop in sequence accuracy. Yet the same transition following exposure to the lights condition produced huge increases in sequence accuracy.

Path dependence has been a difficult problem to resolve in most, if not all, types of conditioning. Miller et al. (1995) described how the Rescorla–Wagner model and most contemporary models of classical conditioning predict path independence, even though an extensive literature has demonstrated path dependence, especially with a history combining reinforced and nonreinforced trials. Path dependence is not limited to conditioning studies: Economics and evolutionary biology have long developed models that incorporate the ways in which decisions and species are influenced by prior decisions, mutations, and selection pressures. The present experiments point out, once again, the need for further model development in behavior analysis that will incorporate path dependence.

Certain questions remain for future research in guided skill learning. One question involves the interpretation of these particular experiments. We observed considerable differences in the effects of lights and reversed-lights conditions, especially in acquisition rates. Different error rates in these conditions would produce differential exposure to an extinguished houselight during time-out. Does stimulus control by the houselight contribute to any of the effects that we have observed? Because the onset of the houselight signaled the beginning of each trial, might a greater reliance on the houselight to begin the response sequence influence the impact of a switch from the reversed-lights to the lights condition?

We believe that the most important question for future research into guided skill learning, in experiments such as these, is to define the interaction between environmental cues (i.e., panel lights) and practice cues that develop as the motor skill is repeated in many trials. Sufficient practice leads to autonomy, in which the skill can be executed in the absence of the instructor. But how? Cue interaction has been a central theme of associative-learning theories, but most of them predict that early learning of environmental cues would block learning of practice cues. Yet we have demonstrated that providing guiding cues contributes to stronger, not weaker, practice cues. The powerful methodology of behavior analysis has much to contribute to a more complete understanding of guided skill learning across species.