Operant (instrumental) conditioning is the process by which animals learn to perform actions that produce reinforcing outcomes. Its study in the laboratory is thought to provide an analogue to voluntary behavior. Thus, operant conditioning in animals has implications for understanding voluntary behaviors that impact human health, such as overeating, drug-taking, and smoking. Extinction procedures may have implications for treating such behavioral excesses (e.g., Bouton, 2014). In extinction, an operant behavior decreases when its reinforcer is removed. However, extinguished operant behavior can readily return and is especially dependent on the context (e.g., operant chambers in the laboratory that differ in tactile, visual, and olfactory properties) for its inhibition. An example of this is the renewal effect, the finding that extinguished responding recovers when the response is tested outside of the context in which it has been extinguished (Bouton, Todd, Vurbic, & Winterbauer, 2011; Crombag & Shaham, 2002; Nakajima, Urushihara, & Masaki, 2002). Most of the evidence suggests that operant extinction results in an inhibitory association between the context and response (Bouton, Trask, & Carranza-Jasso, 2016; Rescorla, 1993, 1997; Todd, 2013; see also Trask, Thrailkill, & Bouton, 2017). Removal from the context in which response inhibition is learned weakens its expression, thus causing a return of behavior.

Extinguished operant responding can also recover in a phenomenon known as resurgence. In a standard resurgence paradigm, a target response, R1, is reinforced and then extinguished. While R1 is extinguished, a newly available replacement response, R2, is reinforced. During a test, both responses are available and neither is reinforced. The typical result is that R1 behavior returns or resurges when reinforcement for R2 is removed (e.g., Leitenberg, Rawson, & Bath, 1970). One interpretation of this result is that resurgence is an ABC-like renewal effect (where extinguished responding recovers in a relatively novel context, C) in which the context is created by the presence or absence of alternative reinforcement (Trask, Schepers, & Bouton, 2015; Winterbauer & Bouton, 2010). In this interpretation, reinforcement for R1 constitutes Context A, reinforcement for R2 constitutes Context B, and the no reinforcement conditions during the test would be analogous to Context C. Resurgence occurs when reinforcers are presented contingently (Bouton & Trask, 2016) or noncontingently during Phase 2 (Trask & Bouton, 2016; Winterbauer & Bouton, 2010), suggesting that the mere presence of reinforcers is enough to create the reinforcement context.

Several factors that reduce resurgence have been identified. In general, higher rates of reinforcement during Phase 2 treatment produce more resurgence, and leaner rates of alternative reinforcement produce less (Bouton & Trask, 2016; Leitenberg, Rawson, & Mulick, 1975; Smith, Smith, Shahan, Madden & Twohig, 2017; Sweeney & Shahan, 2013). Additionally, thinning the rate of alternative reinforcement from high rates to lower rates over the treatment phase also weakens the effect (Sweeney & Shahan, 2013; Winterbauer & Bouton, 2012). Reverse thinning procedures in which alternative reinforcement rates gradually increase throughout the phase can also reduce resurgence relative to treatments that create a more consistent but equivalent overall average rate of reinforcement (Schepers & Bouton, 2015; see also Bouton & Schepers, 2014). Further, Schepers and Bouton (2015) demonstrated that alternating sessions of reinforcement and nonreinforcement for R2 during R1 extinction weakened the resurgence effect relative to animals that received reinforcement at the same average rate throughout R1 extinction (see also Trask, Keim, & Bouton, 2018). Together, the results are consistent with the idea that conditions that encourage generalization between Phase 2 and testing can reduce resurgence. That is, making the alternative reinforcement context (where reinforcement is typically available) more similar to the testing context (where reinforcement is typically not available) results in less resurgence (Trask et al., 2015).

The quality, rather than quantity, of alternative reinforcement can also be important in defining the reinforcement context. In one experiment (Bouton & Trask, 2016; Experiment 2), rats learned to perform an R1 lever-press response for a distinct food reinforcer, O1 (counterbalanced as sucrose pellets or grain-based pellets). In a second phase, R1 was extinguished while responding on a newly available R2 produced the other reinforcer, O2. During a testing phase, both responses were available but not reinforced. For one group, no reinforcers were delivered during the test; resurgence was expected. For a second group, O1 outcomes were delivered freely at the same rate as reinforcers had been earned in Phase 2. In a third group, O2 outcomes were delivered freely at the same rate as they had been earned in Phase 2. If alternative reinforcement creates a unique context in which learning takes place, then delivery of O2 (which created the theoretical reinforcement context in which R1 was extinguished), but not O1 (which was not present during R1 extinction), during testing should maintain the context of R1 extinction and reduce resurgence. During the test, rats that had either no reinforcers or free O1 reinforcers demonstrated the standard resurgence effect. Animals that had the free O2 reinforcers showed no increase in responding; the resurgence effect was completely abolished in this group. The results of the experiment demonstrate that reinforcement delivery itself is not sufficient to abolish resurgence. Instead, the reinforcer had to be specifically associated with the inhibition (extinction) of R1. A later experiment also found that O2 presentations during testing attenuated resurgence, whereas presentations of another reinforcer (O3) that was equally associated with R2, but not R1’s extinction, did not (Trask et al., 2018). Reinforcers associated with extinction can also weaken the ABA renewal effect (Trask & Bouton, 2016). These results are consistent with a long tradition that suggests reinforcers can have discriminative properties as well as reinforcing properties (e.g., Bouton, Rosengard, Achenbach, Peck, & Brooks, 1993; Ostlund & Balleine, 2007; Reid, 1958).

The current experiments were designed to extend these results. They asked whether the resurgence-attenuating effects of delivering O2 during relapse testing (Bouton & Trask, 2016; Trask & Bouton, 2016; Trask et al., 2018) can also be achieved by delivering a more neutral cue associated with the extinction phase. Visual or auditory cues presented during the course of extinction have been shown to be effective in attenuating renewal (Brooks & Bouton, 1994), spontaneous recovery (Brooks & Bouton, 1993; Brooks, 2000), and reinstatement (Brooks & Fava, 2017) of extinguished Pavlovian responding when they are presented during testing. Subsequent analysis determined that the cue was not a conditioned inhibitor (Brooks & Bouton, 1993, 1994). Instead, the cue likely worked to attenuate relapse by enhancing generalization between the extinction and testing phases, making it easier to retrieve extinction. Retrieval cues have likewise been effective in reducing some forms of operant relapse, including spontaneous recovery, reinstatement (Bernal-Gamboa, Gámez, & Nieto, 2017), and renewal (Nieto, Uengoer, & Bernal-Gamboa, 2017; Willcocks & McNally, 2014), though not rapid reacquisition (Willcocks & McNally, 2014). The failure to observe an effect of retrieval cue at slowing reacquisition might be consistent with the possibility that retrieval cues have less impact at reducing operant relapse in procedures in which the animal is earning reinforcers, as is also the case in the resurgence paradigm.

One potential reason that the reinforcing outcomes (O2) were so effective at reducing resurgence (Bouton & Trask, 2016) and renewal (Trask & Bouton, 2016) is that food pellets are motivationally significant and inherently attention-commanding. Visual or auditory stimuli presented in the background might not be. However, cues can acquire motivational or attentional significance through experience. For example, Mackintosh (1975) suggested that cues paired consistently with reinforcers attract attention as the animal learns the cue is a good predictor of the outcome. Although there is also evidence that attention can be directed toward poor predictors of reinforcers (see Kaye & Pearce, 1984; Hall & Pearce, 1979; Pearce & Hall, 1980), several lines of evidence support Mackintosh’s claim. For instance, previously relevant predictors are learned about more readily than previously irrelevant predictors (e.g., Mackintosh & Little, 1969; Roberts, Robbins, & Everitt, 1988). Further, in human predictive tasks, participants pay more attention (as measured by eye gaze) to stimuli that are good predictors than those that are not (e.g., Le Pelley, Mitchell, Beesley, George, & Wills, 2016).

The present experiments therefore asked whether cues associated with the alternative reinforcer during Phase 2 treatment can be used to attenuate response recovery (relapse) by presenting them during testing in the resurgence paradigm. Cues associated with alternative reinforcement during R1’s extinction might command sufficient attention during extinction and thus encourage generalization from extinction to the test.

Experiment 1

The goal of Experiments 1a–c was to test the effect of a cue associated with alternative reinforcement during Phase 2 and explore the conditions under which it might attenuate resurgence by potentially encouraging generalization of an inhibitory S-R association learned during extinction. The experimental designs are illustrated in Table 1. In Experiment 1a, all rats learned to perform an R1 response for an O1 outcome in Phase 1. In Phase 2, R1 responding was extinguished while a newly inserted R2 response produced a different reinforcer, O2. A 2-s stimulus was associated with every delivery of O2. During the test, R1 and R2 responding were both extinguished and examined under two conditions that were administered in a counterbalanced order. In the first condition, R2 still produced the 2-s cue. In the second, it did not. Resurgence was expected in the latter condition, but the hypothesis was that R2-contingent cue presentations would both attenuate R1 resurgence and elevate R2 compared to a session when R2 produced nothing. Experiment 1b then asked whether the cue needed to be associated with the reinforcer during Phase 2, and Experiment 1c asked whether the cue needed to be presented contingently on R2 responding during the final resurgence test.

Table 1 Design table for all experiments. R1 and R2 are separate levers in the chamber. O1 and O2 are counterbalanced grain- or sucrose-based reinforcers. Cues are 2-s presentations of key lights, tones, or a compound of both. A + indicates that the cue coincides with reinforcer delivery and a / indicates that it does not. See text for additional details
Full size table

Experiment 1a: Method

Subjects

The subjects were 16 female Wistar rats obtained from Charles River, Inc. (St. Constance, Quebec, Canada). They were approximately 85–95 days old at the start of the experiment and were individually housed in suspended stainless steel cages in a room maintained on a 16:8-h light:dark cycle. At the beginning of the experiment, all rats were food deprived to 80% of their free-feeding weight and maintained at that level throughout the experiment with a single feeding following each day’s session.

Apparatus

Conditioning proceeded in two sets of four standard conditioning boxes (Med-Associates Model Number: ENV-008-VP, St. Albans, VT, USA) that were housed in different rooms of the laboratory. The sets had been modified as described below for use as separate contexts, although they were not used in that capacity here. Boxes from both sets measured 30.5 cm × 24.1 × 21.0 cm (l × w × h), with side walls and ceilings made of clear acrylic plastic and front and rear walls made of brushed aluminum. Recessed 5.1 cm × 5.1 cm food cups with infrared photobeams positioned approximately 1.2 cm behind the plane of the wall and 1.2 cm above the bottom of the cup were centered in the front wall about 3 cm above the grid. In one set of four boxes, the floor was composed of stainless steel rods (0.5 cm in diameter) in a horizontal plane spaced 1.6 cm center to center, while in the other set of four boxes, the floor was composed of identical rods spaced 3.2 cm apart in two separate horizontal planes, one 0.6 cm lower than the other and horizontally offset by 1.6 cm. The boxes with the planar floor grid had a side wall with black panels (7.6 cm × 7.6 cm) placed in a diagonal arrangement, and there were diagonal stripes on both the ceiling and the back panel, all oriented in the same direction, 2.9 cm wide, and about 4 cm apart. The other boxes, with the staggered floor, were not adorned in any way. Retractable levers (1.9 cm when extended) were positioned approximately 3.2 cm to the right and to the left of the food cup and 6.4 cm above the grid. Both sets of boxes were housed in sound-attenuating chambers, and were continuously illuminated by two 7.5-W incandescent light bulbs mounted on the chamber ceiling. A 2-s 4,500-Hz, 65-dB tone was emitted from a sonalert module mounted directly above the magazine (Med-Associates Model Number: ENV-223HAM).

Food reinforcers consisted of 45-mg MLab Rodent Tablets (5-TUM: 181156; TestDiet, Richmond, IN) and a 45-mg sucrose pellet (5-TUT: 1811251; TestDiet). These were counterbalanced as O1 and O2. The apparatus was controlled by computer equipment located in an adjacent room.

Procedure

Twice-daily sessions were employed throughout every experiment. Each day’s first session began with approximately 15 h of illuminated colony time remaining. Each day’s second session began approximately 2.5 h later. Animals were placed into illuminated conditioning chambers, and the start of each session was indicated by the insertion of the lever(s) as appropriate. All sessions were 30 min in duration, and the end of the session was indicated by retraction of the lever(s).

Magazine training

All animals received magazine training on the day immediately prior to the beginning of Phase 1. At this time, they received two sessions with both levers retracted. During one session, rats received magazine training with their O1 reinforcer. During the other, the O2 reinforcer was delivered to the magazine. Sessions were counterbalanced so that half of the animals received training first with O1 then O2, and half received O2 then O1. On average, 60 food pellets were delivered during each session on a random time 30-s (RT 30-s) schedule of reinforcement.

R1 acquisition (Phase 1)

All animals then received 12 sessions of instrumental conditioning initiated by insertion of the left lever in half animals and the right lever in the other half. In all sessions, presses on the inserted lever (R1) delivered O1 pellets on a VI 30-s schedule of reinforcement. No additional response shaping was necessary.

R1 extinction and R2 acquisition (Phase 2)

All animals then received eight sessions in which R1 presses were extinguished (i.e., produced no reinforcers) and presses to the second lever (R2) were reinforced with the O2 reinforcer on a VI 30-s schedule. Each (R2) lever press activated both a 2-s tone and the delivery of O2 such that the tone and O2 were simultaneously presented. Both the left and the right levers were inserted throughout each session.

Resurgence test (Phase 3)

On the day following the conclusion of Phase 2, all rats received two 10-min test sessions in which both levers were inserted. R1 and R2 presses were recorded, but neither produced a food outcome. During one test, R2 presses produced only the 2-s tone on a VI 30-s schedule. No cues were presented in the other test. The test order was counterbalanced.

Data analysis

Analyses of variance (ANOVAs) were used to assess response rates throughout the experiment. The rejection criterion was p < .05. In Experiment 1a–c, outliers were screened according to the method recommended by Field (2005). If animals were significant outliers in either of the crucial tests (Z > 2) they were removed from all analyses. One animal was removed for being a significant outlier during the cue test (Z = 2.5).

Experiment 1a: Results

The results of Experiment 1a are displayed in Fig. 1. Animals increased their R1 responding in acquisition (Panel A). In Phase 2 (Panel B), R1 responding declined and R2 responding increased. During the testing phase (Panel C), R1 responding was reduced when the cue was present relative to when it was absent, and R2 responding was also elevated when the cue was present relative to when it was not. These conclusions were confirmed by the following statistical analyses.

Fig. 1
figure 1

R1 acquisition throughout Phase 1 (Panel A), R1 extinction and R2 acquisition in Phase 2 (Panel B), and responding during the test for R1 and R2 (Panel C) in Experiment 1a. See text for statistical analysis

Full size image

R1 acquisition (Phase 1)

The animals increased their responding throughout acquisition, as confirmed by an ANOVA on responding over the 12 sessions, which revealed a main effect of session, F (11, 154) = 24.41, MSE = 24.12, p < .001, ηp2 = .64, 95% CI [.52–.68].

R1 extinction and R2 acquisition (Phase 2)

Throughout Phase 2, animals decreased their R1 responding. This was confirmed by an ANOVA conducted on R1 responding which revealed a main effect of session, F (7, 98) = 12.74, MSE = 5.81, p < .001, ηp2 = .48, 95% CI [.30–.56]. Animals also increased their responding on R2 throughout Phase 2 as confirmed by an ANOVA assessing responding throughout this phase, F (7, 98) = 16.28, MSE = 43.96, p < .001, ηp2 = .54, 95% CI [.37–.61].

Resurgence test (Phase 3)

A 2 (Session: Cue vs. No Cue) × 2 (Response: R1 vs. R2) ANOVA was run to assess responding on both levers throughout the test sessions. The ANOVA revealed a main effect of response, F (1, 14) = 62.84, MSE = 44.28, p < .001, ηp2 = .82, 95% CI [.53–.89], but no main effect of session, F (1, 14) = 1.40, p > .05. The session by response interaction was significant, F (1, 14) = 8.83, MSE = 8.58, p = .01, ηp2 = .39, 95% CI [.03–.63]. Planned comparisons revealed that animals responded less on the R1 response during the session when R2 produced the cue than in the session without the cue, F (1, 14) = 10.87, p = .005, ηp2 = .44, 95% CI [.03–.63]. Thus, the cue attenuated the resurgence effect. In addition, there was more R2 responding in the presence of the cue than without it, F (1, 14) = 4.59, p = .05, ηp2 = .25, 95% CI [.00–.53].

Experiment 1b: Method

Subjects and apparatus

The subjects were 32 female Wistar rats that were obtained, housed, and maintained exactly as those in Experiment 1a. The apparatus and reinforcers were also the same. As before, a 2-s 4,500- Hz, 65-dB tone was emitted from a sonalert module mounted directly above the magazine (Med-Associates Model Number: ENV-223HAM). In order to increase the salience of the cue, a 2-s illumination of a panel light mounted immediately above the sonalert module also occurred at the same time.

Procedure

Animals were run twice a day.

Magazine training and R1 acquisition (Phase 1)

Magazine training and Phase 1 proceeded exactly as in Experiment 1a.

R1 extinction and R2 acquisition (Phase 2)

All animals then received eight sessions in which R1 presses were extinguished (i.e., produced no reinforcers) and presses on the second lever (R2) were reinforced with O2 on a VI 30-s schedule. In one group, delivery of each food pellet was simultaneous with the onset of the 2-s cue. In the second group, Group Unpaired, the cue was also presented in a response-contingent manner, but on its own separate VI 30-s schedule from the pellet (thus, the cue and reinforcer were presented randomly with respect to each other but still contingent on lever pressing). In this way, the cue and reinforcer were not explicitly paired. Both the left and the right levers were inserted throughout each session.

Resurgence test (Phase 3)

The test was identical to Experiment 1a for all animals.

Data analysis

This was as described in Experiment 1a. Two animals were excluded from Group Paired because they were significant overall outliers during the cue test (Zs = 2.3, 3.2) and one was excluded for outlier performance during the no cue test (Z = 2.8).

Experiment 1b: Results

The results are shown in Fig. 2. As before, R1 responding increased in Phase 1 (Panel A) and declined in Phase 2 (Panel B), when the newly-available and reinforced R2 response increased. During the test, R1 responding was reduced only in animals that had received the cue paired with O2 in Phase 2.

Fig. 2
figure 2

R1 responding throughout Phase 1 (Panel A), R1 and R2 responding throughout Phase 2 (Panel B) and the test (Panel C) in Experiment 1b. Please note that error bars (SEM) are only appropriate for between-subject comparisons

Full size image

R1 acquisition (Phase 1)

All animals increased their responding throughout acquisition, as confirmed by 2 (Group) × 12 (Session) ANOVA on responding over Phase 1 in Experiment 1b revealed a main effect of session, F (11, 297) = 69.53, MSE = 32.94, p < .001, ηp2 = .72, 95% CI [.66–.75], but no main effect of group or interaction, Fs < 1.

R1 extinction and R2 acquisition (Phase 2)

Animals decreased their R1 responding during Phase 2, which was confirmed by a 2 (Group) × 8 (Session) ANOVA that found an effect of session, F (7, 189) = 31.32, MSE = 5.67, p < .001, ηp2 = .54, 95% CI [.43–.60], but no effect of group or an interaction, Fs < 1. The rats also increased their responding on R2, as confirmed by a 2 (Group) × 8 (Session) ANOVA, which found a session effect, F (7, 189) = 53.28, MSE = 45.41, p < .001, ηp2 = .66, 95% CI [.58–.75], but no group effect or interaction, Fs < 1.

Resurgence test (Phase 3)

A 2 (Group) × 2 (Session: Cue vs. No Cue) × 2 (Response: R1 vs. R2) ANOVA was run to assess responding on both levers during testing. This revealed a main effect of response, F (1, 27) = 174.22, MSE = 68.18, p < .001, ηp2 = .87, 95% CI [.74–.91]. The session by response interaction observed in Experiment 1a was not found, F (1, 27) = 1.07, p = .31. In order to assess the important group differences, supplementary analyses were run. A separate 2 (Group) × 2 (Session) ANOVA assessed group differences in R1 responding because differences in this response were the primary interest. This revealed a main effect of session, F (1, 27) = 5.54, MSE = 3.79, p < .05, ηp2 = .17, 95% CI [.00–.40], but no effect of group, F = 2.87, p = .10 or interaction, F = 2.18, p = .15. Importantly, planned comparisons revealed that Group Paired showed reduced responding during the test in which responding produced the cue relative to the session with no cue, F (1, 27) = 6.65, p < .02, ηp2 = .20, 95% CI [.01–.43]. There was no corresponding difference in Group Unpaired, F < 1. Further, Group Paired showed suppressed responding relative to Group Unpaired during the test with the cue, F (1, 27) = 5.28, MSE = 8.14, p < .05, ηp2 = .16, 95% CI [.00–.39], but the groups did not differ during the test without the cue, F < 1.

Experiment 1c: Method

Subjects and apparatus

The subjects were 32 female Wistar rats obtained, housed, and maintained exactly as those in Experiment 1a and b. The apparatus and reinforcers were the same as Experiment 1a and b. The compound tone/light cue from Experiment 1b was used here. Twice-daily sessions were used throughout the experiment, as before.

Magazine training, R1 acquisition (Phase 1), and R1 extinction and R2 acquisition (Phase 2)

Magazine training, Phase 1, and Phase 2 training proceeded identically to Experiment 1a.

Resurgence test (Phase 3)

On the day following the conclusion of Phase 2, all rats received two 10-min test sessions with both levers inserted. R1 and R2 presses were recorded, but neither produced a food outcome. During one test, the 2-s cue was presented. For the contingent group, this was contingent on R2 responding according to an RI 30-s schedule (as during Phase 2); for the noncontingent group, the cue was presented noncontingently on an RT 30-s schedule. No cues were presented in the second test for either group.

Data analysis

One animal from Group Noncontingent was a significant outlier in both tests (Z = 2.2 in the cue test; Z = 3.5 in the uncued test).

Experiment 1c: Results

The results are shown in Fig. 3. As usual, the course of R1 acquisition (Panel A) and its extinction and replacement with R2 (Panel B) proceeded without incident. During the test (Panel C) R1 responding was reduced when the cue was present and R2 responding was increased when the cue was present. This was true regardless of whether or not the cue was contingent on R2 responding.

Fig. 3
figure 3

R1 acquisition throughout Phase 1 (Panel A), R1 extinction and R2 acquisition throughout Phase 2 (Panel B), and R1 and R2 responding during the test (Panel C) in Experiment 1c. Please note that error bars (SEM) are only appropriate for between-subject comparisons

Full size image

R1 acquisition (Phase 1)

All animals increased their responding throughout acquisition, as confirmed by a 2 (Group) × 12 (Session) ANOVA. This revealed a main effect of session, F (11, 319) = 69.32, MSE = 22.32, p < .001, ηp2 = .71, 95% CI [.64–.74], but neither a main effect of group nor a significant interaction, Fs < 1.

R1 extinction and R2 acquisition (Phase 2)

Throughout Phase 2, animals decreased their R1 responding. This was confirmed by a 2 (Group) × 8 (Session) ANOVA, which found a main effect of session, F (7, 203) = 29.74, MSE = 3.62, p < .001, ηp2 = .51, 95% CI [.40–.57], but no main effect of group or a group by session interaction, Fs < 1. The rats also increased their responding on R2 throughout the phase. This was confirmed by a 2 (Group) × 8 (Session) ANOVA, which found a main effect of session, F (7, 203) = 52.82, MSE = 41.63, p < .001, ηp2 = .65, 95% CI [.56–.69], but no group effect or interaction, Fs < 1.

Test

A 2 (Group) × 2 (Session: Cue vs. No Cue) × 2 (Response: R1 vs. R2) ANOVA was run to assess responding on both levers during the test sessions. This revealed a main effect of response, F (1, 29) = 77.29, MSE = 112.27, p < .001, ηp2 = .73, 95% CI [.52–.82] and a main effect of session, F (1, 29) = 4.49, MSE = 13.87, p < .05, ηp2 = .13, 95% CI [.00–.36]. The session by response interaction was significant, F (1, 29) = 13.80, MSE = 7.89, p = .001, ηp2 = .32, 95% CI [.07–.53]. No other main effects or interactions were significant (largest Fs < 1). Thus, in the presence of the cue, R2 responses were elevated and R1 responses were suppressed as compared to sessions in which the cue was not present. Planned comparisons revealed that, when collapsed across group (as there was no significant interaction involving this factor), animals responded more on the R2 response during the session when R2 produced the cue than in the session without the cue, F (1, 30) = 8.65, p < .01, ηp2 = .23, 95% CI [.02–.44]. R1 responding appeared to show a trend in the opposite direction but failed to reach significance, F (1, 30) = 1.90, p = .18. Based on the findings from Experiments 1a and 1b (where the same cue presented in the same manner significantly decreased R1 responding), the a priori hypothesis was that the cue would function similarly here. Thus, all animals from Experiment 1 (a through c) who had the reinforcer paired with the cue (i.e., all animals except for Group Unpaired of Experiment 1b; n = 60) were combined to examine responding during both test sessions. These results are detailed in Fig. 4. A 2 (Session: Cue vs. No Cue) × 2 (Response: R1 vs. R2) found a main effect of response, F (1, 59) = 184.23, MSE = 86.13, p < .001, ηp2 = .76, 95% CI [.64–.82], and a session by response interaction, F (1, 59) = 27.68, MSE = 7.94, p < .001, ηp2 = .32, 95% CI [.13–.47], but no effect of session, F (1, 59) = 2.58, p = .11. Planned comparisons found that in sessions when R2 produced the cue, R1 was decreased relative to sessions in which it did not, F (1, 59) = 16.35, p < .001, ηp2 = .22, 95% CI [.06–.38], and R2 was increased, F (1, 59) = 11.78, p = .001, ηp2 = .17, 95% CI [.03–.33].

Fig. 4
figure 4

Responding during the R1 and R2 test collapsed across all animals from Experiment 1 (a–c) with the exception of Group Unpaired from Experiment 1b. See text for statistical analysis

Full size image

It is worth noting that the rats in each experiment showed resurgence during testing in the sense that performance during the control (i.e., uncued) test was significantly higher than that observed during the final session of extinction [Experiment 1a: t (14) = 5.21, p < .001; Experiment 1b: t (28) = 7.60, p < .001; Experiment 1c: t (30) = 5.53, p < .001]. Spontaneous recovery was not considered likely because the Phase-2 procedure involved running extinction sessions until spontaneous recovery had been eliminated at the start of each session. However, since the experiments utilized twice-daily sessions, another appropriate statistical comparison is between responding during the uncued test and the first 10 min of the first session of the last extinction day (Session 7). (Session 7 is a more conservative control for spontaneous recovery because it occurred approximately 22 h since the previous extinction experience.) These comparisons also suggested significant resurgence during testing [Experiment 1a: t (14) = 2.55, p < .05; Experiment 1b: t (28) = 4.50, p < .001; Experiment 1c: t (30) = 2.31, p < .05]. The idea that spontaneous recovery was not a factor is further supported by the fact that there were no differences between responding during the first 10 min of the last two first-of-the-day extinction sessions (Sessions 5 and 7): Experiment 1a: t (14) = 0.92, p = .37; Experiment 1b: t (28) = 1.99, p = .06 (decrease from 5 – 7); Experiment 1c: t (30) = 0.52, p = .61.

Discussion

The results of Experiments 1a, b, and c suggest that a cue that is associated with alternative reinforcement during Phase 2 of the resurgence paradigm can reduce resurgence when it is presented during testing. The results of Experiment 1b further suggest that the cue must be associated with alternative reinforcement in Phase 2 for it to be effective during the test. This result suggests that, without association with a reinforcer, an audiovisual cue may not be sufficiently salient. As noted earlier, one explanation is that a cue that predicts nothing may not be salient or attention-commanding; according to the Mackintosh (1975) model, a stimulus consistently paired with reinforcement will command increasing attention. Finally, the results of Experiment 1c suggest that a cue associated with alternative reinforcement during Phase 2 may weaken resurgence regardless of whether it is contingent or not contingent on R2 during the test. The effectiveness of the noncontingent cue is consistent with findings reported using a noncontingent reinforcer (O2) as a retrieval cue in attenuating both resurgence (Bouton & Trask, 2016; Trask et al., 2018) and renewal (Trask & Bouton, 2016).

One possibility is that the present paired cue, which came on at the same time as the pellet was delivered and then stayed on for about 1.75 s afterward during Phase 2, could have developed inhibitory properties that allowed it to suppress resurgence of R1. However, I am not aware of any evidence that such a mixture of simultaneous and backward conditioning procedures can imbue a stimulus with net inhibition. Perhaps in contrast, the cue tested in Group Unpaired in Experiment 1b could have had inhibitory properties because it was not paired with the reinforcer during Phase 2 and was actually presented contingent on a response that potentially predicted it. But it did not have the same effect on R1. And while the paired cue did suppress (or inhibit) R1 responding, its effect on R2 was evidently to enhance it (e.g., Fig. 4). These observations might make it difficult to accept the possibility that the effective cue worked through conditioned inhibitory properties.

An alternative account of the effects of the cue associated with Phase-2 reinforcement is that it provided conditioned reinforcement (e.g., Williams, 1994) during testing. A conditioned reinforcer might reduce resurgence of R1 by promoting R2 behavior, which might increase response competition. However, the results of Experiment 1c suggest that the cue was equally effective at promoting R2 behavior (and reducing R1 behavior) regardless of whether or not it was contingent on R2 during the test. Such results suggest that the presence of the cue might only need to encourage generalization between extinction and the test. Experiment 2 was designed to explore this possibility further.

Experiment 2

Extinction of instrumental behavior is thought to result in the formation of an inhibitory S – R association (e.g., Bouton et al., 2016; Rescorla, 1993, 1997; Todd, 2013; Todd, Vurbic, & Bouton, 2014; see Trask et al., 2017, for a review). In resurgence, removal of the cues associated with response inhibition might result in a return (renewal) of the original behavior. According to this view, if a cue is to reduce resurgence effectively, it would need to be featured in a session in which R1 is directly extinguished. (Notice that this was always true in Experiments 1a–c.) Experiment 2 (illustrated in Table 1) was therefore designed to explicitly test the hypothesis. The experiment used a within-subject design in which animals were given two types of alternating Phase 2 sessions following the usual Phase 1 training. In the first type of Phase-2 session, R1 was extinguished while R2 produced O2 together with a cue (Cue 1, either a tone or light counterbalanced). In the second type of session, R1 was unavailable (i.e., the lever was retracted) and R2 produced O2, which now coincided with a different cue (Cue 2, light or tone, counterbalanced). The hypothesis was that, due to its presence during R1 extinction sessions, Cue 1, but not Cue 2 (which was not present during R1 extinction), would be able to attenuate resurgence when R2 produced it during a resurgence test. Note that this was expected even though the two cues were equally associated with O2, and therefore should have had equal conditioned reinforcing properties.

Method

Subjects and apparatus

The subjects were 24 female Wistar rats obtained, housed, and maintained as before. The apparatus was the same.

Procedure

Magazine training and R1 Acquisition (Phase 1)

Magazine training and R1 acquisition proceeded as in Experiments 1a–c.

R1 extinction and R2 acquisition (Phase 2)

All animals then received eight sessions in which R1 was extinguished (i.e., produced no reinforcers) and presses to the second lever (R2) were reinforced with O2 on a VI 30-s schedule. Onset of a 2-s cue, Cue 1 (counterbalanced as tone or light) coincided with each O2 delivery. These R1 extinction sessions were double-alternated with sessions in which only R2 was available and produced O2. The R1 lever remained retracted throughout these sessions and was therefore not extinguished. Onset of a 2-s cue, Cue 2 (counterbalanced as the light or tone) coincided with the delivery of O2 in these latter sessions. Thus, while Cue 1 and Cue 2 were equally associated with R2 and O2, only Cue 1 was associated with the extinction of R1. Half the animals received daily sessions in the order of Cue1, Cue 2, Cue 2, Cue1, and half received them in the order of Cue 2, Cue 1, Cue 1, Cue 2.

Resurgence test (Phase 3)

On the day following the conclusion of Phase 2, all rats received three 5-min test sessions in which both levers were inserted. R1 and R2 presses were recorded, but neither produced an outcome. During one test, R2 presses produced Cue 1 on a VI 30-s schedule. During a second test, R2 presses produced Cue 2. No cues were presented in the third test. Testing order was fully counterbalanced.

Results

The results of Experiment 2 are depicted in Fig. 5. Animals increased R1 responding throughout Phase 1 (Panel A). During Phase 2 (Panel B), R1 responding decreased in sessions in which it was available (when R2 produced O2 and Cue 1). R2 responding increased in both Cue 1 and Cue 2 sessions. During the crucial test (Panel C), R1 responding was reduced in sessions in which R2 produced Cue 1 relative to both the test in which R2 produced nothing and the test in which R2 produced Cue 2.

Fig. 5
figure 5

R1 acquisition throughout Phase 1 (Panel A), R1 extinction and R2 acquisition during sessions in which R2 produced Cue 1 and R2 acquisition during sessions in which R2 produced Cue 2 in Phase 2 (Panel B), and responding during the test for R1 and R2 (Panel C) in Experiment 2. See text for statistical analysis

Full size image

R1 acquisition (Phase 1)

The animals increased their responding throughout acquisition, as confirmed by an ANOVA on responding over the 12 sessions, which revealed a main effect of session, F (11, 253) = 41.38, MSE = 31.71, p < .001, ηp2 = .64, 95% CI [.56–.68].

R1 extinction and R2 acquisition (Phase 2)

Animals decreased their R1 responding during the sessions in which R1 was extinguished, as confirmed by an ANOVA that revealed a main effect of session, F (7, 161) = 55.74, MSE = 3.07, p < .001, ηp2 = .71, 95% CI [.62–.75]. Animals also increased their responding on R2 throughout Phase 2 during both sessions where R2 produced O2 and Cue 1 and sessions where R2 produced O2 and Cue 2. This was confirmed by a 2 (Cue 1 vs. Cue 2) × 8 (Session) ANOVA assessing R2 responding throughout this phase. This found a main effect of session, F (7, 161) = 32.02, MSE = 142.89, p < .001, ηp2 = .60, 95% CI [.47–.64]. Responding during the Cue 2 sessions (in which R1 was absent) was slightly higher, as revealed by a main effect of cue (1, 23) = 7.67, MSE = 44.45, p < .05, ηp2 = .25, 95% CI [.01, .49]. No interaction was found, F (7, 161) = 1.21, MSE = 24.81, p = .30.

Resurgence Test (Phase 3)

A 3 (Session: Cue 1 vs. Cue 2 vs. No Cue) × 2 (Response: R1 vs. R2) ANOVA assessed R1 and R2 responding during the test sessions. This revealed a main effect of response, F (1, 23) = 68.00, MSE = 259.79, p < .001, ηp2 = .75, 95% CI [.51–.84], but no main effect of session, F < 1. Importantly, the session by response interaction was significant, F (2, 46) = 10.94, MSE = 41.31, p < .001, ηp2 = .32, 95% CI [.10–.48]. Planned comparisons revealed that animals responded less on R1 during the session when R2 produced Cue 1 than in the session without the cue, p < .001, and in the session where R2 produced Cue 2, p = .001. R1 responding did not differ between Cue 2 and No Cue sessions, p = 1.00. Thus, only the cue that had been present during sessions in which R1 was extinguished attenuated the resurgence effect. Planned comparisons assessing R2 responding demonstrated that sessions in which R2 produced Cue 1, R2 responding was elevated relative to no cue sessions, p < .05, but did not differ from sessions where R2 produced Cue 2, p = .09. R2 responding did not differ in sessions where R2 produced Cue 2 or no cue, p = .21.

As in Experiment 1, animals showed a significant resurgence effect when performance from the final day of extinction was compared with that in the control test, t (23) = 7.45, p < .001. Resurgence was also evident when responding in the first 5 min of Session 7 was compared to responding during the 5-minute control test, t (23) = 4.89, p < .001. Again, no spontaneous recovery was evident when comparing the first 5 min of Session 5 to the first 5 min of Session 7, t (23) = 0.08, p = .94.

Discussion

Cue 1, but not Cue 2, attenuated R1 resurgence in the final test. This result suggests that mere pairings between the cue and O2 are not sufficient to allow a cue associated with alternative reinforcement to attenuate the resurgence effect. This result continues to argue against a role for conditioned reinforcement. Instead, the cue must also occur in sessions in which R1 is being extinguished. This finding may accord with several other studies demonstrating that extinction results in new learning of response inhibition in the extinction context (Bouton et al., 2016; Rescorla, 1997; Todd, 2013). While Cues 1 and 2 could both have entered into excitatory associations with R2 and/or O2, only Cue 1 could have been associated with any new inhibitory learning about R1, as Cue 2 never occurred in sessions during which R1 was extinguished. Thus, presenting Cue 1 during the test, but not Cue 2, would have increased the generalization of response inhibition from extinction to the test. This result may explain a finding recently reported by Craig, Browning, and Shahan (2017) in which a cue that was associated with reinforcement during both Phase 1 and Phase 2 failed to attenuate resurgence significantly (although the authors concluded that the conditioned reinforcer attenuated resurgence, the effect was not statistically reliable [reported p = .128]). The present results suggest that the cue might have been more effective if it had been more exclusively featured in extinction (Phase 2). It is interesting to note that while Cue 1 was presented in sessions in which R1 was available and not reinforced, the cue itself was not explicitly or temporally linked to occasions in which R1 occurred and was not reinforced. This suggests that the cue might influence R1 responding in a similar manner to a contextual cue, which are not usually thought to have a programmed, direct relationship with the response aside from their presence during learning.

General discussion

The current experiments examined the circumstances and mechanisms through which a cue presented in Phase 2 of a resurgence paradigm can come to attenuate resurgence when presented in the final resurgence test. Experiment 1a initially established the effect, and Experiment 1b then demonstrated that the cue was effective only if it had been explicitly paired with that reinforcer. That result is consistent with the Mackintosh (1975) model of attention. Experiment 1c then established that the cue attenuated resurgence if it was presented either contingently or noncontingently on R2 responding during testing. Experiment 2 then suggested that the cue has to be presented in sessions in which R1 was extinguished in order to be able to attenuate R1 resurgence later. A second cue, not presented in sessions when R1 was extinguished (but with an otherwise similar history), was not effective at attenuating resurgence. Overall, the results are consistent with the view that methods that encourage attention to a stimulus presented during R1 extinction result in the greatest likelihood that the cue will be salient enough to reduce resurgence during testing, and that the reduction in resurgence depends on the cue being present during sessions of R1 extinction. To my knowledge, these are the first results demonstrating that an initially neutral cue that is a unique feature of extinction can attenuate the resurgence effect.

The results extend previous work demonstrating that a reinforcer associated with sessions in which R1 is extinguished can attenuate both resurgence (Bouton & Trask, 2016) and renewal (Trask & Bouton, 2016) of an extinguished instrumental response. Notably, the present resurgence-attenuating effects appear to have been dissociated from any conditioned reinforcing properties and Pavlovian associations. This was first suggested by Experiment 1c, in which the cue suppressed R1 responding during testing whether or not it was contingent on R2. Experiment 2 expanded on this result by showing that only a cue that was presented in sessions in which R1 was extinguished would attenuate resurgence; a cue that had similar pairings with both R2 and the reinforcer did not. The pattern suggests that rather than working directly through their association with either R2 or the reinforcer, cues reduce resurgence through their association with the new learning that occurs during extinction. A growing literature suggests that in extinction, the contextual cues present come to directly suppress the response through an inhibitory S-R (or context-R) association. That is, animals learn to inhibit a specific response in the presence of specific contextual cues in which it was extinguished (Bouton et al., 2016; Rescorla, 1993, 1997; Todd, 2013; Todd et al., 2014). Perhaps the present cue operates in a similar manner. In Experiment 2, only a cue that was associated with extinction of R1 could be associated with the inhibition of R1. Thus, according to a response inhibition account of extinction, only that cue could control inhibition of R1.

However, it should be noted that despite the failure of a cue not associated with alternative reinforcement to attenuate resurgence in the current Experiment 1b, several studies have shown that more neutral cues (e.g., those that have never been paired with alternative reinforcement) on their own can attenuate other forms of instrumental relapse, including renewal (Nieto et al., 2017; Willcocks & McNally, 2014), spontaneous recovery, and reinstatement (Bernal-Gamboa et al., 2017). In one representative study, Nieto et al. (2017) trained rats to perform two responses (R1 and R2) to receive food reinforcement, each in a distinct context (Context A and Context B, respectively). Responding was then extinguished in the opposite context (i.e., R1 in B and R2 in A; see Todd, 2013). During extinction of R1, a 5-s tone played approximately twice every minute not contingent on responding. Animals were then tested for each response in its acquisition and extinction contexts. For a crucial group, presentations of the extinction cue occurred in both renewal tests for R1 and R2. While an overall renewal effect was seen (e.g., responding was higher on each response in its renewal context than in its extinction context), renewal was weakened on R1 relative to R2. This suggests that a cue associated with extinction reduced the renewal effect and that this effect is limited to responses extinguished in the presence of that cue. According to the authors, these results further demonstrate that extinction learning results in formation of an inhibitory S-R association, as the cue only weakened the response that it was extinguished with and did not transfer to another response (as would be expected if the cue were functioning as a negative occasion setter).

One element of the resurgence paradigm that differs from spontaneous recovery, renewal, and reinstatement is that it involves reinforcement of an alternative response during extinction of the target response. We have argued (e.g., Trask et al., 2015) that the presence of alternative reinforcement in the resurgence paradigm is itself salient enough to act as a contextual cue that serves to suppress behavior, and that this alternative reinforcer does seem to have equal and similar ability to control behavior as physical context (Trask & Bouton, 2016). Including alternative reinforcement for an alternative behavior during extinction could have many effects. The presence of the reinforcer itself could potentially draw attention toward itself and away from less salient aspects of the situation. That is, a reinforcer is likely to attract more attention and interaction than, for example, a brief illumination of a panel light. Perhaps making the cue relevant in the present experiments increased attention to that cue (e.g., Mackintosh, 1975) when limited attention resources would otherwise have been directed toward the reinforcer. It is notable that a cue that has never been paired with a reinforcer is also not effective at weakening rapid reacquisition, a relapse phenomenon that also involves multiple presentations of a reinforcer (Willcocks & McNally, 2014). In contrast, Kincaid, Spence, and Lattal (2015) demonstrated in pigeons that illumination of the unique key light that set the occasion for R2 responding in Phase 2 could attenuate R1 resurgence on another key when the R2 key light continued to be illuminated during testing. This finding, combined with the current results, suggests that in order for a cue to attenuate relapse in situations where alternative reinforcement is present, it has to be both salient enough to attract some attention as well as associated with R1 extinction.

The results are generally consistent with the context hypothesis of resurgence (Trask et al., 2015; Winterbauer & Bouton, 2010), which suggests that resurgence is an ABC-like renewal effect in which contexts B and C are created by the presence and absence of reinforcers. One implication of that view is that cues that occur during sessions in which R1 is extinguished can increase the generalization between the extinction contexts and testing contexts when they are presented during the test. Other accounts of resurgence, such as a behavioral-momentum based model (Shahan & Sweeney, 2011) and the resurgence as choice (i.e., RaC) model (Shahan & Craig, 2017), would fail to account for the present findings because they do not invoke mechanisms that would allow a treatment cue to have any impact on resurgence responding. The behavioral-momentum based model (Shahan & Sweeney, 2011) suggests that removal of the reinforcer during the test should reduce its disruptive effect on R1, causing a resurgence of the response. In the current experiments, neither the test with the cue nor the test without the cue have any reinforcers present. Thus, according to this view, there should be no difference in resurgence responding in the presence or absence of the cue. A possible extension of the model might allow a conditioned reinforcer to act in the place of a primary reinforcer during resurgence testing and thus work similarly to disrupt R1 responding and weaken resurgence. However, given the current evidence against a role for conditioned reinforcement (Experiments 1c and 2), even this extension seems unprepared to account for the results.

The resurgence as choice model (Shahan & Craig, 2017) also focuses on the reinforcement rate and its reinforcing (rather than discriminative) properties. This model suggests that resurgence occurs as a function of the recency and cumulative history of reinforcement for R1 and R2. It emphasizes the matching law (Herrnstein, 1961) and suggests that the same principles that govern choice produce resurgence. Essentially, resurgence of R1 is proposed to occur because placing R2 on extinction decreases the value of that response. This results in a relative increase in the response value of R1, causing R1 responding to increase. However, this model also provides no mechanism that would predict or explain the present effects of the cue. Overall, both the behavioral momentum and resurgence-as-choice models fail to account for the current findings because they give no role to the discriminative effects of cues and reinforcers in controlling instrumental performance, which is the crucial process emphasized by the context view of resurgence.

We (Bouton & Schepers, 2014; Bouton, Thrailkill, Bergeria, & Davis, 2017; Winterbauer, Lucke, & Bouton, 2013) and others (Craig, Nall, Madden, & Shahan, 2016; Quick, Pyszczynski, Colston, & Shahan, 2011) have noted that resurgence may have implications for contingency management (CM) treatments in humans with behavior-related health problems such as drug dependence. In CM treatments, patients earn vouchers to be exchanged for goods contingent on proof of abstinence (e.g., drug-free urine samples). While this reduces the drug-taking behavior (Higgins, Sigmon, & Heil, 2011; Petry & Martin, 2002; Rawson et al., 2006), the behavior can return (or resurge) when CM is discontinued and alternative reinforcement ceases (Roll, Chudzynski, Cameron, Howell, & McPherson, 2013; see Davis, Kurti, Skelly, Redner, White, & Higgins, 2016). Understanding mechanisms of resurgence (and how to reduce it) may add to our understanding of the relapse seen following CM treatment.

While there are several notable differences between CM and resurgence (e.g., the lack of a contingency between abstinence and reinforcement [Bouton & Schepers, 2014] and the inability to place human behavior on extinction [Bouton et al., 2017]), in general, both the second phase of a resurgence paradigm and CM are effective at reducing target behavior and leave the suppressed behavior susceptible to relapse following the cessation of that phase (Davis et al., 2016; Petry, Martin, Cooney, & Kranzler, 2000). These similarities suggest that, despite procedural differences, factors that work to reduce resurgence may also be effective in reducing relapse following the cessation of CM treatments. The present studies might suggest that a cue associated with reinforcement in the treatment phase may serve to weaken relapse after CM if it is occasionally presented after treatment is terminated. In one potential example of this, Higgins, Budney, Bickel, and Badger (1994) demonstrated that cocaine abstinence was highest in participants whose significant other participated in the treatment. One explanation of this finding is that, like the treatment cues in the current experiments, the presence of the significant other at treatment made the treatment situation generalize better to the situations where relapse was more likely. Other preparations that reduce resurgence (such as the current cue paired with alternative reinforcement) may also function to reduce relapse following contingency management treatments.

In conclusion, the present experiments demonstrate that a cue paired with alternative reinforcement during sessions in which R1 was extinguished can be used to attenuate resurgence of that response when they are presented during the test. Additionally, these cues need to be sufficiently attention-commanding to attenuate resurgence, as cues not paired with the reinforcer during R1 extinction sessions did not weaken resurgence. Further, the resurgence-attenuating effects seem not to depend on Pavlovian S-O associations, as a cue with an equal Pavlovian history did not serve to weaken resurgence in the present Experiment 2. Instead, the cues may work by enhancing generalization between the sessions in which R1 is extinguished and the testing session, increasing transfer of the inhibitory learning acquired during extinction. These are, to my knowledge, the first results demonstrating that previously neutral cues associated with extinction can attenuate the resurgence effect.