Resurgence is the return of an extinguished operant behavior that occurs when reinforcement for an alternative, second behavior that has replaced it is discontinued (Leitenberg, Rawson, & Bath, 1970; Lieving & Lattal, 2003; Sweeney & Shahan, 2013b; Winterbauer & Bouton, 2010). In a typical study of resurgence, a target response (R1) is first reinforced in an initial phase (Phase 1). Following acquisition, a response elimination phase (Phase 2) is conducted in which R1 is extinguished while a newly introduced response, R2, is reinforced. Finally, in a test phase (Phase 3), R2 reinforcement is discontinued (i.e., both responses are placed in extinction). The removal of reinforcement results in an increase, or resurgence, of R1 behavior. Resurgence is similar to several other forms of relapse that occur after the extinction of an operant behavior (e.g., renewal, reinstatement, and spontaneous recovery) in suggesting that extinction does not erase the original learning (e.g., Bouton, 2014; Bouton, Winterbauer, & Todd, 2012a; Bouton, Winterbauer, & Vurbic, 2012b).

At least three explanations of resurgence have been suggested. Leitenberg et al. (1970) originally noted that resurgence might occur because reinforcement of R2 can cause this behavior to interfere with R1 behavior so much that the animal does not have the opportunity to learn that responding on R1 is no longer reinforced. Although such a mechanism might contribute to resurgence in some circumstances, animals often emit a large number of R1 responses in extinction, and resurgence can still occur under Phase 2 conditions that cause a higher level of R1 responding in a resurgence group than in an extinction control group that does not demonstrate resurgence during testing (Winterbauer & Bouton, 2010). Such a result seems incompatible with the idea that R1 resurges when it has received too little extinction.

Shahan and Sweeney (2011) proposed an alternative explanation of resurgence in the form of a quantitative model based on behavioral momentum theory (e.g., Craig, Nevin, & Odum, 2014; Nevin & Grace, 2000). The model assumes that adding reinforcement for R2 to the extinction of R1 has two fundamental effects on R1 performance. First, the alternative reinforcement disrupts R1 beyond the disruption from a typical extinction procedure. Second, the alternative reinforcement also increases the strength of R1 through a process (contextual conditioning) that is assumed to strengthen all behavior. When the alternative reinforcement is discontinued during testing, the disruption of R1 behavior is removed, but the increased strength of R1 remains, resulting in resurgence. According to the model, schedules that include leaner programmed rates of R2 reinforcement during the final session of Phase 2 (and thus cause less final disruption and strengthening of R1 behavior) should result in lower levels of resurgence. Consistent with this prediction, resurgence is reduced by using very lean reinforcement rates during the response elimination phase (Leitenberg, Rawson, & Mulick, 1975; Sweeney & Shahan, 2013b). Resurgence (defined as an increase in responding from Phase 2 to testing) is also reduced by using schedules that are “thinned” from high rates to low rates across Phase 2 (Schepers & Bouton, 2015; Winterbauer & Bouton, 2012), although thinning itself can produce an increase in R1 responding (termed “early resurgence” by Winterbauer & Bouton, 2012). However, a reverse thinning procedure in which reinforcement rates start the phase at a lean rate and then increase during training also weakens resurgence (Bouton & Schepers 2014; Schepers & Bouton, 2015). Thus, contrary to the model, differences in resurgence have been demonstrated in experimental and control groups that receive the same rates of reinforcement for R2 in the final response elimination session. As a further example, Schepers and Bouton (2015, Exp. 3) gave one group of rats alternating response elimination sessions in which R2 was reinforced (on a variable-interval [VI] 10-s schedule) in odd-numbered sessions and extinguished in even-numbered sessions. After the final VI 10-s session, reinforcement was discontinued. No resurgence was observed in this group, but substantial resurgence was observed in a group that had received VI 10-s during every session. In addition, substantial resurgence was evident in a group that received the alternating group’s overall average rate of reinforcement during each session of the phase (a VI 17.5-s reinforcement schedule). Such results suggest that, inconsistent with the model, the reinforcement rate, either calculated during the last session of response elimination (Shahan & Sweeney, 2011) or averaged over the response elimination phase (Sweeney & Shahan, 2013a), is not necessarily a good predictor of the strength of the resurgence effect.

A third account of resurgence emphasizes the discriminative properties of the reinforcer delivered for R2 responding in Phase 2. Winterbauer and Bouton (2010; see also Bouton & Swartzentruber, 1991) suggested that resurgence is a form of the renewal effect, in which extinguished responding recovers when the background context changes when alternate reinforcement is discontinued (e.g., Bouton, Todd, Vurbic, & Winterbauer, 2011). On this view, the discontinuation of reinforcement during the test creates a change of context from the one that prevailed during R1 extinction. Previous research in Pavlovian conditioning has established that the presence or absence of reinforcers (food pellets) can provide a “reinforcer context” that controls conditioned or extinction performance that is associated with it (Bouton, Rosengard, Achenbach, Peck, & Brooks, 1993). Thus, given the well-documented context-specificity of extinction learning, the animal in a resurgence experiment might learn to inhibit R1 in the “context” of reinforcement of R2; when that context is changed by discontinuing R2 reinforcement, response recovery will occur. Such a perspective is consistent with the finding, mentioned above, that alternating exposure to VI-10 reinforcement and extinction for R2 can abolish the resurgence effect (Schepers & Bouton, 2015). This is because the animal theoretically learns to inhibit R1 when few reinforcers are in the background, conditions more like the extinction conditions that prevail during the resurgence test. The perspective is also consistent with the finding that reverse and forward thinning procedures both reduce resurgence (Bouton & Schepers, 2014; Schepers & Bouton, 2015), with forward thinning arguably being the more effective in reducing it (Schepers & Bouton, 2015). (Forward thinning may be more effective than reverse thinning because the change of context is less abrupt as the animal enters the test phase.) The contextual view importantly emphasizes the discriminative properties of reinforcers presented during Phase 2, rather than their reinforcing and disrupting effects. In doing so, it is consistent with a long theoretical tradition in learning theory (e.g., Ostlund & Balleine, 2007; Reid, 1958; Sheffield, 1949).

The purpose of the present experiments was to further contrast predictions of the behavioral momentum (Shahan & Sweeney, 2011) and contextual (Winterbauer & Bouton, 2010) accounts of resurgence. In Experiment 1, we parametrically explored the prediction, common to both views, that resurgence should be an inverse function of the rate of reinforcement delivered in Phase 2. In Experiment 2, we then pitted the two perspectives against one another more directly. In that experiment, rats earned one reinforcer (O1) for R1 in the first phase and a different reinforcer (O2) for R2 when R1 was extinguished in the second. We then examined the effects of response-independent presentations of either O1 or O2 during resurgence testing. Although the contextual view’s emphasis on the discriminative properties of the reinforcer suggests that O2 presentations (but not O1 presentations) should prevent resurgence, behavioral momentum theory does not make such a prediction. In Experiment 3, we examined the effects of adding noncontingent O2 reinforcers during a simple extinction procedure and tested its effects on reinstatement, the procedure in which extinguished responding returns after noncontingent reinforcement delivery (e.g., Baker, Steinwald, & Bouton, 1991; Franks & Lattal, 1976; Reid, 1958; Rescorla & Skucy, 1969). Overall, the results of Experiments 2 and 3 are inconsistent with the momentum-based resurgence model (Shahan & Sweeney, 2011) and point to a role for the discriminative properties of the reinforcer when it is presented during or following response elimination by extinction.

Experiment 1

The first experiment pursued the finding that a thinner reinforcement schedule during the response elimination phase would reduce or eliminate the resurgence effect. As we noted earlier, response elimination procedures that have thinned the reinforcement schedule for R2 from either rich to sparse (Schepers & Bouton, 2015; Sweeney & Shahan, 2013b; Winterbauer & Bouton, 2012) or sparse to rich (Bouton & Schepers, 2014; Schepers & Bouton, 2015) have weakened the resurgence effect. Leitenberg et al. (1975) also found that resurgence was weaker when a second behavior was reinforced at a constant lean rate (VI 240-s) during extinction of the first behavior than the resurgence seen when the second response was reinforced at a high rate (VI 30-s). Sweeney and Shahan (2013b) similarly found that whereas alternative reinforcement with a high rate of reinforcement (VI 10-s) resulted in resurgence, using a lean rate (VI 100-s) abolished the effect. However, Winterbauer and Bouton (2010) found no difference in resurgence between VI 10-s and VI 30-s. The parametric relationship between VI schedule and resurgence has not been firmly established.

Experiment 1 therefore explored the parameter space more systematically. After initial reinforcement of R1 on a VI 30-s reinforcement schedule, different groups received reinforcement of R2 on either a VI 30-s, VI 60-s, VI 90-s, or VI 120-s schedule during the extinction of R1. R1 and R2 were then tested in extinction. Given Winterbauer and Bouton’s (2012) finding that thinning to a VI 120-s schedule can reduce resurgence, we expected little resurgence in the VI 120-s group. The question was how the strength of resurgence was parametrically related to R2’s rate of reinforcement.

Method

Subjects

The subjects were 32 female Wistar rats obtained from Charles River, Inc. (St. Constance, Quebec). The rats were approximately 85–95 days old at the start of the experiment and were individually housed in suspended stainless steel cages in a room maintained on a 16:8-h light:dark cycle. At the beginning of the experiment, all rats were food deprived to 80 % of their free-feeding weight and maintained at that level throughout the experiment with a single feeding following each day’s session.

Apparatus

Conditioning proceeded in two sets of four standard conditioning boxes (Med-Associates Model Number: ENV-008-VP, St. Albans, VT) that were housed in different rooms of the laboratory. The sets had been modified as described below for use as separate contexts, although they were not used in that capacity here. Boxes from both sets measured 30.5 cm × 24.1 cm × 21.0 cm (l × w × h), with side walls and ceilings made of clear acrylic plastic and front and rear walls made of brushed aluminum. Recessed 5.1 cm × 5.1 cm foodcups with infrared photobeams positioned approximately 1.2 cm behind the plane of the wall and 1.2 cm above the bottom of the cup were centered in the front wall about 3 cm above the grid. In one set of four boxes, the floor was composed of stainless steel rods (0.5 cm in diameter) in a horizontal plane spaced 1.6 cm center to center, whereas in the other set of four boxes, the floor was composed of identical rods spaced 3.2 cm apart in two separate horizontal planes, one 0.6 cm lower than the other and horizontally offset by 1.6 cm. The boxes with the planar floor grid had a side wall with black panels (7.6 cm × 7.6 cm) placed in a diagonal arrangement, and there were diagonal stripes on both the ceiling and the back panel, all oriented in the same direction, 2.9 cm wide, and about 4 cm apart. The other boxes, with the staggered floor, were not adorned in any way. Retractable levers (1.9 cm when extended) were positioned approximately 3.2 cm to the right and to the left of the food cup and 6.4 cm above the grid. Both sets of boxes were housed in sound-attenuating chambers and were continuously illuminated by two 7.5-W incandescent light bulbs mounted on the chamber ceiling. The apparatus was controlled by computer equipment located in an adjacent room. Food reinforcers consisted of 45-mg MLab Rodent Tablets (5-TUM: 181156; TestDiet, Richmond, IN).

Procedure

Following a day in which one session of magazine training occurred (see below), twice-daily sessions were employed throughout the experiment. Each day’s first session began with approximately 15 h of illuminated colony time remaining. Each day’s second session began with approximately 12 h of illuminated time remaining. Animals were placed into illuminated conditioning chambers, and the start of each session was indicated by the insertion of the lever(s) as appropriate. All sessions were 30 min in duration, and the end of the session was indicated by retraction of the lever(s).

Magazine training

All animals received magazine training on the day immediately prior to the beginning of Phase 1. At this time, the rats received a single session with both levers retracted. An average of 60 food pellets were delivered during this session on a random time 30-s (RT 30-s) schedule of reinforcement.

R1 acquisition (Phase 1)

All animals then received 12 sessions of instrumental conditioning initiated by insertion of the left lever in half animals and the right lever in the other half. In all sessions, presses on the lever (R1) delivered pellets on a VI 30-s schedule of reinforcement. No additional response shaping was necessary.

R1 extinction and R2 acquisition (Phase 2)

All animals then received ten sessions in which R1 presses were extinguished and presses to the second lever (R2) were reinforced. Both the left and right levers were inserted throughout each session. R1 presses had no programmed consequences, but presses on the new second lever (R2) earned pellets on either a VI 30-s, VI 60-s, VI 90-s, or VI 120-s schedule of reinforcement for the different groups (n = 8). All groups started with R2 reinforced on a VI 30-s schedule, but they were quickly shaped to their terminal schedule (VI 60-s, VI 90-s, or VI 120-s) over the first two sessions of the phase. For Group VI 60, this shaping consisted of one session in which R2 presses were reinforced on a VI 30-s schedule for 15 min and a VI 60-s schedule for the next 15 min. Groups VI 90 and VI 120 received treatments identical to that in Group VI 60 in the first session of the phase. In the second session, R2 in Group VI 90 was reinforced on a VI 60-s schedule for 15 min and then at VI 90-s for the last 15, whereas R2 in Group VI 120 was reinforced on a VI 60-s schedule for the first 10 min, a VI 90-s schedule during the second 10 min, and a VI 120-s schedule during the last 10 min. All rats were on their terminal schedules of reinforcement by the start of the third session of Phase 2.

Resurgence test (Phase 3)

On the day following the conclusion of Phase 2, all groups received a final test session with both levers inserted. R1 and R2 presses were recorded, but neither had scheduled consequences.

Data analysis

Analyses of variance (ANOVAs) were used to assess response rates throughout the experiment. The rejection criterion was p < .05 for all analyses.

Results

The results of the experiment are presented in Fig. 1. Responding on R1 was readily learned in Phase 1 (top left) and was extinguished in Phase 2 (top middle). Responding on R2 was also readily learned in Phase 2 (lower middle); although the rate of responding appeared most rapid in the VI 30 condition, R2 responding was statistically similar among the groups. However, resurgence of R1 during the final test (top right) was restricted to the groups that had received the richest schedules of reinforcement for R2 responding during Phase 2.

Fig. 1
figure 1

Results of Experiment 1. Upper panels: Mean R1 responding (± SEMs) during acquisition (left), extinction (middle), and testing (right). Lower panels: Mean R2 responding (± SEMs) during its acquisition and testing. All groups received extinction of R1 at the same time that R2 was introduced and were reinforced on either a VI 30-s, VI 60-s, VI 90-s, or VI 120-s schedule of reinforcement. Note the changes in the y-axes between panels; error bars are only appropriate for between-group comparisons

Full size image

R1 acquisition

A 4 (Group) × 12 (Session) ANOVA on R1 responding during acquisition revealed a main effect of session, F(11, 308) = 63.00, MSE = 42.04, p < .001, η p 2 = .69, but no main effect of group or group by session interaction, Fs < 1.

R1 extinction and R2 acquisition

R1 extinction was assessed with a 4 (Group) × 10 (Session) ANOVA. Although we observed a main effect of session, F(9, 252) = 38.78, MSE = 10.91, p < .001, η p 2 = .58, no main effect of group or group by session interaction emerged, Fs < 1. A similar ANOVA performed on R2 acquisition revealed a main effect of session, F(9, 252) = 16.42, MSE = 34.18, p < .001, η p 2 = .37, but neither a main effect of group, F(3, 28) = 2.79, MSE = 984.70, p > .05, nor a significant interaction, F < 1.

Resurgence test

A 4 (Group) × 2 (Session) ANOVA compared the groups’ responding during the last 15 min of Phase 2 to the first 15 min of Phase 3. It revealed a significant overall effect of session, F(1, 28) = 24.29, MSE = 2.96, p < .001, η p 2 = .46, suggesting resurgence, but no significant main effect of group, F < 1. Importantly, there was a significant group by session interaction, F(3, 28) = 4.81, MSE = 2.96, p < .01, η p 2 = .34, suggesting that resurgence differed among the groups. Pairwise comparisons revealed that Group VI 30, F(1, 28) = 9.09, p < .01, η p 2 = .25, and Group VI 60, F(1, 28) = 28.11, p < .001, η p 2 = .50, both increased responding between extinction and testing (resurgence), but that Groups VI 90 and VI 120 did not, Fs (1, 28) < 1.40. A trend analysis on the difference scores (first 15 min of the test – last 15 min of Phase 2) revealed a significant linear trend, F(1, 28) = 7.31, p < .05, η p 2 = .21, suggesting that resurgence decreased as a linear function of VI schedule. The quadratic trend was not significant, F(1, 28) = 2.42, p = .13.

We would note that the increase in responding between the last extinction session and the resurgence test could have been due in part to spontaneous recovery. Indeed, responding increased as a function of time between the last two sessions of Phase 2. A similar 4 (Group) × 2 (Session) ANOVA on responding during the last 15 min of Session 9 (the second-to-last Phase 2 session) and the first 15 min of Session 10 (the last session of Phase 2) revealed a significant effect of session, F(1, 28) = 16.08, MSE = 2.76, p < .001, η p 2 = .37. But, importantly, the group by session interaction did not approach reliability, F(3, 28) = 2.00, MSE = 2.76, p > .10. Thus, the differential effects of the group treatments did not appear until the resurgence test (Session 10 vs. 11). The mean responding rates in the last 15 min of Session 9 and the first 15 min of Session 10 were 4.2 and 5.9, respectively.

An ANOVA on R2 responding comparing the final 15 min of Phase 2 to the first 15 min of the resurgence test revealed a significant main effect of session, F(1, 28) = 47.45, MSE = 57.76, p < .001, η p 2 = .63, but neither a group effect, F(3, 28) = 2.53, MSE = 239.03, p > .05, nor a group by session interaction, F < 1. As before, a trend analysis was conducted on the R2 difference scores (first 15 min of test – last 15 min of extinction). Neither the linear (F < 1) nor the quadratic (F = 1.03) trend was significant.

Discussion

The results confirmed that resurgence decreases systematically as a function of the average interreinforcer interval (VI schedule) used in the response elimination phase. The fact that reinforcement of R2 on either a VI 90-s or a VI 120-s schedule while R1 was being extinguished prevented resurgence is consistent with the context hypothesis, which suggests that rats given infrequent reinforcers during Phase 2 might learn to inhibit R1 in a context that is increasingly similar to the (extinction) resurgence test context. It is also consistent with the momentum-based model (Shahan & Sweeney, 2011), which suggests that thinner schedules cause less conditioning of the background (which would otherwise serve to increase the strength of R1).

It is worth noting that the groups all received 30-min sessions in Phase 2, and that they consequently differed in the numbers of reinforcers they earned in Phase 2, as well as in their rates of reinforcement. We are not aware of any theory that predicts an effect of varying the number of reinforcers, however. It is also worth noting that the leanest reinforcement schedules used in Phase 2 (e.g., VI 120-s) differed most from the VI 30-s schedule used in Phase 1; the context hypothesis might therefore predict a faster loss of R1 responding due to differential generalization decrements. The fact that the groups did not differ in R1 responding in extinction could be due to the fact that the leanest reinforcement schedules might also generate the least response competition from R2 (although the groups also did not differ in their rates of R2 responding). Regardless, the overall pattern of results may be consistent with the fact that although context change can have an impact on both operant responding and operant extinction, operant extinction appears to be more affected by context change (e.g., Bouton & Todd, 2014).

Experiment 2

The context hypothesis’s emphasis on the discriminative, rather than the reinforcing, properties of the reinforcer uniquely predicts that changing the qualitative properties of the reinforcer between phases might have effects in the resurgence design. In Experiment 2, three groups of rats therefore first received reinforcement of R1 on a VI 30-s schedule with one pellet outcome (O1; either a grain-based or a sucrose food pellet, counterbalanced). Then, when R1 was extinguished in the next phase, R2 was reinforced (also on a VI 30-s schedule) with an alternate reinforcer (O2, the sucrose or grain-based pellet). According to the context hypothesis, R1 extinction would be learned in the presence of O2 presentations. To test this possibility, the groups received different treatments during testing. Group None was simply tested with both R1 and R2 in extinction; resurgence of R1 was expected in this group. In contrast, Group O2 received response-independent presentations of O2 during the test. Since O2 was presented noncontingently at the same rate that it was earned during response elimination, either the context hypothesis or the Shahan and Sweeney (2011) model would predict no resurgence in this group (see Lieving & Lattal, 2003, Exp. 3). However, the third group, Group O1, received response-independent presentations of O1, rather than O2, during testing. If the O1 pellet is sufficiently different from the O2 pellet that provided the context of R1’s extinction during Phase 2, the context hypothesis predicts response recovery in this group. In contrast, neither the Shahan–Sweeney model, nor the original behavioral momentum theory from which it was developed, distinguishes between different types of reinforcers (see Grimes & Shull, 2001; Shahan & Burke, 2004). Instead, the model would assume that delivery of O1 at the same rate as O2 in the response elimination phase would continue to suppress performance of R1.

Method

Subjects and apparatus

The subjects were 24 naïve female Wistar rats of the same age and stock as those in Experiment 1. Their housing and maintenance conditions, as well as the apparatus, were also the same. Two reinforcers were used: the 45-mg grain-based rodent food pellet used in Experiment 1 as well as a 45-mg sucrose pellet (5-TUT: 1811251; TestDiet, Richmond, IN, USA).

Procedure

The animals received two sessions of training every day. The first session began with approximately 15 h of illuminated colony time remaining, whereas the second began with approximately 13 h of illuminated time remaining. As before, the start of each session was indicated by insertion of the lever(s). All sessions were 30 min in duration, and with the lever(s) being retracted at the end.

Magazine training

All animals received two magazine training sessions, one with each type of reinforcer, on the day immediately prior to beginning Phase 1. Pellets were delivered during each session (with both levers retracted) on an RT 30-s schedule, resulting in approximately 60 pellets delivered during each session. Pellet type was counterbalanced so that half the rats were trained first with their O1 pellet (further counterbalanced as grain or sucrose) and the other half were trained first with their O2 pellet (also counterbalanced as sucrose or grain).

R1 acquisition (Phase 1)

All animals then received 12 sessions of instrumental conditioning of R1, which was the left lever for half the animals and the right lever for the other half. In all sessions, presses on R1 delivered O1 on a VI 30-s schedule of reinforcement.

R1 extinction and R2 acquisition (Phase 2)

All rats then received 12 sessions in which R1 was extinguished and R2 was reinforced. Both the left and the right levers were inserted throughout each session. R1 presses had no scheduled consequences, but presses on the newly inserted second lever (R2) earned O2 on a VI 30-s schedule of reinforcement.

Resurgence test (Phase 3)

The next day, all animals received a single test session with both levers inserted. For all groups, R1 and R2 responses had no programmed consequences (i.e., were extinguished). Group None received no other events during the test. Group O1 received noncontingent deliveries of O1 (delivered previously in Phase 1) on an RT 30-s schedule throughout the test. Group O2 received noncontingent delivery of O2 (previously delivered in Phase 2) on an RT 30-s schedule throughout the test.

Data analysis

As in Experiment 1, ANOVAs were used to assess response rates throughout the experiment, with a criterion of p < .05 for all analyses.

Results

The main results of the experiment are summarized in Fig. 2. R1 responding was readily acquired in Phase 1, when O1 served as the reinforcer, and extinguished in Phase 2. R2 responding was also readily acquired in Phase 2, when O2 now served as the reinforcer. In the resurgence test, we found an increase in R1 responding in Groups None and O1, but Group O2 showed no change in R1 performance from extinction.

Fig. 2
figure 2

Results of Experiment 2. Upper panels: Mean R1 responding (± SEMs) during acquisition (left), extinction (middle), and testing (right). Lower panels: Mean R2 responding (± SEMs) during its acquisition and testing. All groups received reinforcement of R1 with one outcome (O1) and then reinforcement of R2 with a second outcome (O2) while R1 was extinguished. During the test, rats received extinction of both R1 and R2 with response-independent presentations of either O1, O2, or no reinforcers (Group None). Note the changes in the y-axes between panels; error bars are only appropriate for between-group comparisons

Full size image

R1 acquisition

The acquisition of R1 responding is shown in the upper left of Fig. 2. A 3 (Group) × 12 (Session) ANOVA revealed a main effect of session, F(11, 231) = 54.61, MSE = 49.17, p < .001, η p 2 = .72, but no main effect of group or group by session interaction, Fs < 1.

R1 extinction and R2 acquisition

R1 extinction is shown in the upper middle of Fig. 2. A 3 (Group) × 12 (Session) ANOVA uncovered a main effect of session, F(11, 231) = 19.81, MSE = 6.23, p < .001, η p 2 = .49, but no group effect, F < 1, or interaction, F(22, 231) = 1.19, MSE = 6.23, p > .05. A similar ANOVA on R2 acquisition (lower middle panel of the figure) also revealed a main effect of session, F(11, 231) = 41.75, MSE = 72.57, p < .001, η p 2 = .67, but no group effect or interaction, Fs < 1.

Resurgence test

As in Experiment 1, to assess resurgence, a 2 (Session) × 3 (Group) ANOVA compared the groups’ R1 responding on the last 15 min of Phase 2 and the first 15 min of the resurgence test (Phase 3). A main effect of session, F(1, 21) = 10.87, MSE = 11.58, p < .01, η p 2 = .34, indicated an overall increase in responding, but there was no main effect of group, F < 1. The group by session interaction fell short of the conventional rejection criterion, F(2, 21) = 2.95, MSE = 11.58, p = .075, η p 2 = .22. Planned pairwise comparisons revealed a significant increase in R1 responding in Group None, F(1, 21) = 10.64, p < .01, η p 2 = .37, as well as in Group O1, F(1, 21) = 6.12, p < .05, η p 2 = .23. In contrast, Group O2 showed no change in performance, F < 1. No group differences were detected during the test, ps > .05, although we did observe a trend toward significance when comparing Group None and Group O2, p = .08. A supplemental analysis of R1 responding in the first 5 min of the test (and the last 5 min of extinction) supported the same conclusions. Here, Group None responded with rates of 0.2 and 5.3 responses/min in the last 5 min of extinction and the first 5 min of testing, whereas Group O1 had corresponding rates of 1.1 and 6.8 and Group O2 had rates of 1.8 and 1.3. Once again, a main effect of session was apparent, F(1, 21) = 10.86, MSE = 13.09, p < .01, η p 2 = .34, but no main effect of group, F(2, 21) = 1.53, MSE = 15.18, p > .05. Here, the group by session interaction was significant, F(2, 21) = 3.63, MSE = 13.09, p < .05, η p 2 = .26. As before, Group None, F(1, 21) = 8.10, η p 2 = .28, p = .01, and Group O1, F(1, 21) = 9.93, p < .01, η p 2 = .32, showed significant increases in R1 responding between extinction and testing, whereas Group O2 did not, F < 1. Group O2 responded less than Group O1 throughout the first 5 min of the test, p < .05. No other group differences were detected.

As we noted in the previous experiment, the increase in responding between the last extinction session and the resurgence test could have been due in part to spontaneous recovery. A similar 3 (Group) × 2 (Session) ANOVA on responding during the last 15 min of Session 11 (the second-to-last Phase 2 session) and the first 15 min of Session 12 (the last session of Phase 2) also revealed a significant effect of session, F(1, 21) = 10.29, MSE = 1.67, p < .01, η p 2 = .33. But, importantly, the Group × Session interaction did not approach reliability, F(2, 21) < 1. Thus, the differential effects of the group treatments again were restricted to the resurgence test (Sessions 12 vs. 13). The mean responding rates in the last 15 min of Session 9 and the first 15 min of Session 10 were 0.8 and 1.5, respectively.

Regarding R2 responding during the resurgence test (lower right of Fig. 2), a 2 (Session) × 3 (Group) ANOVA run to assess R2 responding from the last 15 min of Phase 2 to the first 15 min of Phase 3 showed a significant main effect of session, F(1, 21) = 27.95, MSE = 225.60, p < .001, η p 2 = .57, but no main effect of group, F(2, 21) = 1.40, MSE = 624.33, p > .05, nor a group by session interaction, F < 1. A 2 (Session) × 3 (Group) ANOVA was run to assess responding on R2 from the last 5 min of extinction to the first 5 min of test. The mean response rates during the last 5 min of Phase 2 were 61.5, 62.7, and 48.6 responses per minute for Group None, Group O1, and Group O2, respectively, and 30.3, 35.6, and 27.0 responses per minute during the first 5 min of the test. Although there was a significant main effect of session, F(1, 21) = 26.75, MSE = 318.83, p < .001, η p 2 = .56, we found no main effect of group, as well as no significant interaction, Fs < 1.

Discussion

As expected, removal of O2 during the test caused a resurgence of R1 responding in Group None. In contrast, the noncontingent presentations of O2 in Group O2, which theoretically served to maintain the “context” associated with R1’s extinction, prevented resurgence from occurring. Even more important, presentation of a different reinforcer in Group O1 allowed responding to recover. Evidently, specific presentation of O2 is necessary to inhibit or suppress R1 responding. The results are not anticipated by the Shahan–Sweeney model, and instead suggest a role for the discriminative properties of O2 in controlling the inhibition of R1 performance. The results further extend the work of Lieving and Lattal (2003), who found that noncontingent reinforcers presented in the test attenuated the resurgence effect. The results of the present experiment suggest that this effect depends on the reinforcer being the one that was specifically presented during R1’s extinction.

One issue in Experiments 1 and 2 is whether the response recovery observed during resurgence testing was due to spontaneous recovery of the R1 response that might result from the passage of time between sessions rather than the removal of R2 reinforcement. To our knowledge, no experiment on resurgence has included a spontaneous recovery control that receives another session of the Phase 2 treatment during testing (see Bouton & Schepers, 2014; Cançado & Lattal, 2011; Kincaid, Lattal, & Spence, 2015; Lieving & Lattal, 2003; Schepers & Bouton, 2015; Sweeney & Shahan, 2013b; Winterbauer & Bouton, 2010, 2011, 2012). Our analyses of the results of both experiments indicated that although R1 responding did increase as a function of time between the last two sessions of Phase 2, the crucial differences between the groups that were present during resurgence testing were not present then. Thus, the effects of the groups’ treatments were linked to resurgence testing. That observation, along with the present finding that noncontingent O2 presentations during testing prevented response recovery, strongly suggests that the recovery observed during resurgence testing was at least partly due to the removal of the reinforcers delivered in Phase 2.

It is notable that presentations of O1 during testing (Group O1) did not increase R1 above the level observed in the group that received no reinforcers at all (Group None). Since O1 had been a feature of Phase 1, when R1 had been reinforced, it would have been possible to observe R1 at a level higher than that in Group None. Such a result, analogous to reinstatement after extinction (e.g., Baker et al., 1991; Franks & Lattal, 1976; Reid, 1958; Rescorla & Skucy, 1969), would have been expected if the rats had received simple extinction of R1, without additional reinforcement of R2 with O2 (Delamater, 1997). However, in the resurgence design used here, reinforcers were presented in both Phases 1 and 2, and this is not true of simple extinction. It is known that presenting the reinforcer in extinction can eliminate the reinstatement effect it produces when it is presented again after extinction (e.g., Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011). Consequently, if there were some degree of generalization between the present O1 and O2, which would not be surprising given that both were 45-mg food pellets with similar motivational attributes, then any stimulus features common to both O1 and O2 would be associated with the inhibition of R1 during Phase 2. Presentation of O1 should therefore have less impact when extinction involves O2 reinforcers than when it occurs without them. According to this analysis, the lack of reinstatement by O1 was due to O2’s presence in extinction. The next experiment was therefore designed to explore this possibility.

Experiment 3

In Experiment 3, rats again received reinforcement of R1 with O1 in an initial training phase. Then, in a second phase, R1 was simply extinguished, without reinforcement of a new behavior (R2). There were two groups. Group O2 received extinction of R1 in the presence of response-independent O2 presentations delivered on a VT 30-s schedule. Although O2 was presented noncontingently, it was presented at the same rate at which it had been earned by all groups in the resurgence design of Experiment 2, and it was expected to have a similar effect (see Winterbauer & Bouton, 2010, Exp. 4). The second group of the present experiment, Group Ext, received simple R1 extinction without any presentations of O2. In the test phase, both groups were tested for responding under three conditions presented in a counterbalanced order: one in which no pellets were delivered, one in which O2 was delivered noncontingently, and one in which O1 was delivered noncontingently. If O1 and O2 control behavior according to their discriminative properties, we should once again expect a recovery of R1 responding in the O2 group when O2 was removed. In contrast, presenting no reinforcers during the test should have little effect on the performance in Group Ext. However, Group Ext should show reinstatement of extinguished R1 responding when O1 was presented during the test. And if the argument made above is correct, and some generalization did indeed occur between O2 and O1, then we might find comparatively little of a reinstating effect of O1 presentations after extinction in Group O2.

Method

Subjects and apparatus

The subjects were 36 naïve female Wistar rats of the same age and stock as those in Experiments 1 and 2, housed and maintained under the same conditions. The two reinforcers (grain-based and sucrose-based pellets) from Experiment 2 were also used in this experiment.

Procedure

Throughout magazine training and acquisition, the subjects received two daily 30-min sessions, as in Experiments 1 and 2. During extinction, the subjects received one daily 30-min session. All sessions began with left lever insertion after a 2-min delay following introduction to the experimental chamber, and ended with lever retraction. The right lever remained retracted throughout the entire experiment.

Magazine training

Magazine training proceeded exactly as in Experiment 2. Each animal was assigned to an experimental chamber and then given one session with its O1 reinforcer (grain or sucrose pellet, counterbalanced) delivered noncontingently on an RT 30-s schedule and another session with its O2 reinforcer (sucrose or grain pellet, counterbalanced) delivered the same way. Sessions were counterbalanced so that half the animals received O1 first and the other half received O2 first.

Acquisition

Over the next six days, the rats received 12 sessions in which responding on the left lever was reinforced with O1 according to a VI 30-s reinforcement schedule.

Extinction

Animals were then divided into two groups (ns = 18, a number that allowed for counterbalancing the test order; see below). For the next eight daily sessions, responding on the left lever had no scheduled consequences (i.e., it was extinguished). For the animals in Group O2, however, O2 reinforcers were delivered independent of responding according to an RT 30-s schedule throughout the session and during the 2-min delay before lever insertion. The animals in Group Ext received no O2 reinforcers. In order to control for exposure to O2, each animal in Group Ext was matched to an animal in Group O2 and received the same number of O2 reinforcers, but this time in a food cup in the homecage 3.5–5 h following the day’s session. The rats in Group O2 were given equal exposure to an empty food cup in the homecage at this time.

Test

On the final day of the experiment, all rats were given three 5-min test sessions. In the O1 test, O1 was delivered on an RT 30-s schedule throughout the 5-min session and the 2-min delay. In the O2 test, O2 reinforcers were delivered in the same way. In the None (N) test, no reinforcers were delivered. The testing order was counterbalanced, so that three animals in each group received tests in the order O1O2N, O1NO2, O2O1N, O2NO1, NO1O2, and NO2O1. The interval between tests was approximately 60 min.

Data analysis

As before, ANOVAs were conducted to assess responding, with a rejection criterion of p < .05. Two animals in Group O2 failed to extinguish by the end of extinction relative to the other animals (final day of extinction: Ms = 9.13 and 8.33, Zs ≥ 3.10; Field, 2005) and were therefore excluded from all analyses. One animal in Group Extinction was also excluded from all analyses because she was a significant group outlier during the None test (Z = 3.44; Field, 2005).

Results

The results of the experiment are summarized in Fig. 3. As is shown at top, both groups acquired R1 in the first phase and then decreased their performance in extinction. The presentation of noncontingent O2 reinforcers during extinction initially seemed to suppress, but then significantly enhanced, the rate of responding in extinction relative to Group Ext. During the final tests, only Group O2 showed a significant resurgence effect when tested without pellets in the None condition, and only Group Ext, which received simple extinction (without O2 presentations), showed a significant reinstatement effect when given the O1 pellet noncontingently in the O1 condition.

Fig. 3
figure 3

Results of Experiment 3. (A) Mean responding (± SEMs) during acquisition and extinction of the operant response. During acquisition, responding led to one outcome (O1). In extinction, one of the groups (O2) had response-independent presentations of a second outcome (O2), whereas the other group (Ext) received simple extinction. (B) Mean responding (± SEMs) during testing. The three panels compare the groups’ responding during the last session of extinction (Ext 8) with responding during tests in the presence of response-independent presentations of O1 (O1 Test, left), O2 (O2 Test, center), or no reinforcers at all (None Test, right). Note the changes in the y-axes between the panels in A; error bars are only appropriate for between-group comparisons

Full size image

Acquisition

All animals increased responding throughout acquisition (shown in the top left panel of Fig. 3), as we confirmed with a 2 (Group) × 12 (Session) ANOVA that revealed a significant main effect of session, F(11, 241) = 79.93, MSE = 31.60, p < .001, η p 2 = .78. Neither the main effect of group nor the interaction was significant, Fs < 1.

Extinction

Responding in extinction is shown in the top right panel of Fig. 3. Both groups decreased responding throughout the extinction phase. However, the suppression of performance was slower in Group O2. This was confirmed by a 2 (Group) × 8 (Session) ANOVA that revealed a main effect of session, F(7, 217) = 68.07, MSE = 9.15, p < .001, η p 2 = .69, and a significant group by session interaction, F(7, 217) = 4.46, MSE = 9.15, p < .001, η p 2 = .13. There was no main effect of group, F(1, 31) = 1.72, p = .20. On the final day of extinction, Group O2 was still significantly elevated relative to Group Extinction, F(1, 31) = 6.68, MSE = 1.34, p < .05, η p 2 = .18.

Test

The results of the test are shown across the bottom three panels of Fig. 3. Separate ANOVAs were conducted to assess the change in responding from the last day of extinction to each test. In the O1 test (shown in the bottom left panel of Fig. 3), a 2 (Group) × 2 (Session) ANOVA revealed both a significant main effect of session, F(1, 31) = 6.67, MSE = 2.56, p < .05, η p 2 = .18, and a significant group by session interaction, F(1, 31) = 4.13, MSE = 2.56, p = .05, η p 2 = .12. We found no main effect of group, F < 1. Pairwise comparisons revealed that whereas Group O2 did not change its performance from the last day of extinction, F < 1, Group Ext showed a significant increase in responding, F(1, 31) = 10.98, p < .01, η p 2 = .26. In the O2 test (shown in the bottom middle panel of Fig. 3), a similar ANOVA revealed no main effect of session, no main effect of group, and no interaction between the two, largest F = 2.46, p = .13. Follow-up comparisons showed that neither Group O2, F < 1, nor Group Ext responded significantly differently between sessions, although there was a trend in Group Ext, F(1, 31) = 3.52, p = .07, η p 2 = .10. In the None test, during which no reinforcers were delivered (right bottom panel of Fig. 3), we observed a significant main effect of session, F(1, 31) = 16.97, MSE = 3.39, p < .001, η p 2 = .35, a significant main effect of group, F(1, 31) = 7.66, MSE = 8.65, p < .01, η p 2 = .20, and a significant interaction between the two, F(1, 31) = 4.65, MSE = 3.29, p < .05, η p 2 = .13. Pairwise comparisons revealed that whereas Group O2 showed a significant increase in responding from the last day of extinction to the None test, F(1, 31) = 19.12, p < .001, η p 2 = .38, Group Ext did not, F(1, 31) = 1.98, p = .17. Additionally, the animals in Group O2 responded significantly more than did the animals in Group Ext during this test, F(1, 31) = 6.85, p = .01, η p 2 = .18.

Discussion

After R–O1 training, the rats in this experiment received either simple extinction (Group Ext) or extinction plus response-independent presentations of O2 (Group O2). The effect of presenting O2 in extinction was primarily to slow the loss of responding. However, in the tests that followed extinction, the main findings were (1) that Group O2 showed an increase in responding when tested without pellets (and Group Ext did not), and (2) that Group Ext showed an increase in responding when tested with O1 (whereas Group O2 showed less of this effect). These results are consistent with the idea that O1 and O2 control responding through their discriminative properties. In addition, other results suggest that some partial generalization took place between the pellets that served as O1 and O2 here. As we noted above, noncontingent O2 in extinction slowed the loss of responding. It is already known that presentations of the same reinforcer slow response loss in extinction (Baker, 1990) and reduce the reinstating effect of that reinforcer (Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011). Such findings are often explained by acknowledging that the reinforcer sets the occasion for the next response (e.g., Ostlund & Balleine 2007; Reid, 1958); O2 might have done so in the present experiment because it was not discriminated perfectly from O1. A second result suggesting generalization between O1 and O2 is that O2 caused a tendency (p = .07) to reinstate responding when it was presented noncontingently after extinction in Group Ext. Either result is consistent with the idea that there was some partial generalization between the two food pellets that provided O1 and O2. The results are not compatible with the idea that reinforcer presentations suppress or disrupt responding (e.g., Shahan & Sweeney, 2011). Indeed, in order to make the augmenting effects of noncontingent reinforcer presentations that we observed in both extinction (Group O2) and testing (Group Ext) more compatible with behavioral momentum theory, it would be necessary to implement a different model (Podlesnik & Shahan, 2010).

We would note that the response-enhancing effects of O1 and O2 presentations after extinction could be consistent with other views—for example, that reinforcer presentations could increase the incentive motivation for operant responding (e.g., Stewart, de Wit, & Eikelboom, 1984). However, the challenge for any such view would be to explain simultaneously the response suppressing effects of O2 that were revealed during testing of Group O2 in the None condition, when O2 associated with extinction was removed, or in the basic resurgence design (e.g., Exp. 2). The simplest account of the overall pattern may be to accept a role for the discriminative properties of O2.

A somewhat surprising finding was that O1 did not cause as much recovery of R1 responding in Group O2 as it had in an analogous group (Group O1) in Experiment 2. In contrast to Experiment 2, in the present experiment O2 was delivered independently of an explicit response during R1 extinction, and the effects of O1, O2, and no reinforcer presentations were then tested within subjects. It is not clear why these differences would cause the discrepancy observed here. It is worth noting that O1 might have a neophobic effect when it is relatively unexpected; recall that the rats had been exposed to O2 but not to O1 for the last eight days. The possibility that a relatively novel reinforcer can suppress behavior may also be consistent with the fact that O2 presentations initially appeared to suppress responding in Group O2 at the start of extinction, although that trend was not statistically significant.

General discussion

The results of the present experiments provide new information that may be relevant for understanding resurgence. In Experiment 1, when R2 was reinforced during the response elimination phase with different VI schedules that varied systematically from VI 30 s to VI 120 s, the amount of resurgence was linearly related to the reinforcement rate (and inversely related to the average interval between reinforcers). In Experiment 2, when R1 was reinforced with one reinforcer (O1) and R2 was reinforced with a different reinforcer (O2) during response elimination, resurgence was prevented when O2, but not O1, was presented noncontingently during testing. And in Experiment 3, the presence of O2 reinforcers during extinction reduced the ability of O1 to reinstate behavior above no reinforcer delivery, suggesting that some generalization took place between the O1 pellet context and the O2 pellet context. Overall, the results support the idea that in resurgence, R1 responding is influenced by the discriminative properties of the reinforcers that are present during response elimination.

The results are generally consistent with the contextual account of resurgence (e.g., Bouton et al. 2012a; Winterbauer & Bouton, 2010). That view accepts what may be the inherent context specificity of extinction and suggests that resurgence might occur when the context created by reinforcement for R2 changes between response elimination and extinction testing. In Experiment 1, the approach predicted that leaner reinforcement schedules in the response elimination phase would allow the rat to learn R1 extinction under sparse reinforcement conditions that increasingly resembled the extinction conditions (context) of testing. In Experiment 2, the account predicted that only noncontingent presentations of the reinforcer used during response elimination training (O2) would suppress R1, and hence reduce the resurgence effect. O1 was contrastingly ineffective because it had not been associated, as O2 had, with the extinction of R1. The results of Experiment 3 further suggest some generalization between the O1 context and the O2 contexts, and less generalization to the “no-pellet” (i.e., None) context. Overall, the results emphasize the role of the discriminative (rather than the reinforcing) properties of reinforcers in controlling extinguished R1 responding in the resurgence paradigm.

As we have noted throughout the article, the results are less consistent with the quantitative model proposed by Shahan and Sweeney (2011) that extended behavioral momentum theory. Although the model correctly predicts the results of Experiment 1, in which the amount of resurgence was directly related to the rate of reinforcement for R2 during response elimination, it does not anticipate the results of Experiments 2 or 3. In Experiment 2, in which noncontingent O2 but not O1 presentations during testing prevented resurgence, the model fell short because, in keeping with preceding versions of behavioral momentum theory (e.g., Nevin & Grace, 2000), it does not discriminate between the effects of qualitatively different reinforcers (see Grimes & Shull, 2001; Shahan & Burke, 2004). In Experiment 3, reinforcers had an augmenting effect on operant responding, both when O2 was presented during the extinction phase in Group O2 and when O1 was presented during testing in Group Ext; neither finding is consistent with the disruption emphasized by the momentum-based model. It may be more parsimonious to accept that reinforcers have stimulus properties that can serve as discriminative cues controlling extinction or conditioned responding, and that such cues may play a role in causing resurgence during the transition between Phase 2 and testing. Without further expansion, the momentum-based resurgence model (Shahan & Sweeney, 2011) cannot implement such a mechanism.

How easy would it be to modify the momentum-based model to handle the results? Recall that the model emphasizes a release from disruption caused by reinforcers delivered at a high rate in Phase 2 in explaining resurgence. To accommodate the present results suggesting that the similarity between the stimulus conditions of Phase 2 and Phase 3 (testing) is important, the model could add a generalization parameter like the one it uses to describe performance in the transition between Phases 1 and 2. If such a parameter allowed the stimulus properties of the reinforcer to play a role (but see Grimes & Shull, 2001; Shahan & Burke, 2004), it could begin to explain resurgence the way the contextual account does (though only by accepting a new mechanism). But emphasizing generalization between Phases 2 and 3 would raise a new question: What is being generalized? The response suppression caused by reinforcement in Phase 2 is not learned—it is conceptualized as an unconditional disruption of performance. Why then should it generalize? Momentum theory gives no role to new learning in extinction, although new learning does occur (e.g., Bouton & Todd, 2014; Rescorla, 1993, 1997; Todd, Vurbic, & Bouton, 2014). Finally, as noted in the introduction, other results have raised unresolved questions about how the model should calculate reinforcement rate in Phase 2 (Bouton & Schepers, 2014; see especially Schepers & Bouton, 2015). Several assumptions and aspects of the momentum-based model will thus need to be modified for it to provide a compelling account of resurgence.

Other aspects of the present results deserve further discussion. First, in Experiment 2, noncontingent presentations of O1 during testing did not reinstate R1 responding above the level seen in a group that received no reinforcers during testing, even though O1 had arguably been part of the context connected with R1’s original acquisition. This result was replicated in Experiment 3, in which, using a simple extinction procedure, rats that were given free O2 pellets during extinction also failed to show reinstatement of responding by O1. At first glance, both results seem inconsistent with the context hypothesis, and indeed with studies of reinstatement, in which presentations of reinforcers associated with conditioning readily reinstate extinguished responding. However, unlike simple extinction experiments, resurgence experiments involve reinforcer presentations during the sessions in which the target response (R1) is extinguished. Similarly presenting the reinforcer during simple extinction can abolish subsequent reinstatement (e.g., Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011) because it might undermine the reinforcer’s unique association with conditioning. Such a factor was potentially at work in the present experiments to reduce reinstatement by O1: As we showed in Experiment 3, animals without O2 present during extinction (Group Ext) did show a significant reinstatement effect when tested with O1. In addition, it is worth noting that presentation of O1 during testing in Experiments 2 and 3 theoretically created a less complete or dramatic change of background context than the no-reinforcer conditions with which they were compared. This is because the common properties of O1 and O2 (they were both food pellets) would have decreased the contextual change from response elimination to testing relative to a group that received no pellets at all during testing. It thus may not be surprising to observe so little in the way of reinstatement-like results in Experiments 2 and 3 in animals given reinforcement during response elimination. The reinforcement conditions that prevail in extinction appear to play an especially important role in inhibiting R1 performance.

In summary, the present results provide further support for a role for the discriminative properties of the reinforcer in producing resurgence (see also Schepers & Bouton, 2015). When R1 is extinguished while R2 is being reinforced, the animal learns to inhibit R1 in the context of R2 reinforcement. When this context is changed, as it is when the R2 reinforcers are discontinued, or when the identity of the reinforcer is changed (Exp. 2), R1 may recover because extinction is relatively specific to the context in which it is learned. The results may have translational implications for reducing relapse after so-called contingency management (CM) treatments in which unhealthy behaviors such as smoking and drug taking are suppressed, in part, by the reinforcement of alternative behavior (e.g., Higgins, Heil, & Lussier, 2004; Petry, Martin, Cooney, & Kranzler, 2000). Although CM is effective at reducing the target behavior (Heil, Yoon, & Higgins, 2008; Petry et al., 2000), the behavior is susceptible to relapse when reinforcement is discontinued (Higgins, Silverman, & Heil, 2008; Silverman, Kaminski, Higgins, & Brady, 2011), perhaps resembling the resurgence effect. The present results, along with others, suggest that relapse may be prevented if one can make sure that the conditions of treatment generalize to those situations in which relapse may occur. This is perhaps the most general implication of context-based accounts of extinction and voluntary response inhibition (e.g., Bouton, 2014).