Abstract
Resistance to punishment is commonly used to measure the difficulty in refraining from rewarding activities when negative consequences ensue, which is a hallmark of addictive behavior. We recently developed a progressive shock strength (PSS) procedure in which individual rats can titrate the amount of punishment that they are willing to tolerate to obtain food rewards. Here, we investigated the effects of a range of delays (0–12 s) on resistance to punishment measured by PSS break points. As expected from delay discounting principles, we found that delayed shock was less effective as a punisher, as revealed by higher PSS breakpoints. However, this discounting effect was not equally distributed in the population of rats, and the introduction of a delay highlighted the existence of two populations: rats that were sensitive to immediate punishment were also sensitive to delayed shock, whereas rats that were resistant to immediate punishment showed strong temporal discounting of delayed punishment. Importantly, shock-sensitive rats suppressed responding even in subsequent non-punishment sessions, and they differed from shock-resistant rats in anxiety-like behavior, but not in sensitivity to pain. These results show that manipulation of temporal contingencies of punishment in the PSS procedure provides a valuable tool to identify individuals with a double vulnerability to addiction: low sensitivity to aversion and excessive discounting of negative future consequences. Conversely, the shock-sensitive population may provide a model of humans who are vulnerable to opportunity loss due to excessive anxiety.
Similar content being viewed by others
Introduction
Drug addiction is characterized by excessive motivation to consume the drug and continued use in spite of negative consequences [1]. The motivational aspects of addiction and the role of the reward system have been extensively studied [2,3,4,5,6,7,8]; however, the investigation of the behavioral and neurobiological mechanisms underlying the inability to stop drug use in the face of adverse consequences has attracted the attention of the field only in the last decade [9,10,11,12,13,14]. Whereas these processes are intertwined, behavioral and neurobiological evidence suggests that they depend on partially distinct mechanisms [14,15,16].
In animal studies, insensitivity to negative outcomes has been mostly investigated using punishment procedures in which actions leading to a reward are contingently paired with electrical footshocks [9,10,11,12,13]. This punishment produces learning processes that result in the reduction of seeking and taking behaviors [16]. Importantly, differences in sensitivity to punishment have been used as a measure of transition from controlled to uncontrolled drug use and are considered as a marker of addiction [11]. Obtaining a deeper knowledge of the behavioral and neurobiological basis of resistance to punishment is critical for better interpretation of results obtained using these procedures and for understanding the role of punishment processes in psychiatric disorders such as addiction [13, 16].
The emotional impact of events that occur in the future is discounted (i.e. their perceived value decreases as a function of time) [17,18,19]. Differences exist among individuals in the rate of discounting, and individuals who discount rewards more rapidly are considered impulsive and at risk of developing psychiatric disorders such as addiction [18]. Indeed, the negative consequences of addictive behaviors often occur well after the execution of the actions leading to the consumption of the object of addiction. Therefore, individuals who discount punishment too rapidly may not be able to appropriately consider the future consequences of their actions and may be particularly vulnerable to addictive disorders [20, 21]. Importantly, punishments are temporally discounted in much the same way that rewards are less effective when delayed [19]. That is, people [22, 23] and animals [24,25,26,27] are more willing to accept negative consequences when they occur after a delay. Therefore, as previously suggested [13, 16], it is important to investigate the behavioral consequences of introducing a delay between a reward-seeking action and punishment.
We have recently developed a procedure to investigate punishment in rats that, similarly to what is commonly done in humans [28,29,30], individually calibrates the strength of punishment based on the animal’s behavior [31]. This self-adjusting progressive strength of shock (PSS) procedure allows obtaining PSS breakpoints that quantify individual resistance to punishment [31]. Importantly, compared to other punishment procedures [32,33,34,35,36,37,38,39,40,41], the PSS procedure reduces exposure to high levels of shock that could be particularly aversive and persistently affect operant behavior [41], and it, therefore, could be considered a refinement in the 3Rs (replacement, reduction, refinement) principles of animal research [42]. Resistance to punishment in this procedure is sensitive to manipulations of motivation and appears to be both a trait (i.e. PSS breakpoints are highly correlated under a wide variety of conditions) and a state (i.e. PSS breakpoints are influenced by motivational states) [31]. Characterizing behavior in the PSS procedure may help understanding the mechanisms underlying resistance to punishment.
In this study, we used the progressive shock strength procedure to investigate the effects of a range of delays (0–12 s) on the PSS break points. In brief, after training in a fixed-ratio 1 (FR 1) food procedure, rats were tested once per week for resistance to punishment in the PSS procedure or motivation for food in a progressive ratio (PR) procedure. Each delay was tested at least twice, and at the end of operant testing, we assessed anxiety-like behavior and pain sensitivity.
Material and methods
Forty-eight male Sprague-Dawley rats aged 8-9 weeks (Janvier Labs, France), experimentally naive at the beginning of the study, were used in this study. The rats were divided into 2 cohorts of 24 rats which differed in the order of delay presentation. All experiments were conducted during the light phase and in accordance with European Union directives (2010/63/EU) for the care of laboratory animals and approved by the local ethics committees (COMETHEA).
Food restriction
During operant procedures and until the end of the experiment, animals underwent food restriction to limit weight gain and to maintain operant behavior. Food (approximately 20 g/day) was given 1 h after the end of the experimental sessions, and rats had unlimited access to water for the entire duration of the experiment.
General experimental designs
Fig. S1 shows the experimental design for the two cohorts of rats. After 9 training sessions under fixed-ratio 1 (FR1) schedules, rats underwent PSS sessions with varying delays between animals’ lever presses and punishment delivery. For the first 3 PSS sessions, on alternate weeks, we also measured responding in the progressive ratio schedule. Afterward, rats underwent only PSS test sessions, interspersed with normal FR1 schedules. The delay was fixed for a given session. Each delay was tested for 2 to 4 sessions. Tests were performed once per week, every fifth session (normally on Fridays).
Three different delays (3, 6, and 12 s) were tested. The range of delays was chosen based on previous work showing that the ability of rats to learn the association between their behavior and a motivational stimulus depends on the temporal contingency of the response and the stimulus and that beyond 8 s this learning is impaired [43] and aversion to footshock decreases to about 25% of initial value in a temporal discounting procedure [26].
In the first cohort of 24 animals, delays were presented in the following order: 0 s (x3 sessions), 3 s (x2), 12 s (x3), 6 s (x3). To rule out an effect of the order of delays on the effects of shock, delays were presented in a different order in the second cohort of 24 animals: 6 s (x2), 3 s (x2), 0 s (x2), 12 s (x2). 24–72 h after the last operant sessions, we measured anxiety-related behaviors in an open field and pain reactivity in the hot plate.
Since behavior did not differ significantly between the two cohorts, we combined the results.
Food reinforcement apparatus and training procedure
Experimental chambers (MedAssociates, www.medassociates.com) were enclosed individually in sound-attenuation chests. Each experimental chamber had a recessed food tray, and two levers in the right wall. The floor consisted of bars that were connected to shockers (MedAssociates, ENV-414SA) that could deliver footshock, with an electric current set to 0.45 mA. Each chamber was equipped with a food-pellet dispenser, which could deliver 45 mg pellets to the food tray. Experimental events were controlled by computers using the MedAssociates interface and Med-PC IV software; the Med-PC code used to conduct the procedures is available upon request. A diode light was present on each lever. One lever was assigned to be the active lever and the corresponding light was used as a conditioned stimulus for food reinforcement. A third diode light was installed on the opposite wall, and its flashing was used as a discriminative stimulus to indicate that food reinforcement would be associated with a foot shock.
The general training schedule involved 45-minute sessions of a schedule of food reinforcement in which each lever press (FR1) produced a 45-mg sucrose pellet. During these sessions, food availability was signaled by turning off the house light, and delivery of food was accompanied by flashing of the diode light above the lever for 2 s. Subsequently, the house light was turned on for an additional 18-sec time-out period, during which responding had no programmed consequences. Following the time out, a new trial started and the next response on the right lever was again reinforced. Responses on the inactive lever were recorded but never reinforced. Rats initially learned to respond for food during nine sessions under this schedule.
Self-adjusting progressive punishment procedure
The self-adjusting progressive shock strength (PSS) procedure was the same as described by Desmercieres et al (2022). In brief, active lever presses resulted in the delivery of food rewards and foot-shocks of fixed intensity and variable duration. The self-adjusting procedure consisted of steps in which the shock duration was increased if the animal completed 2 trials in the previous step. The duration of the first step was 0 s (no punishment), the second step was a low duration of 0.05 s and subsequent shocks increased at each step for 20 steps. The durations of the steps were: 0, 0.05, 0.06, 0.07, 0.08, 0.09, 0.10, 0.12, 0.13, 0.15, 0.18, 0.20, 0.23, 0.27, 0.31, 0.35, 0.41, 0.47, 0.54, 0.62, 0.71 s. If animals reached the final step, the duration of the shock was not further increased, and all subsequent shocks were set at 0.71 s. If rats did not emit any response for 5 min, shock duration was reset to 0 and the shock progression was reinitialized. The strength of the shock was measured by the electrical charge in millicoulombs (mC) that an animal was willing to receive to self-administer food pellets and was calculated by multiplying the fixed current of the shock (0.45 mA) by the duration in sec. The breakpoint was calculated by the total intensity of the shocks received during the session. We chose this parameter for the main analysis because it incorporates both the willingness to receive a given charge unit and the willingness to restart responding after an eventual punishment-induced pause.
Progressive-ratio schedule
Under the progressive-ratio schedule of food reinforcement, the number of responses required to obtain a food pellet increased with each successive food pellet. The steps of the exponential progression were the same as those previously developed by Roberts and colleagues [44] adapted for food reinforcement [45, 46], based on the equation: response ratio = (5e(0.2 × reinforcer number)) − 5, rounded to the nearest integer. Thus, the values of the steps were 1, 2, 4, 6, 9, 12, 15, 20, 25, 32, 40, 50, 62, 77, 95, 118, 145, 178, 219, 268, 328, 402, 492, 603, and 737. Sessions under the progressive-ratio schedule lasted until 10 min passed without completing a step, which typically occurs within 1 h.
Anxiety-related behaviors: open field
The open-field apparatus (Viewpoint, Lyon, France) consisted of a rectangular arena (50 cm wide * 50 cm long * 40 cm high) of white plexiglass. After a 30-min habituation to the experiment room, rats were placed in the arena for 30 min. Their positions were recorded automatically by a camera and video tracking software (Viewpoint, Lyon, France). The software-defined a virtual square (25 cm * 25 cm) delimiting the center zone and the border zone. Anxiety-like behavior was measured as the percentage (%) of time spent in the center (time in the center/time in the border + time in the center * 100) so that more time spent in the center indicated a lower level of anxiety-like behavior.
Pain: hot plate test
The hot plate (Ugo Basile, model-DS 37) was maintained at 48 °C [47]. After a 10-min habituation to the experiment room, animals were placed into a glass cylinder with 25 cm diameter heated surface and 47 cm walls. The latency before escape or jumping was recorded. Experiments were stopped after a cut-off of 120 s to prevent unnecessary pain or any tissue damage.
Statistical analysis
Med-PC data were analyzed using custom-made, freely available software written in Python, Med_to_csv (https://github.com/hedjour/med_to_csv) which uses raw data files to create complete tables for further analysis in GraphPad Prism. Data were checked for normality of distribution using the Shapiro–Wilk test. In our previous study, in the absence of a delay, PSS data did not show a normal distribution as a function of the electrical charge variable, but they did after the natural logarithmic transformation of the variable. With the introduction of a delay, PSS data did not show normal distribution even after logarithmic transformation. Therefore, for statistical analysis, we used a nonparametric repeated measure Friedman rank test to analyze the effects of different delays in all rats.
To dichotomize individuals into shock-sensitive or resistant groups we used the median split of the PSS break point measured by the total electrical charge sustained at delay 0 sec. Median splits were obtained in each cohort of rats separately. Whereas these choices are arbitrary, it should be noted that similar results were obtained when using PSS break points at 6 or 12 s of delay or when median splits were calculated by merging the two cohorts.
To analyze differences in baseline at the beginning and at the end of the experiment in all rats and differences in pain sensitivity in shock-sensitive or resistant groups, we used the non-parametric Mann-Whitney test. To analyze the effects of delay in all rats, we used the non-parametric Friedmann test. For the investigation of the effects of delays in shock-sensitive or resistant groups, we used two-way ANOVA for repeated measures (using Geisser-Greenhouse correction when needed to account for possible violation of sphericity), followed by Sidak’s post hoc test. Differences were considered significant when p < 0.05.
Results
Food self-administration under basal conditions
Self-administration of food during the entire experiment is shown in Fig. S2 for the first cohort and Fig. S3 for the second cohort. Rats quickly learned to self-administer food under FR1 schedules. The initial baseline (BL), calculated as the mean ± SEM of responses per session during the three sessions before the first shock, was 121.74 ± 1.53, and the final baseline calculated as the average of the last three sessions was 122.29 ± 2.45. Statistical analysis using the Mann–Whitney test did not reveal significant differences between the two baselines (p = 0.0643).
Effects of delay on PSS break points
Increasing the delay from 0 to 3 did not produce significant increases in PSS break point. Only when the delay was further increased to 6 s did the number of active responses significantly increase, from about 22 at delay 0 s to about 48 at delay 6 and about 51 responses when the delay was set to 12 s (Fig. 1A). The shock strength that rats were willing to receive increased from 0.97 to a maximum 8.17 mC at delay 12 s (Fig. 1B). This suggests that delay decreases the magnitude of the punishment effect. Statistical analysis using the non-parametric Friedman test revealed significant differences in both number of responses (p < 0.0001) and shock strength (p < 0.0001).
Number of active responses (A) and PSS breakpoint (total electrical charge sustained) (B) as a function of the delay between the response (and the food delivery) and the footshock. Data are expressed as mean ± SD of active responses (N = 48). Each data point corresponds to the average of at least 2 sessions at a given delay. Friedman nonparametric post-hoc test for repeated measures: ****P < 0.0001.
Intra-correlation of PSS break points at different delays and inter-correlation with PR breakpoints
One of our hypotheses was that individual differences in delay discounting of aversive consequences could interfere with PSS breakpoints so that animals more resistant to immediate punishment would not necessarily be more resistant after the introduction of a delay. Contrary to our prediction, correlation among PSS breakpoints was very high and significant under all conditions (Fig. S4A; Spearman correlation R2 > 0.65 and p < 0.0001 for all values), suggesting that introducing a delay did not alter the relative sensitivity to punishment.
In the first weeks of the experiment, in parallel with PSS we also measured motivation in a PR procedure (Fig. S1). Consistent with our previous results [31], we found no correlation (Spearman correlation R2 = 0.00025, p = 0.91) between PSS and PR breakpoints (Fig. S4A, B) confirming that these tasks do not measure exactly the same behavioral process [31, 34, 37, 48].
Identification of individuals with high or low resistance to shock-induced suppression
Visual analysis of PSS data in Fig. 1 reveals that introducing a delay in the PSS procedures leads to an increase in the variability of behavior among subjects, with some animals showing increases in PSS with low 3 s delays and others showing no or very little increases in the PSS break point even with 12 s delays. Therefore, we decided to use a median split of PSS break points at a delay of 0 s to classify animals as shock sensitive and shock resistant and to better characterize these subgroups. Day-to-day behavior during the entire duration of the procedure is shown in Fig. 2A for Cohort 1 and Fig. 2B for Cohort 2. A few characteristics deserve to be highlighted. First, behavior in FR1 training session was very similar in the two groups during initial training but after repeated shock sessions, the number of responses in the sensitive groups decreased even on the first training session after the shock, with a slower return to baseline with repeated training sessions (Fig. 2A, B and Fig. S5). This behavior suggests that shock-sensitive rats develop conditioned suppression of operant behavior. Second, the responses of shock-sensitive and shock-resistant rats were clearly different in PSS sessions but were similar in PR sessions.
Number of active responses in all the 84 sessions of the experiment for (A) cohort 1 and (B) cohort 2. It should be noticed that during normal training and shock sessions, each active response produces the delivery of one sucrose pellet whereas in PR sessions each subsequent pellet requires an increasing number of responses. N = 12 per group.
Effects of delay in high and low shock-resistant animals
Body weight did not differ between sensitive and resistant animals (sensitive 375.12 ± SD 48.14 g; resistant 371.99 ± SD 56.35 g, Student T-Test: P = 0.84) and did not correlate to PSS breakpoints (Spearman correlation R2 = 0.035, p = 0.20).
PSS break point as a function of delay in sensitive and resistant animals is depicted in Fig. 3. In both sensitive and resistant rats PSS break point increased with increasing delays; however, resistant rats showed significant increases with delays as low as 3 s whereas sensitive rats showed increases only at 12 s of delay. In addition, resistant rats showed significantly higher break points than sensitive rats at all delays. Statistical analysis revealed an effect of sensitivity to shock (Responses: F (1, 46) = 124.4; P < 0.0001; Electrical charge: F (1, 46) = 42.24; P < 0.0001), of delay (Responses: F (2.328, 107.1) = 43.10; P < 0.0001; Geisser-Greenhouse’s epsilon = 0.78; Electrical charge: F (1.999, 91.95) = 31.32; P < 0.0001; Geisser-Greenhouse’s epsilon = 0.67) and a sensitivity X delay interaction (Responses: F (3, 138) = 13.01, P < 0.0001; Electrical charge: F (3138) = 16.13; P < 0.0001).
PSS break point measured as the number of active responses (A) and total electrical charge sustained (B) as a function of the delay between the response (and the food delivery) and the footshock. Data are expressed as mean ± SD of active responses (Males n = 24 per group). Each data point corresponds to the average of at least 2 sessions at a given delay. Notice that the median split was calculated separately in each cohort of rats which results in some overlapping of sensitive and resistant rats at delay 0. Two-way ANOVA for repeated measures: *, **, *** = P < 0.5, P < 0.001 and P < 0.0001 compared to Delay 0 s (D0); ### = P < 0.0001 compared to shock sensitive rats.
Conditioned suppression of operant behavior
As previously noted, after they started to receive shock training, shock-sensitive animals showed a reduction in the number of responses emitted even in training sessions in which shock was absent. Therefore, we calculated the difference between the day before and the day after the PSS session, and we called this measure the “suppression score”. Whereas resistant rats did not show any sign of suppression at any delay, sensitive rats showed significant conditioned suppression that was similar at all delays (Fig. 4). Statistical analysis revealed a significant effect of sensitivity to shock (Responses: F (1, 46) = 44.92; P < 0.0001), no effect of delay and no significant sensitivity X delay interaction.
The suppression score was calculated as the difference in the number of active responses between the day after (D + 1) and the day before (D-1) the shock session. Data are expressed as mean ± SD of active responses (Males n = 24 per group). Two-way ANOVA for repeated measures: ##, ### = P < 0.001, P < 0.0001 compared to shock-resistant rats.
Anxiety-like behavior and pain
Differences in sensitivity to shock could be due to trait-like differences in pain perception or anxiety. To determine whether these parameters influenced behavior, at the end of the operant sessions, we measured pain in a hot plate test and anxiety-like behavior in an open field. We found that sensitivity to pain did not differ between sensitive and resistant rats (Fig. 5A, Mann–Whitney U = 258.5, P = 0.54) and did not correlate with PSS breakpoints (Spearman correlation R2 = 0.010, p = 0.49; Fig. S6A). On the other hand, anxiety-like behavior measured at the end of the experiments, was higher in sensitive compared to resistant rats (Fig. 5B, Mann–Whitney U = 164.5, P < 0.05) and correlated with PSS breakpoints (Spearman correlation R2 = 0.10, p = 0.027, Fig. S6B).
Pain sensitivity in the hot plate test (A) and anxiety-like behavior in an open field (B). These tests were performed at the end of the experiment. Note that the rats that spend more time in the center of the open field, are considered the less anxious. Data are expressed as mean ± SD of active responses (Males n = 24 per group). Mann–Whitney test: * = P < 0.05, P < 0.0001.
Discussion
In this study, we investigated the effects of introducing a delay between reward and punishment in the PSS procedure [31]. When punishment was delayed, rats were willing to tolerate stronger shocks. Importantly, the introduction of a delay highlighted the existence of two populations of rats: one that not only is resistant to immediate punishment but that also shows stronger discounting of future aversive consequences; and another one that is sensitive to punishment and shows less temporal discounting. Finally, resistance to punishment was not associated with stronger appetitive motivation (as measured by the progressive ratio schedule) or with lower pain sensitivity, but it was negatively correlated with the development of conditioned suppression and anxiety-like behavior.
In humans, the negative consequences of maladaptive behaviors such as addiction are mostly delayed [20, 21]. This delay contributes to the difficulty in adapting behavior by reducing drug seeking and taking because, similar to what is commonly found with the positive value of rewards [49], the negative value of punishment diminishes with the delay [22,23,24,25,26,27]. Consistent with those studies, we find that rats were willing to tolerate higher strengths of footshocks when punishment was delayed compared to those tolerated when punishment was immediate. Also, consistent with previous work showing individual differences in delay discounting [27], we found considerable differences in the effects of delay on punishment. In fact, the rats that were more resistant to immediate punishment were also those that were more sensitive to temporal discounting. Therefore, using delayed punishment in the PSS procedures provides a method for identifying a population that experiences difficulties in refraining from seeking desired food especially when the negative consequences are not immediate, a key symptom of eating disorders associated with uncontrollable food consumption such as binge-eating disorder and bulimia.
An unexpected finding in this paper is that shock-sensitive rats not only show a profound reduction in responding on shock days but also a significant reduction on the following days even in the absence of shock. This suppression disappeared and behavior tended to return to baseline only after several sessions without shocks. Thus, the emotional impact of footshock appeared to be a major contributor to the effects of punishment in this study. Using a procedure that separates pavlovian conditioning from operant punishment, Jean-Richard-Dit-Bressel and colleagues showed that under certain conditions, conditioned fear and punishment can be dissociated [40]. In contrast, in the PSS procedure, pavlovian and operant conditioning are not separated, which causes a strong association between punishment and conditioned suppression. Future studies will be needed to investigate whether behavior in the PSS procedure and the procedure of Jean-Richard-Dit-Bressel overlap and whether these procedures identify the same populations of resistant and sensitive rats.
In humans, some individuals could use drugs while managing to maintain control, but others lose control over their drug-taking and become addicted [50, 51]. However, it is difficult to predict who will develop addiction and who won’t. Animal studies can provide useful information about the factors that may increase the risks of developing addiction. For drugs, after 20–30 sessions but not earlier, resistance to punishment behavior develops in a minority of rats [33, 52]. Conversely, in mice models of food addiction, resistance to punishment using conventional fixed-intensity procedures did not increase, and it even decreased over time [53, 54]. In this study, we tested with the PSS procedure over long periods (4–6 months) and in a high number of sessions (>80 sessions for cohort 1 and >60 sessions for cohort 2), and we found no evidence of a progressive loss of control over food seeking. Indeed, animals identified as shock-sensitive or shock-resistant after a few sessions continued to show similar behavior for the entire duration of the experiment. This suggests that at least for food reward, sensitivity to punishment in the PSS procedure is an individual trait that is not appreciably affected by repeated exposure to food reward.
Previous studies have shown that experience with punishment can profoundly affect behavior in future situations [41, 55]. In particular, early experience with a high level of shock may induce long-lasting hypersensitivity to shock and suppress behavior in a persistent manner [41]. Conversely, experience with low levels of shock can induce tolerance to punishment and allow individuals to tolerate higher levels of shock later on [55]. The PSS procedure follows in the latter category. In the present study, changes in PSS were confounded by the introduction of delays, but unpublished results in our lab show that upon repeated exposure, rats reach slightly higher PSS break points compared to initial levels. However, we found that PSS break-points are highly correlated throughout the experiments, suggesting that whereas absolute individual levels may increase, relative resistance to punishment in the PSS procedure is relatively stable over time, regardless of the order of delays to which animals were exposed. This is a potentially useful feature of the PSS procedure.
Anxiety and addiction are often comorbid psychiatric disorders and their relationships are bidirectional [56]. In this study, we found that anxiety-like behavior measured in the open field was higher in shock-sensitive than in shock-resistant rats. If we consider shock-resistant rats as addiction-prone, this result is apparently at odds with previous findings in humans and animals. However, it should be noticed that we measured anxiety-like behavior at the end of the experiment and therefore, the results can be affected by previous training in the PSS and could therefore be the consequence rather than the cause of different behavior in the PSS. Indeed, anxiety-like behavior in sensitive rats was associated not only with lower PSS break points but also with higher conditioned suppression, suggesting that repeated experience of fear in an operant context may have led to diffuse fearful behavior. In previous studies investigating decision-making when one option is associated with increasing risks of receiving an electric shock, the choice of the risky option was not associated with anxiety-like behaviors [57], even though the anxiolytic drug diazepam increased risky behavior [58]. More directly relevant to this study, in a previous study, PSS break point and anxiety-like behavior were not correlated [31]. The main difference between that study and this one is the introduction of delay suggesting that temporal degradation of contingency induced an anxious phenotype. These results are reminiscent of previous studies in which probability degradation of punishment contingency produced anxiety-like behavior [59, 60]. Importantly, in our study shock-sensitive rats showed higher anxiety-like behavior than shock-resistant rats, suggesting that these animals became afraid of their environment even when no real threat was present.
Depending on the circumstances, being relatively resistant or relatively sensitive to punishment could be either adaptive or maladaptive. In our procedure, we found a subset of shock-resistant rats that kept seeking food, but they still adapted their behavior depending on the intensity of the shock or the delay. In contrast, the subset of shock-sensitive rats always stopped responding to food after a few shocks, and their behavior changed little when the punishment was delayed. In addition, sensitive animals showed conditioned suppression in the absence of punishment, a behavior that would not generally be considered optimal in the case of food reward. More generally, shock-sensitive and shock-resistant animals appear to have different strategies to cope with environmental challenges. At the individual level, the conservative (shock-sensitive) strategy may be more adaptive if risks are maintained and frequent whereas the risk (shock-resistant) strategy may be more adaptive if risks are temporary and rare.
This study has several limitations that should be acknowledged. First, we investigated the effects of delay only in male rats. Whereas we previously showed that resistance to punishment for immediate footshock is qualitatively similar between males and females [31], other studies have shown that females are less sensitive to delay discounting of negative consequences than males [27]. In addition, males and females differ in their sensitivity to probabilistic punishment [61]. Therefore, it will be important in the future to investigate the effects of delaying punishment in female rats. Another limitation is that to investigate pain sensitivity, we used the hot plate test, a measure of thermal nociception, whereas our punishment consisted of electrical footshocks. The hot plate can only provide coarse information about differences in pain sensitivity but it cannot exclude that subtle or sensory-specific differences would be revealed by more specific measures of shock-induced pain. However, even though using footshock itself would allow a more specific measure of the sensitivity to the type of stimulus used as a punisher, this approach would have clear disadvantages. Indeed, it has been shown that exposure to shocks could induce tolerance or sensitization to future shocks [41, 55] so that the measure of sensitivity itself and the effect of punishment may interact and greatly complicate the interpretation of the results. To avoid this bias and for its simplicity, we have used thermal nociception in the hot plate as a crude measure of pain sensitivity in punishment procedures as done in similar previous papers [31, 62, 63].
In conclusion, introducing a delay between reward and punishment, we identified a subpopulation of rats that were highly sensitive to shock regardless of the delay, to the extent that their response was suppressed (and food reward was lost) even when shock was discontinued. These same individuals also showed anxiety-like behavior in a novel environment and may represent a population vulnerable to opportunity loss. In contrast, the rest of the rats showed resistance to punishment and this phenotype was even more pronounced when the consequences of food-seeking were delayed. These individuals represent a population that would be particularly at risk of developing addiction-like behavior. Thus, the PSS procedure identifies factors that influence resistance to punishment within a behavioral economics paradigm, and future work should determine whether it can be used to assess vulnerability to developing drug addiction.
Data availability
The data that support the findings of this study are available from the corresponding author (MS) upon reasonable request.
References
Koob GF, Volkow ND. Neurobiology of addiction: a neurocircuitry analysis. Lancet Psychiatry. 2016;3:760–73.
Wise RA, Bozarth MA. A psychomotor stimulant theory of addiction. Psychol Rev. 1987;94:469–92.
Robinson TE, Berridge KC. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res Brain Res Rev. 1993;18:247–91.
Di Chiara G. Drug addiction as dopamine-dependent associative learning disorder. Eur J Pharm. 1999;375:13–30.
Koob GF, Le Moal M. Drug addiction, dysregulation of reward, and allostasis. Neuropsychopharmacology. 2001;24:97–129.
Everitt BJ, Robbins TW. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci. 2005;8:1481–9.
Hyman SE, Malenka RC, Nestler EJ. Neural mechanisms of addiction: the role of reward-related learning and memory. Annu Rev Neurosci. 2006;29:565–98.
Solinas M, Belujon P, Fernagut PO, Jaber M, Thiriet N. Dopamine and addiction: what have we learned from 40 years of research. J Neural Transm (Vienna). 2019;126:481–516.
Vanderschuren LJMJ, Ahmed SH. Animal studies of addictive behavior. Cold Spring Harb Perspect Med. 2013;3:a011932.
Figee M, Pattij T, Willuhn I, Luigjes J, van den Brink W, Goudriaan A, et al. Compulsivity in obsessive–compulsive disorder and addictions. Eur Neuropsychopharmacol. 2016;26:856–68.
Lüscher C, Robbins TW, Everitt BJ. The transition to compulsion in addiction. Nat Rev Neurosci. 2020;21:247–63.
Fredriksson I, Venniro M, Reiner DJ, Chow JJ, Bossert JM, Shaham Y. Animal models of drug relapse and craving after voluntary abstinence: a review. Pharm Rev. 2021;73:1050–83.
George O, Ahmed SH, Gilpin NW. Are we compulsively chasing rainbows? Neuropsychopharmacol. 2022;47:2013–5.
McNally GP, Jean-Richard-dit-Bressel P, Millan EZ, Lawrence AJ. Pathways to the persistence of drug use despite its adverse consequences. Mol Psychiatry. 2023. 30 March 2023. https://doi.org/10.1038/s41380-023-02040-z.
Hu H. Reward and aversion. Annu Rev Neurosci. 2016;39:297–324.
Jean-Richard-Dit-Bressel P, Killcross S, McNally GP. Behavioral and neurobiological mechanisms of punishment: implications for psychiatric disorders. Neuropsychopharmacology. 2018;43:1639–50.
Chapman GB. Temporal discounting and utility for health and money. J Exp Psychol: Learn Mem Cogn. 1996;22:771–91.
Story G, Vlaev I, Seymour B, Darzi A, Dolan R. Does temporal discounting explain unhealthy behavior? A systematic review and reinforcement learning perspective. Front Behav Neurosci. 2014;8:76.
Bickel WK, Koffarnus MN, Moody L, Wilson AG. The behavioral- and neuro-economic process of temporal discounting: a candidate behavioral marker of addiction. Neuropharmacology. 2014;76:518–27.
Bickel WK, Marsch LA. Toward a behavioral economic understanding of drug dependence: delay discounting processes. Addiction. 2001;96:73–86.
Acuff SF, MacKillop J, Murphy JG. A contextualized reinforcer pathology approach to addiction. Nat Rev Psychol. 2023;2:309–23.
Banks RK, Vogel-Sprott M. Effect of delayed punishment on an immediately rewarded response in humans. J Exp Psychol. 1965;70:357–9.
Estle SJ, Green L, Myerson J, Holt DD. Differential effects of amount on temporal and probability discounting of gains and losses. Mem Cogn. 2006;34:914–28.
Baron A. Delayed punishment of a runway response. J Comp Physiol Psychol. 1965;60:131–4.
Woolverton WL, Freeman KB, Myerson J, Green L. Suppression of cocaine self-administration in monkeys: effects of delayed punishment. Psychopharmacol (Berl). 2012;220:509–17.
Rodríguez W, Bouzas A, Orduña V. Temporal discounting of aversive consequences in rats. Learn Behav. 2018;46:38–48.
Liley AE, Gabriel DBK, Sable HJ, Simon NW. Sex differences and effects of predictive cues on delayed punishment discounting. eNeuro. 2019;6:ENEURO.0225-19.2019.
Apergis-Schoute AM, Gillan CM, Fineberg NA, Fernandez-Egea E, Sahakian BJ, Robbins TW. Neural basis of impaired safety signaling in obsessive compulsive disorder. Proc Natl Acad Sci. 2017;114:3216–21.
Kim AJ, Anderson BA. The effect of concurrent reward on aversive information processing in the brain. NeuroImage. 2020;217:116890.
Kanen JW, Apergis-Schoute AM, Yellowlees R, Arntz FE, van der Flier FE, Price A, et al. Serotonin depletion impairs both Pavlovian and instrumental reversal learning in healthy humans. Mol Psychiatry. 2021;26:7200–10.
Desmercieres S, Lardeux V, Longueville J-E, Hanna M, Panlilio LV, Thiriet N, et al. A self-adjusting, progressive shock strength procedure to investigate resistance to punishment: characterization in male and female rats. Neuropharmacology. 2022;220:109261.
Panlilio LV, Thorndike EB, Schindler CW. Reinstatement of punishment-suppressed opioid self-administration in rats: an alternative model of relapse to drug abuse. Psychopharmacology. 2003;168:229–35.
Deroche-Gamonet V, Belin D, Piazza PV. Evidence for addiction-like behavior in the rat. Science. 2004;305:1014–7.
Pelloux Y, Everitt BJ, Dickinson A. Compulsive drug seeking by rats under punishment: effects of drug taking history. Psychopharmacology. 2007;194:127–37.
Krasnova IN, Marchant NJ, Ladenheim B, McCoy MT, Panlilio LV, Bossert JM, et al. Incubation of methamphetamine and palatable food craving after punishment-induced abstinence. Neuropsychopharmacology. 2014;39:2008–16.
Bentzley BS, Jhou TC, Aston-Jones G. Economic demand predicts addiction-like behavior and therapeutic efficacy of oxytocin in the rat. Proc Natl Acad Sci. 2014;111:11822–7.
Pascoli V, Terrier J, Hiver A, Lüscher C. Sufficiency of mesolimbic dopamine neuron stimulation for the progression to addiction. Neuron. 2015;88:1054–66.
Datta U, Martini M, Sun W. Different functional domains measured by cocaine self-administration under the progressive-ratio and punishment schedules in male Wistar rats. Psychopharmacology. 2018;235:897–907.
Marchant NJ, Campbell EJ, Kaganovsky K. Punishment of alcohol-reinforced responding in alcohol preferring P rats reveals a bimodal population: Implications for models of compulsive drug seeking. Prog Neuro-Psychopharmacol Biol Psychiatry. 2018;87:68–77.
Jean-Richard-dit-Bressel P, Ma C, Bradfield LA, Killcross S, McNally GP. Punishment insensitivity emerges from impaired contingency detection, not aversion insensitivity or reward dominance. eLife. 2019;8:e52765.
Durand A, Girardeau P, Freese L, Ahmed SH. Increased responsiveness to punishment of cocaine self-administration after experience with high punishment. Neuropsychopharmacology. 2021. 24 August 2021. https://doi.org/10.1038/s41386-021-01159-3.
HubrechtCarter. The 3Rs and humane experimental technique: implementing change. Animals. 2019;9:754.
Lattal KA. Delayed reinforcement of operant behavior. J Exp Anal Behav. 2010;93:129–39.
Richardson NR, Roberts DC. Progressive ratio schedules in drug self-administration studies in rats: a method to evaluate reinforcing efficacy. J Neurosci Methods. 1996;66:1–11.
Solinas M, Panlilio LV, Antoniou K, Pappas LA, Goldberg SR. The Cannabinoid CB1 antagonist N- Piperidinyl-5-(4-chlorophenyl)-1-(2,4-dichlorophenyl) -4-methylpyrazole-3-carboxamide (SR-141716A) differentially alters the reinforcing effects of heroin under continuous reinforcement, fixed ratio, and progressive ratio schedules of drug self-administration in rats. J Pharm Exp Ther. 2003;306:93–102.
Solinas M, Goldberg SR. Motivational effects of cannabinoids and opioids on food reinforcement depend on simultaneous activation of cannabinoid and opioid systems. Neuropsychopharmacology. 2005;30:2035–45.
Deuis JR, Dvorakova LS, Vetter I. Methods used to evaluate pain behaviors in rodents. Front. Mol. Neurosci. 2017;10:284.
Datta U, Martini M, Fan M, Sun W. Compulsive sucrose- and cocaine-seeking behaviors in male and female Wistar rats. Psychopharmacology. 2018;235:2395–405.
Odum AL, Becker RJ, Haynes JM, Galizio A, Frye CCJ, Downey H, et al. Delay discounting of different outcomes: review and theory. J Exp Anal Behav. 2020;113:657–79.
Ersche KD, Meng C, Ziauddeen H, Stochl J, Williams GB, Bullmore ET, et al. Brain networks underlying vulnerability and resilience to drug addiction. Proc Natl Acad Sci USA. 2020;117:15253–61.
Swendsen J, Le Moal M. Individual vulnerability to addiction. Ann N. Y Acad Sci. 2011;1216:73–85.
Kasanetz F, Deroche-Gamonet V, Berson N, Balado E, Lafourcade M, Manzoni O, et al. Transition to addiction is associated with a persistent impairment in synaptic plasticity. Science. 2010;328:1709–12.
Domingo-Rodriguez L, Ruiz De Azua I, Dominguez E, Senabre E, Serra I, Kummer S, et al. A specific prelimbic-nucleus accumbens pathway controls resilience versus vulnerability to food addiction. Nat Commun. 2020;11:782.
García-Blanco A, Domingo-Rodriguez L, Cabana-Domínguez J, Fernández-Castillo N, Pineda-Cirera L, Mayneris-Perxachs J, et al. miRNA signatures associated with vulnerability to food addiction in mice and humans. J Clin Investig. 2022;132:e156281.
Miller NE. Learning resistance to pain and fear: effects of overlearning, exposure, and rewarded exposure in context. J Exp Psychol. 1960;60:137–45.
Smith JP, Book SW. Anxiety and substance use disorders: a review. Psychiatr Times. 2010;25:19–23.
Simon NW, Montgomery KS, Beas BS, Mitchell MR, LaSarge CL, Mendez IA, et al. Dopaminergic modulation of risky decision-making. J Neurosci. 2011;31:17460–70.
Mitchell MR, Vokes CM, Blankenship AL, Simon NW, Setlow B. Effects of acute administration of nicotine, amphetamine, diazepam, morphine, and ethanol on risky decision-making in rats. Psychopharmacology. 2011;218:703–12.
Park J, Moghaddam B. Risk of punishment influences discrete and coordinated encoding of reward-guided actions by prefrontal cortex and VTA neurons. eLife 2017;6:e30056.
Jacobs DS, Allen MC, Park J, Moghaddam B. Learning of probabilistic punishment as a model of anxiety produces changes in action but not punisher encoding in the dmPFC and VTA. eLife. 2022;11:e78912.
Chowdhury TG, Wallin-Miller KG, Rear AA, Park J, Diaz V, Simon NW, et al. Sex differences in reward- and punishment-guided actions. Cogn Affect Behav Neurosci. 2019;19:1404–17.
Li Y, Simmler LD, Van Zessen R, Flakowski J, Wan J-X, Deng F, et al. Synaptic mechanism underlying serotonin modulation of transition to cocaine addiction. Science. 2021;373:1252–6.
Degoulet M, Tiran-Cappello A, Combrisson E, Baunez C, Pelloux Y. Subthalamic low-frequency oscillations predict vulnerability to cocaine addiction. Proc Natl Acad Sci USA. 2021;118:e2024121118.
Acknowledgements
This work was supported by the Centre National pour la Recherche Scientifique, the Institut National de la Santé et de la Recherche Médicale, the University of Poitiers, the Nouvelle Aquitaine CPER 2015-2020/FEDER 2014–2020 program “Habisan” and the Nouvelle Aquitaine grant AAPR2020A-2019-8357510 (PI: M. Solinas) and the IRESP/Inca grants « IRESP-19-ADDICTIONS-20 » (PI: M. Solinas) and « IRESP-AAPSPA2021-V1-07 » (PI: N. Thiriet). The contribution of LVP was supported by the Intramural Research Program of the NIH, National Institute on Drug Abuse. We thank Pauline Belujon for helpful comments on a previous version of the manuscript. This study has benefited from the facilities and expertise of PREBIOS platform (Université de Poitiers).
Author information
Authors and Affiliations
Contributions
Conceptualization (MS, NT, LVP, NJ), Data curation (MS, VL, JEL, YV), Formal Analysis (MS, JEL), Funding Acquisition (MS, NT, NJ), Investigation (VL, PML), Methodology, (MS, NT, YV, LVP), Project Administration (MS, NJ), Resources (MS, JEL, VL, LVP, NJ), Writing-original draft (MS), Writing- review and editing (MS, VL, PML, JEL, NT, YV, LVP, NJ).
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Solinas, M., Lardeux, V., Leblanc, PM. et al. Delay of punishment highlights differential vulnerability to developing addiction-like behavior toward sweet food. Transl Psychiatry 14, 155 (2024). https://doi.org/10.1038/s41398-024-02863-6
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41398-024-02863-6
- Springer Nature Limited