Abstract
Rationale
Repeated haloperidol treatment in rodents results in a day-to-day intensification of catalepsy (i.e., sensitization). Prior experiments suggest that this sensitization is context-dependent and resistant to extinction training.
Objectives
The aim of this study was to provide a neurobiological mechanistic explanation for these findings.
Materials and methods
We use a neurocomputational model of the basal ganglia and simulate two alternative models based on the reward prediction error and novelty hypotheses of dopamine function. We also conducted a behavioral rat experiment to adjudicate between these models. Twenty male Sprague–Dawley rats were challenged with 0.25 mg/kg haloperidol across multiple days and were subsequently tested in either a familiar or novel context.
Results
Simulation results show that catalepsy sensitization, and its context dependency, can be explained by “NoGo” learning via simulated D2 receptor antagonism in striatopallidal neurons, leading to increasingly slowed response latencies. The model further exhibits a non-extinguishable component of catalepsy sensitization due to latent NoGo representations that are prevented from being expressed, and therefore from being unlearned, during extinction. In the rat experiment, context dependency effects were not dependent on the novelty of the context, ruling out the novelty model’s account of context dependency.
Conclusions
Simulations lend insight into potential complex mechanisms leading to context-dependent catalepsy sensitization, extinction, and renewal.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Haloperidol is a potent typical antipsychotic used with high affinity to the dopamine (DA) D2 receptors. In laboratory animals, it is used to model the extrapyramidal side effects of neuroleptics therapy: Rats treated with haloperidol show symptoms similar to those observed in Parkinson’s Disease (PD) and include cognitive learning deficits as well as akinesia and rigidity (i.e., catalepsy), effects that are mediated by blockade of striatal D2 receptors (Sanberg 1980).
Interestingly, repeated administration of haloperidol leads to an intensification of catalepsy following each consecutive test—a process known as sensitization (Schmidt and Beninger 2006; Schmidt et al. 1999; Lanis and Schmidt 2001; Frank and Schmidt 2003; Barnes et al. 1990; Antelman et al. 1986). This same catalepsy sensitization does not occur only under repeated administration of haloperidol but also in DA-deficient animals (bilateral striatal 6-OHDA-lesion) (Klein and Schmidt 2003; Srinivasan and Schmidt 2004).
Similarly, antipsychotic medications do not improve symptoms of psychosis until a delay of days up to weeks (Reynolds 1992) implicating the involvement of sensitization processes in the therapy of schizophrenia.
Notably, catalepsy sensitization by haloperidol and by 6-OHDA-lesion is context-dependent: it is observed in the context under which haloperidol administration was originally given, testing in other novel contexts results in a significant decrease of catalepsy (Klein and Schmidt 2003; Srinivasan and Schmidt 2004; see Fig. 1).
Furthermore, although catalepsy expression can be extinguished following repeated injections of placebo instead of haloperidol, the sensitization nevertheless shows a non-extinguishable component: a single dose of haloperidol elicits renewed elevation of catalepsy relative to animals who had not been previously sensitized (Amtage and Schmidt 2003; see Fig. 2). Similar sensitization, extinction, and renewal phenomena are observed in response to drugs of abuse (e.g., Redish et al. 2007), raising the question of whether similar principles apply.
Despite several years of research into these phenomena, a well elaborated mechanistic explanation for these observations is still lacking. A key observation might be that haloperidol-induced catalepsy sensitization could be related to changes in synaptic strength (i.e., learning) within the striatum—a brain region with a high rate of neuroplastic changes modulated by DA (e.g., Calabresi et al. 2007; Centonze et al. 2001; Robinson and Kolb 1997). Indeed, chronic haloperidol enhances synaptic plasticity via D2 receptor blockade (Centonze et al. 2004) and phosphorylation of GluR1 AMPA receptors in striatopallidal neurons (Håkansson et al. 2006). Haloperidol potently blocks dopaminergic D2 receptors which are primarily found in the indirect pathway of the basal ganglia (BG) (Gerfen 2000; Salin et al. 1996; Boraud et al. 2002; Robertson et al. 1992; Gerfen et al. 1995; Surmeier et al. 2007) and increases spike frequency in striatal spiny I neurons (Frank and Schmidt 2003). Neurons in this same pathway are also hyperactive or show abnormal burst firing in Parkinsonism (Albin et al. 1989; Bergman et al. 1998; Mallet et al. 2006).
In this paper, we focus on this well-established striatal D2 mechanism of haloperidol, in an attempt to account for catalepsy sensitization (see “Discussion” for other mechanisms). To explore the complex dynamic interactions among BG sub-regions in response to haloperidol, we use an explicit, computational model of the BG (Frank 2006), which is also grounded by its ability to account for other dopamine-dependent learning-related phenomena and to make predictions that have subsequently been confirmed via pharmacological manipulations in humans (Frank et al. 2004, 2007c; Frank 2005; Frank and O’Reilly 2006; Cools et al. 2006; Santesso et al. 2009; Moustafa et al. 2008). Here, we report that this same model also reproduces the effects of haloperidol on sensitization, context dependency, extinction, and renewal.
Materials and methods
Model: high level overview
The role of the BG can be seen as that of a dynamic modulator of frontal cortical action plans. With respect to motor control, the BG could function as an action selection device (Graybiel 2000; Redgrave et al. 1999; Frank 2005): efferent projections from the motor cortex reach the BG, which then facilitate appropriate motor commands while suppressing those that are inappropriate (Basso and Wurtz 2002; Brown et al. 2004; Gurney et al. 2001; Jiang et al. 2003; Mink 1996; Redgrave et al. 1999). These two functions can be supported by two separate striatofugal neural projections (Albin et al. 1989; Alexander and Crutcher 1990). The direct striatonigral pathway (expressing high levels of D1 receptors) functions as the “Go-” pathway, by facilitating the selection of particular actions when appropriate in a given sensory context. In contrast, neurons originating in the striato–pallidal–nigral pathway (expressing high levels of D2 receptors) function as the “NoGo-” pathway, by detecting the conditions in which a given action should be suppressed and counteracting the Go pathway at the level of BG output (Frank 2005).
According to the reward prediction error (RPE) hypothesis, midbrain dopaminergic neurons signal when outcomes are better or worse than expected via phasic bursts and pauses in firing (Schultz et al. 1997). Reward-associated behavior is potentiated by activation of D1 receptors and synaptic plasticity in the Go pathway following dopamine bursts (Reynolds and Wickens 2002). Conversely, maladaptive behaviors are suppressed via disinhibition of D2 receptors and potentiation of NoGo cells following dopamine dips (Frank 2005). Recent studies support these dual mechanisms of plasticity: D1 stimulation potentiates corticostriatal Go synapses, whereas a lack of D2 receptor stimulation (simulating the effect of a DA dip) was required to potentiate NoGo synapses (Shen et al. 2008).Footnote 1 Furthermore, this NoGo learning effect would be enhanced by D2 receptor sensitivity (Seeman 2008) and enhanced excitability of striatopallidal NoGo cells in the DA-depleted state (Surmeier et al. 2007; Shen et al. 2008).
Some question the RPE hypothesis of DA signaling altogether, suggesting that the timing of DA signals is too early to encode these errors (Redgrave et al. 1999; Redgrave and Gurney 2006). They argue that the functional role of phasic dopaminergic neuron firing is to reinforce the development of a novel action, rather than unpredicted reward per se, in response to a salient or novel stimulus (for recent reviews, see Redgrave and Gurney (2006) and Lisman and Grace (2005)). This theory was addressed in our second experiment (see below).
BG model functionality
The BG model’s basic functionality is to select an appropriate response in the output units when presented with a stimulus in the input (Fig. 3). This selection mechanism involves interactions between various BG nuclei beginning with the striatum, consisting of simulated Go (striatonigral) and NoGo (striatopallidal) units. Activity in Go units facilitates a response by effectively disinhibiting the thalamic units representing that response and allowing reverberatory thalamocortical projections to generate a cortical response. Activity in NoGo units counteracts the Go activity via inhibition of the external segment of globus pallidus (GPe), which has an opposing effect via focused projections from GPe to GPi (Parent and Hazrati 1995).Footnote 2 The substantia nigra pars compacta (SNc) sends dopaminergic projections to the striatum and signals positive or negative RPEs via DA bursts or dips, respectively. DA bursts further excite activated Go neurons via D1 receptors while inhibiting NoGo neurons via D2 receptors (Gerfen 1992; Joel and Weiner 1999; Brown et al. 2004; Frank 2005). In contrast, DA dips have the opposite effect: NoGo neurons have an increased probability of firing, due to removal of DA inhibition onto sensitive D2 receptors. Furthermore, only striatal neurons that are already activated via glutamatergic corticostriatal input (representing the particular stimulus-response conjunction) can increase or decrease their synaptic strengths, according to the Hebbian-like learning rule in our model (see Electronic supplementary material), similar to the three-factor learning rule proposed by Wickens and colleagues (e.g., Reynolds et al. 2001). Thus, DA bursts potentiate only Go synapses associated with the selected response and activated by the input stimulus, leading to positive reinforcement learning. Conversely, DA dips potentiate activated NoGo synapses such that this response will be more likely to be suppressed in future presentations of the same stimulus (Frank 2005, 2006). Implementation details of the original model can be found elsewhere (Frank 2006); additional changes to simulate the tasks here can be found in the Electronic supplementary material.
Catalepsy simulations
A common measurement of catalepsy in rats is the bar test (e.g., Amtage and Schmidt 2003; Klein and Schmidt 2003) in which the animal is placed to stand with its forepaws on an elevated bar and the time until the first movement occurs is taken. But how does one simulate catalepsy in a computational model? Just as rats can take a longer time to descend off the bar, the BG model can take varying amounts of time to facilitate a response. To measure the latency until an action is selected in the model (hereafter, “response time” (RT)), we assessed the number of network processing cycles (see Electronic supplementary material) before a response was selected by the BG action selection network, i.e., until one of the thalamus units was disinhibited by BG circuitry. When a thalamus unit is activated, the corresponding response is swiftly executed (Frank 2006). Thus, catalepsy is associated with longer latencies to gate responses. These same methods were employed in previous model RT analyses (Frank et al. 2007b, d; Moustafa et al. 2008). Because the BG gating system is required to facilitate a cortical response, similar results are obtained by probing output unit activity.
To gain insight into underlying processes leading to different RTs, we additionally probed striatal unit activity. As described above, the BG model simulates Go and NoGo neuronal populations in the striatum, which facilitate and suppress responses, respectively. If a Go population for a given response is more active than its NoGo counterpart, that response is more likely to be facilitated, thus the relative difference in Go–NoGo activity influences the speed at which the response is executed (Moustafa et al. 2008). If NoGo activity is relatively greater than Go activity, as seen in Parkinsonism, a response may not be selected at all by the BG; therefore the NoGo–Go contrast reflects an internal (“hidden variable”) measure of catalepsy (see Electronic supplementary material for the precise computation).
To simulate the partial blockade of postsynaptic D2 receptors by haloperidol, we reduced the strength of the inhibitory SNc D2 projection onto NoGo neurons to 10% that of the original, representing a 90% occupation of D2 receptors by haloperidolFootnote 3.
Reward prediction error model
According to the RPE hypothesis, DA bursts signal unexpected reward and DA dips signal the lack of expected reward. However, in the aforementioned experiments on catalepsy sensitization, neither explicit reward nor punishment was used following motor responses. However, we reasoned that because the bar test is somewhat aversive (the animal does not want to be on the bar and therefore descends), the escape from aversive conditions may be associated with a positive DA burst. Indeed, there is evidence that an offset of an aversive stimulus is associated with increased striatal DA (Jackson and Moghaddam 2004). Accordingly, we applied a small DA burst following response execution during training.
The network was trained for 60 trials in the haloperidol mode in context A (represented by a set of four sensory input units), and then tested in that context and in an untrained context B (corresponding to a different set of input units).Footnote 4 During this testing procedure, the network’s weights were prevented from changing, so as to prevent learning in the test and to permit multiple tests across training (something which would have to be done between subjects in actual experiments). Next, we simulated extinction by continuing training for a further 40 trials with the network switched from haloperidol mode to the intact state (i.e., weights of the SNc→NoGo projections at 100%). Finally, the haloperidol mode was simulated for an additional five test trials, to determine whether the model still demonstrates sensitized catalepsy after extinction.
Novelty model
We further tested whether an implementation of the novelty hypothesis could account for the same findings, and if so, whether the two models make divergent predictions. In this case, it is the novelty of a stimulus, not the RPE associated with it that drives a DA burst. Accordingly, the apparent context dependency of catalepsy sensitization could arise simply because the animal is not familiar with context B; the associated novelty-driven DA burst (Lisman and Grace 2005; Kakade and Dayan 2002) could activate the Go pathway, promote locomotion and exploratory behavior, and thereby lead to reduction in catalepsy. Note that this hypothesis does not require the assumption that any NoGo learning is specific to sensory input units encoding the external context A. Instead, this learning might generalize across contexts, but the context dependency arises due to the novelty of surrounding context B which drives a DA burst that counteracts catalepsy expression.Footnote 5
Behavioral experiment
After completion of simulation studies and on a suggestion from an anonymous reviewer, we conducted a simple behavioral experiment to distinguish between the RPE and novelty models. This experiment is similar to the context challenge experiment (Klein and Schmidt 2003), but an additional group of animals was habituated to context B (without haloperidol) prior to the sensitization phase, in order to eliminate the novelty of this context. If the novelty model is correct, we would expect to see continued catalepsy expression (or a smaller reduction in catalepsy) during the context challenge in this group, because the context is no longer novel and there should therefore not be a novelty-driven DA burst. In contrast, if the RPE model is correct, all animals should show reduced catalepsy in context B regardless of whether it is novel, because stimulus-NoGo learning only occurred in context A.
Methods
Subjects
A total of 20 male Sprague–Dawley rats (230–260 g at the beginning of the experiment), Charles River, Sulzfeld, Germany, were used. The animals were group-housed (four animals per cage, in standard macrolon IV cages) under a 12/12 light–dark cycle with restricted access to food (12 g per animal per day). Access to water was unconfined (i.e., ad libitum).
Substance
The neuroleptic agent haloperidol (Haldol®-injection solution, Janssen, Germany) was diluted in saline (0.9% NaCl), Fresenius, Germany to a concentration of 0.25 mg/ml. Substance administration was carried out subcutaneously (s.c.), at 1 ml/kg body weight, the same concentration used in Amtage and Schmidt (2003) and Klein and Schmidt (2003).
Behavioral testing
To test for catalepsy, the animals performed a bar test. Within that test, a single rat was put gently with its forepaws on a horizontal bar (9 cm above the table surface, diameter of 0.5 cm). The descent latency, as a proxy for the degree of catalepsy, was measured by taking the time interval between the first placement of the animal on the apparatus and its first active paw movement. This procedure is identical to the one used in Amtage and Schmidt (2003) and Klein and Schmidt (2003).
To test for context dependency, two contexts (A and B) were used. The context consisted of a different room (with different lighting) and a different lab coat of the experimenter (in context A, the experimenter wore a white lab coat; while in context B, a black plastic poncho was put over the lab coat). These context cues are very similar to those used in Klein and Schmidt (2003).
Experimental design
The rats were handled for five consecutive days prior to the first catalepsy test. During the following habituation phase, the animals received a (s.c.) saline injection and were tested 60 min later. The treatment during the habituation phase took place in context A for the first (non-habituated) group (n = 10) and in context B for the second (habituated) group (n = 10).
After the habituation phase, catalepsy sensitization was performed for both groups in context A, for a total of 9 days (after the seventh sensitization day, there was a lack of testing for 2 days). On the first day after sensitization (day 17), both groups were tested in context B to induce the context challenge. On day 18, both groups were retested in context A.
Statistics
Statistical analysis was performed using GB STAT 7.0. Multiple values within a group, where tested with the non-parametric Friedman ANOVA. Two values of one group were submitted to the Wilcoxon signed rank test. To compare individual data between two groups, we used the Mann Whitney U test.
Results
Reward prediction error model results
Catalepsy sensitization
During the first 60 training trials in the haloperidol mode, a steady increase of catalepsy (i.e., an increase in model latencies to select a response) can be observed in context A (Fig. 4a). As expected, the RTs are strongly correlated with relatively greater NoGo than Go activity across trials (Fig. 4b), allowing closer analysis of the mechanisms by which catalepsy materializes. This activation difference resulted from Hebbian learning processes in which active neurons adjust their weights. Because simulated haloperidol blocked dopamine from inhibiting NoGo units, the activity of these units increased as seen during DA depletion (e.g., Mallet et al. 2006). As a result, the synaptic weights between the sensory input (context A) units and these NoGo units increased, consistent with the potentiation of corticostriatal synapses in striatopallidal neurons following haloperidol administration (Centonze et al. 2004; Håkansson et al. 2006). Thus, the next time context A was presented, it elicited greater NoGo activity, which in turn further increased synaptic strength between context A and NoGo units, such that each trial of stimulus context presentation led to progressively greater NoGo activity. In contrast, control networks actually show a decrease of NoGo–Go activity, corresponding to greater D1-dependent Go learning to descend from the bar together with an inhibitory effect of DA onto NoGo neurons (via intact D2 receptors). Thus, this model provides a plausible explanation for the catalepsy intensification resulting from D2 receptor blockade.
Context dependency
In contrast to sensitization in context A, simulated catalepsy was roughly constant in context B regardless of the number of training trials with haloperidol in context A. We also confirmed that this context dependency arose due to differences in weights between context input units to the striatum (data not shown). Weights from the context A input neurons to the NoGo neurons increased, while those from context B units did not, due to the dependency of Hebbian learning on both pre- and postsynaptic activation (see equation A-8 in the Electronic supplementary material). Because the model was never trained with context B units active, its NoGo weights to the striatum did not increase. Thus, the model replicates the context dependency of sensitized haloperidol observed in rodents.
Extinction training
The model also captures extinction (Fig. 4a, b). After we switched the model from simulated haloperidol to the intact mode, cataleptic activity progressively decreased, reaching its starting value by the end of extinction. Again, these effects can be explained by examining the weights from the input units to the striatum. Initially, the network exhibited cataleptic activity in context A (despite being in the intact mode), due to prior NoGo learning. However, because the DA units can now inhibit NoGo units, the Go units were now free to fire more (due to less inhibitory competition from NoGo units). The DA bursts following response selection (corresponding to the offset of the aversive stimulus) also led to Go learning during this time, and thus a reduction in catalepsy.
Sensitized component
Finally, the haloperidol-trained network also exhibited a sensitized component that was resistant to extinction. As shown in Fig. 4, a switch back from intact to haloperidol mode in trial 100 was associated with a prominent rise of cataleptic activity (increased RT and associated NoGo activity) in context A. This sensitized catalepsy was observed despite the previous 60 trials of extinction training in which catalepsy was reduced back to baseline and was far greater than that observed in networks which had never undergone haloperidol sensitization. This qualitative pattern of data matches that observed in rats (Amtage and Schmidt 2003).
What are the underlying mechanisms that cause this non-extinguishable componentFootnote 6? Intriguingly, examination of the weights from the input to NoGo units revealed that the weights did not substantially decrease during extinction—that is, there was relatively little unlearning of prior NoGo associations. Instead, the steady decrease of cataleptic activity resulted primarily from an increase in Go weights during extinction. When switching from haloperidol to intact mode, the now intact SNc→NoGo projections inhibited the striatal NoGo neurons, which prevented these neurons from changing their weights due to the Hebbian learning rule. Consequently, the previously learned A→NoGo association was maintained, but was only prevented from being expressed, during extinction. Thus, when the model was ultimately switched back to haloperidol mode, this prior learning was then immediately uncovered. Finally, note that when the model was tested in context B in trial 100, there was not a large increase in cataleptic activity, due to the specificity of learned NoGo weights. Thus, according to this model implementation, we would expect the sensitized component to be context-dependent.
Novelty hypothesis model results
Full details of the novelty model and the results are presented in the online Supplementary Material. In brief, this model produces a similar rise in cataleptic activity due to the same NoGo learning mechanism and also exhibits context dependency due to the novelty of the untrained context (Fig. 5a, b) as well as the non-extinguishable component (Fig. 6a, b) without requiring the assumption of distinct context representations. However, the novelty model predicts that catalepsy sensitization would not be context-dependent if tested in a familiar context.
Behavioral experiment results
The results from the novel experiment designed to adjudicate between the two models are shown in Fig. 7. During the habituation phase (days 1–5), there was no significant increase of catalepsy in both groups. During the sensitization phase (days 6–12 and days 15 and 16), a highly significant increase of descent latency (i.e., sensitization) was observed in both groups (p < 0.0001). The 2 days without testing (days 13 and 14) had no significant effect on catalepsy. Compared to the descent latencies on days 16 and 18, both groups showed a significant attenuation of the descent latencies on day 17 (p < 0.05). There were no significant between-group differences in descent latencies between days 16, 17, and 18, or in the decrease of descent latencies from day 16 to day 17. Thus, there was no effect of novelty on the context dependency of catalepsy expression.
Discussion
In the present study, we explored possible neural mechanisms of haloperidol-induced catalepsy sensitization, using a computational model of the BG (Frank 2006). The model suggests that this catalepsy sensitization reflects a form of “NoGo” learning to suppress action execution, caused by disinhibition of striatopallidal neurons expressing D2 receptors in the basal ganglia. This notion is supported by studies showing that chronic haloperidol administration promotes synaptic potentiation in corticostriatal projections (Centonze et al. 2004), an effect that appears to be specific to NoGo/indirect pathway neurons (Håkansson et al. 2006). Thus, we posit catalepsy sensitization to result from the same mechanism that leads to relatively enhanced “NoGo” reinforcement learning in non-medicated Parkinson’s patients (Frank et al. 2004; Cools et al. 2006), schizophrenic patients treated with antipsychotics (Waltz et al. 2007), and healthy participants with enhanced striatal D2 receptor genetic function (Frank et al. 2007a).
To capture catalepsy sensitization, we measured the response times for the simulated BG networks to select a response, and associated striatal Go and NoGo activations, as a function of experience. Simulated haloperidol led to NoGo unit disinhibition, Hebbian learning in the corticostriatal pathway, and progressively slowed RTs specific to the stimulus context which had been repeatedly paired with simulated drug administration. Thus, sensitization was context-dependent, as observed experimentally (Klein and Schmidt 2003). This catalepsy was incrementally extinguished when switched back to the intact mode, due to Go learning associated with the removal of the aversive stimulus, and inhibition of NoGo representations. Critically, after extinction, when networks were again challenged with simulated haloperidol, they exhibited substantially more catalepsy than a model that was never sensitized in the first place, as seen in rats (Amtage and Schmidt 2003). This latter effect was due to fact that NoGo representations were simply prevented from being expressed, and therefore from being unlearned, during extinction. The subsequent blockade of simulated D2 receptors uncovered this latent NoGo association.
Based on this finding, our model predicts that it may be possible to prevent the development of a non-extinguishable component by blocking Go learning via D1 receptor blockade during extinction. (To prevent the drug from inducing catalepsy itself, this procedure could be executed following the extinction session, which should prevent Go learning consolidation (e.g., Dalley et al. 2005)). In this case, we hypothesize that extinction will occur via unlearning of NoGo representations rather than new Go learning, such that the sensitized component will be entirely (or mostly) extinguishable even when re-challenged with haloperidol. If confirmed, such a result might hold practical importance for understanding and treating Parkinson’s symptoms. Levodopa, the main medication used to improve motor symptoms, induces immediate early gene expression associated with synaptic plasticity in striatonigral (Go), but not striatopallidal (NoGo), neurons (Carta et al. 2005; Knapska and Kaczmarek 2004). As such, the exaggerated potentiation of NoGo synapses in the DA-depleted state (Surmeier et al. 2007; Shen et al. 2008) may remain latent and may be uncovered once levodopa wears off, leading to the return of motor symptoms characterized by classical on/off states (e.g., Chen and Obering 2005).
Sensitization is not a unique property to aversive conditioning. Indeed, this same sensitization process is observed in response to amphetamine and drugs of abuse, where the strength of sensitization predicts relapse (e.g., Robinson and Berridge 2003; Schmidt and Beninger 2006). Furthermore, this sensitization is associated with an increase in striatal synaptic spine density (Li et al. 2004). Our models suggest a similar mechanism for reward-based sensitization, in that phasic DA reinforces contextual cues, but in this case involving postsynaptic D1-mediated Go learning in striatonigral neurons rather than NoGo sensitization in striatopallidal neurons. If our interpretation holds, it may also explain the high rates of relapse following rehabilitation: striatonigral Go neurons may never really unlearn the rewarding associations, which may only be prevented from being expressed during drug-free conditions. Overall, the above explanation is consonant with other evidence that extinction reflects new learning, rather than unlearning of original associations (Pavlov 1927; Bouton 2004; Redish et al. 2007).
It should further be mentioned that catalepsy is not uniquely induced by D2 antagonism. Several reports show that the selective D1 receptor antagonist SCH 23390 induces catalepsy as well (e.g., Morelli and Di Chiara 1985; Undie and Friedman 1988). In a preliminary study, we tested our model in response to simulated D1 receptor blockade and observed an increase in NoGo–Go activity and RTs, much like our haloperidol models. This result is not surprising because blocking the excitatory effect of dopaminergic projections onto striatal Go units leads to a reduction in Go activity, and hence an increase in catalepsy. Furthermore, simulated DA depletion as in Parkinson’s disease (Frank 2005, 2006) led to similar observations of catalepsy sensitization, thus raising the question of whether aspects of catalepsy in PD patients are partially learned via synaptic potentiation.
Prediction error versus novelty models of DA functioning: novel predictions
We also tested the implications of two distinct hypotheses of DA functioning. We showed that both the RPE and novelty hypotheses of phasic DA signals provide reasonable explanations for the observed behavior, but require different assumptions and make different predictions. The RPE model assumed that NoGo neurons learn specific associations to context A, which do not generalize to context B. In contrast, the novelty model need not assume separate NoGo representations of contexts A and B but instead assume that context B elicits a novelty-related DA burst that promotes Go signals and thereby overcomes the catalepsy that would be produced by NoGo activity. This idea is consistent with evidence showing that phasic DA bursts in response to a conditioned stimulus are associated with speeded RTs in that trial (Satoh et al. 2003).
Our behavioral experiment discriminates between these accounts and falsifies the novelty hypothesis: controlled manipulation of context B novelty had no effect on the context dependency of catalepsy expression. This result is thus consistent with the prediction generated by our reward prediction error model, which posits that NoGo learning occurred in striatopallidal neurons linking the sensory context (A) with a NoGo response.
Limitations
Despite our model’s success in accounting for different aspects of haloperidol-induced catalepsy sensitization, extinction, and renewal within an existing framework, the model has several neurobiological limitations that need to be addressed in future work.
First, we focus on haloperidol effects on the D2 receptor (to which it is most strongly bound) in the striatum (where there are by far the greatest number of D2 receptors (Camps et al. 1989), and which has been implicated in PD). However, it must be acknowledged that additionally, D2, D3, and D4 receptors are also likely to be blocked in the frontal cortex, olfactory bulb, amygdala, and hippocampus. Given limited data, it is not clear if these effects play a crucial role in synaptic plasticity changes induced by haloperidol, nor whether these structures are involved in catalepsy expression. Haloperidol effects on synaptic plasticity in the striatal D2 pathway on the other hand are well studied and suffice to provide an explanation for the observed phenomena and derive novel testable predictions.
Another effect not explicitly modeled is that haloperidol can also elevate striatal DA levels via concomitant blockade of presynaptic D2 autoreceptors (Wu et al. 2002; Garris et al. 2003; Frank and O’Reilly 2006). This increased DA would then stimulate D1 receptors, and could therefore actually enhance Go signals. Indeed, these presynaptic effects have been implicated in the delay to catalepsy onset (Garris et al. 2003). In humans, a single low dose of haloperidol can actually enhance Go learning, presumably via preferential presynaptic mechanisms (Frank and O’Reilly 2006). Nevertheless, with higher doses and chronic administration, the postsynaptic effect dominates (likely due to the greater excitability of NoGo than Go cells; Lei et al. (2004); Kreitzer and Malenka (2007)), leading to overall more NoGo activation (and learning). Thus, inclusion of autoreceptors effects would only delay the inevitable occurrence of catalepsy.
D2 receptors can also act via presynaptic heteroreceptors to regulate cortical glutamatergic input to striatum. Thus, blockade of these receptors would lead to stronger cortical input. Because cortical input is stronger onto NoGo than Go neurons (reviewed above), this effect would likely add to that resulting from postsynaptic D2 blockade. Nevertheless, explicit modeling of this mechanism may shed more light on its potential relevance.
Conclusion
In sum, we provided a neurocomputational account for a constellation of findings in the domain of haloperidol-induced catalepsy sensitization. The model used to generate the findings is the same which has accounted for differential patterns of learning in humans on and off DA medications. The current findings extend the generality of the model to observations in a completely different experimental procedure, setting, and species. The behavioral experiment suggests that the reward prediction error model is more suitable than the novelty model to explain the observed phenomena.
Notes
Importantly, whereas D1 receptors require substantial phasic DA bursts to get activated, high-affinity D2 receptors are more sensitive and are inhibited by relatively low levels of tonic DA (e.g., Goto and Grace 2005). In the model, NoGo learning depends on the extent to which DA is removed from the synapse during DA dips, such that longer duration pauses in DA firing would be associated with lower DA levels and stronger learning signals. Notably, larger negative prediction errors are associated with longer DA pause durations of up to 400 ms (Bayer et al. 2007), and the half-life of DA in the striatal synapse is 55–75 ms (Gonon 1997; Venton et al. 2003).
The classical “indirect” pathway (Albin et al. 1989) involved inhibitory GPe projections to the subthalamic nucleus (STN), which then excited GPi. However, more recent evidence embedded in our model suggests that the STN forms part of a third “hyper-direct” pathway linking cortex to GPi (Frank 2006), and that the NoGo pathway involves striatum–GPe–GPi.
This value was chosen arbitrarily; other settings produce the same patterns.
In actuality, this separation of contextual representations is likely to depend on the hippocampus (Nadel and Willner 1980; Myers and Gluck 1994; Rudy and O’Reilly 1999). Because we focus on the striatal mechanism by which haloperidol produces catalepsy, we simplify this hippocampal aspect and simply represent context as a set of different sensory input units. In the ‘novelty’ simulations below, we do not make this assumption.
The key issue here is whether the striatum has access to highly separated contextual input, as would likely be represented in the hippocampus (Nadel and Willner 1980; Myers and Gluck 1994; Rudy and O’Reilly 1999). Our previous simulations assume that they do, given the well-known hippocampal input to ventral striatum. However, the degree to which striatal representations associated with catalepsy expression, likely in the dorsal striatum, are influenced by these contextual inputs is unknown. We therefore simulate the opposite extreme here, in which the contexts are represented identically in the striatal inputs, except for the peripheral influence of novelty-induced DA bursts.
By ‘non-extinguishable’, we mean in the context of experimental procedures that produce robust extinction under placebo. It is of course theoretically possible that a longer extinction phase would not be followed by renewed catalepsy expression when challenged with haloperidol.
References
Albin RL, Young AB, Penney JB (1989) The functional anatomy of basal ganglia disorders. Trends Neurosci 12:366–375
Alexander GE, Crutcher MD (1990) Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci 13:266–271
Amtage J, Schmidt WJ (2003) Context-dependent catalepsy intensification is due to classical conditioning and sensitization. Behav Pharmacol 14(7):563–567
Antelman S, Kocan D, Edwards D, Knopf S, Perel J, Stiller R (1986) Behavioral effects of a single neuroleptic treatment grow with the passage of time. Brain Res 385(1):58–67
Barnes D, Robinson B, Csernansky J, Bellows E (1990) Sensitization versus tolerance to haloperidol-induced catalepsy: multiple determinants. Pharmacol Biochem Behav 36(4):883–887
Basso MA, Wurtz RH (2002) Neuronal activity in substantia nigra pars reticulata during target selection. J Neurosci 22(5):1883–1894
Bayer HM, Lau B, Glimcher PW (2007) Statistics of midbrain dopamine neuron spike trains in the awake primate. J Neurophysiol 98(3):1428–1439
Bergman H, Feingold A, Nini A, Raz A, Slovin H, Abeles M, Vaadia E (1998) Physiological aspects of information processing in the basal ganglia of normal and parkinsonian primates. Trends Neurosci 21:32–38
Boraud T, Bezard E, Bioulac B, Gross E (2002) From single extracellular unit recording in experimental and human Parkinsonism to the development of a functional concept of the role played by the basal ganglia in motor control. Prog Neurobiol 66(4):265–283
Bouton ME (2004) Context and behavioral processes in extinction. Learn Mem 11(5):485–494
Brown JW, Bullock D, Grossberg S (2004) How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades. Neural Netw 17:471–510
Calabresi P, Picconi B, Tozzi A, Di Filippo M (2007) Dopamine-mediated regulation of corticostriatal synaptic plasticity. Trends Neurosci 30(5):211–219
Camps M, Cortes R, Gueye B, Probst A, Palacios JM (1989) Dopamine receptors in the human brain: autoradiographic distribution of D sites. Neuroscience 28:275–290
Carta AR, Tronci E, Pinna A, Morelli M (2005) Different responsiveness of striatonigral and striatopallidal neurons to L-DOPA after a subchronic intermittent L-DOPA treatment. Eur J Neurosci 21(5):1196–1204
Centonze D, Picconi B, Gubellini P, Bernardi G, Calabresi P (2001) Dopaminergic control of synaptic plasticity in the dorsal striatum. Eur J Neurosci 13:1071–1077
Centonze D, Usiello A, Costa C, Picconi B, Erbs E, Bernardi G, Borrelli E, Calabresi P (2004) Chronic haloperidol promotes corticostriatal long-term potentiation by targeting dopamine D2L receptors. J Neurosci 24:8214–8222
Chen JJ, Obering C (2005) A review of intermittent subcutaneous apomorphine injections for the rescue management of motor fluctuations associated with advanced Parkinson’s disease. Clin Ther 27(11):1710–1724
Cools R, Altamirano L, D’Esposito M (2006) Reversal learning in Parkinson’s disease depends on medication status and outcome valence. Neuropsychologia 44:1663–1673
Dalley JW, Lääne K, Theobald DEH, Armstrong HC, Corlett PR, Chudasama Y, Robbins TW (2005) Time-limited modulation of appetitive Pavlovian memory by D1 and NMDA receptors in the nucleus accumbens. Proc Natl Acad Sci USA 102(17):6189–6194
Frank MJ (2005) Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and non-medicated Parkinsonism. J Cogn Neurosci 17:51–72
Frank MJ (2006) Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making. Neural Netw 19:1120–1136
Frank MJ, O’Reilly RC (2006) A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav Neurosci 120:497–517
Frank S, Schmidt W (2003) Burst activity of spiny projection neurons in the striatum encodes superimposed muscle tetani in cataleptic rats. Exp Brain Res 152(4):519–522
Frank MJ, Seeberger LC, O’Reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in Parkinsonism. Science 306:1940–1943
Frank MJ, Moustafa AA, Haughey H, Curran T, Hutchison K (2007a) Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci USA 104:16,311–16,316
Frank MJ, Samanta J, Moustafa AA, Sherman SJ (2007b) Hold your horses: impulsivity, deep brain stimulation and medication in parkinsonism. Science 318:1309–1312
Frank MJ, Santamaria A, O’Reilly RC, Willcutt E (2007c) Testing computational models of dopamine and noradrenaline dysfunction in attention deficit/hyperactivity disorder. Neuropsychopharmacology 32:1583–1599
Frank MJ, Scheres A, Sherman SJ (2007d) Understanding decision making deficits in neurological conditions: insights from models of natural action selection. Philos Trans R Soc Lond B 362:1641–1654
Garris PA, Budygin EA, Phillips PEM, Venton BJ, Robinson DL, Bergstrom BP, Rebec GV, Wightman RM (2003) A role for presynaptic mechanisms in the actions of nomifensine and haloperidol. Neuroscience 118:819–829
Gerfen CR (1992) The neostriatal mosaic: multiple levels of compartmental organization in the basal ganglia. Annu Rev Neurosci 15:285–320
Gerfen CR (2000) Molecular effects of dopamine on striatal projection pathways. Trends Neurosci 23:S64–S70
Gerfen CR, Keefe KA, Gauda EB (1995) D and D dopamine receptor function in the striatum: coactivation of D- and D-dopamine receptors on separate populations of neurons results in potentiated immediate early gene response in D-containing neurons. J Neurosci 15:8167–8176
Gonon FJ (1997) Prolonged and extrasynaptic excitatory action of dopamine mediated by D1 receptors in the rat striatum in vivo. J Neurosci 17:5972–5978
Goto Y, Grace AA (2005) Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior. Nat Neurosci 8:805–812
Graybiel AM (2000) The basal ganglia. Curr Biol 10(14):R509–511
Gurney K, Prescott TJ, Redgrave P (2001) A computational model of action selection in the basal ganglia. II. Analysis and simulation of behaviour. Biol Cybern 84(6):411–423
Håkansson K, Galdi S, Hendrick J, Snyder G, Greengard P, Fisone G (2006) Regulation of phosphorylation of the GluR1 AMPA receptor by dopamine D2 receptors. J Neurochem 96(2):482–488
Jackson ME, Moghaddam B (2004) Stimulus-specific plasticity of prefrontal cortex dopamine neurotransmission. J Neurochem 88(6):1327–1334
Jiang H, Stein BE, Mchaffie JG (2003) Opposing basal ganglia processes shape midbrain visuomotor activity bilaterally. Nature 423(6943):982–986
Joel D, Weiner I (1999) Striatal contention scheduling and the split circuit scheme of basal ganglia–thalamocortical circuitry: from anatomy to behaviour. In: Miller R, Wickens JR (eds) Conceptual advances in brain research: brain dynamics and the striatal complex. Harwood Academic, Amsterdam, pp 209–236
Kakade S, Dayan P (2002) Dopamine: generalization and bonuses. Neural Netw 15:549–559
Klein A, Schmidt WJ (2003) Catalepsy intensifies context-dependently irrespective of whether it is induced by intermittent or chronic dopamine deficiency. Behav Pharmacol 14(1):49–53
Knapska E, Kaczmarek L (2004) A gene for neuronal plasticity in the mammalian brain: Zif268/egr-1/ngfi-a/krox-24/tis8/zenk? Prog Neurobiol 74(4):183–211
Kreitzer AC, Malenka RC (2007) Endocannabinoid-mediated rescue of striatal LTD and motor deficits in parkinson’s disease models. Nature 445(7128):643–647
Lanis A, Schmidt W (2001) NMDA receptor antagonists do not block the development of sensitization of catalepsy, but make its expression state-dependent. Behav Pharmacol 12(2):143
Lei W, Jiao Y, Del Mar N, Reiner A (2004) Evidence for differential cortical input to direct pathway versus indirect pathway striatal projection neurons in rats. J Neurosci 24(38):8289–8299
Li Y, Acerbo MJ, Robinson TE (2004) The induction of behavioural sensitization is associated with cocaine-induced structural plasticity in the core (but not shell) of the nucleus accumbens. Eur J Neurosci 20(6):1647–1654
Lisman JE, Grace AA (2005) The hippocampal-VTA loop: controlling the entry of information into long-term memory. Neuron 46:703–713
Mallet N, Ballion B, Moine CL, Gonon F (2006) Cortical inputs and GABA interneurons imbalance projection neurons in the striatum of parkinsonian rats. J Neurosci 26(14):3875–3884
Mink JW (1996) The basal ganglia: focused selection and inhibition of competing motor programs. Prog Neurobiol 50(4):381–425
Morelli M, Di Chiara G (1985) Catalepsy induced by SCH 23390 in rats. Eur J Pharmacol 117(2):179–185
Moustafa AA, Cohen MX, Sherman SJ, Frank MJ (2008) A role for dopamine in temporal decision making and reward maximization in parkinsonism. J Neurosci 28:12,294–12,304
Myers CE, Gluck MA (1994) Context, conditioning, and hippocampal representation in animal learning. Behav Neurosci 108:835–847
Nadel L, Willner J (1980) Context and conditioning: a place for space. Physiol Psychol 8:218–228
Parent A, Hazrati L (1995) Functional anatomy of the basal ganglia. II. The place of subthalamic nucleus and external pallidum in basal ganglia circuitry. Brain Res Rev 20:128–154
Pavlov IP (1927) Conditioned reflexes: an investigation of the physiological activity of the cerebral cortex. Oxford University Press, London
Redgrave P, Gurney K (2006) The short-latency dopamine signal: a role in discovering novel actions. Nat Rev Neurosci 7(12):967–975
Redgrave P, Prescott TJ, Gurney K (1999) The basal ganglia: a vertebrate solution to the selection problem. Neuroscience 89(4):1009–1023
Redish AD, Jensen S, Johnson A, Kurth-Nelson Z (2007) Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol Rev 114(3):784–805
Reynolds GP (1992) Developments in the drug treatment of schizophrenia. Trends Pharm Sci 13(3):116–121
Reynolds JN, Wickens JR (2002) Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw 15:507–521
Reynolds JNJ, Hyland BI, Wickens JR (2001) A cellular mechanism of reward-related learning. Nature 412:67–69
Robertson GS, Vincent SR, Fibiger HC (1992) D and D dopamine receptors differentially regulate c-fos expression in striatonigral and stiratopallidal neurons. Neuroscience 49:285–296
Robinson TE, Berridge KC (2003) Addiction. Annu Rev Psychol 54:25–53
Robinson T, Kolb B (1997) Persistent structural modifications in nucleus accumbens and prefrontal cortex neurons produced by previous experience with amphetamine. J Neurosci 17(21):8491–8497
Rudy JW, O’Reilly RC (1999) Contextual fear conditioning, conjunctive representations, pattern completion, and the hippocampus. Behav Neurosci 113:867–880
Salin P, Hajji MD, Kerkerian-Le Goff L (1996) Bilateral 6-hydroxydopamine-induced lesion of the nigrostriatal dopamine pathway reproduces the effects of unilateral lesion on substance P but not on enkephalin expression in rat basal ganglia. Eur J Neurosci 8:1746–1757
Sanberg PR (1980) Haloperidol-induced catalepsy is mediated by postsynaptic dopamine receptors. Nature 284:472–473
Santesso D, Evins A, Frank M, Cowman E, Pizzagalli D (2009) Single dose of a dopamine agonist impairs reinforcement learning in humans: Converging evidence from electrophysiology and computational modeling of striatal–cortical function. Hum Brain Mapp (in press)
Satoh T, Nakai S, Sato T, Kimura M (2003) Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci 23:9913–9923
Schmidt WJ, Beninger RJ (2006) Behavioural sensitization in addiction, schizophrenia, parkinson’s disease and dyskinesia. Neurotox Res 10(2):161–166
Schmidt W, Tzschentke T, Kretschmer B (1999) State-dependent blockade of haloperidol-induced sensitization of catalepsy by MK-801. Eur J Neurosci 11(9):3365–3368
Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593
Seeman P (2008) Dopamine D2(high) receptors on intact cells. Synapse 62(4):314–318
Shen W, Flajolet M, Greengard P, Surmeier DJ (2008) Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321(5890):848–851
Srinivasan J, Schmidt W (2004) Intensification of cataleptic response in 6-hydroxydopamine-induced neurodegeneration of substantia nigra is not dependent on the degree of dopamine depletion. Synapse 51(3):213–218
Surmeier DJ, Ding J, Day M, Wang Z, Shen W (2007) D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends Neurosci 30(5):228–235
Undie A, Friedman E (1988) Differences in the cataleptogenic actions of SCH23390 and selected classical neuroleptics. Psychopharmacology 96(3):311–316
Venton BJ, Zhang H, Garris PA, Phillips PEM, Sulzer D, Wightman RM (2003) Real-time decoding of dopamine concentration changes in the caudate-putamen during tonic and phasic firing. J Neurochem 87(5):1284–1295
Waltz JA, Frank MJ, Robinson BM, Gold JM (2007) Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal–cortical dysfunction. Biol Psychiatry 62:756–764
Wu Q, Reith MEA, Walker QD, Kuhn CM, Caroll FI, Garris PA (2002) Concurrent autoreceptor-mediated control of dopamine release and uptake during neurotransmission: an in vivo voltammetric study. J Neurosci 22:6272–6281
Author contributions
All modeling was performed by Thomas V. Wiecki with supervision of Michael J. Frank and Werner J. Schmidt. The behavioral experiment was planned and analyzed by Katrin Riedinger and Andreas von Ameln-Mayerhofer. The paper was written by Thomas V. Wiecki and Michael J. Frank. Katrin Riedinger and Andreas von Ameln-Mayerhofer wrote the section “Behavioral experiment”.
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Author information
Authors and Affiliations
Corresponding author
Additional information
W.J. Schmidt (deceased).
We thank Ferdinand Kluge for the execution of the behavioral experiment and Christian D. Wilms for useful comments on the manuscript. This work was supported by a National Institute of Mental Health grant R01 MH080066-01 awarded to M.J.F.
Electronic supplementary material
Below is the link to the electronic supplementary material
ESM 1
(PDF 67.9 KB)
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Wiecki, T.V., Riedinger, K., von Ameln-Mayerhofer, A. et al. A neurocomputational account of catalepsy sensitization induced by D2 receptor blockade in rats: context dependency, extinction, and renewal. Psychopharmacology 204, 265–277 (2009). https://doi.org/10.1007/s00213-008-1457-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00213-008-1457-4