A neurocomputational account of catalepsy sensitization induced by D2 receptor blockade in rats: context dependency, extinction, and renewal
- First Online:
Repeated haloperidol treatment in rodents results in a day-to-day intensification of catalepsy (i.e., sensitization). Prior experiments suggest that this sensitization is context-dependent and resistant to extinction training.
The aim of this study was to provide a neurobiological mechanistic explanation for these findings.
Materials and methods
We use a neurocomputational model of the basal ganglia and simulate two alternative models based on the reward prediction error and novelty hypotheses of dopamine function. We also conducted a behavioral rat experiment to adjudicate between these models. Twenty male Sprague–Dawley rats were challenged with 0.25 mg/kg haloperidol across multiple days and were subsequently tested in either a familiar or novel context.
Simulation results show that catalepsy sensitization, and its context dependency, can be explained by “NoGo” learning via simulated D2 receptor antagonism in striatopallidal neurons, leading to increasingly slowed response latencies. The model further exhibits a non-extinguishable component of catalepsy sensitization due to latent NoGo representations that are prevented from being expressed, and therefore from being unlearned, during extinction. In the rat experiment, context dependency effects were not dependent on the novelty of the context, ruling out the novelty model’s account of context dependency.
Simulations lend insight into potential complex mechanisms leading to context-dependent catalepsy sensitization, extinction, and renewal.
KeywordsCatalepsy Sensitization Basal ganglia D2 antagonist Haloperidol Computational models
Haloperidol is a potent typical antipsychotic used with high affinity to the dopamine (DA) D2 receptors. In laboratory animals, it is used to model the extrapyramidal side effects of neuroleptics therapy: Rats treated with haloperidol show symptoms similar to those observed in Parkinson’s Disease (PD) and include cognitive learning deficits as well as akinesia and rigidity (i.e., catalepsy), effects that are mediated by blockade of striatal D2 receptors (Sanberg 1980).
Interestingly, repeated administration of haloperidol leads to an intensification of catalepsy following each consecutive test—a process known as sensitization (Schmidt and Beninger 2006; Schmidt et al. 1999; Lanis and Schmidt 2001; Frank and Schmidt 2003; Barnes et al. 1990; Antelman et al. 1986). This same catalepsy sensitization does not occur only under repeated administration of haloperidol but also in DA-deficient animals (bilateral striatal 6-OHDA-lesion) (Klein and Schmidt 2003; Srinivasan and Schmidt 2004).
Similarly, antipsychotic medications do not improve symptoms of psychosis until a delay of days up to weeks (Reynolds 1992) implicating the involvement of sensitization processes in the therapy of schizophrenia.
Despite several years of research into these phenomena, a well elaborated mechanistic explanation for these observations is still lacking. A key observation might be that haloperidol-induced catalepsy sensitization could be related to changes in synaptic strength (i.e., learning) within the striatum—a brain region with a high rate of neuroplastic changes modulated by DA (e.g., Calabresi et al. 2007; Centonze et al. 2001; Robinson and Kolb 1997). Indeed, chronic haloperidol enhances synaptic plasticity via D2 receptor blockade (Centonze et al. 2004) and phosphorylation of GluR1 AMPA receptors in striatopallidal neurons (Håkansson et al. 2006). Haloperidol potently blocks dopaminergic D2 receptors which are primarily found in the indirect pathway of the basal ganglia (BG) (Gerfen 2000; Salin et al. 1996; Boraud et al. 2002; Robertson et al. 1992; Gerfen et al. 1995; Surmeier et al. 2007) and increases spike frequency in striatal spiny I neurons (Frank and Schmidt 2003). Neurons in this same pathway are also hyperactive or show abnormal burst firing in Parkinsonism (Albin et al. 1989; Bergman et al. 1998; Mallet et al. 2006).
In this paper, we focus on this well-established striatal D2 mechanism of haloperidol, in an attempt to account for catalepsy sensitization (see “Discussion” for other mechanisms). To explore the complex dynamic interactions among BG sub-regions in response to haloperidol, we use an explicit, computational model of the BG (Frank 2006), which is also grounded by its ability to account for other dopamine-dependent learning-related phenomena and to make predictions that have subsequently been confirmed via pharmacological manipulations in humans (Frank et al. 2004, 2007c; Frank 2005; Frank and O’Reilly 2006; Cools et al. 2006; Santesso et al. 2009; Moustafa et al. 2008). Here, we report that this same model also reproduces the effects of haloperidol on sensitization, context dependency, extinction, and renewal.
Materials and methods
Model: high level overview
The role of the BG can be seen as that of a dynamic modulator of frontal cortical action plans. With respect to motor control, the BG could function as an action selection device (Graybiel 2000; Redgrave et al. 1999; Frank 2005): efferent projections from the motor cortex reach the BG, which then facilitate appropriate motor commands while suppressing those that are inappropriate (Basso and Wurtz 2002; Brown et al. 2004; Gurney et al. 2001; Jiang et al. 2003; Mink 1996; Redgrave et al. 1999). These two functions can be supported by two separate striatofugal neural projections (Albin et al. 1989; Alexander and Crutcher 1990). The direct striatonigral pathway (expressing high levels of D1 receptors) functions as the “Go-” pathway, by facilitating the selection of particular actions when appropriate in a given sensory context. In contrast, neurons originating in the striato–pallidal–nigral pathway (expressing high levels of D2 receptors) function as the “NoGo-” pathway, by detecting the conditions in which a given action should be suppressed and counteracting the Go pathway at the level of BG output (Frank 2005).
According to the reward prediction error (RPE) hypothesis, midbrain dopaminergic neurons signal when outcomes are better or worse than expected via phasic bursts and pauses in firing (Schultz et al. 1997). Reward-associated behavior is potentiated by activation of D1 receptors and synaptic plasticity in the Go pathway following dopamine bursts (Reynolds and Wickens 2002). Conversely, maladaptive behaviors are suppressed via disinhibition of D2 receptors and potentiation of NoGo cells following dopamine dips (Frank 2005). Recent studies support these dual mechanisms of plasticity: D1 stimulation potentiates corticostriatal Go synapses, whereas a lack of D2 receptor stimulation (simulating the effect of a DA dip) was required to potentiate NoGo synapses (Shen et al. 2008).1 Furthermore, this NoGo learning effect would be enhanced by D2 receptor sensitivity (Seeman 2008) and enhanced excitability of striatopallidal NoGo cells in the DA-depleted state (Surmeier et al. 2007; Shen et al. 2008).
Some question the RPE hypothesis of DA signaling altogether, suggesting that the timing of DA signals is too early to encode these errors (Redgrave et al. 1999; Redgrave and Gurney 2006). They argue that the functional role of phasic dopaminergic neuron firing is to reinforce the development of a novel action, rather than unpredicted reward per se, in response to a salient or novel stimulus (for recent reviews, see Redgrave and Gurney (2006) and Lisman and Grace (2005)). This theory was addressed in our second experiment (see below).
BG model functionality
A common measurement of catalepsy in rats is the bar test (e.g., Amtage and Schmidt 2003; Klein and Schmidt 2003) in which the animal is placed to stand with its forepaws on an elevated bar and the time until the first movement occurs is taken. But how does one simulate catalepsy in a computational model? Just as rats can take a longer time to descend off the bar, the BG model can take varying amounts of time to facilitate a response. To measure the latency until an action is selected in the model (hereafter, “response time” (RT)), we assessed the number of network processing cycles (see Electronic supplementary material) before a response was selected by the BG action selection network, i.e., until one of the thalamus units was disinhibited by BG circuitry. When a thalamus unit is activated, the corresponding response is swiftly executed (Frank 2006). Thus, catalepsy is associated with longer latencies to gate responses. These same methods were employed in previous model RT analyses (Frank et al. 2007b, d; Moustafa et al. 2008). Because the BG gating system is required to facilitate a cortical response, similar results are obtained by probing output unit activity.
To gain insight into underlying processes leading to different RTs, we additionally probed striatal unit activity. As described above, the BG model simulates Go and NoGo neuronal populations in the striatum, which facilitate and suppress responses, respectively. If a Go population for a given response is more active than its NoGo counterpart, that response is more likely to be facilitated, thus the relative difference in Go–NoGo activity influences the speed at which the response is executed (Moustafa et al. 2008). If NoGo activity is relatively greater than Go activity, as seen in Parkinsonism, a response may not be selected at all by the BG; therefore the NoGo–Go contrast reflects an internal (“hidden variable”) measure of catalepsy (see Electronic supplementary material for the precise computation).
To simulate the partial blockade of postsynaptic D2 receptors by haloperidol, we reduced the strength of the inhibitory SNc D2 projection onto NoGo neurons to 10% that of the original, representing a 90% occupation of D2 receptors by haloperidol3.
Reward prediction error model
According to the RPE hypothesis, DA bursts signal unexpected reward and DA dips signal the lack of expected reward. However, in the aforementioned experiments on catalepsy sensitization, neither explicit reward nor punishment was used following motor responses. However, we reasoned that because the bar test is somewhat aversive (the animal does not want to be on the bar and therefore descends), the escape from aversive conditions may be associated with a positive DA burst. Indeed, there is evidence that an offset of an aversive stimulus is associated with increased striatal DA (Jackson and Moghaddam 2004). Accordingly, we applied a small DA burst following response execution during training.
The network was trained for 60 trials in the haloperidol mode in context A (represented by a set of four sensory input units), and then tested in that context and in an untrained context B (corresponding to a different set of input units).4 During this testing procedure, the network’s weights were prevented from changing, so as to prevent learning in the test and to permit multiple tests across training (something which would have to be done between subjects in actual experiments). Next, we simulated extinction by continuing training for a further 40 trials with the network switched from haloperidol mode to the intact state (i.e., weights of the SNc→NoGo projections at 100%). Finally, the haloperidol mode was simulated for an additional five test trials, to determine whether the model still demonstrates sensitized catalepsy after extinction.
We further tested whether an implementation of the novelty hypothesis could account for the same findings, and if so, whether the two models make divergent predictions. In this case, it is the novelty of a stimulus, not the RPE associated with it that drives a DA burst. Accordingly, the apparent context dependency of catalepsy sensitization could arise simply because the animal is not familiar with context B; the associated novelty-driven DA burst (Lisman and Grace 2005; Kakade and Dayan 2002) could activate the Go pathway, promote locomotion and exploratory behavior, and thereby lead to reduction in catalepsy. Note that this hypothesis does not require the assumption that any NoGo learning is specific to sensory input units encoding the external context A. Instead, this learning might generalize across contexts, but the context dependency arises due to the novelty of surrounding context B which drives a DA burst that counteracts catalepsy expression.5
After completion of simulation studies and on a suggestion from an anonymous reviewer, we conducted a simple behavioral experiment to distinguish between the RPE and novelty models. This experiment is similar to the context challenge experiment (Klein and Schmidt 2003), but an additional group of animals was habituated to context B (without haloperidol) prior to the sensitization phase, in order to eliminate the novelty of this context. If the novelty model is correct, we would expect to see continued catalepsy expression (or a smaller reduction in catalepsy) during the context challenge in this group, because the context is no longer novel and there should therefore not be a novelty-driven DA burst. In contrast, if the RPE model is correct, all animals should show reduced catalepsy in context B regardless of whether it is novel, because stimulus-NoGo learning only occurred in context A.
A total of 20 male Sprague–Dawley rats (230–260 g at the beginning of the experiment), Charles River, Sulzfeld, Germany, were used. The animals were group-housed (four animals per cage, in standard macrolon IV cages) under a 12/12 light–dark cycle with restricted access to food (12 g per animal per day). Access to water was unconfined (i.e., ad libitum).
The neuroleptic agent haloperidol (Haldol®-injection solution, Janssen, Germany) was diluted in saline (0.9% NaCl), Fresenius, Germany to a concentration of 0.25 mg/ml. Substance administration was carried out subcutaneously (s.c.), at 1 ml/kg body weight, the same concentration used in Amtage and Schmidt (2003) and Klein and Schmidt (2003).
To test for catalepsy, the animals performed a bar test. Within that test, a single rat was put gently with its forepaws on a horizontal bar (9 cm above the table surface, diameter of 0.5 cm). The descent latency, as a proxy for the degree of catalepsy, was measured by taking the time interval between the first placement of the animal on the apparatus and its first active paw movement. This procedure is identical to the one used in Amtage and Schmidt (2003) and Klein and Schmidt (2003).
To test for context dependency, two contexts (A and B) were used. The context consisted of a different room (with different lighting) and a different lab coat of the experimenter (in context A, the experimenter wore a white lab coat; while in context B, a black plastic poncho was put over the lab coat). These context cues are very similar to those used in Klein and Schmidt (2003).
The rats were handled for five consecutive days prior to the first catalepsy test. During the following habituation phase, the animals received a (s.c.) saline injection and were tested 60 min later. The treatment during the habituation phase took place in context A for the first (non-habituated) group (n = 10) and in context B for the second (habituated) group (n = 10).
After the habituation phase, catalepsy sensitization was performed for both groups in context A, for a total of 9 days (after the seventh sensitization day, there was a lack of testing for 2 days). On the first day after sensitization (day 17), both groups were tested in context B to induce the context challenge. On day 18, both groups were retested in context A.
Statistical analysis was performed using GB STAT 7.0. Multiple values within a group, where tested with the non-parametric Friedman ANOVA. Two values of one group were submitted to the Wilcoxon signed rank test. To compare individual data between two groups, we used the Mann Whitney U test.
Reward prediction error model results
In contrast to sensitization in context A, simulated catalepsy was roughly constant in context B regardless of the number of training trials with haloperidol in context A. We also confirmed that this context dependency arose due to differences in weights between context input units to the striatum (data not shown). Weights from the context A input neurons to the NoGo neurons increased, while those from context B units did not, due to the dependency of Hebbian learning on both pre- and postsynaptic activation (see equation A-8 in the Electronic supplementary material). Because the model was never trained with context B units active, its NoGo weights to the striatum did not increase. Thus, the model replicates the context dependency of sensitized haloperidol observed in rodents.
The model also captures extinction (Fig. 4a, b). After we switched the model from simulated haloperidol to the intact mode, cataleptic activity progressively decreased, reaching its starting value by the end of extinction. Again, these effects can be explained by examining the weights from the input units to the striatum. Initially, the network exhibited cataleptic activity in context A (despite being in the intact mode), due to prior NoGo learning. However, because the DA units can now inhibit NoGo units, the Go units were now free to fire more (due to less inhibitory competition from NoGo units). The DA bursts following response selection (corresponding to the offset of the aversive stimulus) also led to Go learning during this time, and thus a reduction in catalepsy.
Finally, the haloperidol-trained network also exhibited a sensitized component that was resistant to extinction. As shown in Fig. 4, a switch back from intact to haloperidol mode in trial 100 was associated with a prominent rise of cataleptic activity (increased RT and associated NoGo activity) in context A. This sensitized catalepsy was observed despite the previous 60 trials of extinction training in which catalepsy was reduced back to baseline and was far greater than that observed in networks which had never undergone haloperidol sensitization. This qualitative pattern of data matches that observed in rats (Amtage and Schmidt 2003).
What are the underlying mechanisms that cause this non-extinguishable component6? Intriguingly, examination of the weights from the input to NoGo units revealed that the weights did not substantially decrease during extinction—that is, there was relatively little unlearning of prior NoGo associations. Instead, the steady decrease of cataleptic activity resulted primarily from an increase in Go weights during extinction. When switching from haloperidol to intact mode, the now intact SNc→NoGo projections inhibited the striatal NoGo neurons, which prevented these neurons from changing their weights due to the Hebbian learning rule. Consequently, the previously learned A→NoGo association was maintained, but was only prevented from being expressed, during extinction. Thus, when the model was ultimately switched back to haloperidol mode, this prior learning was then immediately uncovered. Finally, note that when the model was tested in context B in trial 100, there was not a large increase in cataleptic activity, due to the specificity of learned NoGo weights. Thus, according to this model implementation, we would expect the sensitized component to be context-dependent.
Novelty hypothesis model results
Behavioral experiment results
In the present study, we explored possible neural mechanisms of haloperidol-induced catalepsy sensitization, using a computational model of the BG (Frank 2006). The model suggests that this catalepsy sensitization reflects a form of “NoGo” learning to suppress action execution, caused by disinhibition of striatopallidal neurons expressing D2 receptors in the basal ganglia. This notion is supported by studies showing that chronic haloperidol administration promotes synaptic potentiation in corticostriatal projections (Centonze et al. 2004), an effect that appears to be specific to NoGo/indirect pathway neurons (Håkansson et al. 2006). Thus, we posit catalepsy sensitization to result from the same mechanism that leads to relatively enhanced “NoGo” reinforcement learning in non-medicated Parkinson’s patients (Frank et al. 2004; Cools et al. 2006), schizophrenic patients treated with antipsychotics (Waltz et al. 2007), and healthy participants with enhanced striatal D2 receptor genetic function (Frank et al. 2007a).
To capture catalepsy sensitization, we measured the response times for the simulated BG networks to select a response, and associated striatal Go and NoGo activations, as a function of experience. Simulated haloperidol led to NoGo unit disinhibition, Hebbian learning in the corticostriatal pathway, and progressively slowed RTs specific to the stimulus context which had been repeatedly paired with simulated drug administration. Thus, sensitization was context-dependent, as observed experimentally (Klein and Schmidt 2003). This catalepsy was incrementally extinguished when switched back to the intact mode, due to Go learning associated with the removal of the aversive stimulus, and inhibition of NoGo representations. Critically, after extinction, when networks were again challenged with simulated haloperidol, they exhibited substantially more catalepsy than a model that was never sensitized in the first place, as seen in rats (Amtage and Schmidt 2003). This latter effect was due to fact that NoGo representations were simply prevented from being expressed, and therefore from being unlearned, during extinction. The subsequent blockade of simulated D2 receptors uncovered this latent NoGo association.
Based on this finding, our model predicts that it may be possible to prevent the development of a non-extinguishable component by blocking Go learning via D1 receptor blockade during extinction. (To prevent the drug from inducing catalepsy itself, this procedure could be executed following the extinction session, which should prevent Go learning consolidation (e.g., Dalley et al. 2005)). In this case, we hypothesize that extinction will occur via unlearning of NoGo representations rather than new Go learning, such that the sensitized component will be entirely (or mostly) extinguishable even when re-challenged with haloperidol. If confirmed, such a result might hold practical importance for understanding and treating Parkinson’s symptoms. Levodopa, the main medication used to improve motor symptoms, induces immediate early gene expression associated with synaptic plasticity in striatonigral (Go), but not striatopallidal (NoGo), neurons (Carta et al. 2005; Knapska and Kaczmarek 2004). As such, the exaggerated potentiation of NoGo synapses in the DA-depleted state (Surmeier et al. 2007; Shen et al. 2008) may remain latent and may be uncovered once levodopa wears off, leading to the return of motor symptoms characterized by classical on/off states (e.g., Chen and Obering 2005).
Sensitization is not a unique property to aversive conditioning. Indeed, this same sensitization process is observed in response to amphetamine and drugs of abuse, where the strength of sensitization predicts relapse (e.g., Robinson and Berridge 2003; Schmidt and Beninger 2006). Furthermore, this sensitization is associated with an increase in striatal synaptic spine density (Li et al. 2004). Our models suggest a similar mechanism for reward-based sensitization, in that phasic DA reinforces contextual cues, but in this case involving postsynaptic D1-mediated Go learning in striatonigral neurons rather than NoGo sensitization in striatopallidal neurons. If our interpretation holds, it may also explain the high rates of relapse following rehabilitation: striatonigral Go neurons may never really unlearn the rewarding associations, which may only be prevented from being expressed during drug-free conditions. Overall, the above explanation is consonant with other evidence that extinction reflects new learning, rather than unlearning of original associations (Pavlov 1927; Bouton 2004; Redish et al. 2007).
It should further be mentioned that catalepsy is not uniquely induced by D2 antagonism. Several reports show that the selective D1 receptor antagonist SCH 23390 induces catalepsy as well (e.g., Morelli and Di Chiara 1985; Undie and Friedman 1988). In a preliminary study, we tested our model in response to simulated D1 receptor blockade and observed an increase in NoGo–Go activity and RTs, much like our haloperidol models. This result is not surprising because blocking the excitatory effect of dopaminergic projections onto striatal Go units leads to a reduction in Go activity, and hence an increase in catalepsy. Furthermore, simulated DA depletion as in Parkinson’s disease (Frank 2005, 2006) led to similar observations of catalepsy sensitization, thus raising the question of whether aspects of catalepsy in PD patients are partially learned via synaptic potentiation.
Prediction error versus novelty models of DA functioning: novel predictions
We also tested the implications of two distinct hypotheses of DA functioning. We showed that both the RPE and novelty hypotheses of phasic DA signals provide reasonable explanations for the observed behavior, but require different assumptions and make different predictions. The RPE model assumed that NoGo neurons learn specific associations to context A, which do not generalize to context B. In contrast, the novelty model need not assume separate NoGo representations of contexts A and B but instead assume that context B elicits a novelty-related DA burst that promotes Go signals and thereby overcomes the catalepsy that would be produced by NoGo activity. This idea is consistent with evidence showing that phasic DA bursts in response to a conditioned stimulus are associated with speeded RTs in that trial (Satoh et al. 2003).
Our behavioral experiment discriminates between these accounts and falsifies the novelty hypothesis: controlled manipulation of context B novelty had no effect on the context dependency of catalepsy expression. This result is thus consistent with the prediction generated by our reward prediction error model, which posits that NoGo learning occurred in striatopallidal neurons linking the sensory context (A) with a NoGo response.
Despite our model’s success in accounting for different aspects of haloperidol-induced catalepsy sensitization, extinction, and renewal within an existing framework, the model has several neurobiological limitations that need to be addressed in future work.
First, we focus on haloperidol effects on the D2 receptor (to which it is most strongly bound) in the striatum (where there are by far the greatest number of D2 receptors (Camps et al. 1989), and which has been implicated in PD). However, it must be acknowledged that additionally, D2, D3, and D4 receptors are also likely to be blocked in the frontal cortex, olfactory bulb, amygdala, and hippocampus. Given limited data, it is not clear if these effects play a crucial role in synaptic plasticity changes induced by haloperidol, nor whether these structures are involved in catalepsy expression. Haloperidol effects on synaptic plasticity in the striatal D2 pathway on the other hand are well studied and suffice to provide an explanation for the observed phenomena and derive novel testable predictions.
Another effect not explicitly modeled is that haloperidol can also elevate striatal DA levels via concomitant blockade of presynaptic D2 autoreceptors (Wu et al. 2002; Garris et al. 2003; Frank and O’Reilly 2006). This increased DA would then stimulate D1 receptors, and could therefore actually enhance Go signals. Indeed, these presynaptic effects have been implicated in the delay to catalepsy onset (Garris et al. 2003). In humans, a single low dose of haloperidol can actually enhance Go learning, presumably via preferential presynaptic mechanisms (Frank and O’Reilly 2006). Nevertheless, with higher doses and chronic administration, the postsynaptic effect dominates (likely due to the greater excitability of NoGo than Go cells; Lei et al. (2004); Kreitzer and Malenka (2007)), leading to overall more NoGo activation (and learning). Thus, inclusion of autoreceptors effects would only delay the inevitable occurrence of catalepsy.
D2 receptors can also act via presynaptic heteroreceptors to regulate cortical glutamatergic input to striatum. Thus, blockade of these receptors would lead to stronger cortical input. Because cortical input is stronger onto NoGo than Go neurons (reviewed above), this effect would likely add to that resulting from postsynaptic D2 blockade. Nevertheless, explicit modeling of this mechanism may shed more light on its potential relevance.
In sum, we provided a neurocomputational account for a constellation of findings in the domain of haloperidol-induced catalepsy sensitization. The model used to generate the findings is the same which has accounted for differential patterns of learning in humans on and off DA medications. The current findings extend the generality of the model to observations in a completely different experimental procedure, setting, and species. The behavioral experiment suggests that the reward prediction error model is more suitable than the novelty model to explain the observed phenomena.
Importantly, whereas D1 receptors require substantial phasic DA bursts to get activated, high-affinity D2 receptors are more sensitive and are inhibited by relatively low levels of tonic DA (e.g., Goto and Grace 2005). In the model, NoGo learning depends on the extent to which DA is removed from the synapse during DA dips, such that longer duration pauses in DA firing would be associated with lower DA levels and stronger learning signals. Notably, larger negative prediction errors are associated with longer DA pause durations of up to 400 ms (Bayer et al. 2007), and the half-life of DA in the striatal synapse is 55–75 ms (Gonon 1997; Venton et al. 2003).
The classical “indirect” pathway (Albin et al. 1989) involved inhibitory GPe projections to the subthalamic nucleus (STN), which then excited GPi. However, more recent evidence embedded in our model suggests that the STN forms part of a third “hyper-direct” pathway linking cortex to GPi (Frank 2006), and that the NoGo pathway involves striatum–GPe–GPi.
This value was chosen arbitrarily; other settings produce the same patterns.
In actuality, this separation of contextual representations is likely to depend on the hippocampus (Nadel and Willner 1980; Myers and Gluck 1994; Rudy and O’Reilly 1999). Because we focus on the striatal mechanism by which haloperidol produces catalepsy, we simplify this hippocampal aspect and simply represent context as a set of different sensory input units. In the ‘novelty’ simulations below, we do not make this assumption.
The key issue here is whether the striatum has access to highly separated contextual input, as would likely be represented in the hippocampus (Nadel and Willner 1980; Myers and Gluck 1994; Rudy and O’Reilly 1999). Our previous simulations assume that they do, given the well-known hippocampal input to ventral striatum. However, the degree to which striatal representations associated with catalepsy expression, likely in the dorsal striatum, are influenced by these contextual inputs is unknown. We therefore simulate the opposite extreme here, in which the contexts are represented identically in the striatal inputs, except for the peripheral influence of novelty-induced DA bursts.
By ‘non-extinguishable’, we mean in the context of experimental procedures that produce robust extinction under placebo. It is of course theoretically possible that a longer extinction phase would not be followed by renewed catalepsy expression when challenged with haloperidol.
All modeling was performed by Thomas V. Wiecki with supervision of Michael J. Frank and Werner J. Schmidt. The behavioral experiment was planned and analyzed by Katrin Riedinger and Andreas von Ameln-Mayerhofer. The paper was written by Thomas V. Wiecki and Michael J. Frank. Katrin Riedinger and Andreas von Ameln-Mayerhofer wrote the section “Behavioral experiment”.
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
- Frank MJ, Moustafa AA, Haughey H, Curran T, Hutchison K (2007a) Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci USA 104:16,311–16,316Google Scholar
- Joel D, Weiner I (1999) Striatal contention scheduling and the split circuit scheme of basal ganglia–thalamocortical circuitry: from anatomy to behaviour. In: Miller R, Wickens JR (eds) Conceptual advances in brain research: brain dynamics and the striatal complex. Harwood Academic, Amsterdam, pp 209–236Google Scholar
- Nadel L, Willner J (1980) Context and conditioning: a place for space. Physiol Psychol 8:218–228Google Scholar
- Pavlov IP (1927) Conditioned reflexes: an investigation of the physiological activity of the cerebral cortex. Oxford University Press, LondonGoogle Scholar
- Santesso D, Evins A, Frank M, Cowman E, Pizzagalli D (2009) Single dose of a dopamine agonist impairs reinforcement learning in humans: Converging evidence from electrophysiology and computational modeling of striatal–cortical function. Hum Brain Mapp (in press)Google Scholar