Abstract
Positive reinforcement helps to control the acquisition of learned behaviours. Here we report a cellular mechanism in the brain that may underlie the behavioural effects of positive reinforcement. We used intracranial self-stimulation (ICSS) as a model of reinforcement learning1, in which each rat learns to press a lever that applies reinforcing electrical stimulation to its own substantia nigra2,3. The outputs from neurons of the substantia nigra terminate on neurons in the striatum in close proximity to inputs from the cerebral cortex on the same striatal neurons4. We measured the effect of substantia nigra stimulation on these inputs from the cortex to striatal neurons and also on how quickly the rats learned to press the lever. We found that stimulation of the substantia nigra (with the optimal parameters for lever-pressing behaviour) induced potentiation of synapses between the cortex and the striatum, which required activation of dopamine receptors. The degree of potentiation within ten minutes of the ICSS trains was correlated with the time taken by the rats to learn ICSS behaviour. We propose that stimulation of the substantia nigra when the lever is pressed induces a similar potentiation of cortical inputs to the striatum, positively reinforcing the learning of the behaviour by the rats.
Similar content being viewed by others
References
Olds, J. & Milner, P. Positive reinforcement produced by electrical stimulation of septal and other regions of the rat brain. J. Comp. Physiol. Psychol. 47, 419–427 (1954).
Beninger, R. J., Bellisle, F. & Milner, P. M. Schedule control of behavior reinforced by electrical stimulation of the brain. Science 196, 547–549 (1977).
Major, R. & White, N. Memory facilitation by self-stimulation reinforcement mediated by the nigro-neostriatal bundle. Physiol. Behav. 20, 723–733 (1978).
Smith, A. D. & Bolam, J. P. The neural network of the basal ganglia as revealed by the study of synaptic connections of identified neurones. Trends Neurosci. 13, 259–265 (1990).
Schultz, W. Multiple reward signals in the brain. Nature Rev. Neurosci. 1, 199–207 (2000).
Mirenowicz, J. & Schultz, W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72, 1024–1027 (1994).
Mirenowicz, J. & Schultz, W. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379, 449–451 (1996).
Matsumoto, N., Hanakawa, T., Maki, S., Graybiel, A. M. & Kimura, M. Role of nigrostriatal dopamine system in learning to perform sequential motor tasks in a predictive manner. J. Neurophysiol. 82, 978–998 (1999).
Aosaki, T., Graybiel, A. M. & Kimura, M. Effect of the nigrostriatal dopamine system on acquired neural responses in the striatum of behaving monkeys. Science 265, 412–415 (1994).
Hodos, W. & Valenstein, E. S. An evaluation of response rate as a measure of rewarding intracranial stimulation. J. Comp. Physiol. Psychol. 55, 80–84 (1962).
Reynolds, R. W. The relationship between stimulation voltage and rate of hypothalamic self-stimulation in the rat. J. Comp. Physiol. Psychol. 51, 193–198 (1958).
Wilson, C. J. & Kawaguchi, Y. The origins of two-state spontaneous membrane potential fluctuations of neostriatal spiny neurons. J. Neurosci. 16, 2397–2410 (1996).
Stern, E. A., Jaeger, D. & Wilson, C. J. Membrane potential synchrony of simultaneously recorded striatal spiny neurons in vivo. Nature 394, 475–478 (1998).
Wilson, C. J. Postsynaptic potentials evoked in spiny neostriatal projection neurons by stimulation of ipsilateral and contralateral neocortex. Brain Res. 367, 201–213 (1986).
Reynolds, J. N. J. & Wickens, J. R. Substantia nigra dopamine regulates synaptic plasticity and membrane potential fluctuations in the rat neostriatum, in vivo. Neuroscience 99, 199–203 (2000).
Wilson, C. J. & Groves, P. M. Spontaneous firing patterns of identified spiny neurons in the rat neostriatum. Brain Res. 220, 67–80 (1981).
Alexander, G. E. & Crutcher, M. D. Preparation for movement: neural representations of intended direction in three motor areas of the monkey. J. Neurophysiol. 64, 133–150 (1990).
Kimura, M. Behaviorally contingent property of movement-related activity of the primate putamen. J. Neurophysiol. 63, 1277–1296 (1990).
Corbett, D. & Wise, R. A. Intracranial self-stimulation in relation to the ascending dopaminergic systems of the midbrain: a moveable electrode mapping study. Brain Res. 185, 1–15 (1980).
Fibiger, H. C., LePiane, F. G., Jakubovic, A. & Phillips, A. G. The role of dopamine in intracranial self-stimulation of the ventral tegmental area. J. Neurosci. 7, 3888–3896 (1987).
Wise, R. A. Addictive drugs and brain stimulation reward. Annu. Rev. Neurosci. 19, 319–340 (1996).
Bielajew, C. & Shizgal, P. Evidence implicating descending fibers in self-stimulation of the medial forebrain bundle. J. Neurosci. 6, 919–929 (1986).
Swanson, L. W. The projections of the ventral tegmental area and adjacent regions: a combined fluorescent retrograde tracer and immunofluorescence study in the rat. Brain Res. Bull. 9, 321–353 (1982).
Suri, R. E. & Schultz, W. A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 91, 871–890 (1999).
Montague, P. R., Dayan, P. & Sejnowski, T. J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936–1947 (1996).
Alexander, G. E., DeLong, M. R. & Strick, P. L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357–381 (1986).
Charpier, S. & Deniau, J. M. In vivo activity-dependent plasticity at cortico-striatal connections: evidence for physiological long-term potentiation. Proc. Natl Acad. Sci. USA 94, 7036–7040 (1997).
Miller, R. M. Meaning and Purpose in the Intact Brain 62–120 (Clarendon Press, Oxford, 1981).
Hunt, G. E., Atrens, D. M. & Jackson, D. M. Reward summation and the effects of dopamine D1 and D2 agonists and antagonists on fixed-interval responding for brain stimulation. Pharmacol. Biochem. Behav. 48, 853–862 (1994).
Paxinos, G. & Watson, C. The Rat Brain in Stereotaxic Coordinates (Academic, New York, 1998).
Acknowledgements
We thank A. Kean and C. Booker for technical assistance. This work was supported by the Health Research Council of New Zealand, Lottery Health Research and the New Zealand Neurological Foundation.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Reynolds, J., Hyland, B. & Wickens, J. A cellular mechanism of reward-related learning. Nature 413, 67–70 (2001). https://doi.org/10.1038/35092560
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1038/35092560
- Springer Nature Limited
This article is cited by
-
Dopamine transients follow a striatal gradient of reward time horizons
Nature Neuroscience (2024)
-
Dopamine and glutamate regulate striatal acetylcholine in decision-making
Nature (2023)
-
Dynamic behaviour restructuring mediates dopamine-dependent credit assignment
Nature (2023)
-
Neuronal and astrocytic CB1R signaling differentially modulates goal-directed behavior and working memory by distinct temporal mechanisms
Neuropsychopharmacology (2023)
-
Spontaneous behaviour is shaped by dopamine in two ways
Nature (2023)