Encyclopedia of Computational Neuroscience

Living Edition
| Editors: Dieter Jaeger, Ranu Jung

Computation with Dopaminergic Modulation

Living reference work entry
DOI: https://doi.org/10.1007/978-1-4614-7320-6_631-3

Definition

Dopamine (DA) is a neuromodulator released by midbrain neurons with widespread projections throughout the brain. Dopaminergic modulation has diverse effects on cellular, motor, and cognitive functions, including reinforcement learning, working memory, and attention. Dysregulation of dopamine also plays a central role in the breakdown of these functions in disorders such as Parkinson’s disease and schizophrenia.

Detailed Description

Dopamine Basics

DA is a neuromodulator released by neurons in two nuclei of the midbrain: the ventral tegmental area (VTA) and the substantia nigra pars compacta (SNc). The axons of these neurons project to a large number of cortical and subcortical areas. Prefrontal cortex, motor cortex, and the striatum are among the most densely innervated areas. DA release tends to be fairly homogeneous across DA neurons, by virtue of electrical coupling between the axons of adjacent neurons, which induces highly synchronous firing. As pointed out by Paul...

Keywords

Prediction Error Reinforcement Learning Pathological Gambling Active Maintenance Gain Modulation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

References

  1. Arnsten AFT (1998) Catecholamine modulation of prefrontal cortical cognitive function. Trends Cogn Sci 2:436–447CrossRefPubMedGoogle Scholar
  2. Berridge KC, Robinson TE (1998) What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Rev 28:309–369CrossRefPubMedGoogle Scholar
  3. Braver TS, Cohen JD (1999) Dopamine, cognitive control, and schizophrenia: the gating model. Prog Brain Res 121:327–349CrossRefPubMedGoogle Scholar
  4. Clarke CR, Geffen GM, Geffen LB (1987) Catecholamines and attention I: animal and clinical studies. Neurosci Biobehav Rev 11:341–352CrossRefGoogle Scholar
  5. Cohen JD, Braver TS, Brown JW (2002) Computational perspectives on dopamine function in prefrontal cortex. Curr Opin Neurobiol 12:223–229CrossRefPubMedGoogle Scholar
  6. Cohen JD, McClure SM, Yu AJ (2007) Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos Trans R Soc Lond B Biol Sci 362:933–942CrossRefPubMedCentralPubMedGoogle Scholar
  7. Cools R, Altamirano L, D’Esposito M (2006) Reversal learning in Parkinson’s disease depends on medication status and outcome valence. Neuropsychologia 44:1663–1673CrossRefPubMedGoogle Scholar
  8. Dagher A, Robbins T (2009) Personality addiction dopamine: Insights from Parkinson’s disease. Neuron 61:502–510CrossRefPubMedGoogle Scholar
  9. Daw ND, Courville AC, Touretzky DS (2006) Representation and timing in theories of the dopamine system. Neural Comput 18:1637–1677CrossRefPubMedGoogle Scholar
  10. Denk F, Walton ME, Jennings KA, Sharp T, Rushworth MF, Bannerman DM (2005) Differential involvement of serotonin and dopamine systems in cost–benefit decisions about delay oreffort. Psychopharmacology (Berl) 179:587–596CrossRefGoogle Scholar
  11. Durstewitz D, Seamans JK (2008) The dual-state theory of prefrontal cortex dopamine function with relevance to COMT genotypes and schizophrenia. Biol Psychiatry 64:739–749CrossRefPubMedGoogle Scholar
  12. Durstewitz D, Kelc M, Güntürkün O (1999) A neurocomputational theory of the dopaminergic modulation of working memory functions. J Neurosci 19:2807–2822PubMedGoogle Scholar
  13. Foote L, Morrison JH (1987) Extrathalamic modulation of cortical function. Annu Rev Neurosci 10:67–95CrossRefPubMedGoogle Scholar
  14. Frank MJ (2005) Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J Cogn Neurosci 17:51–72CrossRefPubMedGoogle Scholar
  15. Frank MJ (2008) Schizophrenia: a computational reinforcement learning perspective. Schizophr Bull 34:1008–1011CrossRefPubMedCentralPubMedGoogle Scholar
  16. Frank MJ, Seeberger L, O'Reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in Parkinsonism. Science 306:1940–1943CrossRefPubMedGoogle Scholar
  17. Glimcher PW (2011) Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc Natl Acad Sci 108:15647–15654CrossRefPubMedCentralPubMedGoogle Scholar
  18. Grace AA (1991) Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis of the etiology of schizophrenia. Neuroscience 41:1–24CrossRefPubMedGoogle Scholar
  19. Hollerman JR, Schultz W (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1:304–309CrossRefPubMedGoogle Scholar
  20. Horvitz JC, Stewart T, Jacobs B (1997) Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Res 759:251–258CrossRefPubMedGoogle Scholar
  21. Houk JC, Adams JL, Barto AG (1995) A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Houk JC, Davis JL, Beiser DG (eds) Models of information processing in the basal ganglia. MIT Press, Cambridge, MA, pp 249–270Google Scholar
  22. Kakade S, Dayan P (2002) Dopamine: generalization and bonuses. Neural Netw 15:549–559CrossRefPubMedGoogle Scholar
  23. Lapish CC, Kroener S, Durstewitz D, Lavin A, Seamans JK (2007) The ability of the mesocortical dopamine system to operate in distinct temporal modes. Psychopharmacology (Berl) 191:609–625CrossRefGoogle Scholar
  24. Levitt JB, Lewis DA, Yoshioka T, Lund JS (1993) Topography of pyramidal neuron intrinsic connections in macaque monkey prefrontal cortex (areas 9 and 46). J Comp Neurol 338:360–376CrossRefPubMedGoogle Scholar
  25. Mazzoni P, Hristova A, Krakauer JW (2007) Why don’t we move faster? Parkinson’s disease, movement vigor, and implicit motivation. J Neurosci 27:7105–7116CrossRefPubMedGoogle Scholar
  26. Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16(5):1936–1947PubMedGoogle Scholar
  27. Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H (2006) Midbrain dopamine neurons encode decisions for future action. Nature Neuroscience 9:1057–1063CrossRefPubMedGoogle Scholar
  28. Moustafa AA, Cohen MX, Sherman SJ, Frank MJ (2008a) A role for dopamine in temporal decision making and reward maximization in Parkinsonism. J Neurosci 28:12294–12304CrossRefPubMedCentralPubMedGoogle Scholar
  29. Moustafa AA, Sherman SJ, Frank MJ (2008b) A dopaminergic basis for working memory, learning, and attentional shifting in Parkinson’s disease. Neuropsychologia 46:3144–3156CrossRefPubMedGoogle Scholar
  30. Moyer JT, Wolf JA, Finkel LH (2007) Effects of dopaminergic modulation on the integrative properties of the ventral striatal medium spiny neuron. J Neurophysiol 98:3731–3748CrossRefPubMedGoogle Scholar
  31. Niv Y, Rivlin-Etzion M (2007) Parkinson’s disease: fighting the will? J Neurosci 27:11777–11779CrossRefPubMedGoogle Scholar
  32. Niv Y, Daw ND, Joel D, Dayan P (2007) Tonic dopamine: opportunity costs and the controlof response vigor. Psychopharmacology (Berl) 191:507–520CrossRefGoogle Scholar
  33. O'Reilly RC, Frank MJ (2006) Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput 18:283–328CrossRefPubMedGoogle Scholar
  34. Roesch MR, Calu DJ, Schoenbaum G (2007) Dopamine neurons encode the better options in rats deciding between differently delayed or sized rewards. Nature Neuroscience 10:1615:1624Google Scholar
  35. Salamone JD, Wisniecki A, Carlson BB, Correa M (2001) Nucleus accumbens dopamine depletions make animals highly sensitive to high fixed ratio requirements but do not impair primary food reinforcement. Neuroscience 5:863–870CrossRefGoogle Scholar
  36. Sawaguchi T, Goldman-Rakic PS (1991) D1 dopamine receptors in prefrontal cortex: involvement in working memory. Science 251:947–950CrossRefPubMedGoogle Scholar
  37. Schultz W (2007) Multiple dopamine functions at different time courses. Annu Rev Neurosci 30:259–288CrossRefPubMedGoogle Scholar
  38. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599CrossRefPubMedGoogle Scholar
  39. Servan-Schreiber D, Printz H, Cohen JD (1990) A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior. Science 249:892–895CrossRefPubMedGoogle Scholar
  40. Sokolowski JD, Salamone JD (1998) The role of accumbens dopamine in lever pressing and response allocation: effects of 6-OHDA injected into core and dorsomedial shell. Pharmacol Biochem Behav 59:557–566CrossRefPubMedGoogle Scholar
  41. Suri RE, Schultz W (1998) Learning of sequential movements by neural network model with dopamine-like reinforcement signal. Exp Brain Res 121:350–354CrossRefPubMedGoogle Scholar
  42. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge, MAGoogle Scholar
  43. Trantham-Davidson H, Neely LC, Lavin A, Seamans JK (2004) Mechanisms underlying differential D1 versus D2 dopamine receptor regulation of inhibition in prefrontal cortex. J Neurosci 24:10652–10659CrossRefPubMedGoogle Scholar
  44. Waltz JA, Frank MJ, Robinson BM, Gold JM (2007) Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-corticaldysfunction. Biol Psychiatry 62:756–764CrossRefPubMedCentralPubMedGoogle Scholar
  45. Wiecki TV, Frank MJ (2010) Neurocomputational models of motor and cognitive deficits in Parkinson’s disease. Prog Brain Res 183:275–297CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Department of Psychology and Neuroscience InstitutePrinceton UniversityPrincetonUSA