Modeling Serotonin’s Contributions to Basal Ganglia Dynamics

  • Pragathi Priyadharsini Balasubramani
  • V. Srinivasa Chakravarthy
  • Balaraman Ravindran
  • Ahmed A. Moustafa
Chapter
Part of the Cognitive Science and Technology book series (CSAT)

Abstract

In addition to dopaminergic input, serotonergic (5-HT) fibers also widely arborize through the basal ganglia circuits and strongly control their dynamics. Although empirical studies show that 5-HT plays many functional roles in risk-based decision making, reward, and punishment learning, prior computational models mostly focus on its role in behavioral inhibition or timescale of prediction. This chapter presents an extended reinforcement learning (RL)-based model of DA and 5-HT function in the BG, which reconciles some of the diverse roles of 5-HT. The model uses the concept of utility function—a weighted sum of the traditional value function expressing the expected sum of the rewards, and a risk function expressing the variance observed in reward outcomes. Serotonin is represented by a weight parameter, used in this combination of value and risk functions, while the neuromodulator dopamine (DA) is represented as reward prediction error as in the classical models. Consistent with this abstract model, a network model is also presented in which medium spiny neurons (MSN) co-expressing both D1 and D2 receptors (D1R–D2R) is suggested to compute risk, while those expressing only D1 receptors are suggested to compute value. This BG model includes nuclei such as striatum, Globus Pallidus externa, Globus Pallidus interna, and subthalamic nuclei. DA and 5-HT are modeled to affect both the direct pathway (DP) and the indirect pathway (IP) composing of D1R, D2R, D1R–D2R projections differentially. Both abstract and network models are applied to data from different experimental paradigms used to study the role of 5-HT: (1) risk-sensitive decision making, where 5-HT controls the risk sensitivity; (2) temporal reward prediction, where 5-HT controls timescale of reward prediction, and (3) reward–punishment sensitivity, where punishment prediction error depends on 5-HT levels. Both the extended RL model (Balasubramani, Chakravarthy, Ravindran, & Moustafa, in Front Comput Neurosci 8:47, 2014; Balasubramani, Ravindran, & Chakravarthy, in Understanding the role of serotonin in basal ganglia through a unified model, 2012) along with their network correlates (Balasubramani, Chakravarthy, Ravindran, & Moustafa, in Front Comput Neurosci 9:76, 2015; Balasubramani, Chakravarthy, Ali, Ravindran, & Moustafa, in PLoS ONE 10(6):e0127542, 2015) successfully explain the three diverse roles of 5-HT in a single framework.

Keywords

Serotonin Dopamine Reinforcement learning Risk Reward Punishment Utility Basal ganglia Parkinson’s disease DA 5-HT D1 receptors Medium spiny neurons D2 receptors D1R–D2R co-expression 

References

  1. Abbott, P. D. a. L. F. (2001). Theoretical neuroscience: Computational and mathematical modeling of neural systems. Cambridge, Massachusetts, London, England: The MIT Press.Google Scholar
  2. Aghajanian, G. K., & Marek, G. J. (2000). Serotonin model of schizophrenia: Emerging role of glutamate mechanisms. Brain Research Reviews, 31(2), 302–312.CrossRefGoogle Scholar
  3. Albin, R. L. (1998). Fuch’s corneal dystrophy in a patient with mitochondrial DNA mutations. Journal of Medical Genetics, 35(3), 258–259.CrossRefGoogle Scholar
  4. Albin, R. L., Young, A. B., & Penney, J. B. (1989). The functional anatomy of basal ganglia disorders. Trends in Neurosciences, 12(10), 366–375.CrossRefGoogle Scholar
  5. Alex, K. D., & Pehek, E. A. (2007). Pharmacologic mechanisms of serotonergic regulation of dopamine neurotransmission. Pharmacology & Therapeutics, 113(2), 296–320.  https://doi.org/10.1016/j.pharmthera.2006.08.004.CrossRefGoogle Scholar
  6. Allen, A. T., Maher, K. N., Wani, K. A., Betts, K. E., & Chase, D. L. (2011). Coexpressed D1-and D2-like dopamine receptors antagonistically modulate acetylcholine release in Caenorhabditis elegans. Genetics, 188(3), 579–590.CrossRefGoogle Scholar
  7. Amemori, K., Gibb, L. G., & Graybiel, A. M. (2011). Shifting responsibly: The importance of striatal modularity to reinforcement learning in uncertain environments. Frontiers in Human Neuroscience, 5, 47.  https://doi.org/10.3389/fnhum.2011.00047.CrossRefGoogle Scholar
  8. Angiolillo, P. J., & Vanderkooi, J. M. (1996). Hydrogen atoms are produced when tryptophan within a protein is irradiated with ultraviolet light. Photochemistry and Photobiology, 64(3), 492–495.CrossRefGoogle Scholar
  9. Araki, K. Y., Sims, J. R., & Bhide, P. G. (2007). Dopamine receptor mRNA and protein expression in the mouse corpus striatum and cerebral cortex during pre-and postnatal development. Brain Research, 1156, 31–45.CrossRefGoogle Scholar
  10. Ashby, F. G., Turner, B. O., & Horvitz, J. C. (2010). Cortical and basal ganglia contributions to habit learning and automaticity. Trends in Cognitive Sciences, 14(5), 208–215.CrossRefGoogle Scholar
  11. Aston-Jones, G., & Cohen, J. D. (2005). An integrative theory of locus coeruleus-norepinephrine function: Adaptive gain and optimal performance. Annual Review of Neuroscience, 28, 403–450.CrossRefGoogle Scholar
  12. Azmitia, E. C. (1999). Serotonin neurons, neuroplasticity, and homeostasis of neural tissue. Neuropsychopharmacology, 21(2 Suppl), 33S–45S.  https://doi.org/10.1016/S0893-133X(99)00022-6.CrossRefGoogle Scholar
  13. Azmitia, E. C. (2001). Modern views on an ancient chemical: Serotonin effects on cell proliferation, maturation, and apoptosis. Brain Research Bulletin, 56(5), 413–424.CrossRefGoogle Scholar
  14. Balasubramani, P. P., Chakravarthy, S., Ravindran, B., & Moustafa, A. A. (2014). An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning. Frontiers in Computational Neuroscience, 8, 47.CrossRefGoogle Scholar
  15. Balasubramani, P. P., Chakravarthy, S., Ravindran, B., & Moustafa, A. A. (2015a). A network model of basal ganglia for understanding the roles of dopamine and serotonin in reward-punishment-risk based decision making. Name. Frontiers in Computational Neuroscience, 9, 76.Google Scholar
  16. Balasubramani, P. P., Chakravarthy, V. S., Ali, M., Ravindran, B., & Moustafa, A. A. (2015b). Identifying the Basal Ganglia network model markers for medication-induced impulsivity in Parkinson’s Disease patients. PLoS ONE, 10(6), e0127542.CrossRefGoogle Scholar
  17. Balasubramani, P. P., Ravindran, B., & Chakravarthy, S. (2012). Understanding the role of serotonin in basal ganglia through a unified model. Paper presented at the International Conference on Artificial Neural Networks, Lausanne, Switzerland.Google Scholar
  18. Bar-Gad, I., & Bergman, H. (2001). Stepping out of the box: Information processing in the neural networks of the basal ganglia. Current Opinion in Neurobiology, 11(6), 689–695.CrossRefGoogle Scholar
  19. Bell, C. (2001). Tryptophan depletion and its implications for psychiatry. The British Journal of Psychiatry, 178(5), 399–405.  https://doi.org/10.1192/bjp.178.5.399.CrossRefGoogle Scholar
  20. Bell, D. E. (1995). Risk, return and utility. Management Science, 41, 23–30.MATHCrossRefGoogle Scholar
  21. Belujon, P., Bezard, E., Taupignon, A., Bioulac, B., & Benazzouz, A. (2007). Noradrenergic modulation of subthalamic nucleus activity: Behavioral and electrophysiological evidence in intact and 6-hydroxydopamine-lesioned rats. The Journal of Neuroscience, 27(36), 9595–9606.CrossRefGoogle Scholar
  22. Bertler, A., & Rosengren, E. (1966). Possible role of brain dopamine. Pharmacological Reviews, 18(1), 769–773.Google Scholar
  23. Bertran-Gonzalez, J., Bosch, C., Maroteaux, M., Matamales, M., Herve, D., Valjent, E., et al. (2008). Opposing patterns of signaling activation in dopamine D1 and D2 receptor-expressing striatal neurons in response to cocaine and haloperidol. Journal of Neuroscience, 28(22), 5671–5685.  https://doi.org/10.1523/JNEUROSCI.1039-08.2008.CrossRefGoogle Scholar
  24. Bertran-Gonzalez, J., Hervé, D., Girault, J.-A., & Valjent, E. (2010). What is the degree of segregation between striatonigral and striatopallidal projections? Front Neuroanat, 4.Google Scholar
  25. Boureau, Y. L., & Dayan, P. (2011). Opponency revisited: Competition and cooperation between dopamine and serotonin. Neuropsychopharmacology, 36(1), 74–97.  https://doi.org/10.1038/npp.2010.151.CrossRefGoogle Scholar
  26. Buhot, M.-C. (1997). Serotonin receptors in cognitive behaviors. Current Opinion in Neurobiology, 7(2), 243–254.CrossRefGoogle Scholar
  27. Calabresi, P., Maj, R., Pisani, A., Mercuri, N. B., & Bernardi, G. (1992). Long-term synaptic depression in the striatum: Physiological and pharmacological characterization. Journal of Neuroscience, 12(11), 4224–4233.Google Scholar
  28. Calabresi, P., Picconi, B., Tozzi, A., Ghiglieri, V., & Di Filippo, M. (2014). Direct and indirect pathways of basal ganglia: A critical reappraisal. Nature Neuroscience, 17(8), 1022–1030.CrossRefGoogle Scholar
  29. Chakravarthy, V. S., & Balasubramani, P. P. (2013). Basal Ganglia System as an engine for exploration. In J. R. Jaeger D. (Ed.), Encyclopedia of Computational Neuroscience. Berlin Heidelberg: SpringerReference (http://www.springerreference.com/). Springer-Verlag.
  30. Chakravarthy, V. S., & Balasubramani, P. P. (2014). Basal Ganglia System as an engine for exploration. Berlin Heidelberg: SpringerReference (http://www.springerreference.com/). Springer-Verlag.
  31. Chakravarthy, V. S., Joseph, D., & Bapi, R. S. (2010). What do the basal ganglia do? A modeling perspective. Biological Cybernetics, 103(3), 237–253.  https://doi.org/10.1007/s00422-010-0401-y.MathSciNetMATHCrossRefGoogle Scholar
  32. Chao, M. Y., Komatsu, H., Fukuto, H. S., Dionne, H. M., & Hart, A. C. (2004). Feeding status and serotonin rapidly and reversibly modulate a Caenorhabditis elegans chemosensory circuit. Proceedings of the National Academy of Science U S A, 101(43), 15512–15517.  https://doi.org/10.1073/pnas.0403369101.CrossRefGoogle Scholar
  33. Cohen, J. D., McClure, S. M., & Angela, J. Y. (2007). Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481), 933–942.CrossRefGoogle Scholar
  34. Cools, R., Nakamura, K., & Daw, N. D. (2011). Serotonin and dopamine: Unifying affective, activational, and decision functions. Neuropsychopharmacology, 36(1), 98–113.  https://doi.org/10.1038/npp.2010.121.CrossRefGoogle Scholar
  35. Cools, R., Robinson, O. J., & Sahakian, B. (2008). Acute tryptophan depletion in healthy volunteers enhances punishment prediction but does not affect reward prediction. Neuropsychopharmacology, 33(9), 2291–2299.  https://doi.org/10.1038/sj.npp.1301598.CrossRefGoogle Scholar
  36. d’Acremont, M., Lu, Z. L., Li, X., Van der Linden, M., & Bechara, A. (2009). Neural correlates of risk prediction error during reinforcement learning in humans. Neuroimage, 47(4), 1929–1939.  https://doi.org/10.1016/j.neuroimage.2009.04.096.CrossRefGoogle Scholar
  37. Dalley, J. W., Everitt, B. J., & Robbins, T. W. (2011). Impulsivity, compulsivity, and top-down cognitive control. Neuron, 69(4), 680–694.CrossRefGoogle Scholar
  38. Daw, N. D., Kakade, S., & Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Network, 15(4–6), 603–616.CrossRefGoogle Scholar
  39. Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441(7095), 876–879.  https://doi.org/10.1038/nature04766.CrossRefGoogle Scholar
  40. Dayan, P., & Huys, Q. (2015). Serotonin’s many meanings elude simple theories. Elife, 4.Google Scholar
  41. Dayan, P., & Huys, Q. J. (2008). Serotonin, inhibition, and negative mood. PLoS Computational Biology, 4(2), e4.CrossRefGoogle Scholar
  42. Dayan, P., & Yu, A. J. (2006). Phasic norepinephrine: A neural interrupt signal for unexpected events. Network: Computation in Neural Systems, 17(4), 335–350.CrossRefGoogle Scholar
  43. Delaville, C., Zapata, J., Cardoit, L., & Benazzouz, A. (2012). Activation of subthalamic alpha 2 noradrenergic receptors induces motor deficits as a consequence of neuronal burst firing. Neurobiology of Diseases, 47(3), 322–330.CrossRefGoogle Scholar
  44. DeLong, M. R. (1990). Primate models of movement disorders of basal ganglia origin. Trends in Neurosciences, 13(7), 281–285.CrossRefGoogle Scholar
  45. Di Giovanni, G., Di Matteo, V., Pierucci, M., & Esposito, E. (2008). Serotonin–dopamine interaction: Electrophysiological evidence. Progress in Brain Research, 172, 45–71.CrossRefGoogle Scholar
  46. Di Mascio, M., Di Giovanni, G., Di Matteo, V., Prisco, S., & Esposito, E. (1998). Selective serotonin reuptake inhibitors reduce the spontaneous activity of dopaminergic neurons in the ventral tegmental area. Brain Research Bulletin, 46(6), 547–554.CrossRefGoogle Scholar
  47. Di Matteo, V., Di Giovanni, G., Pierucci, M., & Esposito, E. (2008a). Serotonin control of central dopaminergic function: Focus on in vivo microdialysis studies. Progress in Brain Research, 172, 7–44.CrossRefGoogle Scholar
  48. Di Matteo, V., Pierucci, M., Esposito, E., Crescimanno, G., Benigno, A., & Di Giovanni, G. (2008b). Serotonin modulation of the basal ganglia circuitry: Therapeutic implication for Parkinson’s disease and other motor disorders. Progress in Brain Research, 172, 423–463.CrossRefGoogle Scholar
  49. Ding, Y., Won, L., Britt, J. P., Lim, S. A. O., McGehee, D. S., & Kang, U. J. (2011). Enhanced striatal cholinergic neuronal activity mediates l-DOPA–induced dyskinesia in parkinsonian mice. Proceedings of the National Academy of Sciences, 108(2), 840–845.CrossRefGoogle Scholar
  50. Divac, I., Fonnum, F., & Storm-Mathisen, J. (1977). High affinity uptake of glutamate in terminals of corticostriatal axons. Nature, 266(5600), 377–378.CrossRefGoogle Scholar
  51. Doya, K. (2002). Metalearning and neuromodulation. Neural Network, 15(4–6), 495–506.CrossRefGoogle Scholar
  52. Eberle-Wang, K., Mikeladze, Z., Uryu, K., & Chesselet, M. F. (1997). Pattern of expression of the serotonin2C receptor messenger RNA in the basal ganglia of adult rats. Journal of Comparative Neurology, 384(2), 233–247.CrossRefGoogle Scholar
  53. Economidou, D., Theobald, D. E., Robbins, T. W., Everitt, B. J., & Dalley, J. W. (2012). Norepinephrine and dopamine modulate impulsivity on the five-choice serial reaction time task through opponent actions in the shell and core sub-regions of the nucleus accumbens. Neuropsychopharmacology, 37(9), 2057–2066.CrossRefGoogle Scholar
  54. Ferre, S., Cortes, R., & Artigas, F. (1994). Dopaminergic regulation of the serotonergic raphe-striatal pathway: Microdialysis studies in freely moving rats. Journal of Neuroscience, 14(8), 4839–4846.Google Scholar
  55. Fox, S. H., Chuang, R., & Brotchie, J. M. (2009). Serotonin and Parkinson’s disease: On movement, mood, and madness. Movement Disorders, 24(9), 1255–1266.  https://doi.org/10.1002/mds.22473.CrossRefGoogle Scholar
  56. Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: A neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. Journal of Cognitive Neuroscience, 17(1), 51–72.  https://doi.org/10.1162/0898929052880093.CrossRefGoogle Scholar
  57. Frank, M. J., Doll, B. B., Oas-Terpstra, J., & Moreno, F. (2009). Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nature Neuroscience, 12(8), 1062–1068.CrossRefGoogle Scholar
  58. Frank, M. J., Samanta, J., Moustafa, A. A., & Sherman, S. J. (2007a). Hold your horses: Impulsivity, deep brain stimulation, and medication in parkinsonism. Science, 318(5854), 1309–1312.CrossRefGoogle Scholar
  59. Frank, M. J., Seeberger, L. C., & O’Reilly R, C. (2004). By carrot or by stick: Cognitive reinforcement learning in Parkinsonism. Science, 306(5703), 1940–1943.  https://doi.org/10.1126/science.1102941.
  60. Gerfen, C. R., Engber, T. M., Mahan, L. C., Susel, Z., Chase, T. N., Monsma, F., et al. (1990). D1 and D2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science, 250(4986), 1429–1432.CrossRefGoogle Scholar
  61. Gerfen, C. R., & Wilson, C. J. (1996). Chapter II The basal ganglia. Handbook of Chemical Neuroanatomy, 12, 371–468.Google Scholar
  62. Gervais, J., & Rouillard, C. (2000). Dorsal raphe stimulation differentially modulates dopaminergic neurons in the ventral tegmental area and substantia nigra. Synapse, 35(4), 281–291.  https://doi.org/10.1002/(sici)1098-2396(20000315)35: 4 < 281::aid-syn6 > 3.0.co;2-a.
  63. Gillette, R. (2006). Evolution and function in serotonergic systems. Integrative and Comparative Biology, 46(6), 838–846.  https://doi.org/10.1093/icb/icl024.CrossRefGoogle Scholar
  64. Halford, J. C., Harrold, J. A., Lawton, C. L., & Blundell, J. E. (2005). Serotonin (5-HT) drugs: Effects on appetite expression and use for the treatment of obesity. Current Drug Targets, 6(2), 201–213.CrossRefGoogle Scholar
  65. Hasbi, A., Fan, T., Alijaniaram, M., Nguyen, T., Perreault, M. L., O’Dowd, B. F., et al. (2009). Calcium signaling cascade links dopamine D1-D2 receptor heteromer to striatal BDNF production and neuronal growth. Proceedings of the National Academy of Science U S A, 106(50), 21377–21382.  https://doi.org/10.1073/pnas.0903676106.CrossRefGoogle Scholar
  66. Hasbi, A., O’Dowd, B. F., & George, S. R. (2010). Heteromerization of dopamine D2 receptors with dopamine D1 or D5 receptors generates intracellular calcium signaling by different mechanisms. Current Opinion in Pharmacology, 10(1), 93–99.  https://doi.org/10.1016/j.coph.2009.09.011.CrossRefGoogle Scholar
  67. Hasbi, A., O’Dowd, B. F., & George, S. R. (2011). Dopamine D1-D2 receptor heteromer signaling pathway in the brain: Emerging physiological relevance. Molecular Brain, 4, 26.  https://doi.org/10.1186/1756-6606-4-26.CrossRefGoogle Scholar
  68. He, Q., Xue, G., Chen, C., Lu, Z., Dong, Q., Lei, X., … Chen, C. (2010). Serotonin transporter gene-linked polymorphic region (5-HTTLPR) influences decision making under ambiguity and risk in a large Chinese sample. Neuropharmacology, 59(6), 518–526.Google Scholar
  69. Heiman, M., Schaefer, A., Gong, S., Peterson, J. D., Day, M., Ramsey, K. E., … Surmeier, D. J. (2008). A translational profiling approach for the molecular characterization of CNS cell types. Cell, 135(4), 738–748.Google Scholar
  70. Hernandez-Echeagaray, E., Starling, A. J., Cepeda, C., & Levine, M. S. (2004). Modulation of AMPA currents by D2 dopamine receptors in striatal medium-sized spiny neurons: Are dendrites necessary? European Journal of Neuroscience, 19(9), 2455–2463.  https://doi.org/10.1111/j.0953-816X.2004.03344.x.CrossRefGoogle Scholar
  71. Houk, J. C., Bastianen, C., Fansler, D., Fishbach, A., Fraser, D., Reber, P. J., … Simo, L. S. (2007). Action selection and refinement in subcortical loops through basal ganglia and cerebellum. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 362(1485), 1573–1583.  https://doi.org/10.1098/rstb.2007.2063.
  72. Humphries, M. D., & Prescott, T. J. (2010). The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. Progress in Neurobiology, 90(4), 385–417.  https://doi.org/10.1016/j.pneurobio.2009.11.003.CrossRefGoogle Scholar
  73. Jiang, L. H., Ashby, C. R., Jr., Kasser, R. J., & Wang, R. Y. (1990). The effect of intraventricular administration of the 5-HT <sub> 3 </sub> receptor agonist 2-methylserotonin on the release of dopamine in the nucleus accumbens: An in vivo chronocoulometric study. Brain Research, 513(1), 156–160.CrossRefGoogle Scholar
  74. Jung, A. B., & Bennett, J. P. (1996). Development of striatal dopaminergic function. I. Pre-and postnatal development of mRNAs and binding sites for striatal D1 (D1a) and D2 (D2a) receptors. Developmental Brain Research, 94(2), 109–120.CrossRefGoogle Scholar
  75. Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263–292.MATHCrossRefGoogle Scholar
  76. Kalva, S. K., Rengaswamy, M., Chakravarthy, V. S., & Gupte, N. (2012). On the neural substrates for exploratory dynamics in basal ganglia: A model. Neural Netw, 32, 65–73.  https://doi.org/10.1016/j.neunet.2012.02.031.CrossRefGoogle Scholar
  77. Kawaguchi, Y., Wilson, C. J., & Emson, P. C. (1990). Projection subtypes of rat neostriatal matrix cells revealed by intracellular injection of biocytin. The Journal of Neuroscience, 10(10), 3421–3438.Google Scholar
  78. Kötter, R., & Wickens, J. (1998). Striatal mechanisms in Parkinson’s disease: New insights from computer modeling. Artificial Intelligence in Medicine, 13(1), 37–55.CrossRefGoogle Scholar
  79. Kravitz, E. A. (2000). Serotonin and aggression: Insights gained from a lobster model system and speculations on the role of amine neurons in a complex behavior. Journal of Comparative Physiology. A, Sensory, Neural, and Behavioral Physiology, 186(3), 221–238.CrossRefGoogle Scholar
  80. Krishnan, R., Ratnadurai, S., Subramanian, D., Chakravarthy, V. S., & Rengaswamy, M. (2011). Modeling the role of basal ganglia in saccade generation: Is the indirect pathway the explorer? Neural Network, 24(8), 801–813.  https://doi.org/10.1016/j.neunet.2011.06.002.CrossRefGoogle Scholar
  81. Kuhnen, C. M., Samanez-Larkin, G. R., & Knutson, B. (2013). Serotonergic genotypes, neuroticism, and financial choices. PLoS ONE, 8(1), e54632.CrossRefGoogle Scholar
  82. Lak, A., Stauffer, W. R., & Schultz, W. (2014). Dopamine prediction error responses integrate subjective value from different reward dimensions. Proceedings of the National Academy of Sciences, 111(6), 2343–2348.CrossRefGoogle Scholar
  83. Le Moine, C., & Bloch, B. (1995). D1 and D2 dopamine receptor gene expression in the rat striatum: Sensitive cRNA probes demonstrate prominent segregation of D1 and D2 mRNAs in distinct neuronal populations of the dorsal and ventral striatum. Journal of Comparative Neurology, 355(3), 418–426.CrossRefGoogle Scholar
  84. Le Moine, C., Normand, E., & Bloch, B. (1991). Phenotypical characterization of the rat striatal neurons expressing the D1 dopamine receptor gene. Proceedings of the National Academy of Sciences, 88(10), 4205–4209.CrossRefGoogle Scholar
  85. Lee, S. P., So, C. H., Rashid, A. J., Varghese, G., Cheng, R., Lanca, A. J., . . . George, S. R. (2004). Dopamine D1 and D2 receptor Co-activation generates a novel phospholipase C-mediated calcium signal. Journal of Biological Chemistry, 279(34), 35671–35678.  https://doi.org/10.1074/jbc.m401923200.
  86. Lobo, M. K., Karsten, S. L., Gray, M., Geschwind, D. H., & Yang, X. W. (2006). FACS-array profiling of striatal projection neuron subtypes in juvenile and adult mouse brains. Nature Neuroscience, 9(3), 443–452.CrossRefGoogle Scholar
  87. Long, A. B., Kuhn, C. M., & Platt, M. L. (2009). Serotonin shapes risky decision making in monkeys. Social Cognitive Affective Neuroscience, 4(4), 346–356.  https://doi.org/10.1093/scan/nsp020.CrossRefGoogle Scholar
  88. Lopez-Ibor, J. (1992). Serotonin and psychiatric disorders. International Clinical Psychopharmacology.Google Scholar
  89. Markowitz, H. (1952). Portfolio selection. The Journal of Finance, 7(1), 77–91.  https://doi.org/10.2307/2975974.Google Scholar
  90. Matamales, M., Bertran-Gonzalez, J., Salomon, L., Degos, B., Deniau, J.-M., Valjent, E., . . . Girault, J.-A. (2009). Striatal medium-sized spiny neurons: identification by nuclear staining and study of neuronal subpopulations in BAC transgenic mice. PLoS One, 4(3), e4770.Google Scholar
  91. Matsuda, W., Furuta, T., Nakamura, K. C., Hioki, H., Fujiyama, F., Arai, R., et al. (2009). Single nigrostriatal dopaminergic neurons form widely spread and highly dense axonal arborizations in the neostriatum. The Journal of Neuroscience, 29(2), 444–453.CrossRefGoogle Scholar
  92. McGeorge, A. J., & Faull, R. L. (1989). The organization of the projection from the cerebral cortex to the striatum in the rat. Neuroscience, 29(3), 503–537.CrossRefGoogle Scholar
  93. Mink, J. W. (1996). The basal ganglia: Focused selection and inhibition of competing motor programs. Progress in Neurobiology, 50(4), 381.CrossRefGoogle Scholar
  94. Morita, K., Morishima, M., Sakai, K., & Kawaguchi, Y. (2012). Reinforcement learning: Computing the temporal difference of values via distinct corticostriatal pathways. Trends in Neurosciences, 35(8), 457–467.CrossRefGoogle Scholar
  95. Moyer, J. T., Wolf, J. A., & Finkel, L. H. (2007). Effects of dopaminergic modulation on the integrative properties of the ventral striatal medium spiny neuron. Journal of Neurophysiology, 98(6), 3731–3748.  https://doi.org/10.1152/jn.00335.2007.CrossRefGoogle Scholar
  96. Murphy, S. E., Longhitano, C., Ayres, R. E., Cowen, P. J., Harmer, C. J., & Rogers, R. D. (2009). The role of serotonin in nonnormative risky choice: The effects of tryptophan supplements on the “reflection effect” in healthy adult volunteers. Journal of Cognitive Neuroscience, 21(9), 1709–1719.  https://doi.org/10.1162/jocn.2009.21122.CrossRefGoogle Scholar
  97. Nadjar, A., Brotchie, J. M., Guigoni, C., Li, Q., Zhou, S.-B., Wang, G.-J., … Bezard, E. (2006). Phenotype of striatofugal medium spiny neurons in parkinsonian and dyskinetic nonhuman primates: a call for a reappraisal of the functional organization of the basal ganglia. The Journal of Neuroscience, 26(34), 8653–8661.Google Scholar
  98. Nakamura, K. (2013). The role of the dorsal raphé nucleus in reward-seeking behavior. Frontiers in Integrative Neuroscience, 7.Google Scholar
  99. Parent, A., & Hazrati, L.-N. (1995). Functional anatomy of the basal ganglia. II. The place of subthalamic nucleus and external pallidium in basal ganglia circuitry. Brain Research Reviews, 20(1), 128–154.CrossRefGoogle Scholar
  100. Perreault, M. L., Fan, T., Alijaniaram, M., O’Dowd, B. F., & George, S. R. (2012). Dopamine D1-D2 receptor heteromer in dual phenotype GABA/glutamate-coexpressing striatal medium spiny neurons: Regulation of BDNF, GAD67 and VGLUT1/2. PLoS ONE, 7(3), e33348.  https://doi.org/10.1371/journal.pone.0033348.CrossRefGoogle Scholar
  101. Perreault, M. L., Hasbi, A., Alijaniaram, M., Fan, T., Varghese, G., Fletcher, P. J., . . . George, S. R. (2010). The dopamine D1-D2 receptor heteromer localizes in dynorphin/enkephalin neurons: increased high affinity state following amphetamine and in schizophrenia. Journal of Biological Chemistry, 285(47), 36625–36634.  https://doi.org/10.1074/jbc.m110.159954.
  102. Perreault, M. L., Hasbi, A., O’Dowd, B. F., & George, S. R. (2011). The dopamine d1-d2 receptor heteromer in striatal medium spiny neurons: Evidence for a third distinct neuronal pathway in Basal Ganglia. Frontiers in Neuroanatomy, 5, 31.  https://doi.org/10.3389/fnana.2011.00031.CrossRefGoogle Scholar
  103. Preuschoff, K., Bossaerts, P., & Quartz, S. R. (2006). Neural differentiation of expected reward and risk in human subcortical structures. Neuron, 51(3), 381–390.  https://doi.org/10.1016/j.neuron.2006.06.024.CrossRefGoogle Scholar
  104. Rashid, A. J., O’Dowd, B. F., Verma, V., & George, S. R. (2007). Neuronal Gq/11-coupled dopamine receptors: An uncharted role for dopamine. Trends in Pharmacological Sciences, 28(11), 551–555.CrossRefGoogle Scholar
  105. Reynolds, J. N., Hyland, B. I., & Wickens, J. R. (2001). A cellular mechanism of reward-related learning. Nature, 413(6851), 67–70.CrossRefGoogle Scholar
  106. Reynolds, J. N., & Wickens, J. R. (2002). Dopamine-dependent plasticity of corticostriatal synapses. Neural Network, 15(4–6), 507–521.CrossRefGoogle Scholar
  107. Robinson, O. J., Cools, R., & Sahakian, B. J. (2012). Tryptophan depletion disinhibits punishment but not reward prediction: Implications for resilience. Psychopharmacology (Berl), 219(2), 599–605.  https://doi.org/10.1007/s00213-011-2410-5.CrossRefGoogle Scholar
  108. Rogers, R. D. (2011). The roles of dopamine and serotonin in decision making: Evidence from pharmacological experiments in humans. Neuropsychopharmacology, 36(1), 114–132.  https://doi.org/10.1038/npp.2010.165.CrossRefGoogle Scholar
  109. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80(1), 1–27.MathSciNetCrossRefGoogle Scholar
  110. Schultz, W. (2013). Updating dopamine reward signals. Current Opinion in Neurobiology, 23(2), 229–238.CrossRefGoogle Scholar
  111. Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P., & Dolan, R. (2012). Serotonin selectively modulates reward value in human decision-making. Journal of Neuroscience, 32(17), 5833–5842.  https://doi.org/10.1523/JNEUROSCI.0053-12.2012.CrossRefGoogle Scholar
  112. Shuen, J. A., Chen, M., Gloss, B., & Calakos, N. (2008). Drd1a-tdTomato BAC transgenic mice for simultaneous visualization of medium spiny neurons in the direct and indirect pathways of the basal ganglia. The Journal of Neuroscience, 28(11), 2681–2685.CrossRefGoogle Scholar
  113. So, C. H., Verma, V., Alijaniaram, M., Cheng, R., Rashid, A. J., O’Dowd, B. F., et al. (2009). Calcium signaling by dopamine D5 receptor and D5-D2 receptor hetero-oligomers occurs by a mechanism distinct from that for dopamine D1-D2 receptor hetero-oligomers. Molecular Pharmacology, 75(4), 843–854.  https://doi.org/10.1124/mol.108.051805.CrossRefGoogle Scholar
  114. Spehlmann, R., & Stahl, S. (1976). Dopamine acetylcholine imbalance in Parkinson’s disease: Possible regenerative overgrowth of cholinergic axon terminals. The Lancet, 307(7962), 724–726.CrossRefGoogle Scholar
  115. Stauffer, W. R., Lak, A., & Schultz, W. (2014). Dopamine reward prediction error responses reflect marginal utility. Current Biology, 24(21), 2491–2500.CrossRefGoogle Scholar
  116. Stopper, C. M., & Floresco, S. B. (2011). Contributions of the nucleus accumbens and its subregions to different aspects of risk-based decision making. Cognitive Affective Behavioral Neuroscience, 11(1), 97–112.  https://doi.org/10.3758/s13415-010-0015-9.CrossRefGoogle Scholar
  117. Stringer, S., Rolls, E., Trappenberg, T., & De Araujo, I. (2002). Self-organizing continuous attractor networks and path integration: Two-dimensional models of place cells. Network: Computation in Neural Systems, 13(4), 429–446.MATHCrossRefGoogle Scholar
  118. Surmeier, D., & Kitai, S. (1993). D 1 and D 2 dopamine receptor modulation of sodium and potassium currents in rat neostriatal neurons. Progress in Brain Research, 99, 309–324.CrossRefGoogle Scholar
  119. Surmeier, D. J., Ding, J., Day, M., Wang, Z., & Shen, W. (2007). D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends in Neurosciences, 30(5), 228–235.  https://doi.org/10.1016/j.tins.2007.03.008.CrossRefGoogle Scholar
  120. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Adaptive computations and machine learning. Bradford: MIT Press.Google Scholar
  121. Suzuki, M., Hurd, Y. L., Sokoloff, P., Schwartz, J. C., & Sedvall, G. (1998). D3 dopamine receptor mRNA is widely expressed in the human brain. Brain Research, 779(1–2), 58–74.CrossRefGoogle Scholar
  122. Swann, A. C., Lijffijt, M., Lane, S. D., Cox, B., Steinberg, J. L., & Moeller, F. G. (2013). Norepinephrine and impulsivity: Effects of acute yohimbine. Psychopharmacology (Berl), 229(1), 83–94.CrossRefGoogle Scholar
  123. Tai, L.-H., Lee, A. M., Benavidez, N., Bonci, A., & Wilbrecht, L. (2012). Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nature Neuroscience, 15(9), 1281–1289.CrossRefGoogle Scholar
  124. Tanaka, S. C., Schweighofer, N., Asahi, S., Shishida, K., Okamoto, Y., Yamawaki, S., et al. (2007). Serotonin differentially regulates short- and long-term prediction of rewards in the ventral and dorsal striatum. PLoS ONE, 2(12), e1333.  https://doi.org/10.1371/journal.pone.0001333.CrossRefGoogle Scholar
  125. Tanaka, S. C., Shishida, K., Schweighofer, N., Okamoto, Y., Yamawaki, S., & Doya, K. (2009). Serotonin affects association of aversive outcomes to past actions. Journal of Neuroscience, 29(50), 15669–15674.  https://doi.org/10.1523/JNEUROSCI.2799-09.2009.CrossRefGoogle Scholar
  126. Tops, M., Russo, S., Boksem, M. A., & Tucker, D. M. (2009). Serotonin: Modulator of a drive to withdraw. Brain and Cognition, 71(3), 427–436.  https://doi.org/10.1016/j.bandc.2009.03.009.CrossRefGoogle Scholar
  127. Valjent, E., Bertran-Gonzalez, J., Hervé, D., Fisone, G., & Girault, J.-A. (2009). Looking BAC at striatal signaling: Cell-specific analysis in new transgenic mice. Trends in Neurosciences, 32(10), 538–547.CrossRefGoogle Scholar
  128. Wang, R., Macmillan, L., Fremeau, R., Jr., Magnuson, M., Lindner, J., & Limbird, L. (1996). Expression of α2-adrenergic receptor subtypes in the mouse brain: Evaluation of spatial and temporal information imparted by 3 kb of 5′ regulatory sequence for the α2A AR-receptor gene in transgenic animals. Neuroscience, 74(1), 199–218.CrossRefGoogle Scholar
  129. Ward, R. P., & Dorsa, D. M. (1996). Colocalization of serotonin receptor subtypes 5-HT2A, 5-HT2C, and 5-HT6 with neuropeptides in rat striatum. Journal of Comparative Neurology, 370(3), 405–414.CrossRefGoogle Scholar
  130. Zhong, S., Israel, S., Xue, H., Ebstein, R. P., & Chew, S. H. (2009a). Monoamine oxidase A gene (MAOA) associated with attitude towards longshot risks. PLoS ONE, 4(12), e8516.CrossRefGoogle Scholar
  131. Zhong, S., Israel, S., Xue, H., Sham, P. C., Ebstein, R. P., & Chew, S. H. (2009b). A neurochemical approach to valuation sensitivity over gains and losses. Proceedings of the Royal Society B: Biological Sciences, 276(1676), 4181–4188.CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • Pragathi Priyadharsini Balasubramani
    • 1
  • V. Srinivasa Chakravarthy
    • 2
  • Balaraman Ravindran
    • 3
  • Ahmed A. Moustafa
    • 4
  1. 1.Department of NeuroscienceUniversity of Rochester Medical CenterRochester, New YorkUSA
  2. 2.Department of BiotechnologyIndian Institute of Technology, MadrasChennaiIndia
  3. 3.Department of Computer Science and EngineeringIndian Institute of Technology, MadrasChennaiIndia
  4. 4.School of Social Sciences and Psychology & Marcs Institute for Brain and BehaviourWestern Sydney UniversitySydneyAustralia

Personalised recommendations