Understanding the Role of Serotonin in Basal Ganglia through a Unified Model

  • Balasubramani Pragathi Priyadharsini
  • Balaraman Ravindran
  • V. Srinivasa Chakravarthy
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7552)

Abstract

We present a Reinforcement Learning (RL)-based model of serotonin which tries to reconcile some of the diverse roles of the neuromodulator. The proposed model uses a novel formulation of utility function, which is a weighted sum of the traditional value function and the risk function. Serotonin is represented by the weightage, α, used in this combination. The model is applied to three different experimental paradigms: 1) bee foraging behavior, which involves decision making based on risk, 2) temporal reward prediction task, in which serotonin (α) controls the time-scale of reward prediction, and 3) reward/punishment prediction task, in which punishment prediction error depends on serotonin levels. The three diverse roles of serotonin – in time-scale of reward prediction, risk modeling, and punishment prediction – is explained within a single framework by the model.

Keywords

Serotonin Dopamine Reinforcement Learning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Schultz, W., Dayan, P., Montague, P.R.: A neural substrate of prediction and reward. Science 275(5306), 1593–1599 (1997)CrossRefGoogle Scholar
  2. 2.
    Cools, R., Nakamura, K., Daw, N.D.: Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology 36(1), 98–113 (2011)CrossRefGoogle Scholar
  3. 3.
    Tops, M., et al.: Serotonin: modulator of a drive to withdraw. Brain Cogn. 71(3), 427–436 (2009)CrossRefGoogle Scholar
  4. 4.
    Rogers, R.D.: The roles of dopamine and serotonin in decision making: evidence from pharmacological experiments in humans. Neuropsychopharmacology 36(1), 114–132 (2011)CrossRefGoogle Scholar
  5. 5.
    Schultz, W.: Dopamine signals for reward value and risk: basic and recent data. Behav. Brain Funct. 6, 24 (2010)CrossRefGoogle Scholar
  6. 6.
    Daw, N.D., Kakade, S., Dayan, P.: Opponent interactions between serotonin and dopamine. Neural Netw. 15(4-6), 603–616 (2002)CrossRefGoogle Scholar
  7. 7.
    Doya, K.: Metalearning and neuromodulation. Neural Netw. 15(4-6), 495–506 (2002)CrossRefGoogle Scholar
  8. 8.
    Tanaka, S.C., et al.: Serotonin differentially regulates short- and long-term prediction of rewards in the ventral and dorsal striatum. PLoS One 2(12), e1333 (2007)Google Scholar
  9. 9.
    Bell, D.E.: Risk, return and utility. Management Science 41, 23–30 (1995)MATHCrossRefGoogle Scholar
  10. 10.
    Montague, P.R., et al.: Bee foraging in uncertain environments using predictive hebbian learning. Nature 377(6551), 725–728 (1995)CrossRefGoogle Scholar
  11. 11.
    Cools, R., Robinson, O.J., Sahakian, B.: Acute tryptophan depletion in healthy volunteers enhances punishment prediction but does not affect reward prediction. Neuropsychopharmacology 33(9), 2291–2299 (2008)CrossRefGoogle Scholar
  12. 12.
    d’Acremont, M., et al.: Neural correlates of risk prediction error during reinforcement learning in humans. Neuroimage 47(4), 1929–1939 (2009)CrossRefGoogle Scholar
  13. 13.
    Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. Adaptive Computations and Machine Learning (1998)Google Scholar
  14. 14.
    Kahneman, D., Tversky, A.: Prospect theory: an analysis of decision under risk. Econometrica 47, 263–292 (1979)MATHCrossRefGoogle Scholar
  15. 15.
    Robinson, O.J., Cools, R., Sahakian, B.J.: Tryptophan depletion disinhibits punishment but not reward prediction: implications for resilience. Psychopharmacology (Berl) 219(2), 599–605 (2012)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Balasubramani Pragathi Priyadharsini
    • 1
  • Balaraman Ravindran
    • 2
  • V. Srinivasa Chakravarthy
    • 1
  1. 1.Department of BiotechnologyIndian Institute of Technology, MadrasChennaiIndia
  2. 2.Department of Computer Science and EngineeringIndian Institute of Technology, MadrasChennaiIndia

Personalised recommendations