A Computational Model of the Role of Serotonin in Reversal Learning

  • Graeme Hattan
  • Bernd Porr
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7426)


It has been shown that the action of serotonin on the orbito-frontal cortex (OFC) is crucial for the inhibition phase of reversal learning. Serotonin has also been shown to facilitate the induction of LTD throughout the prefrontal cortex. We present a biologically realistic, systems level model which proposes a mechanism for the release of serotonin in response to the omission of an expected reward. Serotonin release, as a result of the combination of excitation of the dorsal raphé nucleus (DRN) pathway and the lack of inhibition of the DRN from the lateral habenula, leads to LTD in the OFC and suppression of excitation of the nucleus accumbens shell due to reward predicting sensory stimuli. Behavioural inhibition is controlled via the shell-ventral pallido-mediodorsal pathway, which serves as a feed forward switching mechanism and enables the behavioural inhibition required to achieve reversal learning.


Conditional Stimulus Unconditional Stimulus Ventral Tegmental Area Behavioural Inhibition Orbitofrontal Cortex 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bianco, I.H., Wilson, S.W.: The habenular nuclei: a conserved asymmetric relay station in the vertebrate brain. Philos. Trans. R. Soc. Lond., B, Biol. Sci. 364(1519), 1005–1020 (2009)Google Scholar
  2. 2.
    Brown, J., Bullock, D., Grossberg, S.: How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J. Neurosci. 19(23), 10502–10511 (1999)Google Scholar
  3. 3.
    Clarke, H.F., Dalley, J.W., Crofts, H.S., Robbins, T.W., Roberts, A.C.: Cognitive inflexibility after prefrontal serotonin depletion. Science 304(5672), 878–881 (2004)CrossRefGoogle Scholar
  4. 4.
    Clarke, H.F., Walker, S.C., Crofts, H.S., Dalley, J.W., Robbins, T.W., Roberts, A.C.: Prefrontal serotonin depletion affects reversal learning but not attentional set shifting. J. Neurosci. 25(2), 532–538 (2005)CrossRefGoogle Scholar
  5. 5.
    Clarke, H.F., Walker, S.C., Dalley, J.W., Robbins, T.W., Roberts, A.C.: Cognitive inflexibility after prefrontal serotonin depletion is behaviorally and neurochemically specific. Cereb. Cortex 17(1), 18–27 (2007)CrossRefGoogle Scholar
  6. 6.
    Dayan, P.: Motivated reinforcement learning. In: Advances in Neural Information Processing Systems 13 (2001)Google Scholar
  7. 7.
    Dayan, P., Huys, Q.J.M.: Serotonin in affective control. Annual Review of Neuroscience 32, 95–126 (2009)CrossRefGoogle Scholar
  8. 8.
    Grace, A.A., Floresco, S.B., Goto, Y., Lodge, D.J.: Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends Neurosci. 30(5), 220–227 (2007)CrossRefGoogle Scholar
  9. 9.
    Lapiz-Bluhm, M.D.S., Soto-Piña, A.E., Hensler, J.G., Morilak, D.A.: Chronic intermittent cold stress and serotonin depletion induce deficits of reversal learning in an attentional set-shifting test in rats. Psychopharmacology (Berl.) 202(1-3), 329–341 (2009)CrossRefGoogle Scholar
  10. 10.
    Moren, J., Balkenius, C.: A computational model of emotional learning in the amygdala. In: From Animals to Animats 6, pp. 383–391 (2000)Google Scholar
  11. 11.
    Napier, R.M., Macrae, M., Kehoe, E.J.: Rapid reaquisition in conditioning of the rabbit’s nictitating membrane response. J. Exp. Psychol. Anim. Behav. Process. 18(2), 182–192 (1992)CrossRefGoogle Scholar
  12. 12.
    O’Reilly, R.C., Frank, M.J., Hazy, T.E., Watz, B.: Pvlv: the primary value and learned value pavlovian learning algorithm. Behav. Neurosci. 121(1), 31–49 (2007)CrossRefGoogle Scholar
  13. 13.
    Pavlov, I.P.: Conditioned reflexes. Oxford University Press, Oxford (1927)Google Scholar
  14. 14.
    Porr, B., Wörgötter, F.: Learning with ”relevance”: using a third factor to stabilize hebbian learning. Neural Computation 19(10), 2694–2719 (2007)zbMATHCrossRefGoogle Scholar
  15. 15.
    Schoenbaum, G., Chiba, A.A., Gallagher, M.: Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J. Neurosci. 19(5), 1876–1884 (1999)Google Scholar
  16. 16.
    Schoenbaum, G., Roesch, M.R., Stalnaker, T.A., Takahashi, Y.K.: A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat. Rev. Neurosci. 10(12), 885–892 (2009)Google Scholar
  17. 17.
    Schotanus, S.M., Chergui, K.: Dopamine d1 receptors and group i metabotropic glutamate receptors contribute to the induction of long-term potentiation in the nucleus accumbens. Neuropharmacology 54(5), 837–844 (2008)CrossRefGoogle Scholar
  18. 18.
    Stern, W.C., Johnson, A., Bronzino, J.D., Morgane, P.J.: Effects of electrical stimulation of the lateral habenula on single-unit activity of raphe neurons. Exp. Neurol. 65(2), 326–342 (1979)CrossRefGoogle Scholar
  19. 19.
    Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. MIT, Cambridge (1998)Google Scholar
  20. 20.
    Thompson, A.M., Porr, B., Wörgötter, F.: Learning and reversal learning in the subcortical limbic system: A computational model. Adaptive Behavior 18(3-4), 211 (2010)CrossRefGoogle Scholar
  21. 21.
    Zahm, D.S.: An integrative neuroanatomical perspective on some subcortical substrates of adaptive responding with emphasis on the nucleus accumbens. Neurosci. Biobehav. Rev. 24(1), 85–105 (2000)CrossRefGoogle Scholar
  22. 22.
    Zhong, P., Liu, W., Gu, Z., Yan, Z.: Serotonin facilitates long-term depression induction in prefrontal cortex via p38 mapk/rab5-mediated enhancement of ampa receptor internalization. J. Physiol (Lond.) 586(pt. 18), 4465–4479 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Graeme Hattan
    • 1
  • Bernd Porr
    • 1
  1. 1.Biomedical Engineering, School of EngineeringUniversity of GlasgowUnited Kingdom

Personalised recommendations