The Cerebellum

, 7:433 | Cite as

The Cerebellum Is Involved in Reward-based Reversal Learning

  • Patrizia Thoma
  • Christian Bellebaum
  • Benno Koch
  • Michael Schwarz
  • Irene Daum
Article

Abstract

The cerebellum has recently been discussed in terms of a possible involvement in reward-based associative learning. To clarify the cerebellar contribution, eight patients with focal vascular lesions of the cerebellum and a group of 24 healthy subjects matched for age and IQ were compared on a range of different probabilistic outcome-based associative learning tasks, assessing acquisition, reversal, cognitive transfer, and generalization as well as the effect of reward magnitude. Cerebellar patients showed intact acquisition of stimulus contingencies, while reward-based reversal learning was significantly impaired. In addition, the patients showed slower acquisition of new stimulus contingencies in a second reward-based learning task, which might reflect reduced carry-over effects. Reward magnitude affected learning only during initial acquisition, with better learning on trials with high rewards in patients and control subjects. Overall, the findings suggest that the cerebellum is implicated in reversal learning as a dissociable component of reward-based learning.

Keywords

Cerebellum Reversal learning Acquired equivalence Associative learning Reinforcement 

References

  1. 1.
    Delgado MR (2007) Reward-related responses in the human striatum. Ann N Y Acad Sci 1104:70–88PubMedCrossRefGoogle Scholar
  2. 2.
    Schultz W (2007) Behavioral dopamine signals. Trends Neurosci 30(5):203–210PubMedCrossRefGoogle Scholar
  3. 3.
    Delgado MR, Locke HM, Stenger VA, Fiez JA (2003) Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations. Cogn Affect Behav Neurosci 3(1):27–38PubMedCrossRefGoogle Scholar
  4. 4.
    Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA (2000) Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84(6):3072–3077PubMedGoogle Scholar
  5. 5.
    O’Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304(5669):452–454PubMedCrossRefGoogle Scholar
  6. 6.
    Yacubian J, Glascher J, Schroeder K, Sommer T, Braus DF, Buchel C (2006) Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain. J Neurosci 26(37):9530–9537PubMedCrossRefGoogle Scholar
  7. 7.
    Seymour B, Daw N, Dayan P, Singer T, Dolan R (2007) Differential encoding of losses and gains in the human striatum. J Neurosci 27(18):4826–4831PubMedCrossRefGoogle Scholar
  8. 8.
    Bellebaum C, Koch B, Schwarz M, Daum I (2008) Focal basal ganglia lesions are associated with impairments in reward based reversal learning. Brain 131:829–841PubMedCrossRefGoogle Scholar
  9. 9.
    Cools R, Barker RA, Sahakian BJ, Robbins TW (2001) Enhanced or impaired cognitive function in Parkinson’s disease as a function of dopaminergic medication and task demands. Cereb Cortex 11(12):1136–1143PubMedCrossRefGoogle Scholar
  10. 10.
    Cools R, Clark L, Owen AM, Robbins TW (2002) Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci 22(11):4563–4567PubMedGoogle Scholar
  11. 11.
    O’Doherty J, Kringelbach ML, Rolls ET, Hornak J, Andrews C (2001) Abstract reward and punishment representations in the human orbitofrontal cortex. Nat Neurosci 4(1):95–102PubMedCrossRefGoogle Scholar
  12. 12.
    Tremblay L, Schultz W (1999) Relative reward preference in primate orbitofrontal cortex. Nature 398(6729):704–708PubMedCrossRefGoogle Scholar
  13. 13.
    Holroyd CB, Coles MG (2002) The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol Rev 109(4):679–709PubMedCrossRefGoogle Scholar
  14. 14.
    Daum I, Ackermann H (1997) Neuropsychological abnormalities in cerebellar syndromes—fact or fiction? Int Rev Neurobiol 41:455–471PubMedCrossRefGoogle Scholar
  15. 15.
    Schmahmann JD (2004) Disorders of the cerebellum: ataxia, dysmetria of thought, and the cerebellar cognitive affective syndrome. J Neuropsychiatry Clin Neurosci 16(3):367–378PubMedGoogle Scholar
  16. 16.
    Timmann D, Daum I (2007) Cerebellar contributions to cognitive functions: a progress report after two decades of research. Cerebellum 6(3):159–162PubMedCrossRefGoogle Scholar
  17. 17.
    Ivry RB, Spencer RM, Zelaznik HN, Diedrichsen J (2002) The cerebellum and event timing. Ann N Y Acad Sci 978:302–317PubMedCrossRefGoogle Scholar
  18. 18.
    Leggio MG, Neri P, Graziano A, Mandolesi L, Molinari M, Petrosini L (1999) Cerebellar contribution to spatial event processing: characterization of procedural learning. Exp Brain Res 127(1):1–11PubMedCrossRefGoogle Scholar
  19. 19.
    Raymond JL, Lisberger SG, Mauk MD (1996) The cerebellum: a neuronal learning machine? Science 272(5265):1126–1131PubMedCrossRefGoogle Scholar
  20. 20.
    Timmann D, Drepper J, Calabrese S, Burgerhoff K, Maschke M, Kolb FP et al (2004) Use of sequence information in associative learning in control subjects and cerebellar patients. Cerebellum 3(2):75–82PubMedCrossRefGoogle Scholar
  21. 21.
    Drepper J, Timmann D, Kolb FP, Diener HC (1999) Non-motor associative learning in patients with isolated degenerative cerebellar disease. Brain 122(Pt 1):87–97PubMedCrossRefGoogle Scholar
  22. 22.
    Doyon J, Song AW, Karni A, Lalonde F, Adams MM, Ungerleider LG (2002) Experience-dependent changes in cerebellar contributions to motor sequence learning. Proc Natl Acad Sci U S A 99(2):1017–1022PubMedCrossRefGoogle Scholar
  23. 23.
    Daum I, Schugens MM, Ackermann H, Lutzenberger W, Dichgans J, Birbaumer N (1993) Classical conditioning after cerebellar lesions in humans. Behav Neurosci 107(5):748–756PubMedCrossRefGoogle Scholar
  24. 24.
    Gerwig M, Kolb FP, Timmann D (2007) The involvement of the human cerebellum in eyeblink conditioning. Cerebellum 6(1):38–57PubMedCrossRefGoogle Scholar
  25. 25.
    Blackwood N, Ffytche D, Simmons A, Bentall R, Murray R, Howard R (2004) The cerebellum and decision making under uncertainty. Brain Res Cogn Brain Res 20(1):46–53PubMedCrossRefGoogle Scholar
  26. 26.
    Lalonde R (1994) Cerebellar contributions to instrumental learning. Neurosci Biobehav Rev 18(2):161–170PubMedCrossRefGoogle Scholar
  27. 27.
    Anderson CM, Maas LC, Frederick B, Bendor JT, Spencer TJ, Livni E et al (2006) Cerebellar vermis involvement in cocaine-related behaviors. Neuropsychopharmacology 31(6):1318–1326PubMedGoogle Scholar
  28. 28.
    Martin-Solch C, Magyar S, Kunig G, Missimer J, Schultz W, Leenders KL (2001) Changes in brain activation associated with reward processing in smokers and nonsmokers. A positron emission tomography study. Exp Brain Res 139(3):278–286PubMedCrossRefGoogle Scholar
  29. 29.
    Olbrich HM, Valerius G, Paris C, Hagenbuch F, Ebert D, Juengling FD (2006) Brain activation during craving for alcohol measured by positron emission tomography. Aust N Z J Psychiatry 40(2):171–178PubMedCrossRefGoogle Scholar
  30. 30.
    Bonson KR, Grant SJ, Contoreggi CS, Links JM, Metcalfe J, Weyl HL et al (2002) Neural systems and cue-induced cocaine craving. Neuropsychopharmacology 26(3):376–386PubMedCrossRefGoogle Scholar
  31. 31.
    Volkow ND, Wang GJ, Ma Y, Fowler JS, Zhu W, Maynard L et al (2003) Expectation enhances the regional brain metabolic and the reinforcing effects of stimulants in cocaine abusers. J Neurosci 23(36):11461–11468PubMedGoogle Scholar
  32. 32.
    Grant S, London ED, Newlin DB, Villemagne VL, Liu X, Contoreggi C et al (1996) Activation of memory circuits during cue-elicited cocaine craving. Proc Natl Acad Sci U S A 93(21):12040–12045PubMedCrossRefGoogle Scholar
  33. 33.
    Ramnani N, Elliott R, Athwal BS, Passingham RE (2004) Prediction error for free monetary reward in the human prefrontal cortex. Neuroimage 23(3):777–786PubMedCrossRefGoogle Scholar
  34. 34.
    Tanaka SC, Doya K, Okada G, Ueda K, Okamoto Y, Yamawaki S (2004) Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat Neurosci 7(8):887–893PubMedCrossRefGoogle Scholar
  35. 35.
    Holstege G, Georgiadis JR (2004) The emotional brain: neural correlates of cat sexual behavior and human male ejaculation. Prog Brain Res 143:39–45PubMedCrossRefGoogle Scholar
  36. 36.
    Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80(1):1–27PubMedGoogle Scholar
  37. 37.
    Bellebaum C, Daum I (2008) Learning-related changes in reward expectancy are reflected in the feedback-related negativity. Eur J Neurosci 27(7):1823–1835PubMedCrossRefGoogle Scholar
  38. 38.
    Marr D (1969) A theory of cerebellar cortex. J Physiol 202(2):437–470PubMedGoogle Scholar
  39. 39.
    Schultz W, Dickinson A (2000) Neuronal coding of prediction errors. Annu Rev Neurosci 23:473–500PubMedCrossRefGoogle Scholar
  40. 40.
    Ramnani N (2006) The primate cortico-cerebellar system: anatomy and function. Nat Rev Neurosci 7(7):511–522PubMedCrossRefGoogle Scholar
  41. 41.
    Frank MJ, Seeberger LC, O’eilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306(5703):1940–1943PubMedCrossRefGoogle Scholar
  42. 42.
    Myers CE, Shohamy D, Gluck MA, Grossman S, Kluger A, Ferris S et al (2003) Dissociating hippocampal versus basal ganglia contributions to learning and transfer. J Cogn Neurosci 15(2):185–193PubMedCrossRefGoogle Scholar
  43. 43.
    Weiler JA, Bellebaum C, Daum I (2008) Aging affects acquisition and reversal of reward-based associative learning. Learn Mem 15(4):190–197PubMedCrossRefGoogle Scholar
  44. 44.
    Keri S, Nagy O, Kelemen O, Myers CE, Gluck MA (2005) Dissociation between medial temporal lobe and basal ganglia memory systems in schizophrenia. Schizophr Res 77(2–3):321–328PubMedCrossRefGoogle Scholar
  45. 45.
    Weis S, Klaver P, Reul J, Elger CE, Fernandez G (2004) Temporal and cerebellar brain regions that support both declarative memory formation and retrieval. Cereb Cortex 14(3):256–267PubMedCrossRefGoogle Scholar
  46. 46.
    Matsui T, Hiranu A (1978) An atlas of the human brain for computerized tomography. Fischer, StuttgartGoogle Scholar
  47. 47.
    Dahl G (1972) Reduzierter Wechsler Intelligentest (short version of the Wechsler Intelligence Scale). Hain, MeisenheimGoogle Scholar
  48. 48.
    Wechsler D (1981) The Wechsler Adult Intelligence Scale—revised. The Psychological Corporation, San AntonioGoogle Scholar
  49. 49.
    Nelson HE (1976) A modified card sorting test sensitive to frontal lobe defects. Cortex 12:313–324PubMedGoogle Scholar
  50. 50.
    Stanton ME, Peloso E, Brown KL, Rodier P (2007) Discrimination learning and reversal of the conditioned eyeblink reflex in a rodent model of autism. Behav Brain Res 176(1):133–140PubMedCrossRefGoogle Scholar
  51. 51.
    Robinson S, Sandstrom SM, Denenberg VH, Palmiter RD (2005) Distinguishing whether dopamine regulates liking, wanting, and/or learning about rewards. Behav Neurosci 119(1):5–15PubMedCrossRefGoogle Scholar
  52. 52.
    Goerendt IK, Lawrence AD, Brooks DJ (2004) Reward processing in health and Parkinson’s disease: neural organization and reorganization. Cereb Cortex 14(1):73–80PubMedCrossRefGoogle Scholar
  53. 53.
    Ernst M, Kimes AS, London ED, Matochik JA, Eldreth D, Tata S et al (2003) Neural substrates of decision making in adults with attention deficit hyperactivity disorder. Am J Psychiatry 160(6):1061–1070PubMedCrossRefGoogle Scholar
  54. 54.
    Blakemore SJ, Sirigu A (2003) Action prediction in the cerebellum and in the parietal lobe. Exp Brain Res 153(2):239–245PubMedCrossRefGoogle Scholar
  55. 55.
    Tobler PN, O’Doherty JP, Dolan RJ, Schultz W (2006) Human neural learning depends on reward prediction errors in the blocking paradigm. J Neurophysiol 95(1):301–310PubMedCrossRefGoogle Scholar
  56. 56.
    Gluck MA, Allen MT, Myers CE, Thompson RF (2001) Cerebellar substrates for error correction in motor conditioning. Neurobiol Learn Mem 76(3):314–341PubMedCrossRefGoogle Scholar
  57. 57.
    Joel D, Niv Y, Ruppin E (2002) Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Netw 15(4–6):535–547PubMedCrossRefGoogle Scholar
  58. 58.
    Fiez JA, Petersen SE, Cheney MK, Raichle ME (1992) Impaired non-motor learning and error detection associated with cerebellar damage. A single case study. Brain 115(Pt 1):155–178PubMedCrossRefGoogle Scholar
  59. 59.
    Cools R, Altamirano L, D’Esposito M (2006) Reversal learning in Parkinson’s disease depends on medication status and outcome valence. Neuropsychologia 44(10):1663–1673PubMedCrossRefGoogle Scholar
  60. 60.
    Middleton FA, Strick PL (2001) Cerebellar projections to the prefrontal cortex of the primate. J Neurosci 21(2):700–712PubMedGoogle Scholar
  61. 61.
    Frank MJ, Claus ED (2006) Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychol Rev 113(2):300–326PubMedCrossRefGoogle Scholar
  62. 62.
    Ikai Y, Takada M, Mizuno N (1994) Single neurons in the ventral tegmental area that project to both the cerebral and cerebellar cortical areas by way of axon collaterals. Neuroscience 61(4):925–934PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Patrizia Thoma
    • 1
  • Christian Bellebaum
    • 1
  • Benno Koch
    • 2
  • Michael Schwarz
    • 2
  • Irene Daum
    • 1
  1. 1.Institute of Cognitive Neuroscience, Department of Neuropsychology, Faculty of PsychologyRuhr-University of BochumBochumGermany
  2. 2.Department of Neurology, Klinikum DortmundDortmundGermany

Personalised recommendations