Advertisement

Journal of Comparative Physiology A

, Volume 191, Issue 3, pp 201–211 | Cite as

Expectations and outcomes: decision-making in the primate brain

  • Allison N. McCoy
  • Michael L. Platt
Review

Abstract

Success in a constantly changing environment requires that decision-making strategies be updated as reward contingencies change. How this is accomplished by the nervous system has, until recently, remained a profound mystery. New studies coupling economic theory with neurophysiological techniques have revealed the explicit representation of behavioral value. Specifically, when fluid reinforcement is paired with visually-guided eye movements, neurons in parietal cortex, prefrontal cortex, the basal ganglia, and superior colliculus—all nodes in a network linking visual stimulation with the generation of oculomotor behavior—encode the expected value of targets lying within their response fields. Other brain areas have been implicated in the processing of reward-related information in the abstract: midbrain dopaminergic neurons, for instance, signal an error in reward prediction. Still other brain areas link information about reward to the selection and performance of specific actions in order for behavior to adapt to changing environmental exigencies. Neurons in posterior cingulate cortex have been shown to carry signals related to both reward outcomes and oculomotor behavior, suggesting that they participate in updating estimates of orienting value.

Keywords

Anterior Cingulate Cortex Superior Colliculus Dopamine Neuron Orbitofrontal Cortex Ventral Striatum 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Amador N, Schlag-Rey M, Schlag J (2000) Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. J Neurophysiol 84:2166–2170PubMedGoogle Scholar
  2. Arnauld A, Nichole P (1662) The art of thinking: Port-Royal logic. Translated by Dickoff J, James P. Bobbs-Merrill, IndianapolisGoogle Scholar
  3. Baleydier C, Mauguiere F (1980) The duality of the cingulate gyrus in monkey. Neuroanatomical study and functional hypothesis. Brain 103(3):525–554PubMedGoogle Scholar
  4. Basso MA, Wurtz RH (1997) Modulation of neuronal activity by target uncertainty. Nature 389:66–69CrossRefPubMedGoogle Scholar
  5. Berns GS, McClure SM, Pagnoni G, Montague PR (2001) Predictability modulates human brain response to reward. J Neurosci 21:2793–2798PubMedGoogle Scholar
  6. Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P (2001) Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30:619–639CrossRefPubMedGoogle Scholar
  7. Bussey TJ, Everitt BJ, Robbins TW (1997) Dissociable effects of cingulate and medial frontal cortex lesions on stimulus-reward learning using a novel Pavlovian autoshaping procedure for the rat: implications for the neurobiology of emotion. Behav Neurosci 111:908–919CrossRefPubMedGoogle Scholar
  8. Camerer CF (2003) Behavioral game theory: experiments in strategic interaction (Roundtable Series in Behavioral Economics). Princeton University Press, PrincetonGoogle Scholar
  9. Cavada C, Goldman-Rakic PS (1989) Posterior parietal cortex in rhesus monkey: I. Parcellation of areas based on distinctive limbic and sensory corticocortical connections. J Comp Neurol 287:393–421PubMedGoogle Scholar
  10. Cavada C, Company T, Tejedor J, Cruz-Rizzolo RJ, Reinoso-Suarez F (2000) The anatomical connections of the macaque monkey orbitofrontal cortex. A review. Cereb Cortex 10:220–242CrossRefPubMedGoogle Scholar
  11. Coe B, Tomihara K, Matsuzawa M, Hikosaka O (2002) Visual and anticipatory bias in three cortical eye fields of the monkey during an adaptive decision-making task. J Neurosci 22(12):5081–5090PubMedGoogle Scholar
  12. Colby CL, Duhamel JR, Goldberg ME (1996) Visual, presaccadic, and cognitive activation of single neurons in monkey lateral intraparietal area. J Neurophysiol 76:2841–2852PubMedGoogle Scholar
  13. Critchley HD, Mathias CJ, Dolan RJ (2001) Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron 29:537–545CrossRefPubMedGoogle Scholar
  14. Cromwell HC, Schultz W (2003) Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol 29:29Google Scholar
  15. Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA (2000) Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84:3072–3077PubMedGoogle Scholar
  16. Delgado MR, Locke HM, Stenger VA, Fiez JA (2003) Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations. Cogn Affect Behav Neurosci 3:27–38PubMedGoogle Scholar
  17. Dorris MC, Munoz DP (1998) Saccadic probability influences motor preparation signals and time to saccadic initiation. J Neurosci 18:7015–7026PubMedGoogle Scholar
  18. Elliott R, Newman JL, Longe OA, Deakin JF (2003) Differential response patterns in the striatum and orbitofrontal cortex to financial reward in humans: a parametric functional magnetic resonance imaging study. J Neurosci 23(1):303–307PubMedGoogle Scholar
  19. Gabriel M, Sparenborg S (1987) Posterior cingulate cortical lesions eliminate learning-related unit activity in the anterior cingulate cortex. Brain Res 409:151–157CrossRefPubMedGoogle Scholar
  20. Gabriel M, Orona E, Foster K, Lambert RW (1980) Cingulate cortical and anterior thalamic neuronal correlates of reversal learning in rabbits. J Comp Physiol Psychol 94:1087–1100PubMedGoogle Scholar
  21. Glimcher P (2002) Decisions, decisions, decisions: choosing a biological science of choice. Neuron 36:323–332CrossRefPubMedGoogle Scholar
  22. Glimcher P (2003) Decisions, Uncertainty, and the brain: the science of neuroeconomics. MIT Press, CambridgeGoogle Scholar
  23. Gnadt JW, Andersen RA (1988) Memory related motor planning activity in posterior parietal cortex of macaque. Exp Brain Res 70:216–220PubMedGoogle Scholar
  24. Gold JI, Shadlen MN (2002) Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36:299–308CrossRefPubMedGoogle Scholar
  25. Goldberg ME, Colby CL, Duhamel JR (1990) Representation of visuomotor space in the parietal lobe of the monkey. Cold Spring Harb Symp Quant Biol 55:729–739PubMedGoogle Scholar
  26. Gottlieb JP, Kusunoki M, Goldberg ME (1998) The representation of visual salience in monkey parietal cortex. Nature 391:481–484CrossRefPubMedGoogle Scholar
  27. Hanes DP, Schall JD (1996) Neural control of voluntary movement initiation. Science 274:427–430CrossRefPubMedGoogle Scholar
  28. Harker KT, Whishaw IQ (2002) Impaired spatial performance in rats with retrosplenial lesions: importance of the spatial problem and the rat strain in identifying lesion effects in a swimming pool. J Neurosci 22:1155–1164PubMedGoogle Scholar
  29. Hassani OK, Cromwell HC, Schultz W (2001) Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J Neurophysiol 85:2477–2489PubMedGoogle Scholar
  30. Herrnstein RJ (1961) Relative and absolute strength of response as a function of frequency of reinforcement. J Exp Anal Behav 4:267–272PubMedGoogle Scholar
  31. Herrnstein RJ (1997) The matching law: papers in psychology and economics. Harvard University Press, CambridgeGoogle Scholar
  32. Holroyd CB, Nieuwenhuis S, Yeung N, Cohen JD (2003) Errors in reward prediction are reflected in the event-related brain potential. Neuroreport 14:2481–2484CrossRefPubMedGoogle Scholar
  33. Holroyd CB, Nieuwenhuis S, Yeung N, Nystrom L, Mars RB, Coles MG, Cohen JD (2004) Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals. Nat Neurosci 7:497–498CrossRefPubMedGoogle Scholar
  34. Horvitz JC (2000) Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96:651–656CrossRefPubMedGoogle Scholar
  35. Horvitz JC (2002) Dopamine gating of glutamatergic sensorimotor and incentive motivational input signals to the striatum. Behav Brain Res 137:65–74CrossRefPubMedGoogle Scholar
  36. Ikeda T, Hikosaka O (2003) Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron 39:693–700CrossRefPubMedGoogle Scholar
  37. Kawagoe R, Takikawa Y, Hikosaka O (1998) Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1:411–416CrossRefPubMedGoogle Scholar
  38. Kim JN, Shadlen MN (1999) Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat Neurosci 2:176–185CrossRefPubMedGoogle Scholar
  39. Lauwereyns J, Watanabe K, Coe B, Hikosaka O (2002) A neural correlate of response bias in monkey caudate nucleus. Nature 418:413–417CrossRefPubMedGoogle Scholar
  40. Leon MI, Shadlen MN (1999) Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24:415–425CrossRefPubMedGoogle Scholar
  41. Mackintosh NJ (1975) Blocking of conditioned suppression: role of the first compound trial. J Exp Psychol Anim Behav Process 1:335–345CrossRefPubMedGoogle Scholar
  42. McCoy AN, Crowley JC, Haghighian G, Dean HL, Platt ML (2003) Saccade reward signals in posterior cingulate cortex. Neuron 40:1031–1040CrossRefPubMedGoogle Scholar
  43. Mirenowicz J, Schultz W (1996) Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379:449–451CrossRefPubMedGoogle Scholar
  44. Montague PR, Berns GS (2002) Neural economics and the biological substrates of valuation. Neuron 36:265–284CrossRefPubMedGoogle Scholar
  45. Morecraft RJ, Geula C, Mesulam MM (1993) Architecture of connectivity within a cingulo-fronto-parietal neurocognitive network for directed attention. Arch Neurol 50:279–284PubMedGoogle Scholar
  46. Niki H, Watanabe M (1979) Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res 171:213–224CrossRefPubMedGoogle Scholar
  47. O’Doherty J, Kringelbach ML, Rolls ET, Hornak J, Andrews C (2001) Abstract reward and punishment representations in the human orbitofrontal cortex. Nat Neurosci 4:95–102CrossRefPubMedGoogle Scholar
  48. Olson CR, Musil SY, Goldberg ME (1996) Single neurons in posterior cingulate cortex of behaving macaque: eye movement signals. J Neurophysiol 76:3285–3300PubMedGoogle Scholar
  49. Pagnoni G, Zink CF, Montague PR, Berns GS (2002) Activity in human ventral striatum locked to errors of reward prediction. Nat Neurosci 5:97–98CrossRefPubMedGoogle Scholar
  50. Pavlov IP (1927) Conditioned reflexes. Oxford University Press, LondonGoogle Scholar
  51. Pearce JM, Hall G (1980) A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87:532–552CrossRefPubMedGoogle Scholar
  52. Platt ML (2002) Neural correlates of decisions. Curr Opin Neurobiol 12(2):141–148CrossRefPubMedGoogle Scholar
  53. Platt ML, Glimcher PW (1997) Responses of intraparietal neurons to saccadic targets and visual distractors. J Neurophysiol 78:1574–1589PubMedGoogle Scholar
  54. Platt ML, Glimcher PW (1999) Neural correlates of decision variables in parietal cortex. Nature 400:233–238CrossRefPubMedGoogle Scholar
  55. Platt ML, Lau B, Glimcher PW (2004) Situating the Superior Colliculus within the Gaze Control Network. In: Hall WC, Moschovakis A (eds) The superior colliculus: new approaches for studying sensorimotor integration. CRC Press, New YorkGoogle Scholar
  56. Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning. Variations in the effectiveness of reinforcement and nonreinforcement. In: Black AHaP WF (eds) Classical conditioning. II. Current research and theory. Appleton-Century-Crofts, New YorkGoogle Scholar
  57. Roitman JD, Shadlen MN (2002) Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J Neurosci 22:9475–9489PubMedGoogle Scholar
  58. Rolls ET (2000) The orbitofrontal cortex and reward. Cereb Cortex 10:284–294CrossRefPubMedGoogle Scholar
  59. Salamone JD (1994) The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav Brain Res 61:117–133CrossRefPubMedGoogle Scholar
  60. Sato M, Hikosaka O (2002) Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement. J Neurosci 22:2363–2373PubMedGoogle Scholar
  61. Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1–27PubMedGoogle Scholar
  62. Schultz W (2004) Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology. Curr Opin Neurobiol 14:139–147CrossRefPubMedGoogle Scholar
  63. Schultz W, Dickinson A (2000) Neuronal coding of prediction errors. Annu Rev Neurosci 23:473–500CrossRefPubMedGoogle Scholar
  64. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599CrossRefPubMedGoogle Scholar
  65. Seymour B, O’Doherty JP, Dayan P, Koltzenburg M, Jones AK, Dolan RJ, Friston KJ, Frackowiak RS (2004) Temporal difference models describe higher-order learning in humans. Nature 429:664–667CrossRefPubMedGoogle Scholar
  66. Shadlen MN, Newsome WT (1996) Motion perception: seeing and deciding. Proc Natl Acad Sci U S A 93:628–633CrossRefPubMedGoogle Scholar
  67. Shadlen MN, Newsome WT (2001) Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86:1916–1936PubMedGoogle Scholar
  68. Sherrington CS (1906) The integrative action of the nervous system. Scribner, New YorkGoogle Scholar
  69. Shidara M, Richmond BJ (2002) Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296:1709–1711CrossRefPubMedGoogle Scholar
  70. Shima K, Tanji J (1998) Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282:1335–1338CrossRefPubMedGoogle Scholar
  71. Skinner BF (1981) Selection by consequences. Science 213:501–504PubMedGoogle Scholar
  72. Smith K, Dickhaut J, McCabe K, Pardo JV (2002) Neuronal substrates for choice under ambiquity, risk, gains, and losses. Mangement Science 48:711–718CrossRefGoogle Scholar
  73. Speiser D (ed) (1982) The works of Daniel Bernouilli. Birkhauser, BostonGoogle Scholar
  74. Stephens DW, Krebs JR (1986) Foraging theory. Princeton University Press, PrincetonGoogle Scholar
  75. Stuphorn V, Taylor TL, Schall JD (2000a) Performance monitoring by the supplementary eye field. Nature 408:857–860CrossRefPubMedGoogle Scholar
  76. Stuphorn V, Taylor TL, Schall JD (2000b) Performance monitoring by the supplementary eye field. Nature 408:857–860CrossRefPubMedGoogle Scholar
  77. Sutherland RJ, Whishaw IQ, Kolb B (1988) Contributions of cingulate cortex to two forms of spatial learning and memory. J Neurosci 8:1863–1872PubMedGoogle Scholar
  78. Sutton RS, Barto AG (1981) Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev 88:135–170CrossRefPubMedGoogle Scholar
  79. Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O (2002) Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res 142:284–291CrossRefPubMedGoogle Scholar
  80. Thorndike LL (1898) Animal intelligence: an experimental study of the asssociative processes in animals. Psychol Rev Monogr [Suppl 11]Google Scholar
  81. Tremblay L, Schultz W (1999) Relative reward preference in primate orbitofrontal cortex. Nature 398:704–708CrossRefPubMedGoogle Scholar
  82. Vogt BA, Pandya DN (1987) Cingulate cortex of the rhesus monkey. II. Cortical afferents. J Comp Neurol 262:271–289PubMedGoogle Scholar
  83. Vogt BA, Finch DM, Olson CR (1992) Functional heterogeneity in cingulate cortex: the anterior executive and posterior evaluative regions. Cereb Cortex 2:435–443PubMedGoogle Scholar
  84. Waelti, P, Dickinson, A, Schultz, W 2001Dopamine responses comply with basic assumptions of formal learning theoryNature41243CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag 2004

Authors and Affiliations

  1. 1.Department of NeurobiologyDuke University Medical CenterDurhamUSA
  2. 2.Center for Cognitive NeuroscienceDuke UniversityDurhamUSA
  3. 3.Department of Biological Anthropology and AnatomyDuke UniversityDurhamUSA

Personalised recommendations