Skip to main content

Expectations and outcomes: decision-making in the primate brain

Journal of Comparative Physiology A volume 191, pages 201–211 (2005)Cite this article

Abstract

Success in a constantly changing environment requires that decision-making strategies be updated as reward contingencies change. How this is accomplished by the nervous system has, until recently, remained a profound mystery. New studies coupling economic theory with neurophysiological techniques have revealed the explicit representation of behavioral value. Specifically, when fluid reinforcement is paired with visually-guided eye movements, neurons in parietal cortex, prefrontal cortex, the basal ganglia, and superior colliculus—all nodes in a network linking visual stimulation with the generation of oculomotor behavior—encode the expected value of targets lying within their response fields. Other brain areas have been implicated in the processing of reward-related information in the abstract: midbrain dopaminergic neurons, for instance, signal an error in reward prediction. Still other brain areas link information about reward to the selection and performance of specific actions in order for behavior to adapt to changing environmental exigencies. Neurons in posterior cingulate cortex have been shown to carry signals related to both reward outcomes and oculomotor behavior, suggesting that they participate in updating estimates of orienting value.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2a–c
Fig. 3a, b
Fig. 4a–d
Fig. 5a, b

References

  • Amador N, Schlag-Rey M, Schlag J (2000) Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. J Neurophysiol 84:2166–2170

    CAS  PubMed  Google Scholar 

  • Arnauld A, Nichole P (1662) The art of thinking: Port-Royal logic. Translated by Dickoff J, James P. Bobbs-Merrill, Indianapolis

    Google Scholar 

  • Baleydier C, Mauguiere F (1980) The duality of the cingulate gyrus in monkey. Neuroanatomical study and functional hypothesis. Brain 103(3):525–554

    CAS  PubMed  Google Scholar 

  • Basso MA, Wurtz RH (1997) Modulation of neuronal activity by target uncertainty. Nature 389:66–69

    Article  CAS  PubMed  Google Scholar 

  • Berns GS, McClure SM, Pagnoni G, Montague PR (2001) Predictability modulates human brain response to reward. J Neurosci 21:2793–2798

    CAS  PubMed  Google Scholar 

  • Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P (2001) Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30:619–639

    Article  CAS  PubMed  Google Scholar 

  • Bussey TJ, Everitt BJ, Robbins TW (1997) Dissociable effects of cingulate and medial frontal cortex lesions on stimulus-reward learning using a novel Pavlovian autoshaping procedure for the rat: implications for the neurobiology of emotion. Behav Neurosci 111:908–919

    Article  CAS  PubMed  Google Scholar 

  • Camerer CF (2003) Behavioral game theory: experiments in strategic interaction (Roundtable Series in Behavioral Economics). Princeton University Press, Princeton

    Google Scholar 

  • Cavada C, Goldman-Rakic PS (1989) Posterior parietal cortex in rhesus monkey: I. Parcellation of areas based on distinctive limbic and sensory corticocortical connections. J Comp Neurol 287:393–421

    CAS  PubMed  Google Scholar 

  • Cavada C, Company T, Tejedor J, Cruz-Rizzolo RJ, Reinoso-Suarez F (2000) The anatomical connections of the macaque monkey orbitofrontal cortex. A review. Cereb Cortex 10:220–242

    Article  CAS  PubMed  Google Scholar 

  • Coe B, Tomihara K, Matsuzawa M, Hikosaka O (2002) Visual and anticipatory bias in three cortical eye fields of the monkey during an adaptive decision-making task. J Neurosci 22(12):5081–5090

    CAS  PubMed  Google Scholar 

  • Colby CL, Duhamel JR, Goldberg ME (1996) Visual, presaccadic, and cognitive activation of single neurons in monkey lateral intraparietal area. J Neurophysiol 76:2841–2852

    CAS  PubMed  Google Scholar 

  • Critchley HD, Mathias CJ, Dolan RJ (2001) Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron 29:537–545

    Article  CAS  PubMed  Google Scholar 

  • Cromwell HC, Schultz W (2003) Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol 29:29

    Google Scholar 

  • Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA (2000) Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84:3072–3077

    CAS  PubMed  Google Scholar 

  • Delgado MR, Locke HM, Stenger VA, Fiez JA (2003) Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations. Cogn Affect Behav Neurosci 3:27–38

    CAS  PubMed  Google Scholar 

  • Dorris MC, Munoz DP (1998) Saccadic probability influences motor preparation signals and time to saccadic initiation. J Neurosci 18:7015–7026

    CAS  PubMed  Google Scholar 

  • Elliott R, Newman JL, Longe OA, Deakin JF (2003) Differential response patterns in the striatum and orbitofrontal cortex to financial reward in humans: a parametric functional magnetic resonance imaging study. J Neurosci 23(1):303–307

    CAS  PubMed  Google Scholar 

  • Gabriel M, Sparenborg S (1987) Posterior cingulate cortical lesions eliminate learning-related unit activity in the anterior cingulate cortex. Brain Res 409:151–157

    Article  CAS  PubMed  Google Scholar 

  • Gabriel M, Orona E, Foster K, Lambert RW (1980) Cingulate cortical and anterior thalamic neuronal correlates of reversal learning in rabbits. J Comp Physiol Psychol 94:1087–1100

    CAS  PubMed  Google Scholar 

  • Glimcher P (2002) Decisions, decisions, decisions: choosing a biological science of choice. Neuron 36:323–332

    Article  CAS  PubMed  Google Scholar 

  • Glimcher P (2003) Decisions, Uncertainty, and the brain: the science of neuroeconomics. MIT Press, Cambridge

    Google Scholar 

  • Gnadt JW, Andersen RA (1988) Memory related motor planning activity in posterior parietal cortex of macaque. Exp Brain Res 70:216–220

    CAS  PubMed  Google Scholar 

  • Gold JI, Shadlen MN (2002) Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36:299–308

    Article  CAS  PubMed  Google Scholar 

  • Goldberg ME, Colby CL, Duhamel JR (1990) Representation of visuomotor space in the parietal lobe of the monkey. Cold Spring Harb Symp Quant Biol 55:729–739

    CAS  PubMed  Google Scholar 

  • Gottlieb JP, Kusunoki M, Goldberg ME (1998) The representation of visual salience in monkey parietal cortex. Nature 391:481–484

    Article  CAS  PubMed  Google Scholar 

  • Hanes DP, Schall JD (1996) Neural control of voluntary movement initiation. Science 274:427–430

    Article  CAS  PubMed  Google Scholar 

  • Harker KT, Whishaw IQ (2002) Impaired spatial performance in rats with retrosplenial lesions: importance of the spatial problem and the rat strain in identifying lesion effects in a swimming pool. J Neurosci 22:1155–1164

    CAS  PubMed  Google Scholar 

  • Hassani OK, Cromwell HC, Schultz W (2001) Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J Neurophysiol 85:2477–2489

    CAS  PubMed  Google Scholar 

  • Herrnstein RJ (1961) Relative and absolute strength of response as a function of frequency of reinforcement. J Exp Anal Behav 4:267–272

    CAS  PubMed  Google Scholar 

  • Herrnstein RJ (1997) The matching law: papers in psychology and economics. Harvard University Press, Cambridge

    Google Scholar 

  • Holroyd CB, Nieuwenhuis S, Yeung N, Cohen JD (2003) Errors in reward prediction are reflected in the event-related brain potential. Neuroreport 14:2481–2484

    Article  PubMed  Google Scholar 

  • Holroyd CB, Nieuwenhuis S, Yeung N, Nystrom L, Mars RB, Coles MG, Cohen JD (2004) Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals. Nat Neurosci 7:497–498

    Article  CAS  PubMed  Google Scholar 

  • Horvitz JC (2000) Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96:651–656

    Article  CAS  PubMed  Google Scholar 

  • Horvitz JC (2002) Dopamine gating of glutamatergic sensorimotor and incentive motivational input signals to the striatum. Behav Brain Res 137:65–74

    Article  CAS  PubMed  Google Scholar 

  • Ikeda T, Hikosaka O (2003) Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron 39:693–700

    Article  CAS  PubMed  Google Scholar 

  • Kawagoe R, Takikawa Y, Hikosaka O (1998) Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1:411–416

    Article  CAS  PubMed  Google Scholar 

  • Kim JN, Shadlen MN (1999) Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat Neurosci 2:176–185

    Article  PubMed  Google Scholar 

  • Lauwereyns J, Watanabe K, Coe B, Hikosaka O (2002) A neural correlate of response bias in monkey caudate nucleus. Nature 418:413–417

    Article  CAS  PubMed  Google Scholar 

  • Leon MI, Shadlen MN (1999) Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24:415–425

    Article  CAS  PubMed  Google Scholar 

  • Mackintosh NJ (1975) Blocking of conditioned suppression: role of the first compound trial. J Exp Psychol Anim Behav Process 1:335–345

    Article  CAS  PubMed  Google Scholar 

  • McCoy AN, Crowley JC, Haghighian G, Dean HL, Platt ML (2003) Saccade reward signals in posterior cingulate cortex. Neuron 40:1031–1040

    Article  CAS  PubMed  Google Scholar 

  • Mirenowicz J, Schultz W (1996) Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379:449–451

    Article  CAS  PubMed  Google Scholar 

  • Montague PR, Berns GS (2002) Neural economics and the biological substrates of valuation. Neuron 36:265–284

    Article  CAS  PubMed  Google Scholar 

  • Morecraft RJ, Geula C, Mesulam MM (1993) Architecture of connectivity within a cingulo-fronto-parietal neurocognitive network for directed attention. Arch Neurol 50:279–284

    CAS  PubMed  Google Scholar 

  • Niki H, Watanabe M (1979) Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res 171:213–224

    Article  CAS  PubMed  Google Scholar 

  • O’Doherty J, Kringelbach ML, Rolls ET, Hornak J, Andrews C (2001) Abstract reward and punishment representations in the human orbitofrontal cortex. Nat Neurosci 4:95–102

    Article  CAS  PubMed  Google Scholar 

  • Olson CR, Musil SY, Goldberg ME (1996) Single neurons in posterior cingulate cortex of behaving macaque: eye movement signals. J Neurophysiol 76:3285–3300

    CAS  PubMed  Google Scholar 

  • Pagnoni G, Zink CF, Montague PR, Berns GS (2002) Activity in human ventral striatum locked to errors of reward prediction. Nat Neurosci 5:97–98

    Article  CAS  PubMed  Google Scholar 

  • Pavlov IP (1927) Conditioned reflexes. Oxford University Press, London

    Google Scholar 

  • Pearce JM, Hall G (1980) A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87:532–552

    Article  CAS  PubMed  Google Scholar 

  • Platt ML (2002) Neural correlates of decisions. Curr Opin Neurobiol 12(2):141–148

    Article  CAS  PubMed  Google Scholar 

  • Platt ML, Glimcher PW (1997) Responses of intraparietal neurons to saccadic targets and visual distractors. J Neurophysiol 78:1574–1589

    CAS  PubMed  Google Scholar 

  • Platt ML, Glimcher PW (1999) Neural correlates of decision variables in parietal cortex. Nature 400:233–238

    Article  CAS  PubMed  Google Scholar 

  • Platt ML, Lau B, Glimcher PW (2004) Situating the Superior Colliculus within the Gaze Control Network. In: Hall WC, Moschovakis A (eds) The superior colliculus: new approaches for studying sensorimotor integration. CRC Press, New York

  • Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning. Variations in the effectiveness of reinforcement and nonreinforcement. In: Black AHaP WF (eds) Classical conditioning. II. Current research and theory. Appleton-Century-Crofts, New York

  • Roitman JD, Shadlen MN (2002) Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J Neurosci 22:9475–9489

    CAS  PubMed  Google Scholar 

  • Rolls ET (2000) The orbitofrontal cortex and reward. Cereb Cortex 10:284–294

    Article  CAS  PubMed  Google Scholar 

  • Salamone JD (1994) The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav Brain Res 61:117–133

    Article  CAS  PubMed  Google Scholar 

  • Sato M, Hikosaka O (2002) Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement. J Neurosci 22:2363–2373

    CAS  PubMed  Google Scholar 

  • Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1–27

    CAS  PubMed  Google Scholar 

  • Schultz W (2004) Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology. Curr Opin Neurobiol 14:139–147

    Article  CAS  PubMed  Google Scholar 

  • Schultz W, Dickinson A (2000) Neuronal coding of prediction errors. Annu Rev Neurosci 23:473–500

    Article  CAS  PubMed  Google Scholar 

  • Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599

    Article  CAS  PubMed  Google Scholar 

  • Seymour B, O’Doherty JP, Dayan P, Koltzenburg M, Jones AK, Dolan RJ, Friston KJ, Frackowiak RS (2004) Temporal difference models describe higher-order learning in humans. Nature 429:664–667

    Article  CAS  PubMed  Google Scholar 

  • Shadlen MN, Newsome WT (1996) Motion perception: seeing and deciding. Proc Natl Acad Sci U S A 93:628–633

    Article  CAS  PubMed  Google Scholar 

  • Shadlen MN, Newsome WT (2001) Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86:1916–1936

    CAS  PubMed  Google Scholar 

  • Sherrington CS (1906) The integrative action of the nervous system. Scribner, New York

    Google Scholar 

  • Shidara M, Richmond BJ (2002) Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296:1709–1711

    Article  PubMed  Google Scholar 

  • Shima K, Tanji J (1998) Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282:1335–1338

    Article  CAS  PubMed  Google Scholar 

  • Skinner BF (1981) Selection by consequences. Science 213:501–504

    CAS  PubMed  Google Scholar 

  • Smith K, Dickhaut J, McCabe K, Pardo JV (2002) Neuronal substrates for choice under ambiquity, risk, gains, and losses. Mangement Science 48:711–718

    Article  Google Scholar 

  • Speiser D (ed) (1982) The works of Daniel Bernouilli. Birkhauser, Boston

  • Stephens DW, Krebs JR (1986) Foraging theory. Princeton University Press, Princeton

    Google Scholar 

  • Stuphorn V, Taylor TL, Schall JD (2000a) Performance monitoring by the supplementary eye field. Nature 408:857–860

    Article  CAS  PubMed  Google Scholar 

  • Stuphorn V, Taylor TL, Schall JD (2000b) Performance monitoring by the supplementary eye field. Nature 408:857–860

    Article  CAS  PubMed  Google Scholar 

  • Sutherland RJ, Whishaw IQ, Kolb B (1988) Contributions of cingulate cortex to two forms of spatial learning and memory. J Neurosci 8:1863–1872

    CAS  PubMed  Google Scholar 

  • Sutton RS, Barto AG (1981) Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev 88:135–170

    Article  CAS  PubMed  Google Scholar 

  • Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O (2002) Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res 142:284–291

    Article  PubMed  Google Scholar 

  • Thorndike LL (1898) Animal intelligence: an experimental study of the asssociative processes in animals. Psychol Rev Monogr [Suppl 11]

    Google Scholar 

  • Tremblay L, Schultz W (1999) Relative reward preference in primate orbitofrontal cortex. Nature 398:704–708

    Article  CAS  PubMed  Google Scholar 

  • Vogt BA, Pandya DN (1987) Cingulate cortex of the rhesus monkey. II. Cortical afferents. J Comp Neurol 262:271–289

    CAS  PubMed  Google Scholar 

  • Vogt BA, Finch DM, Olson CR (1992) Functional heterogeneity in cingulate cortex: the anterior executive and posterior evaluative regions. Cereb Cortex 2:435–443

    CAS  PubMed  Google Scholar 

  • P Waelti A Dickinson W Schultz (2001) ArticleTitleDopamine responses comply with basic assumptions of formal learning theory Nature 412 IssueID6842 43 Occurrence Handle10.1038/35083500 Occurrence Handle1:CAS:528:DC%2BD3MXlt1CrsLo%3D Occurrence Handle11452299

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Neurobiology, Duke University Medical Center, 325 Bryan Research Building, Box 3209, Durham, NC, 27710, USA

    Allison N. McCoy & Michael L. Platt

  2. Center for Cognitive Neuroscience, Duke University, Durham, NC, 27710, USA

    Michael L. Platt

  3. Department of Biological Anthropology and Anatomy, Duke University, Durham, NC, 27710, USA

    Michael L. Platt

Authors
  1. Allison N. McCoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael L. Platt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael L. Platt.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

McCoy, A.N., Platt, M.L. Expectations and outcomes: decision-making in the primate brain. J Comp Physiol A 191, 201–211 (2005). https://doi.org/10.1007/s00359-004-0565-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00359-004-0565-9

Keywords

  • Anterior Cingulate Cortex
  • Superior Colliculus
  • Dopamine Neuron
  • Orbitofrontal Cortex
  • Ventral Striatum