Skip to main content
SpringerLink
Log in

Neural encoding of choice during a delayed response task in primate striatum and orbitofrontal cortex

  • Research Article
  • Published:
Experimental Brain Research Aims and scope Submit manuscript

Abstract

Reward outcomes are available in many diverse situations and all involve choice. If there are multiple outcomes each rewarding, then decisions regarding relative value lead to choosing one over another. Important factors related to choice context should be encoded and utilized for this form of adaptive choosing. These factors can include the number of alternatives, the pacing of choice behavior and the possibility to reverse one’s choice. An essential step in understanding if the context of choice is encoded is to directly compare choice with a context in which choice is absent. Neural activity in orbitofrontal cortex and striatum encodes potential value parameters related to reward quality and quantity as well as relative preference. We examined how neural activations in these brain regions are sensitive to choice situations and potentially involved in a prediction for the upcoming outcome selection. Neural activity was recorded and compared between a two-choice spatial delayed response task and an imperative ‘one-option’ task. Neural activity was obtained that extended from the instruction cue to the movement similar to previous work utilizing the identical imperative task. Orbitofrontal and striatal neural responses depended upon the decision about the choice of which reward to collect. Moreover, signals to predictive instruction cues that precede choice were selective for the choice situation. These neural responses could reflect chosen value with greater information on relative value of individual options as well as encode choice context itself embedded in the task as a part of the post-decision variable.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Apicella P, Ljungberg T, Scarnati E, Schultz W (1991) Responses to reward in monkey dorsal and ventral striatum. Exp Brain Res 85(3):491–500

    Article  PubMed  CAS  Google Scholar 

  • Belguermmi A, Bovet D, Pascal A, Prévot-Julliard A, Jalme M, Rat-Fischer L, Leboucher G (2011) Pigeons discriminate between human feeders. Anim Cogn 14(6):909–914. https://doi.org/10.1007/s10071-011-0420-7

    Article  Google Scholar 

  • Berridge KC, Cromwell HC (1990) Motivational-sensorimotor interaction controls aphagia and exaggerated treading after striatopallidal lesions. Behav Neurosci 104(5):778–795

    Article  PubMed  CAS  Google Scholar 

  • Berridge KC, Kringelbach ML (2013) Neuroscience of affect: brain mechanisms of pleasure and displeasure. Curr Opin Neurobiol 23(3):294–303

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Blanchard TC, Hayden BY (2015) Monkeys are more patient in a foraging task than in a standard intertemporal choice task. PLoS One (United States) 10:e0117057

    Article  CAS  Google Scholar 

  • Bouton ME, Todd TP, Miles OW, Leon SP, Epstein LH (2013) Within- and between-session variety effects in a food-seeking habituation paradigm. Appetite 66:10–19

    Article  PubMed  PubMed Central  Google Scholar 

  • Braun S, Hauber W (2011) The dorsomedial striatum mediates flexible choice behavior in spatial tasks. Behav Brain Res (Netherlands) 220:288–293

    Article  Google Scholar 

  • Brown VJ, Robbins TW (1991) Simple and choice reaction time performance following unilateral striatal dopamine depletion in the rat. Impaired motor readiness but preserved response preparation. Brain 114(Pt 1B):513–525

    Article  PubMed  Google Scholar 

  • Cai X, Kim S, Lee D (2011) Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron 69:170–182

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Cardinal RN, Howes NJ (2005) Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neurosci 6:37

    Article  PubMed  PubMed Central  Google Scholar 

  • Catania AC (1963) Concurrent performances: reinforcement interaction and response independence. J Exp Anal Beh 6:253–263

    Article  CAS  Google Scholar 

  • Catania AC, Savgolden T (1980) Preference for free choice over forced choice in pigeons. J Exp Anal Behav 34(1):77–86. https://doi.org/10.1901/jeab.1980.34-77

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Cromwell HC, Schultz W (2003) Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol 89(5):2823–2838

    Article  PubMed  Google Scholar 

  • Cromwell HC, Hassani OK, Schultz W (2005) Relative reward processing in primate striatum. Exp Brain Res 162(4):520–525

    Article  PubMed  Google Scholar 

  • Czoty PW, McCabe C, Nader MA (2005) Effects of the 5-HT(1A) agonist (+/-)-8-hydroxy-2-(di-n-propylamino)tetralin (8-OH-DPAT) on cocaine choice in cynomolgus monkeys. Behav Pharmacol (England) 16:187–191

    Article  CAS  Google Scholar 

  • Farashahi S, Donahue CH, Khorsand P, Seo H, Lee D, Soltani A (2017) Metaplasticity as a neural substrate for adaptive learning and choice under uncertainty. Neuron 94(2):401–414

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Fisher WW, Thompson RH, Piazza CC, Crosland K, Gotjen D (1997) On the relative reinforcing effects of choice and differential consequences. J Appl Behav Anal 30(3):423–438. https://doi.org/10.1901/jaba.1997.30-423

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Freidin E, Kacelnik A (2011) Rational choice, context dependence, and the value of information in European starlings (Sturnus vulgaris). Science 334(6058):1000–1002

    Article  CAS  Google Scholar 

  • Gourley SL, Olevska A, Zimmermann KS, Ressler KJ, Dileone RJ, Taylor JR (2013) The orbitofrontal cortex regulates outcome-based decision-making via the lateral striatum. Eur J Neurosci (France) 38:2382–2388

    Article  Google Scholar 

  • Grabenhorst F, Hernadi I, Schultz W (2016) Primate amygdala neurons evaluate the progress of self-defined economic choice sequences. Elife 5:e18731. https://doi.org/10.7554/eLife.18731

    Article  Google Scholar 

  • Hassani OK, Cromwell HC, Schultz W (2001) Influence of the expectation for different rewards onbehavior-related neuronal activity in primate striatum. J Neurophysiol 85:2477–2489

    Article  PubMed  CAS  Google Scholar 

  • Hernádi I, Grabenhorst F, Schultz W (2015) Planning activity for internally generated reward goals in monkey amygdala neurons. Nat Neurosci (United States) 18:461–469

    Article  CAS  Google Scholar 

  • Hwang J, Kim S, Lee D (2009) Temporal discounting and inter-temporal choice in rhesus monkeys. Front Behav Neurosci (Switzerland) 3:9

    Google Scholar 

  • Iyengar SS, Lepper MR (2000) When choice is demotivating: can one desire too much of a good thing? J Pers Soc Psychol 79(6):995–1006

    Article  PubMed  CAS  Google Scholar 

  • Kacelnik A, Bateson M (1996) Risky theories—the effects of variance on foraging decisions. Integr Comp Biol 36:402–434

    Google Scholar 

  • Kalivas PW, Volkow ND (2005) The neural basis of addiction: a pathology of motivation and choice. Am J Psychiatry (United States) 162:1403–1413

    Article  Google Scholar 

  • Keiflin R, Reese RM, Woods CA, Janak PH (2013) The orbitofrontal cortex as part of a hierarchical neural system mediating choice between two good options. J Neurosci (United States) 33:15989–15998

    Article  CAS  Google Scholar 

  • Kheramin S, Body S, Mobini S, Ho MY, Velazquez-Martinez DN, Bradshaw CM et al (2002) Effects of quinolinic acid-induced lesions of the orbital prefrontal cortex on inter-temporal choice: a quantitative analysis. Psychopharmacology (Berl) 165(1):9–17

    Article  CAS  Google Scholar 

  • Kheramin S, Body S, Ho M, Velazquez-Martinez DN, Bradshaw CM, Szabadi E et al (2003) Role of the orbital prefrontal cortex in choice between delayed and uncertain reinforcers: a quantitative analysis. Behav Process 64(3):239–250

    Article  CAS  Google Scholar 

  • Kheramin S, Body S, Herrera FM, Bradshaw CM, Szabadi E, Deakin JF et al (2005) The effect of orbital prefrontal cortex lesions on performance on a progressive ratio schedule: implications for models of inter-temporal choice. Behav Brain Res 156(1):145–152

    Article  PubMed  CAS  Google Scholar 

  • Kim H, Sul JH, Huh N, Lee D, Jung MW (2009) Role of striatum in updating values of chosen actions. J Neurosci (United States) 29:14701–14712

    Article  CAS  Google Scholar 

  • Klapp S, Abbott J, Coffman K, Greim D, Snider R, Young F (1979) Simple and choice reaction time methods in the study of motor programming. J Mot Behav 11(2):91–101

    Article  PubMed  CAS  Google Scholar 

  • Lak A, Stauffer WR, Schultz W (2016) Dopamine neurons learn relative chosen value from probabilistic rewards. Elife (England). https://doi.org/10.7554/eLife.18044

    Article  Google Scholar 

  • Lau B, Glimcher PW (2008) Value representations in the primate striatum during matching behavior. Neuron 58(3):451–463

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Leung BK, Balleine BW (2013) The ventral striato-pallidal pathway mediates the effect of predictive learning on choice between goal-directed actions. J Neurosci (United States) 33:13848–13860

    Article  CAS  Google Scholar 

  • Louie K, Glimcher PW (2012) Efficient coding and the neural representation of value. Ann N Y Acad Sci 1251:13–32. https://doi.org/10.1111/j.1749-6632.2012.06496.x

    Article  PubMed  Google Scholar 

  • Marcos E, Genovesio A (2016) Determining monkey free choice long before the choice is made: the principal role of prefrontal neurons involved in both decision and motor processes. Front Neural Circuits (Switzerland) 10:75

    Google Scholar 

  • McGraw JJ, Zona LC, Cromwell HC (2017) The effects of ethanol on diverse components of choice in the rat: reward discrimination, preference and relative valuation. Eur J Neurosci (France) 46(3):1837–1849. https://doi.org/10.1111/ejn.13627

    Article  Google Scholar 

  • Melville CL, Rue HC, Rybiski LR, Weatherly JN (1997) Altering reinforcer variety or intensity changes the within-session decrease in responding. Learn Motiv 28(4):609–621

    Article  Google Scholar 

  • Nader MA, Woolverton WL (1992a) Choice between cocaine and food by rhesus monkeys: effects of conditions of food availability. Behav Pharmacol 3(6):635–638

    Article  PubMed  CAS  Google Scholar 

  • Nader MA, Woolverton WL (1992b) Effects of increasing response requirement on choice between cocaine and food in rhesus monkeys. Psychopharmacology (Berl) 108(3):295–300

    Article  CAS  Google Scholar 

  • O’Neill M, Schultz W (2010) Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68(4):789–800

    Article  PubMed  CAS  Google Scholar 

  • Padoa-Schioppa C, Assad JA (2008) The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nat Neurosci 11(1):95–102

    Article  PubMed  CAS  Google Scholar 

  • Palmer CR, Kristan WB Jr (2011) Contextual modulation of behavioral choice. Curr Opin Neurobiol (England) 21:520–526

    Article  CAS  Google Scholar 

  • Pastor-Bernier A, Plott CR, Schultz W (2017) Monkeys choose as if maximizing utility compatible with basic principles of revealed preference theory. Proc Natl Acad Sci USA (United States) 114:E1766-E1775

    Google Scholar 

  • Rainwater A, Sanz E, Palmiter RD, Quintana A (2017) Striatal GPR88 modulates foraging efficiency. J Neurosci (United States) 37:7939–7947

    Article  CAS  Google Scholar 

  • Ramirez-Lugo L, Penas-Rincon A, Angeles-Duran S, Sotres-Bayon F (2016) Choice behavior guided by learned, but not innate, taste aversion recruits the orbitofrontal cortex. J Neurosci (United States) 36:10574–10583

    Article  CAS  Google Scholar 

  • Riceberg JS, Shapiro ML (2017) Orbitofrontal cortex signals expected outcomes with predictive codes when stable contingencies promote the integration of reward history. J Neurosci (United States) 37:2010–2021

    Article  CAS  Google Scholar 

  • Rich EL, Wallis JD (2016) Decoding subjective decisions from orbitofrontal cortex. Nat Neurosci (United States) 19:973–980

    Article  CAS  Google Scholar 

  • Ricker JM, Kopchock RJ 3rd, Drown RM, Cromwell HC (2016a) Effects of striatal lesions on components of choice: Reward discrimination, preference, and relative valuation. Behav Brain Res (Netherlands) 315:130–140

    Article  Google Scholar 

  • Ricker JM, Hatch JD, Powers DD, Cromwell HC (2016b) Fractionating choice: a study on reward discrimination, preference, and relative valuation in the rat (rattus norvegicus). J Comp Psychol (United States) 130:174–186

    Article  Google Scholar 

  • Robbins TW (2002) The 5-choice serial reaction time task: behavioural pharmacology and functional neurochemistry. Psychopharmacol (Berl) 163(3–4):362–380

    Article  CAS  Google Scholar 

  • Rolls BJ, Rowe EA, Rolls ET, Kingston B, Megson A, Gunary R (1981) Variety in a meal enhances food intake in man. Physiol Behav 26(2):215–221

    Article  PubMed  CAS  Google Scholar 

  • Schultz W (2004) Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology. Curr Opin Neurobiol 14(2):139–147

    Article  PubMed  CAS  Google Scholar 

  • Schultz W (2010) Subjective neuronal coding of reward: temporal value discounting and risk. Eur J Neurosci 31(12):2124–2135

    Article  PubMed  Google Scholar 

  • Schultz W (2011) Potential vulnerabilities of neuronal reward, risk, and decision mechanisms to addictive drugs. Neuron 69(4):603–617

    Article  PubMed  CAS  Google Scholar 

  • Schultz W (2015) Neuronal reward and decision signals: From theories to data. Physiol Rev (United States) 95:853–951

    CAS  Google Scholar 

  • Schultz W (2017) Reward prediction error. Curr Biol 27(10):R369–R371. https://doi.org/10.1016/j.cub.2017.02.064

    Article  PubMed  CAS  Google Scholar 

  • Seymour B, McClure SM (2008) Anchors, scales and the relative coding of value in the brain. Curr Opin Neurobiol (England) 18:173–178

    Article  CAS  Google Scholar 

  • Shizgal P (1997) Neural basis of utility estimation. Curr Opin Neurobiol 7(2):198–208

    Article  PubMed  CAS  Google Scholar 

  • St Onge JR, Floresco SB (2010) Prefrontal cortical contribution to risk-based decision making. Cereb Cortex 20(8):1816–1828

    Article  PubMed  Google Scholar 

  • Stauffer WR, Lak A, Schultz W (2014) Dopamine reward prediction error responses reflect marginal utility. Curr Biol (England) 24:2491–2500

    Article  CAS  Google Scholar 

  • Sternberg S, Monsell S, Knoll RL, Wright CE (1978) The latency and duration of rapid movement sequences: comparisons of speech and typing. In: Stelmach GE (ed) Information processing in motor control and learning. Academic Press, New York, pp 117–152

    Chapter  Google Scholar 

  • Strait CE, Sleezer BJ, Hayden BY (2015) Signatures of value comparison in ventral striatum neurons. PLoS Biol (United States) 13:e1002173

    Article  CAS  Google Scholar 

  • Strait CE, Sleezer BJ, Blanchard TC, Azab H, Castagno MD, Hayden BY (2016) Neuronal selectivity for spatial positions of offers and choices in five reward regions. J Neurophysiol 115(3):1098–1111. https://doi.org/10.1152/jn.00325.2015

    Article  PubMed  Google Scholar 

  • Suzuki S (1999) Selection of forced- and free-choice by monkeys (macaca fascicularis). Percept Mot Skills 88:242–250. https://doi.org/10.2466/pms.1999.88.1.242

    Article  Google Scholar 

  • Treit D, Spetch ML, Deutsch JA (1983) Variety in the flavor of food enhances eating in the rat: a controlled demonstration. Physiol Behav 30(2):207–211

    Article  PubMed  CAS  Google Scholar 

  • Tremblay L, Schultz W (1999) Relative reward preference in primate orbitofrontal cortex. Nature (England) 398:704–708

    Article  CAS  Google Scholar 

  • Tsujimoto S, Genovesio A, Wise SP (2009) Monkey orbitofrontal cortex encodes response choices near feedback time. J Neurosci 29(8):2569–2574. https://doi.org/10.1523/JNEUROSCI.5777-08.2009

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Tsutsui K, Grabenhorst F, Kobayashi S, Schultz W (2016) A dynamic code for economic object valuation in prefrontal cortex neurons. Nat Commun (England) 7:12554

    Article  CAS  Google Scholar 

  • Tversky A, Shafir E (1992) Choice under conflict: the dynamics of deferred decision. Psychol Sci 3:358–361

    Article  Google Scholar 

  • van Wingerden M, van der Meij R, Kalenscher T, Maris E, Pennartz CM (2014) Phase-amplitude coupling in rat orbitofrontal cortex discriminates between correct and incorrect decisions during associative learning. J Neurosci (United States) 34:493–505

    Article  CAS  Google Scholar 

  • Watanabe M, Cromwell HC, Tremblay L, Hollerman JR, Hikosaka K, Schultz W (2001) Behavioral reactions reflecting differential reward expectations in monkeys. Exp Brain Res 140(4):511–518

    Article  PubMed  CAS  Google Scholar 

  • Webber ES, Chambers NE, Kostek JA, Mankin DE, Cromwell HC (2015) Relative reward effects on operant behavior: Incentive contrast, induction and variety effects. Behav Process (Netherlands) 116:87–99

    Article  CAS  Google Scholar 

  • Webber ES, Mankin DE, Cromwell HC (2016) Striatal activity and reward relativity: neural signals encoding dynamic outcome valuation. Eneuro 3:ENEURO.0022-16.2016

  • Williams BA (2002) Behavioral contrast redux. Animal Learning Behavior 30:1–20

    Article  PubMed  Google Scholar 

  • Yang X, Gao M, Shi J, Ye H, Chen S (2017) Modulating the activity of the DLPFC and OFC has distinct effects on risk and ambiguity decision-making: a tDCS study. Front Psychol 8:1417. https://doi.org/10.3389/fpsyg.2017.01417

    Article  PubMed  PubMed Central  Google Scholar 

  • Zona LC, Fry BR, Cromwell HC (2017) Effects of anandamide administration on components of reward processing during free choice. Pharmacol Biochem Behav 158:14–21

    Article  PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, JP Scott Center for Neuroscience, Mind and Behavior, Bowling Green State University, Bowling Green, OH, 43403, USA

    Howard C. Cromwell

  2. Centre de Neuroscience Cognitive, UMR-5229 CNRS, Bron, Cedex, France

    Leon Tremblay

  3. Université Claude-Bernard Lyon 1, 69100, Villeurbanne, France

    Leon Tremblay

  4. Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, CB2 3DY, UK

    Wolfram Schultz

Authors
  1. Howard C. Cromwell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leon Tremblay
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wolfram Schultz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Howard C. Cromwell.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cromwell, H.C., Tremblay, L. & Schultz, W. Neural encoding of choice during a delayed response task in primate striatum and orbitofrontal cortex. Exp Brain Res 236, 1679–1688 (2018). https://doi.org/10.1007/s00221-018-5253-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-018-5253-z

Keywords

Search

Navigation