Abstract
Many brain areas are activated by the possibility and receipt of reward. Are all of these brain areas reporting the same information about reward? Or are these signals related to other functions that accompany reward-guided learning and decision-making? Through carefully controlled behavioral studies, it has been shown that reward-related activity can represent reward expectations related to future outcomes, errors in those expectations, motivation, and signals related to goal- and habit-driven behaviors. These dissociations have been accomplished by manipulating the predictability of positively and negatively valued events. Here, we review single neuron recordings in behaving animals that have addressed this issue. We describe data showing that several brain areas, including orbitofrontal cortex, anterior cingulate, and basolateral amygdala signal reward prediction. In addition, anterior cingulate, basolateral amygdala, and dopamine neurons also signal errors in reward prediction, but in different ways. For these areas, we will describe how unexpected manipulations of positive and negative value can dissociate signed from unsigned reward prediction errors. All of these signals feed into striatum to modify signals that motivate behavior in ventral striatum and guide responding via associative encoding in dorsolateral striatum.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ambroggi F, Ishikawa A, Fields HL, Nicola SM (2008) Basolateral amygdala neurons facilitate reward-seeking behavior by exciting nucleus accumbens neurons. Neuron 59:648–661
Amiez C, Joseph JP, Procyk E (2005) Anterior cingulate error-related activity is modulated by predicted reward. Eur J Neurosci 21:3447–3452
Amiez C, Joseph JP, Procyk E (2006) Reward encoding in the monkey anterior cingulate cortex. Cereb Cortex 16:1040–1055
Anderson AK (2005) Affective influences on the attentional dynamics supporting awareness. J Exp Psychol Gen 134:258–281
Anderson BA, Laurent PA, Yantis S (2011) Value-driven attentional capture. Proc Natl Acad Sci USA 108:10367–10371
Anstrom KK, Miczek KA, Budygin EA (2009) Increased phasic dopamine signaling in the mesolimbic pathway during social defeat in rats. Neuroscience 161:3–12
Apicella P, Ljungberg T, Scarnati E, Schultz W (1991) Responses to reward in monkey dorsal and ventral striatum. Exp Brain Res. Experimentelle Hirnforschung. Experimentation cerebrale 85:491–500
Balleine BW, O’Doherty JP (2010) Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacol: Official Publ Am Coll Neuropsychopharmacol 35:48–69
Balleine BW, Daw ND, O’Doherty JP (2008) Multiple forms of value learning and the function of dopamine. In: Glimcher PW, Camerer CF, Fehr E, Poldrack RA (eds) Neuroeconomics: decision making and the brain. Elsevier, Amsterdam
Barto A (ed) (1995) Adaptive critics and the basal ganglia. MIT Press, Cambridge
Bechara A, Damasio H, Damasio AR, Lee GP (1999) Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. J Neurosci 19:5473–5481
Belova MA, Paton JJ, Morrison SE, Salzman CD (2007) Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala. Neuron 55:970–984
Belova MA, Patton JJ, Salzman CD (2008) Moment-to-moment tracking of state value in the amygdala. J Neurosci 28:10023–10030
Berridge KC (2007) The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacology 191:391–431
Berridge KC, Robinson TE (1998) What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Brain Res Rev 28:309–369
Bissonette GB et al (2013) Separate populations of neurons in ventral striatum encode value and motivation. PLoS ONE 8:e64673
Bissonette GB, Gentry RN, Padmala S, Pessoa L, Roesch MR (2014) Impact of appetitive and aversive outcomes on brain responses: linking the animal and human literatures. Frontiers Syst Neurosci 8:24
Blokland A (1998) Reaction time responding in rats. Neurosci Biobehav Rev 22:847–864
Brinschwitz K et al (2010) Glutamatergic axons from the lateral habenula mainly terminate on GABAergic neurons of the ventral midbrain. Neuroscience 168:463–476
Brischoux F, Chakraborty S, Brierley DI, Ungless MA (2009) Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci USA 106:4894–4899
Bromberg-Martin ES, Matsumoto M, Hikosaka O (2010a) Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68:815–834
Bromberg-Martin ES, Matsumoto M, Hikosaka O (2010b) Distinct tonic and phasic anticipatory activity in lateral habenula and dopamine neurons. Neuron 67:144–155
Bryden DW, Johnson EE, Tobia SC, Kashtelyan V, Roesch MR (2011a) Attention for learning signals in anterior cingulate cortex. J Neurosci: Official J Soc Neurosci 31:18266–18274
Bryden DW, Johnson EE, Diao X, Roesch MR (2011b) Impact of expected value on neural activity in rat substantia nigra pars reticulata. Eur J Neurosci 33:2308–2317
Burke KA, Franz TM, Miller DN, Schoenbaum G (2008) The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards. Nature 454:340–344
Burton AC, Bissonette GB, Lichtenberg NT, Kashtelyan V, Roesch MR (2013) Ventral striatum lesions enhance stimulus and response encoding in dorsal striatum. Biol Psychiatry
Burton AC, Bissonette GB, Lichtenberg NT, Kashtelyan V, Roesch MR (2014) Ventral striatum lesions enhance stimulus and response encoding in dorsal striatum. Biol Psychiatry 75:132–139
Burton AC, Nakamura K, Roesch MR (2015) From ventral-medial to dorsal-lateral striatum: neural correlates of reward-guided decision-making. Neurobiol Learn Mem 117:51–59
Cai X, Kim S, Lee D (2011) Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron 69:170–182
Cardinal RN (2006) Neural systems implicated in delayed and probabilistic reinforcement. Neural Netw 19:1277–1301
Cardinal RN, Pennicott DR, Sugathapala CL, Robbins TW, Everitt BJ (2001) Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science 292:2499–2501
Cardinal RN, Parkinson JA, Hall J, Everitt BJ (2002a) Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev 26:321–352
Cardinal RN et al (2002b) Effects of selective excitotoxic lesions of the nucleus accumbens core, anterior cingulate cortex, and central nucleus of the amygdala on autoshaping performance in rats. Behav Neurosci 116:553–567
Carelli RM, Deadwyler SA (1994) A comparison of nucleus accumbens neuronal firing patterns during cocaine self-administration and water reinforcement in rats. J Neurosci 14:7735–7746
Carter CS et al (1998) Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280:747–749
Cassell MD, Wright DJ (1986) Topography of projections from the medial prefrontal cortex to the amygdala in the rat. Brain Res Bull 17:321–333
Catanese J, van der Meer M (2013) A network state linking motivation and action in the nucleus accumbens. Neuron 78:753–754
Churchwell JC, Morris AM, Heurtelou NM, Kesner RP (2009) Interactions between the prefrontal cortex and amygdala during delay discounting and reversal. Behav Neurosci 123:1185–1196
Cousens GA, Otto T (2003) Neural substrates of olfactory discrimination learning with auditory secondary reinforcement. I. Contributions of the basolateral amygdaloid complex and orbitofrontal cortex. Integr Physiol Behav Sci 38:272–294
Cromwell HC, Hassani OK, Schultz W (2005) Relative reward processing in primate striatum. Exp Brain Res. Experimentelle Hirnforschung. Experimentation cerebrale 162:520–525
Daw ND, Kakade S, Dayan P (2002) Opponent interactions between serotonin and dopamine. Neural Networks: Official J Int Neural Network Soc 15:603–616
Daw ND, Niv Y, Dayan P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8:1704–1711
Day JJ, Jones JL, Carelli RM (2011) Nucleus accumbens neurons encode predicted and ongoing reward costs in rats. Eur J Neurosci 33:308–321
Di Chiara G (2002) Nucleus accumbens shell and core dopamine: differential role in behavior and addiction. Behav Brain Res 137:75–114
Di Ciano P, Cardinal RN, Cowell RA, Little SJ, Everitt BJ (2001) Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approach behavior. J Neurosci 21:9471–9477
Dziewiatkowski J et al (1998) The projection of the amygdaloid nuclei to various areas of the limbic cortex in the rat. Folia Morphol (Warsz) 57:301–308
Floresco SB, St Onge JR, Ghods-Sharifi S, Winstanley CA (2008) Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making. Cogn Affect Behav Neurosci 8:375–389
Gallagher M (2000) The amygdala and associative learning. In: Aggleton JP (ed) The amygdala: a functional analysis. Oxford University Press, Oxford, pp 311–330
Ghods-Sharifi S, Floresco SB (2010) Differential effects on effort discounting induced by inactivations of the nucleus accumbens core or shell. Behav Neurosci 124:179–191
Ghods-Sharifi S., St Onge, J.R. & Floresco, S.B. Fundamental contribution by the basolateral amygdala to different forms of decision making. J Neurosci 29, 5251-5259 (2009)
Giertler C, Bohn I, Hauber W (2003) The rat nucleus accumbens is involved in guiding of instrumental responses by stimuli predicting reward magnitude. Eur J Neurosci 18:1993–1996
Gold JI, Shadlen MN (2007) The neural basis of decision making. Annu Rev Neurosci 30:535–574
Goldstein BL et al (2012) Ventral striatum encodes past and predicted value independent of motor contingencies. J Neurosci: Official J Soc Neurosci 32:2027–2036
Gruber AJ, Hussain RJ, O’Donnell P (2009) The nucleus accumbens: a switchboard for goal-directed behaviors. PLoS ONE 4:e5062
Gruber AJ et al (2010) More is less: a disinhibited prefrontal cortex impairs cognitive flexibility. J Neurosci 30:17102–17110
Haber SN (2003) The primate basal ganglia: parallel and integrative networks. J Chem Neuroanat 26:317–330
Haber SN, Knutson B (2010) The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacol: Official Publ Am Coll Neuropsychopharmacol 35:4–26
Haber SN, Fudge JL, McFarland NR (2000) Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 20:2369–2382
Haney RZ, Calu DJ, Takahashi YK, Hughes BW, Schoenbaum G (2010) Inactivation of the central but not the basolateral nucleus of the amygdala disrupts learning in response to overexpectation of reward. J Neurosci 30:2911–2917
Hatfield T, Han JS, Conley M, Gallagher M, Holland P (1996) Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J Neurosci 16:5256–5265
Hayden BY, Heilbronner SR, Pearson JM, Platt ML (2011) Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J Neurosci 31:4178–4187
Hernandez A, Burton AC, O’Donnell P, Schoenbaum G, Roesch MR (2015) Altered basolateral amygdala encoding in an animal model of schizophrenia. J Neurosci 35:6394–6400
Hikosaka O (2010) The habenula: from stress evasion to value-based decision-making. Nat Rev Neurosci 11:503–513
Hillman KL, Bilkey DK (2010) Neurons in the rat anterior cingulate cortex dynamically encode cost-benefit in a spatial decision-making task. J Neurosci 30:7705–7713
Hollerman JR, Tremblay L, Schultz W (1998) Influence of reward expectation on behavior-related neuronal activity in primate striatum. J Neurophysiol 80:947–963
Holroyd CB, Coles MG (2002) The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol Rev 109:679–709
Hong S, Hikosaka O (2008) The globus pallidus sends reward-related signals to the lateral habenula. Neuron 60:720–729
Hong S, Jhou TC, Smith M, Saleem KS, Hikosaka O (2011) Negative reward signals from the lateral habenula to dopamine neurons are mediated by rostromedial tegmental nucleus in primates. J Neurosci 31:11457–11471
Hosokawa T, Kato K, Inoue M, Mikami A (2007) Neurons in the macaque orbitofrontal cortex code relative preference of both rewarding and aversive outcomes. Neurosci Res 57:434–445
Houk J, Adams JL, Barto AG (ed) (1995) A model of how the basal ganglia generate and use neural signals that predict reinforcement
Ikemoto S (2007) Dopamine reward circuitry: two projection systems from the ventral midbrain to the nucleus accumbens-olfactory tubercle complex. Brain Res Rev 56:27–78
Ikemoto S, Panksepp J (1999) The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res Brain Res Rev 31:6–41
Ito M, Doya K (2009) Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J Neurosci 29:9861–9874
Ito S, Stuphorn V, Brown JW, Schall JD (2003) Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302:120–122
Izquierdo AD, Suda RK, Murray EA (2004) Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J Neurosci 24:7540–7548
Janak PH, Chen MT, Caulder T (2004) Dynamics of neural coding in the accumbens during extinction and reinstatement of rewarded behavior. Behav Brain Res 154:125–135
Jhou TC, Fields HL, Baxter MG, Saper CB, Holland PC (2009) The rostromedial tegmental nucleus (RMTg), a GABAergic afferent to midbrain dopamine neurons, encodes aversive stimuli and inhibits motor responses. Neuron 61:786–800
Ji H, Shepard PD (2007) Lateral habenula stimulation inhibits rat midbrain dopamine neurons through a GABA(A) receptor-mediated mechanism. J Neurosci 27:6923–6930
Jo YS, Lee J, Mizumori SJ (2013) Effects of prefrontal cortical inactivation on neural activity in the ventral tegmental area. J Neurosci 33:8159–8171
Joel D, Niv Y, Ruppin E (2002) Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Networks: Official J Int Neural Network Soc 15:535–547
Johansen JP, Tarpley JW, LeDoux JE, Blair HT (2010) Neural substrates for expectation-modulated fear learning in the amygdala and periaqueductal gray. Nat Neurosci 13:979–986
Jones B, Mishkin M (1972) Limbic lesions and the problem of stimulus-reinforcement associations. Exp Neurol 36:362–377
Kalenscher T, Lansink CS, Lankelma JV, Pennartz CM (2010) Reward-associated gamma oscillations in ventral striatum are regionally differentiated and modulate local firing activity. J Neurophysiol 103:1658–1672
Kaping D, Vinck M, Hutchison RM, Everling S, Womelsdorf T (2011) Specific contributions of ventromedial, anterior cingulate, and lateral prefrontal cortex for attentional selection and stimulus valuation. PLoS Biol 9:e1001224
Karachi C et al (2002) Three-dimensional cartography of functional territories in the human striatopallidal complex by using calbindin immunoreactivity. J Comp Neurol 450:122–134
Kaufling J, Veinante P, Pawlowski SA, Freund-Mercier MJ, Barrot M (2009) Afferents to the GABAergic tail of the ventral tegmental area in the rat. J Comp Neurol 513:597–621
Kennerley SW, Wallis JD (2009) Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur J Neurosci 29:2061–2073
Kennerley SW, Walton ME, Behrens TE, Buckley MJ, Rushworth MF (2006) Optimal decision making and the anterior cingulate cortex. Nat Neurosci 9:940–947
Kennerley SW, Dahmubed AF, Lara AH, Wallis JD (2009) Neurons in the frontal lobe encode the value of multiple decision variables. J Cogn Neurosci 21:1162–1178
Kennerley SW, Behrens TE, Wallis JD (2011) Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat Neurosci 14:1581–1589
Kesner RP, Williams JM (1995) Memory for magnitude of reinforcement: dissociation between amygdala and hippocampus. Neurobiol Learn Mem 64:237–244
Kim S, Hwang J, Lee D (2008) Prefrontal coding of temporally discounted values during intertemporal choice. Neuron 59:161–172
Kim H, Sul JH, Huh N, Lee D, Jung MW (2009) Role of striatum in updating values of chosen actions. J Neurosci: Official J Soc Neurosci 29:14701–14712
Klavir O, Genud-Gabai R, Paz R (2013) Functional connectivity between amygdala and cingulate cortex for adaptive aversive learning. Neuron 80:1290–1300
Koch M, Schmid A, Schnitzler HU (2000) Role of muscles accumbens dopamine D1 and D2 receptors in instrumental and Pavlovian paradigms of conditioned reward. Psychopharmacology 152:67–73
Lammel S, Ion DI, Roeper J, Malenka RC (2011) Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron 70:855–862
Lang PJ, Davis M (2006) Emotion, motivation, and the brain: reflex foundations in animal and human research. Prog Brain Res 156:3–29
Lansink CS, Goltstein PM, Lankelma JV, Pennartz CM (2010) Fast-spiking interneurons of the rat ventral striatum: temporal coordination of activity with principal cells and responsiveness to reward. Eur J Neurosci 32:494–508
Lau B, Glimcher PW (2007) Action and outcome encoding in the primate caudate nucleus. J Neurosci 27:14502–14514
Lau B, Glimcher PW (2008) Value representations in the primate striatum during matching behavior. Neuron 58:451–463
Leathers ML, Olson CR (2012) In monkeys making value-based decisions, LIP neurons encode cue salience and not action value. Science 338:132–135
Leathers ML, Olson CR (2013) Response to comment on “in monkeys making value-based decisions, LIP neurons encode cue salience and not action value”. Science 340:430
LeDoux JE (2000) The amygdala and emotion: a view through fear. In: Aggleton JP (ed) The amygdala: a functional analysis. Oxford University Press, New York, pp 289–310
Lex B, Hauber W (2010) The role of nucleus accumbens dopamine in outcome encoding in instrumental and Pavlovian conditioning. Neurobiol Learn Mem 93:283–290
Lichtenberg NT, Kashtelyan V, Burton AC, Bissonette GB, Roesch MR (2013) Nucleus accumbens core lesions enhance two-way active avoidance. Neurosci
Louie K, Glimcher PW (2010) Separating value from choice: delay discounting activity in the lateral intraparietal area. J Neurosci 30:5498–5507
Magno E, Foxe JJ, Molholm S, Robertson IH, Garavan H (2006) The anterior cingulate and error avoidance. J Neurosci 26:4769–4773
Malkova L, Gaffan D, Murray EA (1997) Excitotoxic lesions of the amygdala fail to produce impairment in visual learning for auditory secondary reinforcement but interfere with reinforcer devaluation effects in rhesus monkeys. J Neurosci 17:6011–6020
Matsumoto M, Hikosaka O (2007) Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447:1111–1115
Matsumoto M, Hikosaka O (2009) Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459:837–841
Matsumoto M, Matsumoto K, Abe H, Tanaka K (2007) Medial prefrontal cell activity signaling prediction errors of action values. Nat Neurosci 10:647–656
McCullough LD, Salamone JD (1992) Involvement of nucleus accumbens dopamine in the motor activity induced by periodic food presentation: a microdialysis and behavioral study. Brain Res 592:29–36
McGinty VB, Lardeux S, Taha SA, Kim JJ, Nicola SM (2013) Invigoration of reward seeking by cue and proximity encoding in the nucleus accumbens. Neuron 78:910–922
Monosov IE, Hikosaka O (2012) Regionally distinct processing of rewards and punishments by the primate ventromedial prefrontal cortex. J Neurosci 32:10318–10330
Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci: Official J Soc Neurosci 16:1936–1947
Morrison SE, Saez A, Lau B, Salzman CD (2011) Different time courses for learning-related changes in amygdala and orbitofrontal cortex. Neuron 71:1127–1140
Murray EA (2007) The amygdala, reward and emotion. Trends Cogn Sci 11:489–497
Nakamura K, Santos GS, Matsuzaki R, Nakahara H (2012) Differential reward coding in the subdivisions of the primate caudate during an oculomotor task. J Neurosci 32:15963–15982
Newsome WT, Glimcher PW, Gottlieb J, Lee D, Platt ML (2013) Comment on “in monkeys making value-based decisions, LIP neurons encode cue salience and not action value”. Science 340:430
Nicola SM, Yun IA, Wakabayashi KT, Fields HL (2004) Cue-evoked firing of nucleus accumbens neurons encodes motivational significance during a discriminative stimulus task. J Neurophysiol 91:1840–1865
Nishizawa K et al (2012) Striatal indirect pathway contributes to selection accuracy of learned motor actions. J Neurosci: Official J Soc Neurosci 32:13421–13432
Niv Y, Schoenbaum G (2008) Dialogues on prediction errors. Trends in cognitive sciences 12:265–272
Nunes EJ, Randall PA, Podurgiel S, Correa M, Salamone JD (2013) Nucleus accumbens neurotransmission and effort-related choice behavior in food motivation: effects of drugs acting on dopamine, adenosine, and muscarinic acetylcholine receptors. Neurosci Biobehav Rev 37:2015–2025
O’Doherty J et al (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452–454
Oleson EB, Gentry RN, Chioma VC, Cheer JF (2012) Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidance. J Neurosci: Official J Soc Neurosci 32:14804–14808
Oliveira FT, McDonald JJ, Goodman D (2007) Performance monitoring in the anterior cingulate is not all error related: expectancy deviation and the representation of action-outcome associations. J Cogn Neurosci 19:1994–2004
Omelchenko N, Bell R, Sesack SR (2009) Lateral habenula projections to dopamine and GABA neurons in the rat ventral tegmental area. Eur J Neurosci 30:1239–1250
Ostlund SB, Balleine BW (2007) Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental learning. J Neurosci 27:4819–4825
Padoa-Schioppa C (2007) Orbitofrontal cortex and the computation of economic value. Ann N Y Acad Sci 1121:232–253
Padoa-Schioppa C (2011) Neurobiology of economic choice: a good-based model. Annu Rev Neurosci 34:333–359
Padoa-Schioppa C, Assad JA (2006) Neurons in the orbitofrontal cortex encode economic value. Nature 441:223–226
Padoa-Schioppa C, Assad JA (2008) The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nat Neurosci 11:95–102
Padoa-Schioppa C, Cai X (2011) The orbitofrontal cortex and the computation of subjective value: consolidated concepts and new perspectives. Ann N Y Acad Sci 1239:130–137
Parkinson JA et al (2001) The role of the primate amygdala in conditioned reinforcement. J Neurosci 21:7770–7780
Paus T (2001) Primate anterior cingulate cortex: where motor control, drive and cognition interface. Nat Rev Neurosci 2:417–424
Pearce JM, Hall G (1980) A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87:532–552
Phelps EA, Ling S, Carrasco M (2006) Emotion facilitates perception and potentiates the perceptual benefits of attention. Psychol Sci 17:292–299
Pickens CL et al (2003) Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. J Neurosci 23:11078–11084
Platt ML, Glimcher PW (1999) Neural correlates of decision variables in parietal cortex. Nature 400:233–238
Quilodran R, Rothe M, Procyk E (2008) Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron 57:314–325
Rangel A, Hare T (2010) Neural computations associated with goal-directed choice. Curr Opin Neurobiol 20:262–270
Redish AD (2004) Addiction as a computational process gone awry. Science 306:1944–1947
Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning II: current research and theory. Appleton-Century-Crofts, New York, pp 64–99
Rich EL, Wallis JD (2014) Medial-lateral organization of the orbitofrontal cortex. J Cogn Neurosci 26:1347–1362
Roesch MR, Bryden DW (2011) Impact of size and delay on neural activity in the rat limbic corticostriatal system. Frontiers Neurosci 5:130
Roesch MR, Olson CR (2004) Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304:307–310
Roesch MR, Olson CR (2005a) Neuronal activity in primate orbitofrontal cortex reflects the value of time. J Neurophysiol 94:2457–2471
Roesch MR, Olson CR (2005b) Neuronal activity dependent on anticipated and elapsed delay in macaque prefrontal cortex, frontal and supplementary eye fields, and premotor cortex. J Neurophysiol 94:1469–1497
Roesch MR, Calu DJ, Burke KA, Schoenbaum G (2007a) Should I stay or should I go? Transformation of time-discounted rewards in orbitofrontal cortex and associated brain circuits. Ann N Y Acad Sci 1104:21–34
Roesch MR, Takahashi Y, Gugsa N, Bissonette GB, Schoenbaum G (2007b) Previous cocaine exposure makes rats hypersensitive to both delay and reward magnitude. J Neurosci 27:245–250
Roesch MR, Singh T, Brown PL, Mullins SE, Schoenbaum G (2009) Ventral striatal neurons encode the value of the chosen action in rats deciding between differently delayed or sized rewards. J Neurosci 29:13365–13376
Roesch MR, Calu DJ, Esber GR, Schoenbaum G (2010a) Neural correlates of variations in event processing during learning in basolateral amygdala. J Neurosci: Official J Soc Neurosci 30:2464–2471
Roesch MR, Calu DJ, Esber GR, Schoenbaum G (2010b) All that glitters … dissociating attention and outcome expectancy from prediction errors signals. J Neurophysiol 104:587–595
Roesch MR et al (2012a) Normal aging alters learning and attention-related teaching signals in basolateral amygdala. J Neurosci 32:13137–13144
Roesch MR, Bryden DW, Cerri DH, Haney ZR, Schoenbaum G (2012b) Willingness to wait and altered encoding of time-discounted reward in the orbitofrontal cortex with normal aging. J Neurosci 32:5525–5533
Rolls ET, Sienkiewicz ZJ, Yaxley S (1989) Hunger modulates the responses to gustatory stimuli of single neurons in the caudolateral orbitofrontal cortex of the macaque monkey. EurJ Neurosci 1:53–60
Rothe M, Quilodran R, Sallet J, Procyk E (2011) Coordination of high gamma activity in anterior cingulate and lateral prefrontal cortical areas during adaptation. J Neurosci: Official J Soc Neurosci 31:11110–11117
Rudebeck PH, Murray EA (2008) Amygdala and orbitofrontal cortex lesions differentially influence choices during object reversal learning. J Neurosci: Official J Soc Neurosci 28:8338–8343
Rudebeck PH, Murray EA (2011) Balkanizing the primate orbitofrontal cortex: distinct subregions for comparing and contrasting values. Ann N Y Acad Sci 1239:1–13
Rudebeck PH, Murray EA (2014) The orbitofrontal oracle: cortical mechanisms for the prediction and evaluation of specific behavioral outcomes. Neuron 84:1143–1156
Rudebeck PH, Bannerman DM, Rushworth MF (2008) The contribution of distinct subregions of the ventromedial frontal cortex to emotion, social behavior, and decision making. Cogn Affect Behav Neurosci 8:485–497
Rudebeck PH, Saunders RC, Prescott AT, Chau LS, Murray EA (2013a) Prefrontal mechanisms of behavioral flexibility, emotion regulation and value updating. Nat Neurosci 16:1140–1145
Rudebeck PH, Mitz AR, Chacko RV, Murray EA (2013b) Effects of amygdala lesions on reward-value coding in orbital and medial prefrontal cortex. Neuron 80:1519–1531
Rushworth MF, Behrens TE (2008) Choice, uncertainty and value in prefrontal and cingulate cortex. Nat Neurosci 11:389–397
Rushworth MF, Walton ME, Kennerley SW, Bannerman DM (2004) Action sets and decisions in the medial frontal cortex. Trends Cogn Sci 8:410–417
Rushworth MF, Behrens TE, Rudebeck PH, Walton ME (2007) Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour. Trends Cogn Sci 11:168–176
Saddoris MP, Gallagher M, Schoenbaum G (2005) Rapid associative encoding in basolateral amygdala depends on connections with orbitofrontal cortex. Neuron 46:321–331
Salamone JD (1986) Different effects of haloperidol and extinction on instrumental behaviours. Psychopharmacology 88:18–23
Salamone JD (1994) The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav Brain Res 61:117–133
Salamone JD, Correa M (2002) Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav Brain Res 137:3–25
Salamone JD, Correa M (2012) The mysterious motivational functions of mesolimbic dopamine. Neuron 76:470–485
Salamone JD et al (1991) Haloperidol and nucleus accumbens dopamine depletion suppress lever pressing for food but increase free food consumption in a novel food choice procedure. Psychopharmacology 104:515–521
Salamone JD, Correa M, Nunes EJ, Randall PA, Pardo M (2012) The behavioral pharmacology of effort-related choice behavior: dopamine, adenosine and beyond. J Exp Anal Behav 97:125–146
Sallet J et al (2007) Expectations, gains, and losses in the anterior cingulate cortex. Cogn Affect Behav Neurosci 7:327–336
Samejima K, Ueda Y, Doya K, Kimura M (2005) Representation of action-specific reward values in the striatum. Science 310:1337–1340
Scheffers MK, Coles MG (2000) Performance monitoring in a confusing world: error-related brain activity, judgments of response accuracy, and types of errors. J Exp Psychol Hum Percept Perform 26:141–151
Schoenbaum G (2004) Affect, action, and ambiguity and the amygdala-orbitofrontal circuit. Focus on “combined unilateral lesions of the amygdala and orbital prefrontal cortex impair affective processing in rhesus monkeys”. J Neurophysiol 91:1938–1939
Schoenbaum G, Eichenbaum H (1995) Information coding in the rodent prefrontal cortex. I. Single-neuron activity in orbitofrontal cortex compared with that in pyriform cortex. J Neurophysiol 74:733–750
Schoenbaum G, Esber GR (2010) How do you (estimate you will) like them apples? Integration as a defining trait of orbitofrontal function. Curr Opin Neurobiol 20:205–211
Schoenbaum G, Roesch M (2005) Orbitofrontal cortex, associative learning, and expectancies. Neuron 47:633–636
Schoenbaum G, Chiba AA, Gallagher M (1998) Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat Neurosci 1:155–159
Schoenbaum G, Chiba AA, Gallagher M (1999) Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J Neurosci 19:1876–1884
Schoenbaum G, Chiba AA, Gallagher M (2000) Changes in functional connectivity in orbitofrontal cortex and basolateral amygdala during learning and reversal training. J Neurosci 20:5179–5189
Schoenbaum G, Setlow B, Nugent SL, Saddoris MP, Gallagher M (2003a) Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learn Mem 10:129–140
Schoenbaum G, Setlow B, Saddoris MP, Gallagher M (2003b) Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39:855–867
Schoenbaum G, Roesch MR, Stalnaker TA (2006) Orbitofrontal cortex, decision-making and drug addiction. Trends Neurosci 29:116–124
Schoenbaum G, Gottfried JA, Murray EA, Ramus SJ (2007) Linking affect to action: critical contributions of the orbitofrontal cortex. Preface. Ann N Y Acad Sci 1121:xi–xiii
Schoenbaum G, Roesch MR, Stalnaker TA, Takahashi YK (2009) A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat Rev Neurosci 10:885–892
Schoenbaum G, Takahashi Y, Liu TL, McDannald MA (2011a) Does the orbitofrontal cortex signal value? Ann N Y Acad Sci 1239:87–99
Schoenbaum G, Roesch MR, Stalnaker TA, Takahashi YK (2011b) Orbitofrontal cortex and outcome expectancies: optimizing behavior and sensory perception. In: Gottfried JA (ed) Neurobiology of sensation and reward. Boca Raton, Florida
Schultz W (1997) Dopamine neurons and their role in reward mechanisms. Curr Opin Neurobiol 7:191–197
Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1–27
Schultz W (1999) The reward signal of midbrain dopamine neurons. News Physiol Sci: Int J Physiol Produced Jointly Int Union Physiol Sci Am Physiol Soc 14:249–255
Schultz W, Apicella P, Scarnati E, Ljungberg T (1992) Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12:4595–4610
Setlow B, Schoenbaum G, Gallagher M (2003) Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38:625–636
Shabel SJ, Janak PH (2009) Substantial similarity in amygdala neuronal activity during conditioned appetitive and aversive emotional arousal. Proc Natl Acad Sci USA 106:15031–15036
Shidara M, Richmond BJ (2004) Differential encoding of information about progress through multi-trial reward schedules by three groups of ventral striatal neurons. Neurosci Res 49:307–314
Singh T, McDannald MA, Haney RZ, Cerri DH, Schoenbaum G (2010) Nucleus accumbens core and shell are necessary for reinforcer devaluation effects on pavlovian conditioned responding. Front Integr Neurosci 4:126
Solomon RL, Corbit JD (1974) An opponent-process theory of motivation. I. Temporal dynamics of affect. Psychol Rev 81:119–145
Sripanidkulchai K, Sripanidkulchai B, Wyss JM (1984) The cortical projection of the basolateral amygdaloid nucleus in the rat: a retrograde fluorescent dye study. J Comp Neurol 229:419–431
Stalnaker TA, Franz TM, Singh T, Schoenbaum G (2007) Basolateral amygdala lesions abolish orbitofrontal-dependent reversal impairments. Neuron 54:51–58
Stalnaker TA, Takahashi Y, Roesch MR, Schoenbaum G (2009) Neural substrates of cognitive inflexibility after chronic cocaine exposure. Neuropharmacology 56(Suppl 1):63–72
Stalnaker TA, Calhoon GG, Ogawa M, Roesch MR, Schoenbaum G (2010) Neural correlates of stimulus-response and response-outcome associations in dorsolateral versus dorsomedial striatum. Frontiers Integr Neurosci 4:12
Steinberg EE et al (2013) A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci 16:966–973
Stopper CM, Floresco SB (2011) Contributions of the nucleus accumbens and its subregions to different aspects of risk-based decision making. Cogn Affect Behav Neurosci 11:97–112
Sugrue LP, Corrado GS, Newsome WT (2004) Matching behavior and the representation of value in the parietal cortex. Science 304:1782–1787
Sugrue LP, Corrado GS, Newsome WT (2005) Choosing the greater of two goods: neural currencies for valuation and decision making. Nat Rev Neurosci 6:363–375
Sutton RA, Barto AG (ed) (1998) Reinforcement learning: an introduction
Takahashi Y, Roesch MR, Stalnaker TA, Schoenbaum G (2007) Cocaine exposure shifts the balance of associative encoding from ventral to dorsolateral striatum. Frontiers Integr Neurosci 1:11
Takahashi Y, Schoenbaum G, Niv Y (2008) Silencing the critics: understanding the effects of cocaine sensitization on dorsolateral and ventral striatum in the context of an actor/critic model. Front Neurosci 2:86–99
Takahashi YK et al (2009) The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 62:269–280
Takahashi YK et al (2011) Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat Neurosci 14:1590–1597
Totah NK, Kim YB, Homayoun H, Moghaddam B (2009) Anterior cingulate neurons represent errors and preparatory attention within the same behavioral sequence. J Neurosci 29:6418–6426
Tsujimoto S, Genovesio A, Wise SP (2011) Comparison of strategy signals in the dorsolateral and orbital prefrontal cortex. J Neurosci 31:4583–4592
Tsujimoto S, Genovesio A, Wise SP (2012) Neuronal activity during a cued strategy task: comparison of dorsolateral, orbital, and polar prefrontal cortex. J Neurosci 32:11017–11031
Tye KM, Janak PH (2007) Amygdala neurons differentially encode motivation and reinforcement. J Neurosci: Official J Soc Neurosci 27:3937–3945
Tye KM, Cone JJ, Schairer WW, Janak PH (2010) Amygdala neural encoding of the absence of reward during extinction. J Neurosci: Official J Soc Neurosci 30:116–125
van der Meer MA, Redish AD (2009) Covert expectation-of-reward in rat ventral striatum at decision points. Front Integr Neurosci 3:1
van der Meer MA, Redish AD (2011) Ventral striatum: a critical look at models of learning and evaluation. Curr Opin Neurobiol 21:387–392
van der Meer MA, Johnson A, Schmitzer-Torbert NC, Redish AD (2010) Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron 67:25–32
Wadenberg ML, Ericson E, Magnusson O, Ahlenius S (1990) Suppression of conditioned avoidance behavior by the local application of (-)sulpiride into the ventral, but not the dorsal, striatum of the rat. Biol Psychiatry 28:297–307
Wakabayashi KT, Fields HL, Nicola SM (2004) Dissociation of the role of nucleus accumbens dopamine in responding to reward-predictive cues and waiting for reward. Behav Brain Res 154:19–30
Wallis JD (2007) Orbitofrontal cortex and its contribution to decision-making. Annu Rev Neurosci 30:31–56
Wallis JD, Kennerley SW (2010) Heterogeneous reward signals in prefrontal cortex. Curr Opin Neurobiol 20:191–198
Wallis JD, Kennerley SW (2011) Contrasting reward signals in the orbitofrontal cortex and anterior cingulate cortex. Ann N Y Acad Sci 1239:33–42
Wallis JD, Miller EK (2003) Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur J Neurosci 18:2069–2081
Wallis JD, Anderson KC, Miller EK (2001) Single neurons in prefrontal cortex encode abstract rules. Nature 411:953–956
Walton ME, Devlin JT, Rushworth MF (2004) Interactions between decision making and performance monitoring within prefrontal cortex. Nat Neurosci 7:1259–1265
Wang AY, Miura K, Uchida N (2013) The dorsomedial striatum encodes net expected return, critical for energizing performance vigor. Nat Neurosci 16:639–647
Wilson RC, Takahashi YK, Schoenbaum G, Niv Y (2014) Orbitofrontal cortex as a cognitive map of task space. Neuron 81:267–279
Winstanley CA, Theobald DEH, Cardinal RN, Robbins TW (2004) Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J Neurosci 24:4718–4722
Yin HH, Knowlton BJ (2006) The role of the basal ganglia in habit formation. Nat Rev Neurosci 7:464–476
Yun IA, Wakabayashi KT, Fields HL, Nicola SM (2004) The ventral tegmental area is required for the behavioral and nucleus accumbens neuronal firing responses to incentive cues. J Neurosci 24:2923–2933
Acknowledgements
This work was supported by grants from the NIDA (R01DA031695, MR).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Bissonette, G.B., Roesch, M.R. (2015). Neurophysiology of Reward-Guided Behavior: Correlates Related to Predictions, Value, Motivation, Errors, Attention, and Action. In: Simpson, E., Balsam, P. (eds) Behavioral Neuroscience of Motivation. Current Topics in Behavioral Neurosciences, vol 27. Springer, Cham. https://doi.org/10.1007/7854_2015_382
Download citation
DOI: https://doi.org/10.1007/7854_2015_382
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26933-7
Online ISBN: 978-3-319-26935-1
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)