Skip to main content
SpringerLink
Log in

Multiple Systems for the Motivational Control of Behavior and Associated Neural Substrates in Humans

  • Chapter
  • First Online:
Behavioral Neuroscience of Motivation

Part of the book series: Current Topics in Behavioral Neurosciences ((CTBN,volume 27))

Abstract

In this chapter, we will review evidence about the role of multiple distinct systems in driving the motivation to perform actions in humans. Specifically, we will consider the contribution of goal-directed action selection mechanisms, habitual action selection mechanisms and the influence of Pavlovian predictors on instrumental action selection. We will further evaluate evidence for the contribution of multiple brain areas including ventral frontal and dorsal cortical areas and several distinct parts of the striatum in these processes. Furthermore, we will consider circumstances in which adverse interactions between these systems can result in the decoupling of motivation from incentive valuation and performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 279.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Aarts E, Wallace DL, Dang LC, Jagust WJ, Cools R, D’Esposito M (2014) Dopamine and the cognitive downside of a promised bonus. Psychol Sci. doi:10.1177/0956797613517240

  • Allman MJ, DeLeon IG, Cataldo MF, Holland PC, Johnson AW (2010) Learning processes affecting human decision making: an assessment of reinforcer-selective pavlovian-to-instrumental transfer following reinforcer devaluation. J Exp Psychol Anim Behav Process 36:402

    Article  PubMed  Google Scholar 

  • Ariely D, Gneezy U, Loewenstein G, Mazar N (2009) Large stakes and big mistakes. Rev Eco Stud 76:451–469

    Article  Google Scholar 

  • Balleine BW, Dickinson A (1998) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37:407–419

    Article  CAS  PubMed  Google Scholar 

  • Balleine BW, O’Doherty JP (2010) Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35:48–69

    Article  PubMed  PubMed Central  Google Scholar 

  • Balleine BW, Daw ND, O’Doherty JP (2008) Multiple forms of value learning and the function of dopamine. In: Glimcher PW, Camerer C, Fehr E, Poldrack RA (eds) Neuroeconomics: decision making and the brain. Elsevier, New York, pp 367–385

    Google Scholar 

  • Bartra O, McGuire JT, Kable JW (2013) The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. Neuroimage 76:412–427

    Article  PubMed  PubMed Central  Google Scholar 

  • Bray S, Rangel A, Shimojo S, Balleine B, O’Doherty JP (2008) The neural mechanisms underlying the influence of pavlovian cues on human decision making. J Neurosci 28:5861–5866

    Article  CAS  PubMed  Google Scholar 

  • Broadhurst P (1959) The interaction of task difficulty and motivation: the Yerkes-Dodson Law revived. Acta Psychol 16:321–338

    Article  Google Scholar 

  • Chib VS, Rangel A, Shimojo S, O’oherty JP (2009) Evidence for a common representation of decision values for dissimilar goods in human ventromedial prefrontal cortex. J Neurosci 29:12315–12320

    Article  CAS  PubMed  Google Scholar 

  • Chib VS, De Martino B, Shimojo S, O’Doherty JP (2012) Neural mechanisms underlying paradoxical performance for monetary incentives are driven by loss aversion. Neuron 74:582–594

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Chib VS, Shimojo S, O’Doherty JP (2014) The effects of incentive framing on performance decrements for large monetary outcomes: behavioral and neural mechanisms. J Neurosci 34:14833–14844

    Article  CAS  PubMed  Google Scholar 

  • Corbit LH, Balleine BW (2005) Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of pavlovian-instrumental transfer. J Neurosci 25:962–970

    Article  CAS  PubMed  Google Scholar 

  • Corbit LH, Balleine BW (2011) The general and outcome-specific forms of Pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell. J Neurosci 31:11786–11794

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Daw ND, Niv Y, Dayan P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8:1704–1711

    Article  CAS  PubMed  Google Scholar 

  • Dayan P, Niv Y, Seymour B, Daw ND (2006) The misbehavior of value and the discipline of the will. Neural Netw 19:1153–1160

    Article  PubMed  Google Scholar 

  • de Araujo IE, Kringelbach ML, Rolls ET, McGlone F (2003a) Human cortical responses to water in the mouth, and the effects of thirst. J Neurophysiol 90:1865–1876

    Article  PubMed  Google Scholar 

  • de Araujo IE, Rolls ET, Kringelbach ML, McGlone F, Phillips N (2003b) Taste-olfactory convergence, and the representation of the pleasantness of flavour, in the human brain. Eur J Neurosci 18:2059–2068

    Article  PubMed  Google Scholar 

  • de Araujo IE, Rolls ET, Velazco MI, Margot C, Cayeux I (2005) Cognitive modulation of olfactory processing. Neuron 46:671–679

    Article  PubMed  Google Scholar 

  • de Wit S, Watson P, Harsay HA, Cohen MX, van de Vijver I, Ridderinkhof KR (2012) Corticostriatal connectivity underlies individual differences in the balance between habitual and goal-directed action control. J Neurosci 32:12066–12075

    Article  PubMed  Google Scholar 

  • Delgado MR, Gillis MM, Phelps EA (2008a) Regulating the expectation of reward via cognitive strategies. Nat Neurosci 11:880–881

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Delgado MR, Nearing KI, LeDoux JE, Phelps EA (2008b) Neural circuitry underlying the regulation of conditioned fear and its relation to extinction. Neuron 59:829–838

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Dickinson A (1985) Actions and habits: the development of a behavioural autonomy. Philos Trans R Soc Lond Ser B Biol Sci 308:67–78

    Article  Google Scholar 

  • Dickinson A, Nicholas D, Adams CD (1983) The effect of the instrumental training contingency on susceptibility to reinforcer devaluation. Q J Exp Psychol 35:35–51

    Article  Google Scholar 

  • FitzGerald TH, Dolan RJ, Friston KJ (2014) Model averaging, optimal inference, and habit formation. Front Hum Neurosci 8:457

    Google Scholar 

  • Gillan CM, Robbins TW (2014) Goal-directed learning and obsessive–compulsive disorder. Philos Trans R Soc B Biol Sci 369:20130475

    Article  Google Scholar 

  • Gottfried JA, O’Doherty J, Dolan RJ (2002) Appetitive and aversive olfactory learning in humans studied using event-related functional magnetic resonance imaging. J Neurosci 22:10829–10837

    CAS  PubMed  Google Scholar 

  • Gottfried JA, O’Doherty J, Dolan RJ (2003) Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301:1104–1107

    Article  CAS  PubMed  Google Scholar 

  • Gruner P, Anticevic A, Lee D, Pittenger C (2015) Arbitration between action strategies in obsessive-compulsive disorder. Neuroscientist, 1073858414568317

    Google Scholar 

  • Hall J, Parkinson JA, Connor TM, Dickinson A, Everitt BJ (2001) Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behaviour. Eur J Neurosci 13:1984–1992

    Article  CAS  PubMed  Google Scholar 

  • Hare TA, Schultz W, Camerer CF, O’Doherty JP, Rangel A (2011) Transformation of stimulus value signals into motor commands during simple choice. Proc Natl Acad Sci 108:18120–18125

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Holland PC (2004) Relations between Pavlovian-instrumental transfer and reinforcer devaluation. J Exp Psychol Anim Behav Process 30:104

    Article  PubMed  Google Scholar 

  • Hosokawa T, Kennerley SW, Sloan J, Wallis JD (2013) Single-neuron mechanisms underlying cost-benefit analysis in frontal cortex. J Neurosci 33:17385–17397

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Hunt LT, Dolan RJ, Behrens TE (2014) Hierarchical competitions subserving multi-attribute choice. Nat Neurosci 17:1613–1622

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Knutson B, Fong GW, Adams CM, Varner JL, Hommer D (2001) Dissociation of reward anticipation and outcome with event-related fMRI. NeuroReport 12:3683–3687

    Article  CAS  PubMed  Google Scholar 

  • Kringelbach ML, O’Doherty J, Rolls ET, Andrews C (2003) Activation of the human orbitofrontal cortex to a liquid food stimulus is correlated with its subjective pleasantness. Cereb Cortex 13:1064–1071

    Article  CAS  PubMed  Google Scholar 

  • LaBar KS, Gatenby JC, Gore JC, LeDoux JE, Phelps EA (1998) Human amygdala activation during conditioned fear acquisition and extinction: a mixed-trial fMRI study. Neuron 20:937–945

    Article  CAS  PubMed  Google Scholar 

  • Lau B, Glimcher PW (2007) Action and outcome encoding in the primate caudate nucleus. J Neurosci 27:14502–14514

    Article  CAS  PubMed  Google Scholar 

  • LeDoux J (2003) The emotional brain, fear, and the amygdala. Cell Mol Neurobiol 23:727–738

    Article  PubMed  Google Scholar 

  • Lee SW, Shimojo S, O’Doherty JP (2014) Neural computations underlying arbitration between model-based and model-free learning. Neuron 81:687–699

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Levy DJ, Glimcher PW (2011) Comparing apples and oranges: using reward-specific and reward-general subjective value representation in the brain. J Neurosci 31:14693–14707

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Liljeholm M, Tricomi E, O’Doherty JP, Balleine BW (2011) Neural correlates of instrumental contingency learning: differential effects of action-reward conjunction and disjunction. J Neurosci 31:2474–2480

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Liljeholm M, Wang S, Zhang J, O’Doherty JP (2013) Neural correlates of the divergence of instrumental probability distributions. J Neurosci 33:12519–12527

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Lovibond PF (1983) Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus. J Exp Psychol Anim Behav Process 9:225

    Article  CAS  PubMed  Google Scholar 

  • Maia TV, Frank MJ (2011) From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci 14:154–162

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • McNamee D, Rangel A, O’Doherty JP (2013) Category-dependent and category-independent goal-value codes in human ventromedial prefrontal cortex. Nat Neurosci 16:479–485

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Miller EM, Shankar MU, Knutson B, McClure SM (2014) Dissociating motivation from reward in human striatal activity. J Cogn Neurosci 26:1075–1084

    Article  PubMed  Google Scholar 

  • Mobbs D, Hassabis D, Seymour B, Marchant JL, Weiskopf N, Dolan RJ, Frith CD (2009) Choking on the money reward-based performance decrements are associated with midbrain activity. Psychol Sci 20:955–962

    Article  PubMed  PubMed Central  Google Scholar 

  • Montague PR, Dolan RJ, Friston KJ, Dayan P (2012) Computational psychiatry. Trends Cogn Sci 16:72–80

    Article  PubMed  PubMed Central  Google Scholar 

  • Morris RW, Dezfouli A, Griffiths KR, Balleine BW (2014) Action-value comparisons in the dorsolateral prefrontal cortex control choice between goal-directed actions. Nat Commun 5

    Google Scholar 

  • O’Doherty JP (2007) Lights, camembert, action! The role of human orbitofrontal cortex in encoding stimuli, rewards, and choices. Ann N Y Acad Sci 1121:254–272

    Article  PubMed  Google Scholar 

  • O’Doherty JP (2011) Contributions of the ventromedial prefrontal cortex to goal-directed action selection. Ann N Y Acad Sci 1239:118–129

    Article  PubMed  Google Scholar 

  • O’Doherty J, Kringelbach ML, Rolls ET, Hornak J, Andrews C (2001) Abstract reward and punishment representations in the human orbitofrontal cortex. Nat Neurosci 4(1):95–102

    Google Scholar 

  • O’Doherty J, Rolls ET, Francis S, Bowtell R, McGlone F, Kobal G, Renner B, Ahne G (2000) Sensory-specific satiety-related olfactory activation of the human orbitofrontal cortex. NeuroReport 11:893–897

    Article  PubMed  Google Scholar 

  • O’Doherty J, Winston J, Critchley H, Perrett D, Burt DM, Dolan RJ (2003) Beauty in a smile: the role of medial orbitofrontal cortex in facial attractiveness. Neuropsychologia 41:147–155

    Article  PubMed  Google Scholar 

  • Ostlund SB, Balleine BW (2007) Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental conditioning. J Neurosci 27:4819–4825

    Article  CAS  PubMed  Google Scholar 

  • Parkinson JA, Olmstead MC, Burns LH, Robbins TW, Everitt BJ (1999) Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity byd-amphetamine. J Neurosci 19:2401–2411

    CAS  PubMed  Google Scholar 

  • Pessiglione M, Schmidt L, Draganski B, Kalisch R, Lau H, Dolan RJ, Frith CD (2007) How the brain translates money into force: a neuroimaging study of subliminal motivation. Science 316:904–906

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Plassmann H, O’Doherty J, Rangel A (2007) Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J Neurosci 27:9984–9988

    Article  CAS  PubMed  Google Scholar 

  • Plassmann H, O’Doherty J, Shiv B, Rangel A (2008) Marketing actions can modulate neural representations of experienced pleasantness. Proc Natl Acad Sci U S A 105:1050–1054

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Plassmann H, O’Doherty JP, Rangel A (2010) Appetitive and aversive goal values are encoded in the medial orbitofrontal cortex at the time of decision making. J Neurosci 30:10799–10808

    Article  CAS  PubMed  Google Scholar 

  • Platt ML, Glimcher PW (1999) Neural correlates of decision variables in parietal cortex. Nature 400:233–238

    Article  CAS  PubMed  Google Scholar 

  • Prevost C, Liljeholm M, Tyszka JM, O’Doherty JP (2012) Neural correlates of specific and general Pavlovian-to-Instrumental Transfer within human amygdalar subregions: a high-resolution fMRI study. J Neurosci 32:8383–8390

    Article  CAS  PubMed  Google Scholar 

  • PrĂ©vost C, Pessiglione M, MĂ©tĂ©reau E, ClĂ©ry-Melin M-L, Dreher J-C (2010) Separate valuation subsystems for delay and effort decision costs. J Neurosci 30:14080–14090

    Article  PubMed  Google Scholar 

  • Robbins TW, Everitt BJ (1999) Drug addiction: bad habits add up. Nature 398:567–570

    Article  CAS  PubMed  Google Scholar 

  • Rolls ET, Kringelbach ML, De Araujo IE (2003) Different representations of pleasant and unpleasant odours in the human brain. Eur J Neurosci 18:695–703

    Article  PubMed  Google Scholar 

  • Samejima K, Ueda Y, Doya K, Kimura M (2005) Representation of action-specific reward values in the striatum. Science 310:1337–1340

    Article  CAS  PubMed  Google Scholar 

  • Schmidt L, Lebreton M, ClĂ©ry-Melin M-L, Daunizeau J, Pessiglione M (2012) Neural mechanisms underlying motivation of mental versus physical effort. PLoS Biol 10:e1001266

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Small DM, Zatorre RJ, Dagher A, Evans AC, Jones-Gotman M (2001) Changes in brain activity related to eating chocolate: from pleasure to aversion. Brain 124:1720–1733

    Article  CAS  PubMed  Google Scholar 

  • Smith DV, Hayden BY, Truong T-K, Song AW, Platt ML, Huettel SA (2010) Distinct value signals in anterior and posterior ventromedial prefrontal cortex. J Neurosci 30:2490–2495

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Sohn JW, Lee D (2007) Order-dependent modulation of directional signals in the supplementary and presupplementary motor areas. J Neurosci 27:13655–13666

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Staudinger MR, Erk S, Abler B, Walter H (2009) Cognitive reappraisal modulates expected value and prediction error encoding in the ventral striatum. Neuroimage 47:713–721

    Article  PubMed  Google Scholar 

  • Talmi D, Seymour B, Dayan P, Dolan RJ (2008) Human Pavlovian-instrumental transfer. J Neurosci 28:360–368

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Tricomi E, Balleine BW, O’Doherty JP (2009) A specific role for posterior dorsolateral striatum in human habit learning. Eur J Neurosci 29:2225–2232

    Article  PubMed  PubMed Central  Google Scholar 

  • Voon V, Derbyshire K, RĂĽck C, Irvine M, Worbe Y, Enander J, Schreiber L, Gillan C, Fineberg N, Sahakian B (2014) Disorders of compulsivity: a common bias towards learning habits. Mol Psychiatry 20:345–352

    Google Scholar 

  • Watson P, Wiers R, Hommel B, de Wit S (2014) Working for food you don’t desire. Cues interfere with goal-directed food-seeking. Appetite 79:139–148

    Article  CAS  PubMed  Google Scholar 

  • Wunderlich K, Rangel A, O’Doherty JP (2009) Neural computations underlying action-based decision making in the human brain. Proc Natl Acad Sci U S A 106:17199–17204

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Wunderlich K, Dayan P, Dolan RJ (2012) Mapping value based planning and extensively trained choice in the human brain. Nat Neurosci 15:786–791

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. California Institute of Technology, Pasadena, USA

    John P. O’Doherty

Authors
  1. John P. O’Doherty
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John P. O’Doherty .

Editor information

Editors and Affiliations

  1. New York State Psychiatric Institute, Columbia University Medical Center, New York, New York, USA

    Eleanor H. Simpson

  2. Barnard College, Columbia University Barnard College, New York, New York, USA

    Peter D. Balsam

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

O’Doherty, J.P. (2015). Multiple Systems for the Motivational Control of Behavior and Associated Neural Substrates in Humans. In: Simpson, E., Balsam, P. (eds) Behavioral Neuroscience of Motivation. Current Topics in Behavioral Neurosciences, vol 27. Springer, Cham. https://doi.org/10.1007/7854_2015_386

Download citation

Publish with us

Policies and ethics

Search

Navigation