Skip to main content

From the Reward Circuit to the Valuation System: How the Brain Motivates Behavior

Abstract

In this chapter, we expose how behavioral economists, experimental psychologists, and cognitive neuroscientists joined their efforts and merged two fields of investigation: reward learning and choice behavior. This collaboration was made possible by technical progress—the availability of brain-imaging scanners and conceptual links—the use of motivational value as a key variable. We then present evidence that motivational values are encoded in a so-called brain valuation system (BVS), which essentially comprises the ventral parts of the prefrontal cortex and basal ganglia. Some fundamental properties of the BVS have been uncovered: it encodes values that are personal (subject- and not object-specific), generic (expressed in a common neuronal currency), and automatic (generated even during distractive tasks). Next, we show example situations where the BVS interacts with other brain systems (such as the perceptual, motor, executive, episodic, and mirror systems) that can impact on, or be impacted by, motivational values. These neural interactions might explain a number of psychological phenomena, for instance, incentive motivation (why we put so much effort in a task), delay discounting (why we can resist the temptation of immediate pleasures), or mimetic desires (why we often pursue the same goals as others). Last, we point to unsolved issues, such as how values are encoded at the single-cell level, how the value code incorporates uncertainty, how the values of different features are integrated, how the values are different options are compared, how negative values are represented relative to positive values, etc.

Keywords

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Aarts, H., Custers, R., & Marien, H. (2008). Preparing and motivating behavior outside of awareness. Science, 319(5870), 1639–1639. doi:10.1126/science.1150432.

    PubMed  Google Scholar 

  • Adam, R., Leff, A., Sinha, N., Turner, C., Bays, P., Draganski, B., & Husain, M. (2013). Dopamine reverses reward insensitivity in apathy following globus pallidus lesions. Cortex; A Journal Devoted to the Study of the Nervous System and Behavior, 49(5), 1292–1303. doi:10.1016/j.cortex.2012.04.013.

    PubMed  PubMed Central  Google Scholar 

  • Ariely, D., Loewenstein, G., & Prelec, D. (2003). “Coherent arbitrariness”: Stable demand curves without stable preferences. The Quarterly Journal of Economics, 118(1), 73–106. doi:10.1162/00335530360535153.

    Google Scholar 

  • Baliki, M. N., Chialvo, D. R., Geha, P. Y., Levy, R. M., Harden, R. N., Parrish, T. B., & Apkarian, A. V. (2006). Chronic pain and the emotional brain: Specific brain activity associated with spontaneous fluctuations of intensity of chronic back pain. The Journal of Neuroscience, 26(47), 12165–12173. doi:10.1523/JNEUROSCI.3576 -06.2006.

    PubMed  Google Scholar 

  • Barron, H. C., Dolan, R. J., & Behrens, T. E. J. (2013). Online evaluation of novel choices by simultaneous representation of multiple memories. Nature Neuroscience, 16(10), 1492–1498. doi:10.1038/nn.3515.

    PubMed  PubMed Central  Google Scholar 

  • Bartra, O., McGuire, J. T., & Kable, J. W. (2013). The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. NeuroImage, 76, 412–427. doi:10.1016/j.neuroimage.2013.02.063.

    PubMed  PubMed Central  Google Scholar 

  • Basten, U., Biele, G., Heekeren, H. R., & Fiebach, C. J. (2010). How the brain integrates costs and benefits during decision making. Proceedings of the National Academy of Sciences, 107(50), 21767–21772. doi:10.1073/pnas.0908104107.

    Google Scholar 

  • Bayliss, A. P., Paul, M. A., Cannon, P. R., & Tipper, S. P. (2006). Gaze cuing and affective judgments of objects: I like what you look at. Psychonomic Bulletin & Review, 13(6), 1061–1066. doi:10.3758/BF03213926.

    Google Scholar 

  • Benoit, R. G., Gilbert, S. J., & Burgess, P. W. (2011). A neural mechanism mediating the impact of episodic prospection on farsighted decisions. The Journal of Neuroscience, 31(18), 6771–6779. doi:10.1523/JNEUROSCI.6559-10.2011.

    PubMed  Google Scholar 

  • Bernoulli, D. (1738). Specimen theoriae novae de mensura sortis. Commentarii Academiae Scientiarum Imperialis Petropolitanae, 5, 175–192.

    Google Scholar 

  • Berns, G. S., McClure, S. M., Pagnoni, G., & Montague, P. R. (2001). Predictability modulates human brain response to reward. The Journal of Neuroscience, 21(8), 2793–2798.

    PubMed  Google Scholar 

  • Berridge, K. C. (2004). Motivation concepts in behavioral neuroscience. Physiology & Behavior, 81(2), 179–209. doi:10.1016/j.physbeh.2004.02.004.

    Google Scholar 

  • Bijleveld, E., Custers, R., & Aarts, H. (2010). Unconscious reward cues increase invested effort, but do not change speed-accuracy tradeoffs. Cognition, 115(2), 330–335. doi:10.1016/j.cognition.2009.12.012.

    PubMed  Google Scholar 

  • Bindra, D. (1978). How adaptive behavior is produced: A perceptual-motivational alternative to response-reinforcement. Behavioral and Brain Sciences, 1(01), 41–52.

    Google Scholar 

  • Blair, K., Marsh, A. A., Morton, J., Vythilingam, M., Jones, M., Mondillo, K., et al. (2006). Choosing the lesser of two evils, the better of two goods: Specifying the roles of ventromedial prefrontal cortex and dorsal anterior cingulate in object choice. The Journal of Neuroscience, 26(44), 11379–11386. doi:10.1523/JNEUROSCI.1640- 06.2006.

    PubMed  Google Scholar 

  • Bódi, N., Kéri, S., Nagy, H., Moustafa, A., Myers, C. E., Daw, N., et al.(2009). Reward-learning and the novelty-seeking personality: A between-and within-subjects study of the effects of dopamine agonists on young Parkinson’s patients. Brain: A Journal of Neurology, 132(9), 2385–2395. doi:10.1093/brain/awp094.

    Google Scholar 

  • Bolles, R. C. (1972). Reinforcement, expectancy, and learning. Psychological Review, 79(5), 394. doi:10.1037/h0033120.

    Google Scholar 

  • Bouret, S., & Richmond, B. J. (2010). Ventromedial and orbital prefrontal neurons differentially encode internally and externally driven motivational values in monkeys. The Journal of Neuroscience, 30(25), 8591–8601. doi:10.1523/JNEUROSCI.0049-10.2010.

    PubMed  PubMed Central  Google Scholar 

  • Boyer, P. (2008). Evolutionary economics of mental time travel? Trends in Cognitive Sciences, 12(6), 219–224. doi:10.1016/j.tics.2008.03.003..

    PubMed  Google Scholar 

  • Cabanac, M. (1992). Pleasure: The common currency. Journal of Theoretical Biology, 155(2), 173–200. doi:10.1016/S0022-5193(05)80594-6.

    PubMed  Google Scholar 

  • Camerer, C. F., Loewenstein, G., & Prelec, D. (2004). Neuroeconomics: Why economics needs brains. The Scandinavian Journal of Economics, 106(3), 555–579. doi:10.1111/j.0347-0520.2004.00377.x..

    Google Scholar 

  • Camille, N., Griffiths, C. A., Vo, K., Fellows, L. K., & Kable, J. W. (2011). Ventromedial frontal lobe damage disrupts value maximization in humans. The Journal of Neuroscience, 31(20), 7527–7532. doi:10.1523/JNEUROSCI.6527-10.2011.

    PubMed  PubMed Central  Google Scholar 

  • Campbell-Meiklejohn, D. K., Bach, D. R., Roepstorff, A., Dolan, R. J., & Frith, C. D. (2010). How the opinion of others affects our valuation of objects. Current Biology, 20(13), 1165–1170. doi:10.1016/j.cub.2010.04.055..

    PubMed  PubMed Central  Google Scholar 

  • Chib, V. S., Rangel, A., Shimojo, S., & O’Doherty, J. P. (2009). Evidence for a common representation of decision values for dissimilar goods in human ventromedial prefrontal cortex. The Journal of Neuroscience, 29(39), 12315–12320. doi:10.1523/JNEUROSCI.2575-09.2009.

    PubMed  Google Scholar 

  • Craig, A. D., Reiman, E. M., Evans, A., & Bushnell, M. C. (1996). Functional imaging of an illusion of pain. Nature, 384(6606), 258–260. doi:10.1038/384258a0.

    PubMed  Google Scholar 

  • Craig, A. D., Chen, K., Bandy, D., & Reiman, E. M. (2000). Thermosensory activation of insular cortex. Nature Neuroscience, 3(2), 184–190. doi:10.1038/72131.

    PubMed  Google Scholar 

  • Critchley, H. D., Mathias, C. J., & Dolan, R. J. (2001). Neural activity in the human brain relating to uncertainty and arousal during anticipation. Neuron, 29(2), 537–545. doi:10.1016/S0896-6273(01)00225-2.

    PubMed  Google Scholar 

  • De Martino, B., Fleming, S. M., Garrett, N., & Dolan, R. J. (2013). Confidence in value-based choice. Nature Neuroscience, 16(1), 105–110. doi:10.1038/nn.3279.

    PubMed  PubMed Central  Google Scholar 

  • Dickinson, A., & Balleine, B. (1994). Motivational control of goal-directed action. Animal Learning & Behavior, 22(1), 1–18. doi:10.3758/BF03199951.

    Google Scholar 

  • Fehr, E., & Rangel, A. (2011). Neuroeconomic foundations of economic choice-recent advances. Journal of Economic Perspectives, 25(4), 3–30. doi:10.1257/089533011798394532.

    Google Scholar 

  • Fellows, L. K., & Farah, M. J. (2007). The role of ventromedial prefrontal cortex in decision making: Judgment under uncertainty or judgment per se? Cerebral Cortex, 17(11), 2669–2674. doi:10.1093/cercor/bhl176.

    PubMed  Google Scholar 

  • FitzGerald, T. H., Friston, K. J., & Dolan, R. J. (2012). Action-specific value signals in reward-related regions of the human brain. The Journal of Neuroscience, 32(46), 16417–16423. doi:10.1523/JNEUROSCI.3254-12.2012.

    PubMed  PubMed Central  Google Scholar 

  • Fox, M. D., Snyder, A. Z., Vincent, J. L., & Raichle, M. E. (2007). Intrinsic fluctuations within cortical systems account for intertrial variability in human behavior. Neuron, 56(1), 171–184. doi:10.1016/j.neuron.2007.08.023.

    PubMed  Google Scholar 

  • Frank, M. J., Seeberger, L. C., & O’Reilly, R. C. (2004). By carrot or by stick: Cognitive reinforcement learning in parkinsonism. Science, 306(5703), 1940–1943. doi:10.1126/science.1102941.

    PubMed  Google Scholar 

  • Frith, C. D., & Frith, U. (1999). Interacting minds-a biological basis. Science, 286(5445), 1692–1695. doi:10.1126/science.286.5445.1692.

    PubMed  Google Scholar 

  • Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, G. (1996). Action recognition in the premotor cortex. Brain: A Journal of Neurology, 119(2), 593–609. doi:10.1093/brain/119.2.593.

    Google Scholar 

  • Georgopoulos, A. P., Schwartz, A. B., & Kettner, R. E. (1986). Neuronal population coding of movement direction. Science, 233(4771), 1416–1419. doi:10.1126/science.3749885.

    PubMed  Google Scholar 

  • Gershman, S. J., Pesaran, B., & Daw, N. D. (2009). Human reinforcement learning subdivides structured action spaces by learning effector-specific values. The Journal of Neuroscience, 29(43), 13524–13531. doi:10.1523/JNEUROSCI.2469-09.2009.

    PubMed  PubMed Central  Google Scholar 

  • Girard, R. (1979). Violence and the sacred. Baltimore: Johns Hopkins University Press.

    Google Scholar 

  • Glimcher, P. W. (2009). Neuroeconomics: Decision making and the brain. Amsterdam: Academic. (http://site.ebrary.com/lib/alltitles/docDetail.action?docID=10254642)

    Google Scholar 

  • Gold, J. I., & Shadlen, M. N. (2007). The neural basis of decision making. Annual Review of Neuroscience, 30(1), 535–574. doi:10.1146/annurev.neuro.29.051605.113038.

    PubMed  Google Scholar 

  • Gottfried, J. A., O’Doherty, J., & Dolan, R. J. (2003). Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science, 301(5636), 1104–1107. doi:10.1126/science.1087919.

    PubMed  Google Scholar 

  • Grabenhorst, F., Rolls, E. T., Margot, C., da Silva, M. A. A. P., & Velazco, M. I. (2007). How pleasant and unpleasant stimuli combine in different brain regions: Odor mixtures. The Journal of Neuroscience, 27(49), 13532–13540. doi:10.1523/JNEUROSCI.3337- 07.2007.

    PubMed  Google Scholar 

  • Haber, S. N., & Knutson, B. (2009). The reward circuit: Linking primate anatomy and human imaging. Neuropsychopharmacology: Official Publication of the American College of Neuropsychopharmacology, 35(1), 4–26. doi:10.1038/npp. 2009.129.

    Google Scholar 

  • Hare, T. A., O’Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008). Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. The Journal of Neuroscience, 28(22), 5623–5630. doi:10.1523/JNEUROSCI.1309-UROSCI.

    PubMed  Google Scholar 

  • Hare, T. A., Camerer, C. F., & Rangel, A. (2009). Self-control in decision-making involves modulation of the VMPFC valuation system. Science, 324(5927), 646–648. doi:10.1126/science.1168450.

    PubMed  Google Scholar 

  • Hare, T. A., Camerer, C. F., Knoepfle, D. T., O’Doherty, J. P., & Rangel, A. (2010). Value computations in ventral medial prefrontal cortex during charitable decision making incorporate input from regions involved in social cognition. The Journal of Neuroscience, 30(2), 583–590. doi:10.1523/JNEUROSCI.4089-09.2010.

    PubMed  Google Scholar 

  • Hare, T. A., Malmaud, J., & Rangel, A. (2011a). Focusing attention on the health aspects of foods changes value signals in VMPFC and improves dietary choice. The Journal of Neuroscience, 31(30), 11077–11087. doi:10.1523/JNEUROSCI.6383-10.2011.

    Google Scholar 

  • Hare, T. A., Schultz, W., Camerer, C. F., O’Doherty, J. P., & Rangel, A. (2011b). Transformation of stimulus value signals into motor commands during simple choice. Proceedings of the National Academy of Sciences, 108(44), 18120–18125. doi:10.1073/pnas.1109322108.

    Google Scholar 

  • Harvey, A. H., Kirk, U., Denfield, G. H., & Montague, P. R. (2010). Monetary favors and their influence on neural responses and revealed preference. The Journal of Neuroscience, 30(28), 9597–9602. doi:10.1523/JNEUROSCI.1086-10.2010.

    PubMed  PubMed Central  Google Scholar 

  • Hassabis, D., Kumaran, D., Vann, S. D., & Maguire, E. A. (2007). Patients with hippocampal amnesia cannot imagine new experiences. Proceedings of the National Academy of Sciences, 104(5), 1726–1731. doi:10.1073/pnas.0610561104.

    Google Scholar 

  • Hesselmann, G., Kell, C. A., Eger, E., & Kleinschmidt, A. (2008a). Spontaneous local variations in ongoing neural activity bias perceptual decisions. Proceedings of the National Academy of Sciences, 105(31), 10984–10989. doi:10.1073/pnas.0712043105.

    Google Scholar 

  • Hesselmann, G., Kell, C. A., & Kleinschmidt, A. (2008b). Ongoing activity fluctuations in hMT+ bias the perception of coherent visual motion. The Journal of Neuroscience, 28(53), 14481–14485. doi:10.1523/JNEUROSCI.4398-08.2008.

    Google Scholar 

  • Hikosaka, K., & Watanabe, M. (2000). Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards. Cerebral Cortex, 10(3), 263–271. doi:10.1093/cercor/10.3.263.

    PubMed  Google Scholar 

  • Hikosaka, O., Sakamoto, M., & Usui, S. (1989). Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. Journal of Neurophysiology, 61(4), 814–832.

    PubMed  Google Scholar 

  • Hunt, L. T., Kolling, N., Soltani, A., Woolrich, M. W., Rushworth, M. F. S., & Behrens, T. E. J. (2012). Mechanisms underlying cortical activity during value-guided choice. Nature Neuroscience, 15(3), 470–476. doi:10.1038/nn.3017.

    PubMed  PubMed Central  Google Scholar 

  • Hutcherson, C. A., Plassmann, H., Gross, J. J., & Rangel, A. (2012). Cognitive regulation during decision making shifts behavioral control between ventromedial and dorsolateral prefrontal value systems. The Journal of Neuroscience, 32(39), 13543–13554. doi:10.1523/JNEUROSCI.6387-11.2012.

    PubMed  PubMed Central  Google Scholar 

  • Imperato, A., Mulas, A., & Di Chiara, G. (1986). Nicotine preferentially stimulates dopamine release in the limbic system of freely moving rats. European Journal of Pharmacology, 132(2), 337–338.

    PubMed  Google Scholar 

  • Kable, J. W., & Glimcher, P. W. (2007). The neural correlates of subjective value during intertemporal choice. Nature Neuroscience, 10(12), 1625–1633. doi:10.1038/nn2007.

    PubMed  PubMed Central  Google Scholar 

  • Kable, J. W., & Glimcher, P. W. (2009). The neurobiology of decision: Consensus and controversy. Neuron, 63(6), 733–745. doi:10.1016/j.neuron.2009.09.003.

    PubMed  PubMed Central  Google Scholar 

  • Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263–291. doi:10.2307/1914185.

    Google Scholar 

  • Kennerley, S. W., Behrens, T. E. J., & Wallis, J. D. (2011). Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nature Neuroscience, 14(12), 1581–1589. doi:10.1038/nn.2961.

    PubMed  PubMed Central  Google Scholar 

  • Kim, H., Shimojo, S., & O’Doherty, J. P. (2011). Overlapping responses for the expectation of juice and money rewards in human ventromedial prefrontal cortex. Cerebral Cortex, 21(4), 769–776. doi:10.1093/cercor/bhq145.

    PubMed  Google Scholar 

  • Klucharev, V., Hytönen, K., Rijpkema, M., Smidts, A., & Fernández, G. (2009). Reinforcement learning signal predicts social conformity. Neuron, 61(1), 140–151. doi:10.1016/j.neuron.2008.11.027.

    PubMed  Google Scholar 

  • Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001). Anticipation of increasing monetary reward selectively recruits nucleus accumbens. The Journal of Neuroscience, 21(16), RC159.

    PubMed  Google Scholar 

  • Knutson, B., Fong, G. W., Bennett, S. M., Adams, C. M., & Hommer, D. (2003). A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: Characterization with rapid event-related fMRI. NeuroImage, 18(2), 263–272. doi:10.1016/S1053-8119(02)00057-5.

    PubMed  Google Scholar 

  • Kouneiher, F., Charron, S., & Koechlin, E. (2009). Motivation and cognitive control in the human prefrontal cortex. Nature Neuroscience, 12(7), 939–945. doi:10.1038/nn.2321.

    PubMed  Google Scholar 

  • Krajbich, I., Armel, C., & Rangel, A. (2010). Visual fixations and the computation and comparison of value in simple choice. Nature Neuroscience, 13(10), 1292–1298. doi:10.1038/nn.2635.

    PubMed  Google Scholar 

  • Kringelbach, M. L. (2005). The human orbitofrontal cortex: Linking reward to hedonic experience. Nature Reviews Neuroscience, 6(9), 691–702. doi:10.1038/nrn1747.

    PubMed  Google Scholar 

  • Lauwereyns, J., Watanabe, K., Coe, B., & Hikosaka, O. (2002). A neural correlate of response bias in monkey caudate nucleus. Nature, 418(6896), 413–417. doi:10.1038/nature00892.

    PubMed  Google Scholar 

  • Laxton, A. W., Neimat, J. S., Davis, K. D., Womelsdorf, T., Hutchison, W. D., Dostrovsky, J. O., et al. (2013). Neuronal coding of implicit emotion categories in the subcallosal cortex in patients with depression. Biological Psychiatry, 74(10), 714–719. doi:10.1016/j.biopsych.2013.03.029.

    PubMed  Google Scholar 

  • Le Bouc, R., & Pessiglione, M. (2013). Imaging social motivation: Distinct brain mechanisms drive effort production during collaboration versus competition. The Journal of Neuroscience, 33(40), 15894–15902. doi:10.1523/JNEUROSCI.0143-13.2013.

    PubMed  Google Scholar 

  • Lebreton, M., Jorge, S., Michel, V., Thirion, B., & Pessiglione, M. (2009). An automatic valuation system in the human brain: Evidence from functional neuroimaging. Neuron, 64(3), 431–439. doi:10.1016/j.neuron.2009.09.040..

    PubMed  Google Scholar 

  • Lebreton, M., Kawa, S., d’ Arc, B. F., Daunizeau, J., & Pessiglione, M. (2012). Your goal is mine: Unraveling mimetic desires in the human brain. The Journal of Neuroscience, 32(21), 7146–7157. doi:10.1523/JNEUROSCI.4821-11.2012.

    PubMed  Google Scholar 

  • Lebreton, M., Bertoux, M., Boutet, C., Lehericy, S., Dubois, B., Fossati, P., & Pessiglione, M. (2013). A critical role for the hippocampus in the valuation of imagined outcomes. PLoS Biology, 11(10), e1001684. doi:10.1371/journal.pbio.1001684..

    PubMed  PubMed Central  Google Scholar 

  • Leone, P., Pocock, D., & Wise, R. A. (1991). Morphine-dopamine interaction: Ventral tegmental morphine increases nucleus accumbens dopamine release. Pharmacology Biochemistry and Behavior, 39(2), 469–472. doi:10.1016/0091-3057(91)90210-S.

    Google Scholar 

  • Levy, D. J., & Glimcher, P. W. (2011). Comparing apples and oranges: Using reward-specific and reward-general subjective value representation in the brain. The Journal of Neuroscience, 31(41), 14693–14707. doi:10.1523/JNEUROSCI.2218-11.2011.

    PubMed  PubMed Central  Google Scholar 

  • Levy, D. J., & Glimcher, P. W. (2012). The root of all value: A neural common currency for choice. Current Opinion in Neurobiology, 22(6), 1027–1038. doi:10.1016/j.conb.2012.06.001.

    PubMed  PubMed Central  Google Scholar 

  • Levy, I., Lazzaro, S. C., Rutledge, R. B., & Glimcher, P. W. (2011). Choice from non-choice: Predicting consumer preferences from blood oxygenation level-dependent signals obtained during passive viewing. The Journal of Neuroscience, 31(1), 118–125. doi:10.1523/JNEUROSCI.3214-10.2011.

    PubMed  PubMed Central  Google Scholar 

  • Lim, S.-L., O’Doherty, J. P., & Rangel, A. (2011). The decision value computations in the VMPFC and striatum use a relative value code that is guided by visual attention. The Journal of Neuroscience, 31(37), 13214–13223. doi:10.1523/JNEUROSCI.1246-11.2011.

    PubMed  Google Scholar 

  • Ljungberg, T., Apicella, P., & Schultz, W. (1992). Responses of monkey dopamine neurons during learning of behavioral reactions. Journal of Neurophysiology, 67(1), 145–163.

    PubMed  Google Scholar 

  • Logothetis, N. K. (2008). What we can do and what we cannot do with fMRI. Nature, 453(7197), 869–878. doi:10.1038/nature06976.

    PubMed  Google Scholar 

  • Louie, K., & Glimcher, P. W. (2012). Efficient coding and the neural representation of value. Annals of the New York Academy of Sciences, 1251(1), 13–32. doi:10.1111/j.1749-6632.2012.06496.x.

    PubMed  Google Scholar 

  • McClure, S. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2004a). Separate neural systems value immediate and delayed monetary rewards. Science, 306(5695), 503–507. doi:10.1126/science.1100907.

    Google Scholar 

  • McClure, S. M., Li, J., Tomlin, D., Cypert, K. S., Montague, L. M., & Montague, P. R. (2004b). Neural correlates of behavioral preference for culturally familiar drinks. Neuron, 44(2), 379–387. doi:10.1016/j.neuron.2004.09.019.

    Google Scholar 

  • McClure, S. M., Ericson, K. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2007). Time discounting for primary rewards. The Journal of Neuroscience, 27(21), 5796–5804. doi:10.1523/JNEUROSCI.4246-06.2007.

    PubMed  Google Scholar 

  • McNamee, D., Rangel, A., & O’Doherty, J. P. (2013). Category-dependent and category-independent goal-value codes in human ventromedial prefrontal cortex. Nature Neuroscience, 16(4), 479–485. doi:10.1038/nn.3337.

    PubMed  PubMed Central  Google Scholar 

  • Mirenowicz, J., & Schultz, W. (1994). Importance of unpredictability for reward responses in primate dopamine neurons. Journal of Neurophysiology, 72(2), 1024–1027.

    PubMed  Google Scholar 

  • Montague, P. R., & Berns, G. S. (2002). Neural economics and the biological substrates of valuation. Neuron, 36(2), 265–284. doi:10.1016/S0896-6273(02)00974-1.

    PubMed  Google Scholar 

  • Morishima, Y., Schunk, D., Bruhin, A., Ruff, C. C., & Fehr, E. (2012). Linking brain structure and activation in temporoparietal junction to explain the neurobiology of human altruism. Neuron, 75(1), 73–79. doi:10.1016/j.neuron.2012.05.021.

    PubMed  Google Scholar 

  • O’Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001). Abstract reward and punishment representations in the human orbitofrontal cortex. Nature Neuroscience, 4(1), 95–102. doi:10.1038/82959.

    PubMed  Google Scholar 

  • Olds, J., & Milner, P. (1954). Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. Journal of Comparative and Physiological Psychology, 47(6), 419–427. doi:10.1037/h0058775.

    PubMed  Google Scholar 

  • Padoa-Schioppa, C. (2011). Neurobiology of economic choice: A good-based model. Annual Review of Neuroscience, 34, 333. doi:10.1146/annurev-neuro-061010-113648.

    PubMed  PubMed Central  Google Scholar 

  • Padoa-Schioppa, C., & Assad, J. A. (2006). Neurons in the orbitofrontal cortex encode economic value. Nature, 441(7090), 223–226. doi:10.1038/nature04676.

    PubMed  PubMed Central  Google Scholar 

  • Pagnoni, G., Zink, C. F., Montague, P. R., & Berns, G. S. (2002). Activity in human ventral striatum locked to errors of reward prediction. Nature Neuroscience, 5(2), 97–98. doi:10.1038/nn802.

    PubMed  Google Scholar 

  • Palminteri, S., Lebreton, M., Worbe, Y., Grabli, D., Hartmann, A., & Pessiglione, M. (2009). Pharmacological modulation of subliminal learning in Parkinson’s and Tourette’s syndromes. Proceedings of the National Academy of Sciences, 106(45), 19179–19184. doi:10.1073/pnas.0904035106.

    Google Scholar 

  • Palminteri, S., Justo, D., Jauffret, C., Pavlicek, B., Dauta, A., Delmaire, C., et al. (2012). Critical roles for anterior insula and dorsal striatum in punishment-based avoidance learning. Neuron, 76(5), 998–1009. doi:10.1016/j.neuron.2012.10.017.

    PubMed  Google Scholar 

  • Pascal, B. (1669). Pensées. Paris: G. Desprez. (http://gallica.bnf.fr/ark:/12148/btv1b8606964f).

    Google Scholar 

  • Pasquereau, B., Nadjar, A., Arkadir, D., Bezard, E., Goillandeau, M., Bioulac, B., et al. (2007). Shaping of motor responses by incentive values through the basal ganglia. The Journal of Neuroscience, 27(5), 1176–1183. doi:10.1523/JNEUROSCI.3745- 06.2007.

    PubMed  Google Scholar 

  • Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006). Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature, 442(7106), 1042–1045. doi:10.1038/nature05051.

    PubMed  PubMed Central  Google Scholar 

  • Pessiglione, M., Schmidt, L., Draganski, B., Kalisch, R., Lau, H., Dolan, R. J., & Frith, C. D. (2007). How the brain translates money into force: A neuroimaging study of subliminal motivation. Science, 316(5826), 904–906. doi:10.1126/science.1140459.

    PubMed  PubMed Central  Google Scholar 

  • Pessiglione, M., Petrovic, P., Daunizeau, J., Palminteri, S., Dolan, R. J., & Frith, C. D. (2008). Subliminal instrumental conditioning demonstrated in the human brain. Neuron, 59(4), 561–567. doi:10.1016/j.neuron.2008.07.005.

    PubMed  PubMed Central  Google Scholar 

  • Peters, J., & Büchel, C. (2010a). Episodic future thinking reduces reward delay discounting through an enhancement of prefrontal-mediotemporal interactions. Neuron, 66(1), 138–148. doi:10.1016/j.neuron.2010.03.026.

    Google Scholar 

  • Peters, J., & Büchel, C. (2010b). Neural representations of subjective reward value. Behavioural Brain Research, 213(2), 135–141. doi:10.1016/j.bbr.2010.04.031.

    Google Scholar 

  • Plassmann, H., O’Doherty, J., & Rangel, A. (2007). Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. The Journal of Neuroscience, 27(37), 9984–9988. doi:10.1523/JNEUROSCI.2131- 07.2007.

    PubMed  Google Scholar 

  • Plassmann, H., O’Doherty, J. P., & Rangel, A. (2010). Appetitive and aversive goal values are encoded in the medial orbitofrontal cortex at the time of decision making. The Journal of Neuroscience, 30(32), 10799–10808. doi:10.1523/JNEUROSCI.0788-10.2010.

    PubMed  Google Scholar 

  • Platt, M. L., & Glimcher, P. W. (1999). Neural correlates of decision variables in parietal cortex. Nature, 400(6741), 233–238. doi:10.1038/22268.

    PubMed  Google Scholar 

  • Pouget, A., Dayan, P., & Zemel, R. (2000). Information processing with population codes. Nature Reviews Neuroscience, 1(2), 125–132. doi:10.1038/35039062.

    PubMed  Google Scholar 

  • Preuschoff, K., Bossaerts, P., & Quartz, S. R. (2006). Neural differentiation of expected reward and risk in human subcortical structures. Neuron, 51(3), 381–390. doi:10.1016/j.neuron.2006.06.024.

    PubMed  Google Scholar 

  • Prévost, C., Pessiglione, M., Météreau, E., Cléry-Melin, M.-L., & Dreher, J.-C. (2010). Separate valuation subsystems for delay and effort decision costs. The Journal of Neuroscience, 30(42), 14080–14090. doi:10.1523/JNEUROSCI.2752-10.2010.

    PubMed  Google Scholar 

  • Rangel, A., & Clithero, J. A. (2012). Value normalization in decision making: Theory and evidence. Current Opinion in Neurobiology, 22(6), 970–981. doi:10.1016/j.conb.2012.07.011.

    PubMed  Google Scholar 

  • Rangel, A., & Hare, T. (2010). Neural computations associated with goal-directed choice. Current Opinion in Neurobiology, 20(2), 262–270. doi:10.1016/j.conb.2010.03.001.

    PubMed  Google Scholar 

  • Rangel, A., Camerer, C., & Montague, P. R. (2008). A framework for studying the neurobiology of value-based decision making. Nature Reviews Neuroscience, 9(7), 545–556. doi:10.1038/nrn2357.

    PubMed  Google Scholar 

  • Ratcliff, R., & Rouder, J. N. (1998). Modeling response times for two-choice decisions. Psychological Science, 9(5), 347–356. doi:10.1111/1467-9280.00067.

    Google Scholar 

  • Rescorla, R., & Wagner, A. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. Black & W. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York: Appleton-Century-Crofts.

    Google Scholar 

  • Rick, S., & Loewenstein, G. (2008). Intangibility in intertemporal choice. Philosophical Transactions of the Royal Society B: Biological Sciences, 363(1511), 3813–3824. doi:10.1098/rstb.2008.0150.

    Google Scholar 

  • Rizzolatti, G., Fadiga, L., Gallese, V., & Fogassi, L. (1996). Premotor cortex and the recognition of motor actions. Cognitive Brain Research, 3(2), 131–141. doi:10.1016/0926-6410(95)00038-0.

    PubMed  Google Scholar 

  • Rizzolatti, G., Fogassi, L., & Gallese, V. (2001). Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Reviews Neuroscience, 2(9), 661–670. doi:10.1038/35090060.

    PubMed  Google Scholar 

  • Rochat, L., Linden, M. V., der Renaud, O., Epiney, J.-B., Michel, P., Sztajzel, R., et al. (2013). Poor reward sensitivity and apathy after stroke Implication of basal ganglia. Neurology, 81(19), 1674–1680. doi:10.1212/01.wnl.0000435290.49598.1d.

    PubMed  Google Scholar 

  • Rushworth, M. F., Kolling, N., Sallet, J., & Mars, R. B. (2012). Valuation and decision-making in frontal cortex: One or many serial or parallel systems? Current Opinion in Neurobiology, 22(6), 946–955. doi:10.1016/j.conb.2012.04.011.

    PubMed  Google Scholar 

  • Sadaghiani, S., Hesselmann, G., & Kleinschmidt, A. (2009). Distributed and antagonistic contributions of ongoing activity fluctuations to auditory stimulus detection. The Journal of Neuroscience, 29(42), 13410–13417. doi:10.1523/JNEUROSCI.2592-09.2009.

    PubMed  Google Scholar 

  • Salimpoor, V. N., Bosch, I., van den Kovacevic, N., McIntosh, A. R., Dagher, A., & Zatorre, R. J. (2013). Interactions between the nucleus accumbens and auditory cortices predict music reward value. Science, 340(6129), 216–219. doi:10.1126/science.1231059.

    PubMed  Google Scholar 

  • Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005). Representation of action-specific reward values in the striatum. Science, 310(5752), 1337–1340. doi:10.1126/science.1115270.

    PubMed  Google Scholar 

  • Samuelson, P. A. (1938). A note on the pure theory of consumer’s behaviour. Economica, 5(17), 61–71.

    Google Scholar 

  • Savage, L. J. (1954). The foundations of statistics. New York: Courier Dover.

    Google Scholar 

  • Saxe, R. (2006). Uniquely human social cognition. Current Opinion in Neurobiology, 16(2), 235–239. doi:10.1016/j.conb.2006.03.001.

    PubMed  Google Scholar 

  • Schacter, D. L., Addis, D. R., & Buckner, R. L. (2007). Remembering the past to imagine the future: The prospective brain. Nature Reviews Neuroscience, 8(9), 657–661. doi:10.1038/nrn2213.

    PubMed  Google Scholar 

  • Schmidt, L., d’ Arc, B. F., Lafargue, G., Galanaud, D., Czernecki, V., Grabli, D., et al. (2008). Disconnecting force from money: Effects of basal ganglia damage on incentive motivation. Brain: A Journal of Neurology, 131(5), 1303–1310. doi:10.1093/brain/awn045.

    Google Scholar 

  • Schmidt, L., Cléry-Melin, M.-L., Lafargue, G., Valabrègue, R., Fossati, P., Dubois, B., & Pessiglione, M. (2009). Get aroused and be stronger: Emotional facilitation of physical effort in the human brain. The Journal of Neuroscience, 29(30), 9450–9457. doi:10.1523/JNEUROSCI.1951-09.2009.

    PubMed  Google Scholar 

  • Schmidt, L., Lebreton, M., Cléry-Melin, M.-L., Daunizeau, J., & Pessiglione, M. (2012). Neural mechanisms underlying motivation of mental versus physical effort. PLoS Biology, 10(2), e1001266. doi:10.1371/journal.pbio.1001266.

    PubMed  PubMed Central  Google Scholar 

  • Schultz, W., Apicella, P., & Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. The Journal of Neuroscience, 13(3), 900–913.

    PubMed  Google Scholar 

  • Seitz, A. R., Kim, D., & Watanabe, T. (2009). Rewards evoke learning of unconsciously processed visual stimuli in adult humans. Neuron, 61(5), 700–707. doi:10.1016/j.neuron.2009.01.016.

    PubMed  PubMed Central  Google Scholar 

  • Sescousse, G., Caldú, X., Segura, B., & Dreher, J.-C. (2013). Processing of primary and secondary rewards: A quantitative meta-analysis and review of human functional neuroimaging studies. Neuroscience & Biobehavioral Reviews, 37(4), 681–696. doi:10.1016/j.neubiorev.2013.02.002.

    Google Scholar 

  • Seymour, B., O’Doherty, J. P., Dayan, P., Koltzenburg, M., Jones, A. K., Dolan, R. J., et al. (2004). Temporal difference models describe higher-order learning in humans. Nature, 429(6992), 664–667. doi:10.1038/nature02581.

    PubMed  Google Scholar 

  • Shadlen, M. N., & Newsome, W. T. (2001). Neural basis of a perceptual decision in the parietal cortex (Area LIP) of the rhesus monkey. Journal of Neurophysiology, 86(4), 1916–1936.

    PubMed  Google Scholar 

  • Sheffield, F. D., & Roby, T. B. (1950). Reward value of a non-nutritive sweet taste. Journal of Comparative and Physiological Psychology, 43(6), 471–481. doi:10.1037/h0061365.

    PubMed  Google Scholar 

  • Small, D. M., Zatorre, R. J., Dagher, A., Evans, A. C., & Jones-Gotman, M. (2001). Changes in brain activity related to eating chocolate. From pleasure to aversion. Brain: A Journal of Neurology, 124(9), 1720–1733. doi:10.1093/brain/124.9.1720.

    Google Scholar 

  • Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge: Cambridge University Press.

    Google Scholar 

  • Thorndike, E. L. (1911). Animal intelligence: Experimental studies. New York: Macmillan. (http://archive.org/details/animalintelligen00thor)

    Google Scholar 

  • Thorpe, S. J., Rolls, D. E. T., & Maddison, S. (1983). The orbitofrontal cortex: Neuronal activity in the behaving monkey. Experimental Brain Research, 49(1), 93–115. doi:10.1007/BF00235545.

    PubMed  Google Scholar 

  • Toates, F. M. (1986). Motivational systems (Vol. 4). Cambridge: Cambridge University Press.

    Google Scholar 

  • Tremblay, L., & Schultz, W. (1999). Relative reward preference in primate orbitofrontal cortex. Nature, 398(6729), 704–708. doi:10.1038/19525.

    PubMed  Google Scholar 

  • Tusche, A., Bode, S., & Haynes, J.-D. (2010). Neural responses to unattended products predict later consumer choices. The Journal of Neuroscience, 30(23), 8024–8031. doi:10.1523/JNEUROSCI.0064-10.2010.

    PubMed  Google Scholar 

  • Von Neumann, J., & Morgenstern, O. (1944). Game theory and economic behavior. Princeton: Princeton University Press.

    Google Scholar 

  • Wallis, J. D., & Miller, E. K. (2003). Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. European Journal of Neuroscience, 18(7), 2069–2081. doi:10.1046/j.1460-9568.2003.02922.x.

    PubMed  Google Scholar 

  • Wise, R. A., Leone, P., Rivest, R., & Leeb, K. (1995). Elevations of nucleus accumbens dopamine and DOPAC levels during intravenous heroin self-administration. Synapse, 21(2), 140–148.

    PubMed  Google Scholar 

  • Wunderlich, K., Rangel, A., & O’Doherty, J. P. (2009). Neural computations underlying action-based decision making in the human brain. Proceedings of the National Academy of Sciences, 106(40), 17199–17204. doi:10.1073/pnas.0901077106.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mathias Pessiglione PhD .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media New York

About this chapter

Cite this chapter

Pessiglione, M., Lebreton, M. (2015). From the Reward Circuit to the Valuation System: How the Brain Motivates Behavior. In: Gendolla, G., Tops, M., Koole, S. (eds) Handbook of Biobehavioral Approaches to Self-Regulation. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-1236-0_11

Download citation

Publish with us

Policies and ethics