The Computational Complexity of Valuation and Motivational Forces in Decision-Making Processes

  • A. David Redish
  • Nathan W. Schultheiss
  • Evan C. Carter
Part of the Current Topics in Behavioral Neurosciences book series (CTBN, volume 27)

Abstract

The concept of value is fundamental to most theories of motivation and decision making. However, value has to be measured experimentally. Different methods of measuring value produce incompatible valuation hierarchies. Taking the agent’s perspective (rather than the experimenter’s), we interpret the different valuation measurement methods as accessing different decision-making systems and show how these different systems depend on different information processing algorithms. This identifies the translation from these multiple decision-making systems into a single action taken by a given agent as one of the most important open questions in decision making today. We conclude by looking at how these different valuation measures accessing different decision-making systems can be used to understand and treat decision dysfunction such as in addiction.

Keywords

Neuroeconomonics Valuation Multiple Decision Theory Decision-Making 

References

  1. Ahmed SH (2010) Validation crisis in animal models of drug addiction: beyond non-disordered drug use toward drug addiction. Neurosci Biobehav Rev 35(2):172–184PubMedCrossRefGoogle Scholar
  2. Andrade EB, Ariely D (2009) The enduring impact of transient emotions on decision making. Organ Behav Hum Decis Process 109(1):1–8CrossRefGoogle Scholar
  3. Ariely D, Loewenstein G (2006) The heat of the moment: the effect of sexual arousal on sexual decision making. J Behav Decis Mak 19:87–98CrossRefGoogle Scholar
  4. Atance CM, O’Neill DK (2001) Episodic future thinking. Trends Cogn Sci 5(12):533–539PubMedCrossRefGoogle Scholar
  5. Balleine BW, Dickinson A (1998) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37(4–5):407–419PubMedCrossRefGoogle Scholar
  6. Bandler R, Shipley MT (1994) Columnar organization in the midbrain periaqueductal gray: modules for emotional expression? Trends Neurosci 17(9):379–389PubMedCrossRefGoogle Scholar
  7. Benoit RG, Gilbert SJ, Burgess PW (2011) A neural mechanism mediating the impact of episodic prospection on farsighted decisions. J Neurosci 31(18):6771–6779PubMedCrossRefGoogle Scholar
  8. Bernstein IL (1978) Learned taste aversions in children receiving chemotherapy. Science 200(4347):1302–1303PubMedCrossRefGoogle Scholar
  9. Bernstein IL (1999) Taste aversion learning: a contemporary perspective. Nutrition 15(3):229–234PubMedCrossRefGoogle Scholar
  10. Bickel WK, Yi R, Landes RD, Hill PF, Baxter C (2011) Remember the future: working memory training decreases delay discounting among stimulant addicts. Biol Psychiatry 69(3):260–265Google Scholar
  11. Bouton ME (2007) Learning and behavior: a contemporary synthesis. Sinauer Associates, MassachusettsGoogle Scholar
  12. Breland K, Breland M (1961) The misbehavior of organisms. Am Psychol 16(11):682–684CrossRefGoogle Scholar
  13. Bruner N, Johnson M (2013) Demand curves for hypothetical cocaine in cocaine-dependent individuals. Psychopharmacology 1–9Google Scholar
  14. Buckner RL, Carroll DC (2007) Self-projection and the brain. Trends Cogn Sci 11(2):49–57PubMedCrossRefGoogle Scholar
  15. Burks SV, Carpenter JP, Goette L, Rustichini A (2009) Cognitive skills affect economic preferences, strategic behavior, and job attachment. Proc Nat Acad Sci 106(19):7745–7750PubMedPubMedCentralCrossRefGoogle Scholar
  16. Cantin L, Lenoir M, Augier E, Vanhille N, Dubreucq S, Serre F, Vouillac C, Ahmed SH (2010) Cocaine is low on the value ladder of rats: possible evidence for resilience to addiction. PLoS ONE 5(7):e11592PubMedPubMedCentralCrossRefGoogle Scholar
  17. Carter EC, Pedersen EJ, McCullough ME (2015) Reassessing intertemporal choice: human decision-making is more optimal in a foraging task than in a self-control task. Frontiers Psychol Decis Neurosci 6:95Google Scholar
  18. Cavanagh JF, Eisenberg I, Guitart-Masip M, Huys Q, Frank MJ (2013) Frontal theta overrides Pavlovian learning biases. J Neurosci 33(19):8541–8548PubMedPubMedCentralCrossRefGoogle Scholar
  19. Charness N (1991) Expertise in chess: the balance between knowledge and search. In: Ericsson KA, Smith J (eds) Toward a general theory of expertise: prospects and limits (Chap. 2). Cambridge University Press, Cambridge, pp 39–63Google Scholar
  20. Corbit LH, Balleine BW (2005) Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of Pavlovian-instrumental transfer. J Neurosci 25(4):962–970PubMedCrossRefGoogle Scholar
  21. Corbit LH, Janak PH (2007) Inactivation of the lateral but not medial dorsal striatum eliminates the excitatory impact of Pavlovian stimuli on instrumental responding. J Neurosci 27(51):13977–13981PubMedCrossRefGoogle Scholar
  22. Coricelli G, Critchley HD, Joffily M, O’Doherty JP, Sirigu A, Dolan RJ (2005) Regret and its avoidance: a neuroimaging study of choice behavior. Nat Neurosci 8:1255–1262PubMedCrossRefGoogle Scholar
  23. Costa A, Foucart A, Hayakawa S, Aparici M, Apesteguia J, Heafner J, Keysar B (2014) Your morals depend on language. PLoS ONE 9(4):e94842. doi:10.1371/journal.pone.0094842 PubMedPubMedCentralCrossRefGoogle Scholar
  24. Daw ND, Niv Y, Dayan P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8:1704–1711PubMedCrossRefGoogle Scholar
  25. Dayan P, Niv Y (2008) Reinforcement learning: the good, the bad and the ugly. Curr Opin Neurobiol 18(2):185–196PubMedCrossRefGoogle Scholar
  26. Dayan P, Niv Y, Seymour B, Daw ND (2006) The misbehavior of value and the discipline of the will. Neural Networks 19:1153–1160PubMedCrossRefGoogle Scholar
  27. Dezfouli A, Balleine B (2012) Habits, action sequences and reinforcement learning. Eur J Neurosci 35(7):1036–1051PubMedPubMedCentralCrossRefGoogle Scholar
  28. Doya K (2000) Metalearning, neuromodulation, and emotion. In: Hatano G, Okada N, Tanabe H (eds) Affective Minds. Elsevier, AmsterdamGoogle Scholar
  29. Dutton DG, Aron AP (1974) Some evidence for heightened sexual attraction under conditions of high anxiety. J Pers Soc Psychol 30(4):510–517PubMedCrossRefGoogle Scholar
  30. Flagel SB, Akil H, Robinson TE (2009) Individual differences in the attribution of incentive salience to reward-related cues: implications for addiction. Neuropharmacology 56(Suppl. 1):139–148PubMedPubMedCentralCrossRefGoogle Scholar
  31. Flagel SB, Clark JJ, Robinson TE, Mayo L, Czuj A, Willuhn I, Akers CA, Clinton SM, Phillips PEM, Akil H (2011) A selective role for dopamine in stimulus-reward learning. Nature 469(7328):53–57PubMedPubMedCentralCrossRefGoogle Scholar
  32. Franco-Watkins AM, Pashler H, Rickard TC (2006) Does working memory load lead to greater impulsivity? Commentary on Hinson, Jameson and Whitney (2003). J Exp Psychol Learn Mem Cogn 32(2):443–447PubMedCrossRefGoogle Scholar
  33. Gallistel CR (1990) The organization of learning. MIT Press, CambridgeGoogle Scholar
  34. Gigerenzer G, Goldstein DG (1996) Reasoning the fast and frugal way: models of bounded rationality. Psychol Rev 103:650–669PubMedCrossRefGoogle Scholar
  35. Gilovich T, Griffin D, Kahneman D (eds) (2002) Heuristics and biases: the psychology of intuitive judgement. Cambridge University Press, CambridgeGoogle Scholar
  36. Gläscher J, Daw N, Dayan P, O’Doherty JP (2010) States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66(4):585–595Google Scholar
  37. Glimcher PW (2003) Decisions, uncertainty, and the brain: the science of neuroeconomics. MIT Press, CambridgeGoogle Scholar
  38. Glimcher PW, Camerer C, Poldrack RA (eds) (2008) Neuroeconomics: decision making and the brain. Academic Press, MassachusettsGoogle Scholar
  39. Goldstein A (2000) Addiction: from biology to drug policy. Oxford, New YorkGoogle Scholar
  40. Goodman WK, Rudorfer MV, Maser JD (eds) (2000) Obsessive-compulsive disorder: contemporary issues in treatment. Lawrence Earlbaum, HillsdaleGoogle Scholar
  41. Greene J (2013) Moral tribes: emotion, reason, and the gap between us and them. PenguinGoogle Scholar
  42. Greene JD, Sommerville RB, Nystrom LE, Darley JM, Cohen JD (2001) An fMRI investigation of emotional engagement and moral judgement. Science 293(5537):2105–2108PubMedCrossRefGoogle Scholar
  43. Greene JD, Nystrom LE, Engell AD, Darley JM, Cohen JD (2004) The neural basis of cognitive conflict and control in moral judgement. Neuron 44(2):389–400PubMedCrossRefGoogle Scholar
  44. Hare TA, Malmaud J, Rangel A (2011) Focusing attention on the health aspects of foods changes value signals in vmPFC and improves dietary choice. J Neurosci 31(30):11077–11087PubMedCrossRefGoogle Scholar
  45. Haslam SA, Reicher SD (2012) Contesting the “nature” of conformity: what Milgram and Zimbardo’s studies really show. PLoS Biol 10(11):e1001426PubMedPubMedCentralCrossRefGoogle Scholar
  46. Hassabis D, Maguire EA (2011) The construction system in the brain. In: Bar M (ed) Predictions in the brain: using our past to generate a future. Oxford University Press, Oxford, pp 70–82Google Scholar
  47. Hassabis D, Kumaran D, Vann SD, Maguire EA (2007) Patients with hippocampal amnesia cannot imagine new experiences. Proc Natl Acad Sci 104:1726–1731PubMedPubMedCentralCrossRefGoogle Scholar
  48. Hein G, Lamm C, Brodbeck C, Singer T (2011) Skin conductance response to the pain of others predicts later costly helping. PLoS ONE 6(8):e22759PubMedPubMedCentralCrossRefGoogle Scholar
  49. Hertz J, Krogh A, Palmer RG (1991) Introduction to the theory of neural computation. Addison-Wesley, ReadingGoogle Scholar
  50. Higgins ST, Alessi SM, Dantona RL (2002) Voucher-based incentives: a substance abuse treatment innovation. Addict Behav 27:887–910PubMedCrossRefGoogle Scholar
  51. Hill C (2008) The rationality of preference construction (and the irrationality of rational choice). Minn J Law Sci Technol 9(2):689–742Google Scholar
  52. Hoffman E, McCabe K, Shachat K, Smith V (1994) Preferences, property rights, and anonymity in bargaining games. Game Econ Behav 7:346–380CrossRefGoogle Scholar
  53. Jackson JC, Redish AD (2003) Detecting dynamical changes within a simulated neural ensemble using a measure of representational quality. Network Comput Neural Syst 14:629–645CrossRefGoogle Scholar
  54. Janak P, Tye K (2015) From circuits to behaviour in the amygdala. Nature 517:284–292PubMedPubMedCentralCrossRefGoogle Scholar
  55. Jog MS, Kubota Y, Connolly CI, Hillegaart V, Graybiel AM (1999) Building neural representations of habits. Science 286:1746–1749CrossRefGoogle Scholar
  56. Johnson A, Redish AD (2007) Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J Neurosci 27(45):12176–12189PubMedCrossRefGoogle Scholar
  57. Johnson A, van der Meer MAA, Redish AD (2007) Integrating hippocampus and striatum in decision-making. Curr Opin Neurobiol 17(6):692–697PubMedPubMedCentralCrossRefGoogle Scholar
  58. Johnson A, Jackson J, Redish AD (2008) Measuring distributed properties of neural representations beyond the decoding of local variables—implications for cognition. In: Hölscher C, Munk MHJ (eds) Mechanisms of information processing in the brain: encoding of information in neural populations and networks. Cambridge University Press, Cambridge, pp 95–119Google Scholar
  59. Jones JL, Esber GR, McDannald MA, Gruber AJ, Hernandez A, Mirenzi A, Schoenbaum G (2012) Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 338(6109):953–956PubMedPubMedCentralCrossRefGoogle Scholar
  60. Kable JW, Glimcher PW (2009) The neurobiology of decision: consensus and controversy. Neuron 63(6):733–745PubMedPubMedCentralCrossRefGoogle Scholar
  61. Kahneman D, Tversky A (eds) (2000) Choices, values, and frames. Cambridge University Press, CambridgeGoogle Scholar
  62. Kahneman D, Knetsch JL, Thaler RH (1991) The endowment effect, loss aversion, and status quo bias. J Econ Perspect 5(1):193–206CrossRefGoogle Scholar
  63. Kang MJ, Rangel A, Camus M, Camerer CF (2011) Hypothetical and real choice differentially activate common valuation areas. J Neurosci 31(2):461–468PubMedCrossRefGoogle Scholar
  64. Kathmann N, Rupertseder C, Hauke W, Zaudig M (2005) Implicit sequence learning in obsessive-compulsive disorder: further support for the fronto-striatal dysfunction model. Biol Psychiatry 58(3):239–244PubMedCrossRefGoogle Scholar
  65. Keramati M, Dezfouli A, Piray P (2011) Speed/accuracy trade-off between the habitual and the goal-directed processes. PLoS Comput Biol 7(5):e1002055PubMedPubMedCentralCrossRefGoogle Scholar
  66. Klein G (1999) Sources of power: how people make decisions. MIT Press, CambridgeGoogle Scholar
  67. Kruse JM, Overmier JB, Konz WA, Rokke E (1983) Pavlovian conditioned stimulus effects upon instrumental choice behavior are reinforcer specific. Learn Motiv 14(2):165–181CrossRefGoogle Scholar
  68. Kurlan R (ed) (1993) Handbook of tourette’s syndrome and related tic and behavioral disorders. Marcel Dekker, New YorkGoogle Scholar
  69. Kurth-Nelson Z, Redish AD (2012) A theoretical account of cognitive effects in delay discounting. Eur J Neurosci 35:1052–1064PubMedPubMedCentralCrossRefGoogle Scholar
  70. Laurent V, Leung B, Maidment N, Balleine BW (2012) µ- and δ-opioid-related processes in the accumbens core and shell differentially mediate the influence of reward- guided and stimulus-guided decisions on choice. J Neurosci 32(5):1875–1883PubMedPubMedCentralCrossRefGoogle Scholar
  71. Leckman JF, Riddle MA (2000) Tourette’s syndrome: when habit-forming systems form habits of their own? Neuron 28:349–354PubMedCrossRefGoogle Scholar
  72. LeDoux JE (2000) Emotion circuits in the brain. Annu Rev Neurosci 23:155–184PubMedCrossRefGoogle Scholar
  73. LeDoux JE (2012) Rethinking the emotional brain. Neuron 73:653–676PubMedPubMedCentralCrossRefGoogle Scholar
  74. Lee SW, Shimoko S, O’Doherty JP (2014) Neural computations underlying arbitration between model-based and model-free learning. Neuron 81(3):687–699PubMedPubMedCentralCrossRefGoogle Scholar
  75. Lenoir M, Serre F, Cantin L, Ahmed SH (2007) Intense sweetness surpasses cocaine reward. PLoS ONE 2(8):e698PubMedPubMedCentralCrossRefGoogle Scholar
  76. LeSage MG, Burroughs D, Dufek M, Keyler DE, Pentel PR (2004) Reinstatement of nicotine self-administration in rats by presentation of nicotine-paired stimuli, but not nicotine priming. Pharmacolol Biochem Behav 79(3):507–513CrossRefGoogle Scholar
  77. Lesaint F, Sigaud O, Flagel SB, Robinson TE, Khamassi M (2014) Modelling individual differences in the form of Pavlovian conditioned approach responses: a dual learning systems approach with factored representations. PLoS Comput Biol 10(2):e1003466. doi:10.1371/journal.pcbi.1003466 PubMedPubMedCentralCrossRefGoogle Scholar
  78. Levy DJ, Glimcher PW (2012) The root of all value: a neural common currency for choice. Curr Opin Neurobiol 22:1–12CrossRefGoogle Scholar
  79. Lichtenstein S, Slovic P (eds) (2006) The construction of preference. Cambridge University Press, CambridgeGoogle Scholar
  80. MacCorquodale K, Meehl PE (1954) Edward C. Tolman. In: Estes W (ed) Modern learning theory. Appleton-Century-Crofts, New York, pp 177–266Google Scholar
  81. Maia TV, McClelland JL (2012) A neurocomputational approach to obsessive-compulsive disorder. Trends Cogn Sci 16(1):14–15PubMedCrossRefGoogle Scholar
  82. McNally GP, Johansen JP, Blair HT (2011) Placing prediction into the fear circuit. Trends Neurosci 34(6):283–292PubMedPubMedCentralCrossRefGoogle Scholar
  83. Milgram S (1974/2009) Obedience to authority: an experimental view. Harper Collins, New YorkGoogle Scholar
  84. Mischel W (2014) The marshmallow test: mastering self-control. Little, Brown, and Co, New YorkGoogle Scholar
  85. Mischel W, Shoda Y, Rodriguez ML (1989) Delay of gratification in children. Science 244(4907):933–938PubMedCrossRefGoogle Scholar
  86. Montague PR, Dolan RJ, Friston KJ, Dayan P (2012) Comput Psychiatry. Trends Cogn Sci 16(1):72–80PubMedPubMedCentralCrossRefGoogle Scholar
  87. Nadel L (1994) Multiple memory systems: what and why, an update. In: Schacter DL, Tulving E (eds) Memory systems 1994. MIT Press, Cambridge, pp 39–64Google Scholar
  88. Niv Y, Daw ND, Dayan P (2006a) Choice values. Nat Neurosci 9:987–988PubMedCrossRefGoogle Scholar
  89. Niv Y, Joel D, Dayan P (2006b) A normative perspective on motivation. Trends Cogn Sci 10(8):375–381PubMedCrossRefGoogle Scholar
  90. Nonacs P (2001) State dependent patch use and the marginal value theorem. Behav Ecol 12:71–83CrossRefGoogle Scholar
  91. O’Doherty JP (2004) Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr Opin Neurobiol 14:769–776PubMedCrossRefGoogle Scholar
  92. O’Doherty JP, Kringelbach ML, Rolls ET, Hornak J, Andrews C (2001) Abstract reward and punishment representations in the human orbitofrontal cortex. Nat Neurosci 4:95–102PubMedCrossRefGoogle Scholar
  93. O’Keefe J, Nadel L (1978) The hippocampus as a cognitive map. Clarendon Press, OxfordGoogle Scholar
  94. Padoa-Schioppa C (2008) The syllogism of neuro-economics. Econ Philos 24:449–457PubMedPubMedCentralCrossRefGoogle Scholar
  95. Padoa-Schioppa C (2009) Range-adapting representation of economic value in the orbitofrontal cortex. J Neurosci 29(44):14004–14014PubMedPubMedCentralCrossRefGoogle Scholar
  96. Pavlov I (1927) Conditioned reflexes. Oxford University Press, OxfordGoogle Scholar
  97. Perry AN, Westenbroek C, Becker JB (2013) The development of a preference for cocaine over food identifies individual rats with addiction-like behaviors. PLoS ONE 8(11):e79465PubMedPubMedCentralCrossRefGoogle Scholar
  98. Peters J, Büchel C (2010) Episodic future thinking reduces reward delay discounting through an enhancement of prefrontal-mediotemporal interactions. Neuron 66(1):138–148PubMedCrossRefGoogle Scholar
  99. Peters J, Büchel C (2011) The neural mechanisms of inter-temporal decision-making: understanding variability. Trends Cogn Sci 15(5):227–239PubMedCrossRefGoogle Scholar
  100. Petry NM (2011) Contingency management for substance abuse treatment: a guide to implementing this evidence-based practice. Routledge, LondonGoogle Scholar
  101. Pfeiffer BE, Foster DJ (2013) Hippocampal place-cell sequences depict future paths to remembered goals. Nature 497:74–79PubMedPubMedCentralCrossRefGoogle Scholar
  102. Phelps E, Lempert KM, Sokol-Hessner P (2014) Emotion and decision making: multiple modulatory circuits. Annu Rev Neurosci 37:263–287PubMedCrossRefGoogle Scholar
  103. Plous S (1993) The psychology of judgement and decision-making. McGraw-Hill, New YorkGoogle Scholar
  104. Rand DG, Greene JD, Nowak MA (2012) Spontaneous giving and calculated greed. Nature 489:427–430PubMedCrossRefGoogle Scholar
  105. Rangel A, Clithero JA (2012) Value normalization in decision making: theory and evidence. Curr Opin Neurobiol 22(6):970–981PubMedPubMedCentralCrossRefGoogle Scholar
  106. Rangel A, Hare T (2010) Neural computations associated with goal-directed choice. Curr Opin Neurobiol 20(2):262–270PubMedCrossRefGoogle Scholar
  107. Rangel A, Camerer C, Montague PR (2008) A framework for studying the neurobiology of value-based decision making. Nat Rev Neurosci 9:545–556PubMedPubMedCentralCrossRefGoogle Scholar
  108. Redish AD (2009) Implications of the multiple-vulnerabilities theory of addiction for craving and relapse. Addiction 104(11):1940–1941PubMedCrossRefGoogle Scholar
  109. Redish AD (2013) The mind within the brain: how we make decisions and how those decisions go wrong. Oxford University Press, OxfordGoogle Scholar
  110. Redish AD, Johnson A (2007) A computational model of craving and obsession. Ann New York Acad Sci 1104(1):324–339CrossRefGoogle Scholar
  111. Redish AD, Jensen S, Johnson A, Kurth-Nelson Z (2007) Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol Rev 114(3):784–805PubMedCrossRefGoogle Scholar
  112. Redish AD, Jensen S, Johnson A (2008) A unified framework for addiction: vulnerabilities in the decision process. Behav Brain Sci 31:415–487PubMedPubMedCentralGoogle Scholar
  113. Regier PS, Redish AD (2012) What is the role of decision-making systems in contingency management? Society for Neuroscience AbstractsGoogle Scholar
  114. Robinson MJ, Warlow SM, Berridge KC (2014) Optogenetic excitation of central amygdala amplifies and narrows incentive motivation to pursue one reward above another. J Neurosci 34(50):16567–16580PubMedPubMedCentralCrossRefGoogle Scholar
  115. Sacks O (1985). Witty ticcy ray. In The Man Who Mistook His Wife for a Hat. Simon and SchusterGoogle Scholar
  116. Samuelson PA (1937) A note on measurement of utility. Rev Econ Stud 4(2):155–161CrossRefGoogle Scholar
  117. Sanfey AG (2007) Social decision-making: insights from game theory and neuroscience. Science 318(5850):598–602PubMedCrossRefGoogle Scholar
  118. Sayette MA, Shiffman S, Tiffany ST, Niaura RS, Martin CS, Shadel WG (2000) The measurement of drug craving. Addiction 95(Suppl. 2):S189–S210PubMedPubMedCentralCrossRefGoogle Scholar
  119. Schacter DL, Addis DR (2011) On the nature of medial temporal lobe contributions to the constructive simulation of future events. In: Bar M (ed) Predictions in the brain: using our past to generate a future. Oxford University Press, Oxford, pp 58–69Google Scholar
  120. Schacter DL, Addis DR, Buckner RL (2008) Episodic simulation of future events: concepts, data, and applications. Ann New York Acad Sci 1124:39–60CrossRefGoogle Scholar
  121. Schmidhuber J (2014) Deep learning in neural networks. Neural Networks 61:85–117PubMedCrossRefGoogle Scholar
  122. Simon H (1955) A behavioral model of rational choice. Q J Econ 69:99–118CrossRefGoogle Scholar
  123. Simonson I, Tversky A (1992) Choice in context: tradeoff contrast and extremeness aversion. J Mark Res 29(3):281–295CrossRefGoogle Scholar
  124. Singer T, Seymour B, O’Doherty JP, Stephan KE, Dolan RJ, Frith CD (2006) Empathic neural responses are modulated by the perceived fairness of others. Nature 439(7075):466–469PubMedPubMedCentralCrossRefGoogle Scholar
  125. Singer T, Critchley HD, Preuschoff K (2009) A common role of insula in feelings, empathy and uncertainty. Trends Cogn Sci 13(8):334–340PubMedCrossRefGoogle Scholar
  126. Skinner MD, Aubin H-J (2010) Craving’s place in addiction theory: contributions of the major models. Neurosci Biobehav Rev 34(4):606–623PubMedCrossRefGoogle Scholar
  127. Smith V (2009) Rationality in economics: constructivist and ecological forms. Cambridge University Press, CambridgeGoogle Scholar
  128. Smith KS, Graybiel AM (2013) A dual operator view of habitual behavior reflecting cortical and striatal dynamics. Neuron 79(2):361–374PubMedPubMedCentralCrossRefGoogle Scholar
  129. Smith KS, Tindell AJ, Aldridge JW, Berridge KC (2009) Ventral pallidum roles in reward and motivation. Behav Brain Res 196(2):155–167PubMedPubMedCentralCrossRefGoogle Scholar
  130. Steiner A, Redish AD (2012) Orbitofrontal cortical ensembles during deliberation and learning on a spatial decision-making task. Front Decis Neurosci 6:131Google Scholar
  131. Stephens DW, Krebs JR (1987) Foraging theory. Princeton University Press, PrincetonGoogle Scholar
  132. Stitzer M, Petry N (2006) Contingency management for treatment of substance abuse. Annu Rev Clin Psychol 2:411–434PubMedCrossRefGoogle Scholar
  133. Stott JJ, Redish AD (2014) A functional difference in information processing between orbitofrontal cortex and ventral striatum during decision-making behavior. Philos Trans R Soc B 369(1655). 10.1098/rstb.2013.0472Google Scholar
  134. Streidter G (2005) Principles of brain evolution. Sinauer Associates, SunderlandGoogle Scholar
  135. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, CambridgeGoogle Scholar
  136. Talmi D, Seymour B, Dayan P, Dolan RJ (2008) Human Pavlovian-instrumental transfer. J Neurosci 28(2):360–368PubMedPubMedCentralCrossRefGoogle Scholar
  137. Tiffany ST (1990) A cognitive model of drug urges and drug-use behavior: role of automatic and nonautomatic processes. Psychol Rev 97(2):147–168PubMedCrossRefGoogle Scholar
  138. Tiffany ST (1999) Cognitive concepts of craving. Alcohol Res Health 23(3):215–224PubMedGoogle Scholar
  139. Tiffany ST, Wray J (2009) The continuing conundrum of craving. Addiction 104:1618–1619PubMedCrossRefGoogle Scholar
  140. Tolman EC (1932) Purposive behavior in animals and men. Appleton-Century-Crofts, New YorkGoogle Scholar
  141. Tremblay L, Schultz W (1999) Relative reward preference in primate orbitofrontal cortex. Nature 398(6729):704–708PubMedCrossRefGoogle Scholar
  142. Trope Y, Liberman N (2003) Temporal construal. Psychol Rev 110(3):403–421PubMedCrossRefGoogle Scholar
  143. van der Meer MAA, Redish AD (2011) Ventral striatum: a critical look at models of learning and evaluation. Curr Opin Neurobiol 21(3):387–392PubMedPubMedCentralCrossRefGoogle Scholar
  144. van der Meer MAA, Kurth-Nelson Z, Redish AD (2012) Information processing in decision-making systems. Neuroscientist 18(4):342–359PubMedPubMedCentralCrossRefGoogle Scholar
  145. Wikenheiser AM, Redish AD (2015) Hippocampal theta sequences reflect current goals. Nat Neurosci 18:289–294PubMedPubMedCentralCrossRefGoogle Scholar
  146. Wikenheiser AM, Stephens DW, Redish AD (2013) Subjective costs drive overly-patient foraging strategies in rats on an intertemporal foraging task. Proc Natl Acad Sci USA 110(20):8308–8313PubMedPubMedCentralCrossRefGoogle Scholar
  147. Wilson M, Daly M (2004) Do pretty women inspire men to discount the future? Proc R Soc Lond B 271:S177–S179CrossRefGoogle Scholar
  148. Winecoff A, Clithero JA, Carter RM, Bergman SR, Wang L, Huettel SA (2013) Ventromedial prefrontal cortex encodes economic value. J Neurosci 33(27):11032–11039PubMedPubMedCentralCrossRefGoogle Scholar
  149. Wunderlich K, Dayan P, Dolan RJ (2012) Mapping value based planning and extensively trained choice in the human brain. Nat Neurosci 15(5):786–791PubMedPubMedCentralCrossRefGoogle Scholar
  150. Yang T, Shadlen MN (2007) Probabilistic reasoning by neurons. Nature 447:1075–1080PubMedCrossRefGoogle Scholar
  151. Zak PJ (ed) (2008) Moral markets: the critical role of values in the economy. Princeton University Press, PrincetonGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • A. David Redish
    • 1
  • Nathan W. Schultheiss
    • 1
  • Evan C. Carter
    • 2
  1. 1.Department of NeuroscienceUniversity of MinnesotaMinneapolisUSA
  2. 2.Department of Ecology, Evolution, and BehaviorUniversity of MinnesotaSt. PaulUSA

Personalised recommendations