Skip to main content
SpringerLink
Log in

Intertemporal Bargaining in Habit

  • Original Paper
  • Published:
Neuroethics Aims and scope Submit manuscript

A Letter to this article was published on 01 April 2017

Abstract

Lewis ascribes the stubborn persistence of addictions to habit, itself a normal process that does not imply lack of responsiveness to motivation. However, he suggests that more dynamic processes may be involved, for instance that “our recurrently focused brains inevitably self-organize.” Given hyperbolic delay discounting, a reward-seeking internal marketplace model describes two processes, also normal in themselves, that may give rise to the “deep attachment” to addictive activities that he describes: (1) People learn to interpret current choices as test cases for how they can expect to choose in the future, thus recruiting additional incentive (willpower) against a universal tendency to temporarily prefer smaller, sooner to larger, later rewards. However, when this incentive is not enough, the same interpretation creates incentive to abandon the failed area, leading to the abstinence violation effect and a localized weak will. (2) Normal human value does not come entirely, or even mainly, from expectation of external rewards, but is generated endogenously in imagination. Hyperbolic discounting provides an account of how we learn to cultivate the hedonic importance of occasions for endogenous reward by building appetite. In this account, expectations of the far future have to be rewarded endogenously if they are be as important as currently rewarded alternatives; and this importance is prone to collapse. Both will and hedonic importance are recursive and thus hard to study by controlled experiment, but do represent modelable, reward-based hypotheses about the dynamic nature of habit.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

Notes

  1. From ventral to dorsal striatum in rats, or the analogous dorsomedial to dorsolateral striatum in humans [3].

  2. Even pains and negative emotions must compete for attention by a positive value up front, experienced as an urge [65].

  3. The “intrinsic” rewards that roboticists have begun to model are still inborn, “inherently interesting or enjoyable” [66, 67].

  4. People sometimes value even recent experiences by some means other than the summation of momentary values found over multiple trials with nonhumans [68]. In a pioneering project to observe directly how people evaluate visceral experiences, Kahneman and his co-workers found that “decision utility” is not the integral of momentary experiences [69]. That is, a subject’s estimate of how painful a just-passed laboratory procedure was is the sum of her most extreme and most recent memories of it. So, for instance, adding a period of lesser discomfort at the end of a colonoscopy leads subjects to rate it less aversive. The subjects seem to have been sampling their component experiences rather than adding them up, and doing so without regard to their durations.

References

  1. Sussman, Steve, Nadra Lisha, and Mark Griffiths. 2011. Prevalence of the addictions: a problem of the majority or the minority? Evaluation & the Health Professions 34: 3–56.

    Article  Google Scholar 

  2. Miller, William R., and J. C’de Baca. 2001. Quantum change: when epiphanies and sudden insights transform ordinary lives. New York: Guilford.

    Google Scholar 

  3. Dolan, R.J., and P. Dayan. 2013. Goals and habits in the brain. Neuron 80(2): 312–325. p. 219

    Article  Google Scholar 

  4. Kleinsorge, T. 1999. Response repetition benefits and costs. Acta Psychologica 103(3): 295–310.

    Article  Google Scholar 

  5. Everitt, B.J., and T.W. Robbins. 2013. From the ventral to the dorsal striatum: devolving views of their roles in drug addiction. Neuroscience & Biobehavioral Reviews 37(9): 1946–1954.

    Article  Google Scholar 

  6. Voon, V., K. Derbyshire, C. Rück, M.A. Irvine, Y. Worbe, J. Enander, L.R.N. Schreiber, C. Gillan, N.A. Fineberg, B.J. Sahakian, T.W. Robbins, N.A. Harrison, J. Wood, N.D. Daw, P. Dayan, P. Grant, and E.T. Bullmore. 2015. Disorders of compulsivity: a common bias towards learning habits. Molecular Psychiatry 20(3): 345–352.

    Article  Google Scholar 

  7. Everitt, B.J., and T.W. Robbins. 2005. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nature Neuroscience 22: 3312–3320.

    Google Scholar 

  8. Yin, H.H., and B.J. Knowlton. 2004. Contributions of striatal subregions to place and response learning. Learning and Memory 11(4): 459–463.

    Article  Google Scholar 

  9. Robbins, T.W., and B.J. Everitt. 2007. A role for mesencephalic dopamine in activation: commentary on Berridge (2006). Psychopharmacology 191: 433–437.

    Article  Google Scholar 

  10. Fellows, Lesley K., and Martha J. Farah. 2005. Different underlying impairments in decision-making following ventromedial and dorsolateral frontal lobe damage in humans. Cerebral Cortex 15: 58–63.

    Article  Google Scholar 

  11. Ainslie, George. 2016. Palpating the elephant; Current theories of addiction in the light of hyperbolic delay discounting. In Addiction and choice: rethinking the relationshi, ed. Nick Heather and Gabriel Segal. 236, note#4. Oxford U.

  12. Ainslie, George. 2012. Pure hyperbolic discount curves predict “eyes open” self-control. Theory and Decision 73: 3–34.

    Article  Google Scholar 

  13. Johnson, Matthew W., and Warren K. Bickel. 2002. Within-subject comparison of real and hypothetical money rewards in delay discounting. Journal of the Experimental Analysis of Behavior 77: 129–146.

    Article  Google Scholar 

  14. Kirby, Kris N. 1997. Bidding on the future: evidence against normative discounting of delayed rewards. Journal of Experimental Psychology: General 126: 54–70.

    Article  Google Scholar 

  15. Green, Leonard, and Joel Myerson. 2004. A discounting framework for choice with delayed and probabilistic rewards. Psychological Bulletin 130: 769–792.

    Article  Google Scholar 

  16. Cropper, Maureen L., Sema K. Aydede, and Paul R. Portney. 1992. Rates of time preference for saving lives. American Economic Review 82: 469–472.

    Google Scholar 

  17. Kable, Joseph W., and Paul W. Glimcher. 2007. The neural correlates of subjective value during intertemporal choice. Nature Neuroscience 10: 1625–1633.

    Article  Google Scholar 

  18. Mazur, James E. 1987. An adjusting procedure for studying delayed reinforcement. In Quantitative analyses of behavior V: the effect of delay and of intervening events on reinforcement value, ed. M.L. Commons, J.E. Mazur, J.A. Nevin, and H. Rachlin. Hillsdale: Erlbaum.

    Google Scholar 

  19. Ainslie, George, and Richard J. Herrnstein. 1981. Preference reversal and delayed reinforcement. Animal Learning and Behavior 9: 476–482.

    Article  Google Scholar 

  20. Laibson, David. 1997. Golden eggs and hyperbolic discounting. Quarterly Journal of Economics 62: 443–479.

    Article  Google Scholar 

  21. McClure, S.M., D.L. Laibson, G. Loewenstein, and J.D. Cohen. 2004. The grasshopper and the ant: separate neural systems value immediate and delayed monetary rewards. Science 306: 503–507.

    Article  Google Scholar 

  22. Ainslie, George. 1974. Impulse control in pigeons. Journal of the Experimental Analysis of Behavior 21: 485–489.

    Article  Google Scholar 

  23. Deluty, M.Z., W.G. Whitehouse, M. Millitz, and P. Hineline. 1983. Self-control and commitment involving aversive events. Behavioral Analysis Letters 3: 213–219.

    Google Scholar 

  24. O’Donoghue, Ted, and Matthew Rabin. 1999. Doing it now or later. The American Economic Review 89(1): 103–124.

    Article  Google Scholar 

  25. Baumeister, Roy F., Gailliot Matthew, C. Nathan DeWall, and Megan Oaten. 2006. Self-regulation and personality: how interventions increase regulatory success, and how depletion moderates the effects of traits on behavior. Journal of Personality 74: 1773–1801.

    Article  Google Scholar 

  26. Fudenberg, D., and D. Levine. 2006. A dual-self model of impulse control. American Economic Review 96: 1449–1476.

    Article  Google Scholar 

  27. Ainslie, George, and John Monterosso. 2003. Building blocks of self-control: increased tolerance for delay with bundled rewards. Journal of the Experimental Analysis of Behavior 79: 83–94.

    Article  Google Scholar 

  28. Kirby, Kris N., and Barbarose Guastello. 2001. Making choices in anticipation of similar future choices can increase self-control. Journal of Experimental Psychology: Applied 7: 154–164.

    Google Scholar 

  29. Hofmeyr, André, George Ainslie, Richard Charlton, and Don Ross. 2010. The relationship between addiction and reward bundling: an experiment comparing smokers and non-smokers. Addiction 106: 402–409.

    Article  Google Scholar 

  30. Schelling, Thomas C. 1960. The strategy of conflict, 53–80. Cambridge: Harvard University Press.

    Google Scholar 

  31. Ainslie, George. 2001. Breakdown of will, 105–140. Cambridge: Cambridge University.

    Book  Google Scholar 

  32. Revusky, S., and J. Garcia. 1970. Learned associations over long delays. Psychology of Learning and Motivation 4: 1–84.

    Article  Google Scholar 

  33. Ainslie, George. 2016. Palpating the elephant; current theories of addiction in the light of hyperbolic delay discounting. In Addiction and choice: rethinking the relationship, ed. Nick Heather and Gabriel Segal, 236 . Oxford: Oxford University.note#4

    Google Scholar 

  34. Marlatt, G. Allen, and Judith R. Gordon. 1980. Determinants of relapse: implications for the maintenance of behavior change. In Behavioral medicine: changing health lifestyles, ed. Park O. Davidson and Sheena M. Davidson, 410–452. Oxford: Pergamon.

  35. Curry, S., G.A. Marlatt, and J.R. Gordon. 1987. Abstinence violation effect: validation of an attributional construct with smoking cessation. Journal of Consulting and Clinical Psychology 55: 145–149.

    Article  Google Scholar 

  36. Polivy, J., and C.P. Herman. 1985. Dieting and binging: a causal analysis. American Psychologist 40: 193–201.

    Article  Google Scholar 

  37. Pickard, Hanna, and Serge Ahmed. 2016. How do you know you have a drug problem? The role of knowledge of negative consequences in explaining drug choice in humans and rats. In Addiction and choice: rethinking the relationship, ed. Hanna Pickard and Serge H. Ahmed. London: Routledge.

  38. Gilbert, Daniel T., and Timothy D. Wilson. 2007. Prospection: experiencing the future. Science 317: 1351–1354.

    Article  Google Scholar 

  39. Sabbagh, M.A., and M.A. Callanan. 1998. Metarepresentation in action: 3-, 4-, and 5-year-olds’ developing theories of mind in parent–child conversations. Developmental Psychology 34(3): 491.

    Article  Google Scholar 

  40. Mischel, H.N., and W. Mischel. 1983. The development of children’s knowledge of self-control strategies. Child Development 54: 603–619.

    Article  Google Scholar 

  41. Rosati, A.G., J.R. Stevens, B. Hare, and M.D. Hauser. 2007. The evolutionary origins of human patience: temporal preferences in chimpanzees, bonobos, and human adults. Current Biology 17: 1663–1668.

    Article  Google Scholar 

  42. Millar, A., and D.J. Navarick. 1984. Self-control and choice in humans: effects of video game playing as a positive reinforcer. Learning and Motivation 15: 203–218.

    Article  Google Scholar 

  43. Navarick, D.J. 1986. Human impulsivity and choice: a challenge to traditional operant methodology. Psychological Record 36(3): 343–356.

    Google Scholar 

  44. Ainslie, George. 2006. Motivation must Be momentary. In Understanding choice, explaining behaviour: essays in honour of ole-Jorgen Skog, ed. J. Elster, O. Gjelsvik, A. Hylland, and K. Moene, 11–28. Oslo: Unipub Forlag.

    Google Scholar 

  45. Wilson, James Q., and Richard J. Herrnstein. 1985. Crime and human nature, 45. New York: Simon & Schuster.

    Google Scholar 

  46. Baum, William M. 2005. Understanding Behaviorism. 2d ed. Blackwell.

  47. Ainslie, George. 2013. Grasping the impalpable: the role of endogenous reward in choices, including process addictions. Inquiry 56: 446–469.

    Article  Google Scholar 

  48. Ainslie, George. in press. De gustibus disputare: Hyperbolic delay discounting integrates five approaches to choice. Journal of Economic Methodology.

  49. Mitchell, Jason P., Jessica Schirmer, Daniel L. Ames, and Daniel T. Gilbert. 2011. Medial prefrontal cortex predicts intertemporal choice. Journal of Cognitive Neuroscience 23: 857–866.

    Article  Google Scholar 

  50. Peters, J., and C. Büchel. 2010. Episodic future thinking reduces reward delay discounting through an enhancement of prefrontal-mediotemporal interactions. Neuron 66(1): 138–148.

    Article  Google Scholar 

  51. Wittman, Mark, Kathryn L. Lovero, Scott D. Lane, and Martin P. Paulus. 2010. Now or later? Striatum and insula activation to immediate versus delayed rewards. Journal of Neuroscience, Psychology and Economics 1: 15–26.

    Article  Google Scholar 

  52. Gregorios-Pippas, Lucy, Philippe N. Tobler, and Wolfram Schultz. 2009. Short-term temporal discounting of reward value in human ventral striatum. Journal of Neurophysiology 101: 1507–1523.

    Article  Google Scholar 

  53. King, D.L., P.H. Delfabbro, and M.D. Griffiths. 2011. The role of structural characteristics in problematic video game play: an empirical study. International Journal of Mental Health and Addiction 9: 320–333.

    Article  Google Scholar 

  54. Ascher, M.S., and P. Levounis. 2015. The behavioral addictions. Washington, DC: American Psychiatric Publishing.

    Google Scholar 

  55. Green, Leonard, Astrid Fry, and Joel Myerson. 1994. Discounting of delayed rewards: a life-span comparison. Psychological Science 5: 33–36.

    Article  Google Scholar 

  56. Chapman, G. 2002. Your money or your health: time preferences in trading money for health. Medical Decision Making 22: 410–416.

    Article  Google Scholar 

  57. Gollier, Christian, and Martin L. Weitzman. 2010. How should the distant future be discounted when discount rates are uncertain? Economics Letters 107(3): 350–353.

    Article  Google Scholar 

  58. Schouwenburg, H.C., and J.T. Groenewoud. 2001. Study motivation under social temptation: effects of trait procrastination. Personality and Individual Differences 30: 229–240.

    Article  Google Scholar 

  59. Monterosso, John, and George Ainslie. 1999. Beyond discounting: possible experimental models of impulse control. Psychopharmacology 146: 339–347.

    Article  Google Scholar 

  60. Glimcher, Paul William, Joseph Kable, and Kenway Louie. 2007. Neuroeconomic studies of impulsivity: now or just as soon as possible? American Economic Review 97: 142–147.

    Article  Google Scholar 

  61. Moody, L., and W.K. Bickel. 2015. Symmetrical discounting of the future and the past in heavy smokers and alcohol drinkers. Drug and Alcohol Dependence 156: e157.

    Google Scholar 

  62. Jones, B., and H. Rachlin. 2009. Delay, probability, and social discounting in a public goods game. Journal of the Experimental Analysis of Behavior 91: 61–73.

    Article  Google Scholar 

  63. Brown, R., and R.J. Herrnstein. 1975. Psychology, 146. Boston: Little Brown.

    Google Scholar 

  64. Becker, G., and K. Murphy. 1988. A theory of rational addiction. Journal of Political Economy 96: 675–700.

    Article  Google Scholar 

  65. Ainslie, George. 2010. The core process in addictions and other impulses: hyperbolic discounting versus conditioning and cognitive framing. In What is addiction? ed. Don Ross, Howard Kincaid, David Spurrett, and Peter Collins, 211–245. Cambridge, Massachusetts: MIT.

  66. Ryan, R.M., and E.L. Deci. 2000. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist 55: 68–78.

    Article  Google Scholar 

  67. Singh, S., R.L. Lewis, and A.G. Barto. 2009. Where do rewards come from. Proceedings of the Annual Conference of the Cognitive Science Society 2601–2606.

  68. Mazur, James E. 1986. Choice between single and multiple delayed reinforcers. Journal of the Experimental Analysis of Behavior 46: 67–77.

    Article  Google Scholar 

  69. Kahneman, Daniel. 2000b. Evaluation by moments: past and future. In Choices, values, and frames, ed. Daniel Kahneman and Amos Tversky, 693–708. Cambridge, UK: Cambridge University Press.

Download references

Acknowledgements

This material is the result of work supported with resources and the use of facilities at the Department of Veterans Affairs Medical Center, Coatesville, PA, USA. The opinions expressed are not those of the Department of Veterans Affairs or of the US Government.

Author information

Authors and Affiliations

  1. University of Cape Town, 151 Veterans Affairs Medical Center, Coatesville, PA, 19320, USA

    George Ainslie

Authors
  1. George Ainslie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George Ainslie.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ainslie, G. Intertemporal Bargaining in Habit. Neuroethics 10, 143–153 (2017). https://doi.org/10.1007/s12152-016-9294-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12152-016-9294-3

Keywords

Search

Navigation