Advertisement

Cognitive, Affective, & Behavioral Neuroscience

, Volume 14, Issue 4, pp 1208–1220 | Cite as

State-based versus reward-based motivation in younger and older adults

  • Darrell A. Worthy
  • Jessica A. Cooper
  • Kaileigh A. Byrne
  • Marissa A. Gorlick
  • W. Todd Maddox
Article

Abstract

Recent decision-making work has focused on a distinction between a habitual, model-free neural system that is motivated toward actions that lead directly to reward and a more computationally demanding goal-directed, model-based system that is motivated toward actions that improve one’s future state. In this article, we examine how aging affects motivation toward reward-based versus state-based decision making. Participants performed tasks in which one type of option provided larger immediate rewards but the alternative type of option led to larger rewards on future trials, or improvements in state. We predicted that older adults would show a reduced preference for choices that led to improvements in state and a greater preference for choices that maximized immediate reward. We also predicted that fits from a hybrid reinforcement-learning model would indicate greater model-based strategy use in younger than in older adults. In line with these predictions, older adults selected the options that maximized reward more often than did younger adults in three of the four tasks, and modeling results suggested reduced model-based strategy use. In the task where older adults showed similar behavior to younger adults, our model-fitting results suggested that this was due to the utilization of a win-stay–lose-shift heuristic rather than a more complex model-based strategy. Additionally, within older adults, we found that model-based strategy use was positively correlated with memory measures from our neuropsychological test battery. We suggest that this shift from state-based to reward-based motivation may be due to age related declines in the neural structures needed for more computationally demanding model-based decision making.

Keywords

Aging Computational modeling Decision-making Motivation Reward 

Notes

Acknowledgements

This research was funded by NIA grant AG043425 to D.A.W. and W.T.M. and NIDA grant DA032457 to W.T.M. We thank Anna Anthony for help with data collection. Correspondence should be addressed to W. Todd Maddox (Maddox@psy.utexas.edu) or Darrell A. Worthy (worthyda@tamu.edu).

References

  1. Akaike, H. (1974). A new look at the statistical model identification. Automatic Control, IEEE Transactions On, 19(6), 716–723.CrossRefGoogle Scholar
  2. Bäckman, L., Nyberg, L., Lindenberger, U., Li, S.-C., & Farde, L. (2006). The correlative triad among aging, dopamine, and cognition: current status and future prospects. Neuroscience & Biobehavioral Reviews, 30(6), 791–807.CrossRefGoogle Scholar
  3. Berridge, K. C., & Robinson, T. E. (2003). Parsing reward. Trends in Neurosciences, 26(9), 507–513.PubMedCrossRefGoogle Scholar
  4. Byrne, K. A., & Worthy, D. A. (2013). Do narcissists make better decisions? An investigation of narcissism and dynamic decision-making performance. Personality and Individual Differences, 55(2), 112–117.CrossRefGoogle Scholar
  5. Castel, A. D., Rossi, A. D., & McGillivray, S. (2012). Beliefs about the “hot hand” in basketball across the adult life span. Psychology and Aging, 27(3), 601.PubMedCrossRefGoogle Scholar
  6. Chowdhury, R., Guitart-Masip, M., Lambert, C., Dayan, P., Huys, Q., Düzel, E., & Dolan, R. J. (2013). Dopamine restores reward prediction errors in old age. Nature Neuroscience, 16(5), 648–653.PubMedCentralPubMedCrossRefGoogle Scholar
  7. Cooper, J. A., Worthy, D. A., Gorlick, M. A., & Maddox, W. T. (2013). Scaffolding across the lifespan in history-dependent decision making. Psychology and Aging, 28(2), 505–514.PubMedCentralPubMedCrossRefGoogle Scholar
  8. Curtis, C. E., & D'Esposito, M. (2003). Persistent activity in the prefrontal cortex during working memory. Trends in Cognitive Sciences, 7(9), 415–423.PubMedCrossRefGoogle Scholar
  9. Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans' choices and striatal prediction errors. Neuron, 69(6), 1204–1215.PubMedCentralPubMedCrossRefGoogle Scholar
  10. Denburg, N. L., Tranel, D., & Bechara, A. (2005). The ability to decide advantageously declines prematurely in some normal older persons. Neuropsychologia, 43(7), 1099–1106.PubMedCrossRefGoogle Scholar
  11. Denburg, N. L., Weller, J. A., Yamada, T. H., Shivapour, D. M., Kaup, A. R., LaLoggia, A., & Bechara, A. (2009). Poor decision making among older adults is related to elevated levels of neuroticism. Annals of Behavioral Medicine, 37(2), 164–172.PubMedCentralPubMedCrossRefGoogle Scholar
  12. Eppinger, B., Schuck, N. W., Nystrom, L. E., & Cohen, J. D. (2013). Reduced striatal responses to reward prediction errors in older compared with younger adults. The Journal of Neuroscience, 33(24), 9905–9912.PubMedCentralPubMedCrossRefGoogle Scholar
  13. Eppinger, B., Walter, M., Heekeren, H. R., & Li, S.-C. (2013). Of goals and habits: age-related and individual differences in goal-directed decision-making. Frontiers in Neuroscience, 7.Google Scholar
  14. Fridlund, A., & Delis, D. C. (1987). CVLT research edition administration and scoring software. New York: The Psychological Corporation.Google Scholar
  15. Gershman, S. J., Markman, A. B., & Otto, A. R. (2012). Retrospective revaluation in sequential decision making: A tale of two systems.Google Scholar
  16. Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010). States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66(4), 585–595.PubMedCentralPubMedCrossRefGoogle Scholar
  17. Gureckis, T. M., & Love, B. C. (2009a). Learning in noise: Dynamic decision-making in a variable environment. Journal of Mathematical Psychology, 53(3), 180–193.PubMedCentralPubMedCrossRefGoogle Scholar
  18. Gureckis, T. M., & Love, B. C. (2009b). Short-term gains, long-term pains: How cues about state aid learning in dynamic environments. Cognition, 113(3), 293–313.PubMedCentralPubMedCrossRefGoogle Scholar
  19. Hare, T. A., O'Doherty, J., Camerer, C. F., Schultz, W., & Rangel, A. (2008). Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. The Journal of Neuroscience, 28(22), 5623–5630.PubMedCrossRefGoogle Scholar
  20. Heaton, R. K. (1981). A manual for the Wisconsin card sorting test: Western Psycological Services.Google Scholar
  21. Lau, B., & Glimcher, P. W. (2005). Dynamic response-by-response models of matching behavior in rhesus monkeys. Journal of the Experimental Analysis of Behavior, 84(3), 555–579.PubMedCentralPubMedCrossRefGoogle Scholar
  22. Lezak, M. (1995). Neuropsychological testing. Oxford: University Press.Google Scholar
  23. Li, S.-C., Lindenberger, U., & Sikström, S. (2001). Aging cognition: from neuromodulation to representation. Trends in Cognitive Sciences, 5(11), 479–486.PubMedCrossRefGoogle Scholar
  24. Maddox, W. T., Gorlick, M. A., Worthy, D. A., & Beevers, C. G. (2012). Depressive symptoms enhance loss-minimization, but attenuate gain-maximization in history-dependent decision-making. Cognition.Google Scholar
  25. Mata, R., Josef, A. K., Samanez‐Larkin, G. R., & Hertwig, R. (2011). Age differences in risky choice: A meta‐analysis. Annals of the New York Academy of Sciences, 1235(1), 18–29.PubMedCentralPubMedCrossRefGoogle Scholar
  26. Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006). Midbrain dopamine neurons encode decisions for future action. Nature Neuroscience, 9(8), 1057–1063.PubMedCrossRefGoogle Scholar
  27. O’Doherty, J. P. (2004). Reward representations and reward-related learning in the human brain: Insights from neuroimaging. Current Opinion in Neurobiology, 14(6), 769–776.PubMedCrossRefGoogle Scholar
  28. Otto, A. R., Gershman, S. J., Markman, A. B., & Daw, N. D. (2013). The curse of planning dissecting multiple reinforcement-learning systems by taxing the central executive. Psychological Science, 24(5), 751–761.PubMedCrossRefGoogle Scholar
  29. Otto, A. R., Gureckis, T. M., Markman, A. B., & Love, B. C. (2009). Navigating through abstract decision spaces: Evaluating the role of state generalization in a dynamic decision-making task. Psychonomic Bulletin & Review, 16(5), 957–963.CrossRefGoogle Scholar
  30. Otto, A. R., Raio, C. M., Chiang, A., Phelps, E. A., & Daw, N. D. (2013). Working-memory capacity protects model-based learning from stress. Proceedings of the National Academy of Sciences, 110(52), 20941–20946.CrossRefGoogle Scholar
  31. Otto, A. R., Taylor, E. G., & Markman, A. B. (2011). There are at least two kinds of probability matching: Evidence from a secondary task. Cognition, 118(2), 274–279.PubMedCrossRefGoogle Scholar
  32. Raz, N., Lindenberger, U., Rodrigue, K. M., Kennedy, K. M., Head, D., Williamson, A., & Acker, J. D. (2005). Regional brain changes in aging healthy adults: general trends, individual differences and modifiers. Cerebral Cortex, 15(11), 1676–1689.PubMedCrossRefGoogle Scholar
  33. Reed, A. E., Mikels, J. A., & Simon, K. I. (2008). Older adults prefer less choice than young adults. Psychology and Aging, 23(3), 671.PubMedCentralPubMedCrossRefGoogle Scholar
  34. Salthouse, T. A. (2004). What and when of cognitive aging. Current Directions in Psychological Science, 13(4), 140–144.CrossRefGoogle Scholar
  35. Salthouse, T. A. (2009). When does age-related cognitive decline begin? Neurobiology of Aging, 30(4), 507–514.PubMedCentralPubMedCrossRefGoogle Scholar
  36. Samanez-Larkin, G. R., Kuhnen, C. M., Yoo, D. J., & Knutson, B. (2010). Variability in nucleus accumbens activity mediates age-related suboptimal financial risk taking. The Journal of Neuroscience, 30(4), 1426–1434.PubMedCentralPubMedCrossRefGoogle Scholar
  37. Schwartz, B. (2000). Self-determination: The tyranny of freedom. American Psychologist, 55(1), 79.PubMedCrossRefGoogle Scholar
  38. Schwartz, B. (2009). The paradox of choice: HarperCollins.Google Scholar
  39. Smittenaar, P., FitzGerald, T. H. B., Romei, V., Wright, N. D., & Dolan, R. J. (2013). Disruption of dorsolateral prefrontal cortex decreases model-based in favor of model-free control in humans. Neuron, 80(4), 914–919.PubMedCentralPubMedCrossRefGoogle Scholar
  40. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction (Vol. 1): Cambridge Univ Press.Google Scholar
  41. Wagenmakers, E.-J., & Farrell, S. (2004). AIC model selection using Akaike weights. Psychonomic Bulletin & Review, 11(1), 192–196.CrossRefGoogle Scholar
  42. Wechsler, D. (1997). WAIS-III, Wechsler Adult Intelligence Scale: Administration and Scoring Manual: Psychological Corporation.Google Scholar
  43. West, R. L. (1996). An application of prefrontal cortex function theory to cognitive aging. Psychological Bulletin, 120(2), 272.PubMedCrossRefGoogle Scholar
  44. Worthy, D. A., Gorlick, M. A., Pacheco, J. L., Schnyer, D. M., & Maddox, W. T. (2011). With age comes wisdom decision making in younger and older adults. Psychological Science, 22(11), 1375–1380.PubMedCentralPubMedCrossRefGoogle Scholar
  45. Worthy, D. A., Hawthorne, M. J., & Otto, A. R. (2013). Heterogeneity of strategy use in the Iowa gambling task: A comparison of win-stay/lose-shift and reinforcement learning models. Psychonomic Bulletin & Review, 20, 364–371.CrossRefGoogle Scholar
  46. Worthy, D. A., & Maddox, W. T. (2012). Age-based differences in strategy use in choice tasks. Frontiers in Neuroscience, 5.Google Scholar
  47. Worthy, D. A., Otto, A. R., & Maddox, W. T. (2012). Working-memory load and temporal myopia in dynamic decision making. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(6), 1640–1658.PubMedCentralPubMedGoogle Scholar
  48. Yechiam, E., & Busemeyer, J. R. (2005). Comparison of basic assumptions embedded in learning models for experience-based decision making. Psychonomic Bulletin & Review, 12(3), 387–402.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2014

Authors and Affiliations

  • Darrell A. Worthy
    • 1
  • Jessica A. Cooper
    • 2
  • Kaileigh A. Byrne
    • 1
  • Marissa A. Gorlick
    • 2
  • W. Todd Maddox
    • 2
  1. 1.College StationTexas A&M UniversityAustinUSA
  2. 2.University of Texas at AustinAustinUSA

Personalised recommendations