A Neural Mechanism for Reward Discounting: Insights from Modeling Hippocampal–Striatal Interactions
- 231 Downloads
Decision-making often requires taking into consideration immediate gains as well as delayed rewards. Studies of behavior have established that anticipated rewards are discounted according to a decreasing hyperbolic function. Although mathematical explanations for reward delay discounting have been offered, little has been proposed in terms of neural network mechanisms underlying discounting. There has been much recent interest in the potential role of the hippocampus. Here, we demonstrate that a previously established neural network model of hippocampal region CA3 contains a mechanism that could explain discounting in downstream reward-prediction systems (e.g., basal ganglia). As part of its normal function, the model forms codes for stimuli that are similar to future predicted stimuli. This similarity provides a means for reward predictions associated with future stimuli to influence current decision-making. Simulations show that this “predictive similarity” decreases as the stimuli are separated in time, at a rate that is consistent with hyperbolic discounting.
KeywordsHippocampus Reward discounting Neural network Prediction
I thank Chip Levy, Joe Monaco, Sean Polyn, Steve Yantis, Kechen Zhang, and the anonymous reviewers for helpful discussions and comments on an earlier version of this manuscript. This work was supported by NIH grant R01-DA013165 to Steve Yantis.
- 27.Mariano TY, Bannerman DM, McHugh SB, Preston TJ, Rudebeck PH, Rudebeck SR, Rawlins JNP, Walton ME, Rushworth MFS, Baxter MG, Campbell TG. Impulsive choice in hippocampal but not orbitofrontal cortex-lesioned rats on a nonspatial decision-making maze task. Eur J Neurosci. 2009;30:472–84.PubMedCrossRefGoogle Scholar
- 29.Mazur J. An adjusting procedure for studying delayed reinforcement. Quant Anal Behav Eff Delay Interv Events Reinf Value. 1987;5:55.Google Scholar
- 35.Mitman K, Laurent P, Levy W. Defining time in a minimal hippocampal CA3 model by matching time-span of associative synaptic modification and input pattern duration. In: Neural networks, 2003. Proceedings of the international joint conference on 2003. p. 1631–1636 IEEE vol 3.Google Scholar
- 36.Monaco J, Levy W. T-maze training of a recurrent CA3 model reveals the necessity of novelty-based modulation of LTP in hippocampal region CA3. In Neural networks, 2003. Proceedings of the international joint conference on 2003. p. 1655–1660 IEEE vol 3.Google Scholar