Relative insensitivity to time-out punishments induced by win-paired cues in a rat gambling task
- 100 Downloads
Pairing rewarding outcomes with audiovisual cues in simulated gambling games increases risky choice in both humans and rats. However, the cognitive mechanism through which this sensory enhancement biases decision-making is unknown.
To assess the computational mechanisms that promote risky choice during gambling, we applied a series of reinforcement learning models to a large dataset of choices acquired from rats as they each performed one of two variants of a rat gambling task (rGT), in which rewards on “win” trials were delivered either with or without salient audiovisual cues.
We used a sampling technique based on Markov chain Monte Carlo to obtain posterior estimates of model parameters for a series of RL models of increasing complexity, in order to assess the relative contribution of learning about positive and negative outcomes to the latent valuation of each choice option on the cued and uncued rGT.
Rats which develop a preference for the risky options on the rGT substantially down-weight the equivalent cost of the time-out punishments during these tasks. For each model tested, the reduction in learning from the negative time-outs correlated with the degree of risk preference in individual rats. We found no apparent relationship between risk preference and the parameters that govern learning from the positive rewards.
The emergence of risk-preferring choice on the rGT derives from a relative insensitivity to the cost of the time-out punishments, as opposed to a relative hypersensitivity to rewards. This hyposensitivity to punishment is more likely to be induced in individual rats by the addition of salient audiovisual cues to rewards delivered on win trials.
KeywordsDecision-making Reward Gambling Risk Individual differences Computational modeling
This work was supported by the National Institutes of Health grant R01DA042065 from NIDA and the Swartz Center for Theoretical Neuroscience at Princeton University (AJL) and an operating grant awarded to CAW from the Canadian Institutes for Health Research (CIHR; PJT-162312).
Compliance with ethical standards
All housing conditions and testing procedures were in accordance with the guidelines of the Canadian Council on Animal Care, and all protocols were approved by the Animal Care Committee of the University of British Columbia.
Conflict of interest
In the past 3 years, CAW has consulted for Hogan Lovells LLP and received due compensation. The authors confirm they have no other conflicts of interest or financial disclosures to make.
- Alter A (2017) Irresistible: the rise of addictive technology and the business of keeping us hooked. Penguin Press, New YorkGoogle Scholar
- Barrus MM, Hosking JG, Zeeb FD, Tremblay M, Winstanley CA (2015) Disadvantageous decision-making on a rodent gambling task is associated with increased motor impulsivity in a population of male rats. J Psychiatry Neurosci 40:108–117Google Scholar
- Dow Schull N (2014) Addiction by design: machine gambling in Las Vegas. Princeton University Press, PrincetonGoogle Scholar
- Ferland J-MN, Hynes TJ, Hounjet CD, Lindenbach D, Haar CV, Adams WK, Phillips AG, Winstanley CA (2019) Prior exposure to salient win-paired cues in a rat gambling task increases sensitivity to cocaine self-administration and suppresses dopamine efflux in nucleus accumbens: support for the reward deficiency hypothesis of addiction. J Neurosci 39:1842–1854CrossRefGoogle Scholar
- Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB, Carlin JB, Stern HS, Dunson DB, Vehtari A, et al (2013) Bayesian data analysis (Chapman and Hall/CRC)Google Scholar
- Kruschke J (2014) Doing Bayesian data analysis: a tutorial with R, JAGS, and Stan (Academic)Google Scholar
- Robinson MJF, Fischer AM, Ahuja A, Lesser EN, Maniates H (2016) Roles of “wanting” and “liking” in motivating behavior: gambling, food, and drug addictions. In: Simpson EH, Balsam PD (eds) Behavioral neuroscience of motivation. Cham, Springer, pp 105–136Google Scholar
- Silveira MM, Murch WS, Clark L, Winstanley CA (2016) Chronic atomoxetine treatment during adolescence does not influence decision-making on a rodent gambling task, but does modulate amphetamine’s effect on impulsive action in adulthoodGoogle Scholar
- Stevens L, Betanzos-Espinosa P, Crunelle CL, Vergara-Moragues E, Roeyers H, Lozano O, Dom G, Gonzalez-Saiz F, Vanderplasschen W, Verdejo-García A, Pérez-García M (2013) Disadvantageous decision-making as a predictor of drop-out among cocaine-dependent individuals in long-term residential treatment. Front Psychiatry 4Google Scholar
- Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, CambridgeGoogle Scholar
- Vu MAT, Adalı T, Ba D, Buzsáki G, Carlson D, Heller K, Liston C, Rudin C, Sohal VS, Widge AS, Mayberg HS, Sapiro G, Dzirasa K (2018) A Shared Vision for Machine Learning in Neuroscience. J Neurosci 38 (7):1601-1607Google Scholar
- Watanabe S (2010) Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. J Mach Learn Res 11:3571–3594Google Scholar
- Zhang F, Xiao L, Gu R (2017) Does gender matter in the relationship between anxiety and decision-making? Front Psychol 8Google Scholar