Value-based decision-making battery: A Bayesian adaptive approach to assess impulsive and risky behavior
Using simple mathematical models of choice behavior, we present a Bayesian adaptive algorithm to assess measures of impulsive and risky decision making. Practically, these measures are characterized by discounting rates and are used to classify individuals or population groups, to distinguish unhealthy behavior, and to predict developmental courses. However, a constant demand for improved tools to assess these constructs remains unanswered. The algorithm is based on trial-by-trial observations. At each step, a choice is made between immediate (certain) and delayed (risky) options. Then the current parameter estimates are updated by the likelihood of observing the choice, and the next offers are provided from the indifference point, so that they will acquire the most informative data based on the current parameter estimates. The procedure continues for a certain number of trials in order to reach a stable estimation. The algorithm is discussed in detail for the delay discounting case, and results from decision making under risk for gains, losses, and mixed prospects are also provided. Simulated experiments using prescribed parameter values were performed to justify the algorithm in terms of the reproducibility of its parameters for individual assessments, and to test the reliability of the estimation procedure in a group-level analysis. The algorithm was implemented as an experimental battery to measure temporal and probability discounting rates together with loss aversion, and was tested on a healthy participant sample.
KeywordsDelay discounting Risk seeking Intertemporal choice Loss aversion Bayesian estimation
We thank Zeb Kurth-Nelson for sharing his ideas on the mathematical framework, Nils B. Kroemer for insightful discussions, and Elisabeth Jünger and Christian Sommer for collecting the pilot data. This study was supported by the Deutsche Forschungsgemeinschaft (DFG FOR 1617 Grants RA 1047/2-1, SM 80/7-1, and SM 80/7-2; DFG SPP 1226 Grant SM 80/5-2; and DFG SFB 940/1 and SFB 940/2 grants). Q.J.M.H. and M.S. contributed to the conception and design of the study. A.G. and Q.J.M.H. implemented the mathematical algorithm, which was improved by S.P. for this work. Piloting of participants and data collection were performed by N.B., and N.B. and S.P. drafted the manuscript. All authors provided critical revision of the manuscript for important intellectual content and approved the final version for publication.
- Bickel, W. K., Jarmolowicz, D. P., Mueller, E. T., Koffarnus, M. N., & Gatchalian, K. M. (2012). Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: Emerging evidence. Pharmacology & Therapeutics, 134, 287–297. doi:10.1016/j.pharmthera.2012.02.004 CrossRefGoogle Scholar
- Doyle, J. R. (2013). Survey of time preference, delay discounting models. Judgment and Decision Making, 8, 116–135.Google Scholar
- Green, L., Myerson, J., Shah, A. K., Estle, S. J., & Holt, D. D. (2007). Do adjusting-amount and adjusting-delay procedures produce equivalent estimates of subjective value in pigeons? Journal of the Experimental Analysis of Behavior, 87, 337–347. doi:10.1901/jeab.2007.37-06 CrossRefPubMedPubMedCentralGoogle Scholar
- Krishnan-Sarin, S., Reynolds, B., Duhig, A. M., Smith, A., Liss, T., McFetridge, A.,…Potenza, M. N. (2007). Behavioral impulsivity predicts treatment outcome in a smoking cessation program for adolescent smokers. Drug and Alcohol Dependence, 88, 79–82. doi:10.1016/j.drugalcdep.2006.09.006
- Lovric, M. (2010). International encyclopedia of statistical science. New York, NY: Springer.Google Scholar
- Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative analysis of behavior: Vol. 5. The effect of delay and of intervening events on reinforcement value (pp. 55–73). Hillsdale, NJ: Erlbaum.Google Scholar
- Ripke, S., Hübner, T., Mennigen, E., Müller, K. U., Rodehacke, S., Schmidt, D.,…Smolka, M. N. (2012). Reward processing and intertemporal decision making in adults and adolescents: The role of impulsivity and decision consistency. Brain Research, 1478, 36–47. doi:10.1016/j.brainres.2012.08.034
- Simpson, C. A., & Vuchinich, R. E. (2000). Reliability of a measure of temporal discounting. Psychological Record, 50(1), 3–16.Google Scholar
- Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction (Vol. 1). Cambridge, MA: MIT Press.Google Scholar
- Von Neumann, J., & Morgenstern, O. (1944). Theory of games and economic behavior. Princeton, NJ: Princeton University Press.Google Scholar