Behavior Research Methods

, Volume 50, Issue 1, pp 236–249 | Cite as

Value-based decision-making battery: A Bayesian adaptive approach to assess impulsive and risky behavior

  • Shakoor Pooseh
  • Nadine Bernhardt
  • Alvaro Guevara
  • Quentin J. M. Huys
  • Michael N. Smolka


Using simple mathematical models of choice behavior, we present a Bayesian adaptive algorithm to assess measures of impulsive and risky decision making. Practically, these measures are characterized by discounting rates and are used to classify individuals or population groups, to distinguish unhealthy behavior, and to predict developmental courses. However, a constant demand for improved tools to assess these constructs remains unanswered. The algorithm is based on trial-by-trial observations. At each step, a choice is made between immediate (certain) and delayed (risky) options. Then the current parameter estimates are updated by the likelihood of observing the choice, and the next offers are provided from the indifference point, so that they will acquire the most informative data based on the current parameter estimates. The procedure continues for a certain number of trials in order to reach a stable estimation. The algorithm is discussed in detail for the delay discounting case, and results from decision making under risk for gains, losses, and mixed prospects are also provided. Simulated experiments using prescribed parameter values were performed to justify the algorithm in terms of the reproducibility of its parameters for individual assessments, and to test the reliability of the estimation procedure in a group-level analysis. The algorithm was implemented as an experimental battery to measure temporal and probability discounting rates together with loss aversion, and was tested on a healthy participant sample.


Delay discounting Risk seeking Intertemporal choice Loss aversion Bayesian estimation 


Author note

We thank Zeb Kurth-Nelson for sharing his ideas on the mathematical framework, Nils B. Kroemer for insightful discussions, and Elisabeth Jünger and Christian Sommer for collecting the pilot data. This study was supported by the Deutsche Forschungsgemeinschaft (DFG FOR 1617 Grants RA 1047/2-1, SM 80/7-1, and SM 80/7-2; DFG SPP 1226 Grant SM 80/5-2; and DFG SFB 940/1 and SFB 940/2 grants). Q.J.M.H. and M.S. contributed to the conception and design of the study. A.G. and Q.J.M.H. implemented the mathematical algorithm, which was improved by S.P. for this work. Piloting of participants and data collection were performed by N.B., and N.B. and S.P. drafted the manuscript. All authors provided critical revision of the manuscript for important intellectual content and approved the final version for publication.

Supplementary material

13428_2017_866_MOESM1_ESM.pdf (197 kb)
Figure S1 (PDF 197 kb)
13428_2017_866_MOESM2_ESM.pdf (195 kb)
Figure S2 (PDF 194 kb)


  1. Ainslie, G. (1975). Specious reward: Behavioral theory of impulsiveness and impulse control. Psychological Bulletin, 82, 463–496. doi: 10.1037/H0076860 CrossRefPubMedGoogle Scholar
  2. Baker, F., Johnson, M. W., & Bickel, W. K. (2003). Delay discounting in current and never-before cigarette smokers: Similarities and differences across commodity, sign, and magnitude. Journal of Abnormal Psychology, 112, 382–392.CrossRefPubMedGoogle Scholar
  3. Beck, R. C., & Triplett, M. F. (2009). Test–retest reliability of a group-administered paper-pencil measure of delay discounting. Experimental and Clinical Psychopharmacology, 17, 345–355. doi: 10.1037/a0017078 CrossRefPubMedGoogle Scholar
  4. Bickel, W. K., Jarmolowicz, D. P., Mueller, E. T., Koffarnus, M. N., & Gatchalian, K. M. (2012). Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: Emerging evidence. Pharmacology & Therapeutics, 134, 287–297. doi: 10.1016/j.pharmthera.2012.02.004 CrossRefGoogle Scholar
  5. Bjork, J. M., Hommer, D. W., Grant, S. J., & Danube, C. (2004). Impulsivity in abstinent alcohol-dependent patients: Relation to control subjects and type 1-/type 2-like traits. Alcohol, 34, 133–150. doi: 10.1016/j.alcohol.2004.06.012 CrossRefPubMedGoogle Scholar
  6. Blanco, C., Potenza, M. N., Kim, S. W., Ibáñez, A., Zaninelli, R., Saiz-Ruiz, J., & Grant, J. E. (2009). A pilot study of impulsivity and compulsivity in pathological gambling. Psychiatry Research, 167, 161–168. doi: 10.1016/j.psychres.2008.04.023 CrossRefPubMedPubMedCentralGoogle Scholar
  7. Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10, 433–436. doi: 10.1163/156856897X00357 CrossRefPubMedGoogle Scholar
  8. Burns, K., & Bechara, A. (2007). Decision making and free will: A neuroscience perspective. Behavioral Sciences and the Law, 25, 263–280. doi: 10.1002/bsl.751 CrossRefPubMedGoogle Scholar
  9. Craig, A. R., Maxfield, A. D., Stein, J. S., Renda, C. R., & Madden, G. J. (2014). Do the adjusting-delay and increasing-delay tasks measure the same construct: Delay discounting? Behavioural Pharmacology, 25, 306–315. doi: 10.1097/Fbp.0000000000000055 CrossRefPubMedPubMedCentralGoogle Scholar
  10. Dom, G., D’haene, P., Hulstijn, W., & Sabbe, B. (2006). Impulsivity in abstinent early- and late-onset alcoholics: Differences in self-report measures and a discounting task. Addiction, 101, 50–59. doi: 10.1111/j.1360-0443.2005.01270.x CrossRefPubMedGoogle Scholar
  11. Doyle, J. R. (2013). Survey of time preference, delay discounting models. Judgment and Decision Making, 8, 116–135.Google Scholar
  12. Ert, E., & Erev, I. (2008). The rejection of attractive gambles, loss aversion, and the lemon avoidance heuristic. Journal of Economic Psychology, 29, 715–723. doi: 10.1016/j.joep.2007.06.003 CrossRefGoogle Scholar
  13. Frydman, C., Camerer, C., Bossaerts, P., & Rangel, A. (2011). MAOA-L carriers are better at making optimal financial decisions under risk. Proceedings of the Royal Society B, 278, 2053–2059. doi: 10.1098/rspb.2010.2304 CrossRefPubMedGoogle Scholar
  14. Garvert, M. M., Moutoussis, M., Kurth-Nelson, Z., Behrens, T. E., & Dolan, R. J. (2015). Learning-induced plasticity in medial prefrontal cortex predicts preference malleability. Neuron, 85, 418–428. doi: 10.1016/j.neuron.2014.12.033 CrossRefPubMedPubMedCentralGoogle Scholar
  15. Green, L., & Myerson, J. (2004). A discounting framework for choice with delayed and probabilistic rewards. Psychological Bulletin, 130, 769–792. doi: 10.1037/0033-2909.130.5.769 CrossRefPubMedPubMedCentralGoogle Scholar
  16. Green, L., Fry, A. F., & Myerson, J. (1994). Discounting of delayed rewards: A life-span comparison. Psychological Science, 5, 33–36. doi: 10.1111/j.1467-9280.1994.tb00610.x CrossRefGoogle Scholar
  17. Green, L., Myerson, J., Shah, A. K., Estle, S. J., & Holt, D. D. (2007). Do adjusting-amount and adjusting-delay procedures produce equivalent estimates of subjective value in pigeons? Journal of the Experimental Analysis of Behavior, 87, 337–347. doi: 10.1901/jeab.2007.37-06 CrossRefPubMedPubMedCentralGoogle Scholar
  18. Hare, T., Hakimi, S., & Rangel, A. (2014). Activity in dlPFC and its effective connectivity to vmPFC are associated with temporal discounting. Frontiers in Neuroscience, 8, 50. doi: 10.3389/fnins.2014.00050 CrossRefPubMedPubMedCentralGoogle Scholar
  19. Holt, D. D., Green, L., & Myerson, J. (2012). Estimating the subjective value of future rewards: Comparison of adjusting-amount and adjusting-delay procedures. Behavioural Processes, 90, 302–310. doi: 10.1016/j.beproc.2012.03.003 CrossRefPubMedPubMedCentralGoogle Scholar
  20. Johnson, M. W., & Bickel, W. K. (2002). Within-subject comparison of real and hypothetical money rewards in delay discounting. Journal of the Experimental Analysis of Behavior, 77, 129.CrossRefPubMedPubMedCentralGoogle Scholar
  21. Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263–291. doi: 10.2307/1914185 CrossRefGoogle Scholar
  22. Kahneman, D., & Tversky, A. (1984). Choices, values, and frames. American Psychologist, 39, 341–350. doi: 10.1037/0003-066X.39.4.341 CrossRefGoogle Scholar
  23. Kirby, K. N. (2009). One-year temporal stability of delay-discount rates. Psychonomic Bulletin & Review, 16, 457–462. doi: 10.3758/PBR.16.3.457 CrossRefGoogle Scholar
  24. Kirby, K. N., Petry, N. M., & Bickel, W. K. (1999). Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls. Journal of Experimental Psychology: General, 128, 78–87.CrossRefGoogle Scholar
  25. Koff, E., & Lucas, M. (2011). Mood moderates the relationship between impulsiveness and delay discounting. Personality and Individual Differences, 50, 1018–1022. doi: 10.1016/j.paid.2011.01.016 CrossRefGoogle Scholar
  26. Koffarnus, M. N., & Bickel, W. K. (2014). A 5-trial adjusting delay discounting task: Accurate discount rates in less than one minute. Experimental and Clinical Psychopharmacology, 22, 222–228. doi: 10.1037/a0035973 CrossRefPubMedPubMedCentralGoogle Scholar
  27. Koffarnus, M. N., Jarmolowicz, D. P., Mueller, E. T., & Bickel, W. K. (2013). Changing delay discounting in the light of the competing neurobehavioral decision systems theory: A review. Journal of the Experimental Analysis of Behavior, 99, 32–57. doi: 10.1002/jeab.2 CrossRefPubMedGoogle Scholar
  28. Krishnan-Sarin, S., Reynolds, B., Duhig, A. M., Smith, A., Liss, T., McFetridge, A.,…Potenza, M. N. (2007). Behavioral impulsivity predicts treatment outcome in a smoking cessation program for adolescent smokers. Drug and Alcohol Dependence, 88, 79–82. doi: 10.1016/j.drugalcdep.2006.09.006
  29. Lewi, J., Butera, R., & Paninski, L. (2008). Sequential optimal design of neurophysiology experiments. Neural Computation, 21, 619–687. doi: 10.1162/neco.2008.08-07-594 CrossRefGoogle Scholar
  30. Loewenstein, G. (1988). Frames of mind in intertemporal choice. Management Science, 34, 200–214.CrossRefGoogle Scholar
  31. Lovric, M. (2010). International encyclopedia of statistical science. New York, NY: Springer.Google Scholar
  32. MacKillop, J., & Kahler, C. W. (2009). Delayed reward discounting predicts treatment response for heavy drinkers receiving smoking cessation treatment. Drug and Alcohol Dependence, 104, 197–203. doi: 10.1016/j.drugalcdep.2009.04.020 CrossRefPubMedPubMedCentralGoogle Scholar
  33. MacKillop, J., Amlung, M. T., Few, L. R., Ray, L. A., Sweet, L. H., & Munafò, M. R. (2011). Delayed reward discounting and addictive behavior: A meta-analysis. Psychopharmacology, 216, 305–321. doi: 10.1007/s00213-011-2229-0 CrossRefPubMedPubMedCentralGoogle Scholar
  34. Madden, G. J., & Johnson, P. S. (2010). A delay-discounting primer. In G. J. M. W. K. Bickel (Ed.), Impulsivity: The behavioral and neurological science of discounting (pp. 11–37). Washington, DC: American Psychological Association.CrossRefGoogle Scholar
  35. Madden, G. J., Petry, N. M., Badger, G. J., & Bickel, W. K. (1997). Impulsive and self-control choices in opioid-dependent patients and non-drug-using control patients: Drug and monetary rewards. Experimental and Clinical Psychopharmacology, 5, 256.CrossRefPubMedGoogle Scholar
  36. Madden, G. J., Petry, N. M., & Johnson, P. S. (2009). Pathological gamblers discount probabilistic rewards less steeply than matched controls. Experimental and Clinical Psychopharmacology, 17, 283–290. doi: 10.1037/A0016806 CrossRefPubMedPubMedCentralGoogle Scholar
  37. Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative analysis of behavior: Vol. 5. The effect of delay and of intervening events on reinforcement value (pp. 55–73). Hillsdale, NJ: Erlbaum.Google Scholar
  38. Mitchell, J. M., Fields, H. L., D’Esposito, M., & Boettiger, C. A. (2005). Impulsive responding in alcoholics. Alcoholism: Clinical and Experimental Research, 29, 2158–2169. doi: 10.1097/01.alc.0000191755.63639.4a CrossRefGoogle Scholar
  39. Myerson, J., Green, L., & Warusawitharana, M. (2001). Area under the curve as a measure of discounting. Journal of the Experimental Analysis of Behavior, 76, 235–243. doi: 10.1901/jeab.2001.76-235 CrossRefPubMedPubMedCentralGoogle Scholar
  40. Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437–442. doi: 10.1163/156856897X00366 CrossRefPubMedGoogle Scholar
  41. Petry, N. M. (2003). Discounting of money, health, and freedom in substance abusers and controls. Drug and Alcohol Dependence, 71, 133–141. doi: 10.1016/S0376-8716(03)00090-5 CrossRefPubMedGoogle Scholar
  42. Petry, N. M. (2012). Discounting of probabilistic rewards is associated with gambling abstinence in treatment-seeking pathological gamblers. Journal of Abnormal Psychology, 121, 151–159. doi: 10.1037/A0024782 CrossRefPubMedGoogle Scholar
  43. Rachlin, H., Raineri, A., & Cross, D. (1991). Subjective-probability and delay. Journal of the Experimental Analysis of Behavior, 55, 233–244. doi: 10.1901/jeab.1991.55-233 CrossRefPubMedPubMedCentralGoogle Scholar
  44. Reynolds, B., Richards, J. B., Horn, K., & Karraker, K. (2004). Delay discounting and probability discounting as related to cigarette smoking status in adults. Behavioural Processes, 65, 35–42.CrossRefPubMedGoogle Scholar
  45. Richards, J. B., Zhang, L., Mitchell, S. H., & Wit, H. (1999). Delay or probability discounting in a model of impulsive behavior: Effect of alcohol. Journal of the Experimental Analysis of Behavior, 71, 121–143.CrossRefPubMedPubMedCentralGoogle Scholar
  46. Ripke, S., Hübner, T., Mennigen, E., Müller, K. U., Rodehacke, S., Schmidt, D.,…Smolka, M. N. (2012). Reward processing and intertemporal decision making in adults and adolescents: The role of impulsivity and decision consistency. Brain Research, 1478, 36–47. doi: 10.1016/j.brainres.2012.08.034
  47. Sebastiani, P., & Wynn, H. P. (2000). Maximum entropy sampling and optimal Bayesian experimental design. Journal of the Royal Statistical Society: Series B, 62, 145–157.CrossRefGoogle Scholar
  48. Shead, N. W., & Hodgins, D. C. (2009). Probability discounting of gains and losses: Implications for risk attitudes and impulsivity. Journal of the Experimental Analysis of Behavior, 92, 1–16. doi: 10.1901/Jeab.2009.92-1 CrossRefPubMedPubMedCentralGoogle Scholar
  49. Silberberg, A., Murray, P., Christensen, J., & Asano, T. (1988). Choice in the repeated-gambles experiment. Journal of the Experimental Analysis of Behavior, 50, 187–195. doi: 10.1901/jeab.1988.50-187 CrossRefPubMedPubMedCentralGoogle Scholar
  50. Simpson, C. A., & Vuchinich, R. E. (2000). Reliability of a measure of temporal discounting. Psychological Record, 50(1), 3–16.CrossRefGoogle Scholar
  51. Smith, C. L., & Hantula, D. A. (2008). Methodological considerations in the study of delay discounting in intertemporal choice: A comparison of tasks and modes. Behavior Research Methods, 40, 940–953. doi: 10.3758/brm.40.4.940 CrossRefPubMedGoogle Scholar
  52. Stanger, C., Ryan, S. R., Fu, H., Landes, R. D., Jones, B. A., Bickel, W. K., & Budney, A. J. (2012). Delay discounting predicts adolescent substance abuse treatment outcome. Experimental and Clinical Psychopharmacology, 20, 205–212. doi: 10.1037/a0026543 CrossRefPubMedGoogle Scholar
  53. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction (Vol. 1). Cambridge, MA: MIT Press.Google Scholar
  54. Takahashi, T., Takagishi, H., Nishinaka, H., Makino, T., & Fukui, H. (2014). Neuroeconomics of psychopathy: Risk taking in probability discounting of gain and loss predicts psychopathy. Neuroendocrinology Letters, 35, 510–517.PubMedGoogle Scholar
  55. Thaler, R. H., & Johnson, E. J. (1990). Gambling with the house money and trying to break even: The effects of prior outcomes on risky choice. Management Science, 36, 643–660.CrossRefGoogle Scholar
  56. Tom, S. M., Fox, C. R., Trepel, C., & Poldrack, R. A. (2007). The neural basis of loss aversion in decision-making under risk. Science, 315, 515–518. doi: 10.1126/science.1134239 CrossRefPubMedGoogle Scholar
  57. Tversky, A., & Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and Uncertainty, 5, 297–323.CrossRefGoogle Scholar
  58. Vincent, B. T. (2016). Hierarchical Bayesian estimation and hypothesis testing for delay discounting tasks. Behavior Research Methods, 48, 1608–1620. doi: 10.3758/s13428-015-0672-2 CrossRefPubMedGoogle Scholar
  59. Von Neumann, J., & Morgenstern, O. (1944). Theory of games and economic behavior. Princeton, NJ: Princeton University Press.Google Scholar
  60. Yechiam, E., Busemeyer, J. R., Stout, J. C., & Bechara, A. (2005). Using cognitive models to map relations between neuropsychological disorders and human decision-making deficits. Psychological Science, 16, 973–978. doi: 10.1111/j.1467-9280.2005.01646.x CrossRefPubMedGoogle Scholar
  61. Yoon, H., & Chapman, G. B. (2016). A closer look at the yardstick: A new discount rate measure with precision and range. Journal of Behavioral Decision Making, 29, 470–480. doi: 10.1002/bdm.1890 CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  • Shakoor Pooseh
    • 1
  • Nadine Bernhardt
    • 1
  • Alvaro Guevara
    • 1
    • 2
  • Quentin J. M. Huys
    • 3
    • 4
  • Michael N. Smolka
    • 1
  1. 1.Department of Psychiatry and PsychotherapyTechnische Universität DresdenDresdenGermany
  2. 2.Escuela de MatemáticaUniversidad de Costa RicaCiudad universitaria Rodrigo Facio BrenesCosta Rica
  3. 3.Translational Neuromodeling Unit, Hospital of PsychiatryUniversity of Zürich and Swiss Federal Institute of TechnologyZurichSwitzerland
  4. 4.Psychiatry, Psychosomatics, and PsychotherapyUniversity of ZürichZurichSwitzerland

Personalised recommendations