Choosing how long to wait in order to optimize reward is a complex decision. We embedded these decisions within a video-game environment in which the amount of reward smoothly increased the longer one waited. The availability of external cues varied in order to determine how they affected the decision to wait to achieve the goal of maximizing the reward rate. As a group, people were most optimal when they could directly observe the growth in reward, and this information overshadowed a static color cue that did not require extended observation. These results were considered within the context of improving the choice between acting versus waiting in order to maximize reward rates.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Ainslie, E., & Herrnstein, R. J. (1981). Preference reversal and delayed reinforcement. Animal Learning & Behavior, 9, 476–482.
Danckert, J. (2019). Boredom: Managing the delicate balance between exploration and exploitation. In J. R. Velasco (Ed.), Boredom is in your mind: A shared psychological-philosophical approach (pp. 37–53): Springer.
Donahoe, J. W. (1970). Stimulus control within response sequences. In J. H. Reynierse (Ed.), Current issues in animal learning: A colloquium (pp. 233–293). University of Nebraska Press.
Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforcement. Appleton-Century-Crofts. https://doi.org/10.1037/10627-000
Holland, J. H. (1975). Adaptation in natural and artificial systems. University of Michigan Press.
Jimura, K., Myerson, J., Hilgard, J., Braver, T. S., & Green, L. (2009). Are people really more patient than other animals? Evidence from human discounting of real liquid rewards. Psychonomic Bulletin & Review, 16, 1071–1075.
Johnson, E. J., & Payne, J. W. (1985). Effort and accuracy in choice. Management Science, 31, 395–414.
Mazur, J. E. (1987). An adjusting delay procedure for studying delayed reinforcement. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. C. Rachlin (Eds.), The effect of delay and intervening events on reinforcement value (Vol. 5, pp. 55–73). Erlbaum.
McGuire, J. T., & Kable, J. W. (2012). Decision makers calibrate behavioral persistence on the basis of time-interval experience. Cognition, 124, 216–226. https://doi.org/10.1016/j.cognition.2012.03.008
Pastore, A., Esposito, U., & Vasilaki, E. (2015). Modelling stock-market investors as reinforcement learning agents. IEEE International Conference on Evolving and Adaptive Intelligent Systems, 1–6. https://doi.org/10.1109/EAIS.2015.7368789
Przybylski, A. K., Ryan, R. M., & Rigby, C. S. (2009). The motivating role of violence in video games. Personality and Social Psychology Bulletin, 35, 243–259. https://doi.org/10.1177/0146167208327216
Rachlin, H. C., Raineri, A., & Cross, D. (1991). Subjective probability and delay. Journal of the Experimental Analysis of Behavior, 55, 233–244.
Reynolds, B., & Schiffbauer, R. (2004). Measuring state changes in human delay discounting: An experiential discounting task. Behavioural Processes, 67, 343–356.
Richards, J. B., Mitchell, S. H., de Wit, H., & Seiden, L. S. (1997). Determination of discount functions in rats with an adjusting-amount procedure. Journal of the Experimental Analysis of Behavior, 67, 353–366.
Schachter, S., Ouellette, R., Whittle, B., & Gerin, W. (1987). Effects of trend and of profit or loss on the tendency to sell stock. Basic and Applied Social Psychology, 8, 259–271.
Skinner, B. F. (1938). The behavior of organisms. Appleton-Century-Crofts.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. MIT Press.
Webb, T. (2020). Delay discounting, self-control training, and academic achievement center utilization (Doctoral dissertation). Southern Illinois University. (Proquest Number 27831770)
Yakobi, O., & Danckert, J. (2021). Boredom proneness is associated with noisy decision-making, not risk-taking. Experimental Brain Research, 239, 1807–1825. https://doi.org/10.1007/s00221-021-06098-5
Young, M. E., & McCoy, A. W. (2015). A delay discounting task produces a greater likelihood of waiting than a deferred gratification task. Journal of the Experimental Analysis of Behavior, 103, 180–195. https://doi.org/10.1002/jeab.119
Young, M. E., Sutherland, S. C., Cole, J., & Nguyen, N. (2011a). Waiting to decide helps in the face of whether uncertainty but not when uncertainty. Learning & Behavior, 39, 115–124.
Young, M. E., Webb, T. L., & Jacobs, E. A. (2011b). Deciding when to “cash in” when outcomes are continuously improving: An escalating interest task. Behavioural Processes, 88, 101–110.
Young, M. E., Webb, T. L., Rung, J. M., & Jacobs, E. A. (2013a). Sensitivity to changing contingencies in an impulsivity task. Journal of the Experimental Analysis of Behavior, 99, 335–345.
Young, M. E., Webb, T. L., Rung, J. M., & McCoy, A. W. (2014). Outcome probability versus magnitude: When waiting benefits one at the cost of the other. PLOS ONE, 9(6), Article e98996. https://doi.org/10.1371/journal.pone.0098996
Young, M. E., Webb, T. L., Sutherland, S. C., & Jacobs, E. A. (2013b). Magnitude effects for experienced rewards at short delays in the escalating interest task. Psychonomic Bulletin & Review, 20, 302–309.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Young, M.E., Howatt, B.C. Learning when to wait and when to act. Learn Behav (2021). https://doi.org/10.3758/s13420-021-00482-5
- Video game
- Dynamic decision-making