Skip to main content

Learning when to wait and when to act

Abstract

Choosing how long to wait in order to optimize reward is a complex decision. We embedded these decisions within a video-game environment in which the amount of reward smoothly increased the longer one waited. The availability of external cues varied in order to determine how they affected the decision to wait to achieve the goal of maximizing the reward rate. As a group, people were most optimal when they could directly observe the growth in reward, and this information overshadowed a static color cue that did not require extended observation. These results were considered within the context of improving the choice between acting versus waiting in order to maximize reward rates.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. Ainslie, E., & Herrnstein, R. J. (1981). Preference reversal and delayed reinforcement. Animal Learning & Behavior, 9, 476–482.

    Article  Google Scholar 

  2. Danckert, J. (2019). Boredom: Managing the delicate balance between exploration and exploitation. In J. R. Velasco (Ed.), Boredom is in your mind: A shared psychological-philosophical approach (pp. 37–53): Springer.

    Chapter  Google Scholar 

  3. Donahoe, J. W. (1970). Stimulus control within response sequences. In J. H. Reynierse (Ed.), Current issues in animal learning: A colloquium (pp. 233–293). University of Nebraska Press.

    Google Scholar 

  4. Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforcement. Appleton-Century-Crofts. https://doi.org/10.1037/10627-000

  5. Holland, J. H. (1975). Adaptation in natural and artificial systems. University of Michigan Press.

    Google Scholar 

  6. Jimura, K., Myerson, J., Hilgard, J., Braver, T. S., & Green, L. (2009). Are people really more patient than other animals? Evidence from human discounting of real liquid rewards. Psychonomic Bulletin & Review, 16, 1071–1075.

    Article  Google Scholar 

  7. Johnson, E. J., & Payne, J. W. (1985). Effort and accuracy in choice. Management Science, 31, 395–414.

    Article  Google Scholar 

  8. Mazur, J. E. (1987). An adjusting delay procedure for studying delayed reinforcement. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. C. Rachlin (Eds.), The effect of delay and intervening events on reinforcement value (Vol. 5, pp. 55–73). Erlbaum.

    Google Scholar 

  9. McGuire, J. T., & Kable, J. W. (2012). Decision makers calibrate behavioral persistence on the basis of time-interval experience. Cognition, 124, 216–226. https://doi.org/10.1016/j.cognition.2012.03.008

    Article  PubMed  PubMed Central  Google Scholar 

  10. Pastore, A., Esposito, U., & Vasilaki, E. (2015). Modelling stock-market investors as reinforcement learning agents. IEEE International Conference on Evolving and Adaptive Intelligent Systems, 1–6. https://doi.org/10.1109/EAIS.2015.7368789

  11. Przybylski, A. K., Ryan, R. M., & Rigby, C. S. (2009). The motivating role of violence in video games. Personality and Social Psychology Bulletin, 35, 243–259. https://doi.org/10.1177/0146167208327216

    Article  PubMed  Google Scholar 

  12. Rachlin, H. C., Raineri, A., & Cross, D. (1991). Subjective probability and delay. Journal of the Experimental Analysis of Behavior, 55, 233–244.

    Article  Google Scholar 

  13. Reynolds, B., & Schiffbauer, R. (2004). Measuring state changes in human delay discounting: An experiential discounting task. Behavioural Processes, 67, 343–356.

    Article  Google Scholar 

  14. Richards, J. B., Mitchell, S. H., de Wit, H., & Seiden, L. S. (1997). Determination of discount functions in rats with an adjusting-amount procedure. Journal of the Experimental Analysis of Behavior, 67, 353–366.

    Article  Google Scholar 

  15. Schachter, S., Ouellette, R., Whittle, B., & Gerin, W. (1987). Effects of trend and of profit or loss on the tendency to sell stock. Basic and Applied Social Psychology, 8, 259–271.

  16. Skinner, B. F. (1938). The behavior of organisms. Appleton-Century-Crofts.

    Google Scholar 

  17. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. MIT Press.

    Google Scholar 

  18. Webb, T. (2020). Delay discounting, self-control training, and academic achievement center utilization (Doctoral dissertation). Southern Illinois University. (Proquest Number 27831770)

    Google Scholar 

  19. Yakobi, O., & Danckert, J. (2021). Boredom proneness is associated with noisy decision-making, not risk-taking. Experimental Brain Research, 239, 1807–1825. https://doi.org/10.1007/s00221-021-06098-5

  20. Young, M. E., & McCoy, A. W. (2015). A delay discounting task produces a greater likelihood of waiting than a deferred gratification task. Journal of the Experimental Analysis of Behavior, 103, 180–195. https://doi.org/10.1002/jeab.119

    Article  PubMed  Google Scholar 

  21. Young, M. E., Sutherland, S. C., Cole, J., & Nguyen, N. (2011a). Waiting to decide helps in the face of whether uncertainty but not when uncertainty. Learning & Behavior, 39, 115–124.

    Article  Google Scholar 

  22. Young, M. E., Webb, T. L., & Jacobs, E. A. (2011b). Deciding when to “cash in” when outcomes are continuously improving: An escalating interest task. Behavioural Processes, 88, 101–110.

    Article  Google Scholar 

  23. Young, M. E., Webb, T. L., Rung, J. M., & Jacobs, E. A. (2013a). Sensitivity to changing contingencies in an impulsivity task. Journal of the Experimental Analysis of Behavior, 99, 335–345.

  24. Young, M. E., Webb, T. L., Rung, J. M., & McCoy, A. W. (2014). Outcome probability versus magnitude: When waiting benefits one at the cost of the other. PLOS ONE, 9(6), Article e98996. https://doi.org/10.1371/journal.pone.0098996

  25. Young, M. E., Webb, T. L., Sutherland, S. C., & Jacobs, E. A. (2013b). Magnitude effects for experienced rewards at short delays in the escalating interest task. Psychonomic Bulletin & Review, 20, 302–309.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Michael E. Young.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

ESM 1

(PDF 302 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Young, M.E., Howatt, B.C. Learning when to wait and when to act. Learn Behav (2021). https://doi.org/10.3758/s13420-021-00482-5

Download citation

Keywords

  • Waiting
  • Video game
  • Dynamic decision-making