Psychonomic Bulletin & Review

, Volume 26, Issue 6, pp 1911–1916 | Cite as

The role of uncertainty in attentional and choice exploration

  • Adrian R. WalkerEmail author
  • David Luque
  • Mike E. Le Pelley
  • Tom Beesley


The exploitation-exploration (EE) trade-off describes how, when making a decision, an organism must often choose between a safe alternative with a known pay-off, and one or more riskier alternatives with uncertain pay-offs. Recently, the concept of the EE trade-off has been extended to the examination of how organisms distribute limited attentional resources between several stimuli. This work suggests that when the rules governing the environment are certain, participants learn to “exploit” by attending preferentially to cues that provide the most information about upcoming events. However, when the rules are uncertain, people “explore” by increasing their attention to all cues that may provide information to help in predicting upcoming events. In the current study, we examine how uncertainty affects the EE trade-off in attention using a contextual two-armed bandit task, where participants explore with both their attention and their choice behavior. We find evidence for an influence of uncertainty on the EE trade-off in both choice and attention. These findings provide support for the idea of an EE trade-off in attention, and that uncertainty is a primary motivator for exploration in both choice and attentional allocation.


Attention Human associative learning Exploration Uncertainty 



This work was supported by an Australian Research Council Discovery Project (DP140103268), and a Research Training Program scholarship from the Australian Department of Education and Training.


  1. Beesley, T., Nguyen, K. P., Pearson, D., & Le Pelley, M. E. (2015). Uncertainty and predictiveness determine attention to cues during human associative learning. Quarterly Journal of Experimental Psychology, 68, 2175–2199.CrossRefGoogle Scholar
  2. Braunlich, K., & Love, B. C. (2019). Occipitotemporal representations reflect individual differences in conceptual knowledge. Journal of Experimental Psychology: General, 148, 1192–1203.Google Scholar
  3. Cohen, J. D., McClure, S. M., & Yu, A. J. (2007). Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481), 933–942.CrossRefGoogle Scholar
  4. Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441, 876–879.CrossRefGoogle Scholar
  5. Easdale, L. C., Le Pelley, M. E., & Beesley, T. (2019). The onset of uncertainty facilitates the learning of new associations by increasing attention to cues. Quarterly Journal of Experimental Psychology, 72, 193–208.Google Scholar
  6. Herrnstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4(3), 267–272.CrossRefGoogle Scholar
  7. Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263-291.CrossRefGoogle Scholar
  8. Kleiner, M., Brainard, D., Pelli, D., (2007). What’s new in Psychtoolbox-3. Perception, 36, 1.Google Scholar
  9. Knox, W. B., Otto, A. R., Stone, P., & Love, B. C. (2012). The nature of belief-directed exploratory choice in human decision-making. Frontiers in Psychology, 2, 1–12.CrossRefGoogle Scholar
  10. Le Pelley, M. E., Beesley, T., & Griffiths, O. (2011). Overt attention and predictiveness in human contingency learning. Journal of Experimental Psychology: Animal Behavior Processes, 37, 220–229.PubMedGoogle Scholar
  11. Le Pelley, M. E., & McLaren, I. P. L. (2003). Learned associability and associative change in human causal learning. Quarterly Journal of Experimental Psychology, 56B, 68–79.CrossRefGoogle Scholar
  12. Luque, D., Vadillo, M. A., Le Pelley, M. E., & Beesley, T. (2017). Prediction and uncertainty in associative learning: Examining controlled and automatic components of learned attentional biases. Quarterly Journal of Experimental Psychology, 70, 1485–1503.CrossRefGoogle Scholar
  13. Mehlhorn, K., Newell, B. R., Todd, P. M., Lee, M. D., Morgan, K., Braithwaite, V. A., . . . Gonzalez, C. (2015). Unpacking the exploration-exploitation tradeoff: A synthesis of human and animal literatures. Decision, 2, 191–215.CrossRefGoogle Scholar
  14. Niv, Y., Daniel, R., Geana, A., Gershman, S. J., Leong, Y. C., Radulescu, A., & Wilson, R. C. (2015). Reinforcement learning in multidimensional environments relies on attention mechanisms. Journal of Neuroscience, 35, 8145–8157.CrossRefGoogle Scholar
  15. Rehder, B., & Hoffman, A. B. (2005). Eyetracking and selective attention in category learning. Cognitive Psychology, 51(1), 1–41.Google Scholar
  16. Schulz, E., Konstantinidis, E., & Speekenbrink, M. (2018). Putting bandits into context: How function learning supports decision making. Journal of Experimental Psychology: Learning, Memory, & Cognition, 44, 927-943.Google Scholar
  17. Shanks, D. R., Tunney, R. J., & McCarthy, J. D. (2002). A re-examination of probability matching and rational choice. Journal of Behavioral Decision Making, 15, 233–250.CrossRefGoogle Scholar
  18. Speekenbrink, M., & Konstantinidis, E. (2015). Uncertainty and exploration in a restless bandit problem. Topics in Cognitive Science, 7, 351–367.CrossRefGoogle Scholar
  19. Wasserman, E. A., Elek, S. M., Chatlosh, D. L., & Baker, A. G. (1993). Rating causal relations: Role of probability in judgments of response-outcome contingency. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 174–188.Google Scholar

Copyright information

© The Psychonomic Society, Inc. 2019

Authors and Affiliations

  1. 1.School of PsychologyUniversity of New South WalesSydneyAustralia
  2. 2.Universidad Autónoma de MadridMadridSpain
  3. 3.Lancaster UniversityLancasterUK

Personalised recommendations