Attention, Perception, & Psychophysics

, Volume 79, Issue 8, pp 2299–2309 | Cite as

How humans react to changing rewards during visual foraging

  • Jinxia Zhang
  • Xue Gong
  • Daryl Fougnie
  • Jeremy M. Wolfe


Much is known about the speed and accuracy of search in single-target search tasks, but less attention has been devoted to understanding search in multiple-target foraging tasks. These tasks raise and answer important questions about how individuals decide to terminate searches in cases in which the number of targets in each display is unknown. Even when asked to find every target, individuals quit before exhaustively searching a display. Because a failure to notice targets can have profound effects (e.g., missing a malignant tumor in an X-ray), it is important to develop strategies that could limit such errors. Here, we explored the impact of different reward patterns on these failures. In the Neutral condition, reward for finding a target was constant over time. In the Increasing condition, reward increased for each successive target in a display, penalizing early departure from a display. In the Decreasing condition, reward decreased for each successive target in a display. The experimental results demonstrate that observers will forage for longer (and find more targets) when the value of successive targets increases (and the opposite when value decreases). The data indicate that observers were learning to utilize knowledge of the reward pattern and to forage optimally over the course of the experiment. Simulation results further revealed that human behavior could be modeled with a variant of Charnov’s Marginal Value Theorem (MVT) (Charnov, 1976) that includes roles for reward and learning.


Human foraging Optimal foraging Search termination Reward pattern Visual search 



This research was supported by grants to J.M.W. from ONR MURI (N000141010278), NIH-NEI (EY017001), Hewlett-Packard, Google, CELEST (NSF SBE-0354378) and by the National Natural Science Fund of China (Grant numbers 61233011, 61374006, 61473086); Major Program of National Natural Science Foundation of China (Grant number 11190015); Natural Science Foundation of Jiangsu (Grant number BK20170692, BK20131300); the Innovation Fund of Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education (Nanjing University of Science and Technology, Grant number JYB201601); the Innovation Fund of Key Laboratory of Measurement and Control of Complex Systems of Engineering(Southeast University, Grant number MCCSE2017B01); the Fundamental Research Funds for the Central Universities (2242016k30009).


  1. Anderson, B. A., Laurent, P. A. & Yantis, S. (2011), ‘Value-driven attentional capture’, Proceedings of the National Academy of Sciences of the United States of America 108(25), 10367–10371.CrossRefPubMedPubMedCentralGoogle Scholar
  2. Anderson, B. A., Laurent, P. A. & Yantis, S. (2012), ‘Generalization of value-based attentional priority’, Visual Cognition 20(6), 647–658.CrossRefGoogle Scholar
  3. Anderson, B. A. & Yantis, S. (2013), ‘Persistence of value-driven attentional capture.’, Journal of Experimental Psychology: Human Perception and Performance 39(1), 6.PubMedGoogle Scholar
  4. Ashman, C., Yu, J. & Wolfman, D. (2000), ‘Satisfaction of search in osteoradiology’, American Journal of Roentgenology 177, 252–253.Google Scholar
  5. Berbaum, K., Caldwell, R., Schartz, K., Thompson, B. & Franken E. Jr. (2007), ‘Does computer-aided diagnosis for lung tumors change satisfaction of search in chest radiography?’, Academic Radiology 14, 1069–1076.CrossRefPubMedPubMedCentralGoogle Scholar
  6. Berbaum, K., Franken E. Jr., Caldwell, R. & Schartz, K. (2006), ‘Can a checklist reduce SOS errors in chest radiography?’, Academic Radiology 13, 296–304.CrossRefPubMedGoogle Scholar
  7. Berbaum, K., Franken, E., Dorfman, D., Caldwell, R. & Krupinski, E. (2000), ‘Role of faulty decision making in the satisfaction of search effect in chest radiography’, Academic Radiology 7, 1098–1106.CrossRefPubMedGoogle Scholar
  8. Berbaum, K., Franken, E., Dorfman, D., Rooholamini, S., Kathol, M., Barloon, T., Behlke, F., Sato, Y., Lu, C. & el Khoury, G. (1990), ‘Satisfaction of search in diagnostic radiology’, Investigative Radiology 25, 133–140.CrossRefPubMedGoogle Scholar
  9. Brainard, D. H. (1997), ‘The psychophysics toolbox’, Spatial vision 10, 433–436.CrossRefPubMedGoogle Scholar
  10. Cain, M. S., Vul, E., Clark, K. & Mitroff, S. R. (2012). A Bayesian optimal foraging model of human visual search. Psychological Science. 1–8.Google Scholar
  11. Charnov, E. L. (1976), ‘Optimal foraging, the marginal value theorem’, Theoretical Population Biology 9(2), 129–136.CrossRefPubMedGoogle Scholar
  12. Constantino, S. M. & Daw, N. D. (2015), ‘Learning the opportunity cost of time in a patch-foraging task’, Cognitive, Affective, & Behavioral Neuroscience 15(4), 837–853.CrossRefGoogle Scholar
  13. Della Libera, C. & Chelazzi, L. (2009), ‘Learning to attend and to ignore is a matter of gains and losses’, Psychological Science 20(6), 778–784.CrossRefPubMedGoogle Scholar
  14. Desimone, R. & Duncan, J. (1995), ‘Neural mechanisms of selective visual attention’, Annual Review of Neuroscience 18, 193–222.CrossRefPubMedGoogle Scholar
  15. Evans, K. K., Birdwell, R. L. & Wolfe, J. M. (2013), ‘If you don’t find it often, you often don’t find it: why some cancers are missed in breast cancer screening’, PLoS One 8(5), e64366.CrossRefPubMedPubMedCentralGoogle Scholar
  16. Fleck, M., Samei, E. & Mitroff, S. (2010), ‘Generalized "satisfaction of search": adverse influences on dual-target search accuracy’, Journal of Experimental Psychology: Applied 16, 60–71.PubMedGoogle Scholar
  17. Fougnie, D., Cormiea, S. M., Zhang, J., Alvarez, G. A. & Wolfe, J. M. (2015), ‘Winter is coming: how humans forage in a temporally structured environment’, Journal of Vision 15, 1–17.CrossRefPubMedPubMedCentralGoogle Scholar
  18. Franken, E., Berbaum, K., Lu, C., Kannam, S., Dorfman, D., Warnock, N., Simonson, T. & Pelsang R.E. (1994), ‘Satisfaction of search in the detection of plain-film abnormalities in abdominal contrast studies’, Investigative Radiology 29, 403–409.CrossRefPubMedGoogle Scholar
  19. Hickey, C., Chelazzi, L. & Theeuwes, J. (2010a), ‘Reward changes salience in human vision via the anterior cingulate’, Journal of Neuroscience 30(33), 11096–11103.CrossRefPubMedGoogle Scholar
  20. Hickey, C., Chelazzi, L. & Theeuwes, J. (2010b), ‘Reward guides vision when it’s your thing: trait reward-seeking in reward-mediated visual priming’, PLoS One 5(11), e14087.CrossRefPubMedPubMedCentralGoogle Scholar
  21. Hills, T. T., Jones, M. N. & Todd, P. M. (2012), ‘Optimal foraging in semantic memory.’, Psychological Review 119(2), 431–440.CrossRefPubMedGoogle Scholar
  22. Hutchinson, J. M. C., Wilke, A. & Todd, P. M. (2008), ‘Patch leaving in humans: can a generalist adapt its rules to dispersal of items across patches?’, Animal Behaviour 75, 1331–1349.CrossRefGoogle Scholar
  23. Koch, C. & Ullman, S. (1985), ‘Shifts in selective visual attention: towards the underlying neural circuitry’, Human Neurobiology 4, 219–227.PubMedGoogle Scholar
  24. McNamara, J. (1985). An optimal sequential policy for controlling a Markov renewal process. Journal of Applied Probability. 324–335.Google Scholar
  25. Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D. & Iverson, G. (2009), ‘Bayesian t tests for accepting and rejecting the null hypothesis’, Psychonomic Bulletin & Review 16(2), 225–237.CrossRefGoogle Scholar
  26. Samuel, S., Kundel, H., Nodine, C. & Toto, L. (1995), ‘Mechanisms of satisfaction of search: eye position recordings in the reading of chest radiographs’, Radiology 194, 895–902.CrossRefPubMedGoogle Scholar
  27. Stephens, D. W., Brown, J. S. & Ydenberg, R. C. (2007). Foraging: behavior and ecology, University of Chicago Press.Google Scholar
  28. Stephens, D. W. & Krebs, J. R. (1986). Foraging theory, Princeton University Press.Google Scholar
  29. Treisman, A. M. & Gelade, G. (1980), ‘A feature-integration theory of attention’, Cognitive Psychology 12, 97–136.CrossRefPubMedGoogle Scholar
  30. Tsotsos, J., Culhane, S., Wai, W., Lai, Y., Davis, N. & Nuflo, F. (1995), ‘Modeling visual attention via selective tuning’, Artificial Intelligence 78, 507–545.CrossRefGoogle Scholar
  31. Wang, L., Yu, H. & Zhou, X. (2013), Interaction between value and perceptual salience in value-driven attentional capture’, Journal of Vision 13(3).Google Scholar
  32. Wilke, A., Hutchinson, J., Todd, P. & Czienskowski, U. (2009), ‘Fishing for the right words: decision rules for human foraging behavior in internal search tasks.’, Cognitive Science 33, 497–529.CrossRefPubMedGoogle Scholar
  33. Wolfe, J. (1994), ‘Guided search 2.0: a revised model of visual search’, Psychonomic Bulletin & Review 1(2), 202–238.CrossRefGoogle Scholar
  34. Wolfe, J. (2012), Approaches to visual search: feature integration theory and guided search. In: Nobre, A.C. & Kastner, S. (Eds) Oxford Handbook of Attention, New York: Oxford University Press.Google Scholar
  35. Wolfe, J. M. (2013), ‘When is it time to move to the next raspberry bush? Foraging rules in human visual search’, Journal of Vision 13, 1–17.Google Scholar
  36. Wolfe, J. M., Brunelli, D. N., Rubinstein, J. & Horowitz, T. S. (2013), ‘Prevalence effects in newly trained airport checkpoint screeners: trained observers miss rare targets, too’, Journal of vision 13(3), 33–33.CrossRefPubMedPubMedCentralGoogle Scholar
  37. Zhang, J., Gong, X., Fougnie, D. & Wolfe, J. M. (2015), ‘Using the past to anticipate the future in human foraging behavior’, Vision Research 111, 66–74.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© The Psychonomic Society, Inc. 2017

Authors and Affiliations

  • Jinxia Zhang
    • 1
    • 2
  • Xue Gong
    • 3
  • Daryl Fougnie
    • 4
  • Jeremy M. Wolfe
    • 2
    • 5
  1. 1.Key Laboratory of Measurement and Control of CSE, Ministry of Education, School of AutomationSoutheast UniversityNanjingChina
  2. 2.Visual Attention LaboratoryBrigham and Women’s HospitalCambridgeUSA
  3. 3.Psychiatry Neuroimaging LaboratoryBrigham and Women’s HospitalBostonUSA
  4. 4.Science DivisionNew York University Abu DhabiAbu DhabiUnited Arab Emirates
  5. 5.Harvard Medical SchoolBostonUSA

Personalised recommendations