Advertisement

Cognitive Computation

, Volume 3, Issue 1, pp 48–63 | Cite as

If Visual Saliency Predicts Search, Then Why? Evidence from Normal and Gaze-Contingent Search Tasks in Natural Scenes

  • Tom FoulshamEmail author
  • Geoffrey Underwood
Article

Abstract

The Itti and Koch (Vision Research 40: 1489–1506, 2000) saliency map model has inspired a wealth of research testing the claim that bottom-up saliency determines the placement of eye fixations in natural scenes. Although saliency seems to correlate with (although not necessarily cause) fixation in free-viewing or encoding tasks, it has been suggested that visual saliency can be overridden in a search task, with saccades being planned on the basis of target features, rather than being captured by saliency. Here, we find that target regions of a scene that are salient according to this model are found quicker than control regions (Experiment 1). However, this does not seem to be altered by filtering features in the periphery using a gaze-contingent display (Experiment 2), and a deeper analysis of the eye movements made suggests that the saliency effect is instead due to the meaning of the scene regions. Experiment 3 supports this interpretation, showing that scene inversion reduces the saliency effect. These results suggest that saliency effects on search may have nothing to do with bottom-up saccade guidance.

Keywords

Attention Scene perception Saliency map models Eye movements Visual search 

Notes

Acknowledgments

TF is supported by a Commonwealth Fellowship from the Government of Canada. GJU was supported by project grant EP/E006329/1 from the EPSRC (UK). We are also grateful to Laurent Itti et al. for making the saliency map model available, and the comments of an anonymous reviewer.

References

  1. 1.
    Buswell GT. How people look at pictures: a study of the psychology of perception in art. Chicago: University of Chicago Press; 1935.Google Scholar
  2. 2.
    Yarbus AL. Eye movements and vision. New York: Plenum; 1967.Google Scholar
  3. 3.
    Itti L, Koch C. A saliency-based search mechanism for overt and covert shifts of visual attention. Vis Res. 2000;40(10–12):1489–506.PubMedCrossRefGoogle Scholar
  4. 4.
    Treisman A, Gelade G. A feature-integration theory of attention. Cogn Psychol. 1980;12:97–136.PubMedCrossRefGoogle Scholar
  5. 5.
    Koch C, Ullman S. Shifts in selective visual attention: towards the underlying neural circuitry. Human Neurobiol. 1985;4:219–27.Google Scholar
  6. 6.
    Mannan S, Ruddock K, Wooding D. The relationship between the locations of spatial features and those of fixations made during visual examination of briefly presented images. Spat Vis. 1996;10(10):65–188.Google Scholar
  7. 7.
    Reinagel P, Zador AM. Natural scene statistics at the centre of gaze. Network-Computation In Neural Systems. 1999;10(4):341–50.CrossRefGoogle Scholar
  8. 8.
    Tatler BW, Baddeley RJ, Gilchrist ID. Visual correlates of fixation selection: effects of scale and time. Vis Res. 2005;45(5):643–59.PubMedCrossRefGoogle Scholar
  9. 9.
    Itti L, Koch C. Computational modelling of visual attention. Nat Rev Neurosci. 2001;2(3):194–203.PubMedCrossRefGoogle Scholar
  10. 10.
    Parkhurst D, Law K, Niebur E. Modeling the role of salience in the allocation of overt visual attention. Vis Res. 2002;42(1):107–23.PubMedCrossRefGoogle Scholar
  11. 11.
    Peters RJ, et al. Components of bottom-up gaze allocation in natural images. Vis Res. 2005;45(18):2397–416.PubMedCrossRefGoogle Scholar
  12. 12.
    Henderson JM, et al. Visual saliency does not account for eye movements during visual search in real-world scenes. In: van Gompel R, et al., editors. Eye movements: a window on mind and brain. Amsterdam: Elsevier; 2007. p. 537–62.Google Scholar
  13. 13.
    Foulsham T, Underwood G. What can saliency models predict about eye movements?. Spatial and sequential aspects of fixations during encoding and recognition. J Vis. 2008;8(6):1–17.PubMedCrossRefGoogle Scholar
  14. 14.
    Harding G, Bloj M. Real and predicted influence of image manipulations on eye movements during scene recognition. J Vis. 2010;10(2):8.1–17.Google Scholar
  15. 15.
    Underwood G, Foulsham T, Humphrey K. Saliency and scan patterns in the inspection of real-world scenes: eye movements during encoding and recognition. Vis Cogn. 2009;17(6–7):812–34.CrossRefGoogle Scholar
  16. 16.
    Itti L. Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Vis Cogn. 2005;12(6):1093–123.CrossRefGoogle Scholar
  17. 17.
    Dorr M, Gegenfurtner KR, Barth E. The contribution of low-level features at the centre of gaze to saccade target selection. Vis Res. 2009;49(24):2918–26.PubMedCrossRefGoogle Scholar
  18. 18.
    Underwood G. Cognitive processes in eye guidance: algorithms for attention in image processing. Cogn Comput. 2009;1:64–76.CrossRefGoogle Scholar
  19. 19.
    Underwood G, et al. Eye movements during scene inspection: a test of the saliency map hypothesis. Eur J Cogn Psychol. 2006;18(3):321–42.CrossRefGoogle Scholar
  20. 20.
    Foulsham T, Underwood G. How does the purpose of inspection influence the potency of visual saliency in scene perception? Perception. 2007;36:1123–38.PubMedCrossRefGoogle Scholar
  21. 21.
    Henderson JM, Malcolm GL, Schandl C. Searching in the dark: cognitive relevance drives attention in real-world scenes. Psychon Bull Rev. 2009;16(5):850–6.PubMedCrossRefGoogle Scholar
  22. 22.
    Chen X, Zelinsky GJ. Real-world visual search is dominated by top-down guidance. Vis Res. 2006;46(24):4118–33.PubMedCrossRefGoogle Scholar
  23. 23.
    Foulsham T, Underwood G. Does conspicuity enhance distraction? Saliency and eye landing position when searching for objects. Quart J Exp Psychol. 2009;62(6):1088–98.CrossRefGoogle Scholar
  24. 24.
    Navalpakkam V, Itti L. Modeling the influence of task on attention. Vis Res. 2005;45(2):205–31.PubMedCrossRefGoogle Scholar
  25. 25.
    Torralba A, et al. Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. Psychol Rev. 2006;113(4):766–86.PubMedCrossRefGoogle Scholar
  26. 26.
    Cutsuridis V. A cognitive model of saliency, attention, and picture scanning. Cogn Comput. 2009;1:292–9.CrossRefGoogle Scholar
  27. 27.
    McConkie GW, Rayner K. Span of effective stimulus during a fixation in reading. Percept Psychophys. 1975;17(6):578–86.CrossRefGoogle Scholar
  28. 28.
    Rayner K. Eye movements in reading and information processing: 20 years of research. Psychol Bull. 1998;124(3):372–422.PubMedCrossRefGoogle Scholar
  29. 29.
    Geisler WS, Perry JS, Najemnik J. Visual search: the role of peripheral information measured using gaze-contingent displays. J Vis. 2006;6(9):858–73.PubMedCrossRefGoogle Scholar
  30. 30.
    Castelhano MS, Henderson JM. Initial scene representations facilitate eye movement guidance in visual search. J Exp Psychol Hum Percept Perform. 2007;33(4):753–63.PubMedCrossRefGoogle Scholar
  31. 31.
    Loschky LC, McConkie GW. Investigating spatial vision and dynamic attentional selection using a gaze-contingent multiresolutional display. J Exp Psychol Appl. 2002;8(2):99–117.PubMedCrossRefGoogle Scholar
  32. 32.
    Foulsham T, Teszka R, Kingstone A. Saccade control in natural images is shaped by the information visible at fixation: evidence from asymmetric gaze-contingent windows. Atten Percept Psychophys. (in press).Google Scholar
  33. 33.
    Valentine T. Upside-down faces—a review of the effect of inversion upon face recognition. Br J Psychol. 1988;79:471–91.PubMedGoogle Scholar
  34. 34.
    Gauthier I, Tarr MJ. Becoming a ‘‘greeble’’ expert: exploring mechanisms for face recognition. Vis Res. 1997;37(12):1673–82.PubMedCrossRefGoogle Scholar
  35. 35.
    Husk JS, Bennett PJ, Sekuler AB. Inverting houses and textures: Investigating the characteristics of learned inversion effects. Vis Res. 2007;47(9):3350–9.PubMedCrossRefGoogle Scholar
  36. 36.
    Kelley TA, Chun MM, Chua KP. Effects of scene inversion on change detection of targets matched for visual salience. J Vis. 2003;3(1):1–5.PubMedCrossRefGoogle Scholar
  37. 37.
    Loftus GR, Mackworth NH. Cognitive determinants of fixation location during picture viewing. J Exp Psychol Hum Percept Perform. 1978;4(4):565–72.PubMedCrossRefGoogle Scholar
  38. 38.
    Henderson JM, Weeks PA, Hollingworth A. The effects of semantic consistency on eye movements during complex scene viewing. J Exp Psychol Hum Percept Perform. 1999;25(1):210–28.CrossRefGoogle Scholar
  39. 39.
    Underwood G, Foulsham T. Visual saliency and semantic incongruency influence eye movements when inspecting pictures. Quart J Exp Psychol. 2006;59(11):1931–49.CrossRefGoogle Scholar
  40. 40.
    van Diepen PMJ, d’Ydewalle G. Early peripheral and foveal processing in fixations during scene perception. Visual Cognition. 2003;10(1):79–100.CrossRefGoogle Scholar
  41. 41.
    Henderson JM. Eye-movement control during visual object processing—effects of initial fixation position and semantic constraint. Can J Exp Psychol. 1993;47(1):79–98.PubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of British ColumbiaVancouverCanada
  2. 2.School of PsychologyUniversity of NottinghamNottinghamUK

Personalised recommendations