Attention, Perception, & Psychophysics

, Volume 79, Issue 6, pp 1593–1614 | Cite as

Measuring and modeling salience with the theory of visual attention

  • Alexander Krüger
  • Jan Tünnermann
  • Ingrid Scharlau
Article

Abstract

For almost three decades, the theory of visual attention (TVA) has been successful in mathematically describing and explaining a wide variety of phenomena in visual selection and recognition with high quantitative precision. Interestingly, the influence of feature contrast on attention has been included in TVA only recently, although it has been extensively studied outside the TVA framework. The present approach further develops this extension of TVA’s scope by measuring and modeling salience. An empirical measure of salience is achieved by linking different (orientation and luminance) contrasts to a TVA parameter. In the modeling part, the function relating feature contrasts to salience is described mathematically and tested against alternatives by Bayesian model comparison. This model comparison reveals that the power function is an appropriate model of salience growth in the dimensions of orientation and luminance contrast. Furthermore, if contrasts from the two dimensions are combined, salience adds up additively.

Keywords

Salience Visual attention Bayesian inference Theory of visual attention Computational modeling 

References

  1. Avraham, T, Yeshurun, Y, & Lindenbaum, M (2008). Predicting visual search performance by quantifying stimuli similarities. Journal of Vision, 8, 9–9. doi:10.1167/8.4.9 CrossRefPubMedGoogle Scholar
  2. Betz, T., Kietzmann, T.C., Wilming, N., & König, P. (2010). Investigating task-dependent top-down effects on overt visual attention. Journal of Vision, 10, 15. doi:10.1167/10.3.15 CrossRefPubMedGoogle Scholar
  3. Bundesen, C. (1990). A theory of visual attention. Psychological Review, 97, 523–547. doi:10.1037/0033-295X.97.4.523 CrossRefPubMedGoogle Scholar
  4. Bundesen, C. (1998). A computational theory of visual attention. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 353, 1271–1281. doi:10.1098/rstb.1998.0282 CrossRefPubMedPubMedCentralGoogle Scholar
  5. Bundesen, C., Habekost, T., & Kyllingsbæk, S. (2005). A neural theory of visual attention: Bridging cognition and neurophysiology. Psychological Review, 112, 291–328.CrossRefPubMedGoogle Scholar
  6. Bundesen, C., Habekost, T., & Kyllingsbæk, S. (2011). A neural theory of visual attention and short-term memory (NTVA). Neuropsychologia, 49, 1446–1457. doi:10.1016/j.neuropsychologia.2010.12.006 CrossRefPubMedGoogle Scholar
  7. Bundesen, C., Vangkilde, S., & Petersen, A. (2015). Recent developments in a computational theory of visual attention (TVA). Vision Research, 116(Part B), 210–218. doi:10.1016/j.visres.2014.11.005 10.1016/j.visres.2014.11.005 CrossRefPubMedGoogle Scholar
  8. Carrasco, M. (2011). Visual attention: The past 25 years. Vision Research, 51, 1484–1525. doi:10.1016/j.visres.2011.04.012 CrossRefPubMedPubMedCentralGoogle Scholar
  9. Chan, L. K. H., & Hayward, W. G. (2009). Feature integration theory revisited: Dissociating feature detection and attentional guidance in visual search. Journal of Experimental Psychology: Human Perception and Performance, 35, 119–132. doi:10.1037/0096-1523.35.1.119 PubMedGoogle Scholar
  10. Chan, L. K. H., & Hayward, W. G. (2014). No attentional capture for simple visual search: Evidence for a dual-route account. Journal of Experimental Psychology. Human Perception and Performance, 40, 2154–2166. doi:10.1037/a0037897 CrossRefPubMedGoogle Scholar
  11. Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18, 193–222. doi:10.1146/annurev.ne.18.030195.001205 CrossRefPubMedGoogle Scholar
  12. Dombrowe, I. C., Olivers, C. N. L., & Donk, M. (2010). The time course of color- and luminance-based salience effects. Frontiers in Psychology, 1.. doi:10.3389/fpsyg.2010.00189
  13. Donk, M., & Soesman, L. (2011). Object salience is transiently represented whereas object presence is not: Evidence from temporal order judgment. Perception, 40, 63–73.CrossRefPubMedGoogle Scholar
  14. Donk, M., & van Zoest, W. (2008). Effects of saliences are short-lived. Psychological Science, 19, 733–739. doi:10.1111/j.1467-9280.2008.02149.x CrossRefPubMedGoogle Scholar
  15. Duncan, J., & Humphreys, G. W. (1989). Visual search and stimulus similarity. Psychological Review, 96, 433–458. doi:10.1037/0033-295X.96.3.433 CrossRefPubMedGoogle Scholar
  16. Einhüser, W., & König, P. (2003). Does luminance-contrast contribute to a saliency map for overt visual attention? European Journal of Neuroscience, 17, 1089–1097. doi:10.1046/j.1460-9568.2003.02508.x CrossRefGoogle Scholar
  17. Ellermeier, W., & Faulhammer, G. (2000). Empirical evaluation of axioms fundamental to Stevens’s ratio-scaling approach: I. Loudness production. Perception and Psychophysics, 62, 1505–1511. doi:10.3758/BF03212151 CrossRefPubMedGoogle Scholar
  18. Finke, K., Bublak, P., Krummenacher, J., Kyllingsbaek, S., Müller, H. J., & Schneider, W. X. (2005). Usability of a theory of visual attention (TVA) for parameter-based measurement of attention I: Evidence from normal subjects. Journal of the International Neuropsychological Society, 11, 832–842.CrossRefPubMedGoogle Scholar
  19. Finke, K., Dodds, C. M., Bublak, P., Regenthal, R., Baumann, F., Manly, T., & Müller, U. (2010). Effects of modafinil and methylphenidate on visual attention capacity: A TVA-based study. Psychopharmacology, 210, 317–329. doi:10.1007/s00213-010-1823-x CrossRefPubMedGoogle Scholar
  20. Frey, H. P., Honey, C., & König, P. (2008). What’s color got to do with it? The influence of color on visual attention in different categories. Journal of Vision, 8, 6–6. doi:10.1167/8.14.6 CrossRefPubMedGoogle Scholar
  21. Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis, 1, 1–19.CrossRefGoogle Scholar
  22. Habekost, T (2015). Clinical TVA-based studies: A general review. Cognition, 290.. doi:10.3389/fpsyg.2015.00290
  23. Huang, L., & Pashler, H. (2005). Quantifying object salience by equating distractor effects. Vision Research, 45, 1909–1920. doi:10.1016/j.visres.2005.01.013 CrossRefPubMedGoogle Scholar
  24. Hung, J., Driver, J., & Walsh, V. (2005). Visual selection and posterior parietal cortex: Effects of repetitive transcranial magnetic stimulation on partial report analyzed by Bundesen’s theory of visual attention. The Journal of Neuroscience, 25, 9602–9612. doi:10.1523/JNEUROSCI.0879-05.2005 CrossRefPubMedGoogle Scholar
  25. Itti, L., & Koch, C. (2001a). Computational modelling of visual attention. Nature Reviews Neuroscience, 2, 194–203. doi:10.1038/35058500 CrossRefPubMedGoogle Scholar
  26. Itti, L., & Koch, C. (2001b). Feature combination strategies for saliency-based visual attention systems. Journal of Electronic Imaging, 10, 161–169. doi:10.1117/1.1333677 CrossRefGoogle Scholar
  27. Kerzel, D., Schönhammer, J., Burra, N., Born, S., & Souto, D. (2011). Saliency changes appearance. PLoS ONE, 6, e28292. doi:10.1371/journal.pone.0028292 CrossRefPubMedPubMedCentralGoogle Scholar
  28. Koch, C., & Ullman, S. (1985). Shifts in selective visual attention: Towards the underlying neural circuitry. Human Neurobiology, 4, 219–227.PubMedGoogle Scholar
  29. Koehler, K., Guo, F., Zhang, S., & Eckstein, M. P. (2014). What do saliency models predict? Journal of Vision, 14, 14. doi:10.1167/14.3.14 CrossRefPubMedPubMedCentralGoogle Scholar
  30. Koene, A. R., & Zhaoping, L. (2007). Feature-specific interactions in salience from combined feature contrasts: Evidence for a bottom-up saliency map in V1. Journal of Vision, 7, 6. doi:10.1167/7.7.6 CrossRefPubMedGoogle Scholar
  31. Kruschke, J. (2014). Doing Bayesian data analysis: A tutorial with R, JAGS and Stan. Boston: Academic Press. Krüger.Google Scholar
  32. Krüger, A., Tünnermann, J., & Scharlau, I. (2016). Fast and conspicuous? Quantifying salience with the theory of visual attention. Advances in Cognitive Psychology, 12(1), 20–38. doi:10.5709/acp-0184-1 CrossRefPubMedPubMedCentralGoogle Scholar
  33. Kuss, M., Jäkel, F., & Wichmann, F. (2005). A Bayesian inference for psychometric functions. Journal of Vision, 5, 8. doi:10.1167/5.5.8 CrossRefGoogle Scholar
  34. Kyllingsbæk, S. (2006). Modeling visual attention. Behavior Research Methods, 38, 123–133. doi:10.3758/BF03192757 CrossRefPubMedGoogle Scholar
  35. Lee, M. D., & Wagenmakers, E.-J. (2014). Bayesian cognitive modeling: A practical course. Cambridge University Press.Google Scholar
  36. Li, Z. (2002). A saliency map in primary visual cortex. Trends in Cognitive Sciences, 6, 9–16. doi:10.1016/S1364-6613(00)01817-9 CrossRefPubMedGoogle Scholar
  37. Little, R. J. (2006). Calibrated Bayes. The American Statistician, 60, 213–223. doi:10.1198/000313006X117837 CrossRefGoogle Scholar
  38. Logan, G. D. (2004). Cumulative progress in formal theories of attention. Annual Review of Psychology, 55, 207–234. doi:10.1146/annurev.psych.55.090902.141415 CrossRefPubMedGoogle Scholar
  39. Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. NY, USA: Henry Holt and Co., Inc.Google Scholar
  40. Mathôt, S., Schreij, D., & Theeuwes, J. (2012). OpenSesame: An open-source, graphical experiment builder for the social sciences. Behavior Research Methods, 977(44), 314–324. doi:10.3758/s13428-011-0168-7 CrossRefGoogle Scholar
  41. Matthias, E., Bublak, P., Müller, H. J., Schneider, W. X., Krummenacher, J., & Finke, K (2010). The influence of alertness on spatial and nonspatial components of visual attention. Journal of Experimental Psychology. Human Perception and Performance, 36, 38–56. doi:10.1037/a0017602 CrossRefPubMedGoogle Scholar
  42. Nordfang, M., Dyrholm, M., & Bundesen, C. (2013). Identifying bottom-up and top-down components of attentional weight by experimental analysis and computational modeling. Journal of Experimental Psychology: General, 142, 510–535. doi:10.1037/a0029631 CrossRefGoogle Scholar
  43. Nothdurft, H.-C. (1993a). The conspicuousness of orientation and motion contrast. Spatial Vision, 7, 341–363. doi:10.1163/156856893X00487 CrossRefPubMedGoogle Scholar
  44. Nothdurft, H. C. (1993b). The role of features in preattentive vision: Comparison of orientation, motion and color cues. Vision Research, 33, 1937–1958.CrossRefPubMedGoogle Scholar
  45. Nothdurft, H.-C. (2000). Salience from feature contrast: Additivity across dimensions. Vision Research, 40, 1183–1201. doi:10.1016/S0042-6989(00)00031-6 CrossRefPubMedGoogle Scholar
  46. Onat, S., Açık, A., Schumann, F., & König, P. (2014). The contributions of image content and behavioral relevancy to overt attention. PLoS ONE, 9, e93254. doi:10.1371/journal.pone.0093254 CrossRefPubMedPubMedCentralGoogle Scholar
  47. Peirce, J. W. (2007). PsychoPy - Psychophysics software in Python. Journal of Neuroscience Methods, 162, 8–13. doi:10.1016/j.jneumeth.2006.11.017 CrossRefPubMedPubMedCentralGoogle Scholar
  48. Petersen, A., Kyllingsbæk, S., & Bundesen, C. (2012). Measuring and modeling attentional dwell time. Psychonomic Bulletin and Review, 19, 1029–1046. doi:10.3758/s13423-012-0286-y CrossRefPubMedGoogle Scholar
  49. Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. In Proceedings of the 3rd international workshop on distributed statistical computing.Google Scholar
  50. Schubert, T., Finke, K., Redel, P., Kluckow, S., Müller, H., & Strobach, T. (2015). Video game experience and its influence on visual attention parameters: An investigation using the framework of the Theory of Visual Attention (TVA). Acta Psychologica, 157, 200–214. doi:10.1016/j.actpsy.2015.03.005 CrossRefPubMedGoogle Scholar
  51. Schütz, A. C., Braun, D. I., & Gegenfurtner, K. R. (2011). Eye movements and perception: A selective review. Journal of Vision, 11, 9. doi:10.1167/11.5.9 CrossRefPubMedGoogle Scholar
  52. Shibuya, H., & Bundesen, C. (1988). Visual selection from multielement displays: Measuring and modeling effects of exposure duration. Journal of Experimental Psychology: Human Perception and Performance, 14, 591–600. doi:10.1037/0096-1523.14.4.591 PubMedGoogle Scholar
  53. Spence, C., & Parise, C. (2010). Prior-entry: A review. Consciousness and Cognition, 19, 364–379. doi:10.1016/j.concog.2009.12.001 CrossRefPubMedGoogle Scholar
  54. Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & Van Der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(4), 583–639. doi:10.1111/1467-9868.00353 CrossRefGoogle Scholar
  55. Stevens, S. S. (1957). On the psychophysical law. Psychological Review, 64, 153–181. doi:10.1037/h0046162 CrossRefPubMedGoogle Scholar
  56. Taagepera, R. (2008). Making Social Sciences More Scientific: The Need for Predictive Models: The Need for Predictive Models. OUP Oxford.Google Scholar
  57. Treue, S. (2003). Visual attention: the where, what, how and why of saliency. Current Opinion in Neurobiology, 13, 428–432. doi:10.1016/S0959-4388(03)00105-3 CrossRefPubMedGoogle Scholar
  58. Tünnermann, J., & Scharlau, I. (2016). Peripheral visual cues: Their fate in processing and effects on attention and temporal-order perception. Frontiers in Psychology, 7. doi:10.3389/fpsyg.2016.01442
  59. Tünnermann, J., Petersen, A., & Scharlau, I. (2015). Does attention speed up processing? Decreases and increases of processing rates in visual prior entry. Journal of Vision, 15, 1–1. doi:10.1167/15.3.1 CrossRefPubMedGoogle Scholar
  60. Tünnermann, J., Krüger, A., & Scharlau, I. (2017). Measuring attention and visual processing speed by model-based analysis of temporal-order judgments. Journal of Visualized Experiments(119), e54856. doi:10.3791/54856
  61. Vangkilde, S., Bundesen, C., & Coull, J. T. (2011). Prompt but inefficient: Nicotine differentially modulates discrete components of attention. Psychopharmacology, 218, 667–680. doi:10.1007/s00213-011-2361-x CrossRefPubMedPubMedCentralGoogle Scholar
  62. Vangkilde, S., Coull, J. T., & Bundesen, C. (2012). Great expectations: Temporal expectation modulates perceptual processing speed. Journal of Experimental Psychology: Human Perception and Performance, 38, 1183–1191. doi:10.1037/a0026343 PubMedGoogle Scholar
  63. Van Zoest, W., Donk, M., & Van der Stigchel, S. (2012). Stimulus-salience and the time-course of saccade trajectory deviations. Journal of Vision, 12, 16. doi:10.1167/12.8.16 CrossRefPubMedGoogle Scholar
  64. Wichmann, F. A., & Hill, N. J. (2001). The psychometric function: I. Fitting, sampling, and goodness of fit. Perception and Psychophysics, 63, 1293–1313. doi:10.3758/BF03194544 CrossRefPubMedGoogle Scholar
  65. Wolfe, J. M. (2007). Guided search 4.0: Current progress with a model of visual search. In Gray, W. (Ed.), Integrated Models of Cognitive Systems (pp. 99–119). New York : Oxford.CrossRefGoogle Scholar
  66. Wolfe, J. M., & Horowitz, T. S. (2004). What attributes guide the deployment of visual attention and how do they do it? Nature Reviews Neuroscience, 5, 495–501. doi:10.1038/nrn1411 CrossRefPubMedGoogle Scholar
  67. Wolfe, J. M., Cave, K. R., & Franzel, S. L. (1989). Guided search: An alternative to the feature integration model for visual search. Journal of Experimental Psychology. Human Perception and Performance, 15, 419–433.CrossRefPubMedGoogle Scholar
  68. Zhao, Q., & Koch, C. (2011). Learning a saliency map using fixated locations in natural scenes. Journal of Vision, 11, 9–9. doi:10.1167/11.3.9 CrossRefPubMedGoogle Scholar

Copyright information

© The Psychonomic Society, Inc. 2017

Authors and Affiliations

  • Alexander Krüger
    • 1
  • Jan Tünnermann
    • 1
  • Ingrid Scharlau
    • 1
  1. 1.Faculty of Arts and HumanitiesPaderborn UniversityPaderbornGermany

Personalised recommendations