Abstract
Saliency detection is a useful tool for video-based, real-time Computer Vision applications. It allows to select which locations of the scene are the most relevant and has been used in a number of related assistive technologies such as life-logging, memory augmentation and object detection for the visually impaired, as well as to study autism and the Parkinson’s disease. Many works focusing on different aspects of saliency have been proposed in the literature, defining saliency in different ways depending on the task. In this paper we perform an experimental analysis focusing on three levels where saliency is defined in different ways, namely visual attention modelling, salient object detection and salient object segmentation. We review the main evaluation datasets specifying the level of saliency which they best describe. Through the experiments we show that the performances of the saliency algorithms depend on the level with respect to which they are evaluated and on the nature of the stimuli used for the benchmark. Moreover, we show that the eye fixation maps can be effectively used to perform salient object detection and segmentation, which suggests that pre-attentive bottom-up information can be still exploited to improve high level tasks such as salient object detection. Finally, we show that benchmarking a saliency detection algorithm with respect to a single dataset/saliency level, can lead to erroneous results and conclude that many datasets/saliency levels should be considered in the evaluations.
Chapter PDF
Similar content being viewed by others
Keywords
References
Ghosh, J., Grauman, K.: Discovering important people and objects for egocentric video summarization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1346–1353 (2012)
Hodges, S., Berry, E., Wood, K.: SenseCam: a wearable camera that stimulates and rehabilitates autobiographical memory. Memory 19(7), 685–96 (2011)
Lu, Z., Grauman, K.: Story-driven summarization for egocentric video. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2714–2721 (2013)
Thakoor, K.A., Marat, S., Nasiatka, P.J., McIntosh, B.P., Sahin, F.E., Tanguay, A.R., Weiland, J.D., Itti, L.: Attention biased speeded up robust features (AB-SURF): A neurally-inspired object recognition algorithm for a wearable aid for the visually-impaired. In: IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 1–6 (2013)
Freeth, M., Foulsham, T., Chapman, P.: The influence of visual saliency on fixation patterns in individuals with autism spectrum disorders. Neuropsychologia 49(1), 156–160 (2011)
Amso, D., Haas, S., Tenenbaum, E., Markant, J., Sheinkopf, S.J.: Bottom-up attention orienting in young children with autism. Journal of Autism and Developmental Disorders 44(3), 664–673 (2014)
Mannan, S.K., Hodgson, T.L., Husain, M., Kennard, C.: Eye movements in visual search indicate impaired saliency processing in parkinson’s disease. Progress in Brain Research 171, 559–562 (2008)
Kastner, S., Ungerleider, L.G.: Mechanisms of visual attention in the human cortex. Annual Review of Neuroscience 23(1), 315–341 (2000)
Itti, L., Koch, C.: Computational modelling of visual attention. Nature Reviews. Neuroscience 2(3), 194–203 (2001)
Rolls, E.T., Deco, G.: Attention in natural scenes: neurophysiological and computational bases. Neural Networks 19(9), 1383–1394 (2006)
Koch, K., McLean, J., Segev, R., Freed, M.A., Berry II, M.J., Balasubramanian, V., Sterling, P.: How much the eye tells the brain. Current Biology 16(14), 1428–1434 (2006)
Treisman, A., Gelade, G.: A feature-integration theory of attention. Cognitive Psychology 136(1), 97–136 (1980)
Borji, A., Itti, L.: State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 35(1), 185–207 (2013)
Duncan, K., Sarkar, S.: Saliency in images and video: a brief survey. IET Computer Vision 6(6), 514–523 (2012)
Koch, C., Ullman, S.: Shifts in selective visual attention: towards the underlying neural circuitry. Human Neurobiology 4(4), 219–227 (1985)
Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 20(11), 1254–1259 (1998)
Le Meur, O., Le Callet, P., Barba, D., Thoreau, D.: A coherent computational approach to model bottom-up visual attention. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 28(5), 802–817 (2006)
Kootstra, G., Nederveen, A., De Boer, B.: Paying attention to symmetry. In: Proceedings of the British Machine Vision Conference (BMVC), pp. 1115–1125 (2008)
Hou, X., Zhang, L.: Saliency detection: A spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. (800), pp. 1–8 (2007)
Hou, X., Harel, J., Koch, C.: Image Signature: Highlighting Sparse Salient Regions. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 34(1), 194–201 (2011)
Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1597–1604 (2009)
Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. Advances in Neural Information Processing Systems 19, 545 (2007)
Garcia-Diaz, A., Fdez-Vidal, X.R., Pardo, X.M., Dosil, R.: Saliency from hierarchical adaptation through decorrelation and variance normalization. Image and Vision Computing 30(1), 51–64 (2012)
Yarbus, A.L., Haigh, B., Rigss, L.A.: Eye movements and vision (1967)
Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 34(10), 1915–1926 (2012)
Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 33(2), 353–367 (2011)
Jiang, H., Wang, J., Yuan, Z., Liu, T., Zheng, N., Li, S.: Automatic salient object segmentation based on context and shape prior. In: British Machine Vision Conference (BMVC), vol. 3, p. 7 (2011)
Borji, A., Sihite, D.N., Itti, L.: Salient object detection: a benchmark. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 414–429. Springer, Heidelberg (2012)
Borji, A., Sihite, D.N., Itti, L.: Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study. IEEE Transactions on Image Processing (TIP) 22(1), 55–69 (2013)
Judd, T., Ehinger, K., Durand, F., Torralba, A.: Learning to predict where humans look. In: IEEE International Conference on Computer Vision (ICCV) (2009)
Cheng, M.M., Mitra, N.J., Huang, X., Torr, P.H.S., Hu, S.M.: Salient object detection and segmentation. Tsinghua University, Technical report(2011)
Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.H.: Saliency detection via graph-based manifold ranking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3166–3173 (2013)
Nguyen, T.V., Xu, M., Gao, G., Kankanhalli, M., Tian, Q., Yan, S.: Static saliency vs. dynamic saliency: a comparative study. In: ACM International Conference on Multimedia, pp. 987–996 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Furnari, A., Farinella, G.M., Battiato, S. (2015). An Experimental Analysis of Saliency Detection with Respect to Three Saliency Levels. In: Agapito, L., Bronstein, M., Rother, C. (eds) Computer Vision - ECCV 2014 Workshops. ECCV 2014. Lecture Notes in Computer Science(), vol 8927. Springer, Cham. https://doi.org/10.1007/978-3-319-16199-0_56
Download citation
DOI: https://doi.org/10.1007/978-3-319-16199-0_56
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16198-3
Online ISBN: 978-3-319-16199-0
eBook Packages: Computer ScienceComputer Science (R0)