An Experimental Analysis of Saliency Detection with Respect to Three Saliency Levels

  • Antonino Furnari
  • Giovanni Maria Farinella
  • Sebastiano Battiato
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8927)

Abstract

Saliency detection is a useful tool for video-based, real-time Computer Vision applications. It allows to select which locations of the scene are the most relevant and has been used in a number of related assistive technologies such as life-logging, memory augmentation and object detection for the visually impaired, as well as to study autism and the Parkinson’s disease. Many works focusing on different aspects of saliency have been proposed in the literature, defining saliency in different ways depending on the task. In this paper we perform an experimental analysis focusing on three levels where saliency is defined in different ways, namely visual attention modelling, salient object detection and salient object segmentation. We review the main evaluation datasets specifying the level of saliency which they best describe. Through the experiments we show that the performances of the saliency algorithms depend on the level with respect to which they are evaluated and on the nature of the stimuli used for the benchmark. Moreover, we show that the eye fixation maps can be effectively used to perform salient object detection and segmentation, which suggests that pre-attentive bottom-up information can be still exploited to improve high level tasks such as salient object detection. Finally, we show that benchmarking a saliency detection algorithm with respect to a single dataset/saliency level, can lead to erroneous results and conclude that many datasets/saliency levels should be considered in the evaluations.

Keywords

Saliency detection Visual attention modelling Salient object detection Salient object segmention Saliency levels Datasets for saliency evaluation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ghosh, J., Grauman, K.: Discovering important people and objects for egocentric video summarization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1346–1353 (2012)Google Scholar
  2. 2.
    Hodges, S., Berry, E., Wood, K.: SenseCam: a wearable camera that stimulates and rehabilitates autobiographical memory. Memory 19(7), 685–96 (2011)CrossRefGoogle Scholar
  3. 3.
    Lu, Z., Grauman, K.: Story-driven summarization for egocentric video. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2714–2721 (2013)Google Scholar
  4. 4.
    Thakoor, K.A., Marat, S., Nasiatka, P.J., McIntosh, B.P., Sahin, F.E., Tanguay, A.R., Weiland, J.D., Itti, L.: Attention biased speeded up robust features (AB-SURF): A neurally-inspired object recognition algorithm for a wearable aid for the visually-impaired. In: IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 1–6 (2013)Google Scholar
  5. 5.
    Freeth, M., Foulsham, T., Chapman, P.: The influence of visual saliency on fixation patterns in individuals with autism spectrum disorders. Neuropsychologia 49(1), 156–160 (2011)CrossRefGoogle Scholar
  6. 6.
    Amso, D., Haas, S., Tenenbaum, E., Markant, J., Sheinkopf, S.J.: Bottom-up attention orienting in young children with autism. Journal of Autism and Developmental Disorders 44(3), 664–673 (2014)CrossRefGoogle Scholar
  7. 7.
    Mannan, S.K., Hodgson, T.L., Husain, M., Kennard, C.: Eye movements in visual search indicate impaired saliency processing in parkinson’s disease. Progress in Brain Research 171, 559–562 (2008)CrossRefGoogle Scholar
  8. 8.
    Kastner, S., Ungerleider, L.G.: Mechanisms of visual attention in the human cortex. Annual Review of Neuroscience 23(1), 315–341 (2000)CrossRefGoogle Scholar
  9. 9.
    Itti, L., Koch, C.: Computational modelling of visual attention. Nature Reviews. Neuroscience 2(3), 194–203 (2001)CrossRefGoogle Scholar
  10. 10.
    Rolls, E.T., Deco, G.: Attention in natural scenes: neurophysiological and computational bases. Neural Networks 19(9), 1383–1394 (2006)CrossRefMATHGoogle Scholar
  11. 11.
    Koch, K., McLean, J., Segev, R., Freed, M.A., Berry II, M.J., Balasubramanian, V., Sterling, P.: How much the eye tells the brain. Current Biology 16(14), 1428–1434 (2006)CrossRefGoogle Scholar
  12. 12.
    Treisman, A., Gelade, G.: A feature-integration theory of attention. Cognitive Psychology 136(1), 97–136 (1980)CrossRefGoogle Scholar
  13. 13.
    Borji, A., Itti, L.: State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 35(1), 185–207 (2013)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Duncan, K., Sarkar, S.: Saliency in images and video: a brief survey. IET Computer Vision 6(6), 514–523 (2012)CrossRefGoogle Scholar
  15. 15.
    Koch, C., Ullman, S.: Shifts in selective visual attention: towards the underlying neural circuitry. Human Neurobiology 4(4), 219–227 (1985)Google Scholar
  16. 16.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  17. 17.
    Le Meur, O., Le Callet, P., Barba, D., Thoreau, D.: A coherent computational approach to model bottom-up visual attention. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 28(5), 802–817 (2006)CrossRefGoogle Scholar
  18. 18.
    Kootstra, G., Nederveen, A., De Boer, B.: Paying attention to symmetry. In: Proceedings of the British Machine Vision Conference (BMVC), pp. 1115–1125 (2008)Google Scholar
  19. 19.
    Hou, X., Zhang, L.: Saliency detection: A spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. (800), pp. 1–8 (2007)Google Scholar
  20. 20.
    Hou, X., Harel, J., Koch, C.: Image Signature: Highlighting Sparse Salient Regions. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 34(1), 194–201 (2011)Google Scholar
  21. 21.
    Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1597–1604 (2009)Google Scholar
  22. 22.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. Advances in Neural Information Processing Systems 19, 545 (2007)Google Scholar
  23. 23.
    Garcia-Diaz, A., Fdez-Vidal, X.R., Pardo, X.M., Dosil, R.: Saliency from hierarchical adaptation through decorrelation and variance normalization. Image and Vision Computing 30(1), 51–64 (2012)CrossRefGoogle Scholar
  24. 24.
    Yarbus, A.L., Haigh, B., Rigss, L.A.: Eye movements and vision (1967)Google Scholar
  25. 25.
    Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 34(10), 1915–1926 (2012)CrossRefGoogle Scholar
  26. 26.
    Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 33(2), 353–367 (2011)CrossRefGoogle Scholar
  27. 27.
    Jiang, H., Wang, J., Yuan, Z., Liu, T., Zheng, N., Li, S.: Automatic salient object segmentation based on context and shape prior. In: British Machine Vision Conference (BMVC), vol. 3, p. 7 (2011)Google Scholar
  28. 28.
    Borji, A., Sihite, D.N., Itti, L.: Salient object detection: a benchmark. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part II. LNCS, vol. 7573, pp. 414–429. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  29. 29.
    Borji, A., Sihite, D.N., Itti, L.: Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study. IEEE Transactions on Image Processing (TIP) 22(1), 55–69 (2013)CrossRefMathSciNetGoogle Scholar
  30. 30.
    Judd, T., Ehinger, K., Durand, F., Torralba, A.: Learning to predict where humans look. In: IEEE International Conference on Computer Vision (ICCV) (2009)Google Scholar
  31. 31.
    Cheng, M.M., Mitra, N.J., Huang, X., Torr, P.H.S., Hu, S.M.: Salient object detection and segmentation. Tsinghua University, Technical report(2011)Google Scholar
  32. 32.
    Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.H.: Saliency detection via graph-based manifold ranking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3166–3173 (2013)Google Scholar
  33. 33.
    Nguyen, T.V., Xu, M., Gao, G., Kankanhalli, M., Tian, Q., Yan, S.: Static saliency vs. dynamic saliency: a comparative study. In: ACM International Conference on Multimedia, pp. 987–996 (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Antonino Furnari
    • 1
  • Giovanni Maria Farinella
    • 1
  • Sebastiano Battiato
    • 1
  1. 1.Department of Mathematics and Computer ScienceUniversity of CataniaCataniaItaly

Personalised recommendations