The Visual Computer

, Volume 32, Issue 5, pp 611–623 | Cite as

A generalized nonlocal mean framework with object-level cues for saliency detection

  • Guangyu Zhong
  • Risheng Liu
  • Junjie CaoEmail author
  • Zhixun Su
Original Article


Nonlocal mean (NM) is an efficient method for many low-level image processing tasks. However, it is challenging to directly utilize NM for saliency detection. This is because that conventional NM method can only extract the structure of the image itself and is based on regular pixel-level graph. However, saliency detection usually requires human perceptions and more complex connectivity of image elements. In this paper, we propose a novel generalized nonlocal mean (GNM) framework with the object-level cue which fuses the low-level and high-level cues to generate saliency maps. For a given image, we first use uniqueness to describe the low-level cue. Second, we adopt the objectness algorithm to find potential object candidates, then we pool the object measures onto patches to generate two high-level cues. Finally, by fusing these three cues as an object-level cue for GNM, we obtain the saliency map of the image. Extensive experiments show that our GNM saliency detector produces more precise and reliable results compared to state-of-the-art algorithms.


Generalized nonlocal mean  Saliency detection Objectness cue 



Risheng Liu is supported by the National Natural Science Foundation of China (No. 61300086), the Fundamental Research Funds for the Central Universities and the Open Project Program of the State Key Laboratory of CAD&CG, Zhejiang University, Zhejiang, China (No. A1404). Junjie Cao is supported by the National Natural Science Foundation of China (No. 61363048 and 61173102). Zhixun Su is supported by National Natural Science Foundation of China (Nos. 61173103 and 91230103) and National Science and Technology Major Project (No. 2013ZX04005021)).


  1. 1.
    Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: CVPR (2011)Google Scholar
  2. 2.
    Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: CVPR (2007)Google Scholar
  3. 3.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. TPAMI 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  4. 4.
    Jiang, Z., Davis, L.S.: Submodular salient region detection. In: CVPR (2013)Google Scholar
  5. 5.
    Pan, J., Su, Z., Bian, M., Liu, R.: Saliency detection based on an edge-preserving filter. In: ICIP (2013)Google Scholar
  6. 6.
    Wei, Y., Wen, F., Zhu, W., Sun, J.: Geodesic saliency using background priors. In: ECCV (2012)Google Scholar
  7. 7.
    Xie, Y., Lu, H., Yang, M.H.: Bayesian saliency via low and mid level cues. TIP 22(5), 1689–1698 (2013)MathSciNetGoogle Scholar
  8. 8.
    Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.H.: Saliency detection via graph-based manifold ranking. In: CVPR (2013)Google Scholar
  9. 9.
    Shi, Y., Yi, Y., Yan, H., Dai, J., Zhang, M., Kong, J.: Region contrast and supervised locality-preserving projection-based saliency detection. TVC, 1–15 (2014)Google Scholar
  10. 10.
    Margolin, R., Zelnik-Manor, L., Tal, A.: Saliency for image manipulation. TVC 29(5), 381–392 (2013)CrossRefGoogle Scholar
  11. 11.
    Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. TPAMI 33(2), 353–367 (2011)CrossRefGoogle Scholar
  12. 12.
    Shen, X., Wu, Y.: A unified approach to salient object detection via low rank matrix recovery. In: CVPR (2012)Google Scholar
  13. 13.
    Yang, J., Yang, M.H.: Top-down visual saliency via joint crf and dictionary learning. In: CVPR (2012)Google Scholar
  14. 14.
    Ma, Y.F., Zhang, H.J.: Contrast-based image attention analysis by using fuzzy growing. In: ACM MM (2003)Google Scholar
  15. 15.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: NIPS (2006)Google Scholar
  16. 16.
    Perazzi, F., Krahenbuhl, P., Pritch, Y., Hornung, A.: Saliency filters: contrast based filtering for salient region detection. In: CVPR (2012)Google Scholar
  17. 17.
    Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. In: CVPR (2013)Google Scholar
  18. 18.
    Judd, T., Ehinger, K., Durand, F., Torralba, A.: Learning to predict where humans look. In: ICCV (2009)Google Scholar
  19. 19.
    Yang, C., Zhang, L., Lu, H.: Graph-regularized saliency detection with convex-hull-based center prior. SPL 20, 637–640 (2013)MathSciNetGoogle Scholar
  20. 20.
    Jia, Y., Han, M.: Category-independent object-level saliency detection. In: ICCV (2013)Google Scholar
  21. 21.
    Jiang, P., Ling, H., Yu, J., Peng, J.: Salient region detection by ufo: uniqueness, focusness and objectness. In: ICCV (2013)Google Scholar
  22. 22.
    Jiang, H., Wang, J., Yuan, Z., Wu, Y., Zheng, N., Li, S.: Salient object detection: a discriminative regional feature integration approach. In: CVPR (2013)Google Scholar
  23. 23.
    Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: CVPR (2005)Google Scholar
  24. 24.
    van Beek, P., Yang, J., Yamamoto, S., Ueda, Y.: Image deblurring and denoising with non-local regularization constraint. In: SPIE (2010)Google Scholar
  25. 25.
    Lindenbaum, M., Fischer, M., Bruckstein, A.: On Gabor’s contribution to image enhancement. Pattern Recogn. 27(1), 1–8 (1994)CrossRefGoogle Scholar
  26. 26.
    Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. TPAMI 12(7), 629–639 (1990)CrossRefGoogle Scholar
  27. 27.
    Cheng, M.M., Zhang, Z., Lin, W.Y., Torr, P.H.S.: BING: binarized normed gradients for objectness estimation at 300 fps. In: CVPR (2014)Google Scholar
  28. 28.
    Yaroslavsky, L.P.: Digital Picture Processing: An Introduction. Springer, New York (1985)CrossRefGoogle Scholar
  29. 29.
    Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: International Conference on Computer Vision (ICCV) (1998)Google Scholar
  30. 30.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. TPAMI 34(11), 2274–2282 (2012)CrossRefGoogle Scholar
  31. 31.
    Gopalakrishnan, V., Hu, Y., Rajan, D.: Random walks on graphs to model saliency in images. In: CVPR (2009)Google Scholar
  32. 32.
    Teuber, H.: Physiological psychology. Annu. Rev. Neurosci. 6(1), 267–296 (1955)Google Scholar
  33. 33.
    Wolfe, J.M., Horowitz, T.S.: What attributes guide the deployment of visual attention and how do they do it? Nat. Rev. Neurosci. 5(6), 495–501 (2004)CrossRefGoogle Scholar
  34. 34.
    Desimone, R., Duncan, J.: Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18(1), 193–222 (1995)CrossRefGoogle Scholar
  35. 35.
    Koch, C., Ullman, S.: Shifts in selective visual attention: towards the underlying neural circuitry. Hum. Neurobiol. 4, 219–227 (1985)Google Scholar
  36. 36.
    Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: CVPR (2009)Google Scholar
  37. 37.
    Li, X., Lu, H., Zhang, L., Ruan, X., Yang, M.H.: Saliency detection via dense and sparse reconstruction. In: ICCV (2013)Google Scholar
  38. 38.
    Jiang, H., Wang, J., Yuan, Z., Liu, T., Zheng, N., Li, S.: Automatic salient object segmentation based on context and shape prior. In: BMVC (2011)Google Scholar
  39. 39.
    Cheng, M.M., Warrell, J., Lin, W.Y., Zheng, S., Vineet, V., Crook, N.: Efficient salient region detection with soft image abstraction. In: ICCV (2013)Google Scholar
  40. 40.
    Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1915–1926 (2012)CrossRefGoogle Scholar
  41. 41.
    Seo, H.J., Milanfar, P.: Static and space-time visual saliency detection by self-resemblance. J Vision 9(12), 1–27 (2009)CrossRefGoogle Scholar
  42. 42.
    Achanta, R., Estrada, F., Wils, P., Süsstrunk, S.: Salient region detection and segmentation. In: ICVS (2008)Google Scholar
  43. 43.
    Bylinskii, Z., Judd, T., Borji, A., Itti, L., Durand, F., Oliva, A., Torralba, A.: MIT saliency benchmark. Accessed 20 March 2015
  44. 44.
    Vikram, T.N., Tscherepanow, M., Wrede, B.: A saliency map based on sampling an image into random rectangular regions of interest. Pattern Recogn. 45(9), 3114–3124 (2012)CrossRefGoogle Scholar
  45. 45.
    Aytekin, C., Kiranyaz, S., Gabbouj, M.: Automatic object segmentation by quantum cuts. In: ICPR (2014)Google Scholar
  46. 46.
    Riche, N., Duvinage, M., Mancas, M., Gosselin, B., Dutoit, T.: Saliency and human fixations: state-of-the-art and study of comparison metrics. In: ICCV (2013)Google Scholar
  47. 47.
    Borji, A., Tavakoli, H.R., Sihite, D.N., Itti, L.: Analysis of scores, datasets, and models in visual saliency prediction. In: ICCV (2013)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Guangyu Zhong
    • 1
  • Risheng Liu
    • 2
  • Junjie Cao
    • 1
    Email author
  • Zhixun Su
    • 1
  1. 1.School of Mathematical SciencesDalian University of TechnologyDalianChina
  2. 2.School of Software TechnologyDalian University of TechnologyDalianChina

Personalised recommendations