Computational Visual Media

, Volume 1, Issue 4, pp 309–320 | Cite as

SaliencyRank: Two-stage manifold ranking for salient object detection

  • Wei Qi
  • Ming-Ming Cheng
  • Ali Borji
  • Huchuan Lu
  • Lian-Fa Bai
Open Access
Research Article

Abstract

Salient object detection remains one of the most important and active research topics in computer vision, with wide-ranging applications to object recognition, scene understanding, image retrieval, context aware image editing, image compression, etc. Most existing methods directly determine salient objects by exploring various salient object features. Here, we propose a novel graph based ranking method to detect and segment the most salient object in a scene according to its relationship to image border (background) regions, i.e., the background feature. Firstly, we use regions/super-pixels as graph nodes, which are fully connected to enable both long range and short range relations to be modeled. The relationship of each region to the image border (background) is evaluated in two stages: (i) ranking with hard background queries, and (ii) ranking with soft foreground queries. We experimentally show how this two-stage ranking based salient object detection method is complementary to traditional methods, and that integrated results outperform both. Our method allows the exploitation of intrinsic image structure to achieve high quality salient object determination using a quadratic optimization framework, with a closed form solution which can be easily computed. Extensive method evaluation and comparison using three challenging saliency datasets demonstrate that our method consistently outperforms 10 state-of-theart models by a big margin.

Keywords

salient object detection manifold ranking visual attention saliency 

References

  1. [1]
    Borji, A.; Itti, L. State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 1, 185–207, 2013.MathSciNetCrossRefGoogle Scholar
  2. [2]
    Toet, A. Computational versus psychophysical bottom–up image saliency: A comparative evaluation study. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 11, 2131–2146, 2011.CrossRefGoogle Scholar
  3. [3]
    Itti, L. Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Transactions on Image Processing Vol. 13, No. 10, 1304–1318, 2004.CrossRefGoogle Scholar
  4. [4]
    Rother, C.; Kolmogorov, V.; Blake, A. “GrabCut”: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics Vol. 23, No.3, 309–314, 2004.CrossRefGoogle Scholar
  5. [5]
    Sharma, G.; Jurie, F.; Schmid, C. Discriminative spatial saliency for image classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 3506–3513, 2012.Google Scholar
  6. [6]
    Avidan, S.; Shamir, A. Seam carving for content-aware image resizing. ACM Transactions on Graphics Vol. 26, No. 3, Article No. 10, 2007.CrossRefGoogle Scholar
  7. [7]
    Zhang, G.-X.; Cheng, M.-M.; Hu, S.-M.; Martin, R. R. A shape-preserving approach to image resizing. Computer Graphics Forum Vol. 28, No. 7, 1897–1906, 2009.Google Scholar
  8. [8]
    Zhang, L.; Huang, H. Hierarchical narrative collage for digital photo album. Computer Graphics Forum Vol. 31, No. 7, 2173–2181, 2012.CrossRefGoogle Scholar
  9. [9]
    Shen, C.; Zhao, Q. Webpage saliency. In: Lecture Notes in Computer Science, Vol. 8695. Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Eds. Springer International Publishing, 33–46, 2014.Google Scholar
  10. [10]
    Mahadevan, V.; Vasconcelos, N. Saliency-based discriminant tracking. In: Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, 1007–1013, 2009.Google Scholar
  11. [11]
    Shen, X.; Wu, Y. A unified approach to salient object detection via low rank matrix recovery. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 853–860, 2012.Google Scholar
  12. [12]
    Yang, C.; Zhang, L.; Lu, H.; Ruan, X.; Yang, M.-H. Saliency detection via graph-based manifold ranking. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 3166–3173, 2013.Google Scholar
  13. [13]
    Li, X.; Lu, H.; Zhang, L.; Ruan, X.; Yang, M.-H. Saliency detection via dense and sparse reconstruction. In: Proceedings of IEEE International Conference on Computer Vision, 2976–2983, 2013.Google Scholar
  14. [14]
    Achanta, R.; Hemami, S.; Estrada, F.; Süsstrunk, S. Frequency-tuned salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1597–1604, 2009.Google Scholar
  15. [15]
    Cheng, M.-M.; Mitra, N. J.; Huang, X.; Torr, P. H. S.; Hu, S.-M. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 37, No. 3, 569–582, 2015.CrossRefGoogle Scholar
  16. [16]
    Liu, T.; Yuan, Z.; Sun, J.; Wang, J.; Zheng, N.; Tang, X.; Shum, H.-Y. Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 2, 353–367, 2011.CrossRefGoogle Scholar
  17. [17]
    Yan, Q.; Xu, L.; Shi, J.; Jia, J. Hierarchical saliency detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1155–1162, 2013.Google Scholar
  18. [18]
    Borji, A.; Sihite, D. N.; Itti, L. Salient object detection: A benchmark. In: Proceedings of the 12th European Conference on Computer Vision, Vol. II, 414–429, 2012.Google Scholar
  19. [19]
    Itti, L.; Koch, C.; Niebur, E. A model of saliencybased visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 20, No. 11, 1254–1259, 1998.CrossRefGoogle Scholar
  20. [20]
    Harel, J.; Koch, C.; Perona, P. Graph-based visual saliency. In: Proceedings of Advances in Neural Information Processing Systems 19, 2006. Available at http://papersnipscc/paper/3095-graphbased-visual-saliencypdf.Google Scholar
  21. [21]
    Hou, X.; Zhang, L. Saliency detection: A spectral residual approach. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1–8, 2007.Google Scholar
  22. [22]
    Zhang, L.; Tong, M. H.; Marks, T. K.; Shan, H.; Cottrell, G. W. SUN: A Bayesian framework for saliency using natural statistics. Journal of Vision Vol. 8, No. 7, Article No. 32, 2008.CrossRefGoogle Scholar
  23. [23]
    Rahtu, E.; Kannala, J.; Salo, M.; Heikkilä, J. Segmenting salient objects from images and videos. In: Proceedings of the 11th European Conference on Computer Vision: Part V, 366–379, 2010.Google Scholar
  24. [24]
    Goferman, S.; Zelnik-Manor, L.; Tal, A. Contextaware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. 10, 1915–1926, 2012.CrossRefGoogle Scholar
  25. [25]
    Jiang, B.; Zhang, L.; Lu, H.; Yang, C.; Yang, M.-H. Saliency detection via absorbing Markov chain. In: Proceedings of IEEE International Conference on Computer Vision, 1665–1672, 2013.Google Scholar
  26. [26]
    Jiang, P.; Ling, H.; Yu, J.; Peng, J. Salient region detection by UFO: Uniqueness, focusness and objectness. In: Proceedings of IEEE International Conference on Computer Vision, 1976–1983, 2013.Google Scholar
  27. [27]
    Jiang, H.; Wang, J.; Yuan, Z.; Wu, Y.; Zheng, N.; Li, S. Salient object detection: A discriminative regional feature integration approach. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2083–2090, 2013.Google Scholar
  28. [28]
    Li, X.; Li, Y.; Shen, C.; Dick, A.; Hengel, A. V. D. Contextual hypergraph modeling for salient object detection. In: Proceedings of IEEE International Conference on Computer Vision, 3328–3335, 2013.Google Scholar
  29. [29]
    Zhang, J.; Sclaroff, S. Saliency detection: A Boolean map approach. In: Proceedings of IEEE International Conference on Computer Vision, 153–160, 2013.Google Scholar
  30. [30]
    Kim, J.; Han, D.; Tai, Y.-W.; Kim, J. Salient region detection via high-dimensional color transform. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 883–890, 2014.Google Scholar
  31. [31]
    Jiang, Z.; Davis, L. S. Submodular salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2043–2050, 2013.Google Scholar
  32. [32]
    Margolin, R.; Tal, A.; Zelnik-Manor, L. What makes a patch distinct? In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1139–1146, 2013.Google Scholar
  33. [33]
    Liu, R.; Cao, J.; Lin, Z.; Shan, S. Adaptive partial differential equation learning for visual saliency detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 3866–3873, 2014.Google Scholar
  34. [34]
    Li, N.; Ye, J.; Ling, H.; Yu, J. Saliency detection on light field. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2806–2813, 2014.Google Scholar
  35. [35]
    Jiang, M.; Huang, S.; Duan, J.; Zhao, Q. SALICON: Saliency in context. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1072–1080, 2015.Google Scholar
  36. [36]
    Cheng, M.-M.; Mitra, N. J.; Huang, X.; Hu, S.-M. SalientShape: Group saliency in image collections. The Visual Computer Vol. 30, No. 4, 443–453, 2014.Google Scholar
  37. [37]
    Fu, H.; Cao, X.; Tu, Z. Cluster-based co-saliency detection. IEEE Transactions on Image Processing Vol. 22, No. 10, 3766–3778, 2013.MathSciNetCrossRefGoogle Scholar
  38. [38]
    Wei, Y.; Wen, F.; Zhu, W.; Sun, J. Geodesic saliency using background priors. In: Proceedings of the 12th European Conference on Computer Vision, Vol. III, 29–42, 2012.Google Scholar
  39. [39]
    Zhu, W.; Liang, S.; Wei, Y.; Sun, J. Saliency optimization from robust background detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2814–2821, 2014.Google Scholar
  40. [40]
    Zhou, D.; Weston, J.; Gretton, A.; Bousquet, O.; Schölkopf, B. Ranking on data manifolds. In: Proceedings of Advances in Neural Information Processing Systems 16, 2004. Available at http://papersnipscc/paper/2447-ranking-on-datamanifolds. pdf.Google Scholar
  41. [41]
    Achanta, R.; Shaji, A.; Smith, K.; Lucchi, A.; Fua, P.; Süsstrunk, S. SLIC superpixels compared to stateof-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. 11, 2274–2282, 2012.CrossRefGoogle Scholar
  42. [42]
    Jiang, H.; Wang, J.; Yuan, Z.; Liu, T.; Zheng, N. Automatic salient object segmentation based on context and shape prior. In: Proceedings of the British Machine Vision Conference, 110.1–12, 2011.Google Scholar
  43. [43]
    Canny, J. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 8, No. 6, 679–698, 1986.CrossRefGoogle Scholar
  44. [44]
    Krähenbühl, P.; Koltun, V. Efficient inference in fully connected CRFs with Gaussian edge potentials. In: Proceedings of Advances in Neural Information Processing Systems 24, 2011. Available at http://papersnipscc/paper/4296-efficientinference-in-fully-connected-crfs-with-gaussian-edgepotentials. pdf.Google Scholar
  45. [45]
    Duan, L.; Wu, C.; Miao, J.; Qing, L.; Fu, Y. Visual saliency detection by spatially weighted dissimilarity. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 473–480, 2011.Google Scholar
  46. [46]
    Perazzi, F.; Krahenbuhl, P.; Pritch, Y.; Hornung, A. Saliency filters: Contrast based filtering for salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 733–740, 2012.Google Scholar
  47. [47]
    Cheng, M.-M.; Warrell, J.; Lin, W.-Y.; Zheng, S.; Vineet, V.; Crook, N. Efficient salient region detection with soft image abstraction. In: Proceedings of IEEE International Conference on Computer Vision, 1529–1536, 2013.Google Scholar

Copyright information

© The Author(s) 2015

Authors and Affiliations

  • Wei Qi
    • 1
  • Ming-Ming Cheng
    • 2
  • Ali Borji
    • 3
  • Huchuan Lu
    • 4
  • Lian-Fa Bai
    • 1
  1. 1.Nanjing University of Science and TechnologyNanjingChina
  2. 2.Nankai UniversityTianjinChina
  3. 3.University of WisconsinMilwaukeeUSA
  4. 4.Dalian University of TechnologyDalianChina

Personalised recommendations