Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 17, pp 22213–22230 | Cite as

Adaptive saliency cuts

  • Yuantian Wang
  • Tongwei RenEmail author
  • Sheng-Hua Zhong
  • Yan Liu
  • Gangshan Wu
Article
First Online:
  • 61 Downloads

Abstract

Saliency cuts aims to segment salient objects from a given saliency map. The existing saliency cuts methods are fixed to the input cues. It limits their performance when the input cues are changed. In this paper, we propose a novel saliency cuts method named adaptive saliency cuts, which takes advantage of all the input cues in a unified framework and adjusts its components adaptively. Given a saliency map, we first generate segmentation seeds with adaptive triple thresholding. Next, we extend GrabCut by combining different input cues, and use it to generate a rough-labeled map of salient objects. Finally, we refine the boundaries of the salient objects with adaptive initialized segmentation, and produce an accurate binary mask. To the best of our knowledge, this method is the first adaptive saliency cuts method for different input cues. We validated the proposed method on MSRA10K and NJU2000. The experimental results demonstrate that our method outperforms the state-of-the-art methods.

Keywords

Saliency cuts Segmentation seeds generation Rough-labeled map generation Object boundary refinement Adaptive GrabCut 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This work is supported by National Science Foundation of China (61321491, 61202320), and Collaborative Innovation Center of Novel Software Technology and Industrialization.

References

  1. 1.
    Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: International Conference on Pattern Recognition, pp 1597–1604Google Scholar
  2. 2.
    Banica D, Agape A, Ion A, Sminchisescu C (2013) Video object segmentation by salient segment chain composition. In: International Conference on Computer Vision Workshop, pp 283–290Google Scholar
  3. 3.
    Bao BK, Liu G, Xu C, Yan S (2012) Inductive robust principal component analysis. IEEE Trans Image Process 21(8):3794–3800MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bao BK, Zhu G, Shen J, Yan S (2013) Robust image analysis with sparse representation on quantized visual features. IEEE Trans Image Process 22(3):860–871MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Blake A, Rother C, Brown M, Perez P, Torr P (2004) Interactive image segmentation using an adaptive gmmrf model. In: European Conference on Computer Vision, pp 428–441Google Scholar
  6. 6.
    Borji A, Cheng M, Jiang H, Li J (2015) Salient object detection: A benchmark. IEEE Trans Image Process 24(12):5706–5722MathSciNetCrossRefGoogle Scholar
  7. 7.
    Carreira J, Sminchisescu C (2012) Cpmc: Automatic object segmentation using constrained parametric min-cuts. IEEE Trans Pattern Anal Mach Intell 34(7):1312–1328CrossRefGoogle Scholar
  8. 8.
    Cheng MM, Zhang GX, Mitra NJ, Huang X, Hu SM (2011) Global contrast based salient region detection. In: International Conference on Pattern Recognition, pp 409–416Google Scholar
  9. 9.
    Fu Y, Cheng J, Li Z, Lu H (2008) Saliency cuts: An automatic approach to object segmentation. In: International Conference on Pattern Recognition, pp 1–4Google Scholar
  10. 10.
    Gao Z, Zhang H, Xu G, Xue Y, Hauptmann A (2014) Multi-view discriminative and structured dictionary learning with group sparsity for human action recognition. Signal Process 112(C):83–97Google Scholar
  11. 11.
    Gao Z, Zhang L, Chen MY, Hauptmann A, Zhang H, Cai AN (2014) Enhanced and hierarchical structure algorithm for data imbalance problem in semantic extraction under massive video dataset. Multimed Tools Appl 68(3):641–657CrossRefGoogle Scholar
  12. 12.
    Gao Z, Li S, Zhu YJ, Wang C, Zhang H (2017) Collaborative sparse representation leaning model for rgbd action recognition. J Vis Commun Image Represent 48(C):442–452CrossRefGoogle Scholar
  13. 13.
    Ge L, Ju R, Ren T, Wu G (2015) Interactive rgb-d image segmentation using hierarchical graph cut and geodesic distance. In: Pacific-Rim Conference on MultimediaGoogle Scholar
  14. 14.
    Giroinieto X, Martos M, Mohedano E, Ponttuset J (2014) From global image annotation to interactive object segmentation. Multimed Tools Appl 70(1):475–493CrossRefGoogle Scholar
  15. 15.
    Guo Y, Gu X, Chen Z, Chen Q, Wang C (2007) Adaptive video presentation for small display while maximize visual information. Lect Notes Comput Sci 4781:322–332CrossRefGoogle Scholar
  16. 16.
    Guo Y, Gu X, Chen Z, Chen Q, Wang C (2007) Denoising saliency map for region of interest extraction. In: International Conference on Advances in Visual Information Systems, pp 205–215Google Scholar
  17. 17.
    Guo J, Ren T, Huang L, Bei J (2017) Saliency detection on sampled images for tag ranking. Multimedia Systems:1–13.  https://doi.org/10.1007/s00530-017-0546-9
  18. 18.
    Hou X, Zhang L (2007) Saliency detection: A spectral residual approach. In: International Conference on Pattern Recognition, pp 1–8Google Scholar
  19. 19.
    Huang Y, Wang S (2008) Multilevel thresholding methods for image segmentation with otsu based on qpso. In: International Conference on Image and Signal Processing, pp 701–705Google Scholar
  20. 20.
    Ju R, Liu Y, Ren T, Ge L, Wu G (2015) Depth-aware salient object detection using anisotropic center-surround difference. Signal Process Image Commun 38(C):115–126CrossRefGoogle Scholar
  21. 21.
    Ju R, Ren T, Wu G (2015) Stereosnakes: contour based consistent object extraction for stereo images. In: IEEE International Conference on Computer Vision, pp 1724–1732Google Scholar
  22. 22.
    Kass M, Witkin AP, Terzopoulos D (1988) Snakes: Active contour models. Int J Comput Vis 1(4):321–331CrossRefzbMATHGoogle Scholar
  23. 23.
    Leibe B, Leonardis A, Schiele B (2004) Combined object categorization and segmentation with an implicit shape model. European Conference on Computer Vision Workshop on Statistical Learning in Computer Vision:17–32Google Scholar
  24. 24.
    Li S, Ju R, Ren T, Wu G (2015) Saliency cuts based on adaptive triple thresholding. In: IEEE International Conference on Image Processing, pp 4609–4613Google Scholar
  25. 25.
    Liu J, Ren T, Wang Y, Zhong SH, Bei J, Chen S (2017) Object proposal on rgb-d images via elastic edge boxes. Neurocomputing 236:134–146CrossRefGoogle Scholar
  26. 26.
    Otsu N (2007) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66CrossRefGoogle Scholar
  27. 27.
    Peng H, Li B, Xiong W, Hu W, Ji R (2014) Rgbd salient object detection: A benchmark and algorithms. In: European Conference on Computer Vision, pp 92–109Google Scholar
  28. 28.
    Rother C, Kolmogorov V, Blake A (2004) “grabcut”: Interactive foreground extraction using iterated graph cuts. ACM Trans Graph 23(3):309–314CrossRefGoogle Scholar
  29. 29.
    Rother C, Minka T, Blake A, Kolmogorov V (2006) Cosegmentation of image pairs by histogram matching - incorporating a global constraint into mrfs. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 993–1000Google Scholar
  30. 30.
    Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905CrossRefGoogle Scholar
  31. 31.
    Song H, Liu Z, Du H, Sun G, Le MO, Ren T (2017) Depth-aware salient object detection and segmentation via multiscale discriminative saliency fusion and bootstrap learning. IEEE Trans Image Process PP(99):1–1MathSciNetGoogle Scholar
  32. 32.
    Tang J, Shu X, Qi G, Li Z, Wang M, Yan S, Jain R (2017) Tri-clustered tensor completion for social-aware image tag refinement. IEEE Trans Pattern Anal Mach Intell 39(8):1662–1674CrossRefGoogle Scholar
  33. 33.
    Wang C, Xue Y, Zhang H, Xu G, Gao Z (2016) Object segmentation of indoor scenes using perceptual organization on rgb-d images. In: International Conference on Wireless Communications and Signal Processing, pp 1–5Google Scholar
  34. 34.
    Wang Y, Huang L, Ren T, Zhang Y (2017) Saliency cuts on rgb-d images. In: International Conference on Internet Multimedia Computing and ServiceGoogle Scholar
  35. 35.
    Wang Y, Huang L, Ren T, Zhong SH, Liu Y, Wu G (2017) Object proposal via depth connectivity constrained grouping. In: Pacific-Rim Conference on MultimediaGoogle Scholar
  36. 36.
    Xu N, Bansal R, Ahuja N (2007) Object segmentation using graph cuts based active contours. In: International Conference on Pattern Recognition, vol 2, pp II–46–53Google Scholar
  37. 37.
    Yang Y, Xu D, Nie F, Yan S, Zhuang Y (2010) Image clustering using local discriminant models and global integration. IEEE Trans Image Process 19 (10):2761–2773MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Yang Y, Nie F, Xu D, Luo J, Zhuang Y, Pan Y (2012) A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Trans Pattern Anal Mach Intell 34(4):723CrossRefGoogle Scholar
  39. 39.
    Yang C, Zhang L, Lu H, Xiang R, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3166–3173Google Scholar
  40. 40.
    Ye L, Liu Z, Li L, Shen L, Bai C, Wang Y (2017) Salient object segmentation via effective integration of saliency and objectness. IEEE Trans Multimed PP(99):1–1Google Scholar
  41. 41.
    Zhang Y, Hong C, Wang C (2010) An efficient real time rectangle speed limit sign recognition system, pp 34–38Google Scholar
  42. 42.
    Zhang H, Zha ZJ, Yang Y, Yan S, Gao Y, Chua TS (2013) Attribute-augmented semantic hierarchy:towards bridging semantic gap and intention gap in image retrieval. In: Acm International Conference on Multimedia, pp 33–42Google Scholar
  43. 43.
    Zhang H, Shang X, Luan H, Wang M, Chua TS (2016) Learning from collective intelligence: Feature learning using social images and tags. Acm Trans Multimed Comput Commun Appl 13(1):1CrossRefGoogle Scholar
  44. 44.
    Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2814–2821Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.State Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina
  2. 2.College of Computer Science and Software EngineeringShenzhen UniversityShenzhenChina
  3. 3.Department of ComputingThe Hong Kong Polytechnic UniversityHong KongChina

Personalised recommendations

Cite article

Buy options