Abstract
Saliency cuts aims to segment salient objects from a given saliency map. The existing saliency cuts methods are fixed to the input cues. It limits their performance when the input cues are changed. In this paper, we propose a novel saliency cuts method named adaptive saliency cuts, which takes advantage of all the input cues in a unified framework and adjusts its components adaptively. Given a saliency map, we first generate segmentation seeds with adaptive triple thresholding. Next, we extend GrabCut by combining different input cues, and use it to generate a rough-labeled map of salient objects. Finally, we refine the boundaries of the salient objects with adaptive initialized segmentation, and produce an accurate binary mask. To the best of our knowledge, this method is the first adaptive saliency cuts method for different input cues. We validated the proposed method on MSRA10K and NJU2000. The experimental results demonstrate that our method outperforms the state-of-the-art methods.
Keywords
Saliency cuts Segmentation seeds generation Rough-labeled map generation Object boundary refinement Adaptive GrabCutNotes
Acknowledgements
This work is supported by National Science Foundation of China (61321491, 61202320), and Collaborative Innovation Center of Novel Software Technology and Industrialization.
References
- 1.Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: International Conference on Pattern Recognition, pp 1597–1604Google Scholar
- 2.Banica D, Agape A, Ion A, Sminchisescu C (2013) Video object segmentation by salient segment chain composition. In: International Conference on Computer Vision Workshop, pp 283–290Google Scholar
- 3.Bao BK, Liu G, Xu C, Yan S (2012) Inductive robust principal component analysis. IEEE Trans Image Process 21(8):3794–3800MathSciNetCrossRefzbMATHGoogle Scholar
- 4.Bao BK, Zhu G, Shen J, Yan S (2013) Robust image analysis with sparse representation on quantized visual features. IEEE Trans Image Process 22(3):860–871MathSciNetCrossRefzbMATHGoogle Scholar
- 5.Blake A, Rother C, Brown M, Perez P, Torr P (2004) Interactive image segmentation using an adaptive gmmrf model. In: European Conference on Computer Vision, pp 428–441Google Scholar
- 6.Borji A, Cheng M, Jiang H, Li J (2015) Salient object detection: A benchmark. IEEE Trans Image Process 24(12):5706–5722MathSciNetCrossRefGoogle Scholar
- 7.Carreira J, Sminchisescu C (2012) Cpmc: Automatic object segmentation using constrained parametric min-cuts. IEEE Trans Pattern Anal Mach Intell 34(7):1312–1328CrossRefGoogle Scholar
- 8.Cheng MM, Zhang GX, Mitra NJ, Huang X, Hu SM (2011) Global contrast based salient region detection. In: International Conference on Pattern Recognition, pp 409–416Google Scholar
- 9.Fu Y, Cheng J, Li Z, Lu H (2008) Saliency cuts: An automatic approach to object segmentation. In: International Conference on Pattern Recognition, pp 1–4Google Scholar
- 10.Gao Z, Zhang H, Xu G, Xue Y, Hauptmann A (2014) Multi-view discriminative and structured dictionary learning with group sparsity for human action recognition. Signal Process 112(C):83–97Google Scholar
- 11.Gao Z, Zhang L, Chen MY, Hauptmann A, Zhang H, Cai AN (2014) Enhanced and hierarchical structure algorithm for data imbalance problem in semantic extraction under massive video dataset. Multimed Tools Appl 68(3):641–657CrossRefGoogle Scholar
- 12.Gao Z, Li S, Zhu YJ, Wang C, Zhang H (2017) Collaborative sparse representation leaning model for rgbd action recognition. J Vis Commun Image Represent 48(C):442–452CrossRefGoogle Scholar
- 13.Ge L, Ju R, Ren T, Wu G (2015) Interactive rgb-d image segmentation using hierarchical graph cut and geodesic distance. In: Pacific-Rim Conference on MultimediaGoogle Scholar
- 14.Giroinieto X, Martos M, Mohedano E, Ponttuset J (2014) From global image annotation to interactive object segmentation. Multimed Tools Appl 70(1):475–493CrossRefGoogle Scholar
- 15.Guo Y, Gu X, Chen Z, Chen Q, Wang C (2007) Adaptive video presentation for small display while maximize visual information. Lect Notes Comput Sci 4781:322–332CrossRefGoogle Scholar
- 16.Guo Y, Gu X, Chen Z, Chen Q, Wang C (2007) Denoising saliency map for region of interest extraction. In: International Conference on Advances in Visual Information Systems, pp 205–215Google Scholar
- 17.Guo J, Ren T, Huang L, Bei J (2017) Saliency detection on sampled images for tag ranking. Multimedia Systems:1–13. https://doi.org/10.1007/s00530-017-0546-9
- 18.Hou X, Zhang L (2007) Saliency detection: A spectral residual approach. In: International Conference on Pattern Recognition, pp 1–8Google Scholar
- 19.Huang Y, Wang S (2008) Multilevel thresholding methods for image segmentation with otsu based on qpso. In: International Conference on Image and Signal Processing, pp 701–705Google Scholar
- 20.Ju R, Liu Y, Ren T, Ge L, Wu G (2015) Depth-aware salient object detection using anisotropic center-surround difference. Signal Process Image Commun 38(C):115–126CrossRefGoogle Scholar
- 21.Ju R, Ren T, Wu G (2015) Stereosnakes: contour based consistent object extraction for stereo images. In: IEEE International Conference on Computer Vision, pp 1724–1732Google Scholar
- 22.Kass M, Witkin AP, Terzopoulos D (1988) Snakes: Active contour models. Int J Comput Vis 1(4):321–331CrossRefzbMATHGoogle Scholar
- 23.Leibe B, Leonardis A, Schiele B (2004) Combined object categorization and segmentation with an implicit shape model. European Conference on Computer Vision Workshop on Statistical Learning in Computer Vision:17–32Google Scholar
- 24.Li S, Ju R, Ren T, Wu G (2015) Saliency cuts based on adaptive triple thresholding. In: IEEE International Conference on Image Processing, pp 4609–4613Google Scholar
- 25.Liu J, Ren T, Wang Y, Zhong SH, Bei J, Chen S (2017) Object proposal on rgb-d images via elastic edge boxes. Neurocomputing 236:134–146CrossRefGoogle Scholar
- 26.Otsu N (2007) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66CrossRefGoogle Scholar
- 27.Peng H, Li B, Xiong W, Hu W, Ji R (2014) Rgbd salient object detection: A benchmark and algorithms. In: European Conference on Computer Vision, pp 92–109Google Scholar
- 28.Rother C, Kolmogorov V, Blake A (2004) “grabcut”: Interactive foreground extraction using iterated graph cuts. ACM Trans Graph 23(3):309–314CrossRefGoogle Scholar
- 29.Rother C, Minka T, Blake A, Kolmogorov V (2006) Cosegmentation of image pairs by histogram matching - incorporating a global constraint into mrfs. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 993–1000Google Scholar
- 30.Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905CrossRefGoogle Scholar
- 31.Song H, Liu Z, Du H, Sun G, Le MO, Ren T (2017) Depth-aware salient object detection and segmentation via multiscale discriminative saliency fusion and bootstrap learning. IEEE Trans Image Process PP(99):1–1MathSciNetGoogle Scholar
- 32.Tang J, Shu X, Qi G, Li Z, Wang M, Yan S, Jain R (2017) Tri-clustered tensor completion for social-aware image tag refinement. IEEE Trans Pattern Anal Mach Intell 39(8):1662–1674CrossRefGoogle Scholar
- 33.Wang C, Xue Y, Zhang H, Xu G, Gao Z (2016) Object segmentation of indoor scenes using perceptual organization on rgb-d images. In: International Conference on Wireless Communications and Signal Processing, pp 1–5Google Scholar
- 34.Wang Y, Huang L, Ren T, Zhang Y (2017) Saliency cuts on rgb-d images. In: International Conference on Internet Multimedia Computing and ServiceGoogle Scholar
- 35.Wang Y, Huang L, Ren T, Zhong SH, Liu Y, Wu G (2017) Object proposal via depth connectivity constrained grouping. In: Pacific-Rim Conference on MultimediaGoogle Scholar
- 36.Xu N, Bansal R, Ahuja N (2007) Object segmentation using graph cuts based active contours. In: International Conference on Pattern Recognition, vol 2, pp II–46–53Google Scholar
- 37.Yang Y, Xu D, Nie F, Yan S, Zhuang Y (2010) Image clustering using local discriminant models and global integration. IEEE Trans Image Process 19 (10):2761–2773MathSciNetCrossRefzbMATHGoogle Scholar
- 38.Yang Y, Nie F, Xu D, Luo J, Zhuang Y, Pan Y (2012) A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Trans Pattern Anal Mach Intell 34(4):723CrossRefGoogle Scholar
- 39.Yang C, Zhang L, Lu H, Xiang R, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3166–3173Google Scholar
- 40.Ye L, Liu Z, Li L, Shen L, Bai C, Wang Y (2017) Salient object segmentation via effective integration of saliency and objectness. IEEE Trans Multimed PP(99):1–1Google Scholar
- 41.Zhang Y, Hong C, Wang C (2010) An efficient real time rectangle speed limit sign recognition system, pp 34–38Google Scholar
- 42.Zhang H, Zha ZJ, Yang Y, Yan S, Gao Y, Chua TS (2013) Attribute-augmented semantic hierarchy:towards bridging semantic gap and intention gap in image retrieval. In: Acm International Conference on Multimedia, pp 33–42Google Scholar
- 43.Zhang H, Shang X, Luan H, Wang M, Chua TS (2016) Learning from collective intelligence: Feature learning using social images and tags. Acm Trans Multimed Comput Commun Appl 13(1):1CrossRefGoogle Scholar
- 44.Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2814–2821Google Scholar