Skip to main content

Advertisement

SpringerLink
Log in

RGB-D image saliency detection from 3D perspective

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the advent of stereo camera saliency object detection for RGB-D image is attracting more and more interest. Most existing algorithms treat RGB-D image as one RGB image and one depth map, then measure saliency map independently, and last fuse them. They disregard the fact that human visual system operates in real 3D environments. The paper proposed saliency object detection for RGB-D image from 3D perspective. It regards object as three dimensional structures, and redefines boundary conception in RGB-D image, and regards space boundary including top, down, left, right, front, back plane in real 3D environment as background. It incorporates 3D compactness feature, in which salient objects typically have 3D compact spatial distributions, into color and depth feature to express similarity among supervoxels and applies manifold ranking by six boundary planes to generate six saliency maps, and then integrates them to get the RGB-D saliency map from background view. In the end it refines saliency map by high confident salient seeds from foreground view. Experiment results show that six planes of RGB-D image are superior to four sides of RGB image as background seeds and 3D compactness plays an important role in saliency measurement. Our approach outperforms other state-of-the-art algorithms on NLPR RGBD 1000 benchmark.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Achanta R, Shaji A, Smith K et al (2012) SLIC superpixels compared to state-of-the-art superpixel methods [J]. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  2. Ahmad J, Sajjad M, Mehmood I et al (2015) Saliency-weighted graphs for efficient visual content description and their applications in real-time image retrieval systems[J]. J Real-Time Image Proc 13(3):431–447

    Article  Google Scholar 

  3. Chaib S, Gu Y, Yao H et al (2016) A VHR scene classification method integrating sparse PCA and saliency computing[C]. Geoscience and Remote Sensing Symposium (IGARSS). 2016 IEEE International, pp 2742–2745

  4. Cheng Y, Fu H, Wei X et al (2014) Depth enhanced saliency detection method[C]. Proceedings of international conference on internet multimedia computing and service. ACM, pp 23

  5. Desingh K, Madhava K K, Rajan D et al (2013) Depth really matters: improving visual salient region detection with depth[C]. British machine vision conference (BMVC), pp 98.1–98.11

  6. Feng D, Barnes N, You S et al (2016) Local background enclosure for RGB-D salient object detection[C]. Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on. IEEE, pp 2343–2350

  7. Guo J, Ren T, Bei J et al (2015) Salient object detection in RGB-D image based on saliency fusion and propagation[C]. Proceedings of the 7th international conference on internet multimedia computing and service. ACM, pp 59

  8. Hong S, You T, Kwak S et al (2015) Online tracking by learning discriminative saliency map with convolutional neural network[C]. International Conference on Machine Learning (ICML), pp 597–606

  9. Hussain CA, Rao DV, Masthani SA (2016) Robust pre-processing technique based on saliency detection for content based image retrieval systems[J]. Procedia Computer Science 85:571–580

    Article  Google Scholar 

  10. Hussain CA, Rao DV, Masthani SA (2016) Image retrieval using saliency content[C]. Electrical, electronics, and optimization techniques (ICEEOT), international conference on. IEEE, pp 840–844

  11. Jiang L, Koch A, Zell A (2015) Salient regions detection for indoor robots using RGB-D data[C]. 2015 IEEE International conference on robotics and automation (ICRA). IEEE, pp 1323–1328

  12. Ju R, Ge L, Geng W et al (2014) Depth saliency based on anisotropic center-surround difference[C]. 2014 IEEE international conference on image processing (ICIP). IEEE, pp 1115–1119

  13. Ju R, Liu Y, Ren T et al (2015) Depth-aware salient object detection using anisotropic center-surround difference[J]. Signal Process Image Commun 38:115–126

    Article  Google Scholar 

  14. Khanna MT, Rai K, Chaudhury S et al (2015) Perceptual depth preserving saliency based image compression[C]. Proceedings of the 2nd international conference on perception and machine intelligence. ACM, pp 218–223

  15. Krause EA, Zillich M, Williams TE et al (2014) Learning to recognize novel objects in one shot through human-robot interactions in natural language dialogues[C]. AAAI, pp 2796–2802

  16. Lang C, Nguyen T V, Katti H et al (2012) Depth matters: influence of depth cues on visual saliency[M]. Computer Vision–ECCV 2012. Springer Berlin Heidelberg, pp 101–115

    Chapter  Google Scholar 

  17. Li Z, Liu G, Zhang D et al (2016) Robust single-object image segmentation based on salient transition region[J]. Pattern Recogn 52:317–331

    Article  Google Scholar 

  18. Liu GH, Yang JY, Li ZY (2015) Content-based image retrieval using computational visual attention model[J]. Pattern Recogn 48(8):2554–2566

    Article  Google Scholar 

  19. Margolin R, Zelnik-Manor L, Tal A (2014) How to evaluate foreground maps?[C]. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 248–255

  20. Oh SJ, Benenson R, Khoreva A et al (2017) Exploiting saliency for object segmentation from image level labels[C]. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4410–4419

  21. Otsu N (1979) A threshold selection method from gray-level histograms[J]. IEEE Trans Syst Man Cybern 9(1):62–66

    Article  Google Scholar 

  22. Papushoy A, Bors AG (2015) Image retrieval based on query by saliency content[J]. Digital Signal Processing 36:156–173

    Article  MathSciNet  Google Scholar 

  23. Peng H, Li B, Xiong W et al (2014) Rgbd salient object detection: a benchmark and algorithms[C]. European conference on computer vision. Springer, Cham, pp 92–109

    Google Scholar 

  24. Qi M, Wang Y (2016) DEEP-CSSR: scene classification using category-specific salient region with deep features[C]. 2016 IEEE international conference on image processing (ICIP). IEEE, pp 1047-1051

  25. Quo J, Ren T, Bei J (2016) Salient object detection for RGB-D image via saliency evolution[C]. 2016 IEEE international conference on multimedia and expo (ICME). IEEE, pp 1–6

  26. Ren J, Gong X, Yu L et al (2015) Exploiting global priors for RGB-D saliency detection[C]. 2015 IEEE conference on computer vision and pattern recognition workshops (CVPRW). IEEE, pp 25–32

  27. Song H, Liu Z, Du H et al (2015) Saliency detection for RGBD images[C]. Proceedings of the 7th international conference on internet multimedia computing and service. ACM, pp 72

  28. Srivastava S, Mukherjee P, Lall B (2016) Adaptive image compression using saliency and KAZE features[C]. International conference on signal processing and communications, pp:1–5

  29. Sun J, Wu J, Yu H et al (2016) Boosting image retrieval framework with salient objects[C]. Audio, language and image processing (ICALIP), 2016 international conference on. IEEE, pp 241–245

  30. Tamura Y, Akashi T, Yano S et al (2016) Human visual attention model based on analysis of magic for smooth human–robot interaction[J]. Int J Soc Robot 8(5):685–694

    Article  Google Scholar 

  31. Tang Y, Tong R, Tang M et al (2016) Depth incorporating with color improves salient object detection[J]. Vis Comput 32(1):111–121

    Article  Google Scholar 

  32. Wu P, Duan L, Kong L (2015) RGB-D salient object detection via feature fusion and multi-scale enhancement[C]. CCF Chinese Conference on Computer vision. Springer Berlin Heidelberg, pp 359–368

    Google Scholar 

  33. Xue H, Gu Y, Li Y et al (2015) RGB-D saliency detection via mutual guided manifold ranking[C]. Image Processing (ICIP), 2015 IEEE international conference on. IEEE, pp 666–670

  34. Yang C, Zhang L, Lu H et al (2013) Saliency detection via graph-based manifold ranking[C]. Computer vision and pattern recognition (CVPR), 2013 IEEE Conference on. IEEE, pp 3166–3173

  35. Zhang X, Chen X (2016) Robust sketch-based image retrieval by saliency detection[C]. International conference on multimedia modeling. Springer, Cham, pp 515–526

    Chapter  Google Scholar 

  36. Zhang F, Du B, Zhang L (2015) Saliency-guided unsupervised feature learning for scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing 53(4):2175–2184

    Article  Google Scholar 

  37. Zhang L, Chen J, Qiu B (2016) Region-of-interest coding based on saliency detection and directional wavelet for remote sensing images[J]. IEEE Geosci Remote Sens Lett 14(1):23–27

    Article  Google Scholar 

  38. Zhou D, Weston J, Gretton A et al (2004) Ranking on data manifolds[C]. Adv Neural Inf Proces Syst 16:169–176

    Google Scholar 

  39. Zhou L, Yang Z, Yuan Q et al (2015) Salient region detection via integrating diffusion-based compactness and local contrast[J]. IEEE Trans Image Process 24(11):3308–3320

    Article  MathSciNet  Google Scholar 

  40. Zhu S, Bo Y, He L (2016) Robust multi-feature visual tracking with a saliency-based target descriptor[C]. Control conference (CCC), 2016 35th Chinese. IEEE, pp 5008–5013

  41. Zhu G, Wang J, Wu Y et al (2016) MC-HOG correlation tracking with saliency proposal[C]. Thirtieth AAAI Conference on Artificial Intelligence. AAAI Press, pp 3690–3696

  42. Zhu W, Liang S, Wei Y et al Saliency optimization from robust background detection[C]. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2814–2821

Download references

Acknowledgements

We thank Prof. Jianguo Wu from Anhui University for helping with acquisition of funding. We also thank all anonymous reviewers for their valuable comments. This research is supported by National Key Technology Research and Development Program of the Ministry of Science and Technology of China (2015BAK24B01), Key Program of Natural Science Project of Educational Commission of Anhui Province, China (KJ2015A009), Open issues on Co-Innovation Center for Information Supply & Assurance Technology, Anhui University (ADXXBZ201610).

Author information

Authors and Affiliations

  1. Key Laboratory of Intelligent Computing & Signal Processing, Ministry of Education, Anhui University, Hefei, China

    Zhengyi Liu, Tengfei Song & Feng Xie

  2. Co-Innovation Center for Information Supply & Assurance Technology, Anhui University, Hefei, China

    Zhengyi Liu, Tengfei Song & Feng Xie

Authors
  1. Zhengyi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tengfei Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengyi Liu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Z., Song, T. & Xie, F. RGB-D image saliency detection from 3D perspective. Multimed Tools Appl 78, 6787–6804 (2019). https://doi.org/10.1007/s11042-018-6319-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6319-4

Keywords

Search

Navigation