Skip to main content
SpringerLink
Log in

A multilayer backpropagation saliency detection algorithm and its applications

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Saliency detection is an active topic in the multimedia field. Most previous works on saliency detection focus on 2D images. However, these methods are not robust against complex scenes which contain multiple objects or complex backgrounds. Recently, depth information supplies a powerful cue for saliency detection. In this paper, we propose a multilayer backpropagation saliency detection algorithm based on depth mining by which we exploit depth cue from three different layers of images. The proposed algorithm shows a good performance and maintains the robustness in complex situations. Experiments’ results show that the proposed framework is superior to other existing saliency approaches. Besides, we give two innovative applications by this algorithm, such as scene reconstruction from multiple images and small target object detection in video.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Achanta R, Estrada F, Wils P, Sstrunk S (2008) Salient region detection and segmentation. Comput Vis Syst 5008:66–75

    Article  Google Scholar 

  2. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009, pp 1597–1604

  3. Alexe B, Deselaers T, Ferrari V (2012) Measuring the objectness of image windows. IEEE Trans Pattern Anal Mach Intell 34(11):2189–202

    Article  Google Scholar 

  4. Chang CH, Liang CK, Chuang YY (2011) Content-aware display adaptation and interactive editing for stereoscopic images. IEEE Trans Multimedia 13(4):589–601

    Article  Google Scholar 

  5. Chang KY, Liu TL, Chen HT, Lai SH (2011) Fusing generic objectness and visual saliency for salient object detection. In: IEEE International Conference on Computer Vision, pp 914–921

  6. Cheng MM, Zhang GX, Mitra NJ, Huang X, Hu SM (2011) Global contrast based salient region detection. In: Computer Vision and Pattern Recognition, pp 409–416

  7. Cheng MM, Mitra NJ, Huang X, Hu SM (2014) Salientshape: group saliency in image collections. Vis Comput 30(4):443–453

    Article  Google Scholar 

  8. Cheng MM, Warrell J, Lin WY, Zheng S, Vineet V, Crook N (2014) Efficient salient region detection with soft image abstraction. In: IEEE International Conference on Computer Vision, pp 1529–1536

  9. Cheng Y, Fu H, Wei X, Xiao J, Cao X (2014) Depth enhanced saliency detection method. In: Proceedings of international conference on internet multimedia computing and service. ISBN:978-1-4503-2810-4. ACM, New York, NY, USA, pp 23:23–23:27 http://doi.acm.org/10.1145/2632856.2632866

  10. Chunbiao Z, Ge L (2017) A three-pathway psychobiological framework of salient object detection using stereoscopic technology. ICCVW

  11. Chunbiao Z, Ge L et al (2017) An innovative salient object detection using center-dark channel prior. ICCVW

  12. Chunbiao Z, Ge L et al (2017) A multilayer backpropagation saliency detection algorithm based on depth mining. In: International Conference on Computer Analysis of Images and Patterns, pp 14–23

  13. Criminisi A, Prez P, Toyama K (2003) Object removal by exemplar-based inpainting. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 721–728

  14. Geng Y (2012) Leveraging stereopsis for saliency analysis. In: Computer vision and pattern recognition, pp 454–461

  15. Girshick R, Donahue J, Darrell T, Malik J (2013) Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE conference on computer vision and pattern recognition, pp 580–587

  16. He K, Sun J, Tang X (2009) Single image haze removal using dark channel prior. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009, pp 1956–1963

  17. Imamoglu N, Lin W, Fang Y. (2013) A saliency detection model using low-level features based on wavelet transform. IEEE Trans Multimedia 15:96–105

    Article  Google Scholar 

  18. Itti L (2004) Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Press

  19. Jiang H, Wang J, Yuan Z, Liu T, Zheng N, Li S (2011) Automatic salient object segmentation based on context and shape prior. In: British Machine Vision Conference

  20. Jiang B, Zhang L, Huchuan L, Yang C, Yang MH (2013) Saliency detection via absorbing markov chain. In: IEEE International Conference on Computer Vision, pp 1665–1672

  21. Jiang H, Wang J, Yuan Z, Yang W, Zheng N, Li S (2013) Salient object detection: A discriminative regional feature integration approach. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2083–2090

  22. Krahenbuhl P (2012) Saliency filters: Contrast based filtering for salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 733–740

  23. Li J, Tian Y, Duan L, Huang T (2013) Estimating visual saliency through single image optimization. IEEE Signal Process Lett 20(9):845–848

    Article  Google Scholar 

  24. Li H, Lu H, Lin Z, Shen X (2015) Inner and inter label propagation: Salient object detection in the wild. IEEE Trans Image Process Publ IEEE Signal Process Soc 24(10):3176–3186

    Article  MathSciNet  Google Scholar 

  25. Li X, Huchuan L, Zhang L, Xiang R, Yang MH (2013) Saliency detection via dense and sparse reconstruction. In: IEEE International Conference on Computer Vision, pp 2976–2983

  26. Li X, Li Y, Shen C, Dick A, Van Den Hengel A (2013) Contextual hypergraph modeling for salient object detection

  27. Liu F, Gleicher M (2006) Region enhanced scale-invariant saliency detection. In: IEEE International Conference on Multimedia and Expo, pp 1477–1480

  28. Liu R, Cao J, Lin Z, Shan S (2014) Adaptive partial differential equation learning for visual saliency detection. In: Computer Vision and Pattern Recognition, pp 3866–3873

  29. Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum HY (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353

    Article  Google Scholar 

  30. Lou J, Zhu W, Wang H, Ren M (2016) Small target detection combining regional stability and saliency in a color image. Multimedia Tools Appl 76:1–18

    Google Scholar 

  31. Marchesotti L, Cifarelli C, Csurka G (2009) A framework for visual saliency detection with applications to image thumbnailing. In: IEEE International Conference on Computer Vision, pp 2232–2239

  32. Mehrani P (2010) Saliency segmentation based on learning and graph cut refinement

  33. Murray N, Vanrell M, Otazu X, Parraga CA (2011) Saliency estimation using a non-parametric low-level vision model. Comput Vis Pattern Recognit 42(7):433–440

    Google Scholar 

  34. Peng H., Li B, Xiong W, Hu W, Ji R (2014) RGBD Salient object detection: A benchmark and algorithms. Springer International Publishing, Berlin

  35. Qin Y, Huchuan L, Yiqun X, He W (2015) Saliency detection via cellular automata. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 110–119

  36. Ran M, Tal A, Lihi Z-M (2013) What makes a patch distinct. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1139–1146

  37. Shi K, Wang K, Lu J, Lin PL (2013) Pixelwise image saliency by aggregating complementary appearance contrast measures with spatial priors. In: Computer Vision and Pattern Recognition, pp 2115–2122

  38. Shi J, Yan Q, Li X, Jia J (2016) Hierarchical image saliency detection on extended cssd. IEEE Trans Pattern Anal Mach Intell 38(4):717

    Article  Google Scholar 

  39. Siva P, Russell C, Xiang T, Agapito L (2013) Looking beyond the image: Unsupervised learning for object saliency and detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3238–3245

  40. Sun J, Ling H (2011) Scale and object aware image retargeting for thumbnail browsing. In: International Conference on Computer Vision, pp 1511–1518

  41. Sun X, Huang Z, Yin H et al (2017) An integrated model for effective saliency prediction. AAAI

  42. Valenti R, Sebe N, Gevers T (2009) Image saliency by isocentric curvedness and color. In: IEEE International Conference on Computer Vision, pp 2185–2192

  43. Wei Y, Wen F, Zhu W, Sun (2012) Geodesic saliency using background priors. In: European Conference on Computer Vision, pp 29–42

  44. Xie Y, Huchuan L, Yang MH (2013) Bayesian saliency via low and mid level cues. IEEE Trans Image Process Publ IEEE Signal Process Soc 22(5):1689–1698

    MathSciNet  MATH  Google Scholar 

  45. Yang C, Zhang L, Huchuan L, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: Computer Vision and Pattern Recognition, pp 3166–3173

  46. Yao L, Zhang W, Hong L, Xue X (2011) Salient object detection using concavity context. In: International Conference on Computer Vision, pp 233–240

  47. Ying W, Shen X (2012) A unified approach to salient object detection via low rank matrix recovery. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 853–860

  48. You X, Du L, Cheung Y et al (2010) A blind watermarking scheme using new nontensor product wavelet filter banks. IEEE Trans Image Process 19:3271–3284

    Article  MathSciNet  MATH  Google Scholar 

  49. Yu FM, Zhang HJ (2003) Contrast-based image attention analysis by using fuzzy growing. In: Eleventh ACM International Conference on Multimedia, Berkeley, Ca, Usa, November, pp 374–381

  50. Zhao S, Yao H, Gao Y et al (2017) Continuous probability distribution prediction of image emotions via multitask shared sparse regression. IEEE Trans Multimedia 19:632–645

    Article  Google Scholar 

  51. Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2814–2821

  52. Zhu C, Ge L, Wang W, Wang R (2017) Salient object detection with complex scene based on cognitive. In: IEEE Third International Conference on Multimedia Big Data. IEEE, pp 33–37

Download references

Acknowledgments

The first author (Zhu, Chunbiao) thanks his family for their kindness, understanding, encouragement, and support. This work was supported by the grant of National Natural Science Foundation of China(No.U1611461), the grant of Science and Technology Planning Project of Guangdong Province, China(No.2014B090910001) and the grant of Shenzhen Peacock Plan(No.20130408-183003656).

Author information

Authors and Affiliations

  1. School of Electronic and Computer Engineering, Peking University Shenzhen Graduate School, Shenzhen, 518055, China

    Chunbiao Zhu & Ge Li

Authors
  1. Chunbiao Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ge Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ge Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, C., Li, G. A multilayer backpropagation saliency detection algorithm and its applications. Multimed Tools Appl 77, 25181–25197 (2018). https://doi.org/10.1007/s11042-018-5780-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5780-4

Keywords

Search

Navigation