Skip to main content
SpringerLink
Log in

Salient object detection via multiple saliency weights

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Salient object detection aims to emulate the extraordinary capability of human visual system, which has the ability to find the most visually attractive objects in a complex visual scene. The human visual attention is often complicated and affected by many factors. In this paper, we present a novel bottom-up approach to automatically detect salient objects of an image via multiple visual cues. The key idea of our approach is to represent a saliency map of an image as an integration of multiple visual cues (saliency weights), which have been proven to be effective and useful. Specifically, we propose four saliency weights, i.e., local contrast weight, superpixel clarity weight, background probability weight, and central bias weight, to effectively represent each visual cue. To obtain our saliency map, the four resulting saliency weights are integrated in a principled way via multiplication and summation based fusion. Furthermore, we propose a new superpixel-level saliency smoothing approach to optimize the integrated results for producing clean and consistent saliency maps. Our experimental results on three standard benchmark datasets demonstrate that the proposed approach outperforms other saliency detection approaches in terms of the subjective observations and objective evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Achanta R, Estrada F, Wils P, Ssstrunk S (2008) Salient region detection and segmentation. In: Computer vision systems, vol 5008, pp 66–75

  2. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: IEEE Conference on computer vision and pattern recognition (CVPR’2009), pp 1597–1604

  3. Alfano PL, Michel GF (1990) Restricting the field of view: perceptual and performance effects. Percept Motor Skills 70(1):35–45

    Article  Google Scholar 

  4. Alpert S, Galun M, Brandt A, Basri R (2012) Image segmentation by probabilistic bottom-up aggregation and cue integration. IEEE Trans Pattern Anal Mach Intell (TPAMI’2012) 34:315–327

    Article  Google Scholar 

  5. Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. ACM Trans Graph 26(3):10

    Article  Google Scholar 

  6. Barghout-Stein L (1999) On differences between peripheral and foveal pattern masking. Ph.D. thesis, University of California Berkeley

  7. Bellman R (1956) On a routing problem. Tech. rep. DTIC Document

  8. Borji A, Itti L (2013) State-of-the-art in visual attention modeling. IEEE Trans Pattern Anal Mach Intell 35(1):185–207

    Article  Google Scholar 

  9. Bruce N, Tsotsos J (2007) An information theoretic model of saliency and visual search. In: Attention in cognitive systems. Theories and systems from an interdisciplinary viewpoint, vol 4840. Springer, pp 171–183

  10. Cheng M, Mitra N, Huang X, Torr P, Hu S (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell (TPAMI’2015) 37:569–582

    Article  Google Scholar 

  11. Cline D, Hofstetter HW, Griffin JR (1997) Dictionary of visual science. Butterworth-Heinemann

  12. Fang Y, Chen Z, Lin W, Lin CW (2012) Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans Image Process (TIP’2012) 21 (9):3888–3901

    Article  MathSciNet  MATH  Google Scholar 

  13. Fang Y, Lin W, Chen Z, Tsai CM, Lin CW (2014) A video saliency detection model in compressed domain. IEEE Trans Circ Syst Video Technol (TCSVT’2014) 24(1):27–38

    Article  Google Scholar 

  14. Fu H, Chi Z, Feng D (2006) Attention-driven image interpretation with application to image retrieval. Pattern Recog 39:1604–1621

    Article  MATH  Google Scholar 

  15. Fu H, Cao X, Tu Z (2013) Cluster-based co-saliency detection. IEEE Trans Image Process (TIP) 22(10):3766–3778

    Article  MathSciNet  MATH  Google Scholar 

  16. Henriques J (2010) http://www.mathworks.com/matlabcentral/fileexchange/coloredges.m

  17. Hou X, Zhang L (2007) Saliency detection: a spectral residual approach. In: IEEE Conference on computer vision and pattern recognition (CVPR’2007), pp 1–8

  18. Itti L (2004) Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Trans Image Process (TIP’2004) 13:1304–1318

    Article  Google Scholar 

  19. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell (TPAMI’1998) 20:1254–1259

    Article  Google Scholar 

  20. Jiang M, Xu J, Zhao Q (2014) Saliency in crowd. In: European conference on computer vision (ECCV’2014), pp 17–32

  21. Koch C, Ullman S (1987) Shifts in selective visual attention: towards the underlying neural circuitry. In: Matters of intelligence. Springer, pp 115–141

  22. Li Z (2014) Understanding vision: theory, models, and data. Oxford University Press

  23. Li Z, Qin S, Itti L (2011) Visual attention guided bit allocation in video compression. Image Vis Comput 29:1–14

    Article  Google Scholar 

  24. Liu M Y, Tuzel O, Ramalingam S, Chellappa R (2011) Entropy rate superpixel segmentation. In: IEEE Conference on computer vision and pattern recognition (CVPR’2011), pp 2097–2104

  25. Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum HY (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell (TPAMI’2010) 33:353–367

    Article  Google Scholar 

  26. Ma K, Gao G, Ding G, Liu CH, Liu E (2016) Crowd saliency prediction with optimal feature combinations. In: Wireless communications & signal processing (WCSP’2016), pp 1–5

  27. Murray N, Vanrell M, Otazu X, Parraga C (2011) Saliency estimation using a non-parametric low-level vision model. In: 2011 IEEE Conference on computer vision and pattern recognition (CVPR), pp 433–440

  28. Panozzo D, Weber O, Sorkine O (2012) Robust image retargeting via axis-aligned deformation. Comput Graph Forum 31(2pt1):229C236

    Article  Google Scholar 

  29. Perazzi F, Krahenbuhl P, Pritch Y, Hornung A (2012) Saliency filters: contrast based filtering for salient region detection. In: IEEE Conference on computer vision and pattern recognition (CVPR’2012), pp 733–740

  30. Rahtu E, Kannala J, Salo M, Heikkilä J. (2010) Segmenting salient objects from images and videos. In: Computer vision-ECCV 2010. Springer, pp 366–379

  31. Rutishauser U, Walther D, Koch C, Perona P (2004) Is bottom-up attention useful for object recognition? In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition (CVPR’2004), vol 2, pp II–37–II–44

  32. Schmidt RF (1981) Fundamentals of sensory physiology. Springer Science & Business Media

  33. Seo HJ, Milanfar P (2009) Static and space-time visual saliency detection by self-resemblance. J Vis 9:15

    Article  Google Scholar 

  34. Strasburger H, Rentschler I, Jüttner M. (2011) Peripheral vision and pattern recognition: a review. J Vis 11(5):13

    Article  Google Scholar 

  35. Toet A (2011) Computational versus psychophysical bottom-up image saliency: a comparative evaluation study. IEEE Trans Pattern Anal Mach Intell (TPAMI) 33 (11):2131–2146

    Article  Google Scholar 

  36. Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cogn Psychol 12(1):97–136

    Article  Google Scholar 

  37. Wang L, Xue J, Zheng N, Hua G (2011) Automatic salient object extraction with contextual cue. In: IEEE International conference on computer vision (ICCV’2011), pp 105–112

  38. Yan B, Li K, Yang X, Hu T (2015) Seam searching-based pixel fusion for image retargeting. IEEE Trans Circ Syst Vid Technol (TCSVT’2015) 25:15–23

    Article  Google Scholar 

  39. Yang Y, Wang X, Guan T, Shen J, Yu L (2014) A multi-dimensional image quality prediction model for user-generated images in social networks. Inf Sci 281:601–610

    Article  Google Scholar 

  40. Zhang L, Tong MH, Marks TK, Shan H, Cottrell GW (2008) Sun: a bayesian framework for saliency using natural statistics. J Vis 8(7):32

    Article  Google Scholar 

  41. Zhaoping L, Zhaoping L (2007) Theoretical understanding of the early visual processes by data compression and data selection. Netw Comput Neural Syst 17 (4):301–34

    Article  Google Scholar 

  42. Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. In: 2014 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2814–2821

Download references

Acknowledgments

This work was supported in part by NSFC (Grant No.: 61370158; 61522202).

Author information

Authors and Affiliations

  1. School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai, 201203, China

    Weimin Tan & Bo Yan

Authors
  1. Weimin Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Yan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tan, W., Yan, B. Salient object detection via multiple saliency weights. Multimed Tools Appl 76, 25091–25107 (2017). https://doi.org/10.1007/s11042-017-4725-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4725-7

Keywords

Search

Navigation