Skip to main content
SpringerLink
Log in

A hybrid approach using color spatial variance and novel object position prior for salient object detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Salient object detection in a real-time environment demands high accuracy with less computation time. It is a changeling task to investigate a saliency model which improves saliency detection accuracy as well as reduces computation time. In this paper, we propose a hybrid model that improves detection accuracy with low computational time. The input image is simplified and clustered at multiple scales using SLIC and K-means clustering algorithms respectively. Gaussian Mixture Model (GMM) is developed on various color components of the digital image at multiple scales. The parameters of GMM are learnt using Expectation Maximization (EM) algorithm. The spatial variance of each color component is determined using GMM parameters and hence object position is estimated. Further, spatial variance of color components and object position is exploited to compute saliency map at a scale level. Afterwards, all the saliency maps generated across various scales are linearly combined to produce the final saliency map. The performance of the proposed model is compared in terms of Precision, Recall, F-Measure, Area under the Curve (AUC), Receiver Operating Characteristics (ROC) and Mean Absolute Error (MAE). Extensive experiments on six publicly available datasets viz. MSRA10K, DUT-OMRON, ECSSD, PASCAL-S, SED2, and THUR15K show that the proposed model outperforms or comparable against 11 state-of-the-art methods of the last decade. The key features of the proposed method are object completeness and efficiency in terms of computational time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Achanta R, Hemami S, Estrada F, Süsstrunk S (2009) Frequency-tuned salient region detection. In: IEEE International conference on computer vision and pattern recognition (CVPR 2009), CONF, pp 1597–1604

  2. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S, et al. (2010) Slic superpixels. Ecole Polytechnique fédéral de Lausssanne (EPFL). Tech Rep 149300:155–162

    Google Scholar 

  3. Alpert S, Galun M, Basri R, Brandt A (2007) Image segmentation by probabilistic bottom-up aggregation and cue integration. In: IEEE Conference on computer vision and pattern recognition, 2007. CVPRn’07. IEEE, pp 1–8

  4. Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. In: ACM Transactions on graphics (TOG), vol 26, ACM, pp 10

  5. Bishop CM (2006) Pattern recognition and machine learning. Springer

  6. Borji A, Cheng MM, Jiang H, Li J (2015) Salient object detection: a benchmark. IEEE Trans Image Process 12(24):5706–5722

    Article  MathSciNet  Google Scholar 

  7. Borji A, Itti L (2013) State-of-the-art in visual attention modeling. IEEE Trans Pat Anal Mach Intel 35(1):185–207

    Article  Google Scholar 

  8. Chang KY, Liu TL, Chen HT, Lai SH (2011) Fusing generic objectness and visual saliency for salient object detection. In: 2011 International conference on computer vision, IEEE, pp 914–921

  9. Chen T, Cheng MM, Tan P, Shamir A, Hu SM (2009) Sketch2photo: Internet image montage. In: ACM Transactions on graphics (TOG), vol 28, ACM, pp 124

  10. Cheng MM, Mitra NJ, Huang X, Hu SM (2014) Salientshape: Group saliency in image collections. Vis Comput 30(4):443–453

    Article  Google Scholar 

  11. Cheng MM, Mitra NJ, Huang X, Torr PH, Hu SM (2015) Global contrast based salient region detection. IEEE Trans Pat Anal Mach Intel 37(3):569–582

    Article  Google Scholar 

  12. Duan L, Wu C, Miao J, Qing L, Fu Y (2011) Visual saliency detection by spatially weighted dissimilarity. In: 2011 IEEE conference on Computer vision and pattern recognition (CVPR), IEEE, pp 473–480

  13. Erdem E, Erdem A (2013) Visual saliency estimation by nonlinearly integrating features using region covariances. J Vis 13(4):11–11

    Article  Google Scholar 

  14. Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88 (2):303–338

    Article  Google Scholar 

  15. GAO D (2007) The discriminant center-surround hypothesis for bottom-up saliency. In: Proc Conference on neural information processing systems (NIPS), 2007

  16. GOFERMAN S (2010) Context-aware saliency detection. In: Proc IEEE Conf Computer vision and pattern recognition, 2010, pp 2376–2383

  17. Gopalakrishnan V, Hu Y, Rajan D (2010) Random walks on graphs for salient object detection in images. IEEE Trans Image Process 19(12):3232–3242

    Article  MathSciNet  Google Scholar 

  18. Harel J, Koch C, Perona P (2007) Graph-based visual saliency. In: Advances in neural information processing systems, pp 545–552

  19. He S, Lau RW, Liu W, Huang Z, Yang Q (2015) Supercnn: a superpixelwise convolutional neural network for salient object detection. Int J Comput Vis 115(3):330–344

    Article  MathSciNet  Google Scholar 

  20. Hou X, Zhang L (2007) Saliency detection: a spectral residual approach. In: IEEE conference on computer vision and pattern recognition, 2007. CVPR’07. IEEE, pp 1–8

  21. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pat Anal Mach Intel 20(11):1254–1259

    Article  Google Scholar 

  22. Jiang B, Zhang L, Lu H, Yang C, Yang MH (2013) Saliency detection via absorbing markov chain. In: Proceedings of the IEEE international conference on computer vision, pp 1665–1672

  23. Jiang H, Wang J, Yuan Z, Liu T, Zheng N, Li S (2011) Automatic salient object segmentation based on context and shape prior. In: BMVC, vol 6, pp 9

  24. Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: a discriminative regional feature integration approach

  25. Judd T, Durand F, Torralba A (2012) A benchmark of computational models of saliency to predict human fixations. Technical report, MIT tech report

  26. Ko BC, Nam JY (2006) Object-of-interest image segmentation based on human attention and semantic region clustering. JOSA A 23(10):2462–2470

    Article  Google Scholar 

  27. Kumar N (2018) Thresholding in salient object detection: a survey. Multimedia Tools and Applications 77(15):19,139–19,170

    Article  Google Scholar 

  28. Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5455–5463

  29. Li J, Levine MD, An X, Xu X, He H (2013) Visual saliency based on scale-space analysis in the frequency domain. IEEE Trans Pat Anal Mach Intel 35(4):996–1010. https://doi.org/10.1109/TPAMI.2012.147

    Article  Google Scholar 

  30. Li X, Lu H, Zhang L, Ruan X, Yang MH (2013) Saliency detection via dense and sparse reconstruction. In: Proceedings of the IEEE international conference on computer vision, pp 2976–2983

  31. Li Y, Hou X, Koch C, Rehg JM, Yuille AL (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 280–287

  32. Liu GH, Yang JY (2019) Exploiting color volume and color difference for salient region detection. IEEE Trans Image Process 28(1):6–16

    Article  MathSciNet  Google Scholar 

  33. Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum HY (2011) Learning to detect a salient object. IEEE Trans Pat Anal Mach Intel 33 (2):353–367

    Article  Google Scholar 

  34. Ma YF, Zhang HJ (2003) Contrast-based image attention analysis by using fuzzy growing. In: Proceedings of the eleventh ACM international conference on Multimedia, ACM, pp 374–381

  35. Marat S, Guironnet M, Pellerin D (2007) Video summarization using a visual attention model. In: 2007 15Th european signal processing conference, IEEE, pp 1784–1788

  36. Margolin R, Tal A, Zelnik-Manor L (2013) What makes a patch distinct?. Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 1139–1146

  37. Moon TK (1996) The expectation-maximization algorithm. IEEE Signal Processing Magazine 13(6):47–60

    Article  Google Scholar 

  38. Murray N, Vanrell M, Otazu X, Parraga CA (2011) Saliency estimation using a non-parametric low-level vision model. In: CVPR 2011, IEEE, pp 433–440

  39. Park M, Kumar M, Loui AC (2011) Saliency detection using region-based incremental center-surround distance. In: 2011 IEEE International symposium on multimedia, IEEE, pp 249–256

  40. Perazzi F, Krähenbühl P, Pritch Y, Hornung A (2012) Saliency filters: Contrast based filtering for salient region detection. In: 2012 IEEE Conference on computer vision and pattern recognition, IEEE, pp 733–740

  41. Ren Z, Gao S, Chia LT, Tsang IWH (2013) Region-based saliency detection and its application in object recognition. IEEE Trans Circ Syst Vid Technol 24(5):769–779

    Article  Google Scholar 

  42. Ren Z, Hu Y, Chia LT, Rajan D (2010) Improved saliency detection based on superpixel clustering and saliency propagation. In: Proceedings of the 18th ACM international conference on Multimedia, ACM, pp 1099–1102

  43. Roy S, Das S (2013) Spatial variance of color and boundary statistics for salient object detection. In: 2013 Fourth national conference on computer vision, pattern recognition, image processing and graphics (NCVPRIPG), IEEE , pp 1–4

  44. Seo HJ, Milanfar P (2009) Static and space-time visual saliency detection by self-resemblance. J Vis 9(12):15–15

    Article  Google Scholar 

  45. Siagian C, Itti L (2007) Rapid biologically-inspired scene classification using features shared with visual attention. IEEE Trans Pat Anal Mach Intel 29(2):300–312

    Article  Google Scholar 

  46. Singh N, Arya R, Agrawal R (2016) A convex hull approach in conjunction with gaussian mixture model for salient object detection. Digital Signal Processing 55:22–31

    Article  Google Scholar 

  47. Singh N, Arya R, Agrawal R (2017) A novel position prior using fusion of rule of thirds and image center for salient object detection. Multimedia Tools and Applications 76(8):10,521–10,538

    Article  Google Scholar 

  48. Singh N, Arya R, Agrawal R (2018) Performance enhancement of salient object detection using superpixel based gaussian mixture model. Multimedia Tools and Applications, pp 1–19

  49. Snowden RJ (2002) Visual attention to color: Parvocellular guidance of attentional resources? Psychol Sci 13(2):180–184

    Article  Google Scholar 

  50. Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1155–1162

  51. Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3166–3173

  52. Zeng Y, Feng M, Lu H, Yang G, Borji A (2018) An unsupervised game-theoretic approach to saliency detection. IEEE Trans Image Process 27(9):4545–4554

    Article  MathSciNet  Google Scholar 

  53. Zhai Y, Shah M (2006) Visual attention detection in video sequences using spatiotemporal cues. In: Proceedings of the 14th ACM international conference on Multimedia, ACM, pp 815–824

  54. Zhang J, Sclaroff S (2016) Exploiting surroundedness for saliency detection: a boolean map approach. IEEE Trans Pat Anal Mach Intel 38(5):889–902

    Article  Google Scholar 

  55. Zhang L, Tong MH, Marks TK, Shan H, Cottrell GW (2008) Sun: a bayesian framework for saliency using natural statistics. J Vis 8(7):32–32

    Article  Google Scholar 

  56. Zhang W, Wu QJ, Wang G, Yin H (2010) An adaptive computational model for salient object detection. IEEE Transactions on Multimedia 12 (4):300–316

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Institute of Technology Uttarakhand, Uttarakhand, India

    Vivek Kumar Singh & Nitin Kumar

  2. Motilal Nehru National Institute of Technology Allahabad, Uttar Pradesh, India

    Navjot Singh

Authors
  1. Vivek Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nitin Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Navjot Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vivek Kumar Singh.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singh, V.K., Kumar, N. & Singh, N. A hybrid approach using color spatial variance and novel object position prior for salient object detection. Multimed Tools Appl 79, 30045–30067 (2020). https://doi.org/10.1007/s11042-020-09467-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09467-4

Keywords

Search

Navigation