Exploiting color name space for salient object detection

Multimedia Tools and Applications volume 79, pages10873–10897(2020)Cite this article

Abstract

In this paper, we will investigate the contribution of color names for the task of salient object detection. An input image is first converted to color name space, which is consisted of 11 probabilistic channels. By exploiting a surroundedness cue, we obtain a saliency map through a linear combination of a set of sequential attention maps. To overcome the limitation of only using the surroundedness cue, two global cues with respect to color names are invoked to guide the computation of a weighted saliency map. Finally, we integrate the above two saliency maps into a unified framework to generate the final result. In addition, an improved post-processing procedure is introduced to effectively suppress image backgrounds while uniformly highlight salient objects. Experimental results show that the proposed model produces more accurate saliency maps and performs well against twenty-one saliency models in terms of three evaluation metrics on three public data sets.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Notes

  1. 1.

    http://lear.inrialpes.fr/people/vandeweijer/color_names.html

  2. 2.

    http://cs-people.bu.edu/jmzhang/BMS/BMS.html

  3. 3.

    http://www.escience.cn/people/jianli/DataBase.html

  4. 4.

    http://www.cse.cuhk.edu.hk/leojia/projects/hsaliency/

  5. 5.

    http://www.loujing.com/rpc-saliency/

  6. 6.

    http://www.escience.cn/people/chengong/Codes.html

  7. 7.

    http://cg.cs.tsinghua.edu.cn/people/~cmm/Saliency/Index.htm

  8. 8.

    http://mmcheng.net/msra10k/

  9. 9.

    http://mmcheng.net/salobjbenchmark/

  10. 10.

    http://research.microsoft.com/en-us/um/people/jiansun/SalientObject/salient_object.htm

  11. 11.

    http://ivrl.epfl.ch/supplementary_material/RK_CVPR09/

  12. 12.

    http://www.cse.cuhk.edu.hk/leojia/projects/hsaliency/dataset.html

References

  1. 1.

    Achanta R, Süsstrunk S (2010) Saliency detection using maximum symmetric surround. In: Proceedings of the IEEE international conference on image processing, pp 2653–2656

  2. 2.

    Achanta R, Estrada F, Wils P, Süsstrunk S (2008) Salient region detection and segmentation. In: Proceedings of the international conference on computer vision systems, pp 66–75

  3. 3.

    Achanta R, Hemami S, Estrada F, Süsstrunk S (2009) Frequency-tuned salient region detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1597– 1604

  4. 4.

    Berlin B, Kay P (1969) Basic color terms: their universality and evolution. University of California Press, Berkeley

    Google Scholar 

  5. 5.

    Borji A, Frintrop S, Sihite DN, Itti L (2012) Adaptive object tracking by learning background context. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 23–30

  6. 6.

    Borji A, Cheng MM, Jiang H, Li J (2015) Salient object detection: a benchmark. IEEE Trans Image Process 24(12):5706–5722

    MathSciNet  Article  Google Scholar 

  7. 7.

    Cheng MM, Zhang G, Mitra NJ, Huang X, Hu SM (2011) Global contrast based salient region detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 409–416

  8. 8.

    Cheng MM, Warrell J, Lin W, Zheng S, Vineet V, Crook N (2013) Efficient salient region detection with soft image abstraction. In: Proceedings of the IEEE international conference on computer vision, pp 1529–1536

  9. 9.

    Cheng MM, Mitra NJ, Huang X, Torr PHS, Hu SM (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582

    Article  Google Scholar 

  10. 10.

    Duan L, Wu C, Miao J, Qing L, Fu Y (2011) Visual saliency detection by spatially weighted dissimilarity. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 473–480

  11. 11.

    Erdem E, Erdem A (2013) Visual saliency estimation by nonlinearly integrating features using region covariances. J Vis 13(4):11:1–20

    Article  Google Scholar 

  12. 12.

    Goferman S, Zelnik-Manor L, Tal A (2010) Context-aware saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2376–2383

  13. 13.

    Gong C, Tao D, Liu W, Maybank S, Fang M, Fu K, Yang J (2015) Saliency propagation from simple to difficult. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2531–2539

  14. 14.

    Gonzalez RC, Woods RE, Eddins SL (2004) Morphological image processing. In: Digital image processing using MATLAB. 1st edn. chap. 9. Pearson Prentice Hall, pp 362–365

  15. 15.

    Guo C, Zhang L (2010) A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans Image Process 19(1):185–198

    MathSciNet  Article  Google Scholar 

  16. 16.

    Hofmann T (1999) Probabilistic latent semantic indexing. In: Proceedings of the ACM SIGIR conference on research and development in information retrieval, pp 50–57

  17. 17.

    Hou X, Zhang L (2007) Saliency detection: a spectral residual approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–8

  18. 18.

    Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  19. 19.

    Klein DA, Frintrop S (2011) Center-surround divergence of feature statistics for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2214–2219

  20. 20.

    Koch C, Ullman S (1985) Shifts in selective visual attention: towards the underlying neural circuitry. Hum Neurobiol 4:219–227

    Google Scholar 

  21. 21.

    Lee YJ, Ghosh J, Grauman K (2012) Discovering important people and objects for egocentric video summarization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1346–1353

  22. 22.

    Li J, Levine MD, An X, He H (2011) Saliency detection based on frequency and spatial domain analysis. In: Proceedings of the British machine vision conference, pp 86:1–11

  23. 23.

    Li J, Levine MD, An X, Xu X, He H (2013) Visual saliency based on scale-space analysis in the frequency domain. IEEE Trans Pattern Anal Mach Intell 35 (4):996–1010

    Article  Google Scholar 

  24. 24.

    Liu T, Sun J, Zheng NN, Tang X, Shum HY (2007) Learning to detect a salient object. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–8

  25. 25.

    Lou J, Ren M, Wang H (2014) Regional principal color based saliency detection. PLoS ONE 9(11):e112475:1–13

    Google Scholar 

  26. 26.

    Lou J, Zhu W, Wang H, Ren M (2017) Small target detection combining regional stability and saliency in a color image. Multimed Tools Appl 76(13):14781–14798. https://doi.org/10.1007/s11042-016-4025-7

    Article  Google Scholar 

  27. 27.

    Ma YF, Zhang HJ (2003) Contrast-based image attention analysis by using fuzzy growing. In: Proceedings of the ACM international conference on multimedia, pp 374–381

  28. 28.

    Margolin R, Tal A, Zelnik-Manor L (2013) What makes a patch distinct? In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1139–1146

  29. 29.

    Martin DR, Fowlkes CC, Malik J (2004) Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans Pattern Anal Mach Intell 26(5):530–549

    Article  Google Scholar 

  30. 30.

    Mazza V, Turatto M, Umiltà C (2005) Foreground-background segmentation and attention: a change blindness study. Psychol Res 69(3):201–210

    Article  Google Scholar 

  31. 31.

    Murray N, Vanrell M, Otazu X, Parraga CA (2011) Saliency estimation using a non-parametric low-level vision model. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 433–440

  32. 32.

    Niebur E, Koch C (1998) Computational architectures for attention. In: The attentive brain, chap. 9. MIT Press, Cambridge, pp 163–186

  33. 33.

    Palmer SE (1999) Vision science: from photons to phenomenology. The MIT Press, Cambridge

    Google Scholar 

  34. 34.

    Qin C, Zhang G, Zhou Y, Tao W, Cao Z (2014) Integration of the saliency-based seed extraction and random walks for image segmentation. Neurocomputing 129:378–391

    Article  Google Scholar 

  35. 35.

    Rahtu E, Kannala J, Salo M, Heikkilä J (2010) Segmenting salient objects from images and videos. In: Proceedings of the European conference on computer vision, pp 366–379

  36. 36.

    Ren Z, Gao S, Chia Lt, Tsang IWh (2014) Region-based saliency detection and its application in object recognition. IEEE Trans Circ Syst Video Technol 24 (5):769–779

    Article  Google Scholar 

  37. 37.

    Rubin E (1958) Figure and ground. In: Readings in perception, pp 194–203

  38. 38.

    Scharfenberger C, Wong A, Clausi DA (2015) Structure-guided statistical textural distinctiveness for salient region detection in natural images. IEEE Trans Image Process 24(1):457–470

    MathSciNet  Article  Google Scholar 

  39. 39.

    Seo HJ, Milanfar P (2009) Static and space-time visual saliency detection by self-resemblance. J Vis 9(12):15: 1–27

    Article  Google Scholar 

  40. 40.

    Shi J, Yan Q, Xu L, Jia J (2016) Hierarchical image saliency detection on extended CSSD. IEEE Trans Pattern Anal Mach Intell 38(4):717–729

    Article  Google Scholar 

  41. 41.

    Soille P (1999) Morphological image analysis: principles and applications, 2nd edn. Springer, Berlin

    Google Scholar 

  42. 42.

    Tavakoli HR, Rahtu E, Heikkilä J (2011) Fast and efficient saliency detection using sparse sampling and kernel density estimation. In: Proceedings of the scandinavian conference on image analysis, pp 666–675

  43. 43.

    Tian H, Fang Y, Zhao Y, Lin W, Ni R, Zhu Z (2014) Salient region detection by fusing bottom-up and top-down features extracted from a single image. IEEE Trans Image Process 23(10):4389–4398

    MathSciNet  Article  Google Scholar 

  44. 44.

    Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cogn Psychol 12(1):97–136

    Article  Google Scholar 

  45. 45.

    Vincent L (1993) Morphological grayscale reconstruction in image analysis: applications and efficient algorithms. IEEE Trans Image Process 2(2):176–201

    Article  Google Scholar 

  46. 46.

    van de Weijer J, Schmid C, Verbeek J (2007) Learning color names from real-world images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–8

  47. 47.

    van de Weijer J, Schmid C, Verbeek J, Larlus D (2009) Learning color names for real-world applications. IEEE Trans Image Process 18(7):1512–1523

  48. 48.

    Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1155–1162

  49. 49.

    Yang C, Zhang L, Lu H (2013) Graph-regularized saliency detection with convex-hull-based center prior. IEEE Signal Process Lett 20(7):637–640

    Article  Google Scholar 

  50. 50.

    Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3166–3173

  51. 51.

    Zhai Y, Shah M (2006) Visual attention detection in video sequences using spatiotemporal cues. In: Proceedings of the ACM international conference on multimedia, pp 815–824

  52. 52.

    Zhang J, Sclaroff S (2013) Saliency detection: a Boolean map approach. In: Proceedings of the IEEE international conference on computer vision, pp 153–160

  53. 53.

    Zhang L, Tong MH, Marks TK, Shan H, Cottrell GW (2008) SUN: a Bayesian framework for saliency using natural statistics. J Vis 8(7):32: 1–20

    Article  Google Scholar 

Download references

Acknowledgements

J. Lou is supported by the Natural Science Foundation of the Higher Education Institutions of Jiangsu Province (No. 19KJB520022), the Science and Technology Special Project of CZIMT (No. 2019-ZXKJ-02), the Cultivation Object of Major Scientific Research Project of CZIMT (No. 2019ZDXM06), the Jiangsu Province Industry University Research Cooperation Project (No. FZ20190200), the Changzhou Key Laboratory of Industrial Internet and Data Intelligence (No. CM20183002), and the QingLan Project of Jiangsu Province (2018). The work of L. Chen, F. Xu, W. Zhu, and M. Ren is supported by the National Natural Science Foundation of China (Nos. 61231014 and 61727802). H. Wang is supported by the National Defense Pre-research Foundation of China (No. 9140A01060115BQ02002) and the National Natural Science Foundation of China (No. 61703209). Q. Xia is supported by the National Natural Science Foundation of China (No. 61403202) and the China Postdoctoral Science Foundation (No. 2014M561654). The authors thank Andong Wang and Haiyang Zhang for helpful discussions regarding this manuscript.

Author information

Affiliations

  1. School of Information Engineering, Changzhou Vocational Institute of Mechatronic Technology, Changzhou, Jiangsu, 213164, People’s Republic of China

    Jing Lou

  2. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, Jiangsu, 210094, People’s Republic of China

    Huan Wang, Longtao Chen, Fenglei Xu, Qingyuan Xia, Wei Zhu & Mingwu Ren

Authors
  1. Jing Lou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Longtao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fenglei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qingyuan Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wei Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mingwu Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jing Lou or Mingwu Ren.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lou, J., Wang, H., Chen, L. et al. Exploiting color name space for salient object detection. Multimed Tools Appl 79, 10873–10897 (2020). https://doi.org/10.1007/s11042-019-07970-x

Download citation

Keywords

  • Saliency
  • Salient object detection
  • Figure-ground segregation
  • Surroundedness
  • Color names
  • Color name space