Skip to main content

Advertisement

SpringerLink
Log in

A novel approach for salient object detection using double-density dual-tree complex wavelet transform in conjunction with superpixel segmentation

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Salient object detection in wavelet domain has recently begun to attract researchers’ effort due to its desired ability to provide multi-scale analysis of an image simultaneously in both frequency and spatial domains. The proposed algorithm exploits the inherent multi-scale structure of the double-density dual-tree complex-oriented wavelet transform (DDDTCWT) to decompose each input image into four approximate sub-band images and 32 high-pass detailed sub-band images at each scale. These 32 detailed high-pass sub-bands at each scale are adequate to represent singularities of any geometric object with high precision and to mimic zooming-in and zooming-out process of human vision system. In the proposed model, we first compute a rough segmented saliency map (RSSM) by fusing multi-scale edge-to-texture features generated from DDDTCWT with segmentation results obtained from bipartite graph partitioning-based segmentation approach. Then, each pixel in RSSM is categorized into either background region or salient region based on a threshold. Finally, the pixels of the two regions are considered as samples to be drawn from a multivariate kernel function whose parameters are estimated using expectation maximization algorithm, to generate a saliency map. The performance of the proposed model is evaluated in terms of precision, recall, F-measure, area under the ROC curve and computation time using six publicly available image datasets. Extensive experimental results on six benchmark datasets demonstrate that the proposed model outperformed the existing 29 state-of-the-art methods in terms of F-measure on all five datasets, recall on four datasets and area under ROC curve on two datasets. In terms of mean recall value, mean F-measure value and mean AUC value on all six datasets, the proposed method outperforms all state-of-the-art methods. The proposed method also takes comparatively less computation time in comparison with many existing spatial domain methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://www.research.microsoft.com/enus/um/people/jiansun/salientobject/salient_object.htm.

  2. http://ivrgwww.epfl.ch/supplementary_material/RK_CVPR09/GroundTruth/binarymasks.zip.

  3. E-mail at “rinki.arya89@gmail.com” or “navjot.singh.09@gmail.com”.

  4. http://elderlab.yorku.ca/~vida/SOD/.

  5. http://www.wisdom.weizmann.ac.il/~vision/Seg_Evaluation_DB.

  6. http://www.wisdom.weizmann.ac.il/~vision/Seg_Evaluation_DB.

References

  1. Achanta R (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  2. Achanta R, Susstrunk S (2010) Saliency detection using maximum symmetric surround. In: Proceedings of 17th IEEE international conference on image processing (ICIP), pp 2653–2656

  3. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: Proceedings of IEEE conference on Computer vision and pattern recognition, pp 1597–1604

  4. Alpert S, Galun M, Brandt A, Basri R (2012) Image segmentation by probabilistic bottom-up aggregation and cue integration. IEEE Trans Pattern Anal Mach Intell 34(2):315–327

    Article  Google Scholar 

  5. Arya R, Singh N, Agrawal R (2015) A novel hybrid approach for salient object detection using local and global saliency in frequency domain. Multimed Tools Appl 75(14):8267–8287

    Article  Google Scholar 

  6. Arya R, Singh N, Agrawal R (2017) A novel combination of second-order statistical features and segmentation using multilayer superpixels for salient object detection. Appl Intell 46(2):254–271

    Article  Google Scholar 

  7. Bian P, Zhang L (2008) Biological plausibility of spectral domain approach for spatiotemporal visual saliency. In: Proceedings of the international conference on neural information processing, pp 251–258

  8. Bian P, Zhang L (2010a) Piecewise frequency domain visual saliency detection. In: Proceedings of IEEE third international conference on information and computing (ICIC), pp 269–272

  9. Bian P, Zhang L (2010) Visual saliency:a biologically plausible contourlet-like frequency domain approach. Cogn Neurodyn 4(3):189–198

    Article  Google Scholar 

  10. Borji A, Itti L (2013) State-of-the-art in visual attention modeling. IEEE Trans Pattern Anal Mach Intell 35(1):185–207

    Article  Google Scholar 

  11. Borji A, Sihite DN, Itti L (2013) Quantitative analysis of human-model agreement in visual saliency modeling:a comparative study. IEEE Trans Image Process 22(1):55–69

    Article  MathSciNet  MATH  Google Scholar 

  12. Borji A, Cheng M-M, Jiang H, Li J (2014) Salient object detection: a survey. In: arXiv preprint arXiv:14115878

  13. Borji A, Cheng M-M, Jiang H, Li J (2015) Salient object detection: a benchmark. IEEE Trans Image Process 24(12):5706–5722

    Article  MathSciNet  MATH  Google Scholar 

  14. Boykov YY, Jolly M-P (2001) Interactive graph cuts for optimal boundary and region segmentation of objects in ND images. In: Proceedings of the eighth IEEE international conference on computer vision, pp 105–112

  15. Bruce N, Tsotsos J (2006) Saliency based on information maximization. In: Advances in neural information processing systems, pp 155–162

  16. Castleman Kenneth R (1996) Digital image processing. Prentice Hall Press, Upper Saddle River

    Google Scholar 

  17. Chen L-Q, Xie X, Fan X, Ma W-Y, Zhang H-J, Zhou H-Q (2003) A visual attention model for adapting images on small displays. Multimed Syst 9(4):353–364

    Article  Google Scholar 

  18. Cheng M-M, Warrell J, Lin W-Y, Zheng S, Vineet V, Crook N (2013) Efficient salient region detection with soft image abstraction. In: Proceedings of IEEE international conference on computer vision, pp 1529–1536

  19. Cheng M, Mitra NJ, Huang X, Torr PH, Hu S (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582

    Article  Google Scholar 

  20. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  21. Daubechies I (1988) Orthonormal bases of compactly supported wavelets. Commun Pure Appl Math 41(7):909–996

    Article  MathSciNet  MATH  Google Scholar 

  22. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7(Jan):1–30

    MathSciNet  MATH  Google Scholar 

  23. Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evolut Comput 1(1):3–18

    Article  Google Scholar 

  24. Fang Y, Lin W, Lee B-S, Lau C-T, Chen Z, Lin C-W (2012) Bottom-up saliency detection model based on human visual sensitivity and amplitude spectrum. IEEE Trans Multimed 14(1):187–198

    Article  Google Scholar 

  25. Fang Y, Chen Z, Lin W, Lin C-W (2012) Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans Image Process 21(9):3888–3901

    Article  MathSciNet  MATH  Google Scholar 

  26. Felzenszwalb PF, Huttenlocher DP (2004) Efficient graph-based image segmentation. Int J Comput Vis 59(2):167–181

    Article  Google Scholar 

  27. Fu K, Gong C, Yang J, Zhou Y, Gu IY-H (2013) Superpixel based color contrast and color distribution driven salient object detection. Sig Process Image Commun 28(10):1448–1463

    Article  Google Scholar 

  28. Fu K, Gong C, Gu IY-H, Yang J (2015) Normalized cut-based saliency detection by adaptive multi-level region merging. IEEE Trans Image Process 24(12):5671–5683

    Article  MathSciNet  MATH  Google Scholar 

  29. Gasparini F, Corchs S, Schettini R (2007) Low-quality image enhancement using visual attention. Opt Eng 46(4):040502

    Article  Google Scholar 

  30. Goferman S, Zelnik-Manor L, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10):1915–1926

    Article  Google Scholar 

  31. Gopalakrishnan V, Hu Y, Rajan D (2010) Random walks on graphs for salient object detection in images. IEEE Trans Image Process 19(12):3232–3242

    Article  MathSciNet  MATH  Google Scholar 

  32. Guo C, Zhang L (2010) A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans Image Process 19(1):185–198

    Article  MathSciNet  MATH  Google Scholar 

  33. Guo C, Ma Q, Zhang L (2008) Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8

  34. Hadizadeh H, Bajic IV (2014) Saliency-aware video compression. IEEE Trans Image Process 23(1):19–33

    Article  MathSciNet  MATH  Google Scholar 

  35. Harel J, Koch C, Perona P (2006) Graph-based visual saliency. In: Proceedings of the advances in neural information processing systems, pp 545–552

  36. Hou X, Zhang L (2007) Saliency detection:A spectral residual approach. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 1–8

  37. Hou X, Harel J, Koch C (2012) Image signature: highlighting sparse salient regions. IEEE Trans Pattern Anal Mach Intell 34(1):194–201

    Article  Google Scholar 

  38. Huang K, Tao D, Yuan Y, Li X, Tan T (2011) Biologically inspired features for scene classification in video surveillance. IEEE Trans Syst Man Cybern B Cybern 41(1):307–313

    Article  Google Scholar 

  39. Huang X, Su Y, Liu Y (2016) Iteratively parsing contour fragments for object detection. Neurocomputing 175:585–598

    Article  Google Scholar 

  40. Huo L, Jiao L, Wang S, Yang S (2016) Object-level saliency detection with color attributes. Pattern Recogn 49:162–173

    Article  Google Scholar 

  41. Imamoglu N, Lin W, Fang Y (2013) A saliency detection model using low-level features based on wavelet transform. IEEE Trans Multimed 15(1):96–105

    Article  Google Scholar 

  42. Iman RL, Davenport JM (1980) Approximations of the critical region of the fbietkan statistic. Commun Stat Theory Methods 9(6):571–595

    Article  MATH  Google Scholar 

  43. Itti L (2000) Models of bottom-up and top-down visual attention. In: Doctoral dissertation California Institute of Technology

  44. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  45. Jian M, Dong J, Ma J (2011) Image retrieval using wavelet-based salient regions. Imaging Sci J 59(4):219–231

    Article  Google Scholar 

  46. Jian M, Lam K-M, Dong J, Shen L (2015) Visual-patch-attention-aware saliency detection. IEEE Trans Cybern 45(8):1575–1586

    Article  Google Scholar 

  47. Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: a discriminative regional feature integration approach. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 2083–2090

  48. Judd T, Durand F, Torralba A (2012) A benchmark of computational models of saliency to predict human fixations. In: MIT technical report technical report

  49. Kannan R, Ghinea G, Swaminathan S (2015) Salient region detection using patch level and region level image abstractions. IEEE Signal Process Lett 22(6):686–690

    Article  Google Scholar 

  50. Karssemeijer N, te Brake GM (1996) Detection of stellate distortions in mammograms. IEEE Trans Med Imaging 15(5):611–619

    Article  Google Scholar 

  51. Kim J, Han D, Tai Y-W, Kim J (2016) Salient region detection via high-dimensional color transform and local spatial support. IEEE Trans Image Process 25(1):9–23

    Article  MathSciNet  MATH  Google Scholar 

  52. Kingsbury N (1999) Image processing with complex wavelets. Philos Trans R Soc Lond A Math Phys Eng Sci 357(1760):2543–2560

    Article  MATH  Google Scholar 

  53. Ko BC, Nam J-Y (2006) Object-of-interest image segmentation based on human attention and semantic region clustering. JOSA A 23(10):2462–2470

    Article  Google Scholar 

  54. Koch C, Ullman S (1987) Shifts in selective visual attention: towards the underlying neural circuitry. In: Proceedings of the matters of intelligence, pp 115–141

  55. Kumar K (2017) An efficient SOM technique for event summarization in multi-view surveillance videos. In: Proceedings of 5th international conference on advanced computing networking and informatics (ICACNI-17), pp 1–6

  56. Kumar K, Shrimankar DD, Singh N (2016) Equal partition based clustering approach for event summarization in videos. In: Proceedings of IEEE conference on signal-image technology and internet-based systems (SITIS), pp 119–126

  57. Kumar K, Shrimankar DD, Singh N (2017) Eratosthenes sieve based key-frame extraction technique for event summarization in videos. Multimed Tools Appl 77(6):7383–7404

    Article  Google Scholar 

  58. Li Z, Itti L (2011) Saliency and gist features for target detection in satellite images. IEEE Trans Image Process 20(7):2017–2029

    Article  MathSciNet  MATH  Google Scholar 

  59. Li Z, Wu X-M, Chang S-F (2012) Segmentation using superpixels: a bipartite graph partitioning approach. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 789–796

  60. Li J, Levine MD, An X, Xu X, He H (2013) Visual saliency based on scale-space analysis in the frequency domain. IEEE Trans Pattern Anal Mach Intell 35(4):996–1010

    Article  Google Scholar 

  61. Li J, Duan L-Y, Chen X, Huang T, Tian Y (2015) Finding the secret of image saliency in the frequency domain. IEEE Trans Pattern Anal Mach Intell 37(12):2428–2440

    Article  Google Scholar 

  62. Liang J, Zhou J, Tong L, Bai X, Wang B (2018) Material based salient object detection from hyperspectral images. Pattern Recogn 76:476–490

    Article  Google Scholar 

  63. Liu Z (2014) Saliency tree: a novel saliency detection framework. IEEE Trans Image Process 23(5):1937–1952

    Article  MathSciNet  MATH  Google Scholar 

  64. Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X et al (2007) Learning to detect a salient object. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 1–8

  65. Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X et al (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367

    Article  Google Scholar 

  66. Ma Y-F, Zhang H-J (2003) Contrast-based image attention analysis by using fuzzy growing. In: Proceedings of ACM international conference on Multimedia, pp 374–381

  67. Marchesotti L, Cifarelli C, Csurka G (2009) A framework for visual saliency detection with applications to image thumbnailing. In: Proceedings of 12th IEEE international conference on computer vision, pp 2232–2239

  68. Murray N, Vanrell M, Otazu X, Parraga CA (2011) Saliency estimation using a non-parametric low-level vision model. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 433–440

  69. Naqvi SS, Browne WN, Hollitt C (2016) Salient object detection via spectral matting. Pattern Recogn 51:209–224

    Article  Google Scholar 

  70. Navalpakkam V, Itti L (2006) An integrated model of top-down and bottom-up attention for optimizing detection speed. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp 2049–2056

  71. Ninassi A, Meur OL, Callet PL, Barbba D (2007) Does where you gaze on an image affect your perception of quality? Applying visual attention to image quality metric. In: Proceedings of IEEE international conference on image processing, pp II-169

  72. Park J, Lee J-Y, Tai Y-W, Kweon IS (2012) Modeling photo composition and its application to photo re-arrangement. In: Proceedings of IEEE international conference on image processing (ICIP), pp 2741–2744

  73. Perazzi F, Krahenbuhl P, Pritch Y, Hornung A (2012) Saliency filters: contrast based filtering for salient region detection. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 733–740

  74. Rother C, Bordeaux L, Hamadi Y, Blake A (2006) Autocollage. ACM Trans Graph (TOG) 25(3):847–852

    Article  Google Scholar 

  75. Rutishauser U (2004) Is bottom-up attention useful for object recognition? In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp II-37

  76. Santella A, Agrawala M, DeCarlo D, Salesin D, Cohen M (2006) Gaze-based interaction for semi-automatic photo cropping. In: Proceedings of SIGCHI conference on human factors in computing systems, pp 771–780

  77. Selesnick IW (2001) The double density DWT. In: Proceedings of wavelets in signal and image analysis, pp 39–66

  78. Selesnick IW (2004) The double-density dual-tree DWT. IEEE Trans Signal Process 52(5):1304–1314

    Article  MathSciNet  MATH  Google Scholar 

  79. Selesnick IW, Baraniuk RG, Kingsbury NG (2005) The dual-tree complex wavelet transform. IEEE Signal Process Mag 22(6):123–151

    Article  Google Scholar 

  80. Shen X, Wu Y (2012) A unified approach to salient object detection via low rank matrix recovery. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 853–860

  81. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Mach Intell 22(8):888–905

    Article  Google Scholar 

  82. Singh N, Arya R, Agrawal R (2014) A novel approach to combine features for salient object detection using constrained particle swarm optimization. Pattern Recogn 47(4):1731–1739

    Article  Google Scholar 

  83. Singh N, Arya R, Agrawal R (2016) A convex hull approach in conjunction with Gaussian mixture model for salient object detection. Digit Signal Process 55:22–31

    Article  Google Scholar 

  84. Singh N, Arya R, Agrawal R (2016b) A novel position prior using fusion of rule of thirds and image center for salient object detection. Multimed Tools Appl 76:1–18

    Google Scholar 

  85. Singh N, Arya R, Agrawal RK (2017) Performance enhancement of salient object detection using superpixel based Gaussian mixture model. Multimed Tools Appl 77:1–19

    Google Scholar 

  86. Sun J, Lu H, Liu X (2015) Saliency region detection based on Markov absorption probabilities. IEEE Trans Image Process 24(5):1639–1649

    Article  MathSciNet  MATH  Google Scholar 

  87. Tian Q, Sebe N, Lew MS, Loupias E, Huang TS (2001) Content-based image retrieval using wavelet-based salient points. In: Photonics west 2001-electronic imaging, pp 425–436

  88. Wang Y-S, Tai C-L, Sorkine O, Lee T-Y (2008) Optimized scale-and-stretch for image resizing. ACM Trans Graph (TOG) 27(5):118

    Article  Google Scholar 

  89. Wang K, Lin L, Lu J, Li C, Shi K (2015) PISA:pixelwise image saliency by aggregating complementary appearance contrast measures with edge-preserving coherence. IEEE Trans Image Process 24(10):3019–3033

    Article  MathSciNet  MATH  Google Scholar 

  90. Xie Y, Lu H, Yang M-H (2013) Bayesian saliency via low and mid level cues. IEEE Trans Image Process 22(5):1689–1698

    Article  MathSciNet  MATH  Google Scholar 

  91. Yang C, Zhang L, Lu H, Ruan X, Yang M-H (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 3166–3173

  92. Yu Y, Wang B, Zhang L (2009) Pulse discrete cosine transform for saliency-based visual attention. In: Proceedings of IEEE 8th international conference on development and learning, pp 1–6

  93. Yu J-G, Xia G-S, Samal A, Tian J (2016) Globally consistent correspondence of multiple feature sets using proximal Gauss-Seidel relaxation. Pattern Recogn 51:255–267

    Article  Google Scholar 

  94. Zhang L, Tong MH, Marks TK, Shan H, Cottrell GW (2008) SUN: a Bayesian framework for saliency using natural statistics. J Vis 8(7):32

    Article  Google Scholar 

  95. Zhu Z, Wahid K, Babyn P, Yang R (2013) Compressed sensing-based MRI reconstruction using complex double-density dual-tree DWT. Int J Biomed Imaging 2013:907501. https://doi.org/10.1155/2013/907501

    Article  Google Scholar 

  96. Zhu L, Klein DA, Frintrop S, Cao Z, Cremers AB (2014) A multisize superpixel approach for salient object detection based on multivariate normal distribution estimation. IEEE Trans Image Process 23(12):5094–5107

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors express their gratitude to the University Grant Commission (UGC), India, and DST-Purse, India, for the obtained financial support in performing this research work.

Author information

Authors and Affiliations

  1. School of Computer and Systems Sciences, Jawaharlal Nehru University, New Delhi, 110067, India

    Rinki Arya & R. K. Agrawal

  2. Motilal Nehru National Institute of Technology Allahabad, Allahabad, 211004, India

    Navjot Singh

Authors
  1. Rinki Arya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. K. Agrawal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Navjot Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rinki Arya.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arya, R., Agrawal, R.K. & Singh, N. A novel approach for salient object detection using double-density dual-tree complex wavelet transform in conjunction with superpixel segmentation. Knowl Inf Syst 60, 327–361 (2019). https://doi.org/10.1007/s10115-018-1243-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-018-1243-5

Keywords

Search

Navigation