Skip to main content
SpringerLink
Log in

Deep supervised visual saliency model addressing low-level features

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Deep neural networks detect visual saliency with semantic information. These high-level features locate salient regions efficiently but pay less attention to structure preservation. In our paper, we emphasize crucial low-level features for deep neural networks in order to preserve local structure and integrity of objects. The proposed framework consists of an image enhancement network and a saliency prediction network. In the first part of our model, we segment the image with a superpixel based unit-linking pulse coupled neural network (PCNN) and generate a weight map representing contrast and spatial properties. With the help of these low-level features, a fully convolutional network (FCN) is employed to compute saliency map in the second part. The weight map enhances the input channels of the FCN, meanwhile refines the output prediction with polished details and contours of salient objects. We demonstrate the superior performance of our model against other state-of-the-art approaches through experimental results on five benchmark datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 1597–1604

  • Achanta R, Shaji A, Smith K, Luchhi A, Fua P, Susstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. Proc IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  • Akram T, Khan MA, Sharif M, Yasmin M (2018) Skin lesion segmentation and recognition using multichannel saliency estimation and M-SVM on selected serially fused features. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-018-1051-5

    Article  Google Scholar 

  • Alpert S, Galun M, Basri R, Brandt A (2012) Image segmentation by probabilistic bottom–up aggregation and cue integration. IEEE Trans Pattern Anal Mach Intell 34(2):315–327

    Article  Google Scholar 

  • Amin-Naji M, Aghagolzadeh A, Ezoji M (2019) CNNs hard voting for multi-focus image fusion. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-019-01199-0

    Article  Google Scholar 

  • Borji A, Cheng MM, Hou Q, Jiang H, Li J (2015) Salient object detection: a benchmark. IEEE Trans Image Proc 24(12):5706–5722

    Article  MathSciNet  Google Scholar 

  • Chen T, Lin L, Liu L, Luo X, Li X (2016) DISC: deep image saliency computing via progressive representation learning. IEEE Trans Neural Netw Learn Syst 27(6):1135–1149

    Article  MathSciNet  Google Scholar 

  • Cheng MM, Zhang GX, Mitra NJ, Huang X, Hu SM (2011) Global contrast based salient region detection. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 409–416

  • Donoser M, Urschler M, Hirzer M, Bischof H (2009) Saliency driven total variation segmentation. In Proceedings of the IEEE International Conference on computer vision, pp 817–824

  • Eckhorn R, Reitboeck HJ, Arndt M, Dicke P (1990) Feature linking via synchronization among distributed assemblies: simulations of results from cat visual cortex. Neural Comput 2(3):293–307

    Article  Google Scholar 

  • Felzenszwalb P, Huttenlocher D (2004) Efficient graph-based image segmentation. Int J Comput Vis 59(2):167–181

    Article  Google Scholar 

  • Gao Y, Wang M, Zha ZJ, Shen J, Li X, Wu X (2013) Visualtextual joint relevance learning for tag-based social image search. IEEE Trans Image Process 22(1):363–376

    Article  MathSciNet  Google Scholar 

  • Gordo A, Almazán J, Revaud J, Larlus D (2016) Deep image retrieval: learning global representations for image search. In: Proceedings of the European Conference on computer vision, pp 241–257

  • Gu X (2008) Feature extraction using unit-linking pulse coupled neural network and its applications. Neural Process Lett 27(1):25–41

    Article  Google Scholar 

  • He S, Lau RWH, Liu W, Huang Z, Yang Q (2015) SuperCNN: a superpixelwise convolutional neural network for salient object detection. Int J Comput Vis 115(3):330–344

    Article  MathSciNet  Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 770–778

  • Hou Q, Cheng MM, Hu X, Borji A, Tu Z, Torr P (2019) Deeply supervised salient object detection with short connections. IEEE Trans Pattern Anal Mach Intell 41(4):815–828

    Article  Google Scholar 

  • Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint 1704:04861

    Google Scholar 

  • Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  • Jia Y, Shelhamer E, Donahue J et al (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on multimedia, pp 675–678

  • Jiang B, Zhang L, Lu H, Yang C, Yang MH (2013) Saliency detection via absorbing Markov chain. In: Proceedings of the 2013 IEEE International Conference on computer vision, pp 1665–1672

  • Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: a discriminative regional feature integration approach. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 2083–2090

  • Johnson JL, Ritter D (1993) Observation of periodic waves in a pulse-coupled neural network. Opt Lett 18:1253–1255

    Article  Google Scholar 

  • Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  • Lee G, Tai YW, Kim J (2016) Deep saliency with encoded low level distance map and high level features. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 660–668

  • Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 5455–5463

  • Li G, Yu Y (2018) Contrast-oriented deep neural networks for salient object detection. IEEE Trans Neural Netw Learn Syst 29(12):6038–6051

    Article  Google Scholar 

  • Li X, Lu H, Zhang L, Xiang R, Yang MH (2013) Saliency detection via dense and sparse reconstruction. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 2976–2983

  • Li Y, Hou X, Koch C, Rehg JM, Yuille AL (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 280–287

  • Li X, Zhao L, Wei L, Yang MH, Wu F, Zhuang Y, Ling H, Wang J (2016) Deepsaliency: multi-task deep neural network model for salient object detection. IEEE Trans Image Proc 25(8):3919–3930

    Article  MathSciNet  Google Scholar 

  • Li H, Chen J, Lu H, Chi Z (2017) CNN for saliency detection with low-level feature integration. Neurocomputing 226:212–220

    Article  Google Scholar 

  • Lin L, Wang X, Yang W, Lai JH (2015) Discriminatively trained and-or graph models for object shape detection. IEEE Trans Pattern Anal Mach Intell 37(5):959–972

    Article  Google Scholar 

  • Liu N, Han J (2016) DHSnet: deep hierarchical saliency network for salient object detection. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 678–686

  • Liu T, Sun J, Zheng N, Tang X, Shum H (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367

    Article  Google Scholar 

  • Luo Z, Mishra A, Achkar A, Eichel J, Li S, Jodoin PM (2017) Non-local deep features for salient object detection. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 6593–6601

  • Mai L, Niu Y, Liu F (2013) Saliency aggregation: a data-driven approach. In Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 1131–1138

  • Nie R, He M, Cao J, Zhou D, Liang Z (2018) Pulse coupled neural network based MRI image enhancement using classical visual receptive field for smarter mobile healthcare. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-018-1098-3

    Article  Google Scholar 

  • Orlando JT, Rui S (2002) Image segmentation by histogram thresholding using fuzzy sets. IEEE Trans Image Proc 11(12):1457–1465

    Article  Google Scholar 

  • Reynolds JH, Desimone R (2003) Interacting roles of attention and visual salience in V4. Neuron 37(5):853–863

    Article  Google Scholar 

  • Shelhamer E, Long J, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 3431–3440

  • Shen X, Wu X (2012) A unified approach to salient object detection via low rank matrix recovery. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 853–860

  • Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: Proceedings of the International Conference on learning representation, pp 1–14

  • Wang L, Lu H, Ruan X, Yang MH (2015) Deep networks for saliency detection via local estimation and global search. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 3183–3192

  • Wang Z, Chen T, Li G, Xu R, Lin L (2017) Multi-label image recognition by recurrently discovering attentional regions. In: Proceedings of the IEEE International Conference on computer vision, pp 464–472

  • Wang H, Dai L, Cai Y, Sun X, Chen L (2018) Salient object detection based on multi-scale contrast. Neural Netw 101:47–56

    Article  Google Scholar 

  • Wang L, Wang L, Lu H, Zhang P, Ruan X (2019) Salient object detection with recurrent fully convolutional networks. IEEE Trans Pattern Anal Mach Intell 41(7):1734–1746

    Article  Google Scholar 

  • Wu H, Li G, Luo X (2014) Weighted attentional blocks for probabilistic object tracking. Vis. Comput 30(2):229–243

    Article  Google Scholar 

  • Xiao Z, Shi J, Chang Q (2009) Automatic image segmentation algorithm based on PCNN and fuzzy mutual information. In: Proceedings of the IEEE International Conference on computer and information technology, pp 241–245

  • Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 1155–1162

  • Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 3166–3173

  • Zhang P, Wang D, Lu H (2017) Learning uncertain convolutional features for accurate saliency detection. In: Proceedings of the IEEE International Conference on computer vision, pp 212–221

  • Zhao R, Ouyang W, Li H, Wang X (2015) Saliency detection by multi-context deep learning. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 1265–1274

  • Zhong X, Shih FY (2019) An efficient saliency detection model based on wavelet generalized lifting. Int J Pattern Recogn 33(02):1954006

    Article  Google Scholar 

  • Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 2814–2821

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grants 61771145 and 61371148.

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, Fudan University, Shanghai, 200433, China

    Lecheng Zhou & Xiaodong Gu

Authors
  1. Lecheng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaodong Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaodong Gu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, L., Gu, X. Deep supervised visual saliency model addressing low-level features. J Ambient Intell Human Comput 14, 15659–15672 (2023). https://doi.org/10.1007/s12652-019-01441-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-019-01441-9

Keywords

Search

Navigation