Abstract
Many existing salient object detection methods are dedicated to fusing features from different levels of a pre-trained convolutional neural network (CNN). However, these methods can easily lead to internal discontinuities within the salient objects because of unreasonable feature fusion strategies and short-range dependencies resulting from common convolution and pooling operations. In this paper, we propose a novel non-local duplicate pooling (NLDP) network to overcome these internal discontinuities. NLDP begins by removing the first few convolutional layers of a classic CNN, which have small receptive fields and require large amounts of calculation. A novel duplicate pooling module (DPM) is then used to generate richer and more detailed saliency maps. This is achieved by constructing a double-pathway that can integrating partial feature maps. Within the DPM, a non-local module (NLM) is used to obtain long-range dependencies. This enhances the internal continuities between the saliency maps. Comprehensive experiments conducted on six benchmark datasets have confirmed the increased effectiveness and detection speed of our method in relation to other salient object detection methods.
Similar content being viewed by others
References
Wang W, Shen J, Porikli F (2015) Saliency-aware geodesic video object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3395–3402
Wang W, Shen J, Sun H, et al. (2017) Video co-saliency guided co-segmentation. IEEE Trans Circ Syst Video Technol 28(8):1727–1736
Hong S, You T, Kwak S, et al. (2015) Online tracking by learning discriminative saliency map with convolutional neural network. In: International conference on machine learning, pp 597–606
Cheng MM, Zhang FL, Mitra NJ, et al. (2010) Repfinder: finding approximately repeated scene elements for image editing. ACM Transactions on Graphics (TOG) 29(4):1–8
Zhang GX, Cheng MM, Hu SM, et al. (2009) A shape-preserving approach to image resizing. Computer Graphics Forum 28(7):1897–1906
Craye C, Filliat D, Goudou JF (2016) Environment exploration for object-based visual saliency learning. In: 2016 IEEE international conference on robotics and automation (ICRA), pp 2303–2309
Zhai Y, Shah M (2006) Visual attention detection in video sequences using spatiotemporal cues. In: Proceedings of the 14th ACM international conference on multimedia, pp 815–824
Cheng MM, Mitra NJ, Huang X, et al. (2014) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582
Achanta R, Estrada F, Wils P, et al. (2008) Salient region detection and segmentation. In: International conference on computer vision systems, pp 66–75
Achanta R, Hemami S, Estrada F, et al. (2009) Frequency-tuned salient region detection. In: 2009 IEEE conference on computer vision and pattern recognition, pp 1597–1604
Itti L (1998) A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Trans 20. https://doi.org/10.1109/34.730558
Rosin PL (2009) A simple method for detecting salient regions[J]. Pattern Recogn 42(11):2363–2371. https://doi.org/10.1016/j.patcog.2009.04.021
Yu Z, Wong HS (2007) A rule based technique for extraction of visual attention regions based on real-time clustering[J]. IEEE Trans Multimed 9(4):766–784. https://doi.org/10.1109/TMM.2007.893351
Siva P, Russell C, Xiang T, et al. (2013) Looking beyond the image: unsupervised learning for object saliency and detection. IEEE Conference on Computer Vision and Pattern Recognition, pp 3238–3245
Li H, Ngan KN (2011) A co-saliency model of image pairs[J]. IEEE Trans Image Process 20 (12):3365–3375. https://doi.org/10.1109/TIP.2011.2156803
Fu H, Cao X, Tu Z (2013) cluster-based co-saliency detection. In IEEE Transactions on Image Processing 22:3766–3778. https://doi.org/10.1109/TIP.2013.2260166
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440
Hou Q, Cheng MM, Hu X, et al. (2017) Deeply supervised salient object detection with short connections. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3203–3212
Luo Z, Mishra A, Achkar A, et al. (2017) Non-local deep features for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6609–6617
Zhang P, Wang D, Lu H, et al. (2017) Amulet: aggregating multi-level convolutional features for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 202–211
He S, Jiao J, Zhang X, et al. (2017) Delving into salient object subitizing and detection. In: Proceedings of the IEEE international conference on computer vision, Venice, pp 1059–1067
Hu X, Zhu L, Qin J, et al. (2018) Recurrently aggregating deep features for salient object detection. In: Thirty-second AAAI conference on artificial intelligence
Amirul Islam M, Kalash M, Bruce NDB (2018) Revisiting salient object detection: simultaneous detection, ranking, and subitizing of multiple salient objects. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7142–7150
Liu N, Han J (2016) Dhsnet: deep hierarchical saliency network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 678–686
Zhang D, Han J, Zhang Y (2017) Supervision by fusion: towards unsupervised learning of deep salient object detector. In: Proceedings of the IEEE international conference on computer vision, pp 4048–4056
Zhang L, Dai J, Lu H, et al. (2018) A bi-directional message passing model for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1741–1750
Wang T, Zhang L, Wang S, et al. (2018) Detect globally, refine locally: a novel approach to saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3127–3135
Zhang X, Wang T, Qi J, et al. (2018) Progressive attention guided recurrent network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 714–722
Wang W, Shen J, Dong X, et al. (2018) Salient object detection driven by fixation prediction. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1711–1720
Liu N, Han J, Yang MH (2018) Picanet: learning pixel-wise contextual attention for saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3089–3098
Chen S, Tan X, Wang B, et al. (2018) Reverse attention for salient object detection. In: Proceedings of the European conference on computer vision (ECCV), pp 234–250
Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybern 36(4):193–202
LeCun Y, Boser B, Denker JS, et al. (1989) Backpropagation applied to handwritten zip code recognition. Neural Computation 1(4):541–551
Gao P, Zhang Q, Wang F, Xiao L, Fujita H, Zhang Y (2020) Learning reinforced attentional representation for end-to-end visual tracking. Inf Sci 517:52–67. https://doi.org/10.1016/j.ins.2019.12.084
Gao P, Yuan R, Wang F, et al. (2019) Siamese attentional keypoint network for high performance visual tracking[J]. Knowledge Based Systems 193. https://doi.org/10.1016/j.knosys.2019.105448
Wang X, Girshick R, Gupta A, et al. (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803
Pérez-Hernández F, Tabik S, Lamas A, et al. (2020) Object detection binary classifiers methodology based on deep learning to identify small objects handled similarly: application in video surveillance[J]. Knowledge-Based Systems. https://doi.org/10.1016/j.knosys.2020.105590
Li G, Yu Y (2016) Deep contrast learning for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 478–487
Wu Z, Su L, Huang Q (2019) Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3907–3916
Zhou B, Khosla A, Lapedriza A, et al. (2014) Object detectors emerge in deep scene cnns. arXiv:1412.6856
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Liu X, Zhu X, Li M, et al. (2019) Multiple kernel k-means with incomplete kernels[J]. IEEE Trans Pattern Anal Mach Intell 42(5):1191–1204. https://doi.org/10.1109/TPAMI.2019.2892416
Yu X, Ye X, Gao Q (2020) Infrared handprint image restoration algorithm based on apoptotic mechanism[j]. IEEE Access 8:47334–47343. https://doi.org/10.1109/ACCESS.2020.2979018
Liu S, Huang D (2018) Receptive field block net for accurate and fast object detection. In: Proceedings of the European conference on computer vision (ECCV), pp 385–400
Borji A, Cheng MM, Hou Q, et al. (2019) Salient object detection: A survey. Computational Visual Media 5(2):117–150 https://doi.org/10.1007/s41095-019-0149-9
Borji A, Cheng MM, Jiang H, et al. (2015) Salient object detection: a benchmark. IEEE Transactions on Image Processing 24(12):5706–5722
He K, Zhang X, Ren S, et al. (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Liu JJ, Hou Q, Cheng MM, et al. (2019) A simple pooling-based design for real-time salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3917–3926
Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395–1403
Qin X, Zhang Z, Huang C, et al. (2019) Basnet: boundary-aware salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7479–7489
Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), pp 60–65
Vaswani A, Shazeer N, Parmar N, et al. (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008
Yan Q, Xu L, Shi J, et al. (2013) Hierarchical saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1155–1162
Li Y, Hou X, Koch C, et al. (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 280–287
Yang C, Zhang L, Lu H, et al. (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3166–3173
Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5455–5463
Wang L, Lu H, Wang Y, et al. (2017) Learning to detect salient objects with image-level supervision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 136–145
Liu T, Yuan Z, Sun J, et al. (2010) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367
Wang W, Zhao S, Shen J, et al. (2019) Salient object detection with pyramid attention and salient edges. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1448–1457
Feng M, Lu H, Ding E (2019) Attentive feedback network for boundary-aware salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1623–1632
Wang W, Shen J, Cheng MM, et al. (2019) An iterative and cooperative top-down and bottom-up inference network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5968–5977
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Wang T, Borji A, Zhang L, et al. (2017) A stagewise refinement model for detecting salient objects in images. In: Proceedings of the IEEE international conference on computer vision, pp 4019–4028
Krähenbühl P, Koltun V (2011) Efficient inference in fully connected crfs with gaussian edge potentials. In: Advances in neural information processing systems, pp 109–117
Movahedi V, Elder JH (2010) Design and perceptual validation of performance measures for salient object segmentation. In: 2010 IEEE computer society conference on computer vision and pattern recognition-workshops, pp 49–56
Acknowledgements
This work is supported by National Natural Science Foundation of China (No. 62076062) and National Key Research and Development Program of China (No. 2017YFB1002801). It is also supported by Collaborative Innovation Center of Wireless Communications Technology.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jiao, J., Xue, H. & Ding, J. Non-local duplicate pooling network for salient object detection. Appl Intell 51, 6881–6894 (2021). https://doi.org/10.1007/s10489-020-02147-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-020-02147-8