Skip to main content
SpringerLink
Log in

Multi-attention embedded network for salient object detection

  • Application of soft computing
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Although the salient object detection method based on the fully convolutional neural network has achieved better performance, how to learn effective feature representations in complex scenes to obtain more accurate saliency maps is still a challenge. In order to cope with the above-mentioned challenges, an addition or cascade structure is generally used to fuse feature information between multiple levels. However, these methods are susceptible to the influence of messy background information. The network may regard non-salient objects with similar salient appearances as target predictions, and the prediction results may be incomplete due to different appearance areas of salient objects. We design a network composed of multiple attention mechanisms to selectively integrate deep and shallow feature information, and more effectively deal with the transfer and fusion of features. In this paper, we propose a multi-attention embedded network (MAENet), which introduces attention mechanisms to give different feature information with different weights for handling the transfer and aggregation of features at different levels. The multi-attention feature aggregation (MAFA) module is proposed, which uses the channel attention mechanism to give different weights to the features to be fused, and then uses the spatial attention mechanism to selectively aggregate shallow edge information and deep abstract semantic features to avoid excessive redundant information which affects the saliency mapping, as well as suppressing non-salient areas with “salient” appearance. In addition, The multi-scale feature extraction (MFE) module and the self-attention (SA) module are also proposed for obtaining sufficiently rich and useful multi-scale context information and enhancing the function of the top layer. Finally, the attentional residual refinement (ARR) module is utilized to refine the saliency map after each feature fusion and further improve the input function. MAENet can accurately segment salient objects and provide clear local details. Experimental results on five benchmark datasets show that the proposed method achieves the favorable performance against 14 state-of-the-art methods on popular evaluation metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE, pp 1597–1604

  • Aksac A, Ozyer T, Alhajj R (2017) Complex networks driven salient region detection based on superpixel segmentation. Patt Recogn 66:268–279

    Article  Google Scholar 

  • Basha SM, Rajput DS (2018a) Parsing based sarcasm detection from literal language in tweets. Recent Patents Comput Sci 11(1):62–69

    Article  Google Scholar 

  • Basha SM, Rajput DS (2018b) A supervised aspect level sentiment model to predict overall sentiment on tweeter documents. Int J Metadata Semant Ontol 13(1):33–41

    Article  Google Scholar 

  • Basha SM, Rajput DS (2019) A roadmap towards implementing parallel aspect level sentiment analysis. Multim Tools Appl 78(20):29463–29492

    Article  Google Scholar 

  • Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Patt Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  • Chen S, Tan X, Wang B, Hu X (2018) Reverse attention for salient object detection. In: Proceedings of the European conference on computer vision (ECCV), pp 234–250

  • Cheng MM, Zhang FL, Mitra NJ, Huang X, Hu SM (2010) Repfinder: finding approximately repeated scene elements for image editing. ACM Trans Graphics (TOG) 29(4):1–8

    Article  Google Scholar 

  • Craye C, Filliat D, Goudou JF (2016) Environment exploration for object-based visual saliency learning. In: 2016 IEEE international conference on robotics and automation (ICRA). IEEE, pp 2303–2309

  • Deng Z, Hu X, Zhu L, Xu X, Qin J, Han G, Heng PA (2018) R3net: recurrent residual refinement network for saliency detection. In: Proceedings of the 27th international joint conference on artificial intelligence. AAAI Press, pp 684–690

  • Ebenuwa SH, Sharif MS, Alazab M, Al-Nemrat A (2019) Variance ranking attributes selection techniques for binary classification problem in imbalance data. IEEE Access 7:24649–24666

    Article  Google Scholar 

  • Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  • Fan DP, Gong C, Cao Y, Ren B, Cheng MM, Borji A (2018) Enhanced-alignment measure for binary foreground map evaluation. arXiv preprint arXiv:1805.10421

  • Fang C, Tian H, Zhang D, Zhang Q, Han J, Han J (2021) Densely nested top-down flows for salient object detection. arXiv preprint arXiv:2102.09133

  • Feng M, Lu H, Ding E (2019) Attentive feedback network for boundary-aware salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1623–1632

  • Gadekallu TR, Rajput DS, Reddy MPK, Lakshmanna K, Bhattacharya S, Singh S, Jolfaei A, Alazab M (2020) A novel pca–whale optimization-based deep neural network model for classification of tomato plant diseases using gpu. J Real Time Image Process 1–14

  • Gadekallu TR, Alazab M, Kaluri R, Maddikunta PKR, Bhattacharya S, Lakshmanna K, Parimala M (2021) Hand gesture classification using a novel cnn-crow search algorithm. Complex Intell Syst 1–14

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  • Hou Q, Cheng MM, Hu X, Borji A, Tu Z, Torr PH (2017) Deeply supervised salient object detection with short connections. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3203–3212

  • Hu X, Zhu L, Qin J, Fu CW, Heng PA (2018) Recurrently aggregating deep features for salient object detection. In: Proceedings of the AAAI conference on artificial intelligence, vol 32

  • Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: a discriminative regional feature integration approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2083–2090

  • Lee H, Kim D (2018) Salient region-based online object tracking. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1170–1177

  • Li G, Yu Y (2016) Visual saliency detection based on multiscale deep cnn features. IEEE Trans Image Process 25(11):5012–5024

    Article  MathSciNet  Google Scholar 

  • Li Y, Hou X, Koch C, Rehg JM, Yuille AL (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 280–287

  • Liang J, Zhou J, Tong L, Bai X, Wang B (2018) Material based salient object detection from hyperspectral images. Patt Recogn 76:476–490

    Article  Google Scholar 

  • Liu JJ, Hou Q, Cheng MM, Feng J, Jiang J (2019) A simple pooling-based design for real-time salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3917–3926

  • Liu N, Han J (2016) Dhsnet: deep hierarchical saliency network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 678–686

  • Liu N, Han J, Yang MH (2018) Picanet: learning pixel-wise contextual attention for saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3089–3098

  • Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum HY (2010) Learning to detect a salient object. IEEE Trans Patt Anal Mach Intell 33(2):353–367

    Google Scholar 

  • Ma C, Miao Z, Zhang XP, Li M (2017) A saliency prior context model for real-time object tracking. IEEE Trans Multim 19(11):2415–2424

    Article  Google Scholar 

  • Margolin R, Zelnik-Manor L, Tal A (2014) How to evaluate foreground maps? In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 248–255

  • Mechrez R, Shechtman E, Zelnik-Manor L (2019) Saliency driven image manipulation. Mach Vis Appl 30(2):189–202

    Article  Google Scholar 

  • Mohammadi S, Noori M, Bahri A, Majelan SG, Havaei M (2020) Cagnet: content-aware guidance for salient object detection. Patt Recogn 103:107303

  • Pang Y, Zhao X, Zhang L, Lu H (2020) Multi-scale interactive network for salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9413–9422

  • Perazzi F, Krähenbühl P, Pritch Y, Hornung A (2012) Saliency filters: Contrast based filtering for salient region detection. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 733–740

  • Qin X, He S, Zhang Z, Dehghan M, Jagersand M (2018) Bylabel: a boundary based semi-automatic image annotation tool. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1804–1813

  • Qin X, Zhang Z, Huang C, Gao C, Dehghan M, Jagersand M (2019) Basnet: boundary-aware salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7479–7489

  • Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  • Wang L, Lu H, Wang Y, Feng M, Wang D, Yin B, Ruan X (2017a) Learning to detect salient objects with image-level supervision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 136–145

  • Wang L, Wang L, Lu H, Zhang P, Ruan X (2018a) Salient object detection with recurrent fully convolutional networks. IEEE Trans Patt Anal Mach Intell 41(7):1734–1746

    Article  Google Scholar 

  • Wang T, Borji A, Zhang L, Zhang P, Lu H (2017b) A stagewise refinement model for detecting salient objects in images. In: Proceedings of the IEEE international conference on computer vision, pp 4019–4028

  • Wang T, Zhang L, Wang S, Lu H, Yang G, Ruan X, Borji A (2018b) Detect globally, refine locally: a novel approach to saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3127–3135

  • Wei J, Wang S, Huang Q (2020) F\(^3\)net: fusion, feedback and focus for salient object detection. Proc AAAI Conf Artif Intell 34:12321–12328

    Google Scholar 

  • Woo S, Park J, Lee JY, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  • Wu Z, Su L, Huang Q (2019) Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3907–3916

  • Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1155–1162

  • Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3166–3173

  • Zhang L, Dai J, Lu H, He Y, Wang G (2018a) A bi-directional message passing model for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1741–1750

  • Zhang P, Wang D, Lu H, Wang H, Ruan X (2017) Amulet: Aggregating multi-level convolutional features for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 202–211

  • Zhang P, Liu W, Lu H, Shen C (2018b) Salient object detection by lossless feature reflection. arXiv preprint arXiv:1802.06527

  • Zhang X, Wang T, Qi J, Lu H, Wang G (2018c) Progressive attention guided recurrent network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 714–722

Download references

Author information

Authors and Affiliations

  1. Department of Information Engineering, China Jiliang University, Hangzhou, 310018, China

    Wei He, Chen Pan & Wenlong Xu

  2. Lishui people’s Hospital, Lishui, Zhejiang Province, China

    Ning Zhang

Authors
  1. Wei He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenlong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ning Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

WH was involved in conceptualization, methodology, software, writing—original draft. CP helped in supervision, project administration, writing—review & editing. WX was involved in formal analysis, investigation. NZ helped in data collection and collation

Corresponding author

Correspondence to Chen Pan.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the Natural Science Foundation of Zhejiang Province (Grant LY19F030013), the National Natural Science Foundation of China (Grant No.61672476) and Key R&D Program of Zhejiang Province (Grant No.2020ZJZC02).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, W., Pan, C., Xu, W. et al. Multi-attention embedded network for salient object detection. Soft Comput 25, 13053–13067 (2021). https://doi.org/10.1007/s00500-021-06146-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-021-06146-w

Keywords

Search

Navigation