Advertisement

Learning Noise-Aware Encoder-Decoder from Noisy Labels by Alternating Back-Propagation for Saliency Detection

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12362)

Abstract

In this paper, we propose a noise-aware encoder-decoder framework to disentangle a clean saliency predictor from noisy training examples, where the noisy labels are generated by unsupervised handcrafted feature-based methods. The proposed model consists of two sub-models parameterized by neural networks: (1) a saliency predictor that maps input images to clean saliency maps, and (2) a noise generator, which is a latent variable model that produces noises from Gaussian latent vectors. The whole model that represents noisy labels is a sum of the two sub-models. The goal of training the model is to estimate the parameters of both sub-models, and simultaneously infer the corresponding latent vector of each noisy label. We propose to train the model by using an alternating back-propagation (ABP) algorithm, which alternates the following two steps: (1) learning back-propagation for estimating the parameters of two sub-models by gradient ascent, and (2) inferential back-propagation for inferring the latent vectors of training noisy examples by Langevin Dynamics. To prevent the network from converging to trivial solutions, we utilize an edge-aware smoothness loss to regularize hidden saliency maps to have similar structures as their corresponding images. Experimental results on several benchmark datasets indicate the effectiveness of the proposed model.

Keywords

Noisy saliency Latent variable model Langevin dynamics Alternating back-propagation 

Notes

Acknowledgments

This research was supported in part by the Australia Research Council Centre of Excellence for Robotics Vision (CE140100016).

Supplementary material

504472_1_En_21_MOESM1_ESM.pdf (2.6 mb)
Supplementary material 1 (pdf 2694 KB)

References

  1. 1.
    Arbeláez, P., Pont-Tuset, J., Barron, J., Marques, F., Malik, J.: Multiscale combinatorial grouping. In: IEEE Conference on Computer Vision and Pattern Recognition (2014)Google Scholar
  2. 2.
    Baidu: PaddlePaddle. https://www.paddlepaddle.org.cn
  3. 3.
    Borji, A., Cheng, M.M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24(12), 5706–5722 (2015)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Cheng, M.-M., Mitra, N.J., Huang, X., Hu, S.-M.: SalientShape: group saliency in image collections. Vis. Comput. 30(4), 443–453 (2013).  https://doi.org/10.1007/s00371-013-0867-4CrossRefGoogle Scholar
  5. 5.
    Fan, D.P., Cheng, M.M., Liu, Y., Li, T., Borji, A.: Structure-measure: a new way to evaluate foreground maps. In: International Conference on Computer Vision, pp. 4548–4557 (2017)Google Scholar
  6. 6.
    Fan, D.P., Gong, C., Cao, Y., Ren, B., Cheng, M.M., Borji, A.: Enhanced-alignment measure for binary foreground map evaluation. In: International Joint Conference on Artificial Intelligence, pp. 698–704 (2018)Google Scholar
  7. 7.
    Feng, M., Lu, H., Ding, E.: Attentive feedback network for boundary-aware salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)Google Scholar
  8. 8.
    Goldberger, J., Ben-Reuven, E.: Training deep neural networks using a noise adaptation layer. In: International Conference on Learning Representations (2017)Google Scholar
  9. 9.
    Han, T., Lu, Y., Zhu, S.C., Wu, Y.N.: Alternating back-propagation for generator network. In: AAAI Conference on Artificial Intelligence (2017)Google Scholar
  10. 10.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  11. 11.
    Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  12. 12.
    Jiang, L., Zhou, Z., Leung, T., Li, L.J., Fei-Fei, L.: MentorNet: learning data-driven curriculum for very deep neural networks on corrupted labels. In: International Conference on Machine Learning (2018)Google Scholar
  13. 13.
    Kingma, D., Welling, M.: Auto-encoding variational bayes. In: International Conference on Learning Representations (2014)Google Scholar
  14. 14.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  15. 15.
    Li, D., Rodriguez, C., Yu, X., Li, H.: Word-level deep sign language recognition from video: a new large-scale dataset and methods comparison. In: IEEE Winter Conference on Applications of Computer Vision (2020)Google Scholar
  16. 16.
    Li, G., Yu, Y.: Visual saliency based on multiscale deep features. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5455–5463 (2015)Google Scholar
  17. 17.
    Li, G., Xie, Y., Lin, L.: Weakly supervised salient object detection using image labels. In: AAAI Conference on Artificial Intelligence (2018)Google Scholar
  18. 18.
    Li, X., Yang, F., Cheng, H., Liu, W., Shen, D.: Contour knowledge transfer for salient object detection. In: European Conference on Computer Vision (2018)Google Scholar
  19. 19.
    Liu, N., Han, J., Yang, M.H.: PicaNet: learning pixel-wise contextual attention for saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  20. 20.
    Liu, T., Tao, D.: Classification with noisy labels by importance reweighting. IEEE Trans. Pattern Anal. Mach. Intell. 38(3), 447–461 (2016)CrossRefGoogle Scholar
  21. 21.
    Liu, Y., Zhang, Q., Zhang, D., Han, J.: Employing deep part-object relationships for salient object detection. In: International Conference on Computer Vision (2019)Google Scholar
  22. 22.
    Luo, Z., Mishra, A., Achkar, A., Eichel, J., Li, S., Jodoin, P.M.: Non-local deep features for salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 6609–6617 (2017)Google Scholar
  23. 23.
    Neal, R.M.: MCMC using Hamiltonian dynamics. In: Handbook of Markov Chain Monte Carlo, vol. 54, pp. 113–162 (2010)Google Scholar
  24. 24.
    Nguyen, D.T., et al.: DeepUSPS: deep robust unsupervised saliency prediction with self-supervision. In: Advances in Neural Information Processing Systems (2019)Google Scholar
  25. 25.
    Qin, X., Zhang, Z., Huang, C., Gao, C., Dehghan, M., Jagersand, M.: BASNet: boundary-aware salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)Google Scholar
  26. 26.
    Reed, S.E., Lee, H., Anguelov, D., Szegedy, C., Erhan, D., Rabinovich, A.: Training deep neural networks on noisy labels with bootstrapping. In: International Conference on Learning Representations (2014)Google Scholar
  27. 27.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)Google Scholar
  28. 28.
    Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. In: Advances in Neural Information Processing Systems, pp. 3483–3491 (2015)Google Scholar
  29. 29.
    Tanno, R., Saeedi, A., Sankaranarayanan, S., Alexander, D.C., Silberman, N.: Learning from noisy labels by regularized estimation of annotator confusion. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)Google Scholar
  30. 30.
    Wang, B., Chen, Q., Zhou, M., Zhang, Z., Jin, X., Gai, K.: Progressive feature polishing network for salient object detection. In: AAAI Conference on Artificial Intelligence, pp. 12128–12135 (2020)Google Scholar
  31. 31.
    Wang, L., et al.: Learning to detect salient objects with image-level supervision. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 136–145 (2017)Google Scholar
  32. 32.
    Wang, T., et al.: Detect globally, refine locally: a novel approach to saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  33. 33.
    Wang, W., Shen, J., Dong, X., Borji, A.: Salient object detection driven by fixation prediction. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1711–1720 (2018)Google Scholar
  34. 34.
    Wang, W., Zhao, S., Shen, J., Hoi, S.C.H., Borji, A.: Salient object detection with pyramid attention and salient edges. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1448–1457 (2019)Google Scholar
  35. 35.
    Wang, W., Shen, J., Cheng, M.M., Shao, L.: An iterative and cooperative top-down and bottom-up inference network for salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)Google Scholar
  36. 36.
    Wang, W., Zhao, S., Shen, J., Hoi, S.C.H., Borji, A.: Salient object detection with pyramid attention and salient edges. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)Google Scholar
  37. 37.
    Wang, Y., Yang, Y., Yang, Z., Zhao, L., Wang, P., Xu, W.: Occlusion aware unsupervised learning of optical flow. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  38. 38.
    Wei, J., Wang, S., Huang, Q.: F3Net: fusion, feedback and focus for salient object detection. In: AAAI Conference on Artificial Intelligence (2020)Google Scholar
  39. 39.
    Wei, Y., Wen, F., Zhu, W., Sun, J.: Geodesic saliency using background priors. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7574, pp. 29–42. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33712-3_3CrossRefGoogle Scholar
  40. 40.
    Wu, R., Feng, M., Guan, W., Wang, D., Lu, H., Ding, E.: A mutual learning method for salient object detection with intertwined multi-supervision. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)Google Scholar
  41. 41.
    Wu, Z., Su, L., Huang, Q.: Cascaded partial decoder for fast and accurate salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)Google Scholar
  42. 42.
    Wu, Z., Su, L., Huang, Q.: Stacked cross refinement network for edge-aware salient object detection. In: International Conference on Computer Vision (2019)Google Scholar
  43. 43.
    Xie, J., Gao, R., Nijkamp, E., Zhu, S.C., Wu, Y.N.: Representation learning: a statistical perspective. Annu. Rev. Stat. Appl. 7, 303–335 (2020)MathSciNetCrossRefGoogle Scholar
  44. 44.
    Xie, J., Gao, R., Zheng, Z., Zhu, S.C., Wu, Y.N.: Learning dynamic generator model by alternating back-propagation through time. In: AAAI Conference on Artificial Intelligence, vol. 33, pp. 5498–5507 (2019)Google Scholar
  45. 45.
    Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1155–1162 (2013)Google Scholar
  46. 46.
    Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.: Saliency detection via graph-based manifold ranking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3166–3173 (2013)Google Scholar
  47. 47.
    Yi, K., Wu, J.: Probabilistic end-to-end noise correction for learning with noisy labels. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)Google Scholar
  48. 48.
    Zeng, Y., Zhuge, Y., Lu, H., Zhang, L., Qian, M., Yu, Y.: Multi-source weak supervision for saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)Google Scholar
  49. 49.
    Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning requires rethinking generalization. In: International Conference on Learning Representations (2017)Google Scholar
  50. 50.
    Zhang, D., Han, J., Zhang, Y.: Supervision by fusion: towards unsupervised learning of deep salient object detector. In: International Conference on Computer Vision (2017)Google Scholar
  51. 51.
    Zhang, J., Fan, D.P., Dai, Y., Anwar, S., Saleh, F.S., Zhang, T., Barnes, N.: UC-Net: uncertainty inspired RGB-D saliency detection via conditional variational autoencoders. In: IEEE Conference on Computer Vision and Pattern Recognition (2020)Google Scholar
  52. 52.
    Zhang, J., Yu, X., Li, A., Song, P., Liu, B., Dai, Y.: Weakly-supervised salient object detection via scribble annotations. In: IEEE Conference on Computer Vision and Pattern Recognition (2020)Google Scholar
  53. 53.
    Zhang, J., Zhang, T., Dai, Y., Harandi, M., Hartley, R.: Deep unsupervised saliency detection: a multiple noisy labeling perspective. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  54. 54.
    Zhang, P., Wang, D., Lu, H., Wang, H., Ruan, X.: Amulet: aggregating multi-level convolutional features for salient object detection. In: International Conference on Computer Vision (2017)Google Scholar
  55. 55.
    Zhang, X., Wang, T., Qi, J., Lu, H., Wang, G.: Progressive attention guided recurrent network for salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  56. 56.
    Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 294–310. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01234-2_18CrossRefGoogle Scholar
  57. 57.
    Zhao, J.X., Cao, Y., Fan, D.P., Cheng, M.M., Li, X.Y., Zhang, L.: Contrast prior and fluid pyramid integration for RGBD salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)Google Scholar
  58. 58.
    Zhao, J.X., Liu, J.J., Fan, D.P., Cao, Y., Yang, J., Cheng, M.M.: EGNet: edge guidance network for salient object detection. In: International Conference on Computer Vision (2019)Google Scholar
  59. 59.
    Zhu, W., Liang, S., Wei, Y., Sun, J.: Saliency optimization from robust background detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2814–2821 (2014)Google Scholar
  60. 60.
    Zhu, Y., Xie, J., Liu, B., Elgammal, A.: Learning feature-to-feature translator by alternating back-propagation for generative zero-shot learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 9844–9854 (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Australian National UniversityCanberraAustralia
  2. 2.Cognitive Computing LabBaidu ResearchSunnyvaleUSA
  3. 3.Australian Centre for Robotic VisionBrisbaneAustralia
  4. 4.Data61EveleighAustralia

Personalised recommendations