Advertisement

Weakly-Supervised Semantic Segmentation with Mean Teacher Learning

  • Li Tan
  • WenFeng Luo
  • Meng YangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11935)

Abstract

Weakly-supervised semantic segmentation with image-level labels is a important task as it directly associates high-level semantic to low-level appearance, which can significantly reduce human efforts. Despite the remarkable progress, it is still not as good as fully supervised segmentation methods. To improve the accuracy, in this paper, we proposed a novel framework of weakly-supervised semantic segmentation with mean teacher (WSSS-MT) learning to advance the class estimation of image pixels. More specifically, our proposed framework includes a student network and a teacher network in the segmentation module, which aims to effectively utilize information of the training process. The student learns the semantic segmentation network with an updated supervision, while the teacher uses the exponential moving average of the student to achieve a more accurate estimation of supervision. WSSS-MT employs the trained teacher as final segmentation network. Experimental results on the PASCAL VOC 2012 dataset show that the performance of our framework is better than the competing methods.

Keywords

Weakly-supervised learning Image semantic segmentation Deep learning 

Notes

Acknowledgement

This work is partially supported by the National Natural Science Foundation of China (Grant no. 61772568), the Guangzhou Science and Technology Program (Grant no. 201804010288), and the Fundamental Research Funds for the Central Universities (Grant no. 18lgzd15).

References

  1. 1.
    Kolesnikov, A., Lampert, C.H.: Seed, expand and constrain: three principles for weakly-supervised image segmentation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 695–711. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46493-0_42CrossRefGoogle Scholar
  2. 2.
    Wei, Y., Feng, J., Liang, X., Cheng, M.-M., Zhao, Y., Yan, S.: Object region mining with adversarial erasing: a simple classification to semantic segmentation approach (2017). arXiv preprint arXiv:1703.08448
  3. 3.
    Huang, Z., Wang, X., Wang, J., Liu, W., Wang, J.: Weakly-supervised semantic segmentation network with deep seeded region growing. In: CVPR, pp. 7014–7023 (2018)Google Scholar
  4. 4.
    Qi, X., Liu, Z., Shi, J., Zhao, H., Jia, J.: Augmented feedback in semantic segmentation under image level supervision. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 90–105. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46484-8_6CrossRefGoogle Scholar
  5. 5.
    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2017)Google Scholar
  6. 6.
    Everingham, M., Eslami, S.M.A., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The Pascal visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111(1), 98–136 (2015).  https://doi.org/10.1007/s11263-014-0733-5CrossRefGoogle Scholar
  7. 7.
    Hariharan, B., Arbelaez, P., Bourdev, L., Maji, S., Malik, J.: Semantic contours from inverse detectors. In: Proceedings of ICCV, pp. 991–998. IEEE (2011)Google Scholar
  8. 8.
    Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs (2016). arXiv preprint arXiv:1606.00915
  9. 9.
    Papandreou, G., Chen, L.-C., Murphy, K., Yuille, A.L.: Weakly- and semi-supervised learning of a DCNN for semantic image segmentation. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1742–1750 (2015)Google Scholar
  10. 10.
    Ahn, J., Kwak, S.: Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation (2018). arXiv preprint arXiv:1803.10464
  11. 11.
    Khoreva, A., Benenson, R., Hosang, J., Hein, M., Schiele, B.: Simple does it: weakly supervised instance and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 876–885 (2017)Google Scholar
  12. 12.
    Vernaza, P., Chandraker, M.: Learning random-walk label propagation for weakly-supervised semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017Google Scholar
  13. 13.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). arXiv preprint arXiv:1409.1556
  14. 14.
    Li, K., Wu, Z., Peng, K.-C., Ernst, J., Fu, Y.: Tell me where to look: guided attention inference network (2018). arXiv preprint arXiv:1802.10171
  15. 15.
    Bearman, A., Russakovsky, O., Ferrari, V., Fei-Fei, L.: What’s the point: semantic segmentation with point supervision. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 549–565. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46478-7_34CrossRefGoogle Scholar
  16. 16.
    Koltun, V.: Efficient inference in fully connected CRFs with Gaussian edge potentials. In: Proceedings of NIPS, vol. 2, no. 3, p. 4 (2011)Google Scholar
  17. 17.
    Adams, R., Bischof, L.: Seeded region growing. IEEE TPAMI 16(6), 641–647 (1994)CrossRefGoogle Scholar
  18. 18.
    Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of CVPR, pp. 2921–2929 (2016)Google Scholar
  19. 19.
    Wei, Y., et al.: STC: a simple to complex framework for weakly-supervised semantic segmentation. IEEE Trans. Pattern Recogn. Mach. Intell. 39(11), 2314–2320 (2017)CrossRefGoogle Scholar
  20. 20.
    Hong, S., Oh, J., Han, B., Lee, H.: Learning transferrable knowledge for semantic segmentation with deep convolutional neural network. In: IEEE CVPR (2016)Google Scholar
  21. 21.
    Zheng, S., et al.: Conditional random fields as recurrent neural networks. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1529–1537 (2015)Google Scholar
  22. 22.
    Shimoda, W., Yanai, K.: Distinct class-specific saliency maps for weakly supervised semantic segmentation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 218–234. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46493-0_14CrossRefGoogle Scholar
  23. 23.
    Wang, X., You, S., Li, X., Ma, H.: Weakly-supervised semantic segmentation by iteratively mining common object features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1354–1362 (2018)Google Scholar
  24. 24.
    Pathak, D., Shelhamer, E., Long, J., Darrell, T.: Fully convolutional multi-class multiple instance learning (2014). arXiv preprint arXiv:1412.7144
  25. 25.
    Bachman, P., Alsharif, O., Precup, D.: Learning with Pseudo-Ensembles, December 2014. arXiv:1412.4864 [cs, stat]
  26. 26.
    Jiang, H., Wang, J., Yuan, Z., Wu, Y., Zheng, N., Li, S.: Salient object detection: a discriminative regional feature integration approach. In: Proceedings of CVPR, pp. 2083–2090 (2013)Google Scholar
  27. 27.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of CVPR, pp. 3431–3440 (2015)Google Scholar
  28. 28.
    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Data and Computer ScienceSun Yat-Sen UniversityGuangzhouChina

Personalised recommendations