Advertisement

ISDNet: Importance Guided Semi-supervised Adversarial Learning for Medical Image Segmentation

  • Qingtian Ning
  • Xu ZhaoEmail author
  • Dahong Qian
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11902)

Abstract

Recent deep neural networks have achieved great success in medical image segmentation. However, massive labeled training data should be provided during network training, which is time consuming with intensive labor work and even requires expertise knowledge. To address such challenge, inspired by typical GANs, we propose a novel end-to-end semi-supervised adversarial learning framework for medical image segmentation, called “Importance guided Semi-supervised Deep Networks” (ISDNet). While most existing works based on GANs use a classifier discriminator to achieve adversarial learning, we combine a fully convolutional discriminator and a classifier discriminator to fulfill better adversarial learning and self-taught learning. Specifically, we propose an importance weight network combined with our FCN-based confidence network, which can assist segmentation network to learn better local and global information. Extensive experiments are conducted on the LASC 2013 and the LiTS 2017 datasets to demonstrate the effectiveness of our approach.

Keywords

Medical image segmentation Semi-supervised GAN 

Notes

Acknowledgement

This work is supported by: National Natural Science Foundation of China (61673269, 61273285).

References

  1. 1.
    Lin, D., Dai, J., Jia, J., et al.: ScribbleSup: scribble-supervised convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3159–3167 (2016)Google Scholar
  2. 2.
    Brock, A., Donahue, J., Simonyan, K.: Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096 (2018)
  3. 3.
    Wu, Y., He, K.: Group normalization. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 3–19. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01261-8_1CrossRefGoogle Scholar
  4. 4.
    Tobon-Gomez, C., Geers, A.J., Peters, J., et al.: Benchmark for algorithms segmenting the left atrium from 3D CT and MRI datasets. IEEE Trans. Med. Imaging 34(7), 1460–1473 (2015)CrossRefGoogle Scholar
  5. 5.
    Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01234-2_49CrossRefGoogle Scholar
  6. 6.
    Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
  7. 7.
    Nie, D., Gao, Y., Wang, L., Shen, D.: ASDNet: attention based semi-supervised deep networks for medical image segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 370–378. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00937-3_43CrossRefGoogle Scholar
  8. 8.
    Zhang, J., Ding, Z., Li, W., et al.: Importance weighted adversarial nets for partial domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8156–8164 (2018)Google Scholar
  9. 9.
    Frid-Adar, M., Diamant, I., Klang, E., et al.: GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321, 321–331 (2018)CrossRefGoogle Scholar
  10. 10.
    Hung, W.C., Tsai, Y.H., Liou, Y.T., et al.: Adversarial learning for semi-supervised semantic segmentation. arXiv preprint arXiv:1802.07934 (2018)
  11. 11.
    Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv preprint arXiv:1701.07875 (2017)
  12. 12.
    Gulrajani, I., Ahmed, F., Arjovsky, M., et al.: Improved training of Wasserstein GANS. In: Advances in Neural Information Processing Systems, pp. 5767–5777 (2017)Google Scholar
  13. 13.
    Zhu, Y., Elhoseiny, M., Liu, B., et al.: A generative adversarial approach for zero-shot learning from noisy texts. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1004–1013 (2018)Google Scholar
  14. 14.
    Lai, W.S., Huang, J.B., Ahuja, N., et al.: Deep Laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 624–632 (2017)Google Scholar
  15. 15.
    Hu, L., Kan, M., Shan, S., et al.: Duplex generative adversarial network for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1498–1507 (2018)Google Scholar
  16. 16.
    Dai, J., He, K., Sun, J.: BoxSup: exploiting bounding boxes to supervise convolutional networks for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1635–1643 (2015)Google Scholar
  17. 17.
    Oh, S.J., Benenson, R., Khoreva, A., et al.: Exploiting saliency for object segmentation from image level labels. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5038–5047. IEEE (2017)Google Scholar
  18. 18.
    Pathak, D., Krahenbuhl, P., Darrell, T.: Constrained convolutional neural networks for weakly supervised segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1796–1804 (2015)Google Scholar
  19. 19.
    Bearman, A., Russakovsky, O., Ferrari, V., Fei-Fei, L.: What’s the point: semantic segmentation with point supervision. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 549–565. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46478-7_34CrossRefGoogle Scholar
  20. 20.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Google Scholar
  21. 21.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  22. 22.
    Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  23. 23.
    Luc, P., Couprie, C., Chintala, S., et al.: Semantic segmentation using adversarial networks. arXiv preprint arXiv:1611.08408 (2016)
  24. 24.
    Sandoval, Z., Betancur, J., Dillenseger, J.-L.: Multi-atlas-based segmentation of the left atrium and pulmonary veins. In: Camara, O., Mansi, T., Pop, M., Rhode, K., Sermesant, M., Young, A. (eds.) STACOM 2013. LNCS, vol. 8330, pp. 24–30. Springer, Heidelberg (2014).  https://doi.org/10.1007/978-3-642-54268-8_3CrossRefGoogle Scholar
  25. 25.
    Zuluaga, M.A., Cardoso, M.J., Modat, M., Ourselin, S.: Multi-atlas propagation whole heart segmentation from MRI and CTA using a local normalised correlation coefficient criterion. In: Ourselin, S., Rueckert, D., Smith, N. (eds.) FIMH 2013. LNCS, vol. 7945, pp. 174–181. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-38899-6_21CrossRefGoogle Scholar
  26. 26.
    Liu, X., Shen, Y., Zhang, S., et al.: Segmentation of left atrium through combination of deep convolutional and recurrent neural networks. J. Med. Imaging Health Inform. 8(8), 1578–1584 (2018)CrossRefGoogle Scholar
  27. 27.
    Mortazi, A., Karim, R., Rhode, K., Burt, J., Bagci, U.: CardiacNET: segmentation of left atrium and proximal pulmonary veins from MRI using multi-view CNN. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 377–385. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-66185-8_43CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of AutomationShanghai Jiao Tong UniversityShanghaiChina
  2. 2.Institute of Medical RoboticsShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations