Semi-supervised Medical Image Segmentation via Learning Consistency Under Transformations

  • Gerda BortsovaEmail author
  • Florian Dubost
  • Laurens Hogeweg
  • Ioannis Katramados
  • Marleen de Bruijne
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11769)


The scarcity of labeled data often limits the application of supervised deep learning techniques for medical image segmentation. This has motivated the development of semi-supervised techniques that learn from a mixture of labeled and unlabeled images. In this paper, we propose a novel semi-supervised method that, in addition to supervised learning on labeled training images, learns to predict segmentations consistent under a given class of transformations on both labeled and unlabeled images. More specifically, in this work we explore learning equivariance to elastic deformations. We implement this through: (1) a Siamese architecture with two identical branches, each of which receives a differently transformed image, and (2) a composite loss function with a supervised segmentation loss term and an unsupervised term that encourages segmentation consistency between the predictions of the two branches. We evaluate the method on a public dataset of chest radiographs with segmentations of anatomical structures using 5-fold cross-validation. The proposed method reaches significantly higher segmentation accuracy compared to supervised learning. This is due to learning transformation consistency on both labeled and unlabeled images, with the latter contributing the most. We achieve the performance comparable to state-of-the-art chest X-ray segmentation methods while using substantially fewer labeled images.


Semi-supervised learning Segmentation Chest X-ray 



This research is part of the research project Deep Learning for Medical Image Analysis (DLMedIA) with project number P15-26, funded by the Netherlands Organisation for Scientific Research (NWO). The computations were carried out on the Dutch national e-infrastructure with the support of SURF Cooperative.


  1. 1.
    Baur, C., Albarqouni, S., Navab, N.: Semi-supervised deep learning for fully convolutional networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 311–319. Springer, Cham (2017). Scholar
  2. 2.
    Cheplygina, V., de Bruijne, M., Pluim, J.P.: Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Med. Image Anal. 54, 280–296 (2019)CrossRefGoogle Scholar
  3. 3.
    Dai, W., Dong, N., Wang, Z., Liang, X., Zhang, H., Xing, E.P.: SCAN: structure correcting adversarial network for organ segmentation in chest X-rays. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS-2018. LNCS, vol. 11045, pp. 263–273. Springer, Cham (2018). Scholar
  4. 4.
    Frid-Adar, M., Ben-Cohen, A., Amer, R., Greenspan, H.: Improving the segmentation of anatomical structures in chest radiographs using U-Net with an ImageNet pre-trained encoder. In: Stoyanov, D., et al. (eds.) RAMBO/BIA/TIA -2018. LNCS, vol. 11040, pp. 159–168. Springer, Cham (2018). Scholar
  5. 5.
    Hervella, Á.S., Rouco, J., Novo, J., Ortega, M.: Retinal image understanding emerges from self-supervised multimodal reconstruction. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 321–328. Springer, Cham (2018). Scholar
  6. 6.
    Honari, S., Molchanov, P., Tyree, S., Vincent, P., Pal, C., Kautz, J.: Improving landmark localization with semi-supervised learning. In: CVPR, pp. 1546–1555 (2018)Google Scholar
  7. 7.
    Li, X., Yu, L., Chen, H., Fu, C.W., Heng, P.A.: Semi-supervised skin lesion segmentation via transformation consistent self-ensembling model. In: BMVC (2018)Google Scholar
  8. 8.
    Novikov, A.A., Lenis, D., Major, D., Hladuvka, J., Wimmer, M., Buehler, K.: Fully convolutional architectures for multiclass segmentation in chest radiographs. TMI 37(8), 1865–1876 (2018)Google Scholar
  9. 9.
    Perone, C.S., Cohen-Adad, J.: Deep semi-supervised segmentation with weight-averaged consistency targets. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 12–19. Springer, Cham (2018). Scholar
  10. 10.
    Rasmus, A., Berglund, M., Honkala, M., Valpola, H., Raiko, T.: Semi-supervised learning with ladder networks. In: NIPS, pp. 3546–3554 (2015)Google Scholar
  11. 11.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  12. 12.
    Ross, T., et al.: Exploiting the potential of unlabeled endoscopic video data with self-supervised learning. IJCARS 13, 925–933 (2018)Google Scholar
  13. 13.
    Sajjadi, M., Javanmardi, M., Tasdizen, T.: Regularization with stochastic transformations and perturbations for deep semi-supervised learning. In: NIPS, pp. 1163–1171 (2016)Google Scholar
  14. 14.
    Shah, M.P., Merchant, S.N., Awate, S.P.: MS-Net: mixed-supervision fully-convolutional networks for full-resolution segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 379–387. Springer, Cham (2018). Scholar
  15. 15.
    Shiraishi, J., et al.: Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. Am. J. Roentgenol. 174(1), 71–74 (2000)CrossRefGoogle Scholar
  16. 16.
    Van Ginneken, B., Stegmann, M.B., Loog, M.: Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database. MedIA 10(1), 19–40 (2006)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Gerda Bortsova
    • 1
    Email author
  • Florian Dubost
    • 1
  • Laurens Hogeweg
    • 2
  • Ioannis Katramados
    • 2
  • Marleen de Bruijne
    • 1
    • 3
  1. 1.Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear MedicineErasmus MCRotterdamThe Netherlands
  2. 2.COSMONiOGroningenThe Netherlands
  3. 3.Department of Computer ScienceUniversity of CopenhagenCopenhagenDenmark

Personalised recommendations