3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation

  • Özgün Çiçek
  • Ahmed Abdulkadir
  • Soeren S. Lienkamp
  • Thomas Brox
  • Olaf Ronneberger
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9901)

Abstract

This paper introduces a network for volumetric segmentation that learns from sparsely annotated volumetric images. We outline two attractive use cases of this method: (1) In a semi-automated setup, the user annotates some slices in the volume to be segmented. The network learns from these sparse annotations and provides a dense 3D segmentation. (2) In a fully-automated setup, we assume that a representative, sparsely annotated training set exists. Trained on this data set, the network densely segments new volumetric images. The proposed network extends the previous u-net architecture from Ronneberger et al. by replacing all 2D operations with their 3D counterparts. The implementation performs on-the-fly elastic deformations for efficient data augmentation during training. It is trained end-to-end from scratch, i.e., no pre-trained network is required. We test the performance of the proposed method on a complex, highly variable 3D structure, the Xenopus kidney, and achieve good results for both use cases.

Keywords

Convolutional neural networks 3D Biomedical volumetric image segmentation Xenopus kidney Semi-automated Fully-automated Sparse annotation 

References

  1. 1.
    Emmenlauer, M., Ronneberger, O., Ponti, A., Schwarb, P., Griffa, A., Filippi, A., Nitschke, R., Driever, W., Burkhardt, H.: Xuvtools: free, fast and reliable stitching of large 3D datasets. J. Microscopy 233(1), 42–60 (2009)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Fedorov, A., Beichel, R., Kalpathy-Cramer, J., Finet, J., Fillion-Robin, J.C., Pujol, S., Bauer, C., Jennings, D., Fennessy, F., Sonka, M., et al.: 3D slicer as an image computing platform for the quantitative imaging network. J. Magn. Reson. Imaging 30(9), 1323–1341 (2012)CrossRefGoogle Scholar
  3. 3.
    Hariharan, B., Arbeláez, P., Girshick, R., Malik, J.: Hypercolumns for object segmentation and fine-grained localization. In: Proceeding CVPR, pp. 447–456 (2015)Google Scholar
  4. 4.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. CoRR abs/1502.03167 (2015)Google Scholar
  5. 5.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. In: Proceeding ACMMM, pp. 675–678 (2014)Google Scholar
  6. 6.
    Kleesiek, J., Urban, G., Hubert, A., Schwarz, D., Maier-Hein, K., Bendszus, M., Biller, A.: Deep mri brain extraction: a 3D convolutional neural network for skull stripping. NeuroImage (2016)Google Scholar
  7. 7.
    Lienkamp, S., Ganner, A., Boehlke, C., Schmidt, T., Arnold, S.J., Schäfer, T., Romaker, D., Schuler, J., Hoff, S., Powelske, C., Eifler, A., Krönig, C., Bullerkotte, A., Nitschke, R., Kuehn, E.W., Kim, E., Burkhardt, H., Brox, T., Ronneberger, O., Gloy, J., Walz, G.: Inversin relays frizzled-8 signals to promote proximal pronephros development. PNAS 107(47), 20388–20393 (2010)CrossRefGoogle Scholar
  8. 8.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceeding CVPR, pp. 3431–3440 (2015)Google Scholar
  9. 9.
    Milletari, F., Ahmadi, S., Kroll, C., Plate, A., Rozanski, V.E., Maiostre, J., Levin, J., Dietrich, O., Ertl-Wagner, B., Bötzel, K., Navab, N.: Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound. CoRR abs/1601.07014 (2016)Google Scholar
  10. 10.
    Nieuwkoop, P., Faber, J.: Normal Table of Xenopus laevis (Daudin). Garland, New York (1994)Google Scholar
  11. 11.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Heidelberg (2015). doi:10.1007/978-3-319-24574-4_28 CrossRefGoogle Scholar
  12. 12.
    Seyedhosseini, M., Sajjadi, M., Tasdizen, T.: Image segmentation with cascaded hierarchical models and logistic disjunctive normal networks. In: Proceeding ICCV, pp. 2168–2175 (2013)Google Scholar
  13. 13.
    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. CoRR abs/1512.00567 (2015)Google Scholar
  14. 14.
    Tran, D., Bourdev, L.D., Fergus, R., Torresani, L., Paluri, M.: Deep end2end voxel2voxel prediction. CoRR abs/1511.06681 (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Özgün Çiçek
    • 1
    • 2
  • Ahmed Abdulkadir
    • 1
    • 4
  • Soeren S. Lienkamp
    • 2
    • 3
  • Thomas Brox
    • 1
    • 2
  • Olaf Ronneberger
    • 1
    • 2
    • 5
  1. 1.Computer Science DepartmentUniversity of FreiburgFreiburgGermany
  2. 2.BIOSS Centre for Biological Signalling StudiesFreiburgGermany
  3. 3.University Hospital Freiburg, Renal Division, Faculty of MedicineUniversity of FreiburgFreiburgGermany
  4. 4.Department of Psychiatry and PsychotherapyUniversity Medical Center FreiburgFreiburgGermany
  5. 5.Google DeepMindLondonUK

Personalised recommendations