Abstract
Deep neural networks have increased the accuracy of automatic segmentation, however their accuracy depends on the availability of a large number of fully segmented images. Methods to train deep neural networks using images for which some, but not all, regions of interest are segmented are necessary to make better use of partially annotated datasets. In this paper, we propose the first axiomatic definition of label-set loss functions that are the loss functions that can handle partially segmented images. We prove that there is one and only one method to convert a classical loss function for fully segmented images into a proper label-set loss function. Our theory also allows us to define the leaf-Dice loss, a label-set generalisation of the Dice loss particularly suited for partial supervision with only missing labels. Using the leaf-Dice loss, we set a new state of the art in partially supervised learning for fetal brain 3D MRI segmentation. We achieve a deep neural network able to segment white matter, ventricles, cerebellum, extra-ventricular CSF, cortical gray matter, deep gray matter, brainstem, and corpus callosum based on fetal brain 3D MRI of anatomically normal fetuses or with open spina bifida. Our implementation of the proposed label-set loss functions is available at https://github.com/LucasFidon/label-set-loss-functions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aertsen, M., et al.: Reliability of MR imaging-based posterior fossa and brain stem measurements in open spinal Dysraphism in the era of fetal surgery. Am. J. Neuroradiol. 40(1), 191–198 (2019)
Benkarim, O.M., et al.: Toward the automatic quantification of in utero brain development in 3D structural MRI: a review. Hum. Brain Mapp. 38(5), 2772–2787 (2017)
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49
Danzer, E., Joyeux, L., Flake, A.W., Deprest, J.: Fetal surgical intervention for myelomeningocele: lessons learned, outcomes, and future implications. Dev. Med. Child Neurol. 62(4), 417–425 (2020)
Dmitriev, K., Kaufman, A.E.: Learning multi-class segmentations from single-class datasets. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9501–9511 (2019)
Dorent, R., et al.: Learning joint segmentation of tissues and brain lesions from task-specific hetero-modal domain-shifted datasets. Med. Image Anal. 67, 101862 (2021)
Ebner, M., et al.: An automated framework for localization, segmentation and super-resolution reconstruction of fetal brain MRI. NeuroImage 206, 116324 (2020)
Fang, X., Yan, P.: Multi-organ segmentation over partially labeled datasets with multi-scale feature abstraction. IEEE Trans. Med. Imaging 39(11), 3619–3629 (2020)
Fetit, A.E., et al.: A deep learning approach to segmentation of the developing cortex in fetal brain mri with minimal manual labeling. In: Medical Imaging with Deep Learning, pp. 241–261. PMLR (2020)
Fidon, L., et al.: Generalised Wasserstein Dice score for imbalanced multi-class segmentation using holistic convolutional networks. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 64–76. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75238-9_6
Fidon, L., et al.: A spatio-temporal atlas of the developing fetal brain with spina bifida aperta. Open Research Europe (2021)
Gholipour, A., et al.: A normative spatiotemporal MRI atlas of the fetal brain for automatic segmentation and analysis of early brain growth. Sci. Rep. 7(1), 1–13 (2017)
Khalili, N., et al.: Automatic brain tissue segmentation in fetal MRI using convolutional neural networks. Magnet. Resonan. Imaging 64, 77–89 (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Milletari, F., Navab, N., Ahmadi, S.A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. IEEE (2016)
Moise Jr., K.J., et al.: Current selection criteria and perioperative therapy used for fetal myelomeningocele surgery. Obstetrics Gynecol. 127(3), 593–597 (2016)
Payette, K., et al.: A comparison of automatic multi-tissue segmentation methods of the human fetal brain using the FeTA dataset. arXiv preprint arXiv:2010.15526 (2020)
Payette, K., Kottke, R., Jakab, A.: Efficient multi-class fetal brain segmentation in high resolution MRI reconstructions with noisy labels. In: Hu, Y., et al. (eds.) ASMUS/PIPPI -2020. LNCS, vol. 12437, pp. 295–304. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60334-2_29
Payette, K., et al.: Longitudinal analysis of fetal MRI in patients with prenatal spina bifida repair. In: Wang, Q., et al. (eds.) PIPPI/SUSI -2019. LNCS, vol. 11798, pp. 161–170. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32875-7_18
Ranzini, M., Fidon, L., Ourselin, S., Modat, M., Vercauteren, T.: MONAIfbs: MONAI-based fetal brain MRI deep learning segmentation. arXiv preprint arXiv:2103.13314 (2021)
Roulet, N., Slezak, D.F., Ferrante, E.: Joint learning of brain lesion and anatomy segmentation from heterogeneous datasets. In: International Conference on Medical Imaging with Deep Learning, pp. 401–413. PMLR (2019)
Sacco, A., et al.: Fetal surgery for open spina bifida. Obstetrician Gynaecologist 21(4), 271 (2019)
Shi, G., Xiao, L., Chen, Y., Zhou, S.K.: Marginal loss and exclusion loss for partially supervised multi-organ segmentation. Med. Image Anal. 101979 (2021)
Ulyanov, D., Vedaldi, A., Lempitsky, V.: Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016)
Zarutskie, A., et al.: Prenatal brain imaging for predicting need for postnatal hydrocephalus treatment in fetuses that had neural tube defect repair in utero. Ultrasound Obstetrics Gynecol. 53(3), 324–334 (2019)
Zhou, Y., et al.: Prior-aware neural network for partially-supervised multi-organ segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10672–10681 (2019)
Acknowledgments
This project has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement TRABIT No 765148. This work was supported by core and project funding from the Wellcome [203148/Z/16/Z; 203145Z/16/Z; WT101957], and EPSRC [NS/A000049/1; NS/A000050/1; NS/A000027/1]. TV is supported by a Medtronic / RAEng Research Chair [RCSRF1819\(\backslash \)7\(\backslash \)34].
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Fidon, L. et al. (2021). Label-Set Loss Functions for Partial Supervision: Application to Fetal Brain 3D MRI Parcellation. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12902. Springer, Cham. https://doi.org/10.1007/978-3-030-87196-3_60
Download citation
DOI: https://doi.org/10.1007/978-3-030-87196-3_60
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87195-6
Online ISBN: 978-3-030-87196-3
eBook Packages: Computer ScienceComputer Science (R0)
