Abstract
Automated medical image segmentation using deep neural networks typically requires substantial supervised training. However, these models fail to generalize well across different imaging modalities. This shortcoming, amplified by the limited availability of expert annotated data, has been hampering the deployment of such methods at a larger scale across modalities. To address these issues, we propose M-GenSeg, a new semi-supervised generative training strategy for cross-modality tumor segmentation on unpaired bi-modal datasets. With the addition of known healthy images, an unsupervised objective encourages the model to disentangling tumors from the background, which parallels the segmentation task. Then, by teaching the model to convert images across modalities, we leverage available pixel-level annotations from the source modality to enable segmentation in the unannotated target modality. We evaluated the performance on a brain tumor segmentation dataset composed of four different contrast sequences from the public BraTS 2020 challenge data. We report consistent improvement in Dice scores over state-of-the-art domain-adaptive baselines on the unannotated target modality. Unlike the prior art, M-GenSeg also introduces the ability to train with a partially annotated source modality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Prevedello, L.M., et al.: Challenges related to artificial intelligence research in medical imaging and the importance of image analysis competitions. Radiol. Artif. Intell. 1(1), e180031 (2019)
Dorent, R., et al.: CrossMoDA 2021 challenge: benchmark of cross-modality domain adaptation techniques for vestibular schwannoma and cochlea segmentation. Med. Image Anal. 83, 102628 (2023)
Billot, B., et al.: SynthSeg: domain randomisation for segmentation of brain scans of any contrast and resolution (2021)
Guan, H., Liu, M.: Domain adaptation for medical image analysis: a survey. IEEE Trans. Biomed. Eng. 3, 1173–1185 (2022)
Pei, C., Wu, F., Huang, L.: Disentangle domain features for cross-modality cardiac image segmentation. Med. Image Anal. 71, 102078 (2021)
Li, C., Luo, X., Chen, W., He, Y. Wu, M., Tan, Y.: AttENT: domain-adaptive medical image segmentation via attention-aware translation and adversarial entropy minimization. In: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 952–959. IEEE (2021)
Huo, Y., et al.: SynSeg-Net: synthetic segmentation without target modality ground truth. IEEE Trans. Med. Imaging 38(4), 1016–1025 (2019)
Chen, C., Dou, Q., Chen, H., Qin, J., Heng, P.-A.: Synergistic image and feature adaptation: towards cross-modality domain adaptation for medical image segmentation, vol. 38, pp. 865–872 (2019)
Jiang, J., et al.: Self-derived organ attention for unpaired CT-MRI deep domain adaptation based MRI segmentation. Phys. Med. Biol. 65(20), 205001 (2020)
Zhou, B., Liu, C., Duncan, J.S.: Anatomy-constrained contrastive learning for synthetic segmentation without ground-truth. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 47–56. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_5
Hoffman, J., et al.: CyCADA: cycle-consistent adversarial domain adaptation (2017)
Zhang, Y., Miao, S., Mansi, T., Liao, R.: Task driven generative modeling for unsupervised domain adaptation: application to X-ray image segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 599–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_67
Zhu, J.-Y., Park, T., Isola, P. Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–51. IEEE (2017)
Vorontsov, E., Molchanov, P., Gazda, M., Beckham, C., Kautz, J., Kadoury, S.: Towards annotation-efficient segmentation via image-to-image translation. Med. Image Anal. 82, 102624 (2022)
Liu, X., Niethammer, M., Kwitt, R., Singh, N., McCormick, M., Aylward, S.: Low-rank atlas image analyses in the presence of pathologies. IEEE Trans. Med. Imaging 34(12), 2583–2591 (2015)
Lin, C., Wang, Y., Wang, T., Ni, D.: Low-rank based image analyses for pathological MR image segmentation and recovery. Front. Neurosci. 13(13), 333 (2019)
Changfa, S., Min, X., Xiancheng, Z., Haotian, W., Heng-Da, C.: Multi-slice low-rank tensor decomposition based multi-atlas segmentation: application to automatic pathological liver CT segmentation. Med. Image Anal. 73, 102152 (2021)
Shin, H.-C., et al.: Medical image synthesis for data augmentation and anonymization using generative adversarial networks. In: Gooya, A., Goksel, O., Oguz, I., Burgos, N. (eds.) SASHIMI 2018. LNCS, vol. 11037, pp. 1–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00536-8_1
Mok, T.C.W., Chung, A.C.S.: Learning data augmentation for brain tumor segmentation with coarse-to-fine generative adversarial networks. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11383, pp. 70–80. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11723-8_7
Kim, S., Kim, B., Park, H.: Synthesis of brain tumor multicontrast MR images for improved data augmentation. Med. Phys. 48(5), 2185–2198 (2021)
Drozdzal, M., Vorontsov, E., Chartrand, G., Kadoury, S., Pal, C.: The importance of skip connections in biomedical image segmentation. In: Carneiro, G., et al. (eds.) LABELS/DLMIA 2016. LNCS, vol. 10008, pp. 179–187. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46976-8_19
Oktay, O., et al.: Attention U-Net: learning where to look for the pancreas (2018)
Bakas, S., Reyes, M., Jakab, A., Bauer, S.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge (2018)
Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 4(1), 170117 (2017)
Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015)
Yuan, W., Wei, J., Wang, J., Ma, Q., Tasdizen, T.: Unified attentional generative adversarial network for brain tumor segmentation from multimodal unpaired images. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 229–237. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_26
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Alefsen, M., Vorontsov, E., Kadoury, S. (2023). M-GenSeg: Domain Adaptation for Target Modality Tumor Segmentation with Annotation-Efficient Supervision. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14223. Springer, Cham. https://doi.org/10.1007/978-3-031-43901-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-43901-8_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43900-1
Online ISBN: 978-3-031-43901-8
eBook Packages: Computer ScienceComputer Science (R0)
