Abstract
Generalising deep models to new data from new centres (termed here domains) remains a challenge. This is largely attributed to shifts in data statistics (domain shifts) between source and unseen domains. Recently, gradient-based meta-learning approaches where the training data are split into meta-train and meta-test sets to simulate and handle the domain shifts during training have shown improved generalisation performance. However, the current fully supervised meta-learning approaches are not scalable for medical image segmentation, where large effort is required to create pixel-wise annotations. Meanwhile, in a low data regime, the simulated domain shifts may not approximate the true domain shifts well across source and unseen domains. To address this problem, we propose a novel semi-supervised meta-learning framework with disentanglement. We explicitly model the representations related to domain shifts. Disentangling the representations and combining them to reconstruct the input image allows unlabeled data to be used to better approximate the true domain shifts for meta-learning. Hence, the model can achieve better generalisation performance, especially when there is a limited amount of labeled data. Experiments show that the proposed method is robust on different segmentation tasks and achieves state-of-the-art generalisation performance on two public benchmarks. Code is publicly available at: https://github.com/vios-s/DGNet.
Keywords
- Domain generalisation
- Disentanglement
- Medical image segmentation
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
With the exception of [41] which clusters unlabeled data to generate pseudo labels, but unfortunately is not applicable to segmentation.
References
Achille, A., Soatto, S.: Emergence of invariance and disentanglement in deep representations. JMLR 19(1), 1947–1980 (2018)
Antoniou, A., Edwards, H., Storkey, A.: How to train your MAML. In: Proceedings of the ICLR (2019)
Bernard, O., Lalande, A., Zotti, C., Cervenansky, F., Yang, X., et al.: Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE TMI 37(11), 2514–2525 (2018)
Bian, C., Yuan, C., Wang, J., Li, M., et al.: Uncertainty-aware domain alignment for anatomical structure segmentation. MedIA 64, 101732 (2020)
Campello, V.M., Gkontra, P., Izquierdo, C., et al.: Multi-centre, multi-vendor and multi-disease cardiac segmentation: the M&MS challenge. IEEE Trans. Med. Imag. (2021)
Carlucci, F.M., D’Innocente, A., Bucci, S., Caputo, B., Tommasi, T.: Domain generalisation by solving jigsaw puzzles. In: Proceedings of the CVPR, pp. 2229–2238 (2019)
Chartsias, A., Joyce, T., Papanastasiou, G., et al.: Disentangled representation learning in cardiac image analysis. Med. Image Anal. 58, 101535 (2019)
Chen, C., Qin, C., Qiu, H., et al.: Deep learning for cardiac image segmentation: a review. Front. Cardiovasc. Med. 7(25), 1–33 (2020)
Dewey, B.E., et al.: A disentangled latent space for cross-site MRI harmonization. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12267, pp. 720–729. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59728-3_70
Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)
Dou, Q., Castro, D.C., Kamnitsas, K., Glocker, B.: Domain generalisation via model-agnostic learning of semantic features. Proc, NeurIPS (2019)
Dubuisson, M.P., Jain, A.K.: A modified hausdorff distance for object matching. In: Proceedings of the ICPR, vol. 1, pp. 566–568. IEEE (1994)
Higgins, I., Matthey, L., Pal, A., et al.: beta-VAE: learning basic visual concepts with a constrained variational framework In: Proceedings of the ICLR (2016)
Huang, J., Guan, D., Xiao, A., Lu, S.: FSDR: frequency space domain randomization for domain generalization. In: Proceedings of the CVPR (2021)
Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the ICCV, pp. 1501–1510 (2017)
Huang, X., Liu, M.Y., Belongie, S., Kautz, J.: Multimodal unsupervised image-to-image translation. In: Proceedings of the ECCV, pp. 172–189 (2018)
Ilse, M., Tomczak, J.M., Louizos, C., Welling, M.: Diva: Domain invariant variational autoencoders. In: Proceedings of the MIDL, pp. 322–348. PMLR (2020)
Isensee, F., et al.: Automatic cardiac disease assessment on cine-MRI via time-series segmentation and domain specific features. In: Pop, M., et al. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 120–129. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75541-0_13
Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: NNU-net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021)
Khandelwal, P., Yushkevich, P.: Domain generalizer: a few-shot meta learning framework for domain generalization in medical imaging. In: Albarqouni, S., et al. (eds.) DART/DCL -2020. LNCS, vol. 12444, pp. 73–84. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60548-3_8
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the ICLR (2015)
Li, D., Yang, Y., Song, Y.Z., Hospedales, T.: Learning to generalise: meta-learning for domain generalisation. In: Proceedings of the AAAI (2018)
Li, D., Zhang, J., Yang, Y., Liu, C., Song, Y.Z., Hospedales, T.M.: Episodic training for domain generalisation. In: Proceedings of the ICCV, pp. 1446–1455 (2019)
Li, H., Pan, S.J., Wang, S., Kot, A.C.: Domain generalization with adversarial feature learning. In: Proceedings of the CVPR, pp. 5400–5409 (2018)
Li, H., Wang, Y., Wan, R., et al.: Domain generalisation for medical imaging classification with linear-dependency regularization. In: Proceedings of the NeurIPS (2020)
Li, X., et al.: Difficulty-aware meta-learning for rare disease diagnosis. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 357–366. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_35
Li, Y., Tian, X., Gong, M., et al.: Deep domain generalization via conditional invariant adversarial networks. In: Proceedings of the ECCV, pp. 624–639 (2018)
Li, Y., Chen, J., Xie, X., Ma, K., Zheng, Y.: Self-loop uncertainty: a novel pseudo-label for semi-supervised medical image segmentation. In: Proceedings of the MICCAI, pp. 614–623. Springer, Cham (2020)
Liu, Q., Dou, Q., Heng, P.A.: Shape-aware meta-learning for generalising prostate MRI segmentation to unseen domains. In: Proceedings of the MICCAI, pp. 475–485. Springer, Cham (2020)
Liu, X., Thermos, S., Chartsias, A., et al.: Disentangled representations for domain-generalised cardiac segmentation. In: International Workshop on STACOM (2020)
Liu, X., Thermos, S., Valvano, G., et al.: Metrics for exposing the biases of content-style disentanglement. arXiv preprint arXiv:2008.12378 (2020)
Llera Montero, M., Ludwig, C.J.H., Ponte Costa, R., Malhotra, G., Bowers, J.: The role of disentanglement in generalisation. In: Proceedings of the ICLR (2021)
Locatello, F., et al.: Challenging common assumptions in the unsupervised learning of disentangled representations. In: Proceeding of the ICML, pp. 4114–4124. PMLR (2019)
Ma, W.D.K., Lewis, J., Kleijn, W.B.: The HSIC Bottleneck: Deep Learning without Back-Propagation. In: Proceedings of the AAAI, pp. 5085–5092 (2020)
Muandet, K., Balduzzi, D., Schölkopf, B.: Domain generalisation via invariant feature representation. In: Proceedings of the ICML, pp. 10–18. PMLR (2013)
Paszke, A., Gross, S., Massa, F., Lerer, A., et. al.: PyTorch: an imperative style, high-performance deep learning library. In: Proceedings of the NeurIPS, pp. 8026–8037 (2019)
Pomponio, R., Erus, G., et al.: Harmonization of large MRI datasets for the analysis of brain imaging patterns throughout the lifespan. NeuroImage 208, 116450 (2020)
Prados, F., Ashburner, J., Blaiotta, C., Brosch, T., Carballido-Gamio, J., Cardoso, M.J., Conrad, B.N., Datta, E., Dávid, G., De Leener, B., et al.: Spinal cord grey matter segmentation challenge. Neuroimage 152, 312–329 (2017)
Puyol-Anton, E., Ruijsink, B., Piechnik k., S., Neubauer, S., et al.: Fairness in cardiac MR image analysis: an investigation of bias due to data imbalance in deep learning based segmentation. arXiv preprint arXiv:2106.12387 (2021)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sharifi-Noghabi, H., Asghari, H., Mehrasa, N., Ester, M.: Domain generalisation via semi-supervised meta learning. arXiv preprint arXiv:2009.12658 (2020)
Tao, Q., Yan, W., Wang, Y., Paiman, E.H., Shamonin, et al.: Deep learning-based method for fully automatic quantification of left ventricle function from cine MR images: a multivendor, multicenter study. Radiology 290(1), 81–88 (2019)
Wang, J., Zhou, S., Fang, C., Wang, L., Wang, J.: Meta corrupted pixels mining for medical image segmentation. In: Proceedings of the MICCAI, pp. 335–345. Springer (2020)
Yu, X., Chen, Y., Li, T., Liu, S., Li, G.: Multi-mapping image-to-image translation via learning disentanglement. In: Proceedings of the NeurIPS (2019)
Zhang, L., Wang, X., Yang, D., Sanford, T., et al.: Generalizing deep learning for medical image segmentation to unseen domains via deep stacked transformation. IEEE Trans, Med. Image 39(7), 2531–2540 (2020)
Zhao, S., Gong, M., Liu, T., Fu, H., Tao, D.: Domain generalization via entropy regularization. In: Proceedings of the NeurIPS, vol. 33 (2020)
Acknowledgement
This work was supported by the University of Edinburgh, the Royal Academy of Engineering and Canon Medical Research Europe by a PhD studentship to Xiao Liu. This work was partially supported by the Alan Turing Institute under the EPSRC grant EP/N510129/1. We thank Nvidia for donating a Titan-X GPU. S.A. Tsaftaris acknowledges the support of Canon Medical and the Royal Academy of Engineering and the Research Chairs and Senior Research Fellowships scheme (grant RCSRF1819\(\backslash \)8\(\backslash \)25).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, X., Thermos, S., O’Neil, A., Tsaftaris, S.A. (2021). Semi-supervised Meta-learning with Disentanglement for Domain-Generalised Medical Image Segmentation. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12902. Springer, Cham. https://doi.org/10.1007/978-3-030-87196-3_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-87196-3_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87195-6
Online ISBN: 978-3-030-87196-3
eBook Packages: Computer ScienceComputer Science (R0)
-
Published in cooperation with
http://miccai.org/
