Skip to main content
SpringerLink
Log in

Generating Realistic Brain MRIs via a Conditional Diffusion Probabilistic Model

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 (MICCAI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14227))

Abstract

As acquiring MRIs is expensive, neuroscience studies struggle to attain a sufficient number of them for properly training deep learning models. This challenge could be reduced by MRI synthesis, for which Generative Adversarial Networks (GANs) are popular. GANs, however, are commonly unstable and struggle with creating diverse and high-quality data. A more stable alternative is Diffusion Probabilistic Models (DPMs) with a fine-grained training strategy. To overcome their need for extensive computational resources, we propose a conditional DPM (cDPM) with a memory-efficient process that generates realistic-looking brain MRIs. To this end, we train a 2D cDPM to generate an MRI subvolume conditioned on another subset of slices from the same MRI. By generating slices using arbitrary combinations between condition and target slices, the model only requires limited computational resources to learn interdependencies between slices even if they are spatially far apart. After having learned these dependencies via an attention network, a new anatomy-consistent 3D brain MRI is generated by repeatedly applying the cDPM. Our experiments demonstrate that our method can generate high-quality 3D MRIs that share a similar distribution to real MRIs while still diversifying the training set. The code is available at https://github.com/xiaoiker/mask3DMRI_diffusion and also will be released as part of MONAI, at https://github.com/Project-MONAI/GenerativeModels.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bermudez, C., Plassard, A.J., Davis, L.T., Newton, A.T., Resnick, S.M., Landman, B.A.: Learning implicit brain MRI manifolds with deep learning. In: Medical Imaging 2018: Image Processing, vol. 10574, pp. 408–414. SPIE (2018)

    Google Scholar 

  2. Dar, S.U., Yurt, M., Karacan, L., Erdem, A., Erdem, E., Çukur, T.: Image synthesis in multi-contrast MRI with conditional generative adversarial networks. IEEE Trans. Med. Imaging 38(10), 2375–2388 (2019)

    Article  Google Scholar 

  3. Dorjsembe, Z., Odonchimed, S., Xiao, F.: Three-dimensional medical image synthesis with denoising diffusion probabilistic models. In: Medical Imaging with Deep Learning (2022). https://openreview.net/forum?id=Oz7lKWVh45H

  4. Goodfellow, I., et al.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)

    Google Scholar 

  5. Gretton, A., Borgwardt, K.M., Rasch, M.J., Schölkopf, B., Smola, A.: A kernel two-sample test. J. Mach. Learn. Res. 13(1), 723–773 (2012)

    MathSciNet  MATH  Google Scholar 

  6. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of wasserstein GANs. In: NIPS, vol. 30, pp. 5769–5779 (2017)

    Google Scholar 

  7. Han, C., et al.: GAN-based synthetic brain MR image generation. In: IEEE International Symposium on Biomedical Imaging, pp. 734–738 (2018)

    Google Scholar 

  8. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Adv. Neural Inf. Process. Syst. 33, 6840–6851 (2020)

    Google Scholar 

  9. Jung, E., Luna, M., Park, S.H.: Conditional GAN with an attention-based generator and a 3D discriminator for 3D medical image generation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12906, pp. 318–328. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87231-1_31

  10. Jung, E., Luna, M., Park, S.H.: Conditional GAN with 3D discriminator for MRI generation of Alzheimer’s disease progression. Pattern Recogn. 133, 109061 (2023)

    Article  Google Scholar 

  11. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)

    Google Scholar 

  12. Kwon, G., Han, C., Kim, D.: Generation of 3D brain MRI using auto-encoding generative adversarial networks. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 118–126. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_14

  13. La Barbera, G., et al.: Anatomically constrained CT image translation for heterogeneous blood vessel segmentation. In: British Machine Vision Virtual Conference, p. 776 (2022)

    Google Scholar 

  14. Larsen, A.B.L., Sønderby, S.K., Larochelle, H., Winther, O.: Autoencoding beyond pixels using a learned similarity metric. In: International Conference on Machine Learning, pp. 1558–1566. Proceedings of Machine Learning Research (2016)

    Google Scholar 

  15. Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11) (2008)

    Google Scholar 

  16. Ouyang, J., Adeli, E., Pohl, K.M., Zhao, Q., Zaharchuk, G.: Representation disentanglement for multi-modal brain MRI Analysis. In: Feragen, A., Sommer, S., Schnabel, J., Nielsen, M. (eds.) IPMI 2021. LNCS, vol. 12729, pp. 321–333. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78191-0_25

  17. Pawlowski, N., Coelho de Castro, D., Glocker, B.: Deep structural causal models for tractable counterfactual inference. Adv. Neural Inf. Process. Syst. 33, 857–869 (2020)

    Google Scholar 

  18. Pinaya, W.H., et al.: Brain imaging generation with latent diffusion models. In: Deep Generative Models: DGM4MICCAI 2022, pp. 117–126 (2022)

    Google Scholar 

  19. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, vol. 9351, pp. 234–241 (2015)

    Google Scholar 

  20. Shaw, P., Uszkoreit, J., Vaswani, A.: Self-attention with relative position representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 2, pp. 464–468. Association for Computational Linguistics, New Orleans (2018). https://doi.org/10.18653/v1/N18-2074, https://aclanthology.org/N18-2074

  21. Shin, H.C., et al.: Medical image synthesis for data augmentation and anonymization using generative adversarial networks. In: International Workshop on Simulation and Synthesis in Medical Imaging, vol. 11037 (2018)

    Google Scholar 

  22. Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., Ganguli, S.: Deep unsupervised learning using nonequilibrium thermodynamics. In: International Conference on Machine Learning, pp. 2256–2265. PMLR (2015)

    Google Scholar 

  23. Sun, L., Chen, J., Xu, Y., Gong, M., Yu, K., Batmanghelich, K.: Hierarchical amortized GAN for 3D high resolution medical image synthesis. IEEE J. Biomed. Health Inf. 26(8), 3966–3975 (2022)

    Article  Google Scholar 

  24. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  25. Wolleb, J., Bieder, F., Sandkühler, R., Cattin, P.C.: Diffusion models for medical anomaly detection. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, vol. 13438, pp. 35–45 (2022)

    Google Scholar 

  26. Xing, S., Sinha, H., Hwang, S.J.: Cycle consistent embedding of 3D brains with auto-encoding generative adversarial networks. In: Medical Imaging with Deep Learning (2021)

    Google Scholar 

  27. Yu, B., Zhou, L., Wang, L., Fripp, J., Bourgeat, P.: 3D cGAN based cross-modality MR image synthesis for brain tumor segmentation. In: IEEE 15th International Symposium on Biomedical Imaging, pp. 626–630 (2018)

    Google Scholar 

  28. Zhang, J., et al.: Multi-label, multi-domain learning identifies compounding effects of HIV and cognitive impairment. Med. Image Anal. 75, 102246 (2022)

    Google Scholar 

  29. Zhao, Q., Liu, Z., Adeli, E., Pohl, K.M.: Longitudinal self-supervised learning. Med. Image Anal. 71, 102051 (2021)

    Article  Google Scholar 

  30. Zheng, S., Charoenphakdee, N.: Diffusion models for missing value imputation in tabular data. In: NeurIPS Table Representation Learning (TRL) Workshop (2022)

    Google Scholar 

Download references

Acknowledgement

This work was partly supported by funding from the National Institute of Health (MH113406, DA057567, AA021697, AA017347, AA010723, AA005965, and AA028840), the DGIST R &D program of the Ministry of Science and ICT of KOREA (22-KUJoint-02), Stanford School of Medicine Department of Psychiatry and Behavioral Sciences Faculty Development and Leadership Award, and by the Stanford HAI Google Cloud Credit.

Author information

Authors and Affiliations

  1. Stanford University, Stanford, CA, 94305, USA

    Wei Peng, Ehsan Adeli, Tomas Bosschieter, Qingyu Zhao & Kilian M. Pohl

  2. Daegu Gyeongbuk Institute of Science and Technology, Daegu, South Korea

    Sang Hyun Park

Authors
  1. Wei Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ehsan Adeli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tomas Bosschieter
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sang Hyun Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qingyu Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kilian M. Pohl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kilian M. Pohl .

Editor information

Editors and Affiliations

  1. Icahn School of Medicine, Mount Sinai, NYC, NY, USA, Tel Aviv University, Tel Aviv, Israel

    Hayit Greenspan

  2. Emory University, Atlanta, GA, USA

    Anant Madabhushi

  3. Queen’s University, Kingston, ON, Canada

    Parvin Mousavi

  4. The University of British Columbia, Vancouver, BC, Canada

    Septimiu Salcudean

  5. Yale University, New Haven, CT, USA

    James Duncan

  6. IBM Research, San Jose, CA, USA

    Tanveer Syeda-Mahmood

  7. Johns Hopkins University, Baltimore, MD, USA

    Russell Taylor

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Peng, W., Adeli, E., Bosschieter, T., Park, S.H., Zhao, Q., Pohl, K.M. (2023). Generating Realistic Brain MRIs via a Conditional Diffusion Probabilistic Model. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14227. Springer, Cham. https://doi.org/10.1007/978-3-031-43993-3_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43993-3_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43992-6

  • Online ISBN: 978-3-031-43993-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation