Skip to main content
SpringerLink
Log in

Large-scale 3D non-Cartesian coronary MRI reconstruction using distributed memory-efficient physics-guided deep learning with limited training data

  • Research Article
  • Published:
Magnetic Resonance Materials in Physics, Biology and Medicine Aims and scope Submit manuscript

Abstract

Object

To enable high-quality physics-guided deep learning (PG-DL) reconstruction of large-scale 3D non-Cartesian coronary MRI by overcoming challenges of hardware limitations and limited training data availability.

Materials and methods

While PG-DL has emerged as a powerful image reconstruction method, its application to large-scale 3D non-Cartesian MRI is hindered by hardware limitations and limited availability of training data. We combine several recent advances in deep learning and MRI reconstruction to tackle the former challenge, and we further propose a 2.5D reconstruction using 2D convolutional neural networks, which treat 3D volumes as batches of 2D images to train the network with a limited amount of training data. Both 3D and 2.5D variants of the PG-DL networks were compared to conventional methods for high-resolution 3D kooshball coronary MRI.

Results

Proposed PG-DL reconstructions of 3D non-Cartesian coronary MRI with 3D and 2.5D processing outperformed all conventional methods both quantitatively and qualitatively in terms of image assessment by an experienced cardiologist. The 2.5D variant further improved vessel sharpness compared to 3D processing, and scored higher in terms of qualitative image quality.

Discussion

PG-DL reconstruction of large-scale 3D non-Cartesian MRI without compromising image size or network complexity is achieved, and the proposed 2.5D processing enables high-quality reconstruction with limited training data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

In accordance with the institutional review board, the data acquired in this study contain person-sensitive information, and can only be shared in the context of scientific collaborations.

References

  1. Thedens DR, Irarrazaval P, Sachs TS, Meyer CH, Nishimura DG (1999) Fast magnetic resonance coronary angiography with a three-dimensional stack of spirals trajectory. Magn Reson Med 41(6):1170–1179

    Article  CAS  PubMed  Google Scholar 

  2. Wright KL, Hamilton JI, Griswold MA, Gulani V, Seiberlich N (2014) Non-Cartesian parallel imaging reconstruction. J Magn Reson Imaging 40(5):1022–1040

    Article  PubMed  PubMed Central  Google Scholar 

  3. Chen Y, Lo WC, Hamilton JI, Barkauskas K, Saybasili H, Wright KL et al (2018) Single breath-hold 3D cardiac T1 mapping using through-time spiral GRAPPA. NMR Biomed 31(6):e3923

    Article  PubMed  PubMed Central  Google Scholar 

  4. Irarrazabal P, Nishimura DG (1995) Fast three dimensional magnetic resonance imaging. Magn Reson Med 33(5):656–662

    Article  CAS  PubMed  Google Scholar 

  5. Stehning C, Börnert P, Nehrke K, Eggers H, Dössel O (2004) Fast isotropic volumetric coronary MR angiography using free-breathing 3D radial balanced FFE acquisition. Magn Reson Med 52(1):197–203

    Article  CAS  PubMed  Google Scholar 

  6. Gurney PT, Hargreaves BA, Nishimura DG (2006) Design and analysis of a practical 3D cones trajectory. Magn Reson Med 55(3):575–582

    Article  PubMed  Google Scholar 

  7. Barger AV, Block WF, Toropov Y, Grist TM, Mistretta CA (2002) Time-resolved contrast-enhanced imaging with isotropic resolution and broad coverage using an undersampled 3D projection trajectory. Magn Reson Med 48(2):297–305

    Article  PubMed  Google Scholar 

  8. Feng L, Grimm R, Block KT, Chandarana H, Kim S, Xu J et al (2014) Golden-angle radial sparse parallel MRI: combination of compressed sensing, parallel imaging, and golden-angle radial sampling for fast and flexible dynamic volumetric MRI. Magn Reson Med 72(3):707–717

    Article  PubMed  Google Scholar 

  9. Piccini D, Feng L, Bonanno G, Coppo S, Yerly J, Lim RP et al (2017) Four-dimensional respiratory motion-resolved whole heart coronary MR angiography. Magn Reson Med 77(4):1473–1484

    Article  PubMed  Google Scholar 

  10. Bonanno G, Piccini D, Marchal B, Zenge M, Stuber M (2014) A new binning approach for 3D motion corrected self-navigated whole-heart coronary MRA using independent component analysis of individual coils. In: Proceedings of the 22nd annual meeting of ISMRM, Milan, p 936

  11. Larson AC, White RD, Laub G, McVeigh ER, Li D, Simonetti OP (2004) Self-gated cardiac cine MRI. Magn Reson Med 51(1):93–102

    Article  PubMed  PubMed Central  Google Scholar 

  12. Stehning C, Börnert P, Nehrke K, Dössel O (2005) Free breathing 3D balanced FFE coronary magnetic resonance angiography with prolonged cardiac acquisition windows and intra-RR motion correction. Magn Reson Med 53(3):719–723

    Article  CAS  PubMed  Google Scholar 

  13. Feng L, Coppo S, Piccini D, Yerly J, Lim RP, Masci PG et al (2018) 5D whole-heart sparse MRI. Magn Reson Med 79(2):826–838

    Article  PubMed  Google Scholar 

  14. Mistretta CA, Wieben O, Velikina J, Block W, Perry J, Wu Y et al (2006) Highly constrained backprojection for time-resolved MRI. Magn Reson Med 55(1):30–40

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Feng L, Delacoste J, Smith D, Weissbrot J, Flagg E, Moore WH et al (2019) Simultaneous evaluation of lung anatomy and ventilation using 4D respiratory-motion-resolved ultrashort echo time sparse MRI. J Magn Reson Imaging 49(2):411–422

    Article  PubMed  Google Scholar 

  16. Nam S, Akcakaya M, Basha T, Stehning C, Manning WJ, Tarokh V et al (2013) Compressed sensing reconstruction for whole-heart imaging with 3D radial trajectories: a graphics processing unit implementation. Magn Reson Med 69(1):91–102

    Article  PubMed  Google Scholar 

  17. Hammernik K, Klatzer T, Kobler E, Recht MP, Sodickson DK, Pock T et al (2018) Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med 79(6):3055–3071

    Article  PubMed  Google Scholar 

  18. Schlemper J, Caballero J, Hajnal JV, Price A, Rueckert D (2017) A deep cascade of convolutional neural networks for MR image reconstruction. In: Information processing in medical imaging: 25th international conference, IPMI 2017, Boone, NC, USA, June 25–30, 2017, Proceedings 25. Springer, pp 647–658

  19. Aggarwal HK, Mani MP, Jacob M (2018) MoDL: Model-based deep learning architecture for inverse problems. IEEE Trans Med Imaging 38(2):394–405

    Article  PubMed  PubMed Central  Google Scholar 

  20. Knoll F, Hammernik K, Zhang C, Moeller S, Pock T, Sodickson DK et al (2020) Deep-learning methods for parallel magnetic resonance imaging reconstruction: a survey of the current approaches, trends, and issues. IEEE Signal Process Mag 37(1):128–140

    Article  PubMed  PubMed Central  Google Scholar 

  21. Yaman B, Hosseini SAH, Moeller S, Ellermann J, Uğurbil K, Akçakaya M (2020) Self-supervised learning of physics-guided reconstruction neural networks without fully sampled reference data. Magn Reson Med 84(6):3172–3191

    Article  PubMed  PubMed Central  Google Scholar 

  22. Hammernik K, Küstner T, Yaman B, Huang Z, Rueckert D, Knoll F et al (2023) Physics-driven deep learning for computational magnetic resonance imaging: combining physics and machine learning for improved medical imaging. IEEE Signal Process Mag 40(1):98–114

    Article  PubMed  PubMed Central  Google Scholar 

  23. Piccini D, Littmann A, Nielles-Vallespin S, Zenge MO (2011) Spiral phyllotaxis: the natural way to construct a 3D radial trajectory in MRI. Magn Reson Med 66(4):1049–1056

    Article  PubMed  Google Scholar 

  24. Ramzi Z, Chaithya G, Starck J-L, Ciuciu P (2022) NC-PDNet: a density-compensated unrolled network for 2D and 3D non-Cartesian MRI reconstruction. IEEE Trans Med Imaging 41(7):1625–1638

    Article  PubMed  Google Scholar 

  25. Malavé MO, Baron CA, Koundinyan SP, Sandino CM, Ong F, Cheng JY et al (2020) Reconstruction of undersampled 3D non-Cartesian image-based navigators for coronary MRA using an unrolled deep learning model. Magn Reson Med 84(2):800–812

    Article  PubMed  PubMed Central  Google Scholar 

  26. Deng Z, Yaman B, Zhang C, Moeller S, Akçakaya M (2021) Efficient training of 3D unrolled neural networks for MRI reconstruction using small databases. In: 2021 55th Asilomar conference on signals, systems, and computers. IEEE, pp 886–889.

  27. Chen Z, Chen Y, Xie Y, Li D, Christodoulou AG (2022) Data-consistent non-Cartesian deep subspace learning for efficient dynamic MR image reconstruction. In: 2022 IEEE 19th international symposium on biomedical imaging (ISBI). IEEE, pp 1–5

  28. Kellman M, Zhang K, Markley E, Tamir J, Bostan E, Lustig M et al (2020) Memory-efficient learning for large-scale computational imaging. IEEE Trans Comput Imaging 6:1403–1414

    Article  Google Scholar 

  29. Baron CA, Dwork N, Pauly JM, Nishimura DG (2018) Rapid compressed sensing reconstruction of 3D non-Cartesian MRI. Magn Reson Med 79(5):2685–2692

    Article  PubMed  Google Scholar 

  30. Ramani S, Fessler JA (2013) Accelerated nonCartesian SENSE reconstruction using a majorize-minimize algorithm combining variable-splitting. In: 2013 IEEE 10th international symposium on biomedical imaging. IEEE, pp 704–707

  31. Micikevicius P, Narang S, Alben J, Diamos G, Elsen E, Garcia D et al (2017) Mixed precision training. arXiv:1710.03740

  32. Zhou L, Pan S, Wang J, Vasilakos AV (2017) Machine learning on big data: opportunities and challenges. Neurocomputing 237:350–361

    Article  Google Scholar 

  33. Pruessmann KP, Weiger M, Scheidegger MB, Boesiger P (1999) SENSE: sensitivity encoding for fast MRI. Magn Reson Med 42(5):952–962

    Article  CAS  PubMed  Google Scholar 

  34. Pruessmann KP, Weiger M, Börnert P, Boesiger P (2001) Advances in sensitivity encoding with arbitrary k-space trajectories. Magn Reson Med 46(4):638–651

    Article  CAS  PubMed  Google Scholar 

  35. Fessler JA (2020) Optimization methods for magnetic resonance image reconstruction: Key models and optimization algorithms. IEEE Signal Process Mag 37(1):33–40

    Article  PubMed  PubMed Central  Google Scholar 

  36. Kumar R, Purohit M, Svitkina Z, Vee E, Wang J (2019) Efficient rematerialization for deep networks. In: Advances in neural information processing systems, p 32

  37. Griewank A (1999) An implementation of checkpointing for the reverse or adjoint model of differentiation. ACM Trans Math Softw 26(1):1–19

    Google Scholar 

  38. Jain P, Jain A, Nrusimha A, Gholami A, Abbeel P, Gonzalez J et al (2020) Checkmate: breaking the memory wall with optimal tensor rematerialization. Proc Mach Learn Syst 2:497–511

    Google Scholar 

  39. Etienne A, Botnar RM, Van Muiswinkel AM, Boesiger P, Manning WJ, Stuber M (2002) “Soap-Bubble” visualization and quantitative analysis of 3D coronary magnetic resonance angiograms. Magn Reson Med 48(4):658–666

    Article  PubMed  Google Scholar 

  40. Hutchinson M, Raff U (1988) Fast MRI data acquisition using multiple detectors. Magn Reson Med 6(1):87–91

    Article  CAS  PubMed  Google Scholar 

  41. Ra JB, Rim C (1993) Fast imaging using subencoding data sets from multiple detectors. Magn Reson Med 30(1):142–145

    Article  CAS  PubMed  Google Scholar 

  42. Cruz G, Atkinson D, Henningsson M, Botnar RM, Prieto C (2017) Highly efficient nonrigid motion-corrected 3D whole-heart coronary vessel wall imaging. Magn Reson Med 77(5):1894–1908

    Article  CAS  PubMed  Google Scholar 

  43. Zhang C, Piccini D, Demirel OB, Bonanno G, Yaman B, Stuber M et al (2022) Distributed memory-efficient physics-guided deep learning reconstruction for large-scale 3d non-Cartesian MRI. In: 2022 IEEE 19th international symposium on biomedical imaging (ISBI). IEEE, pp 1–5

  44. Muckley MJ, Stern R, Murrell T, Knoll F (2020) TorchKbNufft: a high-level, hardware-agnostic non-uniform fast Fourier transform. In: ISMRM workshop on data sampling & image reconstruction

  45. Minnema J, Wolff J, Koivisto J, Lucka F, Batenburg KJ, Forouzanfar T et al (2021) Comparison of convolutional neural network training strategies for cone-beam CT image segmentation. Comput Methods Programs Biomed 207:106192

    Article  PubMed  Google Scholar 

  46. Bermejo-Peláez D, Estepar RSJ, Ledesma-Carbayo MJ (2018) Emphysema classification using a multi-view convolutional network. In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018). IEEE, pp 519–522

  47. Ziabari A, Ye DH, Srivastava S, Sauer KD, Thibault J-B, Bouman CA (2018) 2.5 D deep learning for CT image reconstruction using a multi-GPU implementation. In: 2018 52nd Asilomar conference on signals, systems, and computers. IEEE, pp 2044–2049

  48. Prasoon A, Petersen K, Igel C, Lauze F, Dam E, Nielsen M (2013) Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. In: Medical image computing and computer-assisted intervention–MICCAI 2013: 16th international conference, Nagoya, Japan, September 22–26, 2013, Proceedings, Part II 16. Springer, pp 246–253

  49. Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):1–48

    Article  Google Scholar 

  50. Henningsson M, Koken P, Stehning C, Razavi R, Prieto C, Botnar RM (2012) Whole-heart coronary MR angiography with 2D self-navigated image reconstruction. Magn Reson Med 67(2):437–445

    Article  PubMed  Google Scholar 

  51. Küstner T, Munoz C, Psenicny A, Bustin A, Fuin N, Qi H et al (2021) Deep-learning based super-resolution for 3D isotropic coronary MR angiography in less than a minute. Magn Reson Med 86(5):2837–2852

    Article  PubMed  Google Scholar 

  52. Ahmad R, Ding Y, Simonetti OP (2015) Edge sharpness assessment by parametric modeling: application to magnetic resonance imaging. Concepts Magn Reson Part A 44(3):138–149

    Article  CAS  Google Scholar 

  53. Gilton D, Ongie G, Willett R (2021) Deep equilibrium architectures for inverse problems in imaging. IEEE Trans Comput Imaging 7:1123–1133

    Article  Google Scholar 

  54. Güngör A, Askin B, Soydan DA, Top CB, Saritas EU, Çukur T (2023) DEQ-MPI: a deep equilibrium reconstruction with learned consistency for magnetic particle imaging. IEEE Trans Med Imaging

  55. Seiberlich N, Breuer F, Blaimer M, Jakob P, Griswold M (2008) Self-calibrating GRAPPA operator gridding for radial and spiral trajectories. Magn Reson Med 59(4):930–935

    Article  PubMed  Google Scholar 

  56. Akçakaya M, Nam S, Basha TA, Kawaji K, Tarokh V, Nezafat R (2014) An augmented Lagrangian based compressed sensing reconstruction for non-Cartesian magnetic resonance imaging without gridding and regridding at every iteration. PLoS ONE 9(9):e107107

    Article  PubMed  PubMed Central  Google Scholar 

  57. Yurt M, Özbey M, Dar SU, Tinaz B, Oguz KK, Çukur T (2022) Progressively volumetrized deep generative models for data-efficient contextual learning of MR image recovery. Med Image Anal 78:102429

    Article  PubMed  Google Scholar 

  58. Dar SUH, Özbey M, Çatlı AB, Çukur T (2020) A transfer-learning approach for accelerated MRI using deep neural networks. Magn Reson Med 84(2):663–685

    Article  PubMed  Google Scholar 

Download references

Funding

NIH, Grant numbers: R01HL153146, R01EB032830, R21EB028369, P41EB027061.

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering, University of Minnesota, 200 Union Street S.E., Minneapolis, MN, 55455, USA

    Chi Zhang, Omer Burak Demirel, Burhaneddin Yaman & Mehmet Akçakaya

  2. Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA

    Chi Zhang, Omer Burak Demirel, Burhaneddin Yaman, Steen Moeller & Mehmet Akçakaya

  3. Department of Diagnostic and Interventional Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland

    Davide Piccini, Christopher W. Roy & Matthias Stuber

  4. Advanced Clinical Imaging Technology, Siemens Healthineers International, Lausanne, Switzerland

    Davide Piccini & Gabriele Bonanno

  5. Department of Medicine (Cardiology), University of Minnesota, Minneapolis, MN, 55455, USA

    Chetan Shenoy

  6. Center for Biomedical Imaging, Lausanne, Switzerland

    Matthias Stuber

Authors
  1. Chi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Davide Piccini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Omer Burak Demirel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gabriele Bonanno
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christopher W. Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Burhaneddin Yaman
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Steen Moeller
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Chetan Shenoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Matthias Stuber
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Mehmet Akçakaya
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Mehmet Akçakaya: Study conception and design, drafting of manuscript, critical revision. Gabriele Bonanno: Acquisition of data, critical revision. Omer Burak Demirel: Analysis and interpretation of data, critical revision. Steen Moeller: Analysis and interpretation of data, critical revision. Davide Piccini: Acquisition of data, critical revision. Burhaneddin Yaman: Analysis and interpretation of data, critical revision. Chetan Shenoy: Analysis and interpretation of data, critical revision. Matthias Stuber: Acquisition of data, critical revision. Chi Zhang: Study conception and design, drafting of manuscript, critical revision. Christopher W. Roy: Acquisition of data, critical revision.

Corresponding author

Correspondence to Mehmet Akçakaya.

Ethics declarations

Conflict of interest

Davide Piccini and Gabriele Bonanno are employed by Siemens Healthineers AG, Bern, Switzerland. Matthias Stuber receives non-monetary research support from Siemens Healthineers that is covered by a master research agreement handled by his employer (CHUV). Matthias Stuber has a research contract with Circle that is handled by the Tech Transfer Office (PACTT) of his employer (CHUV).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C., Piccini, D., Demirel, O.B. et al. Large-scale 3D non-Cartesian coronary MRI reconstruction using distributed memory-efficient physics-guided deep learning with limited training data. Magn Reson Mater Phy (2024). https://doi.org/10.1007/s10334-024-01157-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10334-024-01157-8

Keywords

Search

Navigation