Avoid common mistakes on your manuscript.
Background
In the field of medical imaging, machine learning and artificial intelligence (AI) have emerged as techniques that are likely to fundamentally transform clinical practice in the coming years [1,2,3,4,5,6,7]. In the context of magnetic resonance imaging (MRI) and magnetic resonance spectroscopy (MRS), AI has the potential to impact all stages of the imaging pipeline, from (1) image acquisition and reconstruction to (2) image analysis and interpretation and (3) diagnosis and prognosis. If realized, this intelligent imaging revolution will lead to accelerated acquisition times, reduced workload for clinicians, reduced costs to the healthcare system, and more personalized treatment decisions for patients.
Due to the increasing availability of data and computing power, numerous AI solutions, primarily based on deep learning, have been proposed over the past few years, and methods are now evolving toward clinical application and usage. AI algorithms have been proposed and studied in the context of scan planning, accelerated acquisition and reconstruction, and image analysis. While current work often focuses only on one part of this full imaging pipeline, deep learning provides many more opportunities to improve the whole workflow of MR image from acquisition to analysis and diagnosis. Future investigations of deep learning approaches will further support the choice of the examination based on actual physiological scan parameters, e.g., heart rate, or on the patient information obtained during the planning. Furthermore, deep learning techniques will support further acceleration in scan time, to allow for real-time interventional MRI [8]. We observe a trend toward embedding different elements of the imaging pipeline (acquisition, reconstruction, post-processing, analysis, diagnosis) in deep learning models and training a network end to end in so-called multi-task networks, or to exploit the additionally available data, and these directions will form the future of learning-based MR imaging. However, challenges relating to the usability, robustness, and reliability of AI algorithms are crucial when used in daily clinical practice, and their performance needs to be ensured and validated for diverse and heterogeneous patient cohorts.
This special issue highlights current developments and unresolved challenges in the push to move AI from a hot research topic to a clinical reality for MRI/MRS acquisition and reconstruction.
Acquisition and reconstruction
MR imaging represents a significant opportunity for AI due to the redundancy and the high dimensionality of the data. AI may help to overcome challenges regarding acquisition time, SNR, the trade-off between spatial and temporal resolutions, and different types of artifacts, e.g., cardiac, and respiratory motion. While deep learning approaches often outperform conventional approaches in terms of pixel-wise quantitative metrics, these approaches can be overconfident or overfit to the underlying application, and therefore attention needs to be paid to the real-world evaluation of these models to ensure reliability for clinical deployment. It is challenging to evaluate the quality and robustness of AI reconstruction approaches, especially for subtle pathologies. In particular, we may find overly optimistic results by using simulated data and neglecting the unprocessed raw k-space data [9].
The robustness of neural networks to changes in anatomy [10] and architectures [11] was studied in the context of static 2D imaging. Small adversarial input perturbations can affect the neural networks differently [12]. Domain shifts in anatomy have also been shown to impact image reconstruction when using unrolled networks and moderate acceleration [10], but are more subtle and difficult to identify. This observation regarding domain shift is different to image analysis tasks, where a subtle change in data characteristics might already lead to, e.g., mis-segmentation. However, as reconstructions serve as the basis for further downstream tasks and considering that when deployed in a clinical environment these solutions might be more likely to encounter out-of-domain data, e.g., other imaging sequences or contrasts, or patients with different pathologies, biases and error propagation could occur. Clearly, hurdles remain to be overcome and better quality assurance mechanisms need to be developed.
In this issue, Heckel et al. provide a very comprehensive review of recent advances in AI reconstruction for robust MRI [13]. They particularly focus on recent advancements in the field where approaches such as end-to-end network training, generative priors, and self-supervised learning play key roles. They discuss new neural network architectures, such as transformers, that are sure to play an increasing role in the field in the coming years. They continue to consider the possibility of AI for image acquisition to be developed in tandem with AI reconstruction algorithms to improve k-space sampling and pulse sequence design. In addition, they introduce advanced applications, such as quantitative and dynamic MRI, and consider the main challenges of AI-based reconstruction which include hallucinations, model instabilities, the difficulty of benchmarking, how to quantify uncertainty, and the need for large diverse dataset for both model development and validation.
Villegas-Martinez et al. and Yang et al. provide focused reviews on cardiovascular [14] and neurological [15] applications, respectively. With respect to the acquisition, common themes which persist regardless of the application are automated scan planning, acquisition parameter selection, and artifact correction, with methods to improve workflow efficiency and reduce operator variability promising to lead to a future with faster, lower-dose and contrast agent-free imaging. Zhou et al. specifically address one of the main challenges of MR acquisition, subject motion, and discuss the use of AI to correct for this [16]. They survey AI methods to reduce or estimate motion artifacts in both image and frequency domains. In addition, they consider how the estimated motion can be used and how possible synergies between the motion estimation and the downstream tasks (e.g., segmentation, parametric mapping, or MR-guided therapy) can be exploited. In keeping with the trend of the other work in this issue, they identify the pressing need to be able to properly validate the accuracy and fidelity of the estimated motion.
Some of the previously discussed challenges are also tackled within this special issue. Original research from Venkatesh et al. introduces SpiNet-QSM for reconstructing quantitative susceptibility maps (QSM) using a Schatten p-norm model-based AI framework [17]. Of particular interest is the learnable p parameter which allows the framework the flexibility to adapt the regularizer to the specific data.
Two studies address hardware limitations with (1) Zhang et al. proposing a physics-guided AI model in 2.5D that yields high-quality coronary MRI reconstructions based on quantitative image quality metrics and the assessment of expert cardiologists as compared to conventional 3D kooshball coronary MRI [18], and (2) Shafique et al. presenting a parallel framework for SVD-based low-rank model for reconstruction of highly accelerated dynamic cardiac MR that achieves > 10 × speedup in computation time [19].
The work of Fujima et al. investigates the obtained image quality and quantitative reliability of model-based deep learning reconstructions in diffusion-weighted imaging. They found significant qualitative and quantitative, in terms of SNR and CNR, differences between conventional and AI-based reconstructions [20].
In the domain of MRS reconstruction, Berto et al. report the results of a conference challenge that aimed to achieve faster imaging time with the use of AI reconstruction [21]. Specifically, the reconstruction focused on gamma-aminobutyric acid (GABA)-edited MRS with just one-quarter of the typical transients. A range of different neural network architectures and training schemes achieved promising results, but the authors acknowledge the challenge of assessing the quality of spectral reconstructions.
Image analysis and interpretation
Post-processing is the most widely studied application of AI in medical imaging. In particular, image analysis tasks such as segmentation and landmark detection are now routinely tackled with AI. It is probable that post-processing AI applications are more mature as they are easier to evaluate compared to acquisition/reconstruction tasks and there tends to be more data available to train such models. For example, quantitative metrics of evaluating automated segmentations are well established, whereas we have seen that evaluating reconstructed images remains a challenge [22]. However, questions about the reliability of AI are still present for post-processing applications in the presence of biases and domain shifts [23].
In line with previously discussed trends, image analysis tasks are now being tackled with joint models using multi-task learning, linking the analysis with the acquisition and reconstruction of the images. Prominent examples of this are the combined reconstruction and segmentation of accelerated MRI, which have been proposed for several different sequences and anatomies [24,25,26,27,28]. Similar synergies between acquisition and image analysis are being made for other analysis steps as well, such as kinetic parameter estimation where research is studying the direct reconstruction of parametric maps from k-space, linking the acquisition and reconstruction directly to the diagnostic task [3, 29]. We envisage this trend to continue and expand in the future, potentially leading to all reconstruction and image analysis tasks being completed by one model in an end-to-end fashion.
Novel learning strategies [30] and the increasing availability of data also foster the trend toward more complex architectures. These networks can better generalize to different domains [31] and/or serve a multitude of tasks [32]. Although this generalizability aspect is desirable, clinical reliability and practical value still need to be investigated.
In this special issue, Suwannasak et al. make such a step to evaluate an AI-based super-resolution algorithm. In particular, the algorithm allows generation of high-resolution images from 2× under sampled low-resolution 3D T1-weighted brain MR images. The authors show the potential to reduce scan time to just 1 min, while their analysis shows that the super-resolution images still give accurate brain volume measurements and preserve image quality [33].
Other analysis applications are presented by Huang et al. who present work on metabolite concentration quantification for in vivo MRS, and their promising results with a CNN-based model aligned well with the ground truth but required further investigation for differentiating metabolites with low concentrations from overlapping signals [34]. Kafali et al. quantified visceral and subcutaneous adipose tissue based on a 3D multi-contrast CNN and achieved excellent performance based on longitudinal data suitable for monitoring the risk factors for cardiometabolic diseases over time in overweight patients [35].
Image diagnosis and prognosis
The advancement of AI has also opened new frontiers to potentially improve the diagnostic accuracy and prognostic capabilities. Developments are crucial for accurate disease characterization and early treatment planning across various specialties, such as oncology, neurology, and cardiovascular imaging. Radiomics and ML models have been leveraged to extract image features [36,37,38], which provide insights into disease characteristics that may be neglected through conventional methods. Additionally, the use of multiparametric MRI protocols facilitates detailed tissue characterization, aiding in the early disease detection [39,40,41], risk stratification [42, 43], or therapy monitoring and guidance [44, 45]. To increase the model robustness and reliability, techniques of data augmentations and synthetic MR images generation were developed [46, 47] to address the need for large and diverse datasets for training. Moreover, hybrid AI models that integrate deep learning with radiomics offer enhanced diagnostic precision and more detailed prognostic assessments from MRI scans [48].
However, operating in a diagnostic and prognostic setting poses several challenges. The diversity of MRI data across different machines and protocols complicates the development of universally applicable AI models. AI applications in the clinical environment require high reliability and reproducibility, which necessitates clinical validations, standardized datasets, and comparisons across MRI machines and sites. The integration of these models into existing clinical workflows requires not only technical compatibility—and in some cases major infrastructural changes—but also substantial training for clinicians to ensure accurate and effective use of these AI tools.
The risk of misdiagnosis emphasizes the need for clinicians to apply and triage their judgment with AI recommendations. Human-interpretable feedback about the output and/or processing of the data needs to be provided and analyzed, to increase understanding and collaboration. Another risk relates to training on imbalanced datasets which can lead to biases in diagnosis and prognosis such as inaccurate predictions for underrepresented patient groups. Furthermore, issues surrounding privacy and security cannot be neglected, given the sensitive nature of patient data involved in these AI systems. It is crucial that we ensure the benefits of AI in MRI without compromising ethical standards and patient care quality.
While the potential benefits of diagnostic and prognostic models are large, so too are these risks and challenges, and as such, these use-cases of AI are proportionally underrepresented for MRI/MRS data. They are also not included in this special issue, but the universal theme of the need robust validation of AI in these cases remains clear.
This special issue does, however, provide valuable insights into a potentially crucial step in this process, the model evaluation. In the paper by Sharma et al., different statistical tests were used for the comparison of deep learning models trained with different hyperparameter settings and it was found that mixed effect models may be more sensitive than ANOVA for identifying such differences [49].
In summary, the papers of this special issue cover a spectrum of applications from the acquisition to the analysis of MRI/MRS data. Together, they illustrate the power of AI and identify the steps that still need to be taken for the full potential impact of AI to be felt in routine clinical practice. As discussed, many of these wheels are already in motion and given the pace of current progress, and sooner rather than later AI algorithms for MRI/MRS acquisition and reconstruction will be in widespread use in hospitals, benefiting patients and clinicians.
References
Budd S, Robinson EC, Kainz B (2021) A survey on active learning and human-in-the-loop deep learning for medical image analysis. Med Image Anal 71:102062
Gassenmaier S et al (2021) Deep learning applications in magnetic resonance imaging: has the future become present? Diagnostics 11(12):2181
Hammernik K et al (2023) Physics-driven deep learning for computational magnetic resonance imaging: combining physics and machine learning for improved medical imaging. IEEE Signal Process Mag 40(1):98–114
Ismail TF et al (2022) Cardiac MR: from theory to practice. Front Cardiovasc Med. https://doi.org/10.3389/fcvm.2022.826283
Lundervold AS, Lundervold A (2019) An overview of deep learning in medical imaging focusing on MRI. Z Med Phys 29(2):102–127
Rashid M, Singh H, Goyal V (2020) The use of machine learning and deep learning algorithms in functional magnetic resonance imaging—a systematic review. Expert Syst 37(6):e12644
Zeng G et al (2021) A review on deep learning MRI reconstruction without fully sampled k-space. BMC Med Imaging 21(1):195
Glielmo P et al (2024) Artificial intelligence in interventional radiology: state of the art. Eur Radiol Exp 8(1):62
Shimron E et al (2022) Implicit data crimes: machine learning bias arising from misuse of public data. Proc Natl Acad Sci 119(13):e2117203119
Hammernik K et al (2021) Systematic evaluation of iterative deep neural networks for fast parallel MRI reconstruction with sensitivity-weighted coil combination. Magn Reson Med 86(4):1859–1872
Küstner T et al (2024) Predictive uncertainty in deep learning–based MR image reconstruction using deep ensembles: evaluation on the fastMRI data set. Magn Reson Med 92(1):289–302
Antun V et al (2020) On instabilities of deep learning in image reconstruction and the potential costs of AI. Proc Natl Acad Sci 117(48):30088–30095
Heckel R et al (2024) Deep learning for accelerated and robust MRI reconstruction: a review. Special Issue MAGMA
Villegas-Martinez M et al. (2024) The beating heart: artificial intelligence for cardiovascular application in the clinic. Special Issue MAGMA
Yang H et al (2024) Artificial Intelligence for Neuro MRI Acquisition: a Review. Special Issue MAGMA
Zhou Z et al (2024) Stop moving: MR motion correction as an opportunity for artificial intelligence. Magn Reson Mater Phy. https://doi.org/10.1007/s10334-023-01144-5
Venkatesh V et al (2024) SpiNet-QSM: model-based deep learning with schatten p-norm regularization for improved quantitative susceptibility mapping. Magn Reson Mater Phy. https://doi.org/10.1007/s10334-024-01158-7
Zhang C et al (2024) Large-scale 3D non-cartesian coronary MRI reconstruction using distributed memory-efficient physics-guided deep learning with limited training data. Magn Reson Mater Phy. https://doi.org/10.1007/s10334-024-01157-8
Shafique M, Qazi SA, Omer H (2023) Compressed SVD-based L + S model to reconstruct undersampled dynamic MRI data using parallel architecture. Magn Reson Mater Phy. https://doi.org/10.1007/s10334-023-01128-5
Fujima N et al (2023) Improvement of image quality in diffusion-weighted imaging with model-based deep learning reconstruction for evaluations of the head and neck. Magn Reson Mater Phy. https://doi.org/10.1007/s10334-023-01129-4
Berto R et al (2024) Results of the 2023 ISBI challenge to reduce GABA-edited MRS acquisition time. Magn Reson Mater Phy. https://doi.org/10.1007/s10334-024-01156-9
Maier-Hein L et al (2024) Metrics reloaded: recommendations for image analysis validation. Nat Methods 21(2):195–212
Castro DC, Walker I, Glocker B (2020) Causality matters in medical imaging. Nat Commun 11(1):3673
Acar M, Çukur T, Öksüz İ (2022) Segmentation-Aware MRI Reconstruction. Springer International Publishing, Cham
Jacob PAM (2021) Reconstruction and Segmentation of Parallel MR Data Using Image Domain Deep-SLR. In: 2021 IEEE 18th International Symposium on Biomedical Imaging IEEE
Huang Q et al. (2019) MRI reconstruction via cascaded channel-wise attention network. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) IEEE
Sun L et al (2019) Joint CS-MRI reconstruction and segmentation with a unified deep network. Information Processing in Medical Imaging: 26th International Conference, IPMI 2019, Hong Kong, China, June 2–7, 2019, Proceedings 26. Springer
Huang Q et al (2019) FR-Net: Joint reconstruction and segmentation in compressed sensing cardiac MRI. Functional Imaging and Modeling of the Heart: 10th International Conference, FIMH 2019, Bordeaux, France, June 6–8, 2019, Proceedings 10. Springer
Feng L, Ma D, Liu F (2022) Rapid MR relaxometry using deep learning: an overview of current techniques and emerging trends. NMR Biomed 35(4):e4416
Chaitanya K et al (2020) Contrastive learning of global and local features for medical image segmentation with limited annotations. Adv Neural Inf Process Syst 33:12546–12558
Ma J et al (2024) Segment anything in medical images. Nat Commun 15(1):654
Moor M et al (2023) Foundation models for generalist medical artificial intelligence. Nature 616(7956):259–265
Suwannasak A et al (2024) Deep learning-based super-resolution of structural brain MRI at 1.5T: application to quantitative volume measurement. Magn Reson Mater Phy. https://doi.org/10.1007/s10334-024-01165-8
Huang YL, Lin YR, Tsai SY (2023) Comparison of convolutional-neural-networks-based method and LCModel on the quantification of in vivo magnetic resonance spectroscopy. Magn Reson Mater Phy Biol Med 15(1):13
Kafali SG et al (2024) Automated abdominal adipose tissue segmentation and volume quantification on longitudinal MRI using 3D convolutional neural networks with multi-contrast inputs. Magn Reson Mater Phy. https://doi.org/10.1007/s10334-023-01146-3
Stanzione A et al (2020) Prostate MRI radiomics: a systematic review and radiomic quality score assessment. Eur J Radiol 129:109095
Ye D-M, Wang H-T, Yu T (2020) The application of radiomics in breast MRI: a review. Technol Cancer Res Treat 19:1533033820916191
Valdora F et al (2018) Rapid review: radiomics and breast cancer. Breast Cancer Res Treat 169(2):217–229
Mazurowski MA et al (2019) Deep learning in radiology: an overview of the concepts and a survey of the state of the art with focus on MRI. J Magn Reson Imaging 49(4):939–954
Khan P et al (2021) Machine learning and deep learning approaches for brain disease diagnosis: principles and recent advances. IEEE Access 9:37622–37655
Shoeibi A et al (2021) Applications of deep learning techniques for automated multiple sclerosis detection using magnetic resonance imaging: a review. Comput Biol Med 136:104697
Chung CT et al (2022) Machine learning techniques for arrhythmic risk stratification: a review of the literature. Int J Arrhythm 23(1):10
Mainenti PP et al (2022) MRI radiomics: a machine learning approach for the risk stratification of endometrial cancer patients. Eur J Radiol 149:110226
Küstner T et al (2022) Development of a hybrid-imaging-based prognostic index for metastasized-melanoma patients in whole-body 18F-FDG PET/CT and PET/MRI data. Diagnostics 12(9):2102
Nogales A et al (2021) A survey of deep learning models in medical therapeutic areas. Artif Intell Med 112:102020
Chlap P et al (2021) A review of medical image data augmentation techniques for deep learning applications. J Med Imaging Radiat Oncol 65(5):545–563
Garcea F et al (2023) Data augmentation for medical imaging: a systematic literature review. Comput Biol Med 152:106391
Ugga L et al (2021) Meningioma MRI radiomics and machine learning: systematic review, quality score assessment, and meta-analysis. Neuroradiology 63(8):1293–1304
Sharma R et al (2023) Learning to deep learning: statistics and a paradigm test in selecting a UNet architecture to enhance MRI. Magn Reson Mater Phy. https://doi.org/10.1007/s10334-023-01127-6
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical standards
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Küstner, T., Qin, C., Sun, C. et al. The intelligent imaging revolution: artificial intelligence in MRI and MRS acquisition and reconstruction. Magn Reson Mater Phy (2024). https://doi.org/10.1007/s10334-024-01179-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10334-024-01179-2