Abstract
Objectives
An appropriate and fast clinical referral suggestion is important for intra-axial mass-like lesions (IMLLs) in the emergency setting. We aimed to apply an interpretable deep learning (DL) system to multiparametric MRI to obtain clinical referral suggestion for IMLLs, and to validate it in the setting of nontraumatic emergency neuroradiology.
Methods
A DL system was developed in 747 patients with IMLLs ranging 30 diseases who underwent pre- and post-contrast T1-weighted (T1CE), FLAIR, and diffusion-weighted imaging (DWI). A DL system that segments IMLLs, classifies tumourous conditions, and suggests clinical referral among surgery, systematic work-up, medical treatment, and conservative treatment, was developed. The system was validated in an independent cohort of 130 emergency patients, and performance in referral suggestion and tumour discrimination was compared with that of radiologists using receiver operating characteristics curve, precision-recall curve analysis, and confusion matrices. Multiparametric interpretable visualisation of high-relevance regions from layer-wise relevance propagation overlaid on contrast-enhanced T1WI and DWI was analysed.
Results
The DL system provided correct referral suggestions in 94 of 130 patients (72.3%) and performed comparably to radiologists (accuracy 72.6%, McNemar test; p = .942). For distinguishing tumours from non-tumourous conditions, the DL system (AUC, 0.90 and AUPRC, 0.94) performed similarly to human readers (AUC, 0.81~0.92, and AUPRC, 0.88~0.95). Solid portions of tumours showed a high overlap of relevance, but non-tumours did not (Dice coefficient 0.77 vs. 0.33, p < .001), demonstrating the DL’s decision.
Conclusions
Our DL system could appropriately triage patients using multiparametric MRI and provide interpretability through multiparametric heatmaps, and may thereby aid neuroradiologic diagnoses in emergency settings.
Clinical relevance statement
Our AI triages patients with raw MRI images to clinical referral pathways in brain intra-axial mass-like lesions. We demonstrate that the decision is based on the relative relevance between contrast-enhanced T1-weighted and diffusion-weighted images, providing explainability across multiparametric MRI data.
Key Points
• A deep learning (DL) system using multiparametric MRI suggested clinical referral to patients with intra-axial mass-like lesions (IMLLs) similar to radiologists (accuracy 72.3% vs. 72.6%).
• In the differentiation of tumourous and non-tumourous conditions, the DL system (AUC, 0.90) performed similar with radiologists (AUC, 0.81–0.92).
• The DL’s decision basis for differentiating tumours from non-tumours can be quantified using multiparametric heatmaps obtained via the layer-wise relevance propagation method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Abbreviations
- AUC:
-
Area under the curve of the receiver operating characteristic curve
- DL:
-
Deep learning
- DWI:
-
Diffusion-weighted imaging
- FLAIR:
-
Fluid-attenuated inversion recovery
- LRP:
-
Layer-wise relevance propagation
- T1CE:
-
Post-contrast T1-weighted
References
Huisman TA (2009) Tumor-like lesions of the brain. Cancer Imaging 9(Spec No A):S10-13
Rovira A (2014) Tumefactive idiopathic inflammatory demyelinating lesions: a diagnostic challenge. Mult Scler 20:634–635
Filippi M, Preziosa P, Banwell BL et al (2019) Assessment of lesions on magnetic resonance imaging in multiple sclerosis: practical guidelines. Brain 142:1858–1875
Chang PD, Malone HR, Bowden SG et al (2017) A multiparametric model for mapping cellularity in glioblastoma using radiographically localized biopsies. AJNR Am J Neuroradiol 38:890–898
Eo T, Shin H, Jun Y, Kim T, Hwang D (2020) Accelerating Cartesian MRI by domain-transform manifold learning in phase-encoding direction. Med Image Anal 63:101689. https://doi.org/10.1016/j.media.2020.101689
Jun Y, Shin H, Eo T, Kim T, Hwang D (2021) Deep model-based magnetic resonance parameter mapping network (DOPAMINE) for fast T1 mapping using variable flip angle method. Med Image Anal 70:102017
Titano JJ, Badgeley M, Schefflein J et al (2018) Automated deep-neural-network surveillance of cranial images for acute neurologic events. Nat Med 24:1337–1341
Kickingereder P, Isensee F, Tursunova I et al (2019) Automated quantitative tumour response assessment of MRI in neuro-oncology with artificial neural networks: a multicentre, retrospective study. Lancet Oncol 20:728–740
Bach S, Binder A, Montavon G, Klauschen F, Müller K-R, Samek W (2015) On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS One 10
Lee J, Kim S, Park I, Eo T, Hwang D (2021) Relevance-CAM: your model already knows where to look. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, pp 14939–14948. https://doi.org/10.1109/CVPR46437.2021.01470
Mongan J, Moy L, Kahn CE (2020) Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers. Radiol Artif Intell 2:e200029
Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp 234–241
Dice LR (1945) Measures of the amount of ecologic association between species. Ecology 26:297–302
Eidel O, Burth S, Neumann J-O et al (2017) Tumor infiltration in enhancing and non-enhancing parts of glioblastoma: a correlation with histopathology. PLoS One 12:e0169292–e0169292
Hayashida Y, Hirai T, Morishita S et al (2006) Diffusion-weighted imaging of metastatic brain tumors: comparison with histologic type and tumor cellularity. AJNR Am J Neuroradiol 27:1419–1425
Obuchowski NA (1995) Multireader, multimodality receiver operating characteristic curve studies: hypothesis testing and sample size estimation using an analysis of variance approach with dependent observations. Acad Radiol 2(Suppl 1):S22-29
Hillis SL (2007) A comparison of denominator degrees of freedom methods for multiple observer ROC analysis. Stat Med 26:596–619
Rauschecker AM, Rudie JD, Xie L et al (2020) Artificial intelligence system approaching neuroradiologist-level differential diagnosis accuracy at brain MRI. Radiology 295:626–637
Lee H, Yune S, Mansouri M et al (2019) An explainable deep-learning algorithm for the detection of acute intracranial haemorrhage from small datasets. Nat Biomed Eng 3:173–182
Windisch P, Weber P, Fürweger C et al (2020) Implementation of model explainability for a basic brain tumor detection using convolutional neural networks on MRI slices. Neuroradiology 62:1515–1518
Yang C, Rangarajan A, Ranka S Visual explanations from deep 3D convolutional neural networks for Alzheimer’s disease classification. In: AMIA annual symposium proceedings, 2018. American Medical Informatics Association, p 1571
Samek W, Binder A, Montavon G, Lapuschkin S, Müller K (2017) Evaluating the visualization of what a deep neural network has learned. IEEE Trans Neural Netw Learn Syst 28:2660–2673
Ebisu T, Tanaka C, Umeda M et al (1996) Discrimination of brain abscess from necrotic or cystic tumors by diffusion-weighted echo planar imaging. Magn Reson Imaging 14:1113–1116
Kim YJ, Chang KH, Song IC et al (1998) Brain abscess and necrotic or cystic brain tumor: discrimination with signal intensity on diffusion-weighted MR imaging. AJR Am J Roentgenol 171:1487–1490
Kim DS, Na DG, Kim KH et al (2009) Distinguishing tumefactive demyelinating lesions from glioma or central nervous system lymphoma: added value of unenhanced CT compared with conventional contrast-enhanced MR imaging. Radiology 251:467–475
Tsui EY, Leung WH, Chan JH, Cheung YK, Ng SH (2002) Tumefactive demyelinating lesions by combined perfusion-weighted and diffusion weighted imaging. Comput Med Imaging Graph 26:343–346
Park SH, Han K, Jang HY et al (2022) Methods for clinical evaluation of artificial intelligence algorithms for medical diagnosis. Radiology 306:20–31
Acknowledgements
This research was partially supported by the Yonsei Signature Research Cluster Program of 2022 (2022-22-0002). We thank Dong Young Kim and Jae Yeon Choi to participate as resident readers in this work.
All authors reviewed the manuscript. H.S. contributed to the deep learning analysis and the writing of the manuscript. J.E.P. contributed to the conceptual design and the writing of the manuscript. Y.J. contributed to software support. T.E. contributed to the visualisation and editing of the manuscript. J.L. contributed to software support and visualisation. J.E.K., D.H.L., H.H.M., and S.I.P. contributed to image analysis. S.K. contributed to statistical analysis. D.H. contributed to conceptual feedback, software support, and project integrity. H.S.K. contributed to the editing of the manuscript, conceptual design, and project integrity.
Funding
This research was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIP) (grant number: NRF-RS202300208227). This research was also supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (2021R1A4A1031437, 2022R1A2C2008983) and Y-BASE R&E Institute a Brain Korea 21, Yonsei University.
Data generated or analysed during the study are available from the corresponding author by request.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Guarantor
The scientific guarantor of this publication is Ho Sung Kim.
Competing interests
The authors declare no competing interests.
Statistics and biometry
The statistical analysis was performed by an expert statistician (S.O.K., 12 years of experience).
Informed consent
Written informed consent was waived by the Institutional Review Board.
Ethical approval
Institutional Review Board approval was obtained.
Study subjects or cohorts overlap
Not applicable.
Methodology
• Retrospective
• Diagnostic or prognostic study
• Performed at a single institution
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Shin, H., Park, J.E., Jun, Y. et al. Deep learning referral suggestion and tumour discrimination using explainable artificial intelligence applied to multiparametric MRI. Eur Radiol 33, 5859–5870 (2023). https://doi.org/10.1007/s00330-023-09710-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00330-023-09710-0