Skip to main content


Log in

Brain MRI Deep Learning and Bayesian Inference System Augments Radiology Resident Performance

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript


Automated quantitative and probabilistic medical image analysis has the potential to improve the accuracy and efficiency of the radiology workflow. We sought to determine whether AI systems for brain MRI diagnosis could be used as a clinical decision support tool to augment radiologist performance. We utilized previously developed AI systems that combine convolutional neural networks and expert-derived Bayesian networks to distinguish among 50 diagnostic entities on multimodal brain MRIs. We tested whether these systems could augment radiologist performance through an interactive clinical decision support tool known as Adaptive Radiology Interpretation and Education System (ARIES) in 194 test cases. Four radiology residents and three academic neuroradiologists viewed half of the cases unassisted and half with the results of the AI system displayed on ARIES. Diagnostic accuracy of radiologists for top diagnosis (TDx) and top three differential diagnosis (T3DDx) was compared with and without ARIES. Radiology resident performance was significantly better with ARIES for both TDx (55% vs 30%; P < .001) and T3DDx (79% vs 52%; P = 0.002), with the largest improvement for rare diseases (39% increase for T3DDx; P < 0.001). There was no significant difference between attending performance with and without ARIES for TDx (72% vs 69%; P = 0.48) or T3DDx (86% vs 89%; P = 0.39). These findings suggest that a hybrid deep learning and Bayesian inference clinical decision support system has the potential to augment diagnostic accuracy of non-specialists to approach the level of subspecialists for a large array of diseases on brain MRI.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others



Artificial intelligence


Convolutional neural network


Top three differential diagnosis


Top diagnosis


Adaptive Radiology Interpretation and Education System


  1. McDonald RJ, Schwartz KM, Eckel LJ, Diehn FE, Hunt CH, Bartholmai BJ, Erickson BJ, and Kallmes DF. The effects of changes in utilization and technological advancements of cross-sectional imaging on radiologist workload. Acad Radiol 2015;22:1191-8

    Article  PubMed  Google Scholar 

  2. Gunderman RB. Biases in radiologic reasoning. AJR Am J Roentgenol 2009;192:561-4

    Article  PubMed  Google Scholar 

  3. Bruno MA, Walker EA, and Abujudeh HH. Understanding and confronting our mistakes: the epidemiology of error in radiology and strategies for error reduction. Radiographics 2015;35:1668-76

    Article  PubMed  Google Scholar 

  4. Tversky A, and Kahneman D. Availability: a heuristic for judging frequency and probability. Cognitive Psychology 1973;5:207-232

    Article  Google Scholar 

  5. Busby LP, Courtier JL, and Glastonbury CM. Bias in radiology: the how and why of misses and misinterpretations. Radiographics 2018;38:236-247

    Article  PubMed  Google Scholar 

  6. European Society of Radiology (ESR). Summary of the proceedings of the International Summit 2015: general and subspecialty radiology. Insights Imaging 2016;7:1-5

    Article  Google Scholar 

  7. LeCun Y, Bengio Y, and Hinton G. Deep learning. Nature 2015;521:436-44

    Article  CAS  PubMed  Google Scholar 

  8. Chartrand G, Cheng PM, Vorontsov E, Drozdzal M, Turcotte S, Pal CJ, Kadoury S, and Tang A. Deep learning: a primer for radiologists. Radiographics 2017;37:2113-2131

    Article  PubMed  Google Scholar 

  9. Chang PJ, Grinband BD, Weinberg M, Bardis M, Khy G, Cadena MY Su, et al. Deep-Learning Convolutional Neural Networks Accurately Classify Genetic Mutations in Gliomas.” AJNR. American Journal of Neuroradiology 2019;39(7):1201–1207.

  10. Rudie JD, Rauschecker AM, Bryan RN, Davatzikos C, and Mohan S. Emerging applications of artificial intelligence in neuro-oncology. Radiology

  11. Rudie JD, Weiss DA, Saluja R, Rauschecker AM, Wang J, Sugrue L, Bakas S, and Colby JB. Multi-disease segmentation of gliomas and white matter hyperintensities in the BraTS data using a 3D convolutional neural network. Front Comput Neurosci 2019;13:84

    Article  PubMed  PubMed Central  Google Scholar 

  12. Lee H, Yune S, Mansouri M, Kim M, Tajmir SH, Guerrier CE, Ebert SA, Pomerantz SR, Romero JM, Kamalian S, Gonzalez RG, Lev MH, and Do S. An explainable deep-learning algorithm for the detection of acute intracranial haemorrhage from small datasets. Nat Biomed Eng Nature Biomedical Engineering 2018. [[Epub ahead of print]].

  13. Kuo W, Hӓne C, Mukherjee P, Malik J, and Yuh EL. Expert-level detection of acute intracranial hemorrhage on head computed tomography using deep learning. Proc Natl Acad Sci U S A 2019;116:22737-22745

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Duong MT, Rudie JD, Wang J, et al. Convolutional Neural Network for Automated FLAIR Lesion Segmentation on Clinical Brain MR Imaging. AJNR Am J Neuroradiol 2019;40:1282–90.

  15. Pearl J. Probabilistic Reasoning in Intelligent Systems. San Mateo, CA: Morgan Kaufmann; 1988.

    Google Scholar 

  16. Bielza C, and Larrañaga P. Bayesian networks in neuroscience: a survey. Front Comput Neurosci 2014;8:131

    Article  PubMed  PubMed Central  Google Scholar 

  17. Tombropoulos R, Shiffman S, and Davidson C. A decision aid for diagnosis of liver lesions on MRI. Proc Annu Symp Comput Appl Med Care 1993.

  18. Kahn CE, Roberts LM, Shaffer KA, and Haddawy P. Construction of a Bayesian network for mammographic diagnosis of breast cancer. Comput Biol Med 1997;27:19-29.

    Article  PubMed  Google Scholar 

  19. Do BH, Langlotz C, and Beaulieu CF. Bone tumor diagnosis using a naïve Bayesian model of demographic and radiographic features. J Digit Imaging 2017;30:640-647

    Article  PubMed  PubMed Central  Google Scholar 

  20. Kahn CE, Laur JJ, and Carrera GF. A Bayesian network for diagnosis of primary bone tumors. J Digit Imaging 2001;14:56-7.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Hu J, Wu W, Zhu B, Wang H, Liu R, Zhang X, Li M, Yang Y, Yan J, Niu F, Tian C, Wang K, Yu H, Chen W, Wan S, Sun Y, and Zhang B. Correction: cerebral glioma grading using bayesian network with features extracted from multiple modalities of magnetic resonance imaging. PloS One2016;11:e0157095

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Herskovits EH, Bryan RN, and Yang F. Automated Bayesian segmentation of microvascular white-matter lesions in the ACCORD-MIND study. Adv Med Sci 2008;53:182-90

    Article  CAS  PubMed  Google Scholar 

  23. Chen Y, Elenee Argentinis JD, and Weber G. IBM Watson: how cognitive computing can be applied to big data challenges in life sciences research. Clin Ther 2016;38:688-701

    Article  PubMed  Google Scholar 

  24. Duda J, Botzolakis E, Bryan RN, Chen P-H, Cook T, Gee J, Mohan S, Nasrallah I, Rauschecker A, and Rudie J. Bayesian network interface for assisting radiology interpretation and education. Proceedings of SPIE 2018;10579:105790S-105790S-10.

  25. Rauschecker AM, Rudie JD, Xie L, Wang J, Duong MT, Botzolakis EJ, Kovalovich AM, Egan J, Cook TC, Bryan RN, Nasrallah IM, Mohan S, and Gee JC. Artificial intelligence system approaching neuroradiologist-level differential diagnosis accuracy at brain MRI. Radiology

  26. Rudie JD, Rauschecker AM, Xie L, Wang J, Botzolakis EJ, Kovalovich A, Egan J, Cook T, Bryan RN, Nasrallah IM, Mohan S, Gee J. Subspecialty-level deep gray matter differential diagnoses with deep learning and bayesian networks on clinical brain MRI: a pilot study. Radiology: Artificial Intelligence. In Press.

  27. Lindsey R, Daluiski A, Chopra S, Lachapelle A, Mozer M, Sicular S, Hanel D, Gardner M, Gupta A, Hotchkiss R, and Potter H. Deep neural network improves fracture detection by clinicians. Proc Natl Acad Sci USA 2018;115:11591-11596

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Krogue JD, Cheng KV, Hwang KM, Toogood P, Meinberg EG, Geiger EJ, Zaid M, McGill KC, Patel R, Sohn JH, Wright A, Darger BF, Padrez KA, Ozhinsky E, Majumdar S, and Pedoia V. Automatic hip fracture identification and functional subclassification with deep learning. Radiology: Artificial Intelligence. 2020;2:e190023-.

  29. Park A, Chute C, Rajpurkar P, Lou J, Ball RL, Shpanskaya K, Jabarkheel R, Kim LH, McKenna E, Tseng J, Ni J, Wishah F, Wittber F, Hong DS, Wilson TJ, Halabi S, Basu S, Patel BN, Lungren MP, Ng AY, and Yeom KW. Deep learning-assisted diagnosis of cerebral aneurysms using the HeadXNet model. JAMA Netw Open 2019;2:e195600

    Article  PubMed  PubMed Central  Google Scholar 

  30. Bien N, Rajpurkar P, Ball RL, Irvin J, Park A, Jones E, Bereket M, Patel BN, Yeom KW, Shpanskaya K, Halabi S, Zucker E, Fanton G, Amanatullah DF, Beaulieu CF, Riley GM, Stewart RJ, Blankenberg FG, Larson DB, Jones RH, Langlotz CP, Ng AY, and Lungren MP. Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. PLoS Med 2018;15:e1002699

    Article  PubMed  PubMed Central  Google Scholar 

  31. Conant EF, Toledano AY, Periaswamy S, Fotin SV, Go J, Boatsman JE, and Hoffmeister JW. Improving accuracy and efficiency with concurrent use of artificial intelligence for digital breast tomosynthesis. Radiol Artif Intell 2019;1:e180096

    Article  PubMed  PubMed Central  Google Scholar 

  32. Schaffter T, Buist DSM, Lee CI, Nikulin Y, Ribli D, Guan Y, Lotter W, Jie Z, Du H, Wang S, Feng J, Feng M, Kim HE, Albiol F, Albiol A, Morrell S, Wojna Z, Ahsen ME, Asif U, Jimeno Yepes A, Yohanandan S, Rabinovici-Cohen S, Yi D, Hoff B, Yu T, Chaibub Neto E, Rubin DL, Lindholm P, Margolies LR, McBride RB, Rothstein JH, Sieh W, Ben-Ari R, Harrer S, Trister A, Friend S, Norman T, Sahiner B, Strand F, Guinney J, Stolovitzky G, Mackey L, Cahoon J, Shen L, Sohn JH, Trivedi H, Shen Y, Buturovic L, Pereira JC, Cardoso JS, Castro E, Kalleberg KT, Pelka O, Nedjar I, Geras KJ, Nensa F, Goan E, Koitka S, Caballero L, Cox DD, Krishnaswamy P, Pandey G, Friedrich CM, Perrin D, Fookes C, Shi B, Cardoso Negrie G, Kawczynski M, Cho K, Khoo CS, Lo JY, Sorensen AG, and Jung H. Evaluation of combined artificial intelligence and radiologist assessment to interpret screening mammograms. JAMA Netw Open 2020;3:e200265

    Article  PubMed  PubMed Central  Google Scholar 

  33. Das SR, Avants BB, Grossman M, and Gee JC. Registration based cortical thickness measurement. Neuroimage 2009;45:867-79

    Article  PubMed  Google Scholar 

  34. Avants BB, Tustison NJ, Song G, Cook PA, Klein A, and Gee JC. A reproducible evaluation of ANTs similarity metric performance in brain image registration. Neuroimage 2011;54:2033-44

    Article  PubMed  Google Scholar 

  35. Smith SM, Jenkinson M, Woolrich MW, Beckmann CF, Behrens TE, Johansen-Berg H, Bannister PR, De Luca M, Drobnjak I, Flitney DE, Niazy RK, Saunders J, Vickers J, Zhang Y, De Stefano N, Brady JM, and Matthews PM. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage 2004;23 Suppl 1:S208-19

    Article  PubMed  Google Scholar 

  36. Parasuraman R, Sheridan TB, and Wickens CD. A model for types and levels of human interaction with automation. IEEE Trans Syst Man Cybern A Syst Hum 2000;30:286-97.

    Article  CAS  PubMed  Google Scholar 

  37. Duong MT, Rauschecker AM, Rudie JD, Chen PH, Cook TS, Bryan RN, and Mohan S. Artificial intelligence for precision education in radiology. Br J Radiol 2019;92:20190389

    Article  PubMed  PubMed Central  Google Scholar 

  38. Zaharchuk G, Gong E, Wintermark M, Rubin D, and Langlotz CP. Deep learning in neuroradiology. AJNR Am J Neuroradiol 2018;39:1-9

    Article  Google Scholar 

  39. Al Yassin A, Salehi Sadaghiani M, Mohan S, Bryan RN, and Nasrallah I. It is about “time”: academic neuroradiologist time distribution for interpreting brain MRIs. Acad Radiol 2018;25:1521-1525

    Article  PubMed  Google Scholar 

Download references


Financial support for this project was provided by an RSNA Resident Research grant (AMR; RR1778). AMR and JDR were also supported by institutional T-32 Training Grants (Penn T32-EB004311-10 and UCSF T32-EB001631-14). The NVIDIA corporation donated two Titan Xp GPUs as part of the NVIDIA GPU grant program (JDR, AMR).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Jeffrey D. Rudie.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rudie, J.D., Duda, J., Duong, M.T. et al. Brain MRI Deep Learning and Bayesian Inference System Augments Radiology Resident Performance. J Digit Imaging 34, 1049–1058 (2021).

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: