Skip to main content
SpringerLink
Log in

Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations

  • Imaging Informatics and Artificial Intelligence
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Artificial intelligence (AI) has the potential to significantly disrupt the way radiology will be practiced in the near future, but several issues need to be resolved before AI can be widely implemented in daily practice. These include the role of the different stakeholders in the development of AI for imaging, the ethical development and use of AI in healthcare, the appropriate validation of each developed AI algorithm, the development of effective data sharing mechanisms, regulatory hurdles for the clearance of AI algorithms, and the development of AI educational resources for both practicing radiologists and radiology trainees. This paper details these issues and presents possible solutions based on discussions held at the 2019 meeting of the International Society for Strategic Studies in Radiology.

Key Points

• Radiologists should be aware of the different types of bias commonly encountered in AI studies, and understand their possible effects.

• Methods for effective data sharing to train, validate, and test AI algorithms need to be developed.

• It is essential for all radiologists to gain an understanding of the basic principles, potentials, and limits of AI.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Abbreviations

ACR:

American College of Radiology

AI:

Artificial intelligence

CADe:

Computer-assisted detection devices

CADt:

Computer-assisted triage

CADx:

Computer-assisted diagnosis

CME:

Continuing medical education

EU:

European Union

FAT:

Fairness, accountability, and transparency

FDA:

Food and Drug Administration

GUIDE-IT:

Guide to Data Sharing of Imaging Trials

VOICE:

Value of Imaging through Comparative Effectiveness

References

  1. Sizing the prize. https://www.pwc.com/gx/en/issues/analytics/assets/pwc-ai-analysis-sizing-the-prize-report.pdf. Accessed 4 Oct 2019

  2. Harris S (2018) AI in medical imaging to top $2 billion by 2023. https://www.signifyresearch.net/medical-imaging/ai-medical-imaging-top-2-billion-2023/. Accessed 4 Oct 2019

  3. Chockley K, Emanuel E (2016) The end of radiology? Three threats to the future practice of radiology. J Am Coll Radiol 13(12):1415–1420

    Article  Google Scholar 

  4. Obermeyer Z, Emanuel EJ (2016) Predicting the future—big data, machine learning, and clinical medicine. N Engl J Med 375(13):1216

    Article  Google Scholar 

  5. Langlotz CL (2019) Will artificial intelligence replace radiologists? Radiology: Artificial Intelligence 1(3):e190058

    Google Scholar 

  6. Prainsack B (2017) Personalized medicine: empowered patients in the 21st century? New York University Press 100

  7. Jha S, Topol EJ (2018) Information and artificial intelligence. J Am Coll Radiol 15(3):509–511

    Article  Google Scholar 

  8. Hagendorff T (2019) The ethics of AI ethics--an evaluation of guidelines. arXiv:1903.03425

  9. Schönberger D (2019) Artificial intelligence in healthcare: a critical analysis of the legal and ethical implications. Int J Law Info Tech 27(2):171–203

    Article  Google Scholar 

  10. Anderson M, Anderson SL (2019) How should AI be developed, validated, and implemented in patient care? AMA J Ethics 21(2):E125–E130

    Article  Google Scholar 

  11. Osoba OA, Welser W IV (2017) An intelligence in our image: The risks of bias and errors in artificial intelligence. https://www.rand.org/pubs/research_reports/RR1744.html. Accessed 4 Oct 2019

  12. Balthazar P, Harri P, Prater A, Safdar NM (2018) Protecting your patients’ interests in the era of big data, artificial intelligence, and predictive analytics. J Am Coll Radiol 15(3):580–586

    Article  Google Scholar 

  13. Wachter S, Mittelstadt B (2019) A right to reasonable inferences: re-thinking data protection law in the age of big data and AI. Colum Bus L Rev. https://doi.org/10.31228/osf.io/mu2kf

  14. Wachter S, Mittelstadt B, Russell C (2017) Counterfactual explanations without opening the black box: automated decisions and the GPDR. Harv JL Tech 31:841

    Google Scholar 

  15. Cath C (2018) Governing artificial intelligence: ethical, legal and technical opportunities and challenges. Phil Trans R Soc A 376. https://doi.org/10.1098/rsta.2018.0080

    Article  Google Scholar 

  16. Rigby MJ (2019) Ethical dimensions of using artificial intelligence in health care. AMA J Ethics 21(2):E121–E124

    Article  Google Scholar 

  17. Thierer A, O’Sullivan A, Russel R (2017) Artificial intelligence and public policy. Mercatus Research Paper. Available via https://www.mercatus.org/system/files/thierer-artificial-intelligence-policy-mr-mercatus-v1.pdf. Accessed 4 Oct 2019

  18. Starmans MP, van der Voort SR, Tovar JMC, Veenland JF, Klein S, Niessen WJ (2020) Radiomics: data mining using quantitative medical image features. In: Zhou KS, Rueckert D, Fichtinger G (eds) Handbook of medical image computing and computer assisted intervention. Academic Press, pp 429–456

  19. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann (2018) Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLoS Med 15(11):e1002683. https://doi.org/10.1371/journal.pmed.1002683

    Article  Google Scholar 

  20. www.grand-challenge.org. Accessed 19 Oct 2019

  21. https://competitions.codalab.org/competitions/17094. Accessed 19 Oct 2019

  22. Maier-Hein L, Eisenmann M, Reinke A et al (2018) Why rankings of biomedical image analysis competitions should be interpreted with care. Nat Commun 9(1):5217

    Article  CAS  Google Scholar 

  23. Halabi SS, Prevedello LM, Kalpathy-Cramer J et al (2019) The RSNA pediatric bone age machine learning challenge. Radiology 290(2):498–503

    Article  Google Scholar 

  24. https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation/overview. Accessed 19 Oct 2019

  25. https://www.acrdsi.org/DSI-Services/TOUCH-AI. Accessed 19 Oct 2019

  26. Naaktgeboren CA, de Groot JA, van Smeden M, Moons KG, Reitsma JB (2013) Evaluating diagnostic accuracy in the face of multiple reference standards. Ann Intern Med 159(3):195–202

    Article  Google Scholar 

  27. Whiting P, Rutjes AWS, Reitsma JB, Glas AS, Bossuyt PMM, Kleijnen J (2004) Sources of variation and bias in studies of diagnostic accuracy: a systematic review. Ann Intern Med 140(3):189–202

    Article  Google Scholar 

  28. Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SC Jr (2009) Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA 301:831–84128

    Article  CAS  Google Scholar 

  29. Leung KY, van der Lijn F, Vrooman HA, Sturkenboom MC, Niessen WJ (2015) IT Infrastructure to support the secondary use of routinely acquired clinical imaging data for research. Neuroinformatics 13(1):65–81

    Article  Google Scholar 

  30. Bosserdt M, Hamm B, Dewey M (2019) Clinical trials in radiology and data sharing: results from a survey of the European Society of Radiology (ESR) research committee. Eur Radiol. https://doi.org/10.1007/s00330-019-06105-y

    Article  Google Scholar 

  31. Dewey M, Bosserdt M, Dodd JD, Thun S, Kressel HY (2019) Clinical imaging research: higher evidence, global collaboration, improved reporting, and data sharing are the grand challenges. Radiology 291:547–552

    Article  Google Scholar 

  32. Alzheimer’s Disease Neuroimaging Initiative. Available via http://adni.loni.usc.edu/. Accessed 19 Oct 2019

  33. The Global Alzheimer’s Association Interactive Network. Available via http://www.gaain.org/. Accessed 19 Oct 2019

  34. Cancer Imaging Archive. Available via https://www.cancerimagingarchive.net/. Accessed 19 Oct 2019

  35. National Cancer Institute. Available via https://imaging.nci.nih.gov/ncia/login.jsf. Accessed 19 Oct 2019

  36. Haase R, Schlattmann P, Gueret P et al (2019) Diagnosis of obstructive coronary artery disease using computed tomography angiography in patients with stable chest pain: meta-analysis of individual patient data. BMJ 365:I1945

    Article  Google Scholar 

  37. Venema E, Mulder MJHL, Roozenbeek B et al (2017) Selection of patients for intra-arterial treatment for acute ischaemic stroke: development and validation of a clinical decision tool in two randomised trials. BMJ 357:j1710. https://doi.org/10.1136/bmj.j1710

    Article  PubMed  PubMed Central  Google Scholar 

  38. Adamson PD, Fordyce CB, McAllister DA, Udelson JE, Douglas PS, Newby DE (2018) Identification of patients with stable chest pain deriving minimal value from coronary computed tomography angiography: an external validation of the PROMISE minimal-risk tool. Int J Cardiol 252:31–34

    Article  Google Scholar 

  39. Tajmir SH, Alkasab TK (2018) Toward augmented radiologists: changes in radiology education in the era of machine learning and artificial intelligence. Acad Radiol 25(6):747–750

    Article  Google Scholar 

  40. Wood MJ, Teneholtz NA, Geis JR, Michalski MH, Andriole KP (2019) The need for a machine learning curriculum for radiologists. J Am Coll Radiol 16(5):740–742

    Article  Google Scholar 

  41. Nguyen GK, Shetty AS (2018) Artificial intelligence and machine learning: opportunities for radiologists in training. J Am Coll Radiol 15(9):1320–1321

    Article  Google Scholar 

  42. Collado-Mesa F, Alvarez E, Arheart K (2018) The role of artificial intelligence in diagnostic radiology: a survey at a single radiology residency training program. J Am Coll Radiol 15(12):1753–1757

    Article  Google Scholar 

  43. Wilson HJ, Daugherty PR (2018) Collaborative intelligence: humans and AI are joining forces. Harv Bus Rev 96(4):114–123

  44. American College of Radiology Data Science Institute. Available via https://www.acrdsi.org/. Accessed 19 Oct 2019

  45. Radiology informatics committee. Available via https://www2.rsna.org/timssnet/About/committee.cfm?c=C0002103. Accessed 19 Oct 2019

  46. Artificial Intelligence at ECR. Available via https://www.myesr.org/ai. Accessed 20 Oct 2019

  47. National Imaging Informatics Curriculum and Course. Available at https://sites.google.com/view/imaging-informatics-course/home. Accessed 29 July 2019

  48. Voice Course for Comparative Effectiveness. Available at https://www.rsna.org/en/education/workshops/VOICE-program. Accessed 29 July 2019

  49. Wartman SA, Combs CD (2018) Medical education must move from the information age to the age of artificial intelligence. Acad Med 93(8):1107–1109

    Article  Google Scholar 

  50. Popenici SAD, Kerr S (2017) Exploring the impact of artificial intelligence on teaching and learning in higher education. Res Pract Technol Enhanc Learn 12(1):22

    Article  Google Scholar 

  51. How to Determine if Your Product is a Medical Device. US Food & Drug Administration. Available via https://www.fda.gov/medical-devices/classify-your-medical-device/product-medical-device. Accessed 19 Oct 2019

  52. Artificial Intelligence and Machine Learning in Software as a Medical Device. US Food & Drug Administration. Available via https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device. Accessed 19 Oct 2019

  53. European Economic Community (1993) 93/42/EEC - Council Directive concerning medical devices. Official Journal of the European Communities. Available via http://ec.europa.eu/growth/single-market/european-standards/harmonised-standards/medical-devices_en. Accessed 19 Oct 2019

  54. European Commission (2018) MDCG 2018–2 Future EU medical device nomenclature – description of requirements. Available via https://ec.europa.eu/docsroom/documents/28668. Accessed 19 Oct 2019

  55. Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC. Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32017R0745. Accessed 19 Oct 2019

  56. Pesapane F, Volonté C, Codari M, Sardanelli F (2018) Artificial intelligence as a medical device in radiology: ethical and regulatory issues in Europe and the United States. Insights Imaging 9(5):745–753 747

    Article  Google Scholar 

  57. European Group on Ethics in Science and New Technologies (2018) Statement on artificial intelligence, robotics and autonomous systems. Brussels. Available at: http://ec.europa.eu/research/ege/pdf/ege_ai_statement_2018.pdf. Accessed 19 Oct 2019

  58. High-Level Expert Group on Artificial Intelligence (2019) Ethics guidelines for trustworthy AI. Brussels. Available at: https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai. Accessed 19 Oct 2019

Download references

Acknowledgments

The authors would like to acknowledge Herbert Y. Kressel, MD, for his advice during the preparation of this manuscript.

Funding

The authors state that this work has not received any funding.

Author information

Authors and Affiliations

  1. Department of Radiology, New York University Robert I Grossman School of Medicine, New York, NY, USA

    Michael P. Recht

  2. Charité – Universitätsmedizin Berlin, Humboldt-Universität and Freie Universität zu Berlin, Berlin, Germany

    Marc Dewey

  3. Department of Radiology, Massachusetts General Hospital, Boston, MA, USA

    Keith Dreyer

  4. Department of Radiology and Biomedical Informatics, Stanford University, Palo Alto, CA, USA

    Curtis Langlotz

  5. Department of Radiology, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands

    Wiro Niessen

  6. Faculty of Applied Sciences, Delft University of Technology, Delft, The Netherlands

    Wiro Niessen

  7. Department of Political Science, University of Vienna, Vienna, Austria

    Barbara Prainsack

  8. Department of Global Health & Social Medicine, King’s College, London, UK

    Barbara Prainsack

  9. Hogan Lovells US LLP, Washington, D.C., USA

    John J. Smith

Authors
  1. Michael P. Recht
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marc Dewey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Keith Dreyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Curtis Langlotz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wiro Niessen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Barbara Prainsack
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. John J. Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael P. Recht.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Michael Recht.

Conflict of interest

Dr. Langlotz reports non-financial support from Nines.ai; personal fees and non-financial support from whiterabbit.ai; non-financial support from Galileo CDS, Inc.; non-financial support from Bunker Hill, Inc.; grants from GE Healthcare; grants from Philips Healthcare; and grants from Siemens Healthineers, outside the submitted work.

Prof. Dewey has received grant support from the FP7 Program of the European Commission for the randomized multicenter DISCHARGE trial (603266-2, HEALTH-2012.2.4.-2). He also received grant support from German Research Foundation (DFG) in the Heisenberg Program (DE 1361/14-1); from graduate program on quantitative biomedical imaging (BIOQIC, GRK 2260/1); for fractal analysis of myocardial perfusion (DE 1361/19-1); from the Priority Programme Radiomics (DE 1361/19-1 and 20-1 in SPP 2177/1). He also received funding from the Berlin University Alliance and from the Digital Health Accelerator of the Berlin Institute of Health. Prof. Dewey has received lecture fees from Canon, Guerbet, Cardiac MR Academy Berlin, and Bayer. Prof. Dewey is also the editor of Cardiac CT, published by Springer, and offers hands-on courses on CT imaging (www.ct-kurs.de). Institutional master research agreements exist with Siemens, General Electric, Philips, and Canon. The terms of these arrangements are managed by the legal department of Charité – Universitätsmedizin Berlin. Professor Dewey holds a joint patent with Florian Michallek on dynamic perfusion analysis using fractal analysis (PCT/EP2016/071551).

John Smith is a partner at Hogan Lovells US LLP and represents medical device companies, including those involved in medical imaging, before FDA.

Dr. Niessen reports other from Quantib, outside the submitted work.

Dr. Recht reports collaborative research agreements with Facebook Artificial Intelligence Research and Collaboration with Amazon Web Services Public Dataset Program, outside the submitted work.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Not applicable

Ethical approval

Institutional Review Board approval was not required as this is a special report.

Methodology

None as this is a special report.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Recht, M.P., Dewey, M., Dreyer, K. et al. Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations. Eur Radiol 30, 3576–3584 (2020). https://doi.org/10.1007/s00330-020-06672-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-020-06672-5

Keywords

Search

Navigation