Skip to main content

Advertisement

SpringerLink
Log in

Artificial intelligence vs. radiologist: accuracy of wrist fracture detection on radiographs

  • Musculoskeletal
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objective

To compare the performances of artificial intelligence (AI) to those of radiologists in wrist fracture detection on radiographs.

Methods

This retrospective study included 637 patients (1917 radiographs) with wrist trauma between January 2017 and December 2019. The AI software used was a deep neuronal network algorithm. Ground truth was established by three senior musculoskeletal radiologists who compared the initial radiology reports (IRR) made by non-specialized radiologists, the results of AI, and the combination of AI and IRR (IR+AI)

Results

A total of 318 fractures were reported by the senior radiologists in 247 patients. Sensitivity of AI (83%; 95% CI: 78–87%) was significantly greater than that of IRR (76%; 95% CI: 70–81%) (p < 0.001). Specificities were similar for AI (96%; 95% CI: 93–97%) and for IRR (96%; 95% CI: 94–98%) (p = 0.80). The combination of AI+IRR had a significantly greater sensitivity (88%; 95% CI: 84–92%) compared to AI and IRR (p < 0.001) and a lower specificity (92%; 95% CI: 89–95%) (p < 0.001). The sensitivity for scaphoid fracture detection was acceptable for AI (84%) and IRR (80%) but poor for the detection of other carpal bones fracture (41% for AI and 26% for IRR).

Conclusions

Performance of AI in wrist fracture detection on radiographs is better than that of non-specialized radiologists. The combination of AI and radiologist’s analysis yields best performances.

Key Points

• Artificial intelligence has better performances for wrist fracture detection compared to non-expert radiologists in daily practice.

• Performance of artificial intelligence greatly differs depending on the anatomical area.

• Sensitivity of artificial intelligence for the detection of carpal bones fractures is 56%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

AI:

Artificial intelligence

CI:

Confidence interval

CT:

Computed tomography

IRR:

Initial radiology report

MRI:

Magnetic resonance imaging

MSK:

Musculoskeletal

NPV:

Negative predictive value

PPV:

Predictive positive value

References

  1. OECD Health statistics (2015) In: European Society of Radiology. https://doi.org/10.1787/health-data-en. Accessed 27 Apr 2020

  2. The Latest Emergency Department Utilization Numbers Are In. In: ACEP Now. https://www.acepnow.com/article/the-latest-emergency-department-utilization-numbers-are-in/. Accessed 27 Apr 2020

  3. Kim DH, MacKinnon T (2018) Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks. Clin Radiol 73:439–445

  4. Kalmet PHS, Sanduleanu S, Primakov S, Wu G, Jochems A, Refaee T, Ibrahim A, Hulst Lv, Lambin P, Poeze M (2020) Deep learning in fracture detection: a narrative review. Acta Orthop 91:215-

  5. Leeper WR, Leeper TJ, Vogt KN, Charyk-Stewart T, Gray DK, Parry NG (2013) The role of trauma team leaders in missed injuries: does specialty matter? J Trauma Acute Care Surg 75:387–390

    Article  PubMed  Google Scholar 

  6. Nakaura T, Higaki T, Awai K, Ikeda O, Yamashita Y (2020) A primer for understanding radiology articles about machine learning and deep learning. Diagn Interv Imaging 101:765–770

    Article  PubMed  Google Scholar 

  7. Roca P, Attye A, Colas L et al (2020) Artificial intelligence to predict clinical disability in patients with multiple sclerosis using FLAIR MRI. Diagn Interv Imaging 101:795–802

    Article  CAS  PubMed  Google Scholar 

  8. Gulshan V, Peng L, Coram M et al (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316:2402–2410

    Article  PubMed  Google Scholar 

  9. Esteva A, Kuprel B, Novoa RA et al (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115–118

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Sirinukunwattana K, Ahmed Raza SE, Tsang YW nul, Snead DRJ, Cree IA, Rajpoot NM (2016) Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging 35:1196–1206

  11. Cireşan DC, Giusti A, Gambardella LM, Schmidhuber J (2013) Mitosis detection in breast cancer histology images with deep neural networks. Med Image Comput Comput Assist Interv 16:411–418

    PubMed  Google Scholar 

  12. Ye J, Li Q, Nie J (2022) Prevalence, characteristics, and associated risk factors of wrist fractures in Americans above 50: the cross-sectional NHANES study. Front Endocrinol 13:800129

    Article  Google Scholar 

  13. Curtis JR, Arora T, Matthews RS et al (2010) Is withholding osteoporosis medication after fracture sometimes rational? A comparison of the risk for second fracture versus death. J Am Med Dir Assoc 11:584–591

    Article  PubMed  PubMed Central  Google Scholar 

  14. Lindsey R, Daluiski A, Chopra S et al (2018) Deep neural network improves fracture detection by clinicians. Proc Natl Acad Sci U S A 115:11591–11596

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Gan K, Xu D, Lin Y et al (2019) Artificial intelligence detection of distal radius fractures: a comparison between the convolutional neural network and professional assessments. Acta Orthop 90:394–400

    Article  PubMed  PubMed Central  Google Scholar 

  16. Chung SW, Han SS, Lee JW et al (2018) Automated detection and classification of the proximal humerus fracture by using deep learning algorithm. Acta Orthop 89:468–473

    Article  PubMed  PubMed Central  Google Scholar 

  17. Adams M, Chen W, Holcdorf D, McCusker MW, Howe PD, Gaillard F (2019) Computer vs human: deep learning versus perceptual training for the detection of neck of femur fractures. J Med Imaging Radiat Oncol 63:27–32

    Article  PubMed  Google Scholar 

  18. Dupuis M, Delbos L, Veil R, Adamsbaum C (2022) External validation of a commercially available deep learning algorithm for fracture detection in children. Diagn Interv Imaging 103:151–159

    Article  PubMed  Google Scholar 

  19. Etli I, Kozaci N, Avci M, Karakoyun OF (2020) Comparison of the diagnostic accuracy of X-ray and computed tomography in patients with wrist injury. Injury 51:651-655

  20. Dias JJ, Thompson J, Barton NJ, Gregg PJ (1990) Suspected scaphoid fractures: the value of radiographs. J Bone Joint Surg Br 72:98–101

    Article  CAS  PubMed  Google Scholar 

  21. Fowler JR, Hughes TB (2015) Scaphoid fractures. Clin Sports Med 34:37–50

    Article  PubMed  Google Scholar 

  22. Olczak J, Fahlberg N, Maki A et al (2017) Artificial intelligence for analyzing orthopedic trauma radiographs. Acta Orthop 88:581–586

    Article  PubMed  PubMed Central  Google Scholar 

  23. Foufi V, Lanteri S, Gaudet-Blavignac C, Remy P, Montet X, Lovis C (2018) Automatic annotation tool to support supervised machine learning for scaphoid fracture detection. Stud Health Technol Inform 255:210–214

    PubMed  Google Scholar 

  24. Yoon AP, Lee Y-L, Kane RL, Kuo C-F, Lin C, Chung KC (2021) Development and validation of a deep learning model using convolutional neural networks to identify scaphoid fractures in radiographs. JAMA Netw Open 4:e216096

  25. Duron L, Ducarouge A, Gillibert A et al (2021) Assessment of an AI aid in detection of adult appendicular skeletal fractures by emergency physicians and radiologists: a multicenter cross-sectional diagnostic study. Radiology 300:120–129

    Article  PubMed  Google Scholar 

  26. Guermazi A, Tannoury C, Kompel AJ et al (2022) Improving radiographic fracture recognition performance and efficiency using artificial intelligence. Radiology 302:627–636

    Article  PubMed  Google Scholar 

  27. Welling RD, Jacobson JA, Jamadar DA, Chong S, Caoili EM, Jebson PJL (2008) MDCT and radiography of wrist fractures: radiographic sensitivity and fracture patterns. AJR Am J Roentgenol 190:10–16

  28. Ozkaya E, Topal FE, Bulut T, Gursoy M, Ozuysal M, Karakaya Z (2022) Evaluation of an artificial intelligence system for diagnosing scaphoid fracture on direct radiography. Eur J Trauma Emerg Surg 48:585–592

    Article  PubMed  Google Scholar 

  29. Langerhuizen DWG, Bulstra AEJ, Janssen SJ et al (2020) Is deep learning on par with human observers for detection of radiographically visible and occult fractures of the scaphoid? Clin Orthop Relat Res 478:2653–2659

    Article  PubMed  PubMed Central  Google Scholar 

  30. Yang T-H, Horng M-H, Li R-S, Sun Y-N (2022) Scaphoid fracture detection by using convolutional neural network. Diagnostics 12:895

    Article  PubMed  PubMed Central  Google Scholar 

  31. Hendrix N, Scholten E, Vernhout B et al (2021) Development and validation of a convolutional neural network for automated detection of scaphoid fractures on conventional radiographs. Radiol Artif Intell 3:e200260

  32. Berbaum KS, Franken EA, Dorfman DD et al (1990) Satisfaction of search in diagnostic radiology. Invest Radiol 25:133–140

    Article  CAS  PubMed  Google Scholar 

  33. Zhou QQ, Tang W, Wang J et al (2021) Automatic detection and classification of rib fractures based on patients’ CT images and clinical information via convolutional neural network. Eur Radiol 31:3815–3825

    Article  PubMed  Google Scholar 

Download references

Funding

The authors state that this work has not received any funding.

Author information

Authors and Affiliations

  1. Department of Radiology - Hotel Dieu Hospital, Assistance Publique-Hopitaux de Paris, Paris, France

    Mathieu Cohen, Julien Puntonet, Michel Crema & Elisabeth Dion

  2. Université Paris Cité, F-75006, Paris, France

    Mathieu Cohen, Julien Puntonet, Julien Sanchez, Elliott Kierszbaum, Philippe Soyer & Elisabeth Dion

  3. Institute of Sports Imaging, French National Institute of Sports (INSEP), Paris, France

    Julien Sanchez & Michel Crema

  4. Department of Radiology– Cochin Hospital, Assistance Publique-Hopitaux de Paris, 75014, Paris, France

    Philippe Soyer

Authors
  1. Mathieu Cohen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Julien Puntonet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Julien Sanchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elliott Kierszbaum
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michel Crema
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Philippe Soyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Elisabeth Dion
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julien Puntonet.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Julien Puntonet.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Methodology

• retrospective

• diagnostic study

• performed at one institution

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

ESM 1

(DOCX 935 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cohen, M., Puntonet, J., Sanchez, J. et al. Artificial intelligence vs. radiologist: accuracy of wrist fracture detection on radiographs. Eur Radiol 33, 3974–3983 (2023). https://doi.org/10.1007/s00330-022-09349-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-022-09349-3

Keywords

Search

Navigation