Skip to main content

Advertisement

SpringerLink
Log in

External Validation of Deep Learning Algorithm for Detecting and Visualizing Femoral Neck Fracture Including Displaced and Non-displaced Fracture on Plain X-ray

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

This study aimed to develop a method for detection of femoral neck fracture (FNF) including displaced and non-displaced fractures using convolutional neural network (CNN) with plain X-ray and to validate its use across hospitals through internal and external validation sets. This is a retrospective study using hip and pelvic anteroposterior films for training and detecting femoral neck fracture through residual neural network (ResNet) 18 with convolutional block attention module (CBAM) +  + . The study was performed at two tertiary hospitals between February and May 2020 and used data from January 2005 to December 2018. Our primary outcome was favorable performance for diagnosis of femoral neck fracture from negative studies in our dataset. We described the outcomes as area under the receiver operating characteristic curve (AUC), accuracy, Youden index, sensitivity, and specificity. A total of 4,189 images that contained 1,109 positive images (332 non-displaced and 777 displaced) and 3,080 negative images were collected from two hospitals. The test values after training with one hospital dataset were 0.999 AUC, 0.986 accuracy, 0.960 Youden index, and 0.966 sensitivity, and 0.993 specificity. Values of external validation with the other hospital dataset were 0.977, 0.971, 0.920, 0.939, and 0.982, respectively. Values of merged hospital datasets were 0.987, 0.983, 0.960, 0.973, and 0.987, respectively. A CNN algorithm for FNF detection in both displaced and non-displaced fractures using plain X-rays could be used in other hospitals to screen for FNF after training with images from the hospital of interest.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

FNF:

Femoral neck fracture

CNN:

Convolution neural network

ResNet:

Residual neural network

CBAM:

Convolutional block attention module

AUC:

Area under the receiver operating characteristic curve

CT:

Computed tomography

MRI:

Magnetic resonance imaging

AP:

Anteroposterior

ICD code:

International Classification of Diseases code

PACS:

Picture archiving and communication system

ROC curve:

Receiver operating characteristic curve

Grad-CAM:

Gradient-weighted Class Activation Mapping

IQR:

Interquartile ranges

SD:

Standard deviation

CI:

Confidence interval

References

  1. Zuckerman JD: Hip fracture. N Engl J Med 334(23):1519-1525, 1996

    Article  CAS  Google Scholar 

  2. Cummings SR, Rubin SM, Black D: The Future of Hip-Fractures in the United-States - Numbers, Costs, and Potential Effects of Postmenopausal Estrogen. Clin Orthop Relat Res (252):163-166, 1990

    Google Scholar 

  3. Melton LJ: Hip fractures: A worldwide problem today and tomorrow. Bone 14:1-8, 1993

    Article  Google Scholar 

  4. Cannon J, Silvestri S, Munro M: Imaging choices in occult hip fracture. J Emerg Med 37(2):144-152, 2009

    Article  Google Scholar 

  5. Richmond J, Aharonoff GB, Zuckerman JD, Koval KJ: Mortality risk after hip fracture. J Orthop Trauma 17(1):53-56, 2003

    Article  Google Scholar 

  6. LeBlanc KE, Muncie HL Jr., LeBlanc LL: Hip fracture: diagnosis, treatment, and secondary prevention. Am Fam Physician 89(12):945-951, 2014

    PubMed  Google Scholar 

  7. Hip Fracture. OrthoInfo from the American Academy of Orthopaedic Surgeons. Available at https://orthoinfo.aaos.org/en/diseases--conditions/hip-fractures. Accessed 10 November 2020

  8. Zuckerman JD, Skovron ML, Koval KJ, Aharonoff G, Frankel VH: Postoperative complications and mortality associated with operative delay in older patients who have a fracture of the hip. J Bone Joint Surg Am 77(10):1551-1556, 1995

    Article  CAS  Google Scholar 

  9. Rudman N, McIlmail D: Emergency Department Evaluation and Treatment of Hip and Thigh Injuries. Emerg Med Clin North Am. 18(1):29-66, 2000

    Article  CAS  Google Scholar 

  10. Bottle A, Aylin P: Mortality associated with delay in operation after hip fracture: observational study. BMJ 332(7547):947-951, 2006

    Article  Google Scholar 

  11. Perron AD, Miller MD, Brady WJ: Orthopedic pitfalls in the ED: radiographically occult hip fracture. Am J Emerg Med 20(3):234-237, 2002

    Article  Google Scholar 

  12. Parker MJ: Missed hip fractures. Arch Emerg Med 9(1):23-27, 1992

    Article  CAS  Google Scholar 

  13. Rizzo PF, Gould ES, Lyden JP, Asnis SE: Diagnosis of occult fractures about the hip. Magnetic resonance imaging compared with bone-scanning. J Bone Joint Surg Am 75(3):395–401, 1993

  14. Jordan RW, Dickenson E, Baraza N, Srinivasan K: Who is more accurate in the diagnosis of neck of femur fractures, radiologists or orthopaedic trainees?. Skeletal Radiol 42(2):173-176, 2013

    Article  CAS  Google Scholar 

  15. Labza S, Fassola I, Kunz B, Ertel W, Krasnici S: Delayed recognition of an ipsilateral femoral neck and shaft fracture leading to preventable subsequent complications: a case report. Patient Saf Surg 11:20, 2017

    Article  Google Scholar 

  16. Lubovsky O, Liebergall M, Mattan Y, Weil Y, Mosheiff R: Early diagnosis of occult hip fractures MRI versus CT scan. Injury 36(6):788-792, 2005

    Article  CAS  Google Scholar 

  17. Deleanu B, Prejbeanu R, Tsiridis E, Vermesan D, Crisan D, Haragus H, et al: Occult fractures of the proximal femur: imaging diagnosis and management of 82 cases in a regional trauma center. World J Emerg Surg 10:55, 2015

    Article  Google Scholar 

  18. Thomas RW, Williams HL, Carpenter EC, Lyons K: The validity of investigating occult hip fractures using multidetector CT. Br J Radiol 89(1060):20150250, 2016

    Article  Google Scholar 

  19. Mandell JC, Weaver MJ, Khurana B: Computed tomography for occult fractures of the proximal femur, pelvis, and sacrum in clinical practice: single institution, dual-site experience. Emerg Radiol 25(3):265-273, 2018

    Article  Google Scholar 

  20. Adams M., Chen W, Holcdorf D, McCusker MW, Howe PD, Gaillard F: Computer vs human: Deep learning versus perceptual training for the detection of neck of femur fractures. J Med Imaging Radiat Oncol 63(1):27-32, 2019

    Article  Google Scholar 

  21. Cheng CT, Ho TY, Lee TY, Chang CC, Chou CC, Chen CC, et al: Application of a deep learning algorithm for detection and visualization of hip fractures on plain pelvic radiographs. Eur Radiol 29(10):5469-5477, 2019

    Article  Google Scholar 

  22. Urakawa T, Tanaka Y, Goto S, Matsuzawa H, Watanabe K, Endo N: Detecting intertrochanteric hip fractures with orthopedist-level accuracy using a deep convolutional neural network. Skeletal Radiol 48(2):239-244, 2019

    Article  Google Scholar 

  23. Badgeley MA, Zech JR, Oakden-Rayner L, Glicksberg BS, Liu M, Gale W, et al: Deep learning predicts hip fracture using confounding patient and healthcare variables. NPJ Digit Med 2:31, 2019

    Article  Google Scholar 

  24. Krogue JD, Cheng KV, Hwang, KM, Toogood P, Meinberg EG, Geiger, EJ, et al: Automatic Hip Fracture Identification and Functional Subclassification with Deep Learning. Radiol Artif Intell 25(2):e190023, 2020

  25. Mutasa S, Varada S, Goel A, Wong TT, Rasiej MJ: Advanced Deep Learning Techniques Applied to Automated Femoral Neck Fracture Detection and Classification. J Digit Imaging 33(5):1209-1217, 2020

    Article  Google Scholar 

  26. International Statistical Classification of Diseases and Related Health Problems 10th Revision. Available at https://icd.who.int/browse10/2019/en. Accessed 10 November 2020

  27. Garden RS: Low-Angle Fixation in Fractures of the Femoral Neck. Journal of Bone and Joint Surgery-British Volume 43(4):647-663, 1961

    Article  Google Scholar 

  28. Frandsen PA, Andersen E, Madsen F, Skjødt T: Garden's classification of femoral neck fractures. An assessment of interobserver variation. J Bone Joint Surg Br 70(4):588–590, 1988

  29. Thorngren KG, Hommel A, Norrman PO, Thorngren J, Wingstrand H: Epidemiology of femoral neck fractures. Injury 33:1-7, 2002

    Article  Google Scholar 

  30. Van Embden D, Rhemrev SJ, Genelin F, Meylaerts SA, Roukema GR: The reliability of a simplified Garden classification for intracapsular hip fractures. Orthop Traumatol Surg Res 98(4):405-408, 2012

    Article  Google Scholar 

  31. He K, Zhang X, Ren S, Sun J: Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778, 2016

  32. Woo S, Park J, Lee J, Kweon IS: CBAM: Convolutional Block Attention Module. Proceedings of the European Conference on Computer Vision (ECCV) 3–19, 2018

  33. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D: Grad-CAM: Visual Explanations From Deep Networks via Gradient-Based Localization. Proceedings of the IEEE International Conference on Computer Vision (ICCV) 618–626, 2017

  34. Youden WJ: Index for rating diagnostic tests. Cancer 3(1):32-35, 1950

    Article  CAS  Google Scholar 

  35. Kwon G, Ryu J, Oh J, Lim J, Kang BK, Ahn C, et al: Deep learning algorithms for detecting and visualising intussusception on plain abdominal radiography in children: a retrospective multicenter study. Sci Rep 10(1):17582, 2020

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We would like to thank eworld editing (www.eworldediting.com) for English language editing.

Funding

This study was supported by the National Research Foundation of Korea (2019R1F1A1063502). This work was also supported by the research fund of Hanyang University (HY-2018) and the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (NRF2019R1A4A1029800).

Author information

Author notes

  1. Junwon Bae and Sangjoon Yu contributed this work equally as first author.

Authors and Affiliations

  1. Department of Emergency Medicine, College of Medicine, Hanyang University, 222 Wangsimni-ro, Seongdong-gu, Seoul, 04763, Republic of Korea

    Junwon Bae, Jaehoon Oh & Myeong Seong Yoon

  2. Department of Computer Science, Hanyang University, 222 Wangsimni-ro, Seongdong-gu, Seoul, 04763, Republic of Korea

    Sangjoon Yu & Tae Hyun Kim

  3. Machine Learning Research Center for Medical Data, Hanyang University, Seoul, Republic of Korea

    Jaehoon Oh, Tae Hyun Kim, Jae Ho Chung & Hayoung Byun

  4. Department of Otolaryngology – Head and Neck Surgery, College of Medicine, Hanyang University, Seoul, Republic of Korea

    Jae Ho Chung & Hayoung Byun

  5. Department of HY, College of Medicine, KIST Bio-Convergence, Hanyang University, Seoul, Republic of Korea

    Jae Ho Chung

  6. Department of Emergency Medicine, College of Medicine, Chung-Ang University, Seoul, Republic of Korea

    Chiwon Ahn

  7. Department of Emergency Medicine, Seoul National University Bundang Hospital, Gyeonggi-do, Republic of Korea

    Dong Keon Lee

Authors
  1. Junwon Bae
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sangjoon Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jaehoon Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tae Hyun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jae Ho Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hayoung Byun
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Myeong Seong Yoon
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Chiwon Ahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Dong Keon Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Oh J and Kim TH conceived the study and designed the trial. Oh J, Bae J, Ahn C, Byun H, Chung JH, and Lee D supervised the trial conduct and were involved in data collection. Yu S, Kim T, and Yoon M analyzed all images and data. Bae J, Yu S, Oh J and Kim TH drafted the manuscript, and all authors substantially contributed to its revision. Oh J and Kim TH take the responsibility for the content of the paper.

Corresponding authors

Correspondence to Jaehoon Oh or Tae Hyun Kim.

Ethics declarations

Ethics Approval

This study was approved by the Institutional Review Boards of Hanyang University Hospital (ref. no. 2020–01-005) and Seoul National University Bundang Hospital (ref. no. B-2002–595-103).

Consent to Participate

The requirement for informed consent were waived by the IRBs of Hanyang University Hospital, and Seoul National University Bundang Hospital. All methods and procedures were carried out in accordance with the Declaration of Helsinki.

Consent for Publication

This manuscript has not been published or presented elsewhere in part or in entirety and is not under consideration by another journal. We have read and understood your journal’s policies, and we believe that neither the manuscript nor the study violates any of these.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bae, J., Yu, S., Oh, J. et al. External Validation of Deep Learning Algorithm for Detecting and Visualizing Femoral Neck Fracture Including Displaced and Non-displaced Fracture on Plain X-ray. J Digit Imaging 34, 1099–1109 (2021). https://doi.org/10.1007/s10278-021-00499-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-021-00499-2

Keywords

Search

Navigation