To develop a deep-learning algorithm for anterior cruciate ligament (ACL) tear detection and to compare its accuracy using two external datasets.
A database of 19,765 knee MRI scans (17,738 patients) issued from different manufacturers and magnetic fields was used to build a deep learning–based ACL tear detector. Fifteen percent showed partial or complete ACL rupture. Coronal and sagittal fat-suppressed proton density or T2-weighted sequences were used. A Natural Language Processing algorithm was used to automatically label reports associated with each MRI exam. We compared the accuracy of our model on two publicly available external datasets: MRNet, Bien et al, USA (PLoS Med 15:e1002699, 2018); and KneeMRI, Stajduhar et al, Croatia (Comput Methods Prog Biomed 140:151-164, 2017). Receptor operating characteristics (ROC) curves, area under the curve (AUC), sensitivity, specificity, and accuracy were used to evaluate our model.
Our neural networks achieved an AUC value of 0.939 for detection of ACL tears, with a sensitivity of 87% (0.875) and a specificity of 91% (0.908). After retraining our model on Bien dataset and Stajduhar dataset, our algorithm achieved AUC of 0.962 (95% CI 0.930–0.988) and 0.922 (95% CI 0.875, 0.962) respectively. Sensitivity, specificity, and accuracy were respectively 85% (95% CI 75–94%, 0.852), 89% (95% CI 82–97%, 0.894), 0.875 (95% CI 0.817–0.933) for Bien dataset, and 68% (95% CI 54–81%, 0.681), 93% (95% CI 89–97%, 0.934), and 0.870 (95% CI 0.821–0.913) for Stajduhar dataset.
Our algorithm showed high performance in the detection of ACL tears with AUC on two external datasets, demonstrating its generalizability on different manufacturers and populations.
This study shows the performance of an algorithm for detecting anterior cruciate ligament tears with an external validation on populations from countries and continents different from the study population.
• An algorithm for detecting anterior cruciate ligament ruptures was built from a large dataset of nearly 20,000 MRI with AUC values of 0.939, sensitivity of 87%, and specificity of 91%.
• This algorithm was tested on two external populations from different other countries: a dataset from an American population and a dataset from a Croatian population. Performance remains high on these two external validation populations (AUC of 0.962 and 0.922 respectively).
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Anterior cruciate ligament
Convolutional neural networks
Intersection over Union
Natural Language Processing
Rectified linear unit
Musahl V, Karlsson J (2019) Anterior cruciate ligament tear. N Engl J Med 380:2341–2348
Shea K, Carey J (2015) Management of anterior cruciate ligament injuries. J Am Acad Orthop Surg 23:e1–e5
Li K, Du J, Huang L, Ni L, Liu T, Yang H (2017) The diagnostic accuracy of magnetic resonance imaging for anterior cruciate ligament injury in comparison to arthroscopy: a meta-analysis. Sci Rep 7:7583
Challen J, Tang Y, Hazratwala K, Stuckey S (2007) Accuracy of MRI diagnosis of internal derangement of the knee in a non-specialized tertiary level referral teaching hospital. Australas Radiol 51:426–431
Singh R, Kalra M, Nitiwarangkul C et al (2018) Deep learning in chest radiography: detection of findings and presence of change. PLoS One 13:e0204155
Chilamkurthy S, Ghosh R, Tanamala S et al (2018) Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet 392:2388–2396
Gyftopoulos S, Lin D, Knoll F, Doshi A, Rodrigues T, Recht M (2019) Artificial intelligence in musculoskeletal imaging: current status and future directions. AJR Am J Roentgenol 213:506–513
Bien N, Rajpurkar P, Ball R et al (2018) Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. PLoS Med 15:e1002699
Štajduhar I, Mamula M, Miletić D, Ünal G (2017) Semi-automated detection of anterior cruciate ligament injury from MRI. Comput Methods Prog Biomed 140:151–164
Rizk B, Brat H, Zille P, Guillin R, Pouchy C (2021) Adam C, et al Meniscal lesion detection and characterization in adult knee MRI: A deep learning model approach with external validation. Phys Med 83:64–71
Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv:1301.3781. Available from: http://arxiv.org/abs/1301.3781
Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:14123555. Available from: http://arxiv.org/abs/1412.3555
Beaufils P, Hulet C, Dhénain M, Nizard R, Nourissat G, Pujol N (2009) Clinical practice guidelines for the management of meniscal lesions and isolated lesions of the anterior cruciate ligament of the knee in adults. Orthop Traumatol Surg Res 95(6):437–442
DiCiccio T, Efron B (1996) Bootstrap confidence intervals. Stat Sci 11:189–228
Liu F, Guan B, Zhou Z et al (2019) Fully automated diagnosis of anterior cruciate ligament tears on knee MR images by using deep learning. Radiol Artif Intell 1:180091
Germann C, Marbach G, Civardi F et al (2020) Deep convolutional neural network-based diagnosis of anterior cruciate ligament tears: performance comparison of homogenous versus heterogeneous knee MRI cohorts with different pulse sequence protocols and 1.5-T and 3-T magnetic field strengths. Invest Radiol 55:499–506
Zhang L, Li M, Zhou Y, Lu G, Zhou Q (2020) Deep learning approach for anterior cruciate ligament lesion detection: evaluation of diagnostic performance using arthroscopy as the reference standard. J Magn Reson Imaging 52:1745–1752
Pinto dos Santos D, Brodehl S, Baeßler B et al (2019) Structured report data can be used to develop deep learning algorithms: a proof of concept in ankle radiographs. Insights Imaging 10:93
Chang P, Wong T, Rasiej M (2019) Deep learning for detection of complete anterior cruciate ligament tear. J Digit Imaging 32:980–986
Van Dyck P, Vanhoenacker F, Gielen J et al (2010) Three tesla magnetic resonance imaging of the anterior cruciate ligament of the knee: can we differentiate complete from partial tears? Skelet Radiol 40:701–707
Phelan N, Rowland P, Galvin R, O’Byrne J (2015) A systematic review and meta-analysis of the diagnostic accuracy of MRI for suspected ACL and meniscal tears of the knee. Knee Surg Sports Traumatol Arthrosc 24:1525–1539
The authors state that this work has not received any funding.
The scientific guarantor of this publication is Pascal Zille.
Conflict of interest
The authors of this manuscript declare relationships with the following companies:
• A.T. was an intern at Incepto Medical, but at the time of submission of the article disclosed no relevant relationships.
• P.Z., C.A., M.C., and M.W. are employed by Incepto Medical.
• G.A. is the founder and Chief Medical Officer of Incepto Medical.
• R.G. disclosed no conflict of interest.
• H.B. disclosed no conflict of interest.
• B.R. disclosed no conflict of interest.
• L.L. has a consulting activity for Incepto Medical.
Statistics and biometry
No complex statistical methods were necessary for this paper.
Written informed consent was obtained from all subjects (patients) in this study.
Institutional Review Board approval was obtained.
Study subjects or cohorts overlap
Some study subjects have been previously reported in Rizk B, Brat H, Zille P, Guillin R, Pouchy C, Adam C, et al Meniscal lesion detection and characterization in adult knee MRI: a deep learning model approach with external validation. Physica Medica. 2021 Mar;83:64–71.
• diagnostic study
• multicenter study
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
*Alexia Tran and Louis Lassalle are joint first authors.
Rights and permissions
About this article
Cite this article
Tran, A., Lassalle, L., Zille, P. et al. Deep learning to detect anterior cruciate ligament tear on knee MRI: multi-continental external validation. Eur Radiol 32, 8394–8403 (2022). https://doi.org/10.1007/s00330-022-08923-z
- Deep learning
- Knee injuries
- Magnetic resonance imaging
- Artificial intelligence