Skip to main content

Advertisement

SpringerLink
Log in

Deep learning in knee imaging: a systematic review utilizing a Checklist for Artificial Intelligence in Medical Imaging (CLAIM)

  • Musculoskeletal
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Purpose

Our purposes were (1) to explore the methodologic quality of the studies on the deep learning in knee imaging with CLAIM criterion and (2) to offer our vision for the development of CLAIM to assure high-quality reports about the application of AI to medical imaging in knee joint.

Materials and methods

A Checklist for Artificial Intelligence in Medical Imaging systematic review was conducted from January 1, 2015, to June 1, 2020, using PubMed, EMBASE, and Web of Science databases. A total of 36 articles discussing deep learning applications in knee joint imaging were identified, divided by imaging modality, and characterized by imaging task, data source, algorithm type, and outcome metrics.

Results

A total of 36 studies were identified and divided into: X-ray (44.44%) and MRI (55.56%). The mean CLAIM score of the 36 studies was 27.94 (standard deviation, 4.26), which was 66.53% of the ideal score of 42.00. The CLAIM items achieved an average good inter-rater agreement (ICC 0.815, 95% CI 0.660–0.902). In total, 32 studies performed internal cross-validation on the data set, while only 4 studies conducted external validation of the data set.

Conclusions

The overall scientific quality of deep learning in knee imaging is insufficient; however, deep learning remains a promising technology for diagnostic or predictive purpose. Improvements in study design, validation, and open science need to be made to demonstrate the generalizability of findings and to achieve clinical applications. Widespread application, pre-trained scoring procedure, and modification of CLAIM in response to clinical needs are necessary in the future.

Key Points

• Limited deep learning studies were established in knee imaging with mean score of 27.94, which was 66.53% of the ideal score of 42.00, commonly due to invalidated results, retrospective study design, and absence of a clear definition of the CLAIM items in detail.

• A previous trained data extraction instrument allowed reaching moderate inter-rater agreement in the application of the CLAIM, while CLAIM still needs improvement in scoring items and result reporting to become a wide adaptive tool in reviews of deep learning studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Abbreviations

AI:

Artificial intelligence

AUC:

Area under the curve

CNN:

Convolutional neural network

CLAIM:

Checklist for Artificial Intelligence in Medical Imaging

DL:

Deep learning

DNN:

Deep neural network

ML:

Machine learning

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

References

  1. Prieto-Alhambra D, Judge A, Javaid MK, Cooper C, Diez-Perez A, Arden NK (2014) Incidence and risk factors for clinically diagnosed knee, hip and hand osteoarthritis: influences of age, gender and osteoarthritis affecting other joints. Ann Rheum Dis 73:1659–1664

    Article  Google Scholar 

  2. Turkiewicz A, Petersson IF, Bjork J et al (2014) Current and future impact of osteoarthritis on health care: a population-based study with projections to year 2032. Osteoarthritis Cartilage 22:1826–1832

    Article  CAS  Google Scholar 

  3. Roemer FW, Demehri S, Omoumi P et al (2020) State of the art: imaging of osteoarthritis-revisited 2020. Radiology 296(1):5–21

    Article  Google Scholar 

  4. Dunn R, Greenhouse J, James D, Ohlssen D, Mesenbrink P (2020) Risk scoring for time to end-stage knee osteoarthritis: data from the Osteoarthritis Initiative. Osteoarthritis Cartilage 28(8):1020–1029

    Article  CAS  Google Scholar 

  5. Zhai G, Sun X, Randel E et al (2021) Phenylalanine is a novel marker for radiographic knee osteoarthritis progression: the MOST study. J Rheumatol 48(1):123–128

    Article  Google Scholar 

  6. Peterfy CG, Guermazi A, Zaim S et al (2004) WORMS of the knee in osteoarthritis. Osteoarthritis Cartilage 12(3):177–190

    Article  CAS  Google Scholar 

  7. Hunter DJ, Guermazi A, Lo GH et al (2011) Evolution of semi-quantitative whole joint assessment of knee OA: MOAKS. Osteoarthritis Cartilage 19(8):990–1002

    Article  CAS  Google Scholar 

  8. Hunter DJ, Lo GH, Gale D et al (2008) The reliability of a new scoring system for knee osteoarthritis MRI and the validity of bone marrow lesion assessment: BLOKS. Ann Rheum Dis 67(2):206–211

    Article  CAS  Google Scholar 

  9. Peterfy CG, van Dijke CF, Janzen DL et al (1994) Quantification of articular cartilage in the knee with pulsed saturation transfer subtraction and fat-suppressed MR imaging: optimization and validation. Radiology 192:485–491

    Article  CAS  Google Scholar 

  10. Eckstein F, Le Graver MP, Charles HC et al (2011) Clinical, radiographic, molecular and MRI-based predictors of cartilage loss in knee osteoarthritis. Ann Rheum Dis 70:1223–1230

    Article  CAS  Google Scholar 

  11. Eckstein F, Wirth W, Guermazi A, Maschek S, Aydemir A (2015) Intra-articular sprifermin not only increases cartilage thickness, but also reduces cartilage loss: location-independent post hoc analysis using MR imaging. Arthritis Rheumatol 67(11):2916–2922

    Article  CAS  Google Scholar 

  12. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    Article  CAS  Google Scholar 

  13. Leung K, Zhang B, Tan J et al (2020) Prediction of total knee replacement and diagnosis of osteoarthritis by using deep learning on knee radiographs: data from the Osteoarthritis Initiative. Radiology 296(3):584–593

    Article  Google Scholar 

  14. Bien N, Rajpurkar P, Ball RL et al (2018) Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. PLoS Med 15(11):e1002699

    Article  Google Scholar 

  15. Tiulpin A, Klein S, Bierma-Zeinstra SMA et al (2019) Multimodal machine learning-based knee osteoarthritis progression prediction from plain radiographs and clinical data. Sci Rep 9(1):20038

    Article  CAS  Google Scholar 

  16. Mongan J, Moy L, Kahn CE Jr (2020) Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers. Radiol Artif Intell 2(2):e200029

    Article  Google Scholar 

  17. McInnes MDF, Moher D, Thombs BD et al (2018) Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: The PRISMA-DTA statement. JAMA 319(4):388–396

    Article  Google Scholar 

  18. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement. BMJ 339:b2535

    Article  Google Scholar 

  19. Moher D, Shamseer L, Clarke M et al (2015) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 4:1

    Article  Google Scholar 

  20. Zhong J, Hu Y, Si L et al (2021) A systematic review of radiomics in osteosarcoma: utilizing radiomics quality score as a tool promoting clinical translation. Eur Radiol 31(3):1526–1535

    Article  Google Scholar 

  21. Cohen JA (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37–46

    Article  Google Scholar 

  22. Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15:155–163

    Article  Google Scholar 

  23. Morales Martinez A, Caliva F, Flament I et al (2020) Learning osteoarthritis imaging biomarkers from bone surface spherical encoding. Magn Reson Med 84(4):2190–2203

    Article  Google Scholar 

  24. Roblot V, Giret Y, Bou Antoun M et al (2019) Artificial intelligence to diagnose meniscus tears on MRI. Diagn Interv Imaging 100(4):243–249

    Article  CAS  Google Scholar 

  25. Das A, Rad P (2020) Opportunities and challenges in explainable artificial intelligence (XAI): a survey. https://arxiv.org/abs/2006.11371

  26. Norman B, Pedoia V, Noworolski A, Link TM, Majumdar S (2019) Applying densely connected convolutional neural networks for staging osteoarthritis severity from plain radiographs. J Digit Imaging 32(3):471–477

    Article  Google Scholar 

  27. Thomas KA, Kidziński Ł, Halilaj E et al (2020) Automated classification of radiographic knee osteoarthritis severity using deep neural networks. Radiol Artif Intell 2(2):e190065

    Article  Google Scholar 

  28. Chang PD, Wong TT, Rasiej MJ (2019) Deep learning for detection of complete anterior cruciate ligament tear. J Digit Imaging 32(6):980–986

    Article  Google Scholar 

  29. Richardson ML (2021) MR protocol optimization with deep learning: a proof of concept. Curr Probl Diagn Radiol 50(2):168–174

    Article  Google Scholar 

  30. Górriz M, Antony J, Mcguinness K, Giró-i-Nieto X, O’Connor NE (2019) Assessing knee OA severity with CNN attention-based end-to-end architectures. In: International Conference on Medical Imaging with Deep Learning. PMLR, pp 197–214

  31. Armanious K, Abdulatif S, Bhaktharaguttu AR et al (2021) Organ-based chronological age estimation based on 3D MRI scans. In: 2020 28th European Signal Processing Conference (EUSIPCO). IEEE, pp 1225–1228

  32. Guan B, Liu F, Haj-Mirzaian A et al (2020) Deep learning risk assessment models for predicting progression of radiographic medial joint space loss over a 48-month follow-up period. Osteoarthritis Cartilage 28(4):428–437

    Article  CAS  Google Scholar 

  33. Sanduleanu S, Woodruff HC, de Jong EEC et al (2018) Tracking tumor biology with radiomics: a systematic review utilizing a radiomics quality score. Radiother Oncol 127(3):349–360

    Article  Google Scholar 

  34. Granzier RWY, van Nijnatten TJA, Woodruff HC et al (2019) Exploring breast cancer response prediction to neoadjuvant systemic therapy using MRI-based radiomics: a systematic review. Eur J Radiol 121:108736

    Article  CAS  Google Scholar 

Download references

Acknowledgements

The authors would like to express their gratitude to Prof. Huan Zhang and Prof. Qian Wang for their constructive discussion and suggestions.

Funding

This study has received funding by the National Natural Science Foundation of China (81771790) and the Medicine and Engineering Combination Project of Shanghai Jiao Tong University (YG2019ZDB09).

Author information

Author notes

  1. Liping Si and Jingyu Zhong contributed equally to this work.

Authors and Affiliations

  1. Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, No. 1111 Xianxia Road, Changning District, Shanghai, 200336, China

    Liping Si, Jingyu Zhong & Weiwu Yao

  2. Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Huashan Road #1954, Shanghai, 200030, China

    Jiayu Huo, Kai Xuan, Zixu Zhuang & Qian Wang

  3. Department of Radiology, Shanghai Jiao Tong University Affiliated Sixth People’s Hospital, Shanghai, 200233, China

    Yangfan Hu

  4. Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China

    Huan Zhang

Authors
  1. Liping Si
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingyu Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiayu Huo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kai Xuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zixu Zhuang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yangfan Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Qian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Huan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Weiwu Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Qian Wang or Weiwu Yao.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Prof. Weiwu Yao.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and Biometry

No complex statistical methods were necessary for this paper, but one of the authors has significant statistical expertise.

Informed Consent

Written informed consent was not required for this study because of the nature of our study, which was a systematic review.

Ethical Approval

Institutional Review Board approval was not required because of the nature of our study, which was a systematic review.

Methodology

• retrospective

• diagnostic or prognostic study

• multicenter study

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

ESM 1

(DOCX 566 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Si, L., Zhong, J., Huo, J. et al. Deep learning in knee imaging: a systematic review utilizing a Checklist for Artificial Intelligence in Medical Imaging (CLAIM). Eur Radiol 32, 1353–1361 (2022). https://doi.org/10.1007/s00330-021-08190-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-021-08190-4

Keywords

Search

Navigation