Skip to main content

Advertisement

SpringerLink
Log in

Automated Classification of Rheumatoid Arthritis, Osteoarthritis, and Normal Hand Radiographs with Deep Learning Methods

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Rheumatoid arthritis and hand osteoarthritis are two different arthritis that causes pain, function limitation, and permanent joint damage in the hands. Plain hand radiographs are the most commonly used imaging methods for the diagnosis, differential diagnosis, and monitoring of rheumatoid arthritis and osteoarthritis. In this retrospective study, the You Only Look Once (YOLO) algorithm was used to obtain hand images from original radiographs without data loss, and classification was made by applying transfer learning with a pre-trained VGG-16 network. The data augmentation method was applied during training. The results of the study were evaluated with performance metrics such as accuracy, sensitivity, specificity, and precision calculated from the confusion matrix, and AUC (area under the ROC curve) calculated from ROC (receiver operating characteristic) curve. In the classification of rheumatoid arthritis and normal hand radiographs, 90.7%, 92.6%, 88.7%, 89.3%, and 0.97 accuracy, sensitivity, specificity, precision, and AUC results, respectively, and in the classification of osteoarthritis and normal hand radiographs, 90.8%, 91.4%, 90.2%, 91.4%, and 0.96 accuracy, sensitivity, specificity, precision, and AUC results were obtained, respectively. In the classification of rheumatoid arthritis, osteoarthritis, and normal hand radiographs, an 80.6% accuracy result was obtained. In this study, to develop an end-to-end computerized method, the YOLOv4 algorithm was used for object detection, and a pre-trained VGG-16 network was used for the classification of hand radiographs. This computer-aided diagnosis method can assist clinicians in interpreting hand radiographs, especially in rheumatoid arthritis and osteoarthritis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Lee DM, Weinblatt ME. Rheumatoid arthritis. The Lancet. 2001 Sep;358(9285):903–11.

  2. Kourilovitch M, Galarza-Maldonado C, Ortiz-Prado E. Diagnosis and classification of rheumatoid arthritis. Journal of autoimmunity. 2014 Feb;48–49:26–30.

  3. Renner WR, Weinstein AS. Early changes of rheumatoid arthritis in the hand and wrist. Radiologic clinics of North America. 1988 Nov;26(6):1185–93.

  4. Smolen JS, Landewé R, Breedveld FC, Buch M, Burmester G, Dougados M, et al. EULAR recommendations for the management of rheumatoid arthritis with synthetic and biological disease-modifying antirheumatic drugs: 2013 update. Annals of the Rheumatic Diseases. 2014 Mar;73(3):492–509.

  5. Zhang Y, Jordan JM. Epidemiology of osteoarthritis. Clinics in geriatric medicine. 2010 Aug;26(3):355–69.

  6. Hayashi D, Roemer FW, Guermazi A. Imaging for osteoarthritis. Annals of Physical and Rehabilitation Medicine. 2016 Jun;59(3):161–9.

  7. Leung GJ, Rainsford KD, Kean WF. Osteoarthritis of the hand I: Aetiology and pathogenesis, risk factors, investigation and diagnosis. Journal of Pharmacy and Pharmacology. 2014 Mar;66(3):339–46.

  8. Ramonda R, Frallonardo P, Musacchio E, Vio S, Punzi L. Joint and bone assessment in hand osteoarthritis. Clinical rheumatology. 2014 Jan;33(1):11–9.

  9. Hill J, Bird H. Patient knowledge and misconceptions of osteoarthritis assessed by a validated self-completed knowledge questionnaire (PKQ-OA). Rheumatology (Oxford, England). 2007 May;46(5):796–800.

  10. Pereira D, Ramos E, Branco J. Osteoarthritis. Acta medica portuguesa. 2015;28(1):99–106.

  11. Singh S, Maxwell J, Baker JA, Nicholas JL, Lo JY. Computer-aided Classification of Breast Masses: Performance and Interobserver Variability of Expert Radiologists versus Residents. Radiology. 2011 Jan;258(1):73–80.

  12. Doi K. Computer-Aided Diagnosis in Medical Imaging: Achievements and Challenges. In Springer, Berlin, Heidelberg; 2009. p. 96–96.

  13. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Medical Image Analysis. 2017 Dec;42:60–88.

  14. Dargan S, Kumar M, Ayyagari MR, Kumar G. A Survey of Deep Learning and Its Applications: A New Paradigm to Machine Learning. Archives of Computational Methods in Engineering 2019 27:4. 2019 Jun 1;27(4):1071–92.

  15. Greenspan H, Van Ginneken B, Summers RM. Guest Editorial Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique. Vol. 35, IEEE Transactions on Medical Imaging. Institute of Electrical and Electronics Engineers Inc.; 2016. p. 1153–9.

  16. Anwar SM, Majid M, Qayyum A, Awais M, Alnowami M, Khan MK. Medical Image Analysis using Convolutional Neural Networks: A Review. Journal of Medical Systems Springer New York LLC; Nov 1, 2018 p. 1–13.

  17. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. 2015 Sep 4;1–14.

  18. Szegedy C, Wei Liu, Yangqing Jia, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2015. p. 1–9.

  19. Szegedy C, Vanhoucke V, Ioffe S, Shlens J. Rethinking the Inception Architecture for Computer Vision. 2016.

  20. He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. Vols. 2016-Decem, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society; 2016 Dec.

  21. Üreten K, Arslan T, Gültekin KE, Demir AND, Özer HF, Bilgili Y, et al. Detection of hip osteoarthritis by using plain pelvic radiographs with deep learning methods. Skeletal Radiology. 2020 Sep 1;49(9):1369–74.

  22. Cicero M, Bilbily A, Colak E, Dowdell T, Gray B, Perampaladas K, et al. Training and Validating a Deep Convolutional Neural Network for Computer-Aided Detection and Classification of Abnormalities on Frontal Chest Radiographs. Investigative Radiology. 2017 May 1;52(5):281–7.

  23. Mednikov Y, Nehemia S, Zheng B, Benzaquen O, Lederman D. Transfer Representation Learning using Inception-V3 for the Detection of Masses in Mammography. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. Institute of Electrical and Electronics Engineers Inc.; 2018. p. 2587–90.

  24. Gupta S, Kumar M, Garg A. Improved object recognition results using SIFT and ORB feature detector. Multimedia Tools and Applications 2019 78:23. 2019 Oct 19;78(23):34157–71.

  25. Gupta S, Thakur K, Kumar M. 2D-human face recognition using SIFT and SURF descriptors of face’s feature regions. The Visual Computer 2020 37:3. 2020 Feb 12;37(3):447–56.

  26. Bansal M, Kumar M, Kumar M. 2D object recognition: a comparative analysis of SIFT, SURF and ORB feature descriptors. Multimedia Tools and Applications. 2021;80(12):18839–57.

  27. Aly GH, Marey M, El-Sayed SA, Tolba MF. YOLO Based Breast Masses Detection and Classification in Full-Field Digital Mammograms. Computer Methods and Programs in Biomedicine. 2020 Mar 1;200:105823.

  28. Bochkovskiy A, Wang C-Y, Liao H-YM. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv. 2020 Apr 22;

  29. Cheng R. A survey: Comparison between Convolutional Neural Network and YOLO in image identification. In: Journal of Physics: Conference Series. Institute of Physics Publishing; 2020. p. 12139.

  30. Singh S, Ahuja U, Kumar M, Kumar K, Sachdeva M. Face mask detection using YOLOv3 and faster R-CNN models: COVID-19 environment. Multimedia Tools and Applications. 2021;80(13):19753–68.

  31. Nie Y, Sommella P, O’Nils M, Liguori C, Lundgren J. Automatic detection of melanoma with yolo deep convolutional neural networks. In: 2019 7th E-Health and Bioengineering Conference, EHB 2019. Institute of Electrical and Electronics Engineers Inc.; 2019.

  32. Kim DH, MacKinnon T. Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks. Clinical Radiology. 2018 May 1;73(5):439–45.

  33. Üreten K, Erbay H, Maraş HH. Detection of hand osteoarthritis from hand radiographs using convolutional neural networks with transfer learning. Turkish Journal of Electrical Engineering & Computer Sciences. 2020 Sep 25;28(5):2968–78.

  34. Üreten K, Erbay H, Maraş HH. Detection of rheumatoid arthritis from hand radiographs using a convolutional neural network. Clinical Rheumatology. 2020;39(4).

  35. Murakami S, Hatano K, Tan J, Kim H, Aoki T. Automatic identification of bone erosions in rheumatoid arthritis from hand radiographs based on deep convolutional neural network. Multimedia Tools and Applications. 2018 May 6;77(9):10921–37.

  36. Sharp JT, Young DY, Bluhm GB, Brook A, Brower AC, Corbett M, et al. How many joints in the hands and wrists should be included in a score of radiologic abnormalities used to assess rheumatoid arthritis? Arthritis & Rheumatism. 1985 Dec;28(12):1326–35.

  37. KELLGREN JH, LAWRENCE JS. Radiological assessment of osteo-arthrosis. Annals of the rheumatic diseases. 1957 Dec;16(4):494–502.

Download references

Acknowledgements

We would like to thank Dr. Abdurrahman Tufan (Gazi University, Faculty of Medicine, Department of Rheumatology), and Dr. Levent Kılıç (Hacettepe University, Faculty of Medicine, Department of Rheumatology) for classifying the radiographs used in this study.

Author information

Authors and Affiliations

  1. Department of Rheumatology, Faculty of Medicine, Kırıkkale University, 71450, Kırıkkale, Turkey

    Kemal Üreten

  2. Department of Computer Engineering, Faculty of Engineering, Çankaya University, 06790, Ankara, Turkey

    Hadi Hakan Maraş

Authors
  1. Kemal Üreten
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hadi Hakan Maraş
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors were fully involved in the preparation of this manuscript and approved the final version.

Corresponding author

Correspondence to Kemal Üreten.

Ethics declarations

Ethics Approval

Ethical approval certificate was obtained from the Non-interventional Clinical Researches Ethics Board in Kırıkkale University. Certificate date: March 25, 2021, Certificate no: 2021.03.11

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Key Points

• Plain hand radiographs are used to the diagnosis and monitoring progression of rheumatoid arthritis and hand osteoarthritis, and the evaluation of plain hand radiographs requires experience.

• Successful studies are carried out in classifying medical images with deep learning methods.

• Deep learning methods can assist physicians in evaluating plain hand radiographs.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Üreten, K., Maraş, H.H. Automated Classification of Rheumatoid Arthritis, Osteoarthritis, and Normal Hand Radiographs with Deep Learning Methods. J Digit Imaging 35, 193–199 (2022). https://doi.org/10.1007/s10278-021-00564-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-021-00564-w

Keywords

Search

Navigation