Advertisement

Toward automatic quantification of knee osteoarthritis severity using improved Faster R-CNN

  • Bin Liu
  • Jianxu LuoEmail author
  • Huan Huang
Original Article
  • 30 Downloads

Abstract

Purpose

Knee osteoarthritis (OA) is a common disease that impairs knee function and causes pain. Radiologists usually review knee X-ray images and grade the severity of the impairments according to the Kellgren–Lawrence grading scheme. However, this approach becomes inefficient in hospitals with high throughput as it is time-consuming, tedious and also subjective. This paper introduces a model for automatic diagnosis of knee OA based on an end-to-end deep learning method.

Method

In order to process the input images with location and classification simultaneously, we use Faster R-CNN as baseline, which consists of region proposal network (RPN) and Fast R-CNN. The RPN is trained to generate region proposals, which contain knee joint and then be used by Fast R-CNN for classification. Due to the localized classification via CNNs, the useless information in X-ray images can be filtered and we can extract clinically relevant features. For the further improvement in the model’s performance, we use a novel loss function whose weighting scheme allows us to address the class imbalance. Besides, larger anchors are used to overcome the problem that anchors don’t match the object when increasing the input size of X-ray images.

Result

The performance of the proposed model is thoroughly assessed using various measures. The results show that our adjusted model outperforms the Faster R-CNN, achieving a mean average precision nearly 0.82 with a sensitivity above 78% and a specificity above 94%. It takes 0.33 s to test each image, which achieves a better trade-off between accuracy and speed.

Conclusion

The proposed end-to-end fully automatic model which is computationally efficient has the potential to achieve the real automatic diagnosis of knee OA and be used as computer-aided diagnosis tools in clinical applications.

Keywords

Knee osteoarthritis X-ray Deep learning Faster R-CNN Focal loss 

Notes

Acknowledgements

This work is supported by the Scientific and Technological Innovation Action Plan of the Science and Technology Commission of Shanghai Municipality under Grant Number 19511121200.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

References

  1. 1.
    Zhang Y, Jordan JM (2010) Epidemiology of osteoarthritis. Clin Geriatr Med 26(3):355–369 CrossRefGoogle Scholar
  2. 2.
    Cross M, Smith E, Hoy D, Nolte S, Ackerman I, Fransen M, Bridgett L, Williams S, Guillemin F, Hill CL (2014) The global burden of hip and knee osteoarthritis: estimates from the global burden of disease 2010 study. Ann Rheum Dis 73(7):1323–1330CrossRefGoogle Scholar
  3. 3.
    Karsdal MA, Michaelis M, Ladel C, Siebuhr AS, Bihlet A, Andersen JR, Guehring H, Christiansen C, Bayjensen AC, Kraus VB (2016) Disease-modifying treatments for osteoarthritis (DMOADs) of the knee and hip: lessons learned from failures and opportunities for the future. Osteoarthr Cartil 24(12):2013–2021CrossRefGoogle Scholar
  4. 4.
    Braun HJ, Gold GE (2012) Diagnosis of osteoarthritis: imaging. Bone 51(2):278–288CrossRefGoogle Scholar
  5. 5.
    Yoo TK, Kim DW, Choi SB, Park JS (2016) Simple scoring system and artificial neural network for knee osteoarthritis risk prediction: a cross-sectional study. PloS ONE 11(2):e0148724CrossRefGoogle Scholar
  6. 6.
    Orlov N, Shamir L, Macura T, Johnston J, Eckley DM, Goldberg IG (2008) WND-CHARM: multi-purpose image classification using compound image transforms. Pattern Recognit Lett 29(11):1684–1693CrossRefGoogle Scholar
  7. 7.
    Oka H, Muraki S, Akune T, Mabuchi A, Suzuki T, Yoshida H, Yamamoto S, Nakamura K, Yoshimura N, Kawaguchi H (2008) Fully automatic quantification of knee osteoarthritis severity on plain radiographs. Osteoarthr Cartil 16(11):1300–1306CrossRefGoogle Scholar
  8. 8.
    Juefei-Xu F, Naresh Boddeti V , Savvides M (2017) Local binary convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 19–28Google Scholar
  9. 9.
    Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Laak JAVD, Ginneken BV, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88CrossRefGoogle Scholar
  10. 10.
    Antony J, McGuinness K, O’Connor NE, Moran K (2016) Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. In: 2016 23rd international conference on pattern recognition (ICPR). IEEE, pp 1195–1200Google Scholar
  11. 11.
    Antony J, McGuinness K, Moran K, O’Connor NE (2017) Automatic detection of knee joints and quantification of knee osteoarthritis severity using convolutional neural networks. In: International conference on machine learning and data mining in pattern recognition. Springer, pp 376–390Google Scholar
  12. 12.
    Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440Google Scholar
  13. 13.
    Tiulpin A, Thevenot J, Rahtu E, Lehenkari P, Saarakkala S (2018) Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach. Sci Rep 8(1):1727CrossRefGoogle Scholar
  14. 14.
    Abedin J, Antony J, McGuinness K, Moran K, O’Connor NE, Rebholz-Schuhmann D, Newell J (2019) Predicting knee osteoarthritis severity: comparative modeling based on patient’s data and plain X-ray images. Sci Rep 9(1):5761CrossRefGoogle Scholar
  15. 15.
    Lin TY, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988Google Scholar
  16. 16.
    Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99Google Scholar
  17. 17.
    Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587Google Scholar
  18. 18.
    He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916CrossRefGoogle Scholar
  19. 19.
    Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448Google Scholar
  20. 20.
    Neubeck A, Van Gool L (2006) Efficient non-maximum suppression. In: 18th international conference on pattern recognition (ICPR’06), vol 3. IEEE, pp 850–855Google Scholar
  21. 21.
    Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788Google Scholar
  22. 22.
    Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271Google Scholar
  23. 23.
    Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv preprint arXiv:1804.02767
  24. 24.
    Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: single shot multibox detector. In: European conference on computer vision. Springer, pp 21–37Google Scholar
  25. 25.
    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778Google Scholar

Copyright information

© CARS 2020

Authors and Affiliations

  1. 1.School of Information Science and EngineeringEast China University of Science and TechnologyShanghaiChina

Personalised recommendations