A Lightweight CNN and Joint Shape-Joint Space (\(JS^2\)) Descriptor for Radiological Osteoarthritis Detection

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1248)


Knee osteoarthritis (OA) is very common progressive and degenerative musculoskeletal disease worldwide creates a heavy burden on patients with reduced quality of life and also on society due to financial impact. Therefore, any attempt to reduce the burden of the disease could help both patients and society. In this study, we propose a fully automated novel method, based on combination of joint shape and convolutional neural network (CNN) based bone texture features, to distinguish between the knee radiographs with and without radiographic osteoarthritis. Moreover, we report the first attempt at describing the bone texture using CNN. Knee radiographs from Osteoarthritis Initiative (OAI) and Multicenter Osteoarthritis (MOST) studies were used in the experiments. Our models were trained on 8953 knee radiographs from OAI and evaluated on 3445 knee radiographs from MOST. Our results demonstrate that fusing the proposed shape and texture parameters achieves the state-of-the art performance in radiographic OA detection yielding area under the ROC curve (AUC) of \(95.21\%\).


Knee osteoarthritis Joint space width Joint shape Bone texture 



The OAI is a public-private partnership comprised of five contracts (N01-AR-2-2258; N01-AR-2-2259; N01-AR-2-2260; N01-AR-2-2261; N01-AR-2-2262) funded by the National Institutes of Health, a branch of the Department of Health and Human Services, and conducted by the OAI Study Investigators. Private funding partners include Merck Research Laboratories; Novartis Pharmaceuticals Corporation, GlaxoSmithKline; and Pfizer, Inc. Private sector funding for the OAI is managed by the Foundation for the National Institutes of Health. This manuscript was prepared using an OAI public use data set and does not necessarily reflect the opinions or views of the OAI investigators, the NIH, or the private funding partners.

Multicenter Osteoarthritis Study (MOST) Funding Acknowledgment. MOST is comprised of four cooperative grants (Felson – AG18820; Torner – AG18832, Lewis – AG18947, and Nevitt – AG19069) funded by the National Institutes of Health, a branch of the Department of Health and Human Services, and conducted by MOST study investigators. This manuscript was prepared using MOST data and does not necessarily reflect the opinions or views of MOST investigators.

We would like to acknowledge the strategic funding of the University of Oulu, Infotech Oulu.

We gratefully acknowledge the help received from Aleksei Tiulpin who extracted the landmarks using BoneFinder® and the support of NVIDIA Corporation with the donation of the Quadro P6000 GPU used in this research.


  1. 1.
    Bijlsma, J.W., Berenbaum, F., Lafeber, F.P.: Osteoarthritis: an update with relevance for clinical practice. Lancet 377, 2115–2126 (2011)CrossRefGoogle Scholar
  2. 2.
    Altman, R.D.: Early management of osteoarthritis. Am. J. Managed Care 16, S41–S47 (2010)Google Scholar
  3. 3.
    Heidari, B.: Knee osteoarthritis prevalence, risk factors, pathogenesis and features: part I. Caspian J. Intern. Med. 2, 205 (2011)Google Scholar
  4. 4.
    Haq, I., Murphy, E., Dacre, J.: Osteoarthritis. Postgrad. Med. J. 79, 377–383 (2003). Scholar
  5. 5.
    Kellgren, J., Lawrence, J.: Radiological assessment of osteo-arthrosis. Ann. Rheum. Dis. 16, 494 (1957)CrossRefGoogle Scholar
  6. 6.
    Köse, Ö., et al.: Inter-and intraobserver reliabilities of four different radiographic grading scales of osteoarthritis of the knee joint. J. Knee Surg. 3, 247–253 (2018)CrossRefGoogle Scholar
  7. 7.
    Lynch, J., Hawkes, D., Buckland-Wright, J.: Analysis of texture in macroradiographs of osteoarthritic knees, using the fractal signature. Phys. Med. Biol. 36, 709 (1991)CrossRefGoogle Scholar
  8. 8.
    Kraus, V.B., et al.: Predictive validity of radiographic trabecular bone texture in knee osteoarthritis: the osteoarthritis research society international/foundation for the national institutes of health osteoarthritis biomarkers consortium. Arthritis Rheumatol. 70, 80–87 (2018)CrossRefGoogle Scholar
  9. 9.
    Hirvasniemi, J., et al.: Quantification of differences in bone texture from plain radiographs in knees with and without osteoarthritis. Osteoarthritis Cartilage 22, 1724–1731 (2014)CrossRefGoogle Scholar
  10. 10.
    Thomson, J., O’Neill, T., Felson, D., Cootes, T.: Automated shape and texture analysis for detection of osteoarthritis from radiographs of the knee. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 127–134 (2015)Google Scholar
  11. 11.
    Janvier, T., Jennane, R., Toumi, H., Lespessailles, E.: Subchondral tibial bone texture predicts the incidence of radiographic knee osteoarthritis: data from the Osteoarthritis Initiative. Osteoarthritis Cartilage 25, 2047–2054 (2017)CrossRefGoogle Scholar
  12. 12.
    Antony, J., McGuinness, K., O’Connor, N.E., Moran, K.: Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. In: 23rd International Conference on Pattern Recognition (ICPR), pp. 1195–1200 (2016)Google Scholar
  13. 13.
    Norman, B., Pedoia, V., Noworolski, A., Link, T.M., Majumdar, S.: Applying densely connected convolutional neural networks for staging osteoarthritis severity from plain radiographs. J. Digit. Imaging 32, 471–477 (2019)CrossRefGoogle Scholar
  14. 14.
    Tiulpin, A., Thevenot, J., Rahtu, E., Lehenkari, P., Saarakkala, S.: Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learningbased approach. Sci. Rep. 8, 1727 (2018)CrossRefGoogle Scholar
  15. 15.
    Górriz, M., Antony, J., McGuinness, K., Giró-i-Nieto, X., O’Connor, N.E.: Assessing knee OA severity with CNN attention-based end-to-end architectures. In: International Conference on Medical Imaging with Deep Learning, pp. 197–214 (2019)Google Scholar
  16. 16.
    Abedin, J., et al.: Predicting knee osteoarthritis severity: comparative modeling based on patient’s data and plain X-ray images. Sci. Rep. 9, 1–11 (2019)CrossRefGoogle Scholar
  17. 17.
    Minciullo, L. & Cootes, T.: Fully automated shape analysis for detection of osteoarthritis from lateral knee radiographs. In: 23rd International Conference on Pattern Recognition (ICPR), pp. 3787–3791 (2016)Google Scholar
  18. 18.
    Haverkamp, D.J., Schiphof, D., Bierma-Zeinstra, S.M., Weinans, H., Waarsing, J.H.: Variation in joint shape of osteoarthritic knees. Arthritis Rheum. 63, 3401–3407 (2011)CrossRefGoogle Scholar
  19. 19.
    Martinez, A.M., et al.: Discovering knee osteoarthritis bone shape features using deep learning. Osteoarthritis Cartilage 27, S386–S387 (2019)CrossRefGoogle Scholar
  20. 20.
    Ahlback, S.: Osteoarthrosis of the knee. a radiographic investigation. Acta Radiol. 227, 7–72 (1968)Google Scholar
  21. 21.
    Geirhos, R., et al.: ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness (2018) arXiv preprint arXiv:1811.12231
  22. 22.
    Bayramoglu, N., Tiulpin, A., Hirvasniemi, J., Nieminen, M.T., Saarakkala, S.: Adaptive segmentation of knee radiographs for selecting the optimal ROI in texture analysis. Osteoarthritis Cartilage (2020). Scholar
  23. 23.
    Buckland-Wright, C.: Subchondral bone changes in hand and knee osteoarthritis detected by radiography. Osteoarthritis Cartilage 12, 10–19 (2004)CrossRefGoogle Scholar
  24. 24.
    Kamibayashi, L., Wyss, U., Cooke, T., Zee, B.: Trabecular microstructure in the medial condyle of the proximal tibia of patients with knee osteoarthritis. Bone 17, 27–35 (1995)CrossRefGoogle Scholar
  25. 25.
    Goldring, S.R.: Role of bone in osteoarthritis pathogenesis. Med. Clin. North Am. 93, 25–35 (2009)CrossRefGoogle Scholar
  26. 26.
    Mehta, N., et al.: Comparison of 2 radiographic techniques for measurement of tibiofemoral joint space width. Orthop. J. Sports Med. 5, 2325967117728675 (2017)Google Scholar
  27. 27.
    Neumann, G., et al.: Location specific radiographic joint space width for osteoarthritis progression. Osteoarthritis Cartilage 17, 761–765 (2009)CrossRefGoogle Scholar
  28. 28.
    Guermazi, A., Hunter, D.J., Roemer, F.W.: Plain radiography and magnetic resonance imaging diagnostics in osteoarthritis: validated staging and scoring. JBJS 91, 54–62 (2009)CrossRefGoogle Scholar
  29. 29.
    Duryea, J., Zaim, S., Genant, H.: New radiographic-based surrogate outcome measures for osteoarthritis of the knee. Osteoarthritis Cartilage 11, 102–110 (2003)CrossRefGoogle Scholar
  30. 30.
    Lindner, C., et al.: Fully automatic segmentation of the proximal femur using random forest regression voting. IEEE Trans. Med. Imaging 32, 1462–1472 (2013)CrossRefGoogle Scholar
  31. 31.
    Aspden, R.M., Saunders, F.: Osteoarthritis as an organ disease: from the cradle to the grave. Eur. Cells Mater. 37, 74–87 (2019)CrossRefGoogle Scholar
  32. 32.
    MacKay, J.W., et al.: Subchondral bone in osteoarthritis: association between MRI texture analysis and histomorphometry. Osteoarthritis Cartilage 25, 700–707 (2017)CrossRefGoogle Scholar
  33. 33.
    Janvier, T., Toumi, H., Harrar, K., Lespessailles, E., Jennane, R.: ROI impact on the characterization of knee osteoarthritis using fractal analysis. In: 2015 International Conference on Image Processing Theory, Tools and Applications (IPTA), 304–308 (2015)Google Scholar
  34. 34.
    Ojala, T., Pietikäinen, M., Mäenpää, T.: Gray scale and rotation invariant texture classification with local binary patterns. In: European Conference on Computer Vision, pp. 404–420 (2000)Google Scholar
  35. 35.
    Lynch, J., Hawkes, D., Buckland-Wright, J.: A robust and accurate method for calculating the fractal signature of texture in macroradiographs of osteoarthritic knees. Med. Inf. 16, 241–251 (1991)Google Scholar
  36. 36.
    Hirvasniemi, J., et al.: Correlation of subchondral bone density and structure from plain radiographs with micro computed tomography ex vivo. Ann. Biomed. Eng. 44, 1698–1709 (2016)CrossRefGoogle Scholar
  37. 37.
    Lin, T.-Y., Maji, S.: Improved bilinear pooling with CNNs in BMVC (2017)Google Scholar
  38. 38.
    Zhang, R.: Making convolutional networks shift-invariant again (2019). arXiv preprint arXiv:1904.11486
  39. 39.
    Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  40. 40.
    Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Advances in NIPS, pp. 8024–8035 (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Research Unit of Medical Imaging, Physics and TechnologyUniversity of OuluOuluFinland
  2. 2.Department of Diagnostic RadiologyOulu University HospitalOuluFinland
  3. 3.Medical Research CenterUniversity of Oulu and Oulu University HospitalOuluFinland

Personalised recommendations