Skip to main content
SpringerLink
Log in

A Coarse-to-Fine Framework for Automated Knee Bone and Cartilage Segmentation Data from the Osteoarthritis Initiative

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Knee osteoarthritis (OA) is a degenerative joint disease that is prevalent in advancing age. The pathology of OA disease is still unclear, and there are no effective interventions that can completely alter the OA disease process. Magnetic resonance (MR) image evaluation is sensitive for depicting early changes of knee OA, and therefore important for early clinical intervention for relieving the symptom. Automated cartilage segmentation based on MR images is a vital step in experimental longitudinal studies to follow-up the patients and prospectively define a new quantitative marker from OA progression. In this paper, we develop a deep learning–based coarse-to-fine approach for automated knee bone, cartilage, and meniscus segmentation with high computational efficiency. The proposed method is evaluated using two-fold cross-validation on 507 MR volumes (81,120 slices) with OA from the Osteoarthritis Initiative (OAI)1 dataset. The mean dice similarity coefficients (DSCs) of femoral bone (FB), tibial bone (TB), femoral cartilage (FC), and tibial cartilage (TC) separately are 99.1%, 98.2%, 90.9%, and 85.8%. The time of segmenting each patient is 12 s, which is fast enough to be used in clinical practice. Our proposed approach may provide an automated toolkit to help computer-aided quantitative analyses of OA images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. R. C. Lawrence et al., Estimates of the prevalence of arthritis and other rheumatic conditions in the United States: Part II, Arthritis & Rheumatism, vol. 58, no. 1, pp. 26-35, 2008.

    Article  Google Scholar 

  2. K. D. Brandt, Diagnosis and nonsurgical management of osteoarthritis. Professional Communications, 2010.

  3. Y. Zhang and J. M. Jordan, Epidemiology of osteoarthritis, Clinics in geriatric medicine, vol. 26, no. 3, pp. 355-369, 2010.

    Article  Google Scholar 

  4. C. R. Chu, A. A. Williams, C. H. Coyle, and M. E. Bowers, Early diagnosis to enable early treatment of pre-osteoarthritis, Arthritis research & therapy, vol. 14, no. 3, p. 212, 2012.

    Article  Google Scholar 

  5. E. Yelin, S. Weinstein, and T. King, The burden of musculoskeletal diseases in the United States, in Seminars in arthritis and rheumatism, 2016, vol. 46, no. 3, p. 259.

    Article  Google Scholar 

  6. A. Mendy, J. Park, and E. R. Vieira, Osteoarthritis and risk of mortality in the USA: a population-based cohort study, International Journal of Epidemiology, vol. 47, no. 6, pp. 1821-1829, 2018.

    Article  Google Scholar 

  7. D. Bhatia, T. Bejarano, and M. Novo, Current interventions in the management of knee osteoarthritis, Journal of pharmacy & bioallied sciences, vol. 5, no. 1, p. 30, 2013.

    Article  Google Scholar 

  8. M. T. Nieminen, V. Casula, M. T. Nevalainen, and S. Saarakkala, Osteoarthritis year in review 2018: imaging, Osteoarthritis and cartilage, vol. 27, no. 3, pp. 401-411, 2019.

    Article  CAS  Google Scholar 

  9. H. Shim, S. Chang, C. Tao, J.-H. Wang, C. K. Kwoh, and K. T. Bae, Knee cartilage: efficient and reproducible segmentation on high-spatial-resolution MR images with the semiautomated graph-cut algorithm method, Radiology, vol. 251, no. 2, pp. 548-556, 2009.

    Article  Google Scholar 

  10. Y. Du, R. Almajalid, J. Shan, and M. Zhang, A novel method to predict knee osteoarthritis progression on MRI using machine learning methods, IEEE transactions on nanobioscience, vol. 17, no. 3, pp. 228-236, 2018.

    Article  Google Scholar 

  11. J. Jaremko, R. Cheng, R. Lambert, A. Habib, and J. Ronsky, Reliability of an efficient MRI-based method for estimation of knee cartilage volume using surface registration, Osteoarthritis and cartilage, vol. 14, no. 9, pp. 914-922, 2006.

    Article  CAS  Google Scholar 

  12. F. Eckstein and W. Wirth, Quantitative cartilage imaging in knee osteoarthritis, Arthritis, vol. 2011, 2011.

  13. J. Fripp, S. Crozier, S. K. Warfield, and S. Ourselin, Automatic segmentation and quantitative analysis of the articular cartilages from magnetic resonance images of the knee, IEEE transactions on medical imaging, vol. 29, no. 1, pp. 55-64, 2009.

    Article  Google Scholar 

  14. Z. Javaid, M. G. Boocock, P. J. McNair, and C. P. Unsworth, Contour interpolated radial basis functions with spline boundary correction for fast 3D reconstruction of the human articular cartilage from MR images, Medical Physics, vol. 43, no. 3, pp. 1187-1199, 2016.

    Article  Google Scholar 

  15. J. G. Lee, S. Gumus, C. H. Moon, C. K. Kwoh, and K. T. Bae, Fully automated segmentation of cartilage from the MR images of knee using a multi‐atlas and local structural analysis method, Medical physics, vol. 41, no. 9, p. 092303, 2014.

    Article  Google Scholar 

  16. Y. Yin, X. Zhang, R. Williams, X. Wu, D. D. Anderson, and M. Sonka, LOGISMOS—layered optimal graph image segmentation of multiple objects and surfaces: cartilage segmentation in the knee joint, IEEE transactions on medical imaging, vol. 29, no. 12, pp. 2023-2037, 2010.

    Article  Google Scholar 

  17. P. M. Cashman, R. I. Kitney, M. A. Gariba, and M. E. Carter, Automated techniques for visualization and mapping of articular cartilage in MR images of the osteoarthritic knee: a base technique for the assessment of microdamage and submicro damage, IEEE transactions on nanobioscience, vol. 99, no. 1, pp. 42-51, 2002.

    Article  Google Scholar 

  18. H. Z. Tameem and U. S. Sinha, Automated image processing and analysis of cartilage MRI: enabling technology for data mining applied to osteoarthritis, in AIP conference proceedings, 2007, vol. 953, no. 1, pp. 262–276: American Institute of Physics.

  19. G. Vincent, C. Wolstenholme, I. Scott, and M. Bowes, Fully automatic segmentation of the knee joint using active appearance models, Medical Image Analysis for the Clinic: A Grand Challenge, vol. 1, p. 224, 2010.

    Google Scholar 

  20. V. Pedoia, S. Majumdar, and T. M. Link, Segmentation of joint and musculoskeletal tissue in the study of arthritis, Magnetic Resonance Materials in Physics, Biology and Medicine, vol. 29, no. 2, pp. 207-221, 2016.

    Article  CAS  Google Scholar 

  21. M. Zhang et al., Development of a rapid knee cartilage damage quantification method using magnetic resonance images, BMC musculoskeletal disorders, vol. 15, no. 1, p. 264, 2014.

    Article  Google Scholar 

  22. M. Zhang, J. B. Driban, L. L. Price, G. H. Lo, E. Miller, and T. E. McAlindon, Development of a rapid cartilage damage quantification method for the lateral tibiofemoral compartment using magnetic resonance images: data from the osteoarthritis initiative, BioMed research international, vol. 2015, 2015.

  23. F. Ambellan, A. Tack, M. Ehlke, and S. Zachow, Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: Data from the Osteoarthritis Initiative, Medical image analysis, vol. 52, pp. 109-118, 2019.

    Article  Google Scholar 

  24. Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, Unet++: Redesigning skip connections to exploit multiscale features in image segmentation, IEEE transactions on medical imaging, vol. 39, no. 6, pp. 1856-1867, 2019.

    Article  Google Scholar 

  25. O. Ronneberger, P. Fischer, and T. Brox, U-net: Convolutional networks for biomedical image segmentation, in International Conference on Medical image computing and computer-assisted intervention, 2015, pp. 234–241: Springer.

  26. B. Park, H. Park, S. M. Lee, J. B. Seo, and N. Kim, Lung segmentation on HRCT and volumetric CT for diffuse interstitial lung disease using deep convolutional neural networks, Journal of Digital Imaging, vol. 32, no. 6, pp. 1019-1026, 2019.

    Article  Google Scholar 

  27. G. Singadkar, A. Mahajan, M. Thakur, and S. Talbar, Deep deconvolutional residual network based automatic lung nodule segmentation, Journal of Digital Imaging, pp. 1–7, 2020.

  28. H. Seim, D. Kainmueller, H. Lamecker, M. Bindernagel, J. Malinowski, and S. Zachow, Model-based auto-segmentation of knee bones and cartilage in MRI data, Proc. Medical Image Analysis for the Clinic: A Grand Challenge. Bejing, China, pp. 215–223, 2010.

  29. M. H. Hesamian, W. Jia, X. He, and P. Kennedy, Deep learning techniques for medical image segmentation: achievements and challenges, Journal of digital imaging, vol. 32, no. 4, pp. 582-596, 2019.

    Article  Google Scholar 

  30. J. Long, E. Shelhamer, and T. Darrell, Fully convolutional networks for semantic segmentation, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431–3440.

  31. K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556, 2014.

  32. K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.

  33. D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980, 2014.

  34. T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, Focal loss for dense object detection, in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2980–2988.

  35. J. M. Jordan et al., Prevalence of knee symptoms and radiographic and symptomatic knee osteoarthritis in African Americans and Caucasians: the Johnston County Osteoarthritis Project, The Journal of rheumatology, vol. 34, no. 1, pp. 172-180, 2007.

    PubMed  Google Scholar 

  36. O. Oktay et al., Attention u-net: learning where to look for the pancreas, arXiv preprint arXiv:1804.03999, 2018.

  37. D. Dreizin, Y. Zhou, Y. Zhang, N. Tirada, and A. L. Yuille, Performance of a deep learning algorithm for automated segmentation and quantification of traumatic pelvic hematomas on CT, Journal of digital imaging, vol. 33, no. 1, pp. 243-251, 2020.

    Article  Google Scholar 

  38. Q. Yu, L. Xie, Y. Wang, Y. Zhou, E. K. Fishman, and A. L. Yuille, Recurrent saliency transformation network: Incorporating multi-stage visual cues for small organ segmentation, in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8280–8289.

Download references

Acknowledgements

We thank Zuse Institute Berlin for kindly providing annotations.

Funding

This work was supported in part by the National Institutes of Health (NIH) under Grant 1R01DE027027-02 and Grant 1U01 AR069395-03 (XZ).

Author information

Author notes

  1. Yang Deng and Lei You contributed equally to this work.

Authors and Affiliations

  1. School of Biomedical Informatics, The University of Texas Health Science Center At Houston, Houston, TX, 77030, USA

    Yang Deng, Lei You, Yanfei Wang & Xiaobo Zhou

Authors
  1. Yang Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei You
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanfei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaobo Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaobo Zhou.

Ethics declarations

Competing Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deng, Y., You, L., Wang, Y. et al. A Coarse-to-Fine Framework for Automated Knee Bone and Cartilage Segmentation Data from the Osteoarthritis Initiative. J Digit Imaging 34, 833–840 (2021). https://doi.org/10.1007/s10278-021-00464-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-021-00464-z

Keywords

Search

Navigation