Abstract
Objectives
It remains difficult to characterize the source of pain in knee joints either using radiographs or magnetic resonance imaging (MRI). We sought to determine if advanced machine learning methods such as deep neural networks could distinguish knees with pain from those without it and identify the structural features that are associated with knee pain.
Methods
We constructed a convolutional Siamese network to associate MRI scans obtained on subjects from the Osteoarthritis Initiative (OAI) with frequent unilateral knee pain comparing the knee with frequent pain to the contralateral knee without pain. The Siamese network architecture enabled pairwise learning of information from two-dimensional (2D) sagittal intermediate-weighted turbo spin echo slices obtained from similar locations on both knees. Class activation mapping (CAM) was utilized to create saliency maps, which highlighted the regions most associated with knee pain. The MRI scans and the CAMs of each subject were reviewed by an expert radiologist to identify the presence of abnormalities within the model-predicted regions of high association.
Results
Using 10-fold cross-validation, our model achieved an area under curve (AUC) value of 0.808. When individuals whose knee WOMAC pain scores were not discordant were excluded, model performance increased to 0.853. The radiologist review revealed that about 86% of the cases that were predicted correctly had effusion-synovitis within the regions that were most associated with pain.
Conclusions
This study demonstrates a proof of principle that deep learning can be applied to assess knee pain from MRI scans.
Key Points
• Our article is the first to leverage a deep learning framework to associate MR images of the knee with knee pain.
• We developed a convolutional Siamese network that had the ability to fuse information from multiple two-dimensional (2D) MRI slices from the knee with pain and the contralateral knee of the same individual without pain to predict unilateral knee pain.
• Our model achieved an area under curve (AUC) value of 0.808. When individuals who had WOMAC pain scores that were not discordant for knees (pain discordance < 3) were excluded, model performance increased to 0.853.
Similar content being viewed by others
Change history
22 July 2020
The original version of this article, published on 13 February 2020, unfortunately contained a mistake.
Abbreviations
- AUC:
-
Area under curve
- BMI:
-
Body mass index
- BML:
-
Bone marrow lesion
- CAM:
-
Class activation mapping
- CNN:
-
Convolutional neural network
- GPU:
-
Graphical processing unit
- KL:
-
Kellgren-Lawrence
- ML:
-
Machine learning
- MOAKS:
-
MRI Osteoarthritis Knee Score
- MRI:
-
Magnetic resonance imaging
- OA:
-
Osteoarthritis
- OAI:
-
Osteoarthritis initiative
- ROC:
-
Receiver operating characteristic
- WOMAC:
-
Western Ontario and McMaster Universities Osteoarthritis Index
References
Cross M, Smith E, Hoy D et al (2014) The global burden of hip and knee osteoarthritis: estimates from the global burden of disease 2010 study. Ann Rheum Dis 73:1323–1330
Neogi T (2013) The epidemiology and impact of pain in osteoarthritis. Osteoarthritis Cartilage 21:1145–1153
Hunter DJ, Guermazi A, Roemer F, Zhang Y, Neogi T (2013) Structural correlates of pain in joints with osteoarthritis. Osteoarthritis Cartilage 21:1170–1178
Felson DT, Chaisson CE, Hill CL et al (2001) The association of bone marrow lesions with pain in knee osteoarthritis. Ann Intern Med 134:541–549
Felson DT, Niu J, Guermazi A et al (2007) Correlation of the development of knee pain with enlarging bone marrow lesions on magnetic resonance imaging. Arthritis Rheum 56:2986–2992
Hill CL, Gale DG, Chaisson CE et al (2001) Knee effusions, popliteal cysts, and synovial thickening: association with knee pain in osteoarthritis. J Rheumatol 28:1330–1337
Hill CL, Hunter DJ, Niu J et al (2007) Synovitis detected on magnetic resonance imaging and its relation to pain and cartilage loss in knee osteoarthritis. Ann Rheum Dis 66:1599–1603
Dimitroulas T, Duarte RV, Behura A, Kitas GD, Raphael JH (2014) Neuropathic pain in osteoarthritis: a review of pathophysiological mechanisms and implications for treatment. Semin Arthritis Rheum 44:145–154
Finan PH, Buenaver LF, Bounds SC et al (2013) Discordance between pain and radiographic severity in knee osteoarthritis: findings from quantitative sensory testing of central sensitization. Arthritis Rheum 65:363–372
Clauw DJ, Witter J (2009) Pain and rheumatology: thinking outside the joint. Arthritis Rheum 60:321–324
O'Neill TW, Felson DT (2018) Mechanisms of osteoarthritis (OA) pain. Curr Osteoporos Rep 16:611–616
Bedson J, Croft PR (2008) The discordance between clinical and radiographic knee osteoarthritis: a systematic search and summary of the literature. BMC Musculoskelet Disord 9:116
Guermazi A, Zaim S, Taouli B, Miaux Y, Peterfy CG, Genant HG (2003) MR findings in knee osteoarthritis. Eur Radiol 13:1370–1386
Sowers MF, Hayes C, Jamadar D et al (2003) Magnetic resonance-detected subchondral bone marrow and cartilage defect characteristics associated with pain and X-ray-defined knee osteoarthritis. Osteoarthritis Cartilage 11:387–393
Yusuf E, Kortekaas MC, Watt I, Huizinga TW, Kloppenburg M (2011) Do knee abnormalities visualised on MRI explain knee pain in knee osteoarthritis? A systematic review. Ann Rheum Dis 70:60–67
Wenham CY, Conaghan PG (2009) Imaging the painful osteoarthritic knee joint: what have we learned? Nat Clin Pract Rheumatol 5:149–158
Litjens G, Kooi T, Bejnordi BE et al (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444
Hamm CA, Wang CJ, Savic LJ et al (2019) Deep learning for liver tumor diagnosis part I: development of a convolutional neural network classifier for multi-phasic MRI. Eur Radiol 29:3338–3347
Kiryu S, Yasaka K, Akai H et al (2019) Deep learning to differentiate parkinsonian disorders separately using single midsagittal MR imaging: a proof of concept study. Eur Radiol 29:6891–6899
Laukamp KR, Thiele F, Shakirin G et al (2019) Fully automated detection and segmentation of meningiomas using deep learning on routine multiparametric MRI. Eur Radiol 29:124–132
Vreemann S, Dalmis MU, Bult P et al (2019) Amount of fibroglandular tissue FGT and background parenchymal enhancement BPE in relation to breast cancer risk and false positives in a breast MRI screening program : a retrospective cohort study. Eur Radiol 29:4678–4690
Wang CJ, Hamm CA, Savic LJ et al (2019) Deep learning for liver tumor diagnosis part II: convolutional neural network interpretation using radiologic imaging features. Eur Radiol 29:3348–3357
Norman B, Pedoia V, Majumdar S (2018) Use of 2D U-net convolutional neural networks for automated cartilage and meniscus segmentation of knee MR imaging data to determine Relaxometry and Morphometry. Radiology 288:177–185
Ambellan F, Tack A, Ehlke M, Zachow S (2019) Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: data from the osteoarthritis initiative. Med Image Anal 52:109–118
Gaj S, Yang M, Nakamura K, Li X (2019) Automated cartilage and meniscus segmentation of knee MRI with conditional generative adversarial networks. Magn Reson Med. https://doi.org/10.1002/mrm.28111
Byra M, Wu M, Zhang X et al (2020) Knee menisci segmentation and relaxometry of 3D ultrashort echo time cones MR imaging using attention U-net with transfer learning. Magn Reson Med 83:1109–1122
Tack A, Mukhopadhyay A, Zachow S (2018) Knee menisci segmentation using convolutional neural networks: data from the osteoarthritis initiative. Osteoarthritis Cartilage 26:680–688
Prasoon A, Petersen K, Igel C, Lauze F, Dam E, Nielsen M (2013) Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. Med Image Comput Comput Assist Interv 16:246–253
Bien N, Rajpurkar P, Ball RL et al (2018) Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. PLoS Med 15:e1002699
Englund M, Niu J, Guermazi A et al (2007) Effect of meniscal damage on the development of frequent knee pain, aching, or stiffness. Arthritis Rheum 56:4048–4054
Eckstein F, Benichou O, Wirth W et al (2009) Magnetic resonance imaging-based cartilage loss in painful contralateral knees with and without radiographic joint space narrowing: data from the osteoarthritis initiative. Arthritis Rheum 61:1218–1225
Cibere J, Sayre EC, Guermazi A et al (2011) Natural history of cartilage damage and osteoarthritis progression on magnetic resonance imaging in a population-based cohort with knee pain. Osteoarthritis Cartilage 19:683–688
Kim HA, Kim I, Song YW et al (2011) The association between meniscal and cruciate ligament damage and knee pain in community residents. Osteoarthritis Cartilage 19:1422–1428
Peterfy CG, Schneider E, Nevitt M (2008) The osteoarthritis initiative: report on the design rationale for the magnetic resonance imaging protocol for the knee. Osteoarthritis Cartilage 16:1433–1441
Fawaz-Estrup F (2004) The osteoarthritis initiative: an overview. Med Health R I 87:169–171
Chopra S, Hadsell R, LeCun Y (2005) Learning a similarity metric discriminatively, with application to face verification. Courant Institute of Mathematical Sciences
Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. 2016 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr). https://doi.org/10.1109/Cvpr.2016.319:2921-2929
Bedson J, Croft PR (2008) The discordance between clinical and radiographic knee osteoarthritis: a systematic search and summary of the literature. BMC Musculoskelet Disord 9:467–411
Wenham CY, Conaghan PG (2009) Imaging the painful osteoarthritic knee joint: what have we learned? Nat Clin Pract Rheumatol 5:149–158
Sayre EC, Guermazi A, Esdaile JM et al (2017) Associations between MRI features versus knee pain severity and progression: data from the Vancouver Longitudinal Study of Early Knee Osteoarthritis. PLoS One 12:e0176833–e0176812
Neogi T, Felson D, Niu J et al (2009) Association between radiographic features of knee osteoarthritis and pain: results from two cohort studies. BMJ 339:b2844–b2844
Javaid MK, Kiran A, Guermazi A et al (2012) Individual magnetic resonance imaging and radiographic features of knee osteoarthritis in subjects with unilateral knee pain: the health, aging, and body composition study. Arthritis Rheum 64:3246–3255
Minciullo L, Parkes MJ, Felson DT, Cootes TF (2018) Comparing image analysis approaches versus expert readers: the relation of knee radiograph features to knee pain. Ann Rheum Dis 77:1606–1609
Dolz J, Desrosiers C, Ben Ayed I (2018) 3D fully convolutional networks for subcortical segmentation in MRI: a large-scale study. Neuroimage 170:456–470
Funding
This work was supported in part by the National Center for Advancing Translational Sciences, National Institutes of Health, through BU-CTSI Grant (1UL1TR001430), a Scientist Development Grant (17SDG33670323) from the American Heart Association, and a Research Award from the Hariri Institute for Computing and Computational Science and Engineering and Digital Health Initiative at Boston University, and NIH grants to VBK, DTF, and TDC (5U01AG-018820 and 1R01AR070139 and supported by the NIHR Manchester Biomedical Research Centre).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Guarantor
The scientific guarantor of this publication is Dr. David Felson (dfelson@bu.edu).
Conflict of interest
Ali Guermazi is shareholder of BICL and consultant to Pfizer, AstraZeneca, TissueGene, Roche, Galapagos, and MerckSerono.
Statistics and biometry
No complex statistical methods were necessary for this paper. Additionally, Dr. Kolachalama has needed statistical expertise.
Informed consent
Written informed consent was not required for this study because de-identified data is publicly available (Osteoarthritis Initiative: https://oai.nih.gov). At the time of enrollment, informed consent and ethical committee approvals were obtained by the OAI investigators.
Ethical approval
Institutional Review Board approval was not required because the data was obtained from a public database (Osteoarthritis Initiative: https://oai.nih.gov).
Study subjects or cohorts overlap
The cohort has been reported in https://oai.nih.gov.
Methodology
• This is a case-control study.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chang, G.H., Felson, D.T., Qiu, S. et al. Assessment of knee pain from MR imaging using a convolutional Siamese network. Eur Radiol 30, 3538–3548 (2020). https://doi.org/10.1007/s00330-020-06658-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00330-020-06658-3