Does the META score evaluating osteoporotic and metastatic vertebral fractures have enough agreement to be used by orthopaedic surgeons with different levels of training?
Differentiating osteoporotic vertebral fractures (OVF) from metastatic vertebral fractures (MVF) is difficult. A magnetic resonance imaging (MRI)-based score (META score) aiming to differentiate OVF and MVF was recently published; however, an independent agreement assessment is required before the score is used. We performed such independent agreement evaluation, including raters with different levels of training.
Sixty-four patients with confirmed OVF or MVF were evaluated by six raters (three spine surgeons and three orthopaedic residents) using the META score. We used the intra-class correlation coefficient (ICC) to evaluate inter- and intra-observer agreement and the kappa statistic (κ) to determine the agreement for individual score criteria. We calculated the area under the receiver-operating characteristic curve (AUC) to establish the score accuracy.
The inter-observer agreement was poor [ICC = 0.22 (0.12–0.33)]; spine surgeons [ICC = 0.75 (0.66–0.83)] had better agreement than that of residents [ICC = 0.06 (− 0.07 to 0.23)]. The intra-observer agreement was poor [ICC = 0.15 (− 0.04 to 0.30)]; both spine surgeons [ICC = 0.21 (0.05–0.41)] and residents exhibited poor agreement [ICC = − 0.06 (− 0.40 to 0.20)]. The agreement for each specific criterion varied from κ = 0.24 to κ = 0.38. The AUC was 0.57 (0.64 for spine surgeons and 0.51 for residents, p < 0.01).
The inter-observer agreement using the META score was adequate for spine surgeons but not for residents; the intra-observer agreement was poor. These results do not support the standard use of the META score to differentiate OVF and MVF.
KeywordsAgreement study Osteoporotic vertebral fractures Metastatic vertebral fractures Magnetic resonance imaging Differential diagnosis
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
- 3.Baur A, Huber A, Ertl-Wagner B, Durr R, Zysk S, Arbogast S, Deimling M, Reiser M (2001) Diagnostic value of increased diffusion weighting of a steady-state free precession sequence for differentiating acute benign osteoporotic fractures from pathologic vertebral compression fractures. AJNR Am J Neuroradiol 22:366–372PubMedGoogle Scholar
- 9.Garnero P, Peterfy C, Zaim S, Schoenharting M (2005) Bone marrow abnormalities on magnetic resonance imaging are associated with type II collagen degradation in knee osteoarthritis: a three-month longitudinal study. Arthritis Rheum 52:2822–2829. https://doi.org/10.1002/art.21366 CrossRefPubMedGoogle Scholar
- 13.Fleiss J (1986) The design and analysis of clinical experiments. Wiley, New York, pp 1–31Google Scholar
- 16.Baker LL, Goodman SB, Perkash I, Lane B, Enzmann DR (1990) Benign versus pathologic compression fractures of vertebral bodies: assessment with conventional spin-echo, chemical-shift, and STIR MR imaging. Radiology 174:495–502. https://doi.org/10.1148/radiology.174.2.2296658 CrossRefPubMedGoogle Scholar
- 19.Chan JH, Peh WC, Tsui EY, Chau LF, Cheung KK, Chan KB, Yuen MK, Wong ET, Wong KP (2002) Acute vertebral body compression fractures: discrimination between benign and malignant causes using apparent diffusion coefficients. Br J Radiol 75:207–214. https://doi.org/10.1259/bjr.75.891.750207 CrossRefPubMedGoogle Scholar
- 23.Niemeyer T, Wolf A, Kluba S, Halm HF, Dietz K, Kluba T (2006) Interobserver and intraobserver agreement of Lenke and King classifications for idiopathic scoliosis and the influence of level of professional training. Spine (Phila Pa 1976) 31:2103–2107. https://doi.org/10.1097/01.brs.0000231434.93884.c9 (Discussion 2108) CrossRefGoogle Scholar
- 24.Clave A, Tristan L, Desseaux A, Gaucher F, Lefevre C, Stindel E (2016) Influence of experience on intra- and inter-observer reproducibility of the Crowe, Hartofilakidis and modified Cochin classifications. Orthop Traumatol Surg Res 102:155–159. https://doi.org/10.1016/j.otsr.2015.12.009 CrossRefPubMedGoogle Scholar
- 26.Urrutia J, Zamora T, Campos M, Yurac R, Palma J, Mobarec S, Prada C (2016) A comparative agreement evaluation of two subaxial cervical spine injury classification systems: the AOSpine and the Allen and Ferguson schemes. Eur Spine J 25:2185–2192. https://doi.org/10.1007/s00586-016-4498-0 CrossRefPubMedGoogle Scholar
- 29.Urrutia J, Zamora T, Yurac R, Campos M, Palma J, Mobarec S, Prada C (2015) An independent interobserver reliability and intraobserver reproducibility evaluation of the new AOSpine thoracolumbar spine injury classification system. Spine (Phila Pa 1976) 40:E54–E58. https://doi.org/10.1097/brs.0000000000000656 CrossRefGoogle Scholar
- 30.Turgut A, Kumbaraci M, Kalenderer O, Ilyas G, Bacaksiz T, Karapinar L (2016) Is surgeons’ experience important on intra- and inter-observer reliability of classifications used for adult femoral neck fracture? Acta Orthop Traumatol Turc 50:601–605. https://doi.org/10.1016/j.aott.2015.11.004 CrossRefPubMedGoogle Scholar
- 31.Urrutia J, Zamora T, Klaber I, Carmona M, Palma J, Campos M, Yurac R (2016) Do thoraco-lumbar spinal injuries classification systems exhibit lower inter- and intra-observer agreement than other fractures classifications?: a comparison using fractures of the trochanteric area of the proximal femur as contrast model. Injury 47:859–864. https://doi.org/10.1016/j.injury.2015.11.016 CrossRefPubMedGoogle Scholar