Advertisement

Does the META score evaluating osteoporotic and metastatic vertebral fractures have enough agreement to be used by orthopaedic surgeons with different levels of training?

  • Julio Urrutia
  • Pablo Besa
  • Sergio Morales
  • Antonio Parlange
  • Sebastian Flores
  • Mauricio Campos
  • Sebastian Mobarec
Original Article

Abstract

Purpose

Differentiating osteoporotic vertebral fractures (OVF) from metastatic vertebral fractures (MVF) is difficult. A magnetic resonance imaging (MRI)-based score (META score) aiming to differentiate OVF and MVF was recently published; however, an independent agreement assessment is required before the score is used. We performed such independent agreement evaluation, including raters with different levels of training.

Methods

Sixty-four patients with confirmed OVF or MVF were evaluated by six raters (three spine surgeons and three orthopaedic residents) using the META score. We used the intra-class correlation coefficient (ICC) to evaluate inter- and intra-observer agreement and the kappa statistic (κ) to determine the agreement for individual score criteria. We calculated the area under the receiver-operating characteristic curve (AUC) to establish the score accuracy.

Results

The inter-observer agreement was poor [ICC = 0.22 (0.12–0.33)]; spine surgeons [ICC = 0.75 (0.66–0.83)] had better agreement than that of residents [ICC = 0.06 (− 0.07 to 0.23)]. The intra-observer agreement was poor [ICC = 0.15 (− 0.04 to 0.30)]; both spine surgeons [ICC = 0.21 (0.05–0.41)] and residents exhibited poor agreement [ICC = − 0.06 (− 0.40 to 0.20)]. The agreement for each specific criterion varied from κ = 0.24 to κ = 0.38. The AUC was 0.57 (0.64 for spine surgeons and 0.51 for residents, p < 0.01).

Conclusion

The inter-observer agreement using the META score was adequate for spine surgeons but not for residents; the intra-observer agreement was poor. These results do not support the standard use of the META score to differentiate OVF and MVF.

Graphical abstract

These slides can be retrieved under Electronic Supplementary Material.

Keywords

Agreement study Osteoporotic vertebral fractures Metastatic vertebral fractures Magnetic resonance imaging Differential diagnosis 

Notes

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Supplementary material

586_2018_5694_MOESM1_ESM.ppt (226 kb)
Supplementary material 1 (PPT 226 kb)

References

  1. 1.
    Kato S, Hozumi T, Yamakawa K, Saito M, Goto T, Kondo T (2015) META: an MRI-based scoring system differentiating metastatic from osteoporotic vertebral fractures. Spine J 15:1563–1570.  https://doi.org/10.1016/j.spinee.2015.03.011 CrossRefPubMedGoogle Scholar
  2. 2.
    Takigawa T, Tanaka M, Sugimoto Y, Tetsunaga T, Nishida K, Ozaki T (2017) Discrimination between malignant and benign vertebral fractures using magnetic resonance imaging. Asian Spine J 11:478–483.  https://doi.org/10.4184/asj.2017.11.3.478 CrossRefPubMedPubMedCentralGoogle Scholar
  3. 3.
    Baur A, Huber A, Ertl-Wagner B, Durr R, Zysk S, Arbogast S, Deimling M, Reiser M (2001) Diagnostic value of increased diffusion weighting of a steady-state free precession sequence for differentiating acute benign osteoporotic fractures from pathologic vertebral compression fractures. AJNR Am J Neuroradiol 22:366–372PubMedGoogle Scholar
  4. 4.
    Lems WF (2007) Clinical relevance of vertebral fractures. Ann Rheum Dis 66:2–4.  https://doi.org/10.1136/ard.2006.058313 CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Lindsay R, Silverman SL, Cooper C, Hanley DA, Barton I, Broy SB, Licata A, Benhamou L, Geusens P, Flowers K, Stracke H, Seeman E (2001) Risk of new vertebral fracture in the year following a fracture. JAMA 285:320–323CrossRefPubMedGoogle Scholar
  6. 6.
    Nevitt MC, Ettinger B, Black DM, Stone K, Jamal SA, Ensrud K, Segal M, Genant HK, Cummings SR (1998) The association of radiographically detected vertebral fractures with back pain and function: a prospective study. Ann Intern Med 128:793–800CrossRefPubMedGoogle Scholar
  7. 7.
    Kubota T, Yamada K, Ito H, Kizu O, Nishimura T (2005) High-resolution imaging of the spine using multidetector-row computed tomography: differentiation between benign and malignant vertebral compression fractures. J Comput Assist Tomogr 29:712–719CrossRefPubMedGoogle Scholar
  8. 8.
    Laredo JD, Lakhdari K, Bellaiche L, Hamze B, Janklewicz P, Tubiana JM (1995) Acute vertebral collapse: CT findings in benign and malignant nontraumatic cases. Radiology 194:41–48.  https://doi.org/10.1148/radiology.194.1.7997579 CrossRefPubMedGoogle Scholar
  9. 9.
    Garnero P, Peterfy C, Zaim S, Schoenharting M (2005) Bone marrow abnormalities on magnetic resonance imaging are associated with type II collagen degradation in knee osteoarthritis: a three-month longitudinal study. Arthritis Rheum 52:2822–2829.  https://doi.org/10.1002/art.21366 CrossRefPubMedGoogle Scholar
  10. 10.
    Shih TT, Huang KM, Li YW (1999) Solitary vertebral collapse: distinction between benign and malignant causes using MR patterns. J Magn Reson Imag 9:635–642CrossRefGoogle Scholar
  11. 11.
    Walter SD, Eliasziw M, Donner A (1998) Sample size and optimal designs for reliability studies. Stat Med 17:101–110CrossRefPubMedGoogle Scholar
  12. 12.
    Hallgren KA (2012) Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol 8:23–34CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Fleiss J (1986) The design and analysis of clinical experiments. Wiley, New York, pp 1–31Google Scholar
  14. 14.
    Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174CrossRefPubMedGoogle Scholar
  15. 15.
    Yuh WT, Zachar CK, Barloon TJ, Sato Y, Sickels WJ, Hawes DR (1989) Vertebral compression fractures: distinction between benign and malignant causes with MR imaging. Radiology 172:215–218.  https://doi.org/10.1148/radiology.172.1.2740506 CrossRefPubMedGoogle Scholar
  16. 16.
    Baker LL, Goodman SB, Perkash I, Lane B, Enzmann DR (1990) Benign versus pathologic compression fractures of vertebral bodies: assessment with conventional spin-echo, chemical-shift, and STIR MR imaging. Radiology 174:495–502.  https://doi.org/10.1148/radiology.174.2.2296658 CrossRefPubMedGoogle Scholar
  17. 17.
    Rupp RE, Ebraheim NA, Coombs RJ (1995) Magnetic resonance imaging differentiation of compression spine fractures or vertebral lesions caused by osteoporosis or tumor. Spine (Phila Pa 1976) 20:2499–2503 (Discussion 2504) CrossRefGoogle Scholar
  18. 18.
    Tan DY, Tsou IY, Chee TS (2002) Differentiation of malignant vertebral collapse from osteoporotic and other benign causes using magnetic resonance imaging. Ann Acad Med Singap 31:8–14PubMedGoogle Scholar
  19. 19.
    Chan JH, Peh WC, Tsui EY, Chau LF, Cheung KK, Chan KB, Yuen MK, Wong ET, Wong KP (2002) Acute vertebral body compression fractures: discrimination between benign and malignant causes using apparent diffusion coefficients. Br J Radiol 75:207–214.  https://doi.org/10.1259/bjr.75.891.750207 CrossRefPubMedGoogle Scholar
  20. 20.
    Jung HS, Jee WH, McCauley TR, Ha KY, Choi KH (2003) Discrimination of metastatic from acute osteoporotic compression spinal fractures with MR imaging. Radiographics 23:179–187.  https://doi.org/10.1148/rg.231025043 CrossRefPubMedGoogle Scholar
  21. 21.
    Schwaiger BJ, Gersing AS, Baum T, Krestan CR, Kirschke JS (2016) Distinguishing benign and malignant vertebral fractures using CT and MRI. Semin Musculoskelet Radiol 20:345–352.  https://doi.org/10.1055/s-0036-1592433 CrossRefPubMedGoogle Scholar
  22. 22.
    Brorson S, Bagger J, Sylvest A, Hrobjartsson A (2002) Improved interobserver variation after training of doctors in the Neer system. A randomised trial. J Bone Jt Surg Br 84:950–954CrossRefGoogle Scholar
  23. 23.
    Niemeyer T, Wolf A, Kluba S, Halm HF, Dietz K, Kluba T (2006) Interobserver and intraobserver agreement of Lenke and King classifications for idiopathic scoliosis and the influence of level of professional training. Spine (Phila Pa 1976) 31:2103–2107.  https://doi.org/10.1097/01.brs.0000231434.93884.c9 (Discussion 2108) CrossRefGoogle Scholar
  24. 24.
    Clave A, Tristan L, Desseaux A, Gaucher F, Lefevre C, Stindel E (2016) Influence of experience on intra- and inter-observer reproducibility of the Crowe, Hartofilakidis and modified Cochin classifications. Orthop Traumatol Surg Res 102:155–159.  https://doi.org/10.1016/j.otsr.2015.12.009 CrossRefPubMedGoogle Scholar
  25. 25.
    Kalenderer O, Agus H, Ozcalabi IT, Ozluk S (2005) The importance of surgeons’ experience on intraobserver and interobserver reliability of classifications used for Perthes disease. J Pediatr Orthop 25:460–464CrossRefPubMedGoogle Scholar
  26. 26.
    Urrutia J, Zamora T, Campos M, Yurac R, Palma J, Mobarec S, Prada C (2016) A comparative agreement evaluation of two subaxial cervical spine injury classification systems: the AOSpine and the Allen and Ferguson schemes. Eur Spine J 25:2185–2192.  https://doi.org/10.1007/s00586-016-4498-0 CrossRefPubMedGoogle Scholar
  27. 27.
    Urrutia J, Zamora T, Yurac R, Campos M, Palma J, Mobarec S, Prada C (2015) An independent inter- and intra-observer agreement evaluation of the AOSpine subaxial cervical spine injury classification system. Spine (Phila Pa 1976).  https://doi.org/10.1097/brs.0000000000001302 Google Scholar
  28. 28.
    Buijze GA, Guitton TG, van Dijk CN, Ring D, Science of Variation G (2012) Training improves interobserver reliability for the diagnosis of scaphoid fracture displacement. Clin Orthop Relat Res 470:2029–2034.  https://doi.org/10.1007/s11999-012-2260-4 CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Urrutia J, Zamora T, Yurac R, Campos M, Palma J, Mobarec S, Prada C (2015) An independent interobserver reliability and intraobserver reproducibility evaluation of the new AOSpine thoracolumbar spine injury classification system. Spine (Phila Pa 1976) 40:E54–E58.  https://doi.org/10.1097/brs.0000000000000656 CrossRefGoogle Scholar
  30. 30.
    Turgut A, Kumbaraci M, Kalenderer O, Ilyas G, Bacaksiz T, Karapinar L (2016) Is surgeons’ experience important on intra- and inter-observer reliability of classifications used for adult femoral neck fracture? Acta Orthop Traumatol Turc 50:601–605.  https://doi.org/10.1016/j.aott.2015.11.004 CrossRefPubMedGoogle Scholar
  31. 31.
    Urrutia J, Zamora T, Klaber I, Carmona M, Palma J, Campos M, Yurac R (2016) Do thoraco-lumbar spinal injuries classification systems exhibit lower inter- and intra-observer agreement than other fractures classifications?: a comparison using fractures of the trochanteric area of the proximal femur as contrast model. Injury 47:859–864.  https://doi.org/10.1016/j.injury.2015.11.016 CrossRefPubMedGoogle Scholar
  32. 32.
    Sanders R (1997) The problem with apples and oranges [editorial]. J Orthop Trauma 11:465–466CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Orthopaedic Surgery, School of MedicinePontificia Universidad Catolica de ChileSantiagoChile

Personalised recommendations