Skip to main content

An inter- and intra-rater agreement assessment of a novel classification of pyogenic spinal infections

Abstract

Purpose

Pola et al. described a clinical-radiological classification of pyogenic spinal infections (PSI) based on magnetic resonance imaging (MRI) features including vertebral destruction, soft tissue involvement, and epidural abscess, along with the neurological status. We performed an inter- and intra-observer agreement evaluation of this classification.

Methods

Complete MRI studies of 80 patients with PSI were selected and classified using the scheme described by Pola et al. by seven evaluators. After a four-week interval, all cases were presented to the same assessors in a random sequence for repeat assessment. We used the weighted kappa statistics (wκ) to establish the inter- and intra-observer agreement.

Results

The inter-observer agreement was substantial considering the main categories (wκ = 0.77; 0.71–0.82), but moderate considering the subtypes (wκ = 0.51; 0.45–0.58). The intra-observer agreement was substantial considering the main types (wκ = 0.65; 0.59–0.71), and moderate considering the subtypes (wκ = 0.58; 0.54–0.63).

Conclusion

The agreement at the main type level indicates that this classification allows adequate communication and may be used in clinical practice; at the subtypes level, the agreement is only moderate.

This is a preview of subscription content, access via your institution.

Fig. 1

References

  1. Duarte RM, Vaccaro AR (2013) Spinal infection: state of the art and management algorithm. Eur Spine J 22:2787–2799. https://doi.org/10.1007/s00586-013-2850-1

    Article  PubMed  PubMed Central  Google Scholar 

  2. Mylona E, Samarkos M, Kakalou E, Fanourgiakis P, Skoutelis A (2009) Pyogenic vertebral osteomyelitis: a systematic review of clinical characteristics. Semin Arthritis Rheum 39:10–17. https://doi.org/10.1016/j.semarthrit.2008.03.002

    CAS  Article  PubMed  Google Scholar 

  3. Fantoni M, Trecarichi EM, Rossi B, Mazzotta V, Di Giacomo G, Nasto LA, Di Meco E, Pola E (2012) Epidemiological and clinical features of pyogenic spondylodiscitis. Eur Rev Med Pharmacol Sci 16(Suppl 2):2–7

    PubMed  Google Scholar 

  4. Lora-Tamayo J, Euba G, Narvaez JA, Murillo O, Verdaguer R, Sobrino B, Narvaez J, Nolla JM, Ariza J (2011) Changing trends in the epidemiology of pyogenic vertebral osteomyelitis: the impact of cases with no microbiologic diagnosis. Semin Arthritis Rheum 41:247–255. https://doi.org/10.1016/j.semarthrit.2011.04.002

    Article  PubMed  Google Scholar 

  5. Pola E, Taccari F, Autore G, Giovannenze F, Pambianco V, Cauda R, Maccauro G, Fantoni M (2018) Multidisciplinary management of pyogenic spondylodiscitis: epidemiological and clinical features, prognostic factors and long-term outcomes in 207 patients. Eur Spine J 27:229–236. https://doi.org/10.1007/s00586-018-5598-9

    Article  PubMed  Google Scholar 

  6. Berbari EF, Kanj SS, Kowalski TJ, Darouiche RO, Widmer AF, Schmitt SK, Hendershot EF, Holtom PD, Huddleston PM 3rd, Petermann GW, Osmon DR, Infectious Diseases Society of A (2015) 2015 Infectious Diseases Society of America (IDSA) clinical practice guidelines for the diagnosis and treatment of native vertebral osteomyelitis in adults. Clin Infect Dis 61:e26-46. https://doi.org/10.1093/cid/civ482

    Article  PubMed  Google Scholar 

  7. Pola E, Autore G, Formica VM, Pambianco V, Colangelo D, Cauda R, Fantoni M (2017) New classification for the treatment of pyogenic spondylodiscitis: validation study on a population of 250 patients with a follow-up of 2 years. Eur Spine J 26:479–488. https://doi.org/10.1007/s00586-017-5043-5

    Article  PubMed  Google Scholar 

  8. Appalanaidu N, Shafafy R, Gee C, Brogan K, Karmani S, Morassi G, Elsayed S (2019) Predicting the need for surgical intervention in patients with spondylodiscitis: the Brighton Spondylodiscitis Score (BSDS). Eur Spine J 28:751–761. https://doi.org/10.1007/s00586-018-5775-x

    Article  PubMed  Google Scholar 

  9. Urrutia J, Besa P, Meissner-Haecker A, Delgado B (2020) An independent validation of the Brighton spondylodiscitis score and a proposal to modify the score. J Am Acad Orthop Surg 28:701–706. https://doi.org/10.5435/JAAOS-D-19-00505

    Article  PubMed  Google Scholar 

  10. Camino Willhuber G, Guiroy A, Zamorano J, Astur N, Valacco M (2021) Independent reliability analysis of a new classification for pyogenic spondylodiscitis. Global Spine J 11:669–673. https://doi.org/10.1177/2192568220919091

    Article  PubMed  Google Scholar 

  11. Rotondi MA, Donner A (2012) A confidence interval approach to sample size estimation for interobserver agreement studies with multiple raters and outcomes. J Clin Epidemiol 65:778–784. https://doi.org/10.1016/j.jclinepi.2011.10.019

    Article  PubMed  Google Scholar 

  12. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174

    CAS  Article  Google Scholar 

  13. Urrutia J, Meissner-Haecker A, Astur N, Valencia M, Yurac R, Camino-Willhuber G, Valacco M (2021) An independent inter- and intraobserver agreement assessment of the AOSpine sacral fracture classification system. Spine J 21:1143–1148. https://doi.org/10.1016/j.spinee.2021.02.005

    Article  PubMed  Google Scholar 

  14. Oner FC, Ramos LM, Simmermacher RK, Kingma PT, Diekerhof CH, Dhert WJ, Verbout AJ (2002) Classification of thoracic and lumbar spine fractures: problems of reproducibility. A study of 53 patients using CT and MRI. Eur Spine J 11:235–245. https://doi.org/10.1007/s00586-001-0364-8

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  15. Urrutia J, Zamora T, Yurac R, Campos M, Palma J, Mobarec S, Prada C (2015) An independent interobserver reliability and intraobserver reproducibility evaluation of the new AOSpine Thoracolumbar Spine Injury Classification System. Spine (Phila PA 1976) 40:E54–58. https://doi.org/10.1097/BRS.0000000000000656

  16. Urrutia J, Zamora T, Yurac R, Campos M, Palma J, Mobarec S, Prada C (2017) An independent inter- and intraobserver agreement evaluation of the AOSpine subaxial cervical spine injury classification system. Spine (Phila PA 1976) 42:298–303. https://doi.org/10.1097/BRS.0000000000001302

  17. Audige L, Bhandari M, Hanson B, Kellam J (2005) A concept for the validation of fracture classifications. J Orthop Trauma 19:401–406. https://doi.org/10.1097/01.bot.0000155310.04886.37

    Article  PubMed  Google Scholar 

  18. Audige L, Bhandari M, Kellam J (2004) How reliable are reliability studies of fracture classifications? A systematic review of their methodologies. Acta Orthop Scand 75:184–194. https://doi.org/10.1080/00016470412331294445

    Article  PubMed  Google Scholar 

  19. Muto M, Giurazza F, Guarnieri G, Izzo R, Diano A (2016) Neuroimaging of spinal instability. Magn Reson Imaging Clin N Am 24:485–494. https://doi.org/10.1016/j.mric.2016.04.003

    Article  PubMed  Google Scholar 

  20. Shah AA, Karhade AV, Bono CM, Harris MB, Nelson SB, Schwab JH (2019) Development of a machine learning algorithm for prediction of failure of nonoperative management in spinal epidural abscess. Spine J 19:1657–1665. https://doi.org/10.1016/j.spinee.2019.04.022

    Article  PubMed  Google Scholar 

  21. Kim SD, Melikian R, Ju KL, Zurakowski D, Wood KB, Bono CM, Harris MB (2014) Independent predictors of failure of nonoperative management of spinal epidural abscesses. Spine J 14:1673–1679. https://doi.org/10.1016/j.spinee.2013.10.011

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

To Dr. Pablo Besa for his assistance in the statistical analyses, and to Dr. Salvatore Lucchesi for his assistance collecting the cases.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Julio Urrutia.

Ethics declarations

Conflict of interest

The authors have no conflict of interest to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Camino-Willhuber, G., Delgado, B., Astur, N. et al. An inter- and intra-rater agreement assessment of a novel classification of pyogenic spinal infections. Eur Spine J 31, 448–453 (2022). https://doi.org/10.1007/s00586-021-07077-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00586-021-07077-8

Keywords

  • Agreement study
  • Pyogenic spinal infections
  • Spondylodiscitis
  • Neurological involvement
  • Vertebral osteomyelitis