Skip to main content

Advertisement

SpringerLink
Log in

Inter-rater reliability in the radiological classification of renal injuries

  • Original Article
  • Published:
World Journal of Urology Aims and scope Submit manuscript

Abstract

Introduction

Although many radiologists invoke the surgical classification of renal injury proposed by the American Association for Surgery in Trauma (AAST), there has been only limited work on the role of the AAST system as an imaging stratification. The aim was to determine the inter-rater reliability (IRR) amongst radiologists and urologists using the AAST system.

Methods

A 1-year retrospective study of consecutive patients with computed tomography (CT) evidence of renal trauma managed at a Level 1 trauma center. Three radiologists and three urologists independently stratified the presentation CT findings according to the AAST renal trauma classification. Agreement between independent raters and mutually exclusive groups was determined utilizing weighted kappa coefficients.

Results

One hundred and one patients were included. Individual inter-observer agreements ranged from 54/101 (53.4%) to 62/101 (61.4%), with corresponding weighted kappa values from 0.61 to 0.69, constituting substantial agreement. Urologists achieved intra-disciplinary agreement in 49 cases (48.5%) and radiologists in 36 cases (35.6%). Six-reader agreement was achieved in 24 cases (23.7%). The AAST grade I injuries had the highest level of agreement, overall.

Conclusion

The finding of substantial IRR amongst radiologists and urologists utilizing the AAST system supports continued use of the broad parameters of the AAST system, with some modification in specific categories with lower agreement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Kawashima A, Sandler CM, Corl FM, West OC, Tamm EP, Fishman EK et al (2001) Imaging of renal trauma: a comprehensive review. Radiogr Rev Publ Radiol Soc N Am Inc 21(3):557–574

    CAS  Google Scholar 

  2. Lee YJ, Oh SN, Rha SE, Byun JY (2007) Renal trauma. Radiol Clin N Am 45(3):581–592, ix

  3. Summerton (Chair) ND, Kitrey ND, Kuehhas FE, Lumen N, Serafetinidis E, Sharma DM (2014) Guidelines on urological trauma. European Association Urolology. Available at https://uroweb.org/wpcontent/uploads/24-Urological-Trauma_LR.pdf

  4. Moore EE, Shackford SR, Pachter HL, McAninch JW, Browner BD, Champion HR et al (1989) Organ injury scaling: spleen, liver, and kidney. J Trauma 29(12):1664–1666

    Article  CAS  PubMed  Google Scholar 

  5. Santucci RA, McAninch JW, Safir M, Mario LA, Service S, Segal MR (2001) Validation of the American Association for the surgery of trauma organ injury severity scale for the kidney. J Trauma 50(2):195–200

    Article  CAS  PubMed  Google Scholar 

  6. Lynch TH, Martinez-Pineiro L, Plas E, Serafetinides E, Turkeri L, Santucci RA et al (2005) EAU guidelines on urological trauma. Eur Urol 47(1):1–15

    Article  PubMed  Google Scholar 

  7. Santucci RA, Wessells H, Bartsch G, Descotes J, Heyns CF, McAninch JW et al (2004) Evaluation and management of renal injuries: consensus statement of the renal trauma subcommittee. BJU Int 93(7):937–954

    Article  CAS  PubMed  Google Scholar 

  8. Martinez-Pineiro L, Djakovic N, Plas E, Mor Y, Santucci RA, Serafetinidis E et al (2010) EAU guidelines on urethral trauma. Eur Urol 57(5):791–803

    Article  PubMed  Google Scholar 

  9. Shariat SF, Roehrborn CG, Karakiewicz PI, Dhami G, Stage KH (2007) Evidence-based validation of the predictive value of the American Association for the surgery of trauma kidney injury scale. J Trauma 62(4):933–939

    Article  PubMed  Google Scholar 

  10. Heller MT, Schnor N (2014) MDCT of renal trauma: correlation to AAST organ injury scale. Clin Imaging 38(4):410–417

    Article  PubMed  Google Scholar 

  11. Kuo RL, Eachempati SR, Makhuli MJ, Reed RL 2nd (2002) Factors affecting management and outcome in blunt renal injury. World J Surg 26(4):416–419

    Article  PubMed  Google Scholar 

  12. Tinkoff G, Esposito TJ, Reed J, Kilgo P, Fildes J, Pasquale M et al (2008) American Association for the Surgery of Trauma Organ Injury Scale I: spleen, liver, and kidney, validation based on the National Trauma Data Bank. J Am Coll Surg 207(5):646–655

    Article  PubMed  Google Scholar 

  13. Alsikafi NF, Rosenstein DI (2006) Staging, evaluation, and nonoperative management of renal injuries. Urol Clin N Am 33(1):13–19

    Article  Google Scholar 

  14. Master VA, McAninch JW (2006) Operative management of renal injuries: parenchymal and vascular. Urol Clin N Am 3(1):21–31, v–vi

  15. Peng N, Wang X, Zhang Z, Fu S, Fan J, Zhang Y (2016) Diagnosis value of multi-slice spiral CT in renal trauma. J X-ray Sci Technol 24(5):649–655

    Article  Google Scholar 

  16. Joseph B, Khalil M, Rhee P (2015) Penetrating injuries to the spleen and kidney: an evolution in progress. Curr Trauma Rep 1(2):76–84

    Article  Google Scholar 

  17. Chong ST, Cherry-Bukowiec JR, Willatt JM, Kielar AZ (2016) Renal trauma: imaging evaluation and implications for clinical management. Abdom Radiol (New York) 41(8):1565–1579

    Article  Google Scholar 

  18. Steenburg SD, Persohn S, Shen C, Dunkle JW, Gussick SD, Petersen MJ et al (2015) Iterative reconstruction improves image quality and preserves diagnostic accuracy in the setting of blunt solid organ injuries. Emerg Radiol 22(1):43–51

    Article  PubMed  Google Scholar 

  19. Smith JK, Kenney PJ (2003) Imaging of renal trauma. Radiol Clin North Am 41(5):1019–1035

    Article  PubMed  Google Scholar 

  20. Heyns CF (2004) Renal trauma: indications for imaging and surgical exploration. BJU Int 93(8):1165–1170

    Article  CAS  PubMed  Google Scholar 

  21. Meng MV, Brandes SB, McAninch JW (1999) Renal trauma: indications and techniques for surgical exploration. World J Urol 17(2):71–77

    Article  CAS  PubMed  Google Scholar 

  22. Heyns CF, De Klerk DP, De Kock ML (1985) Nonoperative management of renal stab wounds. J Urol 134(2):239–242

    Article  CAS  PubMed  Google Scholar 

  23. Husmann DA, Gilling PJ, Perry MO, Morris JS, Boone TB (1993) Major renal lacerations with a devitalized fragment following blunt abdominal trauma: a comparison between nonoperative (expectant) versus surgical management. J Urol 150(6):1774–1777

    Article  CAS  PubMed  Google Scholar 

  24. Hammer CC, Santucci RA (2003) Effect of an institutional policy of nonoperative treatment of grades I to IV renal injuries. J Urol 169(5):1751–1753

    Article  PubMed  Google Scholar 

  25. van der Wilden GM, Velmahos GC, Joseph DK et al (2013) Successful nonoperative management of the most severe blunt renal injuries: a multicenter study of the research consortium of New England centers for trauma. JAMA Surg 148(10):924–931

    Article  PubMed  Google Scholar 

  26. McCombie SP, Thyer I, Corcoran NM, Rowling C, Dyer J, Le Roux A et al (2014) The conservative management of renal trauma: a literature review and practical clinical guideline from Australia and New Zealand. BJU Int 114(Suppl 1):13–21

    Article  PubMed  Google Scholar 

  27. Stein DM, Santucci RA (2015) An update on urotrauma. Curr Opin Urol 25(4):323–330

    PubMed  Google Scholar 

  28. Moudouni SM, Hadj Slimen M, Manunta A, Patard JJ, Guiraud PH, Guille F et al (2001) Management of major blunt renal lacerations: is a nonoperative approach indicated? Eur Urol 40(4):409–414

    Article  CAS  PubMed  Google Scholar 

  29. Matthews LA, Smith EM, Spirnak JP (1997) Nonoperative treatment of major blunt renal lacerations with urinary extravasation. J Urol 157(6):2056–2058

    Article  CAS  PubMed  Google Scholar 

  30. Traub KB, Hua V, Broman S, Stage KH (2001) Introduction of a genitourinary trauma database for use as a multi-institutional urologic trauma registry. J Trauma 51(2):336–339

    Article  CAS  PubMed  Google Scholar 

  31. Santucci RA (2016) 2015 William Hunter Harridge lecture: how did we go from operating on nearly all injured kidneys to operating on almost none of them? Am J Surg 211(3):501–505

    Article  PubMed  Google Scholar 

  32. Chakraverty S, Zealley I, Kessel D (2014) Damage control radiology in the severely injured patient: what the anaesthetist needs to know. Br J Anaesth 113(2):250–257

    Article  CAS  PubMed  Google Scholar 

  33. Kawashima A, Vrtiska TJ, LeRoy AJ, Hartman RP, McCollough CH, Bernard F, King J (2004) CT Urography. Radiogr Rev Publ Radiol Soc N Am Inc 24(suppl1):S35–S54

    Google Scholar 

  34. Bonatti M, Lombardo F, Vezzali N, Zamboni G, Ferro F, Pernter P et al (2015) MDCT of blunt renal trauma: imaging findings and therapeutic implications. Insights Imaging 6(2):261–272

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Reis LO, Kim FJ, Moore EE, Hirano ES, Fraga GP, Nascimento B et al (2013) Update in the classification and treatment of complex renal injuries. Revista do Colegio Brasileiro de Cirurgioes 40(4):347–350

    Article  PubMed  Google Scholar 

  36. Chiron P, Hornez E, Boddaert G, Dusaud M, Bayoud Y, Molimard B et al (2016) Grade IV renal trauma management. A revision of the AAST renal injury grading scale is mandatory. Eur J Trauma Emerg Surg 42(2):237–241

    Article  CAS  PubMed  Google Scholar 

  37. Dugi DD 3rd, Morey AF, Gupta A, Nuss GR, Sheu GL, Pruitt JH (2010) American Association for the surgery of trauma grade 4 renal injury substratification into grades 4a (low risk) and 4b (high risk). J Urol 183(2):592–597

    Article  PubMed  Google Scholar 

  38. Buckley JC, McAninch JW (2011) Revision of current American Association for the surgery of trauma renal injury grading system. J Trauma 70(1):35–37

    Article  PubMed  Google Scholar 

  39. Powers WFT, Beard LN Jr, Adams A, Kotwall CA, Clancy TV, Hope WW (2012) Solid organ injury grading in trauma: accuracy of grading by surgical residents. Am Surg 78(8):834–836

    PubMed  Google Scholar 

  40. du Plessis J, Pitcher R (2015) Towards task shifting? A comparison of the accuracy of acute trauma-radiograph reporting by medical officers and senior radiographers in an African hospital. Pan Afr Med J 21:308

    Article  PubMed  PubMed Central  Google Scholar 

  41. Dayal M, Gamanagatti S, Kumar A (2013) Imaging in renal trauma. World J Radiol 5(8):275–284

    Article  PubMed  PubMed Central  Google Scholar 

  42. Berry KJ, Johnston JE, Mielke PW Jr (2008) Weighted kappa for multiple raters. Percept Mot Skills 107(3):837–848

    Article  PubMed  Google Scholar 

  43. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159–174

    Article  CAS  PubMed  Google Scholar 

  44. Morey AF, Brandes S, Dugi DD 3rd, Armstrong JH, Breyer BN, Broghammer JA et al (2014) Urotrauma: AUA guideline. J Urol 192(2):327–335

    Article  PubMed  PubMed Central  Google Scholar 

  45. Buijze GA, Guitton TG, van Dijk CN, Ring D, The Science of Variation G (2012) Training improves interobserver reliability for the diagnosis of scaphoid fracture displacement. Clin Orthop Relat Res 470(7):2029–2034

    Article  PubMed  PubMed Central  Google Scholar 

  46. Alcott D, Swann R, Grafham A (1999) The effect of training on rater reliability on the scoring of the NART. Br J Clin Psychol 38(4):431–434

    Article  PubMed  Google Scholar 

  47. Lee AY, Wisner DJ, Aminololama-Shakeri S, Arasu VA, Feig SA, Hargreaves J et al (2017) Inter-reader variability in the use of BI-RADS descriptors for suspicious findings on diagnostic mammography: a multi-institution study of 10 academic radiologists. Acad Radiol 24(1):60–66

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to dedicate this work to the late Professor Chris Heyns who conceptualized this study. We are grateful for the countless contributions he has made in the field of Urology.

Author information

Authors and Affiliations

  1. Division of Radiodiagnosis, Department of Medical Imaging and Clinical Oncology, Faculty of Medicine and Health Sciences, Stellenbosch University and Tygerberg Hospital, Francie van Zijl Avenue, Tygerberg, Cape Town, 7505, South Africa

    Elias J. Pretorius, Stephanie Griffith-Richards, Willem G. Cilliers & Richard D. Pitcher

  2. Division of Urology, Department of Surgical Sciences, Faculty of Medicine and Health Sciences, Stellenbosch University and Tygerberg Hospital, Cape Town, South Africa

    Amir D. Zarrabi, Hilgard M. Ackermann, Catharina M. Meintjes & Alexander J. Szpytko

  3. Department of Statistics and Actuarial Science, Stellenbosch University, Stellenbosch, South Africa

    Justin Harvey

  4. Division of Epidemiology and Biostatistics, Department of Global Health, Centre for Evidence Based Healthcare, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa

    Moleen Zunza

Authors
  1. Elias J. Pretorius
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amir D. Zarrabi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stephanie Griffith-Richards
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Justin Harvey
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hilgard M. Ackermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Catharina M. Meintjes
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Willem G. Cilliers
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Moleen Zunza
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Alexander J. Szpytko
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Richard D. Pitcher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elias J. Pretorius.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pretorius, E.J., Zarrabi, A.D., Griffith-Richards, S. et al. Inter-rater reliability in the radiological classification of renal injuries. World J Urol 36, 489–496 (2018). https://doi.org/10.1007/s00345-017-2166-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00345-017-2166-6

Keywords

Search

Navigation