Clinical Orthopaedics and Related Research®

, Volume 470, Issue 8, pp 2220–2226 | Cite as

Reliability of a Complication Classification System for Orthopaedic Surgery

  • Ernest L. SinkEmail author
  • Michael Leunig
  • Ira Zaltz
  • Jennifer Claire Gilbert
  • John Clohisy
  • Academic Network for Conservational Hip Outcomes Research Group
Clinical Research



Quality of health care and safety have been emphasized by various professional and governmental groups. However, no standardized method exists for grading and reporting complications in orthopaedic surgery. Conclusions regarding outcomes are incomplete without a standardized, objective complication grading scheme applied concurrently. The general surgery literature has the Clavien-Dindo classification that meets the above criteria.


We asked whether a previously reported classification would show high intraobserver and interobserver reliabilities when modified for orthopaedic surgery specifically looking at hip preservation surgery. We therefore determined the interreader and intrareader reliabilities of the adapted classification scheme as applied to hip preservation surgery.


We adapted the validated Clavien-Dindo complication classification system and tested its reliability for orthopaedic surgery, specifically hip preservation surgery. There are five grades based on the treatment required to manage the complication and the potential for long-term morbidity. Forty-four complication scenarios were created from a prospective multicenter database of hip preservation procedures and from the literature. Ten readers who perform hip surgery at eight centers in three countries graded the scenarios at two different times. Fleiss’ and Cohen’s κ statistics were performed for interobserver and intraobserver reliabilities, respectively.


The overall Fleiss’ κ value for interobserver reliability was 0.887 (95% CI, 0.855–0.891). The weighted κ was 0.925 (95% CI, 0.894–0.956) for Grade I, 0.838 (95% CI, 0.807–0.869) for Grade II, 0.87 (95% CI, 0.835–0.866) for Grade III, and 0.898 (95% CI, 0.866–0.929) for Grade IV. The Cohen’s κ value for intraobserver reliability was 0.891 (95% CI, 0.857–0.925).


The adapted classification system shows high interobserver and intraobserver reliabilities for grading of complications when applied to orthopaedic surgery looking at complications of hip preservation surgery. This grading scheme may facilitate standardization of complication reporting and make outcome studies more comparable.


Intensive Care Unit Admission Intraobserver Reliability Complication Classification Provide Quality Health Care Chance Adjust Agreement 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We thank Amy Monreal BS, Joseph T. Nguyen MPH, and Pan Zhaoxing PhD for assistance with preparation of this manuscript.

Supplementary material

11999_2012_2343_MOESM1_ESM.doc (102 kb)
Supplementary material 1 (DOC 102 kb)


  1. 1.
    Beaulé PE, Le Duff MJ, Zaragoza E. Quality of life following femoral head-neck osteochondroplasty for femoroacetabular impingement. J Bone Joint Surg Am. 2007; 89:773–779.PubMedCrossRefGoogle Scholar
  2. 2.
    Beck M, Leunig M, Parvizi J, Boutier V, Wyss D, Ganz R. Anterior femoroacetabular impingement: part II. Midterm results of surgical treatment. Clin Orthop Relat Res. 2004; 418:67–73.PubMedCrossRefGoogle Scholar
  3. 3.
    Chun YS, Vauthey JN, Ribero D, Donadon M, Mullen JT, Eng C, Madoff DC, Chang DZ, Ho L, Kopetz S, Wei SH, Curley SA, Abdalla EK. Systemic chemotherapy and two-stage hepatectomy for extensive bilateral colorectal liver metastases: perioperative safety and survival. J Gastrointest Surg. 2007;11:1498–1504; discussion 1504–1505.PubMedCrossRefGoogle Scholar
  4. 4.
    Clavien PA, Barkun J, de Oliveira ML, Vauthey JN, Dindo D, Schulick RD, de Santibañes E, Pekolj J, Slankamenac K, Bassi C, Graf R, Vonlanthen R, Padbury R, Cameron JL, Makuuchi M. The Clavien-Dindo classification of surgical complications: five-year experience. Ann Surg. 2009; 250:187–196.PubMedCrossRefGoogle Scholar
  5. 5.
    Clavien PA, Sanabria JR, Strasberg SM. Proposed classification of complications of surgery with examples of utility in cholecystectomy. Surgery. 1992; 111:518–526.PubMedGoogle Scholar
  6. 6.
    Clohisy JC, Schutz AL, St John L, Schoenecker PL, Wright RW. Periacetabular osteotomy: a systematic literature review. Clin Orthop Relat Res. 2009; 467:2041–2052.PubMedCrossRefGoogle Scholar
  7. 7.
    Clohisy JC, St John LC, Schutz AL. Surgical treatment of femoroacetabular impingement: a systematic review of the literature. Clin Orthop Relat Res. 2010; 468:555–564.PubMedCrossRefGoogle Scholar
  8. 8.
    Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Measurement. 1960; 20:37–46.CrossRefGoogle Scholar
  9. 9.
    Davey JP, Santore RF. Complications of periacetabular osteotomy. Clin Orthop Relat Res. 1999; 363:33–37.PubMedCrossRefGoogle Scholar
  10. 10.
    DeOliveira ML, Winter JM, Schafer M, Cunningham SC, Cameron JL, Yeo CJ, Clavien PA. Assessment of complications after pancreatic surgery: a novel grading system applied to 633 patients undergoing pancreaticoduodenectomy. Ann Surg. 2006; 244: 931–937; discussion 937–939.PubMedCrossRefGoogle Scholar
  11. 11.
    de Santibanes E, Ardiles V, Gadano A, Palavecino M, Pekolj J, Ciardullo M. Liver transplantation: the last measure in the treatment of bile duct injuries. World J Surg. 2008;32:1714–1721.PubMedCrossRefGoogle Scholar
  12. 12.
    Dindo D, Demartines N, Clavien PA. Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg. 2004; 240:205–213.PubMedCrossRefGoogle Scholar
  13. 13.
    Espinosa N, Beck M, Rothenfluh DA, Ganz R, Leunig M. Treatment of femoro-acetabular impingement: preliminary results of labral refixation. Surgical technique. J Bone Joint Surg Am. 2007;89(suppl 2):36–53.PubMedCrossRefGoogle Scholar
  14. 14.
    Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76:378–382.CrossRefGoogle Scholar
  15. 15.
    Goldhahn S, Sawaguchi T, Audigé L, Mundi R, Hanson B, Bhandari M, Goldhahn J. Complication reporting in orthopaedic trials: a systematic review of randomized controlled trials. J Bone Joint Surg Am. 2009; 91:1847–1853.PubMedCrossRefGoogle Scholar
  16. 16.
    Leunig M, Beaulé PE, Ganz R. The concept of femoroacetabular impingement: current status and future perspectives. Clin Orthop Relat Res. 2009; 467:616–622.PubMedCrossRefGoogle Scholar
  17. 17.
    McKay A, Sutherland FR, Bathe OF, Dixon E. Morbidity and mortality following multivisceral resections in compex hepatic and pancreatic surgery. J Gastrointest Surg. 2008; 12:86–90.PubMedCrossRefGoogle Scholar
  18. 18.
    Patel S, Cassuto J, Orloff M, Tsoulfas G, Zand M, Kashyap R, Jain A, Bozorgzadeh A, Abt P. Minimizing morbidity of organ donation: analysis of factors for perioperative complications after living-donor nephrectomy in the United States. Transplantation. 2008; 85:561–565.PubMedCrossRefGoogle Scholar
  19. 19.
    Permapongkosol S, Link RE, Su LM, Romero FR, Bagga HS, Pavlovich CP, Jarrett TW, Kavoussi LR. Complications of 2,775 urological laparoscopic procedures: 1993 to 2005. J Urol. 2007;177:580–585.CrossRefGoogle Scholar
  20. 20.
    Peters CL, Schabel K, Anderson L, Erickson J. Open treatment of femoroacetabular impingement is associated with clinical improvement and low complication rate at short-term followup. Clin Orthop Relat Res. 2010; 468:504–510.PubMedCrossRefGoogle Scholar
  21. 21.
    Philippon MJ, Briggs KK, Yen YM, Kuppersmith DA. Outcomes following hip arthroscopy for femoroacetabular impingement with associated chondrolabral dysfunction: minimum two-year follow-up. J Bone Joint Surg Br. 2009; 91:16–23.PubMedCrossRefGoogle Scholar
  22. 22.
    Reddy SK, Morse MA, Hurwitz HI, Bendell JC, Gan TJ, Hill SE, Clary BM. Addition of bevacizumab to irinotecan-and oxaliplatin-based preoperative chemotherapy regimens does not increase morbidity after resection of colorectal liver metastases. J Am Coll Surg. 2008;206:96–106.PubMedCrossRefGoogle Scholar
  23. 23.
    Reddy SK, Pawlik TM, Zorzi D, Gleisner AL, Ribero D, Assumpcao L, Barbas AS, Abdalla EK, Choti MA, Vauthey JN, Ludwig KA, Mantyh CR, Morse MA, Clary BM. Simultaneous resections of colorectal cancer and synchronous liver metastases: a multi-institutional analysis. Ann Surg Oncol. 2007; 14:3481–3491.PubMedCrossRefGoogle Scholar
  24. 24.
    Ribero D, Abdalla EK, Madoff DC, Donadon M, Loyer EM, Vauthey JN. Portal vein embolization before major hepatectomy and its effects on regeneration, resectability and outcome. Br J Surg. 2007; 94:1386–1394.PubMedCrossRefGoogle Scholar
  25. 25.
    Seda-Neto J, Godoy AL, Carone E, Pugliese V, Fonseca EA, Porta G, Pugliese R, Miura IK, Baggio V, Kondo M, Chapchap P. Left lateral segmentectomy for pediatic live-donor liver transplantation: special attention to segment IV complications. Transplantation. 2008; 86:697–701.PubMedCrossRefGoogle Scholar
  26. 26.
    Sink EL, Beaulé P, Sucato D, Kim YJ, Millis MB, Dayton M, Trousdale RT, Sierra RJ, Zaltz I, Schoenecker P, Monreal A, Clohisy J. Multicenter study of complications following surgical dislocation of the hip. J Bone Joint Surg Am. 2011; 93:1132–1136.PubMedCrossRefGoogle Scholar
  27. 27.
    Sundaram CP, Martin GL, Guise A, Bernie J, Bargman V, Milgrom M, Shalhav A, Govani M, Goggins W. Complications after a 5-year experience with laparoscopic donor nephrectomy: the Indiana University experience. Surg Endosc. 2007;21:724–728.PubMedCrossRefGoogle Scholar
  28. 28.
    Tamura S, Sugawara Y, Kaneko J, Yamashiki N, Kishi Y, Matsui Y, Kokudo N, Makuuchi M. Systematic grading of surgical complications in live liver donors according to Clavien’s system. Transpl Int. 2006; 19:982–987.PubMedCrossRefGoogle Scholar

Copyright information

© The Association of Bone and Joint Surgeons® 2012

Authors and Affiliations

  • Ernest L. Sink
    • 1
    Email author
  • Michael Leunig
    • 2
  • Ira Zaltz
    • 3
  • Jennifer Claire Gilbert
    • 4
  • John Clohisy
    • 5
  • Academic Network for Conservational Hip Outcomes Research Group
  1. 1.Hospital for Special SurgeryNew YorkUSA
  2. 2.Schulthess KlinikZurichSwitzerland
  3. 3.Oakland Orthopaedic SurgeonsRoyal OakUSA
  4. 4.Hospital for Special SurgeryNew YorkUSA
  5. 5.Washington University in St LouisSt LouisUSA

Personalised recommendations