Introduction

Complications following surgical interventions were identified as a global medical problem by the World Health Organisation (WHO) already in 20081. They cause individual suffering and dissatisfaction, reduce the quality of life of those affected and are associated with enormous costs2,3. Today, more and more surgical centres are assessing and comparing the quality of their services on the basis of the results obtained using the Clavien–Dindo classification. The Clavien–Dindo classification (CDC), first published in 19924 and substantially revised and validated in 20045, consists of 5 grades based on the therapeutic consequences of a postoperative complication. In 2014, the same research group complemented the CDC with the comprehensive complication index (CCI®), which reflects an overall postoperative morbidity based on the ranking of all reported CDC complications in a case6. The CDC is now the most widely used classification of postoperative complications and the most commonly used tool to assess and compare quality in surgery7,8,9,10.

In times of limited resources and increasing competition between different healthcare providers, it is not only important to collect this data, but also to make the collection as accurate and reliable as possible. It is well highlighted in the medical literature that consistent reporting of surgical outcome data is often lacking or of poor quality11.

Thus, the aim of this study was to evaluate the accuracy and reliability of the intern and resident-reported CDC and CCI® scores for postoperative complications in elective general and visceral surgery in a district hospital with predominantly low risk cases.

Methods

This retrospective, single-institution study included a prospective database collection from October 2020 to September 2021. All adult patients undergoing elective surgical treatment at the Department of General Surgery at Maennedorf Hospital, Switzerland, were included. All patients gave informed consent to use their data for research purposes via a general consent. The Cantonal Ethics Committee of Zurich concluded this study did not fall under the scope of the Swiss Human Research Act; thus, ethical approval was not required (Human Research Ordinance HRO; RS 810.301). The study was conducted in accordance to local legislation and the Declaration of Helsinki.

The project was initiated in October 2020 to test the quality of outcome reporting using CDC (the highest score) and CCI® Score for all operated patients during their hospitalization. The recording of post-operative complications based on the CDC was carried out by residents, who received a one-hour introduction on the CDC prior to the implementation. The CDC grade for each patient was determined by a ward intern or a first-year resident, and the CCI®score was automatically calculated using a predetermined formula6.

One year after the implementation of the CDC, senior consultant surgeons (I.R. and T.M.) conducted a thorough data audit, which included reviewing discharge summaries, electronic inpatient and outpatient medical records, blood test results, hospital medication lists, hospital notes, and diagnostic tests to confirm or refute the initial CDC grade and CCI® scores reported by the interns and first-year residents. All complications that occurred postoperatively during the hospitalization were carefully investigated.

Surgical procedures were divided into the following five specialties: bariatric surgery (BS), cholecystectomies (CHE), abdominal wall surgery (AWS), colorectal surgery (CRS), and Varia. The Varia group consisted mainly of local excision or surgical wound-conditioning procedures and laparoscopic hiatoplasty.

Statistical analysis

All data were subjected to statistical analysis and data visualisation using Statistical Package for the Social Sciences (SPSS) Version 28.0.1.1 for Mac/Windows (SPSS Inc., Chicago, IL, USA). Descriptive data are reported as numbers and percentages with mean and standard deviation.

Ethical approval

The Cantonal Ethics Committee of Zurich concluded this study did not fall under the scope of the Swiss Human Research Act; thus, ethical approval was not required. All patients gave informed consent to use their data for research purposes via a general consent.

Results

In total, 575 patients received an elective general or visceral surgical operation between 10/2020 and 09/2021 and were included in the analysis. There were 47% female patients, and mean age was 55.0 ± 16.8 years. The mean body mass index (BMI) of the study population was 29.7 ± 7.8 kg/m2. In terms of comorbidities, the majority of the study population was healthy, with 68.4% of patients having a Charlson comorbidity index score of 0 and 14.4% having a score of 1. The mean American Society of Anesthesiologists (ASA) score was 2, with 75% of patients being classified as ASA I (14%) or ASA II (61%) preoperatively. The surgical case mix consisted of 42% (n = 241) abdominal wall surgery, 23% (n = 132) bariatric surgery, 14% (n = 81) colorectal surgery, 13% (n = 76) cholecystectomy, and 8% (n = 45) Varia. Demographic and pre-operative clinical data are summarized in Table 1.

Table 1 Patient`s characteristics.
Full size table

Differences in outcome reporting before and after revision

The interns and first-year residents of the department reported postoperative complications in 7.3% of patients (42/575), while the retrospective review of the cases by senior consultant surgeons revealed a complication rate of 18.3% (105/575). Thus, interns and first-year residents failed to detect 60% (63/105) of patients with complications that occurred during hospitalization. In the 42 cases, in which complications were initially reported, the grading of complications was correct only in 14 cases (33.3%) (Tables 2 and 3). Among the remaining 28 cases with incorrect complication reports, 32% (9/28) showed revised CCI® scores that were almost 20 points higher as previously reported (Fig. 1). The total number of complications in these 9 cases was 51, of which only 13 were initially reported: 100% of grade I complications (17/17), 62% of grade II complications (13/21), 37.5% of grade IIIa complications (3/8), and all grade IVa complications (5/5) were not reported.

Table 2 Characteristics of Clavien–Dindo classification (CDC) reporting: reported versus revised cases.
Full size table
Table 3 Accuracy of complication-reporting.
Full size table
Figure 1
figure 1

Distribution of 9 cases with a CCI® score almost 20 points higher after revision. One line = one case. CCI®, comprehensive complications index.

Full size image

The 105 patients who had a complicated course after the review showed a total of 217 complications of varying degrees according to CDC (grade I: 101, grade II: 79, grade IIIa: 9, grade IIIb: 15, grade IVa: 12, grade IVb: 0, grade V: 1). The review of the Clavien–Dindo grades demonstrated that 94% (95/101) of grade I complications, 63.3% (50/79) of grade II complications, 11.1% (1/9) of grade IIIa complications, 26.6% (4/15) of grade IIIb complications, and 50% (6/12) of grade IVa complications were missed. The single grade V complication was correctly reported (Fig. 2). The mean reported CCI® score of the 42 cases with complications as reported by residents was 30.9 ± 18.1, whereas the mean revised CCI® score of these 42 cases was 39.4 ± 21.1 (Table 4).

Figure 2
figure 2

Number of reported and revised grades (according to Clavien Dindo Classification).

Full size image
Table 4 Comparison of reported CDC Grades before and after control.
Full size table

When comparing the mean CCI® scores among the 105 patients who experienced complications, it was observed that the score was 12.1 (± 18.9) before revision and increased to 23.8 (± 18.6) after revision. The change in CCI® scores after control and revision by experienced surgeons for each individual patient is graphically demonstrated in Fig. 3.

Figure 3
figure 3

All patients with complications (n = 105) and their consecutive CCI® scores before and after control. One line = one patient. CCI®, comprehensive complications index.

Full size image

Corrected postoperative morbidity related to surgical specialties

Overall, a total of 217 complications occurred in 105 of the 575 patients; 180 (83%) were minor (≤ grade II) complications (46.5% grade I and 36.5% grade II); 37 (17%) were major (≥ grade III) complications, which occurred in 27 patients.

The highest rate of postoperative complications was found in the colorectal surgery (CRS) group. 43.2% (35/81) of this population showed complications in the postoperative course with 10 out of all 15 grade IIIb complications and 5 out of all 12 grade IVa complications, respectively, assigned to CRS patients.

The varia surgery group had the highest CCI® score of all complicated cases (35.8 ± 28.2) due to one death. The overall CCI® score of all surgical cases was 4.4 ± 12.2, whereas the overall CCI® score of all cases with complications in all surgical subgroups was 23.8 ± 18.6. The analysis of CDC complication and its CCI® score based on case mix subgroups is shown in Table 5.

Table 5 Characteristic of surgical procedures according to CDC and CCI®
Full size table

Discussion

The results of the present study suggest that reporting of postoperative complications using the CDC and CCI® by inexperienced interns and residents is neither reliable nor accurate in a low volume district hospital.

Consistent and accurate classification of postoperative complications is critical for the comparison of surgical procedures and outcomes, leading to continuous improvements in patient care2,12,13. Currently, the CDC and CCI® are widely used as benchmark tools in surgery to establish a baseline for assessing and evaluating the quality of surgical care and outcomes14,15,16,17,18. Initially, the CDC was validated 30 days postoperatively. However, both the CDC and the CCI® have now been clinically and economically validated for 90 postoperatively19,20.

Improving the quality of healthcare and reducing its costs are essential components of modern medicine. The need for high quality healthcare has influenced surgical practice, the reporting of surgical outcomes, and the establishment of benchmarks for surgical treatments21. Patients and insurers now have greater access to medical information and expect surgeons to be up to date, competent, empathetic, and to provide high-quality care in an efficient manner22,23. The purpose of reporting and classifying adverse surgical outcomes is to investigate the underlying factors that contribute to them and to identify measures to prevent their recurrence in the future. This, in turn, will improve the quality of care and reduce the cost of complications. It also allows surgeons to objectively assess their surgical skills, improve surgical outcomes to the best possible result, reduce patient mortality rates, and improve patients’ postoperative quality of life.

In our study the review of data on postoperative complications reported by residents has shown a significantly high number of unreported complications, with a true overall complication rate of 18.3% (105/575) compared to 7.8% (42/575) before revision by senior consultants. More than 60% of patients with postoperative complications were not reported, and the accuracy of reported complications was only 33.3%. According to the literature, under-reporting of postoperative complications is a well-documented problem in surgery24,25,26,27,28. Several studies have shown that the true incidence of postoperative complications is often higher compared to the numbers reported in medical records and published studies29,30. Such under-reporting can have serious consequences for patient care, as it can obscure the true prevalence and impact of complications and hinder efforts to improve quality and develop evidence-based practice. To address this issue, many researchers have called for better standardization and documentation of postoperative complications, as well as improved education and awareness among healthcare providers. The use of standardized reporting systems, such as the Clavien–Dindo classification, has also been shown to be effective in improving the accuracy and completeness of outcome reporting. But not only the use of a standardized reporting system but also the use of an optimal data source for assessing postoperative complications is crucial. Current data show that the combination of different documentation levels, such as medical documentation, nursing documentation and the use of special morbidity forms, leads to a significant increase in data reliability with regard to the recording of postoperative complications31.

The most commonly missed postoperative complications in our study were Clavien–Dindo grades I (94.1%) and II (63.3%). Due to their relatively mild nature, the incidence of Clavien–Dindo grade I and grade II complications is often underreported32,33. These factors include a lack of standardization in reporting systems, inadequate documentation, limited communication between healthcare providers, a tendency to focus on more serious complications, and ignorance of the impact of minor complications on outcomes, quality of life and healthcare costs. To improve the accuracy and completeness of surgical outcome reporting, it is critical that all postoperative complications, including grade I and grade II complications, are documented and reported. This not only allows for a more accurate assessment of surgical outcomes, but it also identifies areas for improvement and ensures optimal patient care.

Despite training and additional surveillance by ward consultants during the first three months, there was no difference in the accuracy of the reported data compared to the rest of the study period. This shows, firstly, that even experienced surgeons may be insensitive to the accurate assessment of minor complications (those of grade II or lower) and, thus, need to be trained regularly (train the trainer). Secondly, constant monitoring of the documented outcome reporting is needed to secure the necessary data quality.

To our knowledge, our study is one of the first to compare the accuracy and completeness of outcome reporting using the CDC and CCI® in a low volume hospital setting. Dindo et al. found comparable results comparing resident surgical outcome reporting in a university hospital setting33. The authors showed that the training of inexperienced residents in using the CDC did not improve the overall reliability of outcome reporting using the CDC. However, these results do not argue against the use of such a valid outcome reporting system as CDC grading. Rather, it suggests that the importance of recording outcomes has not been equally understood by all protagonists of the healthcare system. There is no doubt that the recording of outcomes is associated with an additional expenditure of time due to the necessity of documentation. However, this should not have a negative impact on the correct outcome recording. In the end, it is necessary to create a culture in which the recording of outcomes is recognized as an essential parameter of an optimized healthcare system and implemented accordingly.

The overall CCI® score of the group of 42 patients with complications initially identified by the interns and junior residents was 30.9 and increased to 39.4 after review by the senior consultants. This relatively high CCI® already demonstrates that more severe complications were preferentially recognized and documented, but that the minor complications were obviously neglected. The average CCI® of all 105 patients who had a more or less severe complicative course after review by senior surgeons was 23.8. This value appears relatively high, considering the low complicative patient collective. However, considering the fact that in this collective every single deviation from the expected postoperative course, no matter how small, was actually evaluated as a CCI®—relevant complication, these data appear to be reliable and robust.

When looking at the average CCI® of all patients divided into the individual surgical subgroups, the extent of postoperative complications largely corresponds to the data in the scientific literature16,34,35,36,37,38.

This study supports previous evidence that the poor quality of data collection by surgical interns and residents is often neglected33. This may be due to the lack of time, lack of incentives, and lack of recognition of proper CDC reporting as a professional responsibility to assess both individual and institutional performance quality. Most of the interns and residents in this study were in their first and second year of training, which may lead to unfamiliarity with the normal, uncomplicated course of surgical treatment and a lack of awareness of minor complications.

Obviously, it is necessary to point out the importance of correct recording of surgical outcomes to interns and junior residents from the very beginning and to familiarize them with the instruments of outcome reporting in the long term. It seems mandatory to point out the importance of correct outcome reporting already in medical school education. If this has not taken place, as is common in Switzerland and Germany, consistent training in the classification and repetitive monitoring of the correct application of the classification during the residency seems sensible and necessary, respectively. In our view, transferring this activity to non-medical staff is not a solution, as experience has shown that the quality of the data recorded correlates with the degree of medical background knowledge.

Limitations

The present study has several limitations. First, the study was conducted in a single surgical institution with a predominance of low-risk surgical cases. Therefore, the generalizability of the findings to other institutions and higher-risk surgical cases may be limited. Future research should aim to evaluate the reliability of the database according to the CDC in a more diverse range of surgical settings.

Second, readmissions were not considered. Thus, only complications that occurred during the hospital admission were taken into account. This also means that complications 30 or 90 days after discharge are not included in this study.

Third, the retrospective design of the study prevented us from establishing causality, and only associations could be inferred from the data.

Forth, although organized teaching was provided to the interns and residents on two occasions, the effectiveness of this intervention was not formally evaluated. Although both interns and residents reported understanding the CDC, it is uncertain whether they were able to accurately apply this knowledge in practice. Further evaluation of educational interventions and their impact on the accuracy of complication reporting may be warranted.

Conclusion

Following their introduction in the early twenty-first century, the CDC and CCI® have been widely adopted by the research community for assessing outcome of surgical interventions. However, the accuracy and reliability of intern and resident reported CDC and CCI® may be suboptimal, potentially leading to inaccurate measures of quality performance. Therefore, it is recommended that qualified personnel verify the accuracy of complication data prior to dissemination for quality assessment purposes.

Date availability

The data set used for this study is available from the corresponding author upon reasonable request. The manuscript was prepared using the STROBE checklist as guidance.