Advertisement

MTL, a composite measure for objectively profiling hospitals on surgical performance in colorectal cancer surgery

  • Julia Hardt
  • Fabian Doyon
  • C. Klinger
  • H. J. Buhr
  • Stefan PostEmail author
Open Access
Original Article
  • 251 Downloads

Abstract

Purpose

MTL is a composite outcome measure based on routine administrative data defined as (a) postoperative mortality and/or (b) postoperative transfer to another hospital and/or (c) length of hospital stay ≥ the prespecified time period. Aim of the present study was to investigate MTL for profiling hospitals on surgical performance in colorectal cancer surgery, using data from the national registers of the German Society of General and Visceral Surgery (DGAV) and to determine the time interval for length of stay with the highest accuracy regarding major complications (Clavien-Dindo grade ≥ 3).

Methods

All patients undergoing colorectal cancer resection between January 2010 and February 2017 were included. MTL rates were calculated and compared to well-established single outcome measures using multivariate regression analysis. For each outcome measure, postoperative complications were tested regarding their predictability.

Results

Data from 14,978 patients were analyzed. Length of stay was significantly prolonged if postoperative complications occurred (p < 0.0001). Thirty-day mortality and the indication for a transfer to another hospital mainly resulted from cardiopulmonary complications. MTL occurs significantly more often than any of the single-outcome parameters. The time interval of 22 days demonstrated the highest accuracy regarding severe complications (Clavien-Dindo grade ≥ 3).

Conclusions

MTL reflects the complete spectrum of postoperative complications. Compared to individual surgical outcome parameters, MTL may have a better discriminatory power and is therefore suitable to mirror surgical quality. Because of its high accuracy regarding surgical major morbidity, 22 days is the best cut-off for length of stay within the German healthcare system.

Keywords

Composite outcome measure Colorectal cancer Colorectal surgery Quality Morbidity 

Introduction

There is a growing need for quality indicators which reliably and objectively measure the outcome quality of surgical performance and clearly identify high- and low-performing hospitals. Composite outcome measures combine different quality indicators in one score to increase the reliability of hospital performance assessment. There is already some valuable evidence from the literature that composite measures may be a more effective approach for capturing a hospital’s surgical quality [1, 2, 3, 4, 5, 6].

Currently, most quality improvement projects focus on single-quality indicators, which may not sufficiently mirror overall quality of the surgical performance in the context of colorectal cancer care [2].

To test the recently introduced composite outcome measure MTL with regard to its suitability for profiling hospitals on surgical performance, we analyzed the complete dataset from the colorectal cancer study, documentation, and quality centre (StuDoQ) registers of the German Society of General and Visceral Surgery (DGAV). The main objective of the present analysis was to find out which time interval covered by MTL has the highest accuracy regarding severe complications defined as Clavien-Dindo grade ≥ 3. The time interval of 30 days was primarily chosen because a follow up of 30 days is traditionally used in the literature when short-term outcomes after surgery are assessed and reported. Therefore, the 30-day interval was established as a common threshold value and trigger point for the three components of MTL.

Methods

All patients undergoing colorectal resection for adenocarcinoma of the colon or rectum between January 2010 and February 2017, registered in the StuDoQ|ColonCancer Register or the StuDoQ|RectalCancer Register of the DGAV, were included in a database retrieval. After exclusion of resections without restoration of bowel continuity (no anastomosis), palliative surgery or surgery for recurrent colorectal cancer, and after excluding cases with incomplete datasets, 14,978 patients with complete datasets for the follow up duration of 30 days were further analyzed. According to data protection laws, the excluded patients were not analyzed and the retrieved datasets were reduced to the necessary outcome measurements. These included patients and tumor characteristics (e.g., height, weight, stage), procedure characteristics (e.g., surgical technique, duration of the surgical procedure) and postoperative outcomes (e.g., surgical complications graded according to the Clavien-Dindo classification, mortality, surgical site infection). Data quality was monitored on-site and verified by the administration particularly on M, T, and L for all hospitals submitting data to the StuDoQ registers in order to apply for certification as a colorectal center (45% of the included hospitals). These independent control mechanisms minimized the risk of detection and attrition bias. The remaining hospitals entered data into the register voluntarily as part of their institutional quality control. MTL30, a recently described composite outcome measure [7], is defined as (a) mortality and/or (b) transfer to another hospital (other than rehabilitation clinic) within 30 days after the index operation, and/or (c) postoperative length of hospital stay ≥ 30 days. Currently, MTL is only used as a quality outcome measure within the StuDoQ registers and within the framework of the voluntary colorectal cancer center certification process. MTL rates were calculated using the register data and compared to well-established single-outcome measures, including mortality, morbidity, and length of stay. For each postoperative day 15 to 30, contingency tables were used to calculate the accuracy of the Clavien-Dindo Classification ≥ 3. The day with the highest accuracy was then used for an alternative MTL classification.

The study was approved by the institutional ethics committee.

Statistics

Patient characteristics were calculated using summary tables. Length of stay analysis was performed via product-limit survival estimates. Comparisons of the outcome measurements with the potential effects (surgical complications, stroke, myocardial infarction) were analyzed applying full model fitted multiple regression analysis. Because of its composite character, Clavien-Dindo was not included in the multiple regression analysis. MTL intends to measure the surgical quality in a hospital. Therefore, the funnel plots were performed at a hospital level, whereas the remaining figures and the contingency tables (Tables 1 and 2) show results at a patient level. Funnel plots and control limits were constructed according to Spiegelhalter et al. Control limits indicate a range in which the values of a quality indicator would statistically be expected to fall [8]. All statistical analyses were performed with SAS version 9.3 using SAS Enterprise Guide 4.2 (SAS, Cary, NC, USA).
Table 1

Contingency table for MTL30 versus Clavien-Dindo ≥ 3

MTL 30

Clavien-Dindo ≥ 3

Clavien-Dindo < 3

Total

Yes

1170

335

1505

No

1730

11,743

13,473

Total

2900

12,078

14,978

Specificity 97.2%; sensitivity 40.3%; accuracy 86.2%

Table 2

Contingency table for MTL22 versus Clavien-Dindo ≥ 3

MTL 22

Clavien-Dindo ≥ 3

Clavien-Dindo < 3

Total

Yes

1705

687

2392

No

1195

11391

12586

Total

2900

12,078

14,978

Specificity 94.3%; sensitivity 58.8%; accuracy 87.4%

Results

The time interval of 22 days demonstrated the highest accuracy (87.4%) regarding surgical major morbidity (Clavien-Dindo ≥ 3) compared to all other tested time intervals, i.e., 20, 23, or 30 days, as shown by Fig. 1. As a subgroup analysis revealed, the accuracy of the time interval of 22 days did not differ between colonic and rectal resections (colon—accuracy 87.8%; rectum—accuracy 86.7%). The differences between MTL30 and 22 concerning accuracy, specificity, and sensitivity are shown in Table 1 and 2. As demonstrated in this contingency table, MTL22 has a sensitivity of 58.8% (specificity 94.3%) regarding the depiction of severe surgical complications (Clavien-Dindo ≥ 3), whereas the sensitivity of MTL30 is only 40.3% (specificity: 97.2%). Hence, MTL22 performs better than MTL30 in depicting surgical major morbidity. Figure 2 illustrates that the odds ratios for almost all—especially surgical—complications are higher for MTL22 than for MTL30. Moreover, Fig. 2 and the corresponding data in the appendix show that MTL22 is not only better than MTL30, but above all better than all its single components. With MTL22, it is possible to map all complications (Appendix Table 4 (a–f)).
Fig. 1

Accuracy for MTLx regarding surgical major morbidity (Clavien-Dindo ≥ 3) depending on the time interval for length of hospital stay (x). The y-axis shows the accuracy, the x-axis the postoperative day

Fig. 2

af Full model fitted multiple regression analysis of the outcome measurements. If 95% confidence limits cross 1 in either direction, the result is not significant. Otherwise, findings are significant. SSI: surgical site infection; p/o: postoperative. p-values can be found in the appendix.

Data from 14,978 patients in 144 hospitals registered in the two colorectal cancer StuDoQ registers was analyzed. Baseline patient and procedure characteristics are presented in Table 3. Length of stay was significantly prolonged if postoperative complication(s) occurred (p < 0.0001): 35% of patients, who experienced at least one postoperative complication, were still in hospital on postoperative day (POD) 22 compared to only 4% of patients with an uneventful postoperative course (Fig. 3). Thirty days after the index operation, the percentage of inpatients among those with a complicated course was still as high as 24% compared to 1% in the group without complications. Among the investigated complications, postoperative myocardial infarction (p = 0.0001), postoperative ventilation (p = 0.0001), postoperative renal failure (p = 0.0001), postoperative pulmonary embolism (p = 0.0001), postoperative pneumonia (p = 0.0001), ileus (p = 0.03), and anastomotic leakage (p = 0.03) were significantly associated with 30-day mortality (Fig. 2). Likewise, the transfer to another hospital (other than rehabilitation clinic) mainly resulted from cardiovascular or pulmonary complications: postoperative myocardial infarction (p = 0.0001), postoperative pneumonia (p = 0.0001), postoperative stroke (p = 0.0001), and overall medical complications (p = 0.0003) were significantly associated with transferring a patient to another hospital. In contrast, there was no significant association between surgical morbidity caused by anastomotic dehiscence, ileus, bleeding, or surgical site infection and transfer to another hospital. Of 288 patients, who were transferred to another hospital, 107 patients (37%) had no complication documented before transfer. Prolonged length of stay was triggered by both surgical and medical morbidity, except for stroke, pulmonary embolism, and renal failure (still an in-patient at POD 22, i.e., MTL22 becomes positive), or stroke, renal failure, and myocardial infarction (still an in-patient at POD 30, i.e. MTL30 becomes positive). Both MTL30 and 22 were significantly associated with all investigated complications. These findings are shown in Fig. 2.
Table 3

Characteristics of 14,978 patients undergoing surgery for colorectal cancer registered in the StuDoQ|ColonCancer and StuDoQ|RectalCancer Registers between 2010 and 2017

 

Number

Percent

Age

Mean/range

70

19–100

Sex

  Female

6687

44.7

  Male

8291

55.4

Surgical procedures

  Anterior rectal resection

1098

7.3

  Low anterior rectal resection

3439

23.0

  Proctocolectomy

61

0.5

  Completion colectomy

3

 

  Multivisceral resection

269

1.8

  Colectomy

219

1.5

  Subtotal colectomy

11

0.1

  Recto-sigmoid resection

882

5.9

  Sigmoid resection

1597

10.7

  Left hemicolectomy

1176

7.9

  Extended left hemicolectomy

384

2.6

  Transverse colon resection

305

2.0

  Right hemicolectomy

4452

29.7

  Extended right hemicolectomy

1082

7.2

MTL 22

  No

12,586

84.0

  Yes

2392

16.0

MTL 30

  No

13,473

90.0

  Yes

1505

10.0

Postoperative hospital stay < 22 days

  No

13,047

87.1

  Yes

1931

12.9

postoperative hospital stay < 30 days

  No

14,028

93.7

  Yes

950

6.3

Transfer to another acute hospital within 30 days

  No

14,690

98.1

  Yes

288

1.9

Mortality within 30 days

  No

14,653

97.8

  Yes

325

2.2

Revisional surgery

  No

13,171

87.9

  Yes

1807

12.1

Clavien-Dindo grade 3 or more

  No

12,078

80.6

  Yes

2900

19.4

Nastomotic leakage

  No

13,751

13,751

  Yes

1227

1227

Abdominal wall dehiscence

  No

14,424

96.3

  Yes

554

3.7

Postoperative bleeding

  No

14,676

98.0

  Yes

302

2.0

Postoperative ileus

  No

14,478

96.7

  Yes

500

3.3

Surgical site infection requiring wound opening

  No

13,702

91.5

  Yes

1276

8.5

Postoperative internal fistula

  No

14,883

99.4

  Yes

95

0.6

Other surgical complications

  No

13,762

91.9

  Yes

1216

V

Unplanned postoperative ventilation > 48 h

  No

14,507

96.9

  Yes

471

3.1

Postoperative renal failure

  No

14,777

98.7

  Yes

201

3.1

Postoperative pulmonary embolism

  No

4892

99.4

  Yes

86

0.6

Postoperative pneumonia

  No

14,323

95.6

  Yes

655

4.4

Postoperative stroke

  No

14,930

99.7

  Yes

48

0.3

Postoperative myocardial infarction

  No

14,889

99.4

  Yes

89

0.6

Fig. 3

Influence of postoperative complications on length of stay—rates of hospitalized patients (p < .0001).

The two funnel plots (Fig. 4a, b) illustrate the hospital variation by hospital volume in percentage of patients with MTL. For MTL22, 10 hospitals (7%) were located above the upper 99.8% control limits and 17 hospitals (12%) lay above the upper 95% control limits (Fig. 4b) compared to 10 hospitals (7%) exceeding the upper 99.8% control limits and 14 hospitals (10%) the upper 95% control limits for MTL30 (Fig. 4a). Seven hospitals (5%) were located below the lower 99.8% control limits and 12 hospitals (8%) below the lower 95% control limits for MTL22 (Fig. 4b), whereas, regarding MTL30, 2 hospitals (1%) lay below the lower 99.8% control limits (Fig. 4a) and 10 hospitals (7%) below the lower 95% control limits.
Fig. 4

a, b (funnel plots): hospital variation in percentage of patients with MTL 30 and 22 scenarios. Dotted lines represent the upper/lower 99.8% control limits, dashed lines the upper/lower 95% control limits. Single crosses represent individual hospitals.

Discussion

The present study investigated MTL, a composite outcome measure for profiling hospitals on surgical performance, using data from 14,978 patients from the colorectal StuDoQ registers of the DGAV. Length of stay was significantly prolonged if postoperative complications occurred. Thirty-day mortality as well as the indication to transfer a patient to another hospital (other than rehabilitation clinic) mainly resulted from cardiovascular and/or pulmonary complications. According to our analysis, MTL occurs significantly more often than any of its components or any of the other established single-outcome parameters. The time interval of 22 days demonstrated the highest accuracy (87%) regarding surgical major morbidity (Clavien-Dindo ≥ 3) compared to all other tested time intervals, i.e., 20, 23, or 30 days. Moreover, MTL22 has a sensitivity of 65% (specificity 92%) regarding the depiction of severe surgical complications (Clavien-Dindo ≥ 3), whereas the sensitivity of MTL30 is only 40% (specificity: 97%), which is, of course, insufficient. Therefore, MTL22 performs better than MTL30 or any other time interval covered by MTL in depicting surgical major morbidity.

However, the results of this study should be viewed in the context of certain limitations. The present analysis was not adjusted for differences in case-mix because—as mentioned in the introduction—in the first step, we aimed at regarding the marker MTL in isolation. As Wiegering et al. recently showed using data from the StuDoQ|ColonCancer Register, MTL30 correlated with the UICC (Union Internationale Contre le Cancer) stage. Accordingly, MTL should probably be implemented as a risk-adjusted QI in order to minimize confounding of the surgical quality measurement by patient- or tumor-related factors [7].

Since the investigated sample consisted exclusively of hospitals submitting their data to the StuDoQ registers, and most of them with the intention either to achieve the certification as a colorectal cancer center or at least benchmark its own performance, this sample is potentially not representative of the average German hospital, which may limit the external validity of the results presented here.

Even though the evidence regarding MTL as a composite measure is scarce and the published evidence only consists of one recent analysis of four organ-specific StuDoQ registers and the data presented here, there is broad evidence of the superiority of composite outcome measures over single parameters regarding a higher discriminatory power and the suitability to reliably identify outliers in the context of quality measurement. Gooiker et al. investigated the internal consistency and construct validity of nine single quality indicators (QIs) for colorectal cancer surgery using data from 85 Dutch hospitals participating in the Dutch Surgical Colorectal Audit in 2010 and found that single QI indeed provide complementary information. However, due to the lack of inter-QI correlation (indicating insufficient internal coherence), individual QI are not suitable as a surrogate for the quality of surgical colorectal cancer care. Therefore, the authors advocate more complex QI or composite outcome measures in order to meet the requirements regarding internal consistency and construct validity [2].

Dimick et al. developed composite outcome measures for four surgical procedures—colectomy, ventral hernia repair, abdominal aortic aneurysm repair, and lower extremity bypass surgery—by combining the established QI morbidity, reoperation, and length of stay and compared the standard American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP) approach for assessing hospital rates of risk-adjusted morbidity with their new composite approach using data from patients undergoing the four studied procedures registered in the ACS-NSQIP between 2008 and 2009. For all four procedures, the composite measure explained a higher proportion of systematic hospital-level variation and performed better at predicting future hospital performance [1].

In summary, composite outcome measures improve the reliability of benchmarking and decrease the misclassification of hospitals by their higher discriminatory power. Hence, composite measures are also advancing in colorectal surgery [3, 4].

A specific advantage of MTL is the combination of the two single components “transfer to another hospital” (T) and “length of stay” (L). MTL is deemed as fulfilled if either the T- or the L-criterion occurs. Large hospitals usually treat postoperative high-grade morbidity in-house, whereas smaller hospitals often transfer patients with postoperative major complications to another hospital with more expertise and infrastructure to manage complicated cases. MTL guarantees more fairness because in both settings—no matter whether the patient is transferred or has a prolonged stay in the primary hospital—MTL becomes “positive” which indicates a complicated course. Transfers to other hospitals due to overcrowding or a lack of resources in the primarily treating hospital do not occur in the German health care system.

Moreover, MTL reflects the complete spectrum of postoperative complications, whereas its single components only depict partial aspects. MTL as well as its single components were mainly triggered by medical complications, such as myocardial infarction and pulmonary embolism. Therefore, it seems rational to continue to assess the “traditional” surgical quality measures, such as anastomotic leakage, in addition to MTL. However, medical complications can quite possibly provide an indication of the surgical quality and especially reflect the indication quality.

Low hospital caseloads and low event rates usually limit the precision of single outcome measures, such as mortality, and result in little hospital variation. MTL as a composite outcome measure of three more or less “sentinel-event” indicators becomes significantly more frequently positive than its single components. As a result, compared to individual outcome parameters, MTL has a better discriminatory power and is suitable to reliably identify outliers and mirror surgical outcome quality even if hospital caseloads and event rates of the classical single outcome measures are low. As visualized in the funnel plots (Fig. 4a, b), MTL even detects hospitals which may have a quality problem if caseloads are well below 100.

Another decisive advantage of MTL, e.g., compared to postoperative complications classified according to the Clavien-Dindo classification, is the fact that MTL does not require separate documentation, but can be derived from routine administrative data. Since the validity of data recorded in clinical databases is often low, MTL must be more valid already in principle than outcome measures depending on additional clinical data collection. According to a study by Dindo et al., clinical databases are managed by residents alone in the majority of European centers which results in a high risk of underreporting of surgical complications as residents fail to document up to 80% of complications, including such severe ones as death [9].

The data presented here speak for the use of MTL22 as an ideal outcome measure for the quality of colorectal cancer surgery within the German health care system. However, the time frame covered by MTL should be adjusted to fit the specific surgical and sociocultural setting. This means that the number after “MTL” may vary according to procedure and health care system. MTL may as well be used in surgical areas other than colorectal cancer surgery, such as hepatic, pancreatic, or bariatric surgery. The present study suggests a methodology of how to calculate the optimal value for a given procedure and health environment in order to achieve best representation of various complications within this single measure.

Although the generalisability of the results presented here may be limited by the fact that this sample of hospitals submitting their data to the StuDoQ registers is potentially not representative of the average German hospital, the advantages of MTL compared to individual surgical outcome measures should hold true for other populations and procedures: its derivation from routine administrative data, its higher discriminatory power, and its suitability to reliably identify outliers and mirror surgical outcome quality. The next step will be the evaluation of the validity of MTL to assess whether MTL measures what it is intended to measure: the quality of surgical care..

Notes

Acknowledgments

This work has been conducted using the StuDoQ|ColonCancer and the StuDoQ|RectalCancer Registers provided by the Study, Documentation and Quality Center (Studien-, Dokumentations- und Qualitätszentrum, StuDoQ) of the German Society of General and Visceral Surgery (Deutsche Gesellschaft für Allgemein- und Viszeralchirurgie, DGAV) with the ID StuDoQ-2016-0001.

Compliance with ethical standards

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Formal consent is not required for this type of study.

Disclosure

The authors declare that they have no conflict of interest.

Submission category

Original article.

References

  1. 1.
    Dimick JB, Staiger DO, Hall BL, Ko CY, Birkmeyer JD (2013) Composite measures for profiling hospitals on surgical morbidity. Ann Surg 257(1):67–72.  https://doi.org/10.1097/SLA.0b013e31827b6be6 CrossRefGoogle Scholar
  2. 2.
    Gooiker GA, Kolfschoten NE, Bastiaannet E, van de Velde CJ, Eddes EH, van der Harst E, Wiggers T, Rosendaal FR, Tollenaar RA, Wouters MW, Dutch Surgical Colorectal Audit g (2013) Evaluating the validity of quality indicators for colorectal cancer care. J Surg Oncol 108(7):465–471.  https://doi.org/10.1002/jso.23420 CrossRefGoogle Scholar
  3. 3.
    Kolfschoten NE, Kievit J, Gooiker GA, van Leersum NJ, Snijders HS, Eddes EH, Tollenaar RA, Wouters MW, Marang-van de Mheen PJ (2013) Focusing on desired outcomes of care after colon cancer resections; hospital variations in ‘textbook outcome’. Eur J Surg Oncol 39(2):156–163.  https://doi.org/10.1016/j.ejso.2012.10.007 CrossRefGoogle Scholar
  4. 4.
    Merkow RP, Hall BL, Cohen ME, Wang X, Adams JL, Chow WB, Lawson EH, Bilimoria KY, Richards K, Ko CY (2013) Validity and feasibility of the american college of surgeons colectomy composite outcome quality measure. Ann Surg 257(3):483–489.  https://doi.org/10.1097/SLA.0b013e318273bf17 CrossRefGoogle Scholar
  5. 5.
    Staiger DO, Dimick JB, Baser O, Fan Z, Birkmeyer JD (2009) Empirically derived composite measures of surgical performance. Med Care 47(2):226–233.  https://doi.org/10.1097/MLR.0b013e3181847574 CrossRefGoogle Scholar
  6. 6.
    Karthaus EG, Lijftogt N, Busweiler LAD, Elsman BHP, Wouters M, Vahl AC, Hamming JF, Dutch Society of Vascular Surgery tSCotDSAAtDIfCA (2017) Textbook outcome: a composite measure for quality of elective aneurysm surgery. Ann Surg 266(5):898–904.  https://doi.org/10.1097/SLA.0000000000002388 CrossRefGoogle Scholar
  7. 7.
    Wiegering A, Wellner U, Seyfried F, Hardt J, Klinger C, Buhr H, Post S (2017) MTL30 as surrogate parameter for quality of surgically treated diseases: establishment based on the StuDoQ register of the German Society for General and Visceral Surgery. Chirurgica.  https://doi.org/10.1007/s00104-017-0479-z
  8. 8.
    Spiegelhalter DJ (2005) Funnel plots for comparing institutional performance. Stat Med 24(8):1185–1202.  https://doi.org/10.1002/sim.1970 CrossRefGoogle Scholar
  9. 9.
    Dindo D, Hahnloser D, Clavien PA (2010) Quality assessment in surgery: riding a lame horse. Ann Surg 251(4):766–771.  https://doi.org/10.1097/SLA.0b013e3181d0d211 CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Department of SurgeryUniversitätsmedizin Mannheim, Medical Faculty Mannheim, University HeidelbergMannheimGermany
  2. 2.German Society of General and Visceral Surgery (DGAV)BerlinGermany

Personalised recommendations