Introduction

The Surviving Sepsis Campaign (SSC) was established in 2002 to assess the public’s and clinicians’ awareness of sepsis, to develop evidence-based guidelines for the management of severe sepsis and septic shock, and to change clinical behavior through the implementation of those guidelines [1–5]. Since then many hospitals, regulators, public health departments, and lay organizations have developed similar initiatives [6, 7] to improve outcomes in patients with severe sepsis and septic shock by facilitating their identification and evidence-based management; many have highlighted an improvement in outcome with reduced mortality [7–10]. At the same time, however, the incidence of severe sepsis and septic shock has increased, which raises the question whether the improvements observed have been due to an overall decline in mortality rather than an improved compliance with sepsis performance metrics [11].

In our previously published results in 15,022 patients, no difference was shown in baseline mortality in hospitals that entered throughout the 2-year study period; i.e., in the SSC study hospitals, the mortality rate was not changing over time [12]. Since that report, the continued participation of some hospitals for 4 years, the introduction of the initiative in newly participating hospitals throughout the 7.5-year study period, and the variation in compliance among participating hospitals enabled the analysis of the longitudinal impact of the performance improvement project and the association between sepsis bundle compliance and mortality.

Methods

Sites and patient selection

The process of participation in the SSC is described elsewhere [12]. Eligible subjects were those having a suspected site of infection, two or more systemic inflammatory response syndrome criteria [13], and one or more organ dysfunction criteria [12]. Clinical characteristics and time of presentation with severe sepsis were collected for longitudinal analysis. Time of presentation was determined through chart review for the diagnosis of severe sepsis and defined in instructions to data collectors on the SSC website and in educational materials. For patients from the emergency department (ED), the time of presentation was defined as the time of triage.

Data collection

Data were entered into the SSC database locally at individual hospitals into pre-established, unmodifiable fields. Data stripped of private health information were submitted every 30 days to the secure master server at the Society of Critical Care Medicine (Mount Prospect, IL).

Institutional review board approval

The global SSC improvement initiative was approved by the Cooper University Hospital Institutional Review Board (IRB) [Camden, NJ] as meeting criteria for exempt status. The US Department of Health and Human Services’ Office for Human Research Protections reiterated that quality improvement activities such as the SSC often qualify for IRB exemption and do not require individual informed consent [14]. Because the Cooper IRB approval would not cover sites outside the United States, those sites were asked, by letter from the SSC, to seek local IRB approval according to local rules and regulations.

Analysis set construction

The study objective was to characterize the results of the first 4 years of participation in the SSC; thus, the analysis includes a maximum of 16 quarters, which occurred over a 7.5 calendar year period. The first 3 months during which a site entered subjects into the database were defined as the first quarter regardless of when those months occurred. Inclusion in the SSC database was limited to sites with at least 20 subjects and at least 3 months of subject enrollment.

Compliance with the SSC sepsis bundles [12] and associated mortality rates were the primary outcome variables in this study. Two additional working variables were created to help characterize the results. The first variable characterized a site as being in either low or high compliance for both the resuscitation and management bundles. Compliance was defined as evidence that all bundle elements were achieved within the indicated time frame (i.e., 6 h for the resuscitation bundle; 24 h for the management bundle). For each site, the proportion of patients in compliance with the resuscitation bundle and the proportion of patients in compliance with the management bundle during the last 2 quarters of participation were generated. Site resuscitation compliance was categorized as low if <15 % (the median of all sites) or high if ≥15 %. Similarly, site management compliance was categorized as low if <20 % (the median of all sites) or high if ≥20 %. This variable was used to descriptively compare sites across compliance status. When running regression analyses, we used either the continuous proportion values at the site level or the dichotomous compliance values at the patient level. The second working variable was site duration, which categorizes years of participation in the SSC as less than 2 years, 2 to <3 years, and 3 or more years. This 3-level variable was used to describe the cohort of patients or sites, while continuous duration in quarters was used in the regression analysis. The duration variable was constant for a site, whereas the site quarter of participation variable described the quarter a particular patient entered the study relative to the first quarter of that site’s participation in the SSC.

The primary outcome in this study is hospital mortality and the sepsis severity score (SSS) [15] is used as an adjustment variable to balance patient clinical characteristics across groups not used in the development of the score.

Statistical analysis

Site-level characteristics are presented as frequencies and percentages and are compared across bundle compliance (high vs. low) using Fisher exact test. Patient-level characteristics are also presented as frequencies and percentages and are compared across compliance using Pearson Chi-square test. Continuous patient-level characteristics are presented as medians along with the interquartile range and are compared across compliance using Wilcoxon rank sum test. The odds of hospital mortality or the odds of individual bundle compliance were estimated using a generalized estimating equation (GEE) population-averaged logistic regression model. The study’s goal was not to predict hospital mortality, but to identify the role of site quarter on hospital mortality; therefore, we used a risk factor modeling approach to determine which covariates to add to the GEE model. Only covariates that acted either as a confounder or as an effect modifier were included. A confounder was identified when its addition to the model changed the odds ratio (OR) associated with the site quarter by more than 10 % in either direction, without considering statistical significance. A covariate that had a statistically significant interaction (p ≤ 0.05) with site quarter was considered to be an effect modifier. This same approach was used when site duration and patient-level compliance (yes vs. no) was the risk factor or when site-level proportion compliance was the risk factor. The hierarchical nature of the SSC data lends itself to GEE population-averaged logistic regression analysis as patients are nested within a particular site. ICU and hospital lengths of stay were natural log transformed to normalize the distribution and stabilize the variance across low and high resuscitation or management compliance and years of SSC participation. Random-effects linear regression was used to estimate the change in length of stay for a 10 % increase in the proportion compliant with the resuscitation bundle [16]. Ratios of the geometric means were produced when the model results were back transformed to the original units of days of ICU or hospital stay. All analyses were run using Stata 12.1 (Stata Corporation, College Station, TX).

Results

A total of 29,470 subjects were included in the analysis set that was constructed from subjects entered into the SSC database from January 1, 2005, through June 30, 2012.

Site characteristics

The majority of sites were from the United States (49.1 %) as opposed to Europe (36.2 %) and South America (14.7 %) (Table 1). High compliance was seen in 102 (46.8 %) sites for the resuscitation bundle and 103 (47.2 %) sites for the management bundle. For both bundles, the highest compliance was in the United States. Distribution of site duration percentages across compliance was similar for both bundles. Sites with fewer than 2 years’ duration had low compliance for the resuscitation (67.2 %) and management (64.3 %) bundles, while the majority of high-compliance sites for the bundles had 2 or more years’ duration (p < 0.001 and p = 0.010, respectively).

Table 1 Site characteristics across low- and high-compliance sites for resuscitation and management bundles
Full size table

Patient characteristics

Table 2 illustrates that most subjects were diagnosed in the ED (55.8 %) and the majority (68.2 %) were from the United States. There was some tendency for subjects whose sepsis was identified in places other than the ED to be in the resuscitation bundle low-compliance sites compared to high-compliance sites (p < 0.001). A greater percentage of the patients from the United States and Europe were in high-compliance sites (13,538 of 20,086 patients [67.4 %] and 3,477 of 6,609 patients [52.6 %], respectively) compared to low-compliance sites (6,548 of 20,086 patients [32.6 %] and 3,132 of 6,609 patients [47.4 %], respectively), while a larger percentage of the South American patients were in low-compliance sites (1,925 of 2,775 patients [69.5 %]) compared to high-compliance sites (846 of 2,775 patients [30.5 %]; overall p < 0.001). Hospital length of stay, ICU length of stay, and length of stay prior to ICU admission were all longer in low- compared with high-performing resuscitation sites (all p < 0.001).

Table 2 Patient clinical characteristics across low- and high-compliance sites for resuscitation bundle
Full size table

The clinical characteristics across management compliance can be found in Supplementary Table 1. There was some tendency for subjects whose sepsis was identified in places other than the ED to be in the management bundle high-compliance compared to low-compliance sites, whereas a higher percentage of those identified in the ED were in low-compliance sites (p < 0.001). A greater percentage of the patients from Europe and South America were in management bundle high-compliance (3,766 of 6,609 [57.0 %] and 1,992 of 2,775 [71.8 %], respectively) compared to low-compliance sites (2,843 of 6,609 [43.0 %] and 783 of 2,775 [28.2 %], respectively), while a larger percentage of US patients were in low-compliance (10,187 of 20,086 [50.7 %]) compared to high-compliance sites (9,899 of 20,086 [49.3 %]; overall p < 0.001). Adjusted results indicate that ICU and hospital lengths of stay are not significantly associated with management bundle compliance.

Association between bundle compliance and hospital mortality

Overall mortality was lower in resuscitation bundle high-compliance sites (29.0 %) compared to low-compliance sites (38.6 %; p < 0.001); mortality differed less dramatically in management bundle high-compliance sites (32.3 %) and low-compliance sites (33.8 %; p = 0.038) (Table 3). Hospital mortality rate drops 0.7 % per site quarter of participation (p < 0.001) while both resuscitation and management bundle compliance increases for the first 2 years of participation, then plateaus for years 2–4 (Fig. 1). However, the proportion compliant with the resuscitation bundle increases with site quarter of participation, and this increase continues with longer participation in the SSC (<2, 2–3, and 3–4 years) (Fig. 2; Supplementary table 2).

Table 3 Hospital mortality across low- and high-compliance sites for resuscitation management bundles
Full size table
Fig. 1
figure 1

Resuscitation (a) and management (b) Surviving Sepsis Campaign (SSC) bundle compliance and hospital mortality (c). All panels are based on an unadjusted generalized estimating equation population-averaged logistic regression model where circles represent observed values, lines are based on the regression, and shaded areas are the 95 % CIs

Full size image
Fig. 2
figure 2

Resuscitation compliance by duration of years of Surviving Sepsis Campaign (SSC) participation (<2 years, a 2 to <3 years, b 3–4 years c) and hospital mortality (panel d). All panels are based on an unadjusted generalized estimating equation population-averaged logistic regression model where circles represent observed values, lines are based on the regression, and shaded areas are the 95 % CIs

Full size image

Table 4 presents the results of two GEE population-averaged logistic regression models. The first uses continuous compliance, either resuscitation or management bundle, as a site-level variable and is measured in the last two quarters of a site’s SSC participation. The second uses compliance as a patient-level variable and measures whether or not a patient’s ICU visit was compliant with all of the resuscitation or all the management bundle. The first model indicates that hospital mortality decreases 4 % (OR 0.96, 95 % CI 0.95–0.97; p < 0.001) for every additional quarter of SSC participation. Hospital mortality drops 5 % (OR 0.95, 95 % CI 0.94–0.97; p < 0.001) for a 10 % increase in the proportion compliant with the resuscitation bundle while the drop is 3 % (OR 0.97, 95 % CI 0.96–0.98; p < 0.001) for a 10 % increase in the proportion compliant with the management bundle. In the second model, hospital mortality decreases 3 % (OR 0.97, 95 % CI 0.96–0.98; p < 0.001) for every additional quarter of SSC participation. Hospital mortality drops 18 % (OR 0.82, 95 % CI 0.76–0.88; p < 0.001) for patient-level compliance with all elements of the resuscitation bundle, while the drop is 14 % (OR 0.76, 95 % CI 0.71–0.81; p < 0.001) if patient-level compliance was achieved for all elements of the management bundle. Both models are adjusted for the SSS, duration of participation in the SSC, and calendar time measured in quarters. Additionally we tested whether the SSS changed over calendar time using a random-effects linear regression model. The slope coefficient in this model (β = −0.05, p = 0.609) indicates that the SSS was stable over the period of the study.

Table 4 Odds of hospital mortality for site quarter of participation, resuscitation bundle compliance, and management bundle compliance for two logistic regression models
Full size table

Adjusted results indicate that the ICU length of stay decreases 4 % (95 % CI 1–7 %; p = 0.012) for a 10 % increase in the proportion compliant with the resuscitation bundle. Hospital length of stay also decreases 4 % (95 % CI 1–7 %; p = 0.031) for a 10 % increase in the proportion compliant with the resuscitation bundle. These results are adjusted for the SSS, duration in the SSC, survival status (died in the ICU; survived the ICU, but died in the hospital; and survived to hospital discharge), and calendar time measured in quarters.

We wanted to be sure that any decrease in hospital mortality was associated with the SSC and not with a global decrease in mortality in patients with the same severity of illness, because our results were adjusted by the SSS. Thus, using a GEE population-average logistic regression model and only a site’s first quarter of participation in SSC (n = 2,592), we regressed mortality on calendar time in quarters, while adjusting for the SSS. The adjusted OR is 0.999 (95 % CI 0.973–1.021; p = 0.805), indicating hospital mortality was stable over the duration of participation. Therefore, initial mortality rates in all sites entering participation over the study period was not changing over time. Note that these results were not confounded by any other SSC covariates nor were there any SSC effect modifiers of this relationship.

Table 5 presents the odds of hospital mortality for the individual elements of the resuscitation and management bundles. For some of the elements, the interaction with duration in the SSC was significant, and thus these cases have three odds ratios for these elements. Overall each of the elements is protective of hospital mortality as indicated by the odds ratio <1. These results are adjusted for the site quarter of participation, SSS, and duration in SSC participation.

Table 5 Hospital mortality adjusted odds ratio modeled individually for each element in bundle compliance using a generalized estimating equation population-averaged logistic regression
Full size table

Discussion

This analysis, over a period of 7.5 years, demonstrates that increased compliance with a global sepsis quality improvement initiative was associated with a 25 % relative risk reduction in mortality. The intervention studied was the introduction of a multifaceted, collaborative change intervention aimed at facilitating adoption of the SSC resuscitation and management bundles. Bundle adoption included identification of physician and nurse champions, introduction of the bundles, education, as well as audit and feedback [12]. Several important messages emerge as a result of this study; these can be viewed as participation, duration, and dose effects. Participation effect is seen in the overall decline in mortality associated with participation in the SSC initiative irrespective of the rate of compliance, suggesting that participation in SSC alone is associated with improved patient outcome. Duration effect is seen in the 7 % decline in the risk of mortality for every additional quarter a site participates in the SSC; therefore, the longer a site participated in the SSC initiative, the better the associated patient outcomes (mortality). Dose effect can be seen in the results of compliance as a continuous variable: for every 10 % increase in compliance at a given site, the risk of hospital mortality decreases 3–5 %; therefore, the more compliant a hospital, the better the associated outcome. In addition, in hospitals identified as high-compliance sites, the associated mortality reduction was significantly greater than in the overall patient population: in high-compliance hospitals that participated at least 3 years, a 36 % relative reduction in mortality was achieved, while hospitals that participated at least 4 years had an associated mortality reduction of 39 %. This association is a major finding in the efforts of the SSC to change clinical behavior and improve outcomes. A recent report suggested that sepsis mortality has been decreasing over the past decade [11], and a study from Kaukonen et al. [17] reported a decline in sepsis mortality from 35.0 to 18.4 % in Australia and New Zealand between 2000 and 2012. Combined with the fact that sepsis mortality was not decreasing over time in the hospitals entering the SSC database throughout the 7.5-year study period, these results make a compelling argument that the beneficial impact of high compliance with the sepsis bundles is not solely due to the reported overall decline in mortality but is related to the change in clinical practice [11]. This has significant implications for the mandated sepsis reporting efforts now underway in some states [18].

This is the largest prospective cohort of sepsis patients ever reported. Since the original report from the SSC [12], similar studies have been published from individual hospital networks [19–21] and national programs [22]. All published studies have demonstrated an association between improved compliance with guidelines-based sepsis bundles and survival. The results of this study of nearly 30,000 patients show, as in the previous study, that increased compliance with the sepsis performance bundles was associated with decreased mortality, reinforcing the strength of the consistently reported favorable association between compliance and mortality. Also important is the statistically significant decline in mortality rate associated with continued participation in the initiative. For every additional quarter of participation, an associated decrease of 7 % in hospital mortality from sepsis occurred. Given the widespread adoption of sepsis performance improvement initiatives, this is an important message for hospitals and bedside practitioners: the survival benefit associated with performance improvement increases over time. Another important finding is the important beneficial impact on resource utilization: ICU and hospital lengths of stay were significantly lower in high-compliance versus low-compliance hospitals.

This study also points to the difference between observational studies and randomized, controlled trials. In our study, the regression analysis (Table 5) revealed that the bundle elements of central venous pressure (CVP) measurement and central venous oxygen saturation (Scvo 2) measurement were both independently associated with a decreased odds ratio of mortality. However, the recently published ProCESS study failed to demonstrate any difference in mortality between early goal-directed therapy (CVP and Scvo 2) or the value of protocolized resuscitation compared with “usual” care [23]. Our study, which is less methodologically robust than a randomized, controlled trial, included these elements in the bundles, along with multiple other elements of care. It is difficult to say which of the elements were most essential to the associated mortality benefit seen with increasing overall bundle compliance.

Our study has several limitations. It was not a randomized, controlled trial, so no direct causation can be claimed for the effect of compliance on mortality. The statistically significant association between duration of participation and higher compliance with decreasing hospital mortality suggests, however, that compliance with performance metrics is at least, in part, responsible for the improved survival. Another limitation is the lack of baseline mortality data. The “baseline” mortality for all sites is mortality after the first quarter of participation in the SSC. The higher-compliance sites demonstrated lower mortality then the low-compliance sites. Rapid declines in associated mortality early in performance improvement initiatives have been reported elsewhere [7, 12]; thus we cannot definitely say if the difference in mortality in low- and high-compliance sites after 3 months in the SSC is due to baseline differences or to highly compliant sites lowering their mortality rates from the beginning. Another limitation is the potential confounding by other hospital quality improvement initiatives on the declining mortality rate seen during participation in the SSC. Other factors, in addition to the SSC sepsis initiative, may have contributed to the decline in hospital mortality. No data were collected on the specific improvement initiatives simultaneously occurring at all participating institutions. However, results of the analysis with compliance as a continuous variable suggest that there was a significant, positive relationship between duration of participation, increasing compliance, and hospital mortality at each site. The methodology of the performance improvement initiative as a continuous time series does not permit a conclusive determination of causality between bundle compliance and reduced mortality.

Another weakness is the lack of rigorous quality control in data entry. The study spanned three continents and more than 200 hospitals. Data collectors were trained through multiple conference calls and onsite meetings, but no rigorous quality checking of data entry was performed. A further weakness rests in the identification of “time zero.” Participants were instructed to establish time zero in the ED at triage time. For the wards and the ICUs, triage time was established through chart review, and the onset of severe sepsis with identified by vital signs and laboratory values. This time zero is less rigorous and therefore less reliable than the time zero established for ED patients. Even at the end of this study, in sites with 3- and 4-years’ worth of data collection, metrics for compliance with such bundle elements as antibiotics, CVP, and Scvo 2 were still <75 % and, in the case of CVP and Scvo 2, <40 %. This may, in part, reflect the use of triage time as time zero because a certain percentage of patients developed severe sepsis and septic shock several hours after arriving in the ED. Of interest, a recent multicenter trial has called into question the value of CVP and Scvo 2 as the sole therapeutic targets in resuscitation of severe sepsis and septic shock [23]. This may certainly impact recommendations for resuscitation and future bundles as additional published data become available.

Conclusion

The results of the 7.5-year study of the SSC performance improvement initiative demonstrated a greater increase in compliance over time in hospitals that continue to participate 2, 3, and up to 4 years. Increasing compliance is associated with a statistically significant decline in mortality rates, and hospital mortality is even lower in hospitals with high versus low bundle compliance. Consistent with the literature, these results lend strength to the argument that performance metrics can be used to drive change in clinical behavior and, therefore, improve the quality of care and lead to decreased mortality in patients with severe sepsis and septic shock.