Introduction

The development of agents aimed at reducing mortality from severe sepsis has been predicated on the hypothesis that death results from sepsis-induced organ dysfunction, the latter being the consequence of an excessive or uncontrolled host response to the infection [1–3]. Fundamental to this hypothesis is the assumption that the host response, at least to some extent, is no longer beneficial once organ dysfunction ensues and that modulation of this response will reduce the severity of organ dysfunction or prevent additional dysfunctions [4]. Therefore, current trial designs allow the enrollment of a heterogeneous population of patients with varying numbers of organ dysfunctions, severity of illness scores, and predicted risk for death [5].

Recent publications [6–8] have challenged this hypothesis, suggesting that the host response may only be detrimental in patients with the most severe degrees of organ dysfunction and highest risk for death. As a potential result, biologic response modifiers, specifically those with anti-inflammatory effects, may only be beneficial in the most severely ill patients and could potentially be ineffective or detrimental in patients with severe sepsis and less severe organ dysfunctions [7]. The idea that biologic response modifiers might exhibit qualitative treatment effects in severe sepsis (i.e. produce beneficial effects in the most severely ill and detrimental effects in the least severely ill) is based primarily on preclinical animal studies and on post hoc analyses of successful and failed clinical trials in patients with severe sepsis [7]. However, a recent meta-analysis of steroid treatment in patients with sepsis and septic shock [9] failed to identify a relationship between increasing treatment benefit associated with steroid therapy and increasing control group mortality.

We therefore undertook a systematic review of all published phase III, randomized, controlled clinical trials in adult patients with severe sepsis or septic shock to determine whether there were data supporting the hypothesis that biologic modifiers might be associated with qualitative treatment effects dependent on disease severity (as assessed by control mortality rates). Understanding whether data from prior clinical trials suggest that these agents might produce differential effects on survival depending on a patient's severity of illness is important in designing future trials of newer agents in severe sepsis. We report the lack of any such data and discuss the advantages and disadvantages of current trial designs in severe sepsis.

Materials and methods

Publications of randomized, placebo-controlled phase III or phase III equivalent studies that tested the effects of specific pharmaceutical interventions aimed at improving survival from severe sepsis were identified by a search of the PubMed database. The following search terms were used, each with restrictions for human studies and randomized controlled trials: sepsis and mortality, and severe sepsis and mortality. An additional check of the PubMed database was conducted using the search terms sepsis or severe sepsis, with restrictions for human studies and meta-analysis. Reference lists from these latter publications were cross-checked against the original search results to identify any additional reports. The PubMed database was searched multiple times throughout the preparation of this manuscript. The final search was conducted on 29 November 2004.

Studies were included in this analysis if they met the following criteria: randomized, double blind, placebo controlled clinical trial; enrollment of adult patients who met the diagnosis of severe sepsis or septic shock; assessment of 28- to 30-day all-cause mortality as the primary outcome; and adequate power (≥ 80%) to detect statistically significant improvements in the primary outcome at the two-sided alpha of 0.05. Studies that compared more than one active therapy arm with placebo were required to include an intent to adjust statistically for two or more comparisons (e.g. Bonferroni procedure) [10]. Likewise, appropriate correction for repeated comparisons at planned interim analyses (e.g. O'Brien–Flemming) was also required to have been prospectively defined if there was a possibility of stopping the study early because of efficacy [10]. The inclusion of these statistical requirements was to ensure appropriate rigor in the conduct of the study. Phase III or phase III equivalent studies were considered large enough to allow statistical interpretation of the overall population and, more importantly, of reported subgroups.

Severe sepsis was defined in all studies as follows: the presence of known or suspected infection; evidence of a systemic response to infection (e.g. fever, hypothermia, tachypnea, tachycardia, leukocytosis or leukopenia); and one or more organ dysfunctions resulting directly from the systemic response to infection (most commonly cardiovascular, respiratory, renal, hematologic or metabolic acidosis). Septic shock was defined as the presence of either hypotension (absolute or relative) or the need for vasopressor support to maintain adequate perfusion and evidence of end-organ hypoperfusion.

The primary end-point of 28-day all-cause mortality was extracted from all studies with no adjustment for imbalance in baseline characteristics between patient treatment groups. Quantitative assessments of outcome at 28 days for subgroups defined by baseline measures of disease severity were also extracted. These subpopulations included groups defined by Simplified Acute Physiology Score [11], Acute Physiology and Chronic Health Evaluation (APACHE) II [12], presence or absence of shock, presence or absence of hypotension, presence or absence of acute respiratory distress syndrome, IL-6 concentration, cardiovascular Sepsis-related Organ Failure Assessment score [13], and presence of single or multiple organ failures. Qualitative assessment of any interaction beween treatment and disease severity was extracted from the results or discussion section of the report.

Data pertaining to the safety of the intervention was also extracted. In particular, the incidence of any post-treatment infectious complications was specifically sought.

Statistical methods

Mortality rates were extracted from publications. Some reports included the total number of patients within severity classes but did not include per treatment sample sizes within severity groups. In these instances, calculations of placebo and treatment sample sizes per groups assumed that patients were evenly divided between treatment groups. The information extracted was used in a logistic regression to determine whether there was a significant interaction between treatment and severity after adjusting for overall treatment and severity effects. One severity classification was selected per study. If multiple severity classes were reported, priority was attributed in the following order: predicted risk for death; APACHE II; shock versus no shock; and remaining available severity measure. Analyses were performed using SAS version 8.02 software (SAS Institute Inc, Cary, NC, USA).

Results

Using the restrictions listed above, 535 and 158 publications were identified for sepsis + mortality and severe sepsis + mortality, respectively. These publications were grouped as potential phase III studies of biologic response modifiers in severe sepsis (n = 43), non-phase III studies of biologic response modifiers in severe sepsis (n = 158), antibiotic studies in severe sepsis (n = 76), nonantibiotic, nonbiologic response modifier studies in severe sepsis (n = 41), and unrelated studies (n = 335). A total of 110 unique reports were identified using the search terms sepsis or severe sepsis and restricted to meta-analyses of human studies, of which nine were specific to severe sepsis. From the initial publication list and review of the references from identified meta-analyses, 22 reports, investigating 17 molecular entities, fulfilled criteria for phase III or equivalent studies aimed at reducing mortality in adult patients with severe sepsis or septic shock (Table 1). A number of additional studies were identified but were not included because they were not considered phase III studies (for example [14–18]), because they lacked statistical adjustment for multiple comparisons (e.g. [19, 20]), or because the 28- to 30-day mortality data were not provided (e.g. [21–23]). Supplemental publications from some studies were reviewed to extract subgroup mortality data [6, 24]. Studies were conducted between January 1987 and July 2003 (Table 1). Table 2 lists the overall and subgroup results for all identified studies.

Table 1 Characteristics of included randomized placebo-controlled clinical studies
Full size table
Table 2 28-Day all-cause mortality by study and by selected subgroups
Full size table

Three studies met the prospectively defined primary end-point of a statistically significant reduction in 28-day all-cause mortality, namely those by Ziegler and coworkers in 1991 [25], Bernard and colleagues in 2001 [26] and Annane and coworkers in 2002 [27]. The control group mortality rates for these three studies were 43%, 31% and 61%, respectively, indicating that the beneficial effects of adjunct therapies could be demonstrated over a wide range of illness severity. Figure 1 shows the results of all trials that failed to meet their primary end-point as prospectively specified in the methods section of each report. The distribution of outcome results for placebo and active treatment groups reside along the line of unity over a placebo mortality range between 20% and 60%. These data do not suggest that a possible explanation for the lack of demonstrated efficacy in these studies resulted from either enrollment of less severe or more severely ill patients (as assessed by the observed placebo mortality rates).

Figure 1
figure 1

Distribution of treatment and placebo mortalities for unsuccessful sepsis trials.

Full size image

Figure 2 shows the subgroup results, as defined by measures of disease severity, from the failed trials referred to above. Again, there is no evidence that the potential efficacy of the therapeutics within these subgroups varied by baseline placebo mortality rates. Logistic regression indicates that although patient severity is related to mortality (P < 0.0001), neither treatment (P = 0.32) nor an interaction between treatment and severity of illness (P = 0.70) was significantly related to mortality. For failed studies reporting survival data for subgroups defined by baseline measures of disease severity, four demonstrated lower mortality in the active treatment arm in subgroups with lower severity of illness. These were the studies by Opal and coworkers in 2004 [28], Abraham and colleagues in 2003 [29], Greenman and coworkers in 1991 [30] and Abraham and colleagues in 1995 [31] (Table 2). In two studies better outcomes were observed in higher risk subgroups whereas higher mortality was observed in the active treatment arms compared with placebo for some of the 'lower risk' subgroups: Fisher and coworkers (1994) [32], Knaus and Harrell (1996) [6], and Abraham and colleagues (2001) [33]. In the first IL-1 receptor antagonist (IL-1ra) study, lower mortality in the IL-1ra treatment group compared with placebo was observed for subgroups with a predicted risk for death of 24% or greater, regardless of dose [6]. However, in the follow-up study that sought to validate this observation [34] the opposite trend was observed.

Figure 2
figure 2

Distribution of treatment and placebo mortalities for sepsis trials by low and high risk patients.

Full size image

In the study of drotrecogin alfa (activated), better outcomes were observed in higher severity subgroups defined by APACHE II scoring and in lower severity subgroups defined by biologic markers of disease severity (i.e. IL-6 level and prothrombin time) [24]. For patients enrolled in the HA-1A study [25] lower mortality was observed in the active treatment arm than in the placebo group. The observed treatment effect was evident in patients with and without shock and with APACHE II scores above and below 25. The study by Annane and colleagues [27] did not report outcomes for subgroups defined by disease severity.

Among all studies, interventions with anticoagulant activity or anti-inflammatory activity did not appear to be harmful in patients with evidence of less coagulopathy (as assessed by coagulation tests) or less inflammation (as assessed by IL-6 levels). Three studies of medications with anticoagulant properties were reported [26, 29, 35]. For both studies in which outcome was reported for subgroups defined by baseline prothrombin times [26, 29], the observed relative reduction in the risk for death approached 50% for patients with normal coagulation status (international normalized ratio <1.2 or prothrombin time <14.5 s). Three studies reported outcomes by baseline IL-6 levels [24, 26, 36, 37]. In the study of drotrecogin alfa (activated) [24] large absolute and relative reductions in mortality were observed in patients with the lowest IL-6 levels, whereas IL-6 levels did not appear to influence the outcome of therapy with BAYx1351 or afelimomab [36, 37].

Safety assessments

Table 3 lists pertinent findings of the safety assessments conducted in each study. Current methodology for reporting and analyzing adverse events captured a number of safety concerns associated with multiple therapies. An increase in the incidence of bleeding complications was noted for all medications with anticoagulant properties. Complications related to allergic reactions were noted for some murine-based proteins. An increase in the incidence of serious cardiac adverse events was noted in a study of a nitric oxide synthase inhibitor. None of the studies listed detected an increase in the incidence of infectious complications related to the administration of medications with either anti-inflammatory and/or anticoagulant properties.

Table 3 Safety assessment
Full size table

Discussion

Recent publications and editorials have suggested that one possible explanation for the discordance between the preclinical efficacy and subsequent clinical failures of many therapeutics investigated in severe sepsis is that these therapies might be expected to reduce mortality only in the most severely ill patients or those patients at highest risk for death [6–8]. Implicit in this explanation is that these therapies must also produce a harmful effect in the 'less severe population' because a benefit was not observed in the overall population. We investigated whether evidence for an interaction between treatment and severity exists within published clinical data from phase III studies of these agents in severe sepsis.

Twenty-two publications investigating 17 different pharmacotherapeutic agents targeting the host response to infection were identified. Only phase III studies were included to reduce potential noise related to small sample size and multiple dosing regimens. Three studies met their prospectively defined end-point. The control mortality rates range between 31% and 61%, indicating that the beneficial effects of adjunct therapies could be demonstrated over a wide range of illness severity. For failed trials, lower control mortality rates did not appear to be an explanation for possible failures (Fig. 1). Furthermore, exploration of the reported subgroups for these studies also failed to demonstrate any possible interaction between treatment and disease severity that could have contributed to the lack of observed efficacy (Fig. 2).

The first phase III study of IL-1ra has frequently been cited as evidence for the existence of a differential effect of treatment based on disease severity [7, 32]. In that study a nonsignificant reduction in mortality was observed in the overall population. For patients with a predicted risk of death of 24% or greater, mortality was significantly lower in the active treatment arm than in the placebo group [6]. Higher mortality was observed in the active treatment arm for patients with a predicted risk of death below 24%. However, in the confirmatory study [34] the exact opposite was observed. A beneficial effect of treatment was observed in patients with a predicted risk for death below 24% (18% mortality for IL-1ra treated patients versus 24% mortality for placebo treated patients), whereas no difference was observed in patients with a 24% or greater predicted risk for death (42% in both treatment arms).

Additionally, interventions with anticoagulant activity or anti-inflammatory activity did not appear to be harmful in patients with evidence of less coagulopathy (as assessed by coagulation tests) or less inflammation (as assessed by IL-6 levels). Almost all investigated therapeutics had some reported anti-inflammatory effects. However, in the three studies that reported outcomes by baseline IL-6 levels [26, 36, 37] the observed treatment effect was either greater for patients with the lowest IL-6 levels or was unrelated to IL-6 level. In the study of drotrecogin alfa (activated) [24] large absolute and relative reductions in mortality were observed in patients with the lowest IL-6 levels. For studies of interventions with anticoagulant properties, outcomes appeared more favorable for the active treatment arms in patients with less coagulopathy at the time of study entry [24, 29]. These observations suggest, with the limitations of subgroup analyses applied, that interventions with anti-inflammatory or anticoagulant properties are probably not harmful in patients with less inflammation or less coagulopathy.

Review of the safety data from each of the published studies indicates that the current reporting system for adverse events appears to be adequate in capturing potential toxicities associated with these therapies. An increased risk for bleeding complications was noted for antithrombin III, tissue factor pathway inhibitor, and drotrecogin alfa (activated), which is consistent with their anticoagulant properties. Multiple studies investigating murine-based antibodies documented allergic and anaphylactic reactions associated with therapy. Antibody formation was also documented. The nitric oxide synthase inhibitor study, in which statistically higher mortality was observed in the active treatment arm, reported a higher incidence of cardiovascular related deaths in the active treatment arm. No study reported adverse events that might be considered related to inhibition of the host response to infection. However, in the study conducted by Bone and coworkers [22] of high dose methylprednisolone (primary end-point: 14-day mortality), mortality attributed to the secondary infections was significantly increased in the methylprednisolone group than in the placebo group. Taken together, these data indicate that, as designed, clinical trial databases capture potential drug-related events and complications associated with investigational therapies.

Interactions between treatment and disease severity can be generally classified into two major categories: quantitative interactions, in which the relative benefit of a drug may be less in less severe disease; and qualitative interactions, in which the drug is truly beneficial at one level of severity and truly detrimental at another. For quantitative interactions a more favorable benefit:risk ratio might exist for patients with more severe disease, assuming that the absolute benefit of therapy is greater for more severely ill patients and that the absolute risk of therapy is uniformly distributed across disease severities. This type of interaction is not uncommon, and physicians and other health care providers commonly weigh the benefits and risks associated with all medicines before administration. Drotrecogin alfa (activated) may exhibit a quantitative interaction particularly from a risk:benefit perspective, because absolute reductions in mortality are larger for populations at higher risk for death whereas the bleeding complications of therapy appear to be independent of disease severity. As a consequence, many regulatory agencies limited the use of drotrecogin alfa (activated) to patients with severe sepsis at higher risk of death [38, 39].

Qualitative interactions, on the other hand, are probably rare [40]. This type of interaction suggests that the biologic effect of the drug (e.g. an anti-inflammatory effect) is beneficial in high disease severity and that the same biologic effect is detrimental in low disease severity. A systematic review of the available published literature does not support the hypothesis that such a qualitative interaction between treatment and severity exists for compounds that target the host response to infection.

Understanding whether data from prior clinical trials suggest that these agents might produce differential effects on survival depending on a patient's severity of illness is important in designing future trials of newer agents in severe sepsis. If present, such a qualitative interaction could warrant a change in the design of phase III trials that currently enroll patients with severe sepsis over a wide range of disease severity (e.g. one to five organ dysfunctions) and predicted risk for death (between 20% and 60% mortality at 28 days from the start of treatment). Additionally, current recommendations for the treatment of severe sepsis assume, to some degree, the absence of interactions between treatment and disease severity. For example, intensive insulin therapy was demonstrated to reduce mortality in a population of nonseptic patients at very low risk for death [41]. Intensive insulin therapy was provided a grade B recommendation for the treatment of patients with severe sepsis [42] – a population of patients at much higher risk for death. Low-dose steroid therapy reduced mortality in a population of patients with severe sepsis and refractory septic shock (hypotension despite vasopressor administration; placebo mortality 63%) [27]. Low-dose steroid therapy was given a grade C recommendation [42] for the treatment of patients with severe sepsis requiring vasopressor therapy but not necessarily with persistent hypotension. In clinical studies, patients who require only vasopressor support have mortality rates ranging between 26% and 43%, depending on vasopressor dose [24, 29, 32, 35].

The lack of evidence for a qualitative interaction between treatment and severity does not preclude that such an interaction may indeed exist. The recently completed study of drotrecogin alfa (activated) in patients with severe sepsis at lower risk of death (ADDRESS) [43] was stopped because of futility, indicating an inability to demonstrate efficacy in low risk patients. The futility of the study might have been driven by an adverse outcome in surgical patients with a single organ dysfunction, but the unfavorable outcome in surgical patients with a single organ dysfunction might not have been driven by lower severity per se, because a similar outcome was not observed in medical patients with a single organ dysfunction. Other potential confounders might include difficulty in making a diagnosis of severe sepsis in postoperative patients with a single organ dysfunction, a delay in treatment of these patients because of the requirement to delay therapy for 12 hours after surgery, and a higher risk for bleeding complications. This adverse finding prompted revision of the product label for drotrecogin alfa (activated) [38, 39].

The observations from the ADDRESS study [43] underscore the need for studies to enroll a heterogeneous population of patients to investigate the safety and efficacy of biologic response modifiers. In the absence of such investigation, physicians will be forced to extrapolate data across populations of critically ill patients, as has been done with insulin and steroid therapy [42]. Consequently, sponsors and principal investigators should consider increasing the sample size for phase III studies beyond that necessary to detect the treatment effect in the overall population in order to allow more robust assessment of treatment effects across clinically relevant subgroups. The use of power calculations conditional on a statistically significant treatment effect being observed in the overall trial might be useful. We would also recommend that the need to assess treatment effects across subgroups be considered when designing the efficacy stopping rules for interim analyses of large phase III studies.

There are multiple limitations to the above analyses. As with any analysis based on literature review, there is always a concern regarding publication bias. Furthermore, analyses based on post hoc subgroups is also biased because those subgroups reported in publications of negative trials may represent those in which a beneficial treatment effect was observed in at least one stratum, leaving an unfavorable effect in the complementary strata. The study of HA-1A was included in this analysis despite concerns over the statistical rigor of the study because the intent of the present analysis was not to demonstrate that any therapy was or was not effective but to determine whether evidence supporting an interaction between treatment and disease severity exists amongst published clinical data. A follow-up study of HA-1A in a similar population of patients with severe sepsis was stopped because of futility [21]. The follow-up study was not included in this analysis because it reported only 14-day all-cause mortality as the primary end-point.

Conclusion

A review of published clinical data does not support the hypothesis that there is a qualitative interaction between treatment and severity associated with anti-inflammatory or other agents used to treat severe sepsis. Clinical studies in severe sepsis should continue to enroll patients over a wide range of disease severity and risk for death, as long as patients enrolled have evidence of sepsis-induced organ dysfunction(s), patients are at an appreciable risk for death (e.g. as evidenced by admission to an intensive care unit), and the potential for benefit outweighs any potential for harm.

Key messages

  • Severe sepsis trials assume that the relative treatment effect of an intervention is independent of the patients' risk for death.

  • Three studies met the prospectively defined primary end-point of a statistically significant reduction in 28-day all-cause mortality.

  • The lack of demonstrated efficacy in the negative studies is not due to enrollment of less severe or more severely ill patients.

  • Interventions with anticoagulant activity or anti-inflammatory activity did not appear to be harmful in patients with evidence of less coagulopathy or inflammation.

  • Clinical studies in severe sepsis should continue to enroll patients over a wide range of disease severity.