The initial aims of the Surviving Sepsis Campaign, which arose from the Declaration of Barcelona in October 2002, were to raise public and professional awareness of severe sepsis and its treatment, the development of practice guidelines and “worldwide standards of care…through the development of global protocols”. These latter aims led to a partnership with the Institute of Healthcare Improvement, which established treatment bundles described as “a group of interventions that, when executed together, result in better outcomes than when implemented individually’.

The first iteration of the guidelines arising from the campaign was published in 2004, with a second in 2008. Whilst generally welcomed, pharmaceutical company funding and promotion led to robust criticism in some quarters, which was equally robustly rebutted [1–3].

Regardless of the controversy surrounding the funding and promotion of the campaign, the campaign continues, and the true test should be whether it has improved the treatment and survival of patients with severe sepsis. In this issue of Intensive Care Medicine, Levy and colleagues seek to address these important questions [4].

Their report analysed compliance with the resuscitation and management bundles, treatments to be completed within 6 and 24 h of the diagnosis of severe sepsis, respectively, using data from over 15,000 subjects treated in 165 hospitals. In addition, they examined the association between compliance with the bundles and in-hospital mortality. They conclude that the campaign was “associated with sustained continuous quality improvement in sepsis care” and “a reduction in reported hospital mortality rates”. Taken at face value, these conclusions suggest that the campaign has been highly effective and may have reduced hospital mortality. Do these conclusions stand up to closer scrutiny?

The first issue to address is the effectiveness of the individual components of the sepsis bundles. The guidelines published in 2004 drew on evidence published predominantly between 2000 and 2003, and subsequent research has called a number of components into question. The CORTICUS study did not confirm that low-dose corticosteroids were beneficial [5], the NICE SUGAR Study reported that targeting tight glycaemic control may be harmful [6], Early Goal Directed Therapy is the subject of no less than three ongoing clinical trials supported by national research funding agencies, and the effect of drotrecogin alfa (activated) is being re-examined in both industry-sponsored and investigator-initiated trials. Thus, whilst the current study can report its effectiveness in changing clinician’s behaviour, increased uptake of the bundles will only represent “continuous quality improvement” if the sum of the parts is beneficial to patients.

The second issue is to critically examine changes in practice occurring during the conduct of the campaign. In a previous report, complete compliance with the bundles after an implementation programme in Spain was relatively low, being only 10.0 and 15.7% for the resuscitation and management bundles, respectively [7]. The current study reports compliance with all resuscitation measures increased from 10.9 to 32.5% and for the management bundle from 18.4 to 25.5%. (Only 34 hospitals contributed data for the whole 2 years, for these hospitals the final figures for compliance were 31.3 and 36.1% respectively.) Compliance with individual components is much higher, but advocates of bundles claim they are more effective if enacted in their entirety. Although the increase in compliance achieves statistical significance, compliance remained relatively low. Further uncertainty arises as the investigators could not differentiate between failed attempts to achieve targets and the absence of an attempt at all. In addition, it is unclear whether some of the changes are artifactual and result from changes in measurement practices rather than changes in treatment. For example, the increased compliance with central venous pressure and oxygen saturation goals may represent increased measurement of these variables rather than changes in patient management.

To determine whether a change in observed mortality could be explained by a changing patient population or reducing mortality trend over time, various logistic regression models were constructed using the baseline risk factors available to the investigators. Screening and identification of patients with severe sepsis were the responsibility of local investigators, and there was no supervision by the campaign; as a result, we do not know whether selective reporting may have introduced bias. Likewise, there were no quality checks made on the data submitted, and these limitations are understandable given the voluntary and unfunded nature of the study. However, the major methodological weakness of the study is the lack of a control group. Ideally, such a complex intervention would be studied in a cluster randomised trial with hospitals randomly assigned to implement the bundles or to continue current practice [8]. However, such studies are challenging, and apart from a few notable exceptions, this methodology is used infrequently in critical care research [9, 10]. An alternative although inferior design would have been to use the hospitals as their own controls by documenting treatment and mortality trends prior to implementing the campaign, the approach used in the study in Spain. Ideally, such a study would conduct a time series analysis to determine whether the rate of change of mortality changed significantly once the campaign was introduced; similar methodology has recently been used to assess the impact of modernising critical care services in England [11]. This consideration is important as there is convincing evidence that the mortality of patients with severe sepsis is decreasing around the world. For example, a database study conducted in Australia and New Zealand, where the Surviving Sepsis Campaign guidelines have not been embraced [12, 13], reported that mortality for patients with severe sepsis or septic shock admitted from emergency departments decreased from 28.1% in 2002 to 21.2% in 2005 [14]. In such circumstances comparing crude mortality rates before and after an intervention ignores the underlying trend and will give rise to misleading conclusions.

How then should we interpret the data presented in the current report, and where should we go from here? It is clear that severe sepsis and septic shock remain major public health issues that are likely to result in increasing morbidity and mortality due to ageing of the population of developed countries. They also remain major killers in the developing world. The underlying goals of the Surviving Sepsis Campaign to increase awareness, develop guidelines and improve practice are as laudable today as when first enunciated in 2002. What has changed since 2002 has been the development of major national and international consortia conducting investigator-initiated research [15], and consequently the realisation that research data are only reliable if the research is of the correct design and conducted to the highest methodological standards [16, 17].

Clinicians seeking to improve the treatment of patients with severe sepsis may choose to implement the Surviving Sepsis Campaign bundles in their entirety as even the most conservative conclusion from the current report is that doing so is unlikely to cause harm; indeed, increased awareness as a result of the campaign may be partly or even predominantly responsible for reduced mortality observed around the world. Others who are less convinced by the primary evidence may take a more conservative approach and await the results of ongoing trials. We should welcome the fact that baseline mortality appears to be decreasing, but there is still much work to be done. A beneficial effect of the guidelines on patient outcomes is currently unproven, and the primary evidence is not yet of sufficient quality to promote the guidelines as a global standard of care.