Background

Mortality from sepsis is persistently high, and may even be rising despite decades of research [1, 2]. Promising pre-clinical immunomodulatory therapies have failed in clinical practice [3–5] perhaps attributable, in part, to differences between human and rodent immunology [6]. However, an alternative explanation is that the heterogeneous etiology, presentation and progression of human sepsis generate confounding factors that distort the interpretation of clinical immunologic studies. Thus, the identification of appropriate controls, diagnostic accuracy, demographic influences and therapies with immunomodulatory off-target effects are critical considerations in interpreting translational work.

We therefore systematically reviewed the clinical experimental design of studies in three key areas of bed-to-benchside immunologic research in sepsis, focusing in particular on comparator groups and the documentation of known confounding factors. We also explored how the investigation of immune mechanisms in other clinical scenarios - trauma and major elective surgery - associated with the development of sepsis may help refine experimental design.

Methods

A Pubmed search was performed for the terms‘Neutrophil respiratory burst’ OR‘Monocyte endotoxin tolerance’ OR‘Lymphocyte apoptosis’ AND‘Sepsis’ OR‘Trauma’ OR‘Surgery’, restricted to adult human studies published between 03 January 1998 and 03 January 2013. The abstract of each paper was manually assessed for suitability. In vitro studies of healthy volunteer cells were excluded.

Clinical demographics

For all eligible manuscripts, we recorded the primary author, year of publication and clinical setting. The number, age, gender, clinical severity score of subjects and their corresponding controls, in whom the same assay of immune function was performed, were compared. The criteria used to define sepsis - complete with evidence for microbiological confirmation and independent adjudication of the sepsis diagnosis - were also recorded. Since immune cell effector function may change over the course of sepsis, we also recorded details of the timing of initial and subsequent blood samples, and the reason for blood sampling itself. Given that a recent report detected differences in genomic markers of inflammation that associate with survival within the first 24 h of intensive care admission [7], we assessed whether samples were obtained within, or beyond, this 24-h window. Since several commonly used therapies used in intensive care medicine exhibit immune modulating effects, we also recorded whether common immunomodulatory agents including antibiotics [8], glucocorticoids [9] and sedative agents [10] were documented. Reporting of pre-existing immunosuppressive or malignant disease - or their specific exclusion - was also recorded.

Study aims

The specific aims of each study were recorded with regard to the experimental context and primary conclusion. The context within which each of the three functional assays was studied was classified as: Pathophysiological - observational mechanistic studies detailing evolution of the assay response in clinical samples; Experimental - use of patient samples for more detailed experimental investigations beyond the assay itself; Clinical outcome - correlation of outcome measure with assay response; Biomarker comparison - correlation of alternative assay with functional assay.

Laboratory samples

We recorded whether an a priori power analysis had been performed to determine the number of subjects/controls needed to refute the primary hypothesis. Sample timing and key aspects of experimental technique were compared between sepsis and control subjects. Associations made between immune cell function and clinical outcome were noted.

Statistics

Data are presented as mean ± SD, or median (interquartile range). Age data in primary studies was used to construct 95% confidence intervals in order to assess whether differences existed between control and study populations (NCSS 8, Kaysville, UT, USA).

Results

Fifty-seven eligible studies were identified, as summarised in Figure 1. Data is displayed into 3 tables for each immune assay, titled "Principal features of studies" (Tables 1, 2 and 3), "Demographic information" (Tables 4, 5 and 6) and "Experimental conduct and exclusion criteria" (Tables 7, 8 and 9).

Figure 1
figure 1

Flow diagram illustrating study identification and inclusion [11–66].

Full size image
Table 1 Principal features of neutrophil respiratory burst studies
Full size table
Table 2 Principal features of monocyte tolerance studies
Full size table
Table 3 Principal features of lymphocyte apoptosis studies
Full size table
Table 4 Demographic information of neutrophil respiratory burst studies
Full size table
Table 5 Demographic information of monocyte tolerance studies
Full size table
Table 6 Demographic information of lymphocyte apoptosis studies
Full size table
Table 7 Experimental conduct and exclusion criteria of neutrophil respiratory burst studies
Full size table
Table 8 Experimental conduct and exclusion criteria of monocyte tolerance studies
Full size table
Table 9 Experimental conduct and exclusion criteria of lymphocyte apoptosis studies
Full size table

Source of experimental control subjects

No studies reported a priori power analyses based on either preceding laboratory data or ex vivo clinical research. The majority of studies (42/57; 74%) used case–control methodology. Control samples were obtained from healthy volunteers in (35/42; 83%), with the remainder using a variety of loosely defined clinical phenotypes (Figure 2, Tables 1, 2 and 3). The exception was elective surgical patients, where preoperative samples served as appropriate controls. Cohort methodology, where samples including controls were obtained serially from the same patient, was employed in 14/57 (25%) of studies. The majority of cohort studies were conducted in elective surgical patients (12/14; 86%).

Figure 2
figure 2

Identification of experimental control groups. The specific details for Hospital/ICU patients are detailed within Tables 1, 2 and 3. Within cohort study pre-insult baseline samples were taken from the study population, allowing them to act as their own experimental control.

Full size image

Age, gender and ethnicity

Advanced age is associated with progressively impaired innate and adaptive immunity [67]. Less than half of case control studies (20/42; 48%) reported the age distribution of both study and control populations. In studies where age was reported, the critically ill patients studied were often older than the control population. Female gender is associated with improved clinical outcomes following sepsis [68, 69] and increased longevity compared to males in general. Information on gender was provided in (26/42; 62%) of case–control studies. Significant variation in the incidence of sepsis has been reported according to ethnicity [70], which may reflect residual confounding or plausible biologic differences in susceptibility. However, only one study reported the ethnicity of patients.

Co-morbidity

Various comorbidities ranging from cardiac failure to active malignancy are associated with important deleterious alteration in effective immune function, independent of those described in sepsis [71, 72]. The majority of studies (34/57; 60%) excluded patients with overt immunosuppression while a minority (8/57; 14%) excluded those with malignancy (Figure 3).

Figure 3
figure 3

Documentation of patients’ comorbid disease.

Full size image

Clinical definition of sepsis

A high proportion of studies (26/33; 79%) defined sepsis in accordance with the ACCP/SCCM [73, 74] or Surviving Sepsis Campaign (2008 update) [75] criteria. Of those studies which used standard consensus conference criteria, (15/26, 58%) included patients with‘sepsis’, (20/26; 77%) included those with‘severe sepsis’ and (24/26, 92%) included those with‘septic shock’. In a large minority of these 26 studies (11/26; 42%), sub-categories defining sepsis were not compared separately, but combined. Immunologic studies in trauma and surgical patient samples usually did not document (18/24; 75%) whether patients developed an infection during the course of the study. In these studies, the majority (5/6) used established consensus conference criteria.

Microbiological definitions of sepsis

Independent adjudication of the definition of sepsis used in studies was undertaken in 17/57 (30%) of studies. Since recent basic laboratory studies have demonstrated that the clinical signs/symptoms of sepsis are frequently mimicked by non-pathogenic molecules [76, 77], we sought to establish whether microbial evidence for sepsis was presented. Microbiological data were provided in 25/57 (44%).

Severity of critical illness

A minority of studies (19/57; 33%) provided data on organ dysfunction related to sepsis severity, such as APACHE-II or SAPS II. When a severity index was used, a wide range was reported within individual studies suggesting substantial heterogeneity. In studies where mortality was reported (4/57; 7%), severity of critical illness was not reported in those patients who survived.

Timing of experimental samples

The timing of the index blood sample obtained from septic patients was described in the majority (26/33; 79%) of cases. However, the criteria for initial sampling were not comparable between studies and was most frequently defined by the severity of sepsis (Figure 4). These triggers included hospital admission (1/26), ICU admission (5/26), proof of infection (2/26), diagnosis of sepsis (5/26), onset of sepsis (14/26; 54%), onset of organ failure (3/24) and onset of septic shock (7/26) - the remaining two samples were from autopsy studies. Multiple criteria for sampling were often used and dependent upon the severity of patient illness. Approximately half of all studies (14/26; 58%) obtained an initial sample within 24 h of hospital admission. Similar patterns of sample timing were described for trauma patients. Repeat samples were often undertaken, but over highly variable intervals that were frequently not defined a priori. By contrast, all 12 studies undertaken in the elective surgical setting obtained preoperative control samples, with subsequent samples taken on predefined postoperative days.

Figure 4
figure 4

Event trigger used for index blood sample to be taken within studies of septic patients.

Full size image

Therapies as potential confounders

Commonly administered therapies in intensive care impact directly on immune function [8–10]. We assessed reporting of three of the commonest therapies with established immunomodulatory properties and found that only up to a quarter of studies documented their use (Figure 5). Specifically, these were sedative agents (4/57; 7%), antibiotics (6/57; 11%) and steroids (15/57; 26%).

Figure 5
figure 5

Documentation of drug exposure of the study population.

Full size image

Experimental conduct and outcomes

There was no apparent relationship between the experimental context of studies and the control groups that were explored (Tables 1, 2 and 3). There are, however, clear associations between the study population studied and experimental outcome (Tables 1, 7, 2, 8, 3 and 9). For example, within the respiratory burst data, there is a consistent increase in respiratory burst identified by sepsis studies. However, since none of these studies used pre-illness samples, it is unclear if the change is a feature of sepsis, or the study population in relation to healthy volunteers. The conflicting results reported by the three surgical studies are difficult to interpret since each study uses a different burst assay, and the magnitude/type of operation varies. Similar patterns are also evident across the monocyte and lymphocyte studies.

Discussion

This systematic review has revealed several important issues in the design and reporting of immunologic phenotype in intensive care/sepsis studies. The studies we selected are representative of the current literature, covering the past 15 years of work in three key areas of sepsis research. Following a preliminary Pubmed search, these three assays were chosen because they represent the most frequently investigation for each immune cell type. These limitations refer to the clinical aspects of the study methodology rather than specific laboratory techniques that we did not assess. These data suggest that the use of surgical patients to model critical illness may overcome several key limitations.

Defining what constitutes an adequate control sample for the immunologic study of sepsis is clearly highly challenging. Case–control studies are frequently used in sepsis research. Our review suggests that case-control studies cannot easily determine whether the observed differences in the experimental readout between the study and control groups is due to sepsis per se, or other differences between the groups including age, comorbidities and treatment interventions. Whereas cohort studies do allow pre-sepsis samples to be taken, the majority of studies are conducted in healthy volunteers free of important comorbidities (e.g. heart failure, cirrhosis) that influence both the development of, and survival from, sepsis [71]. Furthermore, age-, gender- and ethnicity-related differences in immune function are well documented [67–70], yet our data demonstrates that several key demographic details for study and control populations were frequently not reported. Finally, the presence of malignant disease - associated with immunosuppression [72] and disproportionately represented in the ICU population of most healthcare systems - was only documented in a minority of studies.

Sepsis is currently defined using clinical constructs that define syndromes, rather than use biologic and/or molecular criteria. It remains unclear whether there are biologically relevant differences between clinically defined subtypes of sepsis. In other words, changes in immunophenotype associated with progression of sepsis to severe sepsis/septic shock may merely reflect the consequences of clinical interventions and/or indirect effects on organ function that partly reflect pre-existing comorbidities. Furthermore, the specific detection of pathogens, or pathogen-associated molecular patterns, is likely to further impact on the robustness of immunophenotyping since the location and type of micro-organism both regulate host-immune responses [77, 78]. We identified only one study that specified infection site and/or a specific pathogen [34].

Critically ill patients are exposed to a range of therapeutic agents that have well-described immunologic effects. Although immunomodulation by the majority of these agents has been established in vitro, their role in confounding the septic immunophenotype remains unclear. Nevertheless, a myriad of off-target, immune effects have been established in pre-clinical in vivo models. Many antibiotics target mitochondria and eukaryotic protein synthesis [79]. Steroids exert potent pro- and anti-inflammatory properties - including inducing lymphocyte apoptosis [9]. Similarly, sedatives and analgesics exert profound effects on immune cell function [80, 81].

Our data suggest that surgical patients offer important potential advantages for mechanistic studies of sepsis. The incidence of sepsis - as defined by conventional clinical criteria - varies from 6.98% to 12.25%, depending upon the health care system and database interrogated [82]. No other patient population allows the collection of highly phenotyped data and individualised control samples prior to a defined traumatic insult. Since the volume of surgery is huge and large scale outcome data can be collected, potential limitations including comorbidities and concomitant therapies can be controlled for.

Conclusions

We found several important limitations in clinical design associated with translational immunologic studies of human sepsis. Clinical design in mechanistic studies exploring changes in immunophenotype may contribute to the lack of translational therapeutic progress in intensive care medicine. Major elective surgery offers a potential model to overcome many of these methodological limitations.

Take-home message

Systematic review suggests a critical re-evaluation in design of immunologic phenotyping studies conducted in intensive care.

Tweet

Immunological investigation of septic patients presents methodological challenges that are not considered by many recent studies.