Background

Sepsis is a well-recognized worldwide healthcare issue, ultimately resulting in significant mortality [1], morbidity and resource utilization during and after critical illness [2]. Initial consensus definitions relied upon the systemic inflammatory response syndrome (SIRS) to infection as a fundamental aspect of sepsis definition, which was then stratified in severity according to the presence of organ dysfunction (severe sepsis) or vasopressor requirement despite adequate fluid resuscitation (septic shock) [3]. This had an important role in education and the pathophysiological understanding of the transition between a homeostatic and a dys-homeostatic inflammatory response that results in organ dysfunctions.

In a second consensus conference, although a new panel of specialists agreed that SIRS was not necessary to the definition of sepsis, not much was changed. The panel recognized several factors that could be used to identify patients with severe sepsis and associated with increased mortality, ultimately leading to important efforts for the early recognition and treatment of severe sepsis and septic shock. This consensus also developed the PIRO concept (predisposition, infection, response, organ failure); however, this idea was not largely translated to clinical practice [4].

Sepsis definitions based on SIRS criteria have, though, been questioned as of late, as SIRS has been shown to be present in 93% of all patients admitted to intensive care (and therefore overly non-specific) [5]. Furthermore, up to 1 in every 8 patients with infection and organ dysfunction do not meet SIRS criteria [6]. Moreover, the methodology used to capture SIRS criteria may lead to substantial variability in defining sepsis cases [7, 8]. These and other concerns have led to the development and publication of new sepsis definitions (Sepsis-3), which were derived through a data-driven mortality risk stratification [9–11]. Although these new criteria have been validated in large databases, much controversy still surrounds them. Some criticisms include the new definitions’ lack of validation in scenarios outside of high-income countries, and their non-utilization of lactate as a marker of organ dysfunction [12–16].

Therefore, in order to address these concerns, we aimed to evaluate whether the new Sepsis-3 definitions maintain their accuracy in a different setting from those in which the criteria were developed and whether they can discriminate intensive care unit (ICU) mortality better than the previous definitions (Sepsis-2). We also evaluated whether the addition of lactate to the definitions would improve upon the previous and new criteria’s accuracy of ICU mortality discrimination.

Methods

Study design, population and ethical requirements

This was a retrospective cohort study of patients diagnosed with sepsis admitted to an ICU over a 6-year span, from January 2010 to December 2015. Data were retrieved from the prospectively collected database of a 19-bed ICU in a Brazilian, academic, tertiary medical center in São Paulo—Brazil (Hospital das Clínicas, University of São Paulo Medical School). The study protocol followed the tenets of the Declaration of Helsinki. The institutional review board (Comissão para Análise de Projetos de Pesquisa—CAPPesq) reviewed and approved this study (CAPPesq—protocol number 107.443). Since the study was retrospective in nature and did not involve patient identification, informed consent was waived as there was no intervention.

Setting and sepsis management

Our ICU is located in the Hospital das Clínicas, the largest healthcare complex in Latin America. At the time when the study was conducted, it consisted of seven specialized institutes, with a total of 2400 beds. In the Instituto Central, there are 7 ICUs, ours being mainly a Medical ICU, with patients from emergency surgery and trauma being admitted occasionally for logistical reasons.

The ICU is managed by staff as follows: one nursing assistant for every two beds; one nurse for every five beds; one respiratory therapist for every 10 beds; one staff physician for every 5–8 beds; and residents of internal medicine, critical care, physical therapy, nutrition and nursing. The studied ICU is classified as a strained unit as occupation rate has been above 95% since the beginning of the study period, and the mean SAPS 3 of the patients is 60 [17]. Neither quantitative resuscitation nor protocolized care is routinely used for sepsis management. Our approach to sepsis care is described in Additional file 1.

Data collection

Our database is an electronic health chart record fulfilled by physicians and respiratory therapists on a daily basis. Patients were selected using sepsis, severe sepsis or septic shock as an [All field] search term in the syndromic diagnosis fields of the database. Senior intensivists clinically adjudicated these diagnoses based on previous Sepsis-2 consensus. Patients with any acute organ failure were considered to have severe sepsis, while septic shock was defined when patients were on vasopressors despite fluid resuscitation. These were adjudicated independent of the presence of 2 SIRS criteria. The following clinical data were collected: age; gender; worst and best vital signs during the first ICU day; Simplified Acute Physiological Score (SAPS) 3; first day total Sequential Organ Failure Assessment (SOFA) score; syndromic diagnosis; etiological diagnosis; comorbidities; ICU length of stay (LOS); organ support measures; clinical ICU outcomes; and highest lactate level of the first day. Laboratory variables were retrieved from the electronic health database specific to laboratorial data.

Sepsis definitions

Sepsis-2 definitions were primarily used to classify our patients in the database. The categories (sepsis, severe sepsis and septic shock) were defined according to previously published consensus [4]. The new Sepsis-3 categories were defined as follows [9]:

  • Infection without acute organ dysfunction (no-dysfunction category) infected patients with no significant additional organ dysfunction over the previous conditions, that is, a variation of total SOFA <2 over baseline (chronic organ dysfunction) during the first 24 h after ICU admission.

  • Sepsis (sepsis category) infected patients with a total SOFA variation ≥2 over baseline clinical condition.

  • Septic shock (septic shock category) infected patients with persistent hypotension [mean arterial blood pressure (MAP) <65 mmHg] after adequate fluid resuscitation needing vasopressors to keep MAP ≥65 mmHg. Additionally, the hypotension or need for vasopressors must be associated with lactate level >2 mmol/L measured during the first 24 h.

Baseline total SOFA score was considered to be 4 in patients undergoing chronic dialysis, and 2 or 3 in cirrhotic patients, depending on baseline bilirubin levels. For instance, according to the Sepsis-3 definition [9], a patient with chronic renal failure undergoing dialysis was considered to be in the sepsis group only when the total SOFA score was ≥6. Although the original validation of Sepsis-3 definitions did not include patients with ICU LOS <2 days [9], we included all patients in the period described, independently of LOS, in order to pragmatically test the definitions in our ICU.

Outcome

The primary outcome of interest was ICU mortality.

Statistical analysis

Continuous variables were assessed for normality using the Kolmogorov–Smirnov test. Parametric continuous variables were compared between groups using the t test; nonparametric variables were compared using the Mann–Whitney test. Categorical variables were compared between groups using the Chi-squared test.

We assessed the independent association between lactate and outcome regardless of definition (both Sepsis-2 and Sepsis-3) group using a logistic regression model. Six models were built, three for each definition considering three different cutoffs for lactate (as a continuous predictor, stratified at ≤2 or >2 mEq/L and at ≤4 and >4 mEq/L). Since serum lactate measurements were positively skewed, we used log transformation to fit it into the model as a continuous variable. We further assessed the independent association of Sepsis-3 category and lactate level after adjusting for age, severity at baseline (SAPS 3) and use of support measures (renal replacement therapy, mechanical ventilation). Demographic variables (age and sex) were selected based on univariate analysis, with p values < 0.2 used as cutoff. The other confounding variables were selected because of clinical relevance.

The effect of adding lactate (as a continuous variable or stratified at ≤2 or >2 mEq/L or at ≤4 or >4 mEq/L) on model performance was assessed through area under receiver operating characteristic curve (AUROC). AUROCs were compared using DeLong’s test. Additionally, we assessed the impact of adding lactate to the model using continuous net reclassification improvement (NRI) and integrated discrimination index (IDI), as previously suggested [18]. NRI compares classifications from 2 models (in this case, with and without lactate) for changes by outcome for a net calculation of changes in the right correction. IDI integrates the NRI over all possible cutoffs and is equivalent to the difference in discrimination slopes. In this paper, based on the histograms of the probability of ICU mortality distributions, the cutoff values for reclassification were the following: <30; 30–60; and >60%. We do not report categorical NRI since no single probability cutoff has been defined for the decision-making process in caring for septic patients, as discussed in the Sepsis-3 consensus [10].

Missing values were handled as previously suggested for studies in critical care, when missing values are <5% [19]. There were 46 (<5%) missing lactate values of the first day, which were replaced by the first second day value, or by median imputation if there were not data available. All statistical analyses and graphs were done in R-free source software [20].

Results

The general characteristics of patients, as well as support measures, clinical outcomes, source of infection, and comorbidities are given in Table 1. For the whole group, mean age was 52 years, without gender differences. The most frequent infectious source was respiratory, followed by abdominal and urinary tract sources. The most frequent comorbidities were diabetes, heart failure and hypertension. Sixty percentage of patients used vasopressors; 44%, mechanical ventilation; and 24%, renal replacement therapy. ICU mortality was 32%. In univariate analysis, non-survivors had higher mean age, severity at admission (SAPS 3), more organ dysfunctions (SOFA) and higher worst lactate values. No infectious source was associated with mortality, neither comorbidities, except for hematological cancer. They also presented more frequently with septic shock when compared to other categories. Furthermore, invasive support measures were more frequently used in this population.

Table 1 General characteristics, support measures and outcomes of patients
Full size table

Figure 1 depicts that Sepsis-2 definitions show reasonable differentiation in mortality only between septic shock and severe sepsis patients, while Sepsis-3 definitions show clear stratification of mortality among the three categories (no-dysfunction, sepsis and septic shock). Additional file 2: Table 1s shows patients’ re-allocation between both sepsis definitions. Sepsis-3 classification reduced the percentage of septic shock patients as compared to the previous definition.

Fig. 1
figure 1

Mortality stratified according to sepsis definition. a Mortality according to the Sepsis-2 definition (Pearson’s Chi-squared test using the three categories P < 0.001. Pearson’s Chi-squared post hoc analyses are shown in the figure). b Mortality according to the Sepsis-3 definition (Pearson’s Chi-squared test using the three categories P < 0.001. Pearson’s Chi-squared post hoc analyses are shown in the figure)

Full size image

Additional file 3: Figure 1s shows the median lactate levels of each sepsis category in both sepsis definitions. Significant differences between lactate levels occurred only in the septic shock categories for both definitions (Panels A and C) and between survivors and non-survivors in both septic shock definitions (Panels B and D). The mortality rates according to the sepsis definitions, sepsis categories and serum lactate concentrations are shown in Additional file 4: Fig. 2s. Lactate levels higher than 2 mmol/L were not clearly discriminative of mortality in the Sepsis-2 defined categories of severe sepsis or septic shock. In comparison, lactate levels >4 mmol/L clearly marked increased mortality in both the severe sepsis and septic shock categories of Sepsis-2 and in the no-dysfunction and septic shock categories of Sepsis-3.

Table 2 shows the associations of sepsis definitions with mortality and the additional impact of lactate inclusion in the multivariate model. Furthermore, Table 3 shows the association of Sepsis-3 definition with ICU mortality, after adjustment for lactate level, age, illness severity (assessed through SAPS 3), requirement for RRT and mechanical ventilation at any time of ICU stay.

Table 2 Multivariate models using ICU death as a dependent variable and sepsis definition and lactate as independent variables
Full size table
Table 3 Multivariate models using ICU death as a dependent variable and Sepsis-3 definition, lactate, age, SAPS 3 and requirement for RRT and mechanical ventilation as independent variables
Full size table

Table 4 shows the accuracy of ICU death prediction of models using only the sepsis definitions and sepsis definitions added to lactate as a continuous and dichotomous variable. The accuracy significantly improved with the addition of lactate to the model.

Table 4 Predictive accuracy of mortality of sepsis definitions with or without lactate value
Full size table

Further evaluation of lactate’s addition to the sepsis definitions model using the net reclassification index (NRI) and integrated discrimination increment (IDI) is presented in Additional file 2: Table 2s. In Sepsis-2 definitions, NRI as it relates to ICU mortality was 23.1 (P < 0.001) for lactate >2 mmol/L and 39.9 (P < 0.001) for lactate >4 mmol/L. For Sepsis-3 definitions, NRI was 39.9 (P < 0.001) for lactate levels >4 mmol/L. The discrimination slope of the updated model with lactate levels >4 mmol/L, as assessed by IDI, was 3.3 (P < 0.001) for Sepsis-2 and 2.7 (P < 0.001) for Sepsis-3. For lactate levels >2 mmol/L, IDI was 0.7 (P = 0.021) for Sepsis-2 criteria, while it was not significant for Sepsis-3.

Discussion

In this study, we demonstrate that Sepsis-3 is better than Sepsis-2 at stratifying mortality among septic patients admitted in a strained ICU of a developing country. Sepsis-3 criteria clearly categorized septic patients along a spectrum of severity, since mortality increased from 7% in no-dysfunction-infected patients to 25 and 46% for sepsis and septic shock patients, respectively. In contrast, in the Sepsis-2 definitions, only the septic shock criterion was associated with a significantly higher mortality when compared to the severe sepsis and sepsis categories (44, 17 and 13%, respectively). Moreover, a lactate level higher than 4 mmol/L improved the mortality prediction accuracy on both Sepsis-2 and Sepsis-3 definitions (Table 3).

The validity of sepsis definitions based on SIRS criteria has recently been called into question. This is because almost 90% of the patients admitted to an intensive care unit (ICU) meet the SIRS criteria, [5, 21]. Even SIRS criteria’s high sensitivity has been questioned recently, as some patients with known infectious insult and new organ failures do not satisfy 2 SIRS criteria and therefore do not fulfill previous sepsis definitions [6]. Even more importantly, the categories of sepsis, severe sepsis and septic shock should indicate a real spectrum of severity to be considered as different outcome categories [14]. The present study reinforces a lack of accuracy of Sepsis-2 definitions of sepsis, severe sepsis and septic shock in stratifying patients’ mortality risk. Additionally, our data demonstrate Sepsis-3 definitions of infection without organ dysfunction, sepsis and septic shock clearly represent progressive strata of mortality risk (Fig. 1).

High lactate values in sepsis may be the result of perfusion abnormalities or stress hyperlactatemia [22]. Regardless of the reason for its increase, early high lactate value is traditionally associated with worse outcomes in septic [23] and other critically ill patients [24, 25]. The fact that lactate was not maintained as a marker of organ dysfunction in the new sepsis definitions raised several concerns [26, 27], and the Surviving Sepsis Campaign suggested that lactate should continue to be measured to identify dysfunction [27]. In the original validation study of Sepsis-3 definitions, although high lactate levels were associated with higher mortality at different cutoffs regardless of hypotension or vasopressors, lactate was retained only for the characterization of septic shock [9]. In the present study, however, lactate was a prognostic factor for both definitions and reclassified some Sepsis-3-defined patients when its levels were above 4 mmol/L. In the assessment of the new septic shock definitions in the original validation study, lactate levels higher than 2 mmol/L were increasingly associated with higher odds of death [11]. In our study, cutoff levels above 2 mmol/L did not add any prognostic information in this stratum of severity, as expected, because it is included in the definition itself. However, in accordance with the validation paper, higher lactate levels (which we chose to add in the model as >4 mmol/L) were of prognostic significance.

In the validation of clinical criteria for sepsis, unfortunately, there was no clear mention to the added benefit of lactate for patients with SOFA scores <2, possibly because of lactate values missingness in the derivation (about 60%) and validation cohorts (about 90%) [10]. Interestingly, lactate was not retained in the novel qSOFA during model construction. However, the authors state that for a qSOFA score of 1, high lactate values characterized a population with similar risk to patients with a qSOFA score of 2 [10]. In our cohort, for patients without organ dysfunction, lactate levels higher than 4 mmol/L seem to categorize a population at higher risk of death (Table 2 and Additional file 4: Fig. 2s). Therefore, high lactate levels in patients without organ dysfunction as assessed by the SOFA score might improve risk stratification and change management of these patients, since they could be allocated to a higher level of care upfront in order to avoid further deterioration of their clinical status.

Sepsis-3 criteria for identifying septic patients have been extensively questioned since its publication [12, 15]. A possible decrease in awareness of sepsis and a need for massive educational programs to healthcare providers are among the main concerns. In this context, how Sepsis-3 criteria would perform for sepsis detection, especially in countries with high mortality rates, still needs validation. Our study was not designed to assess Sepsis-2 or Sepsis-3 as a screening tool, since only patients admitted to ICU were included. The role of Sepsis-2 criteria for identifying septic patients is undeniable, as was the case in a recent before–after study conducted in Brazil [28]. However, its accuracy to stratify severity can be called into question, as demonstrated by our data. Accordingly, our study clearly shows a better discriminative performance of Sepsis-3 in predicting ICU mortality.

Strengths of this study include the number of patients analyzed, the recent timeframe and the use of statistical methodologies such as net reclassification index, which allowed us to better evaluate the benefit of adding lactate to the new definition. Also, NRI was not used in the original validation study and our data missingness for lactate values was much lower (<5%), which could have yielded some of the above different results. Furthermore, this represents an external assessment of new criteria in a different setting than that of the original validation—an important step toward consistency for its findings.

There are, nevertheless, several limitations to be addressed, in light of which our findings should be interpreted. First, the retrospective nature of this study makes it difficult to elucidate known confounders that could have biased the outcome measures, such as differences in sepsis treatment. Second, this was a single-center study consisting mainly of medical patients and our results may not be generalizable to other centers, including from other developing countries, since there is wide variation in outcomes even within Brazil when comparing different settings [29, 30]. Third, we evaluated only patients admitted to the ICU and, therefore, our findings cannot be extrapolated to patients treated in wards and in the emergency room. This also threatens the generalizability of our results, since the original study captured a population with presumed infection not only in the ICU. Fourth, the validity of Sepsis-3 criteria in this study was assessed based on ICU mortality. Although ICU mortality is not the most appropriate endpoint to be evaluated, it reflects the pre-ICU and ICU care of septic patients, and in our scenario, identified a more specific point to improve the care of septic patients [31, 32].

Conclusions

The new clinical criteria of sepsis proposed by the third international consensus (Sepsis-3) could predict mortality in infected patients admitted to a strained ICU in a middle-income country. The prognostic value of the new definition of sepsis was progressive along all three categories, which denotes a spectrum of gravity of infectious process and could help in risk stratification for future studies in this area of research. This was not observed of the previous definitions (Sepsis-2) in our sample. Serum lactate improved accuracy for values higher than 4 mmol/L in the no-dysfunction and septic shock groups.