Background

Sepsis is among the leading causes of death with an estimated annual global incidence of 31.5 million cases of sepsis and 19.4 million cases of severe sepsis [1]. Although outcomes have improved due to collaborative efforts [2], mortality remains high with estimated in-hospital mortality rates of 17–26%. In patients presenting with septic shock, the estimated mortality is even higher with rates of 46% and higher depending on the definition [3].

Accurate definition, diagnosis, and early recognition of sepsis are crucial for the effective management of patients, as it improves outcomes [4, 5]. Sepsis and septic shock were redefined in 2016 (Sepsis-3), defining sepsis as “life-threatening organ dysfunction caused by a dysregulated host response to infection” [6]. An increase in the sequential organ failure assessment score (SOFA) of ≥ 2 was associated with an in-hospital mortality greater than 10% and proposed for clinical operationalization [6].

As a bedside screening tool for sepsis-related mortality for non-intensive care settings, the quickSOFA (qSOFA) score was introduced to rapidly identify patients with high likelihood of poor sepsis-related outcome, in particular with high risk of sepsis-related mortality [3, 6, 7]. Although, the effort of establishing a new definition and clinical criteria by analyzing large datasets of patient records was widely acknowledged, the validity of these criteria for clinical practice has been questioned [8,9,10,11,12] and their applicability in different settings and subgroups remains unclear [13, 14]. This might particularly apply to specific subgroups such as burn patients [15], surgical patients [16], and hematological cancer patients [14, 17].

Cancer patients and particularly patients with hematological malignancies commonly suffer from immunodeficiency resulting from the underlying disease and/or from immunosuppressive therapy and are therefore at high risk of infection resulting in sepsis [18, 19]. Importantly, population and register-based studies indicate that up to one in five patients admitted to intensive care units (ICU) suffer from underlying malignancies and that sepsis is the leading cause for ICU admission in patients with hematologic malignancies [18, 20,21,22], although the validity of sepsis criteria in hematological cancer patients remains unclear.

We therefore analyzed the discriminative accuracy of the Sepsis-3 scores and the systemic inflammatory response syndrome (SIRS) criteria for sepsis and in-hospital mortality in a multicenter cohort of hematological cancer patients. The primary aim of this study was to assess the impact of different sepsis scores in discriminating sepsis and in-hospital mortality and to validate their use in these patients admitted to ICU or a regular hematology ward.

Methods

This was a retrospective, multicenter study of 450 adult patients (≥ 18 years) with hematological cancer admitted between 2013 and 2018 to a regular ward specialized in hematology or the ICU at one of the following three large academic tertiary care hospitals in Germany (University Hospital of Cologne, University Hospital Magdeburg, University Hospital Essen). The patients were admitted for application of therapy or for treatment of complications consecutive to their underlying disease or therapy such as fever and suspected infection. For the ICU patients included in the study, patients with hematological cancer consecutively admitted to an ICU at one of the three centers were included in the study irrespective of the suspected diagnosis at transfer to the ICU. Patients on the normal hematology ward of one of the three centers were included, when an infection was suspected, suspected infection being defined according to current Surviving Sepsis Campaign guidelines and recommendations (administration of antibiotics excluding prophylactic treatment and sampling of body fluid for culturing) [2, 6]. For patients with multiple hospitalizations, we only included their first admission.

The following clinical data were taken from patients’ medical records documented at admission to ICU or when an infection was suspected: patient age, gender, underlying disease and current status of the malignancy (newly diagnosed, partial or complete remission, stable disease, progressive disease), data on hematopoietic stem cell transplant (HSCT) and source of infection, physiological and laboratory data to determine the sepsis scores such as the SIRS criteria [23], the SOFA and the qSOFA score [6], severity indicators such as the Acute Physiology, Age, Chronic Health Evaluation II (APACHE II) score [24], Glasgow Coma Scale (GCS) [25], acute kidney failure and graft-versus-host disease, measures of outcome such as in-hospital mortality, ICU mortality, hospital length of stay and transfer to ICU. Additionally, lactate levels, administration of vasopressors, and mean arterial pressure (MAP) were collected to determine septic shock according to the Sepsis-3 definition from 2016 [3].

According to current Surviving Sepsis Campaign guidelines and recommendations [2, 6], all patients who received antibiotics (excluding prophylactic treatment) and underwent sampling of body fluids for culturing were defined as patients with suspected infection [6], and patients with “life-threatening organ dysfunction caused by a dysregulated host response to infection” were defined as patients with sepsis [6]. For each scoring system, the standard threshold of ≥ 2 points was applied for a positive SIRS, SOFA, and qSOFA score [6, 26]. The baseline SOFA score to differentiate between preexisting and acute changes in organ dysfunction in order to calculate the acute increase in SOFA was assessed with the help of prior laboratory values, the impact of the underlying disease and its therapies. In case of missing data to determine the SIRS, SOFA, and qSOFA score, the patients were excluded from further analysis of diagnostic accuracy. Patients with sepsis and a vasopressor requirement to maintain a MAP of ≥ 65 mmHg and a serum lactate level > 2 mmol/L in the absence of hypovolemia were diagnosed with septic shock according to the Sepsis-3 definition [6].

Statistical analysis

Dichotomous variables were presented as absolute numbers and percentages and continuous variables as medians and interquartile ranges (IQR). The Mann-Whitney U test was used to compare continuous variables between the two groups “patients with sepsis” and “patients without sepsis” after testing for normal distribution using the Shapiro-Wilk test. Categorical binary variables were compared by the chi-squared test. Two-sided p values < 0.05 were required for statistical significance.

By applying the definition of sepsis as “life-threatening organ dysfunction caused by a dysregulated host response to infection” [6] as reference, the scores were evaluated. In order to assess the accuracy of the three investigated sepsis criteria, SIRS, SOFA, and qSOFA, regarding sepsis and in-hospital mortality, sensitivity, specificity, predictive values, likelihood ratios (LR), and the Youden’s Index were calculated according to standard formulas as previously described, and the area under receiver operating characteristic (AUROC) curves were determined [27,28,29,30]. Pairwise comparison of receiver operating characteristic (ROC) curves was conducted in order to test statistical significance. To further assess the association between sepsis scoring systems and mortality, we also determined the in-hospital mortality for patients with positive and negative sepsis scores and calculated the risk ratio (RR) for each sepsis criterion. All statistical analyses were two-tailed and calculated with Stata/SE 13.

Results

Study population

A total of 450 patients with hematological malignancies were included from ICUs or regular hematology wards of three German tertiary academic hospitals. The median age was 58 years (IQR, 46–66 years), 61% were male, 44% had leukemia, 31% had lymphoma, 18% suffered from myeloma, and 4% from myelodysplastic syndrome. Nearly half the population was in partial or complete remission, about one in four patients were newly diagnosed and about one in five patients suffered from disease progression at admission. About 40% had received a HSCT, of which half the patients an allogeneic (49%; Table 1).

Table 1 Demographics and clinical characteristics of the cohort
Full size table

One hundred eighty patients (40%) presented with sepsis of which 101 (56%) were treated on ICU and an additional 27 patients (15%) were later transferred to an ICU (Table 2). The two subgroups of patients (sepsis vs. no sepsis) had a similar distribution regarding age and gender but differed significantly in platelet count, creatinine and bilirubin. Patients with sepsis were more likely to present with acute kidney failure, lower MAP, and higher heartrates at admission. The measured body temperature was not significantly different between the two groups. The most common source of infection in patients with sepsis was the respiratory tract (32%) followed by central line-associated infections and the gastrointestinal tract both in 9% of the patients (Tables 1 and 2).

Table 2 Outcomes and severity indicators
Full size table

Performance of SIRS, SOFA and qSOFA criteria

Within the group of patients with sepsis, 86% (95% confidence interval [95% CI], 80–90%) of the patients presented with a ≥ 2 positive SIRS criteria, 64% (95% CI, 57–71%) of the patients with a SOFA score ≥ 2, 42% (95% CI, 34–49%) of the patients with a qSOFA score ≥ 2, and 20 patients (12%) fulfilled the Sepsis-3 criteria of septic shock. Thus, the false-negative rate for the SIRS criteria was 14%, for the SOFA score 36%, and for the qSOFA score 59%. Within the group of patients not suffering from sepsis, 76% of the patients presented with ≥ 2 SIRS criteria, 26% with a SOFA score ≥ 2, and 9% with a qSOFA score ≥ 2, representing the number of false positives for each score, respectively (Table 2, 2×2 contingency tables and flow charts for each score are provided in Additional files 1 and 2).

Prognostic performances of the three investigated sepsis criteria for the total cohort are reported in Table 3. The Youden’s Index as an indicator for the performance of dichotomous diagnostic tests was 0.093 for SIRS, 0.380 for SOFA, and 0.325 for qSOFA. The AUROC values for discrimination of sepsis were 0.57 (95% confidence interval [95% CI], 0.53–0.61; p < 0.001) for SIRS, 0.69 (95% CI, 0.64–0.73; p < 0.001) for SOFA, and 0.67 (95% CI, 0.62–0.71; p < 0.001) for qSOFA (Fig. 1). When comparing the ROC curves of SOFA with qSOFA, the areas are not significantly different (p value = 0.309); however, comparing the ROC curves of SOFA or qSOFA with SIRS, respectively, the areas are significantly different (both p values < 0.001).

Table 3 Performance of SIRS, SOFA and qSOFA scores
Full size table
Fig. 1
figure 1

Receiver operating characteristic (ROC) curves (unadjusted) for a sepsis and b in-hospital mortality. The areas under the ROC curves (AUROCs) for panel a are SIRS, 0.57 (95% confidence interval [CI], 0.53–0.61), p < 0.001; SOFA, 0.69 (95% CI, 0.64–0.73), p < 0.001; qSOFA, 0.67 (95% CI, 0.62–0.71), p < 0.001, and for panel b SIRS, 0.49 (95% CI, 0.44–0.54), p < 0.001; SOFA, 0.74 (95% CI, 0.69–0.79), p < 0.001; qSOFA, 0.65 (95% CI, 0.60–0.71), p < 0.001. AUC area under the curve, SIRS systemic inflammatory response syndrome, SOFA sequential (sepsis-related) organ failure assessment, qSOFA quick SOFA

Full size image

In-hospital mortality

For the prediction of in-hospital mortality, SIRS had a sensitivity of 79% (95% CI, 71–86%) and a specificity of 20% (95% CI, 16–24%), SOFA had a sensitivity of 76% (95% CI, 67–83%) and a specificity of 70% (95% CI, 64–74%), and for qSOFA, the sensitivity was 45% (95% CI, 36–55%) and the specificity was 86% (95% CI, 81–89%). The positive likelihood ratio was 3.1 (95% CI, 2.2–4.4) for qSOFA and 2.5 (95% CI, 2.1–3.1) for SOFA. The negative predictive value was 83% for qSOFA and 90% for SOFA, respectively. Figure 1 and Table 3 show the AUROCs and values for the prediction of in-hospital mortality by the three sepsis scores. The highest AUROCs were determined for SOFA (0.74 [95% CI, 0.69–0.79]; p < 0.001) and qSOFA score (0.65 [95% CI, 0.60–0.71]; p < 0.001) compared to SIRS (0.49 [95% CI, 0.44–0.54]; p < 0.001). When comparing the ROC curves of SOFA with qSOFA regarding prediction of in-hospital mortality, there was a significant difference (p value = 0.005).

In the whole cohort, including ICU and non-ICU patients, in-hospital mortality was 24% with an ICU mortality of 44%. Of all patients with sepsis, 40% died compared to 14% of all patients without sepsis (p < 0.001; Table 2).

Focusing on patients with sepsis, the mortality rate in patients with positive and negative SIRS scores was 39% vs. 40% (p = 0.899), respectively. For patients with a qSOFA score ≥ 2, mortality was 49% compared to 33% for those with a qSOFA score < 2 (p = 0.056), and for the SOFA score 56% vs. 11% (p < 0.001), respectively. Thus, the mortality rate for sepsis patients with falsely negative scores was lowest (11%) for the SOFA score. Patients with sepsis with SOFA scores ≥ 2 were about five times more likely to die (risk ratio 5.06 [95% CI, 2.47–10.37]) compared to patients with SOFA scores < 2. Regarding qSOFA, patients with a qSOFA score ≥ 2 were about 1.5 times more likely to die compared to patients with a score < 2 (risk ratio 1.44 [95% CI, 0.99–2.09]). In contrast, there was little difference in mortality in patients with ≥ 2 SIRS scores compared to patients with SIRS scores < 2 (risk ratio 0.97 [95% CI, 0.57–1.63]). In the group of patients with sepsis treated on the ICU, patients with ≥ 2 SOFA scores were more than twice as likely to die as patients with SOFA scores < 2 (risk ratio 2.26 [95% CI, 0.83–6.13]), being the only risk ratio with a significance level < 0.05 in this group (Table 3).

Discussion

As population-based studies show, about one in five patients in the ICU has cancer and the proportion of ICU patients suffering from cancer is expected to increase over the next decades [18, 21, 31]. Patients with hematological malignancies are at particular risk of requiring ICU treatment as therapy- or disease-related immunodeficiency is common in hematological cancer patients frequently leading to infection and sepsis [20, 31]. At the same time, patients with neutropenia and immunodeficiency frequently do not exhibit fever, pus, and other symptoms associated with the infection; thus, especially in patients with hematological malignancies and sepsis, it can be difficult to determine the site of infection (37%). In addition, many patients suffer from polymicrobial infection originating from mucosal barrier injury, which is difficult to diagnose with conventional methods and criteria.

Although the accurate definition and early diagnosis of sepsis indisputably improve patient survival, there is a considerable debate about new definitions and criteria for sepsis [2,3,4,5,6,7,8,9,10,11,12,13, 15,16,17]. Moreover, the discriminatory capacity of different criteria and scores, such as SIRS, SOFA, and qSOFA, for identifying sepsis and predicting in-hospital mortality in hematological cancer patients remains unclear.

The focus of our study was the evaluation of the diagnostic validity of these criteria for the diagnosis of sepsis and the prediction of high risk of in-hospital mortality in a multicentric population of hematological cancer patients. We studied the criteria in analogy to the Sepsis-III study focusing on the criteria and their (bedside) validity, rather than on changes of inflammation markers, which has been studied previously by several groups [32, 33].

In our study, we found a higher non-ICU mortality of 11% compared to the mortality of 3% reported in the original Sepsis-3 study from Seymour and colleagues in a general population [7]. ICU mortality of hematological cancer patients in our study was 44% and even increased up to 50% if patients suffered from sepsis, which is within the reported range for hematological cancer ICU patients of 34–68% [34] and slightly higher than reported for the largest published hematological cancer cohort by Azoulay et al. (39%) [20]. Costa et al. studied a single-center cohort of 450 cancer patients admitted to the ICU, including about 19% of patients with hematological malignancies, and reported an overall mortality of 39% [17]. Soares and colleagues studied a very large multicenter retrospective cohort of 9946 cancer patients, including about 10% (n = 990) of patients with hematologic malignancies, reporting ICU and in-hospital mortality rates of 16% and 25%, respectively [21]. These findings are well in line with previous reports, indicating a higher mortality of hematological cancer ICU patients compared to studies including patients with solid tumors, which often include a higher proportion of patients with post-surgical surveillance [20, 21, 31]. Neutropenia, which is common in hematological cancer patients, but less in patients with solid tumors, might be an important factor. Although earlier data were conflicting, neutropenia was independently associated with an increased mortality in a recent meta-analysis including 4149 (55%) patients with hematological cancer [35], which might contribute to the high mortality we observed.

Regarding the validity and performance of different sepsis scores in our cohort, we found that more than one in six patients (17%) with sepsis did not meet two or more SIRS criteria. This proportion of “SIRS-negative” sepsis patients in our study was slightly higher than previously reported by Kaukonen et al. (12%) in a population-based ICU cohort [36]. Costa et al. reported exclusively on cancer patients with a suspected infection admitted to the ICU and observed about one fifth of patients with sepsis who met less than two SIRS criteria (23%) [17]. However, both studies did not include information on the respective criteria in patients without sepsis [17, 36]. Interestingly, the majority of patients in our study without sepsis met two or more SIRS criteria (76%). In those false-positive patients, almost all patients (90%) had abnormal white blood cell counts (< 4 × 109/L or > 12 × 109/L), which were responsible for most positive SIRS scores in patients without sepsis. This observation of a high proportion of “SIRS-positive” patients without sepsis was similarly reported by several authors in different populations including ICU and non-ICU patients [13]. Therefore SIRS criteria are of limited use in diagnosing and screening for sepsis in hematological cancer patients as abnormal white blood cell counts are a common disease- or therapy-related observation in these populations.

Both scores, SOFA and qSOFA, had a lower proportion of false-positive patients; however, the proportion of patients with sepsis, who did not meet two or more of the criteria, also increased. Thirty-six percent of the patients with sepsis did not meet a positive SOFA score (two or more) and more than half of the patients with sepsis (59%) had less than two of three qSOFA criteria (Table 3). The qSOFA was mainly negative due to a GCS of 15 (in 92% of the patients) and a systolic blood pressure above 100 mmHg (73% of the patients).

Yet, the mortality rate for the 36% of sepsis patients with false-negative SOFA scores was low (11%) compared to the respective mortality rates for SIRS (40%) and qSOFA (33%).

Overall, we found that SOFA was more accurate than qSOFA or SIRS in discriminating hematological cancer patients with sepsis from patients without sepsis (SOFA, AUROC 0.69 [95% CI, 0.64–0.73], p < 0.001; qSOFA, AUROC 0.67 [95% CI, 0.62–0.71], p < 0.001; SIRS, AUROC 0.57 [95% CI, 0.53–0.61], p < 0.001; Table 3). Focusing on ICU patients, the sensitivity in hematological cancer patients was highest for SOFA (88%) and the false-negative rate for ICU patients with sepsis was lower (12%) compared to the other two scoring systems.

Moreover, in-hospital mortality in hematological cancer patients with sepsis was about fivefold higher in “SOFA-positive” patients compared to patients with a SOFA score below two (risk ratio 5.06 [95% CI, 2.47–10.37]; p < 0.001). The discrimination of in-hospital mortality using SOFA (AUROC 0.74 [95% CI, 0.69–0.79]; p < 0.001) was significantly higher compared with SIRS criteria (AUROC 0.49 [95% CI, 0.44–0.54]; p < 0.001), for which the mortality rate was 40% in “SIRS-negative” patients and about 39% in “SIRS-positive” patients (Table 3). Similarly, several authors previously suggested SOFA as predictor of ICU mortality in general ICU populations with suspected infection, and the relationship between SOFA and mortality has been investigated in a variety of subgroups [7, 17, 37,38,39,40].

Importantly, there are significant limitations to the clinical use of the SOFA score, as it requires laboratory measurements and is time-consuming to calculate. However, our results in a hematological cancer population including non-ICU patients suggest that SIRS criteria are not accurate in hematological cancer patients and that also qSOFA, which has been proposed as an effective way of raising suspicion of sepsis on the regular hematology ward might not be applicable in hematological cancer patients, as the number of qSOFA-negative sepsis patients with high mortality was considerable in our study.

Strengths of our study are the large number of hematological cancer patients and the multicentric design including patients from three tertiary academic hospitals. Furthermore, our cohort included a broad range of hematological cancer patients regarding disease status, HSCT, and organ dysfunction. The main limitation of the study was the retrospective study design with data acquisition possibly being prone to bias. In addition, some of the data needed to calculate the SOFA score could not be collected for non-ICU patients. Furthermore, despite the fact that we found a strong association of several characteristics with mortality including indicators of organ failure, we could not determine the impact of each single factor in a multivariate analysis, as the heterogeneity of our study population was too high regarding the sample size. A multivariate analysis would require a much higher number of patients to provide reproducible results and was not a focus of our study.

Conclusions

In conclusion, we found that in our cohort of patients with hematological malignancies, an increase in SOFA score of ≥ 2 had superior prognostic accuracy for discriminating sepsis and estimating in-hospital mortality compared with qSOFA or SIRS criteria. Especially on intensive care, we observed limited validity of SIRS and qSOFA in identifying hematological patients with sepsis and at high risk of death. The significant proportion of sepsis patients with a negative score, irrespective of the scoring system used, underlines the need for a clinical diagnosis of sepsis particularly in hematological cancer patients.