Introduction

Infection remains the leading cause of paediatric mortality worldwide [1]. In most high-income countries, access to care, vaccination campaigns and improvement in intensive care have drastically decreased deaths from infection. Similar to adults, recent paediatric studies have shown an increase in the incidence of invasive infection and septic shock and a relative decrease in mortality [2–4].

Recently, sepsis and septic shock definitions in adults were revised [4–7]. The aim of the Sepsis-3 definition was to help the clinician in the detection of septic shock patients and to treat them according to their risk of death. The sepsis-related organ failure assessment (SOFA) [8] has a prominent place in this definition, but is neither adapted nor validated in paediatric patients. The clinical course of septic shock is affected by the age and immune state of the patient, the virulence of the pathogen, and the haemodynamic adaptation to circulatory failure. Paediatric logistic organ dysfunction (PELOD) is a paediatric organ dysfunction score validated in the severity classification of patients with sepsis [9]. However, similar to the SOFA score, it does not impact patient management at the bedside.

The Surviving Sepsis Campaign [10] and the American College of Critical Care Medicine (ACCM) guidelines for hemodynamic support in neonates and children [11] remain the most recognized standards of care. Specific therapies such as activated protein C [12, 13] or early goal-directed therapy [14, 15] have not been associated with a consistent decrease in mortality. The term septic shock encompasses various aetiologies and immune states for which specific interventions may have variable results [16]. However, the subset of patients who are unresponsive to standard resuscitation are often labelled as having ‘refractory septic shock’ (RSS). RSS is typified by circulatory failure due to septic cardiomyopathy [17–19] with or without vasoplegia [19]. Importantly, effective short-term support of the circulation with newer vasoactive agents or extracorporeal life support (ECLS) [20] means that RSS is potentially reversible [21, 22]. To maximise the impact of these rescue therapies, a robust tool for early identification of RSS is required,

The Infection, Systemic Inflammation and Sepsis section of the European Society of Paediatric and Neonatal Intensive Care (ESPNIC) designated a taskforce to define paediatric RSS [23]. We performed a study linking a two-round Delphi survey with the development of septic shock scores. These scores aimed at an early indentification of RSS patients with a high risk of mortality at any moment during clinical care. The scores were subsequently validated in a multicentre international retrospective cohort.

Materials and methods

Definition drafting

The first part of the Delphi study was a clinical case-based questionnaire. Two members (L.M. and P.T.) of the Infection, Systemic Inflammation and Sepsis section of ESPNIC were assigned to create four clinical cases of septic shock patients with varying levels of shock and organ dysfunction.

The characteristics of the clinical cases were set following a review of the literature on septic shock and a case analysis of septic shock patients, and covered the whole clinical spectrum of disease severity. Five members from the taskforce (G.M., S.N., M.P., M.K., N.J.G.J.) reviewed the cases for consistency and objectivity. All clinical cases were composed of vignettes with specific clinical and biological parameters (later described as criteria and corresponding cut-offs), encompassing the evolution of sepsis. For each vignette, responders were asked to grade the occurrence of RSS (0 = no RSS to 10 = yes) (see supplementary file 1). The questionnaire was then sent to all ESPNIC members via an Internet survey supplier (SurveyMonkey Inc., Palo Alto, CA, USA).

Criteria extracted from the clinical vignettes were ranged according to the distribution of their strength of association with RSS. Criteria graded A (first quartile) and B (second quartile) were selected and compiled in four draft definitions constituted by a minimal definition alone or with up to three additional organ failures. Preliminary definitions were then tested in the second part of the Delphi study. Participants of this second part had access to the results of the first study. They were asked to choose one of the four definitions and to re-score the definition cut-off values issued from the first part’s results. Based on this second round, criteria graded A or B were incorporated into the selected RSS draft definition. (See supplementary file 2 for additional information on methodology).

Development population application and definition adjustment

The RSS draft definition was first tested in a development population. All patients less than 18 years old admitted to the Paediatric Intensive Care Unit (PICU) at Centre A for septic shock, according to consensus definition [24], and requiring vasopressor or inotrope therapy between January 1, 2010 and December 31, 2013 were analysed. Exclusion criteria were: (1) the presence of limitations of life support at septic shock onset; (2) postmenstrual corrected age <37 weeks; and (3) perinatal sepsis defined as sepsis occurring in the first 7 days of life.

Protocols for clinical care of septic shock patients did not change during the 3-year study period. Patients were retrospectively identified using hospital records and data collected. The French Intensive Care Society ethics committee approved this study (CE SRLF 14-38). Clinical variables included in the definition or used to describe the populations or outcomes, and occurring before the occurrence of the composite endpoint PICU mortality and/or ECLS, were recorded (see supplementary file 2). Draft definition criteria identified in the Delphi study were evaluated against two outcomes: PICU mortality, and the composite endpoint PICU mortality and/or ECLS. Multivariable Cox regression was performed to study the association between the variables used in the definition and these two outcomes, to define two RSS scores. An easy to calculate score was named “bedside RSS score” and a more complex one was named “computed RSS score”.

RSS definition assessment in a validation population

Both bedside and computed RSS scores were tested to evaluate their association with mortality and the composite endpoint mortality/need for ECLS. This was done on a validation population consisting of a cohort of PICU-admitted patients, retrospectively enrolled in 6 PICUs in four countries (Australia, France, Netherlands and United Kingdom; Table 1). Inclusion and exclusion criteria were identical in the development and validation population. Data were collected retrospectively (C.W., E.J., L.J.S., L.M., M.K., M.P., N.J.G.J., S.N., S.R., S.R.). L.M. independently reviewed the consistency of the collected data. All six centres obtained local ethical approval for the retrospective analysis of the patient data and waiver of informed consent from legal representatives.

Table 1 Hospitals and patients
Full size table

Statistical analysis

Continuous variables were tested for normality with Kolmogorov–Smirnov test and compared with Student’s t test or Mann–Whitney test, as appropriate. Non-continuous variables were tested with Chi-squared test or Fisher’s exact test, as appropriate. Data were described as frequencies and percentages, means and standard deviations or median and interquartile range, as appropriate. Multivariable Cox’s regression with backward-stepwise method was performed having as outcomes mortality or the composite endpoint mortality/need for ECLS. Covariates inserted in the models were the variables identified through the Delphi process and the patients’ age [3]. Adjusted hazard ratios were used to weight covariates and included in the scores. The model goodness of fit was evaluated with Omnibus test. Receiver operating characteristics (ROC) analysis was then performed with the validation population for both scores and areas under the curves (AUC) were calculated to assess for the discriminative power of these scores. The best thresholds for these scores were obtained with the calculation of sensitivity, specificity, positive and negative predictive values and the Youden’s index (sensitivity + specificity – 1). The DeLong test was used to compare the AUC of both scores [25]. Survival of RSS patients according to both scores have been evaluated by Kaplan–Meier curves and these latter have been tested using Logrank test. A p value of less than 0.05 was considered statistically significant. Statistical analyses were performed with SPSS for Macintosh, v.22.0 (SPSS, Chicago, IL, USA) and GraphPad Prism®, v. 5.0a (GraphPad Software, La Jolla, CA, USA) softwares.

Results

Draft of the definition

From April 9 to July 2 2014, 114/170 (67 %) physicians members of the ESPNIC representing 27 countries and 75 PICUs answered the first round of the Delphi study. The physicians PICU experience was less than 10 years for 35.6 %, 10–20 years for 44.1 % and more than 20 years for 20.3 %. Results from this first round are presented in Table 2. Items graded A (first quartile of association with diagnosis of RSS) or B (second quartile) were selected for the second round. They were condensed into a minimal definition (lactic acidosis with vaso-inotrope dependency and myocardial dysfunction) with additional organ failures graded A or B [hepatic insufficiency, severe acute respiratory distress syndrome (ARDS) with the Berlin definition [26] or failure to achieve effective continuous renal replacement therapy (CRRT)]. The vaso-inotrope dependency was assessed with the use of the vaso-inotrope score [VIS = (epinephrine + norepinephrine in mcg/kg min) × 100 + (dobutamine + dopamine in mcg/kg min) + (vasopressin in mcg/kg min) × 10,000 + (milrinone in mcg/kg min) × 20] [27]. In the second round of the Delphi study, from September 23 to November 21 2014, 61 physicians answered the survey and 37 (60.6 %) selected the minimal definition without additional organ failure. They validated six items determined in the first round as important with a grade A or B for the diagnosis of RSS. The draft definition for RSS was the association of: (1) blood lactate >8 mmol/L or a 1 mmol/L lactate increase after 6 h of resuscitation, (2) vaso-inotrope dependency (VIS >200 mcg/kg min), and (3) myocardial dysfunction, defined as the occurrence of a resuscitation-responsive cardiac arrest in PICU or cardiac ultrasound findings with left ventricle ejection fraction (LVEF) <25 % or a cardiac index <2.2 L/min m2.

Table 2 Results of the first round of the Delphi study: association of clinical criteria with refractory septic shock diagnosis
Full size table

Development population application and definition adjustment

During a 4-year period between January 1, 2010 and December 31, 2013, 78 patients were admitted at the Centre A PICU for septic shock requiring inotropes or vasopressors. Eight were excluded because of limitations of life support on admission. Of the remaining 70 patients, 21 (30 %) died during PICU stay, of whom 11 (15.7 %) were classified in RSS by the draft definition with a mortality rate of 100 % (sensitivity = 52.4 %, specificity = 100 %) and a median delay from shock onset to death of 19 (12–72) h. No comorbidity was found in 47.1 % of the patients while 22 patients (31.4 %) were immunocompromised.

In the multivariable analysis, the model goodness of fit was satisfying for each step (p = 0.577 and p = 0.717 at the first and second steps, respectively). Data from the last step were selected and used to create septic shock scores. Mortality or the need for ECLS was independently associated with the worst VIS [HR = 1.001 (95 % CI: 1.0009–1.0011) p = 0.01], worst arterial lactate [HR = 1.1 (95 % CI: 1.01–1.23), p = 0.032] and presence of myocardial dysfunction [HR = 18, (95 % CI: 3.4–95.4), p = 0.001]. Lactates and VIS (ρ = 0.34; p = 0.008) as well as LVEF and lactates (ρ = –0.67; p < 0.001) were correlated. Age was not significantly associated with the outcome [HR = 0.99, (95 % CI: 0.98–1.01), p = 0.50]. Following this, two septic shock scores (SSS) were constituted:

  • Computed SSS calculated as follows, (cSSS) = 1.001VIS in mcg/kg min + 1.1arterial lactate in mmol/L + 18 (in the presence of myocardial dysfunction).

  • Bedside SSS (bSSS) based on 5 points with coefficient ranked and rounded to have a user-friendly score:

  • VIS > 200 mcg/kg min = 1 point

  • Arterial lactate >8 mmol/L or its increase of 1 mmol/L after 6 h of care = 1 point

  • Myocardial dysfunction as defined above = 3 points.

Validation population

The validation population consisted of 456 patients admitted for septic shock requiring inotropes or vasopressors, with a mortality rate of 17.8 % (Fig. 1). Thirty-two of these patients had limitations of life support on admission and were excluded from analysis. The mortality for the remaining 424 patients was 11.6 %. The main characteristics of both development and validation populations are compared in Table 3. The two studied populations differed significantly in terms of prognostic factors and outcomes. The paediatric index of mortality score (PIM2) [17 (11–21) vs. 7.6 (3–15), p < 0.001], peak blood lactates [4.7 (2.3–7.8) vs. 2.6 (1.6–5.2), p < 0.01] and mortality (30 vs. 11.6 %, p < 0.001) were significantly higher in the development cohort. No comorbidity was found in 52 % of the patients while 60 patients (16.3 %) were immunocompromised. The origin of infection was nosocomial in 27.6 % and community-acquired in 72.4 % of the patients in the validation cohort. A microbiological diagnosis was positive for 317 patients (74.8 %; bacterial in 83.9 %, viral in 12.9 %, fungal in 2.5 % and parasitic in 0.6 %). The most prevalent pathogens were Neisseria meningitidis in 70 patients (22.4 %), Group-A Streptococcus in 38 patients (12 %), Streptococcus pneumoniae in 16 patients (5 %) and Staphylococcus aureus in 29 patients (9.1 %). The pathogen was a Gram-negative bacillus in 73 patients (23 %), including E. coli for 34 patients (10.7 %). Patients intubated for septic shock reached 87 %, and severe ARDS was present in 27.8 % of all patients.

Fig. 1
figure 1

Flow chart of the validation cohort

Full size image
Table 3 Patient characteristics in the development and validation populations
Full size table

Both scores were calculated for all patients and performance analysed with ROC curves (Fig. 2). The discriminative power for association with mortality or need for ECLS was calculated for computed and bedside scores, with AUC = 0.956 (95 % CI: 0.93–0.97) and 0.920 (95 % CI: 0.89–0.94), respectively. The computed score had a higher AUC as compared to the bedside score (p = 0.0092). The performances of both scores for association with death or use of ECLS are presented in Table 4. The bedside SSS with a cut-off at 2 was associated with a positive predictive value of 60.3 % and a negative predictive value of 97.8 % (positive likelihood ratio = 11.6, Youden’s index = 0.765). The computed SSS with a cut-off at 3.5 was associated with a positive predictive value of 55 % and a negative predictive value of 98.5 % (positive likelihood ratio = 9.4, Youden’s index = 0.802). Mortality rate were compared for RSS populations defined by the bedside and the computed scores, with, respectively, 41 deaths among 68 RSS patients (mortality 60.3 %) versus 8 deaths among 356 non-RSS patients (mortality 2.2 %) for the bedside score and 44 deaths among 80 RSS patients (mortality 55 %) versus 5 deaths among 344 non-RSS patients (mortality 1.5 %). Survival curves for each score are presented in Fig. 3 and mortality is significantly higher in RSS patients for both scores (p < 0.001 and p < 0.001).

Fig. 2
figure 2

Receiver operating characteristics curves for the computed septic shock score (cSSS) and the bedside septic shock score (bSSS). Area under the ROC curve in the validation population was 0.956 (95 % CI = 0.93–0.97, p < 0.01) for the cSSS and 0.920 (95 % CI = 0.89–0.94, p < 0.01) for the bSSS

Full size image
Table 4 Predictive performance of septic shock scores and their best threshold values
Full size table
Fig. 3
figure 3

Survival curve of refractory septic shock defined by a a bedside septic shock score ≥2.5, and b a computed septic shock score ≥3.5

Full size image

Discussion

The septic shock scores (bedside and computed) are the first scores that aim at diagnosing refractory septic shock in paediatric patients admitted in PICU for septic shock. The scores were effective to identify the most severely ill patients. A bedside score ≥2 or computed score ≥3.5 defined RSS with a mortality higher than 55 %. Although statistically different, the diagnostic accuracy of both scores seems clinically equivalent with an absolute difference of 0.035 in the AUC. The clinical utility for each score is different. The bSSS is a bedside tool potentially useful for stratifying the severity and assessing the imminent risk of death, helping clinicians to implement rescue therapies. Its simplicity to use may counter-balance the small loss in accuracy. The cSSS is a potentially powerful and discriminating epidemiological tool and its calculation needs a computer or a smartphone application.

Defining a study population has implications on trials outcomes and may explain contradictory outcomes from historical large-scale randomized trials [12, 13, 28]. Recently, the sepsis-3 adult definition has refined the identification of septic shock patients [7]. However, although focusing exclusively on adults, the main advance of the sepsis-3 definition is the stratification of patients into two categories: sepsis and septic shock. In adults, the SOFA score assesses organ failures after sepsis with good correlation to mortality [8]. This adult score is not adapted for paediatric patients. The paediatric index of mortality (PIM2) [29] and the PELOD-2 [30] score are paediatric scores that can be used to compare population of patients but are not specific for sepsis. In our study, the two scores were built with arterial lactate, vaso-inotrope score and septic cardiomyopathy and were associated with poor outcome. Each of those criteria was independently associated with mortality or the need for ECLS. Blood lactate is a widely used biomarker in septic shock patients. The highest value during the first 24 h of PICU was associated with mortality in paediatric septic shock patients [31]. Dynamic values, such as lactime [32] or lactate clearance [33], are better predictors of mortality in adult patients, but they are not used routinely. Interestingly, the best predictor of survival to ECLS therapy in children with septic shock was a low pre-ECLS arterial lactate value [22].

Use of vasopressors or inotropes to treat fluid-unresponsive shock is still the first line of treatment [10, 11]. The VIS is a score used to assess the responsiveness and dependency to vasopressor in cardiac ICUs [27], with a VIS >200 being associated with mortality or need for ECLS in post-operative paediatric cardiac patients with low cardiac output syndrome [34]. In our study, norepinephrine was used in 77.2 % of the patients and an inotrope (dobutamine, dopamine, epinephrine or milrinone) added in 80.2 %. The cut-off chosen in the definition by the two-round survey is higher than the VIS scores found in the literature in patients under ECLS therapy for other indications [22, 35, 36] or alternative vasopressors [37], but similar to calculated inotrope scores in adult patients needing ECLS support for septic shock [21].

The presence of a septic cardiomyopathy was independently associated with increased mortality or need for ECLS in the development population [OR = 18, (95 % CI: 3.4–95.4)]. Septic cardiomyopathy is the component carrying the highest weight in the septic shock scores. Although the pathogenesis of septic cardiomyopathy is multifactorial, it is known to be reversible [36, 37, 38]. LVEF has proven to be sufficient for the diagnosis of low cardiac output with a LVEF <40 % [39]. Diastolic dysfunction and right ventricle dysfunction in sepsis are less studied but their role in septic cardiomyopathy seems to precede systolic left ventricular dysfunction [39]. Cardiac arrest is the ultimate evolution of septic shock. In our study, in both cohorts, 59.7 % of the deceased patients had a resuscitated cardiac arrest in the course of the septic shock. Patients in cardiac arrest due to septic shock can benefit from ECLS therapy with an overall survival of 75 % in one series [22], compared to 32.7 % in our cohort. Improved assessment of septic cardiomyopathy before cardiac arrest occurs is mandatory. Regular evaluation of cardiac output and function can be based on invasive and non-invasive criteria. Non-invasive quantification relies mostly on continuous oesophageal or supra-sternal Doppler or cardiac ultrasound. Importantly, it is well recognized that children with septic shock without apparent need for inotropes can subsequently develop septic cardiomyopathy and low cardiac output [19]. This haemodynamic pattern can be explained by the unmasking of septic cardiomyopathy by restoration of vasomotor tone in resuscitated shock after norepinephrine infusions, as well as over-enthusiastic fluid resuscitation [17].

This study has some limitations. First, the data collection was retrospective and thus is at risk of missing data and information bias. Second, data were collected during the whole septic shock care period with selection of a unique worst value for each item. Thus, the maximal values could be at varying time points in the clinical course. Calculation of the score was based on worst values for each criterion that may not be synchronous. This could have increased the estimated risk of death for patients in the validation population as well as overestimating the AUCROC. However, most patients died shortly after admission (median time to death = 1 day) and the highest values were mostly collected within the first 24 h after admission. Similarly, a delay to death of less than 24 h was also shown in more than 13,000 paediatric sepsis admissions in the North Thames, UK, region [40]. Other scores used similar methodology during their creation, including SOFA and PELOD [8, 30]. Evaluating the kinetics of both scores as well as its composing criteria in the course of the disease is important and needs prospective evaluation. Third, there were significant baseline differences in the characteristics of the two populations (development and validation) and between centres included in the validation cohort. These may be explained by differences in patients’ recruitment and severity. Some of them were known prognostic factors in septic shock such as PIM2 scores and immunosuppression or were part of either the definition (blood lactates) or the outcome (mortality), outlining the patients’ recruitment and severity difference between both cohorts. The high mortality rate in the development cohort ensured the high power of this study to select the criteria associated with refractory septic shock. Meanwhile, the scores have an excellent discriminative value in the validation cohort. This reinforces the quality of the scores in various patients. Finally, this definition is based on a draft definition that has been developed using fictitious clinical cases and an international PICU physician opinion survey. The risk of opinion bias has been adjusted with the modification of this definition and the constitution of the septic shock scores tailored to an actual patient population. This definition is very coherent in real life, shown by the excellent discriminative power of both scores in the validation cohort. In regard of these limitations, a prospective study is warranted for refinement and external validation of this definition.

Conclusion

In conclusion, we have defined refractory septic shock in children as the association of high blood lactate with high vaso-inotrope doses associated with myocardial dysfunction. This definition is based on two septic shock scores showing excellent discriminative power in a multicentre validation population. The RSS Computed Score is a powerful and potentially useful tool to compare patients in future interventional randomized multicenter studies on septic shock. The RSS Bedside Score is easy to calculate and may assist in determining patients who would be suitable for inclusion in clinical trials of rescue therapies.