Prognostic accuracy of Neonatal SOFA score versus SIRS criteria in preterm infants with late-onset sepsis

Neonatal SOFA score was reported as an accurate predictor of mortality while the prognostic accuracy of SIRS criteria is unknown. The aim was to compare neonatal SOFA and SIRS criteria for the prediction of late onset sepsis-related mortality in preterm newborns. Newborns ≤ 32 weeks with late onset sepsis were retrospectively studied. Neonatal SOFA and SIRS criteria were calculated at onset of sepsis (T0), and after 6 ± 1 (T1), 12 ± 3 (T2) and 24 ± 3 h (T3). Outcome was death during antibiotic treatment for late onset sepsis. We studied 112 newborns with gestational age 26.9 ± 2.3 weeks; 11% met the study outcome. Neonatal SOFA was significantly higher in non-survivors vs. survivors at all time intervals; SIRS criteria were significantly higher in non-survivors vs. survivors at T1, T2 and T3. Neonatal SOFA increased over time in non-survivors (p = 0.003). At T0, the area under receiver operating characteristics curve was significantly higher for neonatal SOFA score than SIRS criteria (0.950 vs. 0.569; p = 0.0002), and the best calculated cut-off for T0 neonatal SOFA score was 4. In multivariate analysis T0 and T1 neonatal SOFA were predictors of late onset sepsis-related mortality (p = 0.048 and p < 0.001). Conclusion: Neonatal SOFA score showed greater discriminatory capacity for mortality than SIRS criteria and might be helpful to plan management for patients at higher risk of death. What is Known: • Neonatal SOFA score may be an accurate prognostic tool. • No prognostic score has been fully standardized for septic newborns in NICU. What is New: • Neonatal SOFA score outperformed SIRS criteria for the prediction of prognosis in preterm infants with late onset sepsis. • Neonatal SOFA score assessed at onset of sepsis and 6 hrs later is a predictor of mortality. Supplementary Information The online version contains supplementary material available at 10.1007/s00431-023-05143-5.


Introduction
Late onset sepsis (LOS) is a major cause of morbidity for preterm infants in NICU, affecting 10-30% of very low birthweight infants [1][2][3] with a mortality rate of 7-15% [4,5].According to Sepsis-3 consensus, sepsis definition in adults is centered on organ dysfunction due to dysregulated host response to infection [6] while the definition of sepsis in 1 3 newborns is still based on Systemic Inflammatory Response Syndrome (SIRS) criteria alongside with the presence of infection, as proposed by the International Pediatric Sepsis Consensus Conference in 2005 [7].Prognostic assessment of critically ill septic adults is based on Sequential Organ Failure Assessment (SOFA) score [6] while no scoring system has been fully standardized for newborns in NICU.Neonatal SOFA (nSOFA) was developed as a scoring system for organ dysfunction in preterm infants with LOS [8] and showed valuable prognostic accuracy in newborns with gestational age < 33 weeks [5,8,9].Moreover, the progression of organ failure in preterm infants who die because of LOS showed a definite temporal relationship with death [10].
Although SIRS criteria are diagnostic rather than prognostic tools, both in adults and pediatrics they were compared to SOFA score for the prediction of sepsis-related mortality [11,12].A significantly higher prognostic capacity for sepsisrelated mortality and morbidity was reported for SOFA score than SIRS criteria [11,12], showing that organ impairments rather than signs of inflammation are the key elements for prognostic assessment of patients with sepsis.The prognostic capacity of neonatal SIRS criteria is currently unknown.On these bases we hypothesized that nSOFA is a better prognostic marker of mortality than SIRS criteria in preterm newborns with LOS.Thus, the aim of this study was to compare the accuracy of nSOFA score with SIRS criteria for the prediction of LOS-related mortality in preterm newborns.

Study design and participants
This retrospective single center study was approved by the pediatric local ethics committee.Preterm infants who were born at ≤ 32 weeks gestational age from January 2016 to December 2021 and experienced an episode of LOS during NICU stay at Careggi University Hospital, Florence, Italy, were enrolled in the study.Exclusion criteria were the presence of major congenital abnormalities or genetic syndromes and inborn errors of metabolism.LOS was defined as positive blood culture taken after the first 72 h of life [13].In order to exclude contaminated samples, in cases of blood culture growing coagulase negative Staphylococcus species, patients were considered as having LOS only if C-reactive protein (CRP) was > 10 mg/L and they received antibiotics for > 5 days [14,15].Blood samples for cultures were obtained from peripheral vein (at least 1 mL) [16] with strict adherence to the sterile technique and collected in dedicated vials (BD Bactec TM , Becton Dickinson and Company, Sparks, USA).The primary outcome of the study was the comparison of accuracy of nSOFA and SIRS criteria in predicting LOS-related mortality defined as death occurring during ongoing antibiotic treatment for LOS.
At onset of sepsis (T 0 ) each enrolled patient was sampled for blood culture, complete blood count, CRP and procalcitonin (PCT).Neonatal SOFA score and SIRS criteria were calculated at T 0 , and after 6 ± 1 (T 1 ), 12 ± 3 (T 2 ), and 24 ± 3 h (T 3 ).As per local protocol, cases of LOS received empiric treatment with vancomycin and amikacin or other aminoglycoside; if a previous course of antibiotics had been administered within 7 days before the onset of LOS, different antibiotic regimens including carbapenem and second line anti-staphylococcal drugs were considered.Targeted antibiotic treatment was based on sensitivity of the isolates.All cases of LOS were treated with antibiotics for at least 5 days or until death.
Inotropic drugs (i.e.adrenaline, noradrenaline, dopamine, dobutamine, etc.) and glucocorticoids for cardiovascular impairment were administered and titrated consistently with the findings of functional echocardiography and/or monitoring of systemic arterial pressure and lactate levels, according to the American College of Critical Care guidelines for the treatment of neonatal shock [17].Concomitant treatments, such as sedatives, analgesics, caffeine, ibuprofen and paracetamol for the treatment of patent ductus arteriosus, steroids for purposes other than increasing BP, and parenteral nutrition were administered according to local protocols.
Infants were started on mechanical ventilation when the pH was < 7.20 with PaCO2 > 65 mm Hg, or PaO 2 < 50 mmHg with FIO 2 ≥ 0.50, after surfactant treatment, or if infants had frequent episodes of apnea.Mechanical ventilation was set to maintain a PaCO 2 of 55 to 65 mmHg and 90-95% pulse oxygen saturation (SpO 2 ).All data were extracted from local electronic clinical charts.
A priori rule was established for calculating nSOFA score or the number of SIRS criteria in the event of death during the assessment period.This approach is strongly recommended for studies on adult SOFA score to avoid missing data for patients with potentially high scores, in order to prevent a survivorship bias with paradoxical underestimation of the score for patients experiencing death during the assessment period [18].To date, no consensus exists about the most appropriate method of handling missing data due to early mortality [18].Among the proposed strategies [18] we decided that, in case of death within the first 24 h of onset, the highest recorded value of nSOFA and SIRS criteria would be imputed for the time points following death.We chose this approach because no method considering specific extra penalty for death has been explored to date for newborns and, on the other hand, considering the last recorded instead of the highest value would not account for mortality.

Statistical analysis
The clinical characteristics of enrolled patients were described as mean and SD for continuous parametric variables, median and interquartile range for non-parametric variables, and counts and percentage for discrete variables.Comparisons between groups were performed with Student t test for parametric continuous variables, Mann-Whitney U test for continuous nonparametric variables, such as nSOFA and SIRS criteria, and Chi-squared test for categorical variables.Changes over time of nSOFA score and SIRS criteria within the single groups were analyzed with Friedman test for repeated measures.With the purpose of measuring the discrimination performance of T 0 nSOFA and T 0 SIRS criteria, the receiver operating characteristic curves (ROC) for each score were analyzed to calculate the area under the curve (AUC) and the best cut-off level.The comparison between the AUC of T 0 nSOFA score and T 0 SIRS criteria was performed using the De Long method [19].Variables with P < 0.05 were considered for inclusion in multivariate analysis.
Sample size was calculated assuming an AUC of 0.88 for T 0 nSOFA score [5] and LOS-related mortality of 10% [4,5].In order to detect a difference in AUC between nSOFA and SIRS criteria of 20%, with alpha error = 0.05 and power of 0.80, the calculated sample size was 101.
Data were analyzed with SPSS, version 26.0 (IBM, New York, US).
This study followed the "Strengthening the Reporting of Observational Studies in Epidemiology" (STROBE) guidelines for reporting observational studies.
Neonatal SOFA score was significantly higher in nonsurvivors vs. survivors at T 0 , T 1 , T 2 , and T 3 [Table 2 and Fig. 1a].SIRS criteria were significantly higher in nonsurvivors vs. survivors at T 1 , T 2 , and T 3 but were similar at T 0 (Table 2 and Fig. 1b).Neonatal SOFA score increased during the first 24 h from onset of LOS in non survivors (p = 0.003) while it did not vary in survivors (p = 0.921); SIRS criteria did not change over time both in non survivors (p = 0.908) and survivors (p = 0.712) (Table 2).
In multivariate analysis including gestational age, Gram negatives, T 0 and T 1 nSOFA, and T 1 SIRS criteria, T 0 and T 1 nSOFA remained significantly associated with mortality (p = 0.048 and p < 0.001, respectively) while T 1 SIRS did not (Table 3).We decided not to include BW in the multivariate analysis model since it was collinear with gestational age.If birthweight was included in the model, T 0 and T 1 nSOFA remained significantly associated with mortality (p = 0.049 and p < 0.001, respectively) (Supplementary Table 1).

Discussion
Our study compared, for the first time, the prognostic accuracy of nSOFA score and SIRS criteria in predicting mortality in very preterm infants with LOS and we have demonstrated a greater discrimination capacity of nSOFA.Neonatal SOFA was found to be higher in non-survivors vs. survivors at any time during the first 24 h from sepsis onset and to increase over time in non-survivors, while it did not vary in survivors.Moreover, multivariate analysis showed that both T 0 and T 1 nSOFA scores were independent predictors of mortality.These results confirm previous findings of higher nSOFA score in non-survivors vs. survivors during the first 48 h from onset of LOS in different cohorts of very preterm newborns [5,20,21].The value of T 0 nSOFA AUC (0.9498) indicates high accuracy for the prediction of LOS-related mortality, in agreement with the mean AUC of 0.88 reported in a previous multicenter study [5].We found an optimal cut-off of 4 for T 0 nSOFA to predict LOS-related mortality (sensitivity 92%, specificity 85%).
On the other hand, SIRS criteria did not discriminate between survivors and non-survivors at onset, although they were significantly higher in non-survivors vs. survivors at T 1 , T 2 and T 3. Lack of difference between survivors and non-survivors at T 0 might be consistent with the diagnostic nature of SIRS criteria.At T 1 , T 2 and T 3 significantly higher SIRS criteria in non-survivors might be explained with persistence despite treatment, in comparison to survivors.However, in contrast to nSOFA score, SIRS criteria did not significantly increase over time in non-survivors, indicating poor association with unfavorable progression and outcome.Moreover, we found a sub-optimal AUC (0.5734) for T 0 SIRS criteria and T 1 SIRS criteria failed to predict mortality in multivariate analysis.On a whole, our findings show poor prognostic accuracy of SIRS criteria, partly attributable to the diagnostic nature of SIRS criteria.Globally, these results support the development and validation of specific scores for prognostic purposes.
The comparison of AUC of ROC curves showed significantly better discriminating capacity for T 0 nSOFA vs. T 0 SIRS criteria (p = 0.0002).Similarly, in septic patients admitted to PICU discrimination for in-hospital mortality was significantly higher for pSOFA than SIRS criteria, with AUC of 0.829 vs. 0.727 respectively (p < 0.01) [11] and in critically ill adults with suspected sepsis, an increase in SOFA score of 2 or more points showed a significantly higher discrimination for in-hospital mortality than the presence of at least 2 SIRS criteria [12].A previous study showed the highest sensitivity of nSOFA occurring 24 and 48 h after onset and the highest specificity 6 h after onset [20], while higher AUC was found 12 h after onset in another study [21].Despite these data, we decided to analyze T 0 and T 1 in order to evaluate the potential usefulness of nSOFA and SIRS criteria for early identification of high-risk patients during the course of LOS with the aim of prompting appropriate care in terms of monitoring and limiting organ impairment progression.
In multivariate analysis gestational age was preferred over birthweight because, from a pathophysiological perspective, the immunologic dysfunction observed in preterm newborns and predisposing to LOS and LOS-related mortality is attributable to immaturity itself [22][23][24].Moreover, no small for gestational age infants, defined as birthweight < 3 rd centile for gestational age [25], was included among non-survivors, therefore our study could not detect the impact of such variable on LOS-related mortality.Finally, the inclusion of birthweight in the multivariate analysis did not significantly impact on the model.
Our findings highlight the pivotal importance of organ dysfunction assessment for the prognostic stratification of patients with sepsis as opposed to signs of inflammation.Our data are consistent with organ dysfunction progression demonstrated in newborns dying because of LOS, as oxygen requirement significantly increased from 3 days before death through the day of death, the need for mechanical ventilation and for vasopressors significantly increased from 2 days before death, while platelet count significantly decreased on the day before death [10].
In our population, non survivors presented lower gestational age and birthweight, and higher incidence of complications of prematurity, in comparison to survivors, in agreement with previous observations [1,22], suggesting that baseline characteristics of patients might play a prevalent role in determining the outcome of LOS.However, according to multivariate analysis, for prognostic purposes, baseline characteristics as gestational age and birthweight are outperformed by scores of organ dysfunction.
Our study has some limitations.First, a relatively small number of non-survivors was included, and 4/12 patients died during the assessment period, causing one value for T 2 and 3 values for T 3 of nSOFA and SIRS criteria to be replaced by the maximal observed value for the patient.At present, no specific strategy to appropriately replace missing   data in case of early death has been developed for studies on prognostic scores.Second, the criteria to establish the need for vasopressors or steroids with the purpose of maintaining blood pressure is still a matter of debate in newborns [26].Patients in our cohort received medications for cardiovascular support basing on systemic blood pressure values and/or echocardiographic demonstration of abnormal cardiac function and low cardiac output according to local protocols.However, the monocentric design of our study was a strength, ensuring that the same local protocol was applied to all enrolled patients.Finally, because of the need to limit the number of samples in newborns for the hematologic component of SIRS and nSOFA after T 0 , relying on the latest values could be partially inaccurate.
In conclusion, our data indicate that nSOFA is an accurate prognostic tool for predicting mortality in preterm infants with LOS and shows higher discriminatory capacity for mortality than SIRS criteria.Hence, our findings discourage the use of SIRS criteria as prognostic scores and support the use of nSOFA score for prognostic stratification of preterm infants with suspected or proven LOS.Early identification of the subset of infants at greater risk of death is useful to plan patient-targeted management with the purpose of avoiding detrimental evolution of organ dysfunction and limiting LOS-related mortality.

Fig. 1 a
Fig. 1 a Violin plot of nSOFA score at T 0 , T 1 , T 2 and T 3 for survivors (S) and non-survivors (NS).b Violin plot of SIRS criteria at T 0 , T 1 , T 2 and T 3 for survivors (S) and non-survivors (NS) ◂

Fig. 2
Fig. 2 ROC curves for T 0 nSOFA score and T 0 SIRS criteria

Table 2
Comparison of nSOFA score and SIRS criteria between survivors and non-survivors and variations over time within survivors and non-survivors