Introduction

Sepsis is a life-threatening condition defined as infection that results in organ dysfunction due to a dysregulated host response1. In 2017, an estimated 25 million cases of sepsis were noted globally in children, and over 3 million died2. In Australia and New Zealand, up to 6.5 cases per 100,000 of the paediatric population will develop severe sepsis requiring intensive care unit (ICU) admission3. The direct costs related to children with sepsis requiring ICU admission in Australia and New Zealand was estimated in 2013 at AUD$30.7 m annually4. Even when children survive sepsis, the longer-term impact of sepsis and post-discharge complications can be severe5,6. Timely recognition and initiation of appropriate interventions in children with sepsis has been shown to save lives and improve quality of life7.

Implementation of sepsis protocols and institutional pathways represent best practice recommended by the Surviving Sepsis Campaign, but require substantial investment8. Protocolized sepsis care is rarely cost-saving9. In adults, economic evaluations of the Surviving Sepsis Campaign10, Multiple Urgent Sepsis Therapies protocol11, and mandated protocolized sepsis care12 reported improved mortality, but increased costs13. Paediatric sepsis quality improvement studies have focused on outcomes including survival and length of stay (LOS)14 but have not assessed cost impacts and were often single-centre. It is urgent to assess the cost impact on the health system of paediatric protocolized sepsis care.

We conducted a population-level economic analysis of paediatric patients treated in the Queensland Sepsis Breakthrough Collaborative. This was a quality improvement project initiated by the Queensland Department of Health in response to the Australian National Action Plan for Sepsis15,16. The sepsis collaborative implemented evidence-based pathways in Emergency Departments (EDs) in Queensland for earlier recognition and treatment of sepsis in adults and children aimed at improving sepsis outcomes17,18. The primary objective of this study was to investigate the economic impact on the wider health system of Paediatric Sepsis Pathway (PSP) implementation. In the absence of patient-level data, aggregate data was used to examine sepsis-associated hospitalizations in Queensland, Australia.

Methods

Study design

This study was a population-based multi-site, data-driven simulation model and cost–benefit analysis of acute service utilisation before and after the implementation of an ED PSP. The study was approved by research ethics and governance, including a waiver of individual consent (Children’s Health Queensland HREC/18/QRCH/167) as well as state Public Health Act approval. No identifying information was used, and all research was conducted in accordance with relevant guidelines and regulations. Details of the Queensland Sepsis Collaborative18 and the PSP17 have been previously described. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist19 was used to report outcomes (Supplement 2).

Intervention

The PSP was introduced as a paper-based clinical decision support tool, based on the Surviving Sepsis Campaign and comprised of: systematic screening of paediatric patients to facilitate recognition of possible sepsis; appropriate escalation using decision trees; a protocolized treatment bundle with guides for antibiotic prescription and administration; and a parental information leaflet17,18. The purpose of the PSP was to assist ED physicians to recognize sepsis earlier in the disease process, enabling earlier treatment and better compliance with the sepsis treatment bundle, leading to improved clinical outcomes. Emergency physicians were trained to use the PSP by a dedicated paediatric sepsis clinical nurse consultant. A detailed description of the PSP version 1, used during the study period, can be obtained by contacting the corresponding author, although this has since been superseded by updated versions which can be obtained at: https://clinicalexcellence.qld.gov.au/priority-areas/safety-and-quality/sepsis/sepsis-resources/paediatric-sepsis-pathways.

Study participation was voluntary and restricted to EDs capable of advanced resuscitation, stabilisation and rapid transfer20. Further description of the Queensland hospital capability framework is provided in Supplement 3. Twenty-one Queensland hospitals were initially eligible for involvement, and 16 chose to participate. Participating sites had nominated sepsis teams responsible for leading implementation, supported by central quality improvement advisors from the sepsis collaborative.

Setting and population

Queensland is the second largest state in Australia geographically and has an estimated population of 5.2 million (June 2020). It has the greatest population dispersion of any Australian state, over a geographical area larger than Alaska. The proportion of children (0–14 years) in the state has been relatively steady over the last 5 years at 19.3%, reflecting a paediatric population of approximately one million21. Paediatric care in Queensland is centralized, with quaternary care occurring in a single children’s hospital in Brisbane. Children requiring escalation of care are transported to more specialized centres, with the mode of travel (road, rotary or fixed wing transport) dictated by transport distance.

Inpatient episodes for all patients aged under 18, admitted via ED with an ICD-10 coded diagnosis of sepsis across Queensland, were compared before and after the intervention. Patients with ward-acquired sepsis codes were excluded. Sepsis coding is provided in Supplement 3. Three hospitals piloted the PSP before it was introduced at a further 13 sites, including the state children’s hospital. These 16 sites were compared to 32 hospitals that did not implement the PSP. Sites were categorized into:

PSP-CH: The only dedicated children’s hospital (CH) in the state; implemented the PSP in the ED. This is the paediatric tertiary/quaternary hospital in Queensland and is the primary transfer destination for paediatric patients requiring a significant escalation in care.

PSP-mixed: Two dedicated paediatric EDs and 13 EDs treating a mixed population of adults and children that implemented the PSP. Does not include PSP-CH.

Non-PSP: EDs of 32 acute care hospitals that did not implement the PSP. These include 24 smaller regional facilities without ICU capability and eight tertiary facilities with ICU capability.

Figure 1 displays the likely patient pathways for PSP-mixed and non-PSP sites (panel A) and the PSP-CH (panel B). Patients were assumed to be non-palliative, meaning that mortality was only likely in ICU. In Queensland, the transfer process typically occurs by local referral from lower capability centres to an intensive care specialist through Retrieval Services Queensland. The intensivist provides advice, initiates triage with logistic support by Retrieval Services Queensland, and facilitates transfer to a higher capability tertiary care centre if required. This transfer is then conducted by paediatric intensive care, a designated adult, or paramedic team depending on clinical need.

Figure 1
figure 1

Admission pathway for all hospitals except PSP-CH (A) and PSP-CH only (B). PSP-CH is the only specialist children’s participating hospital.

Full size image

Study periods

The pre-implementation period was from September 2015 to August 2017 for one PSP-mixed pilot site and from July 2016 to June 2018 for all remaining sites, including two additional PSP-mixed pilot sites and all non-PSP sites. A wash-out period was applied while staff were trained in the PSP and supporting infrastructure put into place. This was between September 2017 and July 2018 for the first pilot site, from June 2018 to July 2018 for the remaining two pilot sites, and from June 2018 to January 2019 for the remaining PSP and non-PSP sites.

The PSP implementation period was from July 2018 to July 2020 for the three PSP-mixed pilot sites. For the remaining sites, including PSP-CH and non-PSP sites, the PSP implementation period was from January 2019 to December 2020. A schedule is provided in Supplement 4. The wash-out period was also applied to non-PSP sites as a counterfactual condition to determine state-wide trends in healthcare utilisation independent of the program.

Data collection

This study used available data that are routinely collected in the Queensland Hospital Admitted Patient Data Collection (QHAPDC). Data from QHAPDC were available in aggregate form as summary statistics by pre-intervention, wash-out, and intervention periods for each group, with 24 months of pre- and post-implementation data for each site. Utilisation data included: non-ICU LOS; ICU LOS; ICU admission rate; and the transfer out rate. Non-ICU LOS refers to any amount of time during admission not spent in the ICU.

Outcomes

The primary study outcome was hospital length of stay in bed days, separated into ICU and non-ICU bed days. The value of ICU bed days selected was $5381 (SD $1423), derived from previous research22 and updated to 2021 values using a 3% discount rate, standard for economic evaluations in developed countries such as the USA and Australia23,24. The value of ward bed days could not be derived from the literature. We solicited the cost of a paediatric bed day from Queensland Health and received a mean value plus overheads of $1512 per day based on representative general wards from the hospital-based corporate information system. Due to the uncertainty around this figure, we applied the same level of uncertainty as in the ICU day valuation, which was 26.4% ($400). Interhospital transfer costs were not available, but the transfer rate was analysed for context.

Statistical and economic analysis

Probabilistic sensitivity analysis (PSA) is a method of quantifying uncertainty in model input parameters using Monte Carlo simulation methods25,26. As patient-level data were not accessible to determine whether changes were due to sampling bias, simulation was applied to estimate uncertainty intervals. Summary statistics were used to create appropriate prior distributions, from which 10,000 samples were drawn to compare patient outcomes before and after implementation across PSP and non-PSP sites25.

To estimate statewide changes following implementation, our null hypothesis was that rates would remain unchanged. Expected results were calculated by multiplying post-implementation sample sizes by pre-implementation utilisation. Expected results were then subtracted from observed results to estimate changes associated with the PSP. To determine the counterfactual, or expected change at PSP sites in the program’s absence, difference-in-difference calculations were applied. The percentage change for non-PSP sites was subtracted from the change for PSP sites to account for general care trends across Queensland.

Program costs

The costs of the PSP were calculated by summarizing total labour costs for clinical, administrative, and support staff to develop and deliver the program from 2017 to 2021. Staff time allocated to project implementation was tabulated for both in-kind and budgeted activities. Staff time was then multiplied by salary rates calculated pro-rata based on Queensland Health pay scales27 and converted to 2021 values. A detailed summary of all PSP costs is contained in Supplement 5.

Results

Pre-implementation, there were 819 admissions (409.5/year) and an 18.2% ICU admission rate (149 admissions, or 74.5/year) in the PSP group. In the non-PSP group, there were 236 (118/year) admissions and a 13.1% ICU admission rate (31 admissions, or 15.5/year). This changed post-implementation to 1211 admissions (605.5/year) and a 17.4% ICU admission rate (211 admissions, or 105.5/year) in the PSP group. In the non-PSP group, there were 293 (146.5/year) admissions and 11.9% ICU admission rate (35 admissions, or 17.5/year). Transfers out from PSP-mixed dropped from 25.6% (140, or 70/year) to 17.0% (139, or 69.5/year) and increased in the non-PSP group from 36.0% (85, or 42.5/year) to 39.2% (115, or 57.5/year) between pre- and post-implementation.

Mean LOS dropped from 4.1 (SD 5.7) to 3.7 (SD 4.5) days for PSP-mixed, 10.2 (SD 13.9) to 8.1 (SD 9.6) days for PSP-CH, and 2.8 (SD 4.2) to 2.6 (SD 5.2) for non-PSP sites. Patient mortality in the PSP group was 1.1% (9 deaths, or 4.5/year) pre-implementation and 0.3% (2 deaths, or 1/year) post-implementation. Mortality in the non-PSP group was 0.4% (1 death, or 0.5/year) pre-implementation and 0.7% (2 deaths, or 1/year) post-implementation.

Probabilistic sensitivity analysis

There was a high probability of reductions in LOS in both ICU and non-ICU utilisation across the PSP sites following implementation (Table 1). The most likely reductions in LOS across all PSP sites were 20.8 h in non-ICU utilisation and 9.4 h for ICU utilisation, with 99.7% and 89.8% of simulations showing a reduction, respectively. Rate of admission to ICU was likely unchanged. For non-PSP sites, the most likely changes were a reduction in non-ICU LOS of nearly 5 h and an increase in ICU LOS of over 7 h. These occurred in 74% and 66% of simulations, respectively.

Table 1 Results of probabilistic sensitivity analysis pre- and post-intervention.
Full size table

Policy impact

There was an estimated reduction in total non-ICU days of 1013 (507.5/year, − 15.6%) at PSP sites, compared to a reduction of 55 days (27.5/year, − 6.4%) at non-PSP sites. There was also an estimated reduction in total ICU days of 111 (− 12.9%, 55.5/year), compared to an increase of 11 days (+ 7.0%, 5.5/year) at non-PSP sites. Escalations declined by 69 (− 33.5%, 34.5/year) at PSP-mixed sites, compared to an increase of 9 (+ 8.9%, 4.5/year) at non-PSP sites (Table 2).

Table 2 Summary of policy impacts derived from PSA.
Full size table

Following difference-in-difference analysis, the expected reduction in non-ICU bed days was 596 (− 9.2%, 298/year). The expected reduction in ICU bed days was 172 (− 19.9%, 86/year) and the expected reduction in transfers out was 83 (− 39.6%, 44.5/year). Confidence intervals for these figures are displayed in Table 3.

Table 3 Economic value of freed ICU and non-ICU bed days.
Full size table

Implementation costs

The cost of program implementation over its duration was estimated at $1,729,665 (Supplement 5). The value of freed ICU and non-ICU bed days was $1,826,684 [$872,392, $3,384,590] after difference-in-difference analysis (Table 3). Thus, the most likely outcome was a cost savings of $97,019 [− $857,273, $1,654,925] for the state over 24 months ($48,510 [− $428,636, $827,463] annually), though this was subject to some uncertainty. The PSP was cost-saving in 63.4% of simulations.

Discussion

This study used aggregate administrative hospital data before and after the implementation of a PSP across Queensland. Findings from this economic evaluation highlighted a likely reduction in utilisation rates across several key metrics. The overall policy impact of implementing the program was a substantial reduction in non-ICU and ICU utilisation, albeit subject to uncertainty. By attaching bed day values to these reductions, it was likely that the PSP was cost-neutral or slightly cost saving for the health system.

Our results show a slight reduction in length of stay for PSP-mixed sites, suggesting patients may have received necessary care sooner as early sepsis intervention is associated with reduced LOS28. Significantly more PSP-mixed patients were managed at their local hospital following the PSP. We hypothesize that the intervention may have given clinicians the resources and training to treat more patients locally.

Comparisons with current literature

Current evidence for sepsis bundles shows that sepsis can be detected accurately29 and leads to mortality reductions in the UK30 and New York28,31 following bundle completion in adults. In children, results have been mixed32,33, potentially due to bundle adherence. Lower risk-adjusted in-hospital mortality following bundle application was observed in New York, but only 25% of patients received the bundle within 1 h of diagnosis7. Mortality rates were obtained in this study, but given the low mortality rate of paediatric sepsis patients at baseline (1.1%), we were hesitant to include the impact of mortality reductions as even small changes due to chance could be deemed significant34.

This is the first published study to describe systems-level changes in health care utilisation following the implementation of an ED sepsis recognition, escalation and management protocol in paediatric patients. While many studies have examined the benefits of similar programs on an individual patient basis, none have examined their results from a health system perspective using simulation to enable improved decision making. This study demonstrates that sepsis bundles may lead to modest cost savings at a systems level, though other studies show no difference12 or an increase11 in inpatient costs on a per-patient level.

Healthcare in a geographically large state such as Queensland must cater to both urban and remote regions. Centralization of services is one potential solution35 that has shown positive outcomes for patients36,37. However, centralization necessitates transfers to higher acuity hospitals, a costly process with a burden on patients and their families in a physically large state. Estimates from the UK indicate that these costs, including ongoing treatment of complications and lost productivity from delayed recognition and treatment, may significantly outweigh direct health system costs38. By training clinicians in remote regions to recognize and manage paediatric sepsis sooner, it appears possible to reduce the need for retrievals, enabling more local treatment without negatively affecting patient outcomes. These findings therefore may be applicable to health systems beyond the Australian context, especially in areas with a high degree of geographic remoteness, such as Russia or Canada.

Sepsis coding

The number of paediatric sepsis diagnoses upon admission increased from 819 to 1211 in the PSP-mixed group following implementation while the non-PSP group increased from 236 to 293. Sepsis is often undercoded39; improved recognition upon ED presentation was a key component of the intervention, indicating increases may have been due to improved awareness of the symptoms and signs of sepsis in the ED. Changes to coding practices, a recognized phenomenon in sepsis documentation40,41, between pre- and post-intervention periods from the international classification of diseases may also have contributed to this change42. Paediatric sepsis coding has led to substantial uncertainty in the recognition and diagnosis of sepsis and severe sepsis43, which may have captured a lower acuity patient population. However, ICU admission rate at all sites remained relatively stable pre- and post-implementation, indicating that major changes to acuity were unlikely.

Limitations

Constrained by time and the accessibility of patient-level data, the choice of a population-level analysis nonetheless offered the opportunity to provide robust estimates of net costs that might be expected when implementing similar programs. It is possible that the change in sepsis awareness led to lower acuity patients being diagnosed and coded as sepsis, which could reduce utilisation rates through sampling bias. However, earlier recognition may also have prevented rapid deterioration and led to a lower severity of illness through the prevention of shock. Further research on the PSP should address these limitations using patient-level data.

Accounting for additional factors such as readmission rates, improvements to morbidity and mortality, quality of life, transport costs and other measures of utility could significantly increase the value of the PSP. A study taking a societal perspective of sepsis prevention on mortality found substantial cost savings44.

A slight mismatch in the number of hospitals with ICU capacity in the non-PSP group may have affected validity as a counterfactual. Eight non-PSP hospitals had ICU capacity compared to 16 in the PSP group, and some quality improvement initiatives may have been targeted towards ICU-capable sites over this period. However, there may have also been initiatives targeted towards sites without ICU capability over this period, and as we examined the rate of change rather than absolute change, the risk of bias was deemed low. Unknown factors such as administrative workload may have been a possible source of confounding causing differences in patient care between ICU-capable PSP and non-PSP sites. Another potential confounder was that it was not possible to collect information on compliance with the PSP among participating sites. Re-engagement with participating sites as part of an implementation evaluation may be able to retrospectively assess compliance as well as identify any potential differences between sites implementing and not implementing the PSP. This is a topic for future research and can be informed by an audit of compliance practices and organisational factors.

While there were no other concurrent state-wide sepsis programs, quality improvement is an ongoing process that can lead to incremental improvements in care delivery over time45 including reductions in LOS46. While a greater concentration of higher capability sites in the PSP-mixed group compared to the non-PSP group may have led to improvements unrelated to the PSP, other state-wide programs may have been in place to improve the performance of lower capability sites. Future research using patient-level data and matched cases and controls could identify whether this was possible. It was also likely that there was some ‘bleeding over’ from the hospitals within the PSP to hospitals outside of it, especially between intensive and emergency practitioners who often work within multiple facilities, potentially making these findings more conservative.

This study was not able to directly calculate cost savings from a hospital perspective, instead using valuations of bed days. This may be why our results demonstrate cost savings. When re-evaluating the PSP using patient-level data, future research might incorporate National Hospital Costs Data Collection episode-level costs in a data linkage to improve estimates beyond our extrapolated average bed day values.

Conclusion

This study observed an association between the introduction of the PSP and a reduction in healthcare utilisation. While this study shows that the PSP was likely modestly cost-saving for the health system, a patient-level statistical analysis is required to determine whether this was due to changes in case mix or a result of the PSP.