Out-of-hours discharge from intensive care, in-hospital mortality and intensive care readmission rates: a systematic review and meta-analysis

Search strategy and selection criteria

We registered this systematic review and meta-analysis with PROSPERO (CRD42014010321). We published the protocol (https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-015-0081-8) [21] and followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [22] and MOOSE (Meta-analysis Of Observational Studies in Epidemiology) guidelines [23] where applicable.

To be included, studies had to: report in-hospital mortality and/or ICU readmission rates for all patients aged ≥ 16 years discharged alive from a general surgical, medical or mixed ICU to a lower level of in-hospital care (high dependency or ward-level); report these outcomes separately for patients discharged from ICU out-of-hours and in-hours; and follow-up patients to hospital discharge. We defined “out-of-hours” and “in-hours” as separate time periods in each day of the week with “out-of-hours” including 00:00 and “in-hours” including 12:00. We did not change definitions for the weekend period. Studies that separated weekday and weekend but did not separate in-hours from out-of-hours were excluded. We excluded papers where patient episodes included in the out-of-hours analysis also contributed to the analysis in a larger study. We also excluded studies restricted to specific patient populations (e.g. patients who underwent cardiac surgery, were managed in a specialist neurosurgical intensive care or received liver transplants). We included prospective or retrospective original studies that used quantitative methods of data collection and analysis. All publication languages were included. We did not apply date restrictions. We included unpublished data, where found.

We performed searches in Medline, Embase, Cumulative Index of Nursing and Allied Health Literature (CINAHL), the Cochrane library and OpenGrey. The last search date for all databases was 12 June 2017. Reviews or reports of risk factors at ICU discharge may not refer to out-of-hours in the title or abstract, particularly if the effects were not significant. To address this, we conducted two searches: a general search for all factors associated with post-ICU in-hospital mortality or readmission and a search focused specifically on out-of-hours discharges. A medical librarian (T.P.) guided our search strategy. Details of the search strategy are shown in the supplementary material, Table 1. We undertook additional keyword and citation searches from identified studies using Medline and Web of Knowledge.

Data analysis

We exported all search results to a reference management software programme (Endnote; Thomson Reuters, www.endnote.com), which automatically identified duplicates. Two researchers (P.W. and S.V.) reviewed the initial results in three stages (title, abstract and full text). We resolved disagreements by recourse to the original text. From this list, further searches using relevant keywords (using Medline), and citation searches (using Web of Knowledge) for each paper were conducted.

We compared mortality rates and readmission rates in patients discharged from intensive care out-of-hours versus in-hours. For each study, we calculated risk ratios and 95% confidence intervals for each of the available outcomes, mortality and readmission. We summarised data using a random-effects meta-analysis (to account for the variance we found between studies). We used the DerSimonian and Laird method of computing the between-studies variance [25]. We present results in forest plots using the STATA metan procedure [26]. We aggregated data at the level of individual studies. We assessed consistency using both the χ² test and the I² statistic [27]. Where studies adjusted their analysis for potential confounders, we summarised odds ratios using the same methods.

We pre-specified sensitivity analysis by omitting studies of different quality or risk of bias. We pre-specified subgroup analyses by discharge destination (ward or high dependency unit, as defined by the authors), different definitions of out-of-hours and inclusion of patients receiving palliative care (again as defined by the authors), where there were sufficient studies. As ICU practice and provision is known to vary geographically [20], we undertook post hoc analyses of the effect of out-of-hours discharge on mortality and readmission to an ICU by the main geographical areas of the published studies (United Kingdom, Europe, Australasia, Asia, United States of America with South America and Canada).

We used the Newcastle–Ottawa Scale (NOS) to assess study quality [24]. This tool focuses on three broad areas: selection of groups, comparability of groups and ascertainment of outcome. The final output offers a score out of nine. We selected discharge destination (ward or high dependency unit), age and “admission severity of illness” as potential confounders. Two reviewers (S.V. and P.W.) separately assessed the studies. We resolved disagreements by discussion or referral to a third reviewer if necessary. We assessed the risk of publication bias by visual assessment of funnel plots and Egger’s regression [28]. We assessed study heterogeneity using both the χ² test and the I² statistic [27].

Strengths and weaknesses

We used a clearly defined, peer-reviewed protocol to conduct this review [21] to ensure a rigorous process and to minimise concerns regarding internal validity. A major strength of our study is that we demonstrate that out-of-hours discharge is associated with both an increased risk of death and an increased risk of readmission.

Our findings are consistent across large numbers of patients, different healthcare systems and different definitions of out-of-hours.

For both mortality and ICU readmission, our meta-analyses showed substantial heterogeneity. However, the heterogeneity mainly lies in how large the effects are rather than whether effects are present. Some heterogeneity was explained by differences in the definition of out-of-hours. Stratifying for different out-of-hours definitions decreased heterogeneity in some groups, but the association with mortality remained for all groups. Heterogeneity was lowest when we summarised the adjusted odds ratios for the eight studies reporting them. The association with mortality remained present. The reported duration of the out-of-hours period varied between 9 and 16 h. As a consequence, patients discharged at 20:30 are classified as out-of-hours in 12 studies and in-hours in 5 studies. Although inconsistent, investigators may differ in out-of-hours definitions because of differences in what time services change in their healthcare system. We chose, along with all the included authors, to treat out-of-hours at weekends in the same way as weekdays.

Our findings are limited by the cohort design of all the included studies. However, in the absence of any controlled studies, they provide the best available evidence. The limited data provided by many studies restricted our ability to explore underlying causes. Some were not primarily focused on the effects of out-of-hours discharges and so did not report population characteristics by discharge time. Where disease severity was reported by discharge time, studies differed in the assessment used. Where reported, most studies excluded patients discharged to another healthcare facility [5, 9, 11, 17, 18, 35, 37, 40, 41]. This introduces a potential bias if outcomes by discharge time differ in this group.

Despite being widely used in published systematic reviews, there is disagreement within the literature as to the appropriateness of scoring systems to assess study quality [43, 44, 45]. We used the NOS [24] rather than the ACROBAT-NRSI tool [43] proposed in our protocol. We selected the NOS as a frequently used and more suitable tool for the database cohort studies included. Studies commonly failed to define whether patients discharged in or out-of-hours were discharged to a ward or to high dependency area. This tended to occur in large multisite studies where high dependency facilities were not reported. As only two (small) studies scored less than seven, we did not undertake the planned sub-group analysis according to study quality.

There remains a risk of other studies not reporting outcomes related to out-of-hours discharge where no effect was seen. However, both the funnel plots and Eggers regression for mortality suggest publication bias against publishing studies with high out-of-hours discharge mortality rates. This could be explained by a reluctance of single centres to publish data associated with a perceived poor practice of out-of-hours discharge with high mortality. Our meta-analysis may therefore under- rather than over-estimate effects associated with out-of-hours discharge. As a result, our findings of increased mortality and increased ICU readmission associated with out-of-hours discharge appear robust.

Comparison to what is previously known

Prior to our meta-analysis, it remained unclear whether out-of-hours discharge was associated with increased mortality and readmission rates. Many studies that alone were not large enough to show a statistically significant effect (and could have been misinterpreted as there being no effect) contributed to our overall findings [9, 11, 16, 17, 18, 37, 38, 39, 40, 41]. One previous meta-analysis of the association between time of discharge from ICU and hospital mortality has been undertaken [46]. However, this meta-analysis did not include substantial amounts of relevant information, meta-analysing only 14 studies (of which we excluded 4 as they contained data duplicated within larger studies [19, 29, 30, 31] and did not study ICU readmission. The absence of registration or a published peer-reviewed protocol may in part explain these weaknesses.

Where increased post-ICU mortality or readmission has been associated with out-of-hours discharge, there has been debate as to whether out-of-hours discharge is causally associated with worse outcomes or simply defines a patient group who are more at risk [12, 14]. Differences in disease severity at admission to ICU have convincingly been shown not to explain the excess mortality and readmission rates found with out-of-hours discharge [1, 2, 8, 13, 15, 35]. The presence of treatment limitations at ICU admission also did not account for the effect [1]. In contrast, two studies [19, 36] corrected out an increased risk of out-of-hours discharge found on univariate analysis by including steps in their statistical model that corrected for factors suggestive of differences in care [continuing therapies such as dialysis and parenteral nutrition, decreased conscious state, increased therapeutic intervention scores (TISS) scores]. The findings are not contradictory; rather, the question asked is different. The first approach suggests that the worse outcomes associated with out-of-hours discharge are not explained by differences in patients at the point of admission to ICU (baseline covariates). The second provides differences in the care pathway that help explain the worse outcomes. It is whether these differences are of significance when a patient is discharged out-of-hours that is of interest, rather than any concept that crossing the threshold of an ICU between particular hours is causally associated with mortality. The combination of the findings of both types of study suggest that differences in management of patients discharged out-of-hours in part explain the worse outcomes seen. Other findings support this idea. Goldfrad and colleagues [2] found only 44.1% of discharges at night were fully ready for discharge in comparison to 86.3% in the day. Premature discharge was the main determinant of increased mortality associated with night-time discharge in their model. There is also evidence that patients with a high treatment need are disproportionately discharged at night [13]. Patients discharged out-of-hours also have higher severity of illness on their last ICU day [10, 11] and a greater incidence of treatments normally delivered in ICUs [19]. All of these might suggest that premature discharge in part explains our findings. One recent study [19] suggests that premature discharge is not a factor; however, the incidence of documented premature discharge is so low in comparison to previous studies [2], and the incidence of delayed discharge so high, that the meaning of this is unclear. Studies to date differ as to whether including differences in the presence of treatment limitations in statistical models corrects out the increased risks of out-of-hours discharge [1, 11, 15, 19, 36]. It seems unlikely that these patients would commonly benefit from readmission to an ICU.

Meaning

Our study resolves the question of whether out-of-hours discharge is associated with worse outcomes. The association with increased mortality and with readmission is substantial. Only the magnitude of the association remains somewhat uncertain. The retrospective non-randomised nature of all the studies undertaken prevents the attribution or non-attribution of causation (both for the studies and our analysis). Whether these increases in mortality and readmission result from patient differences, differences in care or a combination remains unclear. Our meta-analysis does, however, resolve a key area of debate. Disproportionate out-of-hours discharge of patients who will no longer benefit from intensive care cannot explain the increased mortality, as more of this group are readmitted.

Future work

Further work is required to explain why out-of-hours discharge is so strongly associated with post-ICU mortality and readmission, and how these adverse outcomes can be addressed. Future studies should aim to measure and account for confounders appropriately. To investigate the question of whether discharge out-of-hours results in poor outcomes studies should account for patient factors such as age and admission illness severity, which are not altered by differences in treatment in or after the ICU. To investigate the causes of differences found, studies should measure and account for differences in care. These would include measures of ICU exposure (length of ICU stay), illness severity at the point of discharge and ongoing care requirements and quantification of post-discharge care. The contribution of post-discharge care to differences in out-of-hours outcomes has so far only been explored to a limited extent in a single study [19], despite clearly being key to further understanding the problem [12].