Out-of-hours discharge from intensive care, in-hospital mortality and intensive care readmission rates: a systematic review and meta-analysis

Abstract

Purpose

Discharge from an intensive care unit (ICU) out of hours is common. We undertook a systematic review and meta-analysis to explore the association between time of discharge and mortality/ICU readmission.

Methods

We searched Medline, Embase, Web of Knowledge, CINAHL, the Cochrane Library and OpenGrey to June 2017. We included studies reporting in-hospital mortality and/or ICU readmission rates by ICU discharge “out-of-hours” and “in-hours”. Inclusion was limited to patients aged ≥ 16 years discharged alive from a non-specialist ICU to a lower level of hospital care. Studies restricted to specific diseases were excluded. We assessed study quality using the Newcastle Ottowa Scale. We extracted published data, summarising using a random-effects meta-analysis.

Results

Our searches identified 1961 studies. We included unadjusted data from 1,191,178 patients from 18 cohort studies (presenting data from 1994 to 2014). “Out of hours” had multiple definitions, beginning between 16:00 and 22:00 and ending between 05:59 and 09:00. Patients discharged out of hours had higher in-hospital mortality [relative risk (95% CI) 1.39 (1.24, 1.57) p < 0.0001] and readmission rates [1·30 (1.19, 1.42), p < 0.001] than patients discharged in hours. Heterogeneity was high (I² 90.1% for mortality and 90.2% for readmission), resulting from differences in effect size rather than the presence of an effect.

Conclusions

Out-of-hours discharge from an ICU is strongly associated with both in-hospital death and ICU readmission. These effects persisted across all definitions of “out of hours” and across healthcare systems in different geographical locations. Whether these increases in mortality and readmission result from patient differences, differences in care, or a combination remains unclear.

Introduction

The days in hospital following discharge from an intensive care unit (ICU) are high risk. In multi-centre studies, in-hospital mortality rates after ICU discharge are between 4.0 and 13.3% [1, 2], and account for one-third of all in-hospital deaths in patients treated in an ICU. These findings compare unfavourably with in-hospital mortality in other “high-risk” patient groups, cardiothoracic (2.7%) or upper gastrointestinal (2.4%) surgery [3, 4]. While in hospital, patients discharged from an ICU remain at high risk of requiring re-admission to an ICU [5,6,7]. Readmission to an ICU is associated with substantially higher mortality rates than a single admission [1, 5, 8].

Whether out-of-hours discharge from an ICU to a ward is associated with these poor outcomes is unclear, with studies showing differing results [1, 9,10,11]. Where an association has been found, opinions differ as to whether out-of-hours discharge from an ICU results in differences in care that cause these outcomes [12,13,14]. Observed outcome differences may also be explained because the population discharged out-of-hours differs from that discharged in-hours rather than there being differences in care. There are reasons why these differences in population might occur. If discharge out-of-hours results from bed pressures (more patients requiring admission to the ICU than available beds) [12], patients thought unlikely to benefit from further ICU support may be discharged preferentially. In this case, it would be expected that readmission rates in those discharged out-of-hours should be lower than in those discharged in-hours. Alternatively, if the patients are discharged before the point they no longer need ICU care, mortality and readmissions may increase.

Some researchers have looked specifically at out-of-hours discharge as a factor in post-ICU mortality or readmission [2, 8, 11, 15], and other cohort studies have included the effect of out-of-hours discharge in broader studies of mortality and readmission rates [5, 16,17,18]. To the best of our knowledge, a robust systematic review of the association of out-of-hours discharge with in-hospital mortality and ICU readmission, including both types of study, has never been undertaken. Synthesis of this information is important because out-of-hours discharge remains common [19]. If associated with post-ICU mortality or readmission, it is highly amenable to system change. If not, discharge at night may be a reasonable course to optimally manage ICU occupancy.

Our primary objective was to determine whether discharge from a general medical, surgical or mixed medical–surgical ICU out-of-hours in comparison to in-hours is associated with subsequent in-hospital mortality. Our secondary objective was to determine whether out-of-hours discharge in this population in comparison to in-hours discharge is associated with ICU readmission. As both ICU provision and practice differs internationally [20], we looked for geographical effects on these outcomes.

Methods

Search strategy and selection criteria

We registered this systematic review and meta-analysis with PROSPERO (CRD42014010321). We published the protocol (https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-015-0081-8) [21] and followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [22] and MOOSE (Meta-analysis Of Observational Studies in Epidemiology) guidelines [23] where applicable.

To be included, studies had to: report in-hospital mortality and/or ICU readmission rates for all patients aged ≥ 16 years discharged alive from a general surgical, medical or mixed ICU to a lower level of in-hospital care (high dependency or ward-level); report these outcomes separately for patients discharged from ICU out-of-hours and in-hours; and follow-up patients to hospital discharge. We defined “out-of-hours” and “in-hours” as separate time periods in each day of the week with “out-of-hours” including 00:00 and “in-hours” including 12:00. We did not change definitions for the weekend period. Studies that separated weekday and weekend but did not separate in-hours from out-of-hours were excluded. We excluded papers where patient episodes included in the out-of-hours analysis also contributed to the analysis in a larger study. We also excluded studies restricted to specific patient populations (e.g. patients who underwent cardiac surgery, were managed in a specialist neurosurgical intensive care or received liver transplants). We included prospective or retrospective original studies that used quantitative methods of data collection and analysis. All publication languages were included. We did not apply date restrictions. We included unpublished data, where found.

We performed searches in Medline, Embase, Cumulative Index of Nursing and Allied Health Literature (CINAHL), the Cochrane library and OpenGrey. The last search date for all databases was 12 June 2017. Reviews or reports of risk factors at ICU discharge may not refer to out-of-hours in the title or abstract, particularly if the effects were not significant. To address this, we conducted two searches: a general search for all factors associated with post-ICU in-hospital mortality or readmission and a search focused specifically on out-of-hours discharges. A medical librarian (T.P.) guided our search strategy. Details of the search strategy are shown in the supplementary material, Table 1. We undertook additional keyword and citation searches from identified studies using Medline and Web of Knowledge.

Data analysis

We exported all search results to a reference management software programme (Endnote; Thomson Reuters, www.endnote.com), which automatically identified duplicates. Two researchers (P.W. and S.V.) reviewed the initial results in three stages (title, abstract and full text). We resolved disagreements by recourse to the original text. From this list, further searches using relevant keywords (using Medline), and citation searches (using Web of Knowledge) for each paper were conducted.

Two researchers (S.V. and P.W.) extracted summary estimate data (relative risk or odds ratios, where reported) independently from each identified study. We used data extraction tables that we piloted before use. We extracted type of publication (academic paper or conference paper), publication date, study type, setting, eligibility criteria, proportion and strategy for missing data, and definitions of in-hours and out-of-hours. We extracted data on the numbers of patients included in the study, numbers of deaths and readmissions, demographic data including illness severity (where reported by in-hours and out-of-hours), co-variates used in multi-variate analysis and main conclusions (see Tables 1, 2 and 3). We extracted additional data to determine risk of bias using the Newcastle–Ottawa Scale [24]. Where data or details were missing, we contacted the authors by e-mail.

Table 1 Characteristics of included studies

Table 2 Cohort characteristics by study

Table 3 Unadjusted and adjusted odds ratios with included co-variates for mortality and readmission

We compared mortality rates and readmission rates in patients discharged from intensive care out-of-hours versus in-hours. For each study, we calculated risk ratios and 95% confidence intervals for each of the available outcomes, mortality and readmission. We summarised data using a random-effects meta-analysis (to account for the variance we found between studies). We used the DerSimonian and Laird method of computing the between-studies variance [25]. We present results in forest plots using the STATA metan procedure [26]. We aggregated data at the level of individual studies. We assessed consistency using both the χ² test and the I² statistic [27]. Where studies adjusted their analysis for potential confounders, we summarised odds ratios using the same methods.

We pre-specified sensitivity analysis by omitting studies of different quality or risk of bias. We pre-specified subgroup analyses by discharge destination (ward or high dependency unit, as defined by the authors), different definitions of out-of-hours and inclusion of patients receiving palliative care (again as defined by the authors), where there were sufficient studies. As ICU practice and provision is known to vary geographically [20], we undertook post hoc analyses of the effect of out-of-hours discharge on mortality and readmission to an ICU by the main geographical areas of the published studies (United Kingdom, Europe, Australasia, Asia, United States of America with South America and Canada).

We used the Newcastle–Ottawa Scale (NOS) to assess study quality [24]. This tool focuses on three broad areas: selection of groups, comparability of groups and ascertainment of outcome. The final output offers a score out of nine. We selected discharge destination (ward or high dependency unit), age and “admission severity of illness” as potential confounders. Two reviewers (S.V. and P.W.) separately assessed the studies. We resolved disagreements by discussion or referral to a third reviewer if necessary. We assessed the risk of publication bias by visual assessment of funnel plots and Egger’s regression [28]. We assessed study heterogeneity using both the χ² test and the I² statistic [27].

Results

We identified 1961 papers, of which 329 were deemed potentially eligible and reviewed at abstract. A total of 154 full text papers were reviewed following abstract screening (Fig. 1). We identified 34 articles eligible for inclusion (4 conference abstracts and 30 papers), of which 16 were subsequently excluded. Nine studies were excluded because they included data also reported in a larger study (i.e. large national database studies) [6, 13, 19, 29,30,31,32,33,34]. Four studies were deemed ineligible on further review. We contacted the authors of three papers: two were excluded as we could not obtain patient numbers, and one paper contained reporting errors which the authors were unable to resolve.

Fig. 1

Prisma flowchart

We included 18 studies (14 papers and 4 conference abstracts) in the meta-analysis [1, 2, 5, 8,9,10,11, 15,16,17,18, 35,36,37,38,39,40,41]. The 18 studies (9 multicentre and 9 single centre) included 1,191,178 patients. The characteristics of the included studies are shown in Table 1. Study size ranged from 296 to 263,082 patients. Study duration varied between 5 months and 9 years. ICU admission periods spanned 1994–2014. Nine papers reported both mortality and readmission, seven reported mortality only and two readmission only.

Definitions of out-of-hours varied, starting between 16:00 and 22:00 and ending between 05:59 and 09:00. Two studies [2, 37] performed more than one analysis using different definitions of out-of-hours. As 13 of the other 16 studies defined out-of-hours as commencing between 18:00 and 22:00, we selected the definition starting between these times for inclusion in this analysis (Table 1). All studies presented data for the same time periods at the weekend as in the week.

Five of seven studies that compared illness severity between in-hours and out-of-hours discharges found significantly higher severity of illness at admission in the out-of-hours group (Table 2) [1, 2, 8, 10, 35]. Different measures of illness severity were used, preventing pooling of data. Two of eight studies that compared age between in-hours and out-of-hours discharge found significant differences (both Australasian studies finding patients discharged at night to be slightly younger) [1, 8]. None of the five studies that compared gender between in-hours and out-of-hours found significant differences. The absence of data in many of the included studies, combined with the different measures of illness severity used prevented post hoc analysis to investigate whether differences between in-hours and out-of-hours populations accounted for differences in outcome.

We included 16 studies containing data on 927,046 patients in the mortality analysis. Figure 2 shows the association between out-of-hours ICU discharge and mortality. The pooled relative risk estimate for discharge at night (95% CI) was 1.39 (1.24, 1.57), p < 0·0001. Out-of-hours discharge was associated with significant increases in in-hospital mortality for all definitions of out-of-hours (supplementary material, Fig. 1). Overall heterogeneity was high (I² statistic 90.1%), mainly arising from differences in the size (rather than the presence and direction) of the effect in studies defining out-of-hours commencing 18:00–21:59.

Fig. 2

Forest plot of the association between out-of-hours discharge and mortality

The effect of out-of-hours discharge on mortality remained for four of the five geographical areas: UK [relative risk (RR) 1.41 95% CI 1.27, 1.57]; Australasia (RR 1.65, 95% CI 1.40, 1.94); Europe (RR 1.38, 95% CI 1.08, 1.76); and United States of America with South America and Canada (RR 1.31, 95% CI 1.23, 1.40). Asia included only one small study and found no effect (RR 0.41, 95% CI 0.10, 1.63) (supplementary material, Fig. 2). Discharge out-of-hours remained significantly associated with subsequent in-hospital mortality in six of eight included studies that undertook multivariate analysis (Table 3) [1, 2, 8, 10, 15, 35].

We included 11 studies, including 1,156,904 patients in the ICU readmission analysis. Figure 3 shows the association between out-of-hours discharge and readmission to an ICU. The pooled risk estimate for discharge out-of-hours (95% CI) was 1.30 (1.19, 1.42). Heterogeneity was high (I² statistic 90.2%). Heterogeneity arose from differences in effect size rather than the presence or direction of effect [42].

Fig. 3

Forest plot of the association between out-of-hours discharge and ICU readmission

The effect of out-of-hours discharge on readmission remained when analysed for studies in Australasia (RR 1.18, 95% CI 1.09, 1.28), Europe (RR 3.02, 95% CI 2.41, 3.79) and United States of America with South America and Canada (RR 1.14, 95% CI 1.07, 1.21). The effect in the UK was borderline (RR 1.42, 95% CI 1.00, 2.02) (supplementary material, Fig. 3).

Table 3 shows studies that adjusted for potential confounders. We show the confounders used (for which there was no consensus). The summary adjusted odds ratio (95% CI) for mortality was 1.33, (1.30, 1.36), p < 0.001. For comparison, the unadjusted odds ratio was 1.33, (1.28, 1.62), p < 0.001. Analysing only studies that adjusted for potential confounders reduced heterogeneity (the eight studies reporting adjustment tended to be larger studies). One study undertook multivariate adjustment for readmission (out-of-hours discharge remained significant) [38]. We were unable to perform planned sub-group analyses of discharge destination and palliation status due to inconsistent reporting of these data. Too few studies in each group meant we were unable to perform sub-group analysis of out-of-hours definition.

Funnel plots and Egger’s regressions for the effect of out-of-hours discharge on mortality and readmission are shown (supplementary material, Figs. 4, 5, 6, 7). Both funnel plots and Egger’s regression suggest there may be some publication bias whereby studies showing a strong association between mortality and out-of-hours discharge, particularly smaller studies, are not published (p = 0.014). This was not as obvious for studies of readmission (p = 0.057), but this may have been due to a smaller sample of studies.

Quality assessment findings are shown in supplementary material, Table 2. Most studies scored well, between seven and nine out of nine. However, only five studies defined whether the two patient groups were discharged from ICU to a ward or higher dependency area. To assess the influence of each study on bias, we omitted each study in turn (supplementary material, Figs. 8, 9). Removal of any individual study did not remove the effect for either mortality or readmission. The largest effect for mortality occurred when removing a study including the majority of Australasian ICUs [1] reducing the RR (95% CI) to 1.36 (1.26, 1.47). For ICU readmission, both funnel and regression plots supported removal of two major outliers [9, 38]. Removing the outliers reduced the heterogeneity but did not significantly change the RR (supplementary material, Figs. 6, 7, 8, 9).

Discussion

To the best of our knowledge, our meta-analysis brings together the available data on the effects of out-of-hours discharge from intensive care on subsequent hospital mortality and ICU readmission for the first time.

We included 18 studies enrolling 1,191,178 patients. We found that out-of-hours discharge is associated with around a 41% increase in subsequent in-hospital mortality and a 30% increased risk of deteriorating to require readmission to an ICU. The effects persisted across different healthcare systems in different geographical areas and across different definitions of out-of-hours.

Strengths and weaknesses

We used a clearly defined, peer-reviewed protocol to conduct this review [21] to ensure a rigorous process and to minimise concerns regarding internal validity. A major strength of our study is that we demonstrate that out-of-hours discharge is associated with both an increased risk of death and an increased risk of readmission.

Our findings are consistent across large numbers of patients, different healthcare systems and different definitions of out-of-hours.

For both mortality and ICU readmission, our meta-analyses showed substantial heterogeneity. However, the heterogeneity mainly lies in how large the effects are rather than whether effects are present. Some heterogeneity was explained by differences in the definition of out-of-hours. Stratifying for different out-of-hours definitions decreased heterogeneity in some groups, but the association with mortality remained for all groups. Heterogeneity was lowest when we summarised the adjusted odds ratios for the eight studies reporting them. The association with mortality remained present. The reported duration of the out-of-hours period varied between 9 and 16 h. As a consequence, patients discharged at 20:30 are classified as out-of-hours in 12 studies and in-hours in 5 studies. Although inconsistent, investigators may differ in out-of-hours definitions because of differences in what time services change in their healthcare system. We chose, along with all the included authors, to treat out-of-hours at weekends in the same way as weekdays.

Our findings are limited by the cohort design of all the included studies. However, in the absence of any controlled studies, they provide the best available evidence. The limited data provided by many studies restricted our ability to explore underlying causes. Some were not primarily focused on the effects of out-of-hours discharges and so did not report population characteristics by discharge time. Where disease severity was reported by discharge time, studies differed in the assessment used. Where reported, most studies excluded patients discharged to another healthcare facility [5, 9, 11, 17, 18, 35, 37, 40, 41]. This introduces a potential bias if outcomes by discharge time differ in this group.

Despite being widely used in published systematic reviews, there is disagreement within the literature as to the appropriateness of scoring systems to assess study quality [43,44,45]. We used the NOS [24] rather than the ACROBAT-NRSI tool [43] proposed in our protocol. We selected the NOS as a frequently used and more suitable tool for the database cohort studies included. Studies commonly failed to define whether patients discharged in or out-of-hours were discharged to a ward or to high dependency area. This tended to occur in large multisite studies where high dependency facilities were not reported. As only two (small) studies scored less than seven, we did not undertake the planned sub-group analysis according to study quality.

There remains a risk of other studies not reporting outcomes related to out-of-hours discharge where no effect was seen. However, both the funnel plots and Eggers regression for mortality suggest publication bias against publishing studies with high out-of-hours discharge mortality rates. This could be explained by a reluctance of single centres to publish data associated with a perceived poor practice of out-of-hours discharge with high mortality. Our meta-analysis may therefore under- rather than over-estimate effects associated with out-of-hours discharge. As a result, our findings of increased mortality and increased ICU readmission associated with out-of-hours discharge appear robust.

Comparison to what is previously known

Prior to our meta-analysis, it remained unclear whether out-of-hours discharge was associated with increased mortality and readmission rates. Many studies that alone were not large enough to show a statistically significant effect (and could have been misinterpreted as there being no effect) contributed to our overall findings [9, 11, 16,17,18, 37,38,39,40,41]. One previous meta-analysis of the association between time of discharge from ICU and hospital mortality has been undertaken [46]. However, this meta-analysis did not include substantial amounts of relevant information, meta-analysing only 14 studies (of which we excluded 4 as they contained data duplicated within larger studies [19, 29,30,31] and did not study ICU readmission. The absence of registration or a published peer-reviewed protocol may in part explain these weaknesses.

Where increased post-ICU mortality or readmission has been associated with out-of-hours discharge, there has been debate as to whether out-of-hours discharge is causally associated with worse outcomes or simply defines a patient group who are more at risk [12, 14]. Differences in disease severity at admission to ICU have convincingly been shown not to explain the excess mortality and readmission rates found with out-of-hours discharge [1, 2, 8, 13, 15, 35]. The presence of treatment limitations at ICU admission also did not account for the effect [1]. In contrast, two studies [19, 36] corrected out an increased risk of out-of-hours discharge found on univariate analysis by including steps in their statistical model that corrected for factors suggestive of differences in care [continuing therapies such as dialysis and parenteral nutrition, decreased conscious state, increased therapeutic intervention scores (TISS) scores]. The findings are not contradictory; rather, the question asked is different. The first approach suggests that the worse outcomes associated with out-of-hours discharge are not explained by differences in patients at the point of admission to ICU (baseline covariates). The second provides differences in the care pathway that help explain the worse outcomes. It is whether these differences are of significance when a patient is discharged out-of-hours that is of interest, rather than any concept that crossing the threshold of an ICU between particular hours is causally associated with mortality. The combination of the findings of both types of study suggest that differences in management of patients discharged out-of-hours in part explain the worse outcomes seen. Other findings support this idea. Goldfrad and colleagues [2] found only 44.1% of discharges at night were fully ready for discharge in comparison to 86.3% in the day. Premature discharge was the main determinant of increased mortality associated with night-time discharge in their model. There is also evidence that patients with a high treatment need are disproportionately discharged at night [13]. Patients discharged out-of-hours also have higher severity of illness on their last ICU day [10, 11] and a greater incidence of treatments normally delivered in ICUs [19]. All of these might suggest that premature discharge in part explains our findings. One recent study [19] suggests that premature discharge is not a factor; however, the incidence of documented premature discharge is so low in comparison to previous studies [2], and the incidence of delayed discharge so high, that the meaning of this is unclear. Studies to date differ as to whether including differences in the presence of treatment limitations in statistical models corrects out the increased risks of out-of-hours discharge [1, 11, 15, 19, 36]. It seems unlikely that these patients would commonly benefit from readmission to an ICU.

Meaning

Our study resolves the question of whether out-of-hours discharge is associated with worse outcomes. The association with increased mortality and with readmission is substantial. Only the magnitude of the association remains somewhat uncertain. The retrospective non-randomised nature of all the studies undertaken prevents the attribution or non-attribution of causation (both for the studies and our analysis). Whether these increases in mortality and readmission result from patient differences, differences in care or a combination remains unclear. Our meta-analysis does, however, resolve a key area of debate. Disproportionate out-of-hours discharge of patients who will no longer benefit from intensive care cannot explain the increased mortality, as more of this group are readmitted.

Future work

Further work is required to explain why out-of-hours discharge is so strongly associated with post-ICU mortality and readmission, and how these adverse outcomes can be addressed. Future studies should aim to measure and account for confounders appropriately. To investigate the question of whether discharge out-of-hours results in poor outcomes studies should account for patient factors such as age and admission illness severity, which are not altered by differences in treatment in or after the ICU. To investigate the causes of differences found, studies should measure and account for differences in care. These would include measures of ICU exposure (length of ICU stay), illness severity at the point of discharge and ongoing care requirements and quantification of post-discharge care. The contribution of post-discharge care to differences in out-of-hours outcomes has so far only been explored to a limited extent in a single study [19], despite clearly being key to further understanding the problem [12].

Conclusion

Out-of-hours discharge from an ICU is associated with substantial increases in subsequent in-hospital mortality and ICU readmission. These risks remain across different healthcare settings, different geographical areas and different definitions of out-of-hours.