Background

Intensive care unit (ICU) readmissions pose an important clinical problem because they are associated with patient harm, inefficiencies and higher costs [1–4]. Patients readmitted to the ICU experience more adverse events, with in-hospital mortality rates up to six times higher than non-readmitted patients [5]. Readmitted patients reduce ICU bed availability and it is possible that ICU facilities could be used more efficiently if ICU readmissions could be prevented [1–4].

Risk factors for ICU readmission and in-hospital mortality following ICU discharge include patient characteristics, such as age, co-morbidities and severity of illness [5, 6], and organisational factors, such as discharge time and the availability of step-down facilities [5–8]. A substantial amount of variation in patient outcomes between hospitals may be explained by the organisation of the ICU discharge process [9], which consists of four essential components: decision making, planning and preparation, patient transport and follow-up. The ICU discharge process is a complex process in which many healthcare professionals are involved [10]. Deficits in communication, coordination of care, and information exchange between ICU and general ward professionals [11–13] may increase the risk of a suboptimal handover, severe adverse events, ICU readmissions and mortality [14]. Patients discharged from the ICU are particularly vulnerable to poor handovers due to the complicated physiology [15] and the substantial decrease in monitoring when these patients are transferred from the ICU to a general ward [16, 17].

Several methods and instruments are available which aim to improve the quality of the discharge of ICU patients to general wards, such as a liaison nurse and handover forms [18]. Evidence of the effectiveness of these interventions, however, is limited [18, 19] and the actual use of ICU discharge practices vary between ICUs [20].

Variation in ICU readmissions and in-hospital mortality after ICU discharge between hospitals may indicate potential for improvement and be explained by the ICU discharge practices which have been implemented. Insight into associations between ICU discharge practices and patient outcomes can provide evidence for professionals on ways to improve their ICU discharge process, and possibly, reduce adverse patient outcomes.

The aims of this study were: (1) to describe variation in rates of ICU readmissions within 48 h and post-ICU in-hospital mortality in individual hospitals; (2) to describe current ICU discharge practices in Dutch hospitals; and (3) to study the association between ICU discharge practices and rates of ICU readmissions within 48 h and post-ICU in-hospital mortality. We hypothesized that the implementation of ICU discharge practices would be associated with lower rates of ICU readmissions and lower rates of post-ICU in-hospital mortality.

Methods

The design of the study was pre-specified and published [21].

Patient data and outcomes

The Dutch National Intensive Care Evaluation (NICE) registry collects demographic, physiological, clinical and organizational data from ICUs. To ensure that the data are of a high quality, ICU employees are trained in how to score patients, the data are checked before being read into the database, and data quality audits are carried out [22–25].

We used data from the NICE registry on ICU admissions, for reasons other than cardiac surgery, between 1st January and 31st December 2011. We did not examine admissions following cardiac surgery, because cardiac surgery is only performed in a small number of hospitals in the Netherlands and these patients have a low risk of ICU readmission or post-ICU in-hospital mortality [1, 26]. We excluded admissions, in which the patient died during the initial ICU admission or was discharged from the ICU and hospital simultaneously, because these patients were not at risk for ICU readmission or post-ICU in-hospital mortality. We also excluded admissions not fulfilling the Acute Physiology and Chronic Health Evaluation (APACHE) IV inclusion criteria [26] and with missing data on type of admission, reason for discharge, APACHE physiology score, APACHE reason for admission or discharge location (Additional file 1).

We defined an initial ICU admission as a patient’s first ICU admission within a single hospital stay and an ICU readmission as the first ICU readmission within 48 h of the initial ICU discharge, but within the same hospital stay. We choose a time frame of 48 h, as readmissions within this period have a stronger relationship with ICU interventions, such as mechanical ventilation, and discharge circumstances, than later readmissions [27]. We defined post-ICU in-hospital mortality as the death of the patient after the initial ICU admission ended, but before he or she was discharged from the hospital.

ICU discharge practices

Members of an expert panel, consisting of one internal medicine consultant, two intensive care consultants and two researchers, selected eight ICU discharge practices described in scientific literature and clinical guidelines [18, 19, 28–36] to examine in this study. We present these eight practices in Table 1. They were the use of: (1) ICU discharge criteria [29, 30]; (2) a bed manager [31, 32]; (3) early discharge planning [33] (4) step-down facilities [28, 30]; (5) medication reconciliation [32, 34]; (6) verbal and written handover [28, 30, 32]; (7) monitoring of post-ICU patients [36]; and (8) consulting ICU nurses [35]. We extracted data on the use of step-down facilities from the NICE registry. We collected data on the use of the other seven ICU discharge practices using an online questionnaire (Additional file 2), sent to all Dutch ICUs in May 2012. We sent reminders after 9 days and after 3 weeks and contacted the non-responding ICUs by telephone a month after initially sending out the questionnaire. We transformed the data on the use of the eight discharge practices into dichotomous variables to indicate the presence or absence of a discharge practice on a specific ICU (Additional file 3). We summed the eight dichotomous variables into a combined practice score, representing the number of discharge practices incorporated into the discharge process in each ICU.

Table 1 ICU discharge practices
Full size table

Statistical analyses

We calculated the standardised readmission and post-ICU mortality rates for each hospital by dividing the observed number of readmissions or deaths by the expected number of readmissions or deaths. The expected number of readmissions or deaths was the sum of the predicted probabilities of readmission or death obtained from separate prediction models. Readmission rates were corrected for ICU level (in which level 1 are the least and level 3 the most advanced ICUs), age, cardiovascular insufficiency, cirrhosis, hematological malignancy, cardio vascular accident, medical or surgical admission type, planned admission, mechanical ventilation in the first 24 h of admission, chronic renal insufficiency, chronic dialysis, chronic obstructive pulmonary disease, respiratory insufficiency, neoplasm, immunological insufficiency, gastrointestinal bleeding, acute renal failure, confirmed infection, vasopressors, and logit transformed APACHE IV mortality probability [26]. Mortality rates were corrected for ICU level (in which level 1 are the least and level 3 the most advanced ICUs), age, cardiovascular insufficiency, cirrhosis, hematological malignancy, cardio vascular accident, medical or surgical admission type, planned admission, mechanical ventilation in the first 24 h of admission, chronic renal insufficiency, chronic dialysis, chronic obstructive pulmonary disease, respiratory insufficiency, neoplasm, immunological insufficiency, gastrointestinal bleeding, acute renal failure, confirmed infection, vasopressors, diabetes, cerebrovasculair accident, CPR, dysrhythmia, and logit transformed APACHE IV mortality probability [26]. We assessed the discrimination of the prediction models using the area under the receiver operating characteristic (ROC) curve [37] and the calibration using the Hosmer-Lemeshow goodness-of-fit statistic Ĉ with 10 groups [38]. We presented the standardized rates in funnel plots with 95 and 99.8% control limits. We obtained the control limits under the assumption that the natural logarithms of the standardized rates follow a normal distribution [39]. ICUs outside the control limits can be interpreted as deviating significantly from the national rates.

We analysed the univariate association between ICU readmission and post-ICU in-hospital mortality and the eight ICU discharge practices using generalized estimation equations with a logit link function and robust variance estimators [40], while correcting for patient factors. We applied the Bonferroni correction to correct for multiple testing [41], and hence viewed the association between a ICU discharge practice and ICU readmission or post-ICU in-hospital mortality if p-value < 0.0056 (0.05/9). We performed the statistical analyses using IBM SPSS Statistics and R 2.13.0.

Results

We extracted 59,181 first admissions to ICUs in 82 hospitals from the NICE registry (Fig. 1). We excluded 17,141 (Additional file 1) and included 42,040 admissions (71.0%) when calculating standardised readmission and post-ICU mortality rates. The ICUs were in six (7.3%) university hospitals, 29 (35.4%) teaching hospitals, and 47 (57.3%) general hospitals. We present the patient characteristics in Table 2.

Fig. 1
figure 1

Flowchart of patients. NICE: national intensive care evaluation; ICU: intensive care unit; APACHE: acute physiology and chronic health evaluation

Full size image
Table 2 Patient characteristics
Full size table

Rates of ICU readmissions and post-ICU in-hospital mortality

We found a crude ICU readmission rate of 2.9% (1,216/42,040). The standardized rates varied between 0.14 and 2.7 with, by definition, an overall target rate of 1.00. In Fig. 2, we present a funnel plot of the standardized rates of ICU readmissions against the number of ICU admissions per ICU in 2011. In total, 65 (79.3%) ICUs fall within the 95% control limits, three (3.7%) above the upper, and 14 (17.1%) below the lower 95% control limits. One (1.2%) hospital falls above the upper and two (2.4%) hospitals fall below the lower 99.8% control limits. The calibration (Ĉ = 18.1, p-value = 0.0205) and discrimination (area under the ROC curve = 0.63) of the standardization model for ICU readmissions were poor.

Fig. 2
figure 2

Standardized rates of ICU readmission within 48 h. Readmission rates were corrected for ICU level (in which level 1 are the least and level 3 the most advanced ICUs), age, cardiovascular insufficiency, cirrhosis, haematological malignancy, cardio vascular accident, medical or surgical admission type, planned admission, mechanical ventilation in the first 24 h of admission, chronic renal insufficiency, chronic dialysis, chronic obstructive pulmonary disease, respiratory insufficiency, neoplasm, immunological insufficiency, gastrointestinal bleeding, acute renal failure, confirmed infection, vasopressors, and logit transformed APACHE IV mortality probability [26]

Full size image

We found a crude hospital mortality rate of 6.7% (2,811/42,040). The standardized rates ranged between 0.1 and 2.1. In Fig. 3, we present a funnel plot of these rates against the number of ICU admissions in 2011. Five hospitals (6.1%) have an adjusted post-ICU in-hospital mortality rate above the upper and nine (11.0%) below the lower 95% control limits. Four hospitals (4.9%) fall below the lower 99.8% control limits. Although the discrimination (area under the ROC curve = 0.82) of the standardization model for post-ICU in-hospital mortality was good, the calibration (Ĉ = 38.9, p-value < 0.0001) was poor.

Fig. 3
figure 3

Standardized rates of post-ICU in-hospital mortality. Mortality rates were corrected for ICU level (in which level 1 are the least and level 3 the most advanced ICUs), age, cardiovascular insufficiency, cirrhosis, haematological malignancy, cardio vascular accident, medical or surgical admission type, planned admission, mechanical ventilation in the first 24 h of admission, chronic renal insufficiency, chronic dialysis, chronic obstructive pulmonary disease, respiratory insufficiency, neoplasm, immunological insufficiency, gastrointestinal bleeding, acute renal failure, confirmed infection, vasopressors, diabetes, cerebrovasculair accident, CPR, dysrhythmia, and logit transformed APACHE IV mortality probability [26]

Full size image

ICU discharge practices, ICU readmission and post-ICU in-hospital mortality

We had data from the NICE registry and a completed questionnaire for 75 ICUs (Additional file 4). To study the association between ICU discharge practices and ICU readmissions and post-ICU mortality, we excluded 3542 admissions to the non-participating ICUs. Hence, we used data on 38,498 admissions (65.1%) to 75 ICUs in the analyses on associations between ICU discharge practices and ICU readmission and post-ICU in-hospital mortality (Fig. 1). These ICUs were in six university hospitals (8.0%), 28 tertiary medical teaching hospitals (37.3%), and 41 general hospitals (54.7%). We present the results from the questionnaire in detail in Additional file 4.

Table 3 shows the percentages of ICUs, which had implemented each of the discharge practices, and the odds ratios of the univariate association between the implementation of each discharge practice and ICU readmission and post-ICU in-hospital mortality. Following the Bonferroni correction (p < (0,05/9) = p < 0.0056), none of the ICU discharge practices nor the total number of ICU discharge practices implemented by each ICU were associated with the standardized rates of readmission or mortality.

Table 3 Rates of individual practices and odds ratios of univariate association with patient outcomes
Full size table

Discussion

The objective of our study was to describe variation in ICU readmissions within 48 h and post-ICU in-hospital mortality and to study the association of these patient outcomes with the implementation of ICU discharge practices. Using funnel plots, we found that 20.8% of the ICUs fell outside the 95% control limits and 3.6% outside the 99.8% control limits with respect to ICU readmission and 17.1 and 4.9% with respect to post-ICU in-hospital mortality. The substantial proportion of ICUs with standardized readmission or mortality rates falling outside the control limits, suggests that there is more variation between hospitals on these patient outcomes than would be expected and that there is room for quality improvement. The ICUs with lower readmission or mortality can be viewed as best practices and ICUs with higher readmission or mortality could benefit from their views and experiences. The extent of this variation is consistent with that reported in studies on ICU length of stay and mortality [42, 43]. To study and to possibly explain the found variation, we subsequently studied the implementation of ICU discharge practices and their association with the occurrence of IC readmissions and post-ICU in hospital mortality rate.

In this study, we also found that ICU discharge practices vary. We had hypothesized that such variation could indicate that the ICU discharge process could be optimized and, hence, potentially improve patient outcomes and reduce healthcare costs [44, 45]. However, we were unable to demonstrate an association between ICU discharge practices and rates of ICU readmissions or post-ICU in-hospital mortality. In addition, implementing a higher number of ICU discharge practices was not associated with better patient outcomes. Results of previous studies reporting about the association between the use of patient safety and quality improvement practices, and patient safety outcomes are diffuse. Some studies showed that compliance to discharge practices was associated with lower hospital complications and mortality rates [46, 47], while others showed no association [48].

An important strength of this study is our large dataset covering more than 90% of all Dutch ICUs. In addition, we included the APACHE IV mortality probability in our case-mix correction models. Currently, the APACHE IV is the best performing model for case-mix correction for in-hospital mortality following ICU admission in the Netherlands [25, 26].

Our study has some limitations. We strived to minimize the effects of case-mix differences between ICUs by presenting case-mix adjusted standardized rates for quantifying variation in patient outcomes. However, our case-mix correction models have not been externally validated and, in our dataset, the calibration of the models for ICU readmission and post-ICU mortality and the discrimination of the model for ICU readmission were poor. This means that these models may not adequately correct for case-mix differences between hospitals, potentially resulting in more hospitals than expected falling outside the control limits [49]. In addition, the variation between the rates of ICU readmission and post-ICU in-hospital mortality may still result from chance [50], overdispersion [39, 51], an incorrect method for determining the control limits [52], or registration problems within the hospitals.

The number of admissions included in our analyses for some ICUs was very low. Therefore, even when using funnel plots, there is a low probability of detecting that these ICUs are performing differently from national rates [53].

We found no significant association between ICU discharge practices and patient outcomes which may be due to several limitations of our study. First, the power to detect a reduction in post-ICU mortality and ICU readmission rate was limited because we measured each of the discharge practices at hospital level and, although the response rate to the questionnaire was 91.5%, the number of ICUs was limited. Furthermore, some practices were present in almost every ICU. Second, the use of discharge practices were measured using a self-reported questionnaire, which may be susceptible to bias. Overestimation of own practices and socially desirable answers could have influenced our findings. However, Scholle and colleagues found only minor overestimation in their study and concluded that self-assessment could be useful for quality improvement purposes [54]. Third, in our regression models we used patient data, such as severity of illness and the APACHE IV reasons for ICU admission, measured at the time of ICU admission. Ideally, data representing the patient’s condition at the time of ICU discharge would be used. However, these data are not available in the NICE registry. Fourth, we had no data on whether patients were discharged from the ICU for palliative care on the ward. This could have led to an overestimation of the mortality rates.

Clinical handover has been identified as a key process in improving quality of care and patient safety and reducing adverse patient outcomes [55, 56]. Quantification of variation is a tool for uncovering suboptimal quality of care and may identify potential for improvement [57–59]. We found both variation in patient outcomes and in discharge practices and reasoned that this indicates potential for improving patient outcomes and subsequently, reducing healthcare costs [44, 45]. However, we were not able to identify a relation between ICU discharge practices and patient outcomes. Further research is necessary to find factors, which may influence these patient outcomes, in order to improve quality of care. For example organisational factors, such as staffing and experience and skills of (ICU) personnel. Unfortunately, we were not able to include them in our research due to the lack of data of these factors. Exploratory research into the differences between the hospitals falling above the upper and below the lower control limits in our funnel plots may give insight into factors influencing quality of care.

Conclusion

Causes of ICU readmissions and post-ICU in-hospital mortality are likely to vary between hospitals. Although interventions to reduce the rates of these events have been described in the literature, our study shows that none of them are associated with better outcomes in the Netherlands. Examining individual ICU readmissions or post-ICU in-hospital mortalities locally may provide ICUs insight into potential areas for improvement in their own ICU discharge process.