Introduction

Ideally, care for critically ill patients is optimal, 24 h per day. Unfortunately, this is not the case. During so-called off hours, staffing is often reduced, and diagnostic and therapeutic procedures might take longer or are postponed until office hours. Because treatment in the first hours after admission to the intensive care unit (ICU) is related to outcome [1, 2], admission outside office hours might be associated with increased mortality [3, 4].

However, this increased mortality outside office hours has been questioned by several other studies [5–10], suggesting that the association was confounded by differences in the definition of office hours, differences in study population, study size, healthcare organisation or insufficient correction for case mix and illness severity. As a consequence of these differences in methodology and organisational differences between countries, it remains unclear whether there is a relationship between admission outside office hours and increased hospital mortality.

We hypothesised that mortality during off hours is higher than during office hours and that previous studies that did not find this result were probably underpowered or defined office hours too broadly. We therefore analysed hospital mortality in relation to admission time in a very large database with stringent case-mix correction. We also analysed whether sample size could have influenced the results of previous studies. This approach enables us to analyse the relationship between admission time and hospital mortality in the most robust way thus far.

Methods

Patient data

Since 1996 the National Intensive Care Evaluation (NICE) foundation has collected data on admissions to ICUs in The Netherlands. NICE started with 6 ICUs in 1996, and in 2008 more than 70 ICUs were participating in this registry, accounting for 80% of all Dutch ICUs, and approximately 50,000 admissions were included in this registry in 2008. Details about inclusions and exclusions in the Dutch registry have been published previously [11]. In short, the participating ICUs are mixed medical-surgical units located in university hospitals (n = 7), teaching hospitals (n = 25) or non-teaching hospitals (n = 38). A data set of about 100 items is collected for each individual patient. Based upon this data several prediction models can be calculated, such as APACHE II [12], which are used to correct the crude hospital mortality for illness severity at admission.

Data collection takes place in a prospective and standardised manner according to strict and uniform definitions and is subject to stringent data quality checks. Additionally, site visits are performed to ensure the quality of the collected data. This has been shown to ensure high quality of data [13]. The data is aggregated at a central point (NICE Foundation, Amsterdam, The Netherlands). All patient identifying information, such as name and patient identification number, is encrypted. Data are analysed and stored in an anonymous way and are not traceable to any patient, and therefore informed consent was not needed.

All patient records in this database between 2002 and 2008 were included in our analysis. According to the original APACHE II exclusion criteria, cardiopulmonary surgical patients, re-admissions to the ICU, patients who were discharged or died within 8 h after admission and burn patients were excluded.

Defining office hours

Office hours were defined by the presence of a fully qualified intensivist available for patient care. In The Netherlands, three levels of intensive care are identified. The minimal demands for the three levels of intensive care are available in the ESM. Despite organisational differences, commonly intensivists are available from 08:00 to 22:00 hours. Therefore, we defined 08:00–22:00 hours during weekdays and 09:00–18:00 hours during the weekends as office hours. A weekday was defined as a day from 00:00 hours until 24:00 hours, and the weekend was defined as from Saturday 00:00 hours until Sunday 24:00 hours.

Data analysis

Hospital mortality by day of week, in or outside weekend and during office hours versus off hours were examined first using univariate analysis and then with multiple logistic regression adjusted for case mix. Adjustment for case mix was undertaken by using the Dutch APACHE II model. To date, there have been no studies that have shown that prognostic models are stable over time, in a new setting, and with different case mixes [14, 15]. As a result of medical progress and advancement of science, it is expected that the model’s performance will decline over time [16]. To account for this decline in performance we recalibrated the APACHE II before using it for case-mix correction. This involved fitting a new logistic regression equation with in-hospital mortality as dependent variable and the original covariates (APACHE II score, APACHE II reason for admission and admission type) as independent variables. Accordingly, customisation does not change the influence of individual covariates included in the model but modifies their joint influence on the observed mortality in the external dataset [17–19].

Three different recalibrated logistic regression models were used to analyse the association between in-hospital mortality and off hours versus office hours (model 1), weekdays versus weekend (model 2) and day of week (model 3). For better risk adjustment, patients admitted during office hours versus off hours, weekend versus weekday, and day of the week were matched based on a calculated propensity score. This score expresses the probability that a patient falls in the office hours or off hours, weekday or weekend, or day of week group, and its log odds was added as a covariate in the recalibrated APACHE II model for adjustment.

When the incidence of an outcome of interest is common in the study population (>10%), the adjusted odds ratio derived from the logistic regression could under- or overestimate the risk ratio. Therefore, the associations between admission time and mortality are reported as relative risk and their 95% confidence interval (95% CI). The relative risks were approximated from the adjusted odds ratio derived from the three logistic regression models [18] and considered statistically significant if the confidence interval did not contain zero.

Categorising a continuous variable (admission time) into office hours and off hours may conceal the behaviour of the variables over time. Therefore, the relationship between admission time and hospital mortality was also graphically inspected using locally weighted scatterplot smoothing (LOWESS). In simple terms, at each time point, e.g. 15:00 hours, simple models are fitted to the mortality data in the “vicinity” of the point, e.g. from 13:00 to 17:00 hours, with points closer to 15:00 hours having more influence than those farther away. This procedure is performed for all data points. Instead of the scatter plot of 0 and 1 values of mortality over time we now have a smoothed curve that reflects the underlying structure of the data.

To investigate whether smaller sample size could explain the results of previous articles on this subject that showed no relation between mortality and time of admission, we used 300 random sub-samples from our research dataset with sample size equal to that in two previous articles: 56,250 and 6,725 patients [6, 10]. Data were analysed using the R2.6.2 statistical environment and SPSS 16.0 (Chicago, IL, USA).

Results

From January 2002 to January 2009, a total of 149,894 patients eligible according to the APACHE II inclusion criteria were included in the analyses. Baseline characteristics of these patients are shown in Table 1. Table 2 shows the distribution of patients according to admission day, weekend versus weekdays and off hours versus office hours. This table shows that the predicted and observed mortality of patients during the weekends and off hours is higher than during the week and office hours, respectively.

Table 1 Baseline characteristics of the study population (N = 149,894)
Full size table
Table 2 Characteristics of the patients admitted per time variable (N = 149,894)
Full size table

Figure 1 shows the actual observed mortality and the predicted mortality probability of the recalibrated APACHE II model in relation to admission time. The lowest line shows the difference between the observed and predicted mortality proportion. This figure shows qualitatively that the APACHE II model predictions closely follow the observed mortality proportion over all admission times. The difference between the observed and predicted mortality is minimal (the lowest line is almost horizontal) except for the hours close to midnight. The lowest mortality is seen during office hours, especially during the afternoon. Then, mortality slowly increases again, with a peak around 05:00–06:00 hours in the morning. The figure shows that the observed proportion minus the predicted probability during office hours was lower than during off hours (17.5% versus 22.7%). In the whole off hours region the difference between observed and predicted mortality was positive (meaning that there was more mortality than predicted), which was not the case in the office hours region. This observation provides face validity for treating office hours versus off hours as a dichotomous variable in the logistic regression model. The graph also suggests that the effect of off hours on mortality would have been even stronger if off hours had been defined as, say, 20:00–08:00 hours.

Fig. 1
figure 1

Relationship between observed and predicted mortality (in percentages) in relationship to admission time. The predicted mortality is based upon the APACHE II model and observed mortality. The APACHE II model has been recalibrated to better fit the Dutch ICU population (see “Methods” section for explanation). The upper lines are overlapping, which shows that the model correctly predicts mortality in the general Dutch ICU population. The difference between the upper lines is minimal (almost horizontal lower line). To illustrate the pattern of the difference between the lines, the lower figure blows up this difference. Both the model as well as the observed mortality change with admission time. The lowest mortality is seen during office hours (08:00–22:00 hours), and the highest mortality is seen during off hours (22:00–08:00 hours)

Full size image

Table 3 shows the relative risk and 95% confidence intervals of the three different models, investigating the days of weeks, weekend versus weekdays and off hours versus office hours after case-mix correction. Admissions during off hours have a significantly higher mortality risk [RR 1.059 (1.031–1.088)] than admissions during office hours. The same holds for admissions during the weekend [RR 1.103 (1.071–1.136)] in comparison with weekdays, although the mortality risk is already increased for patients admitted on Friday [RR 1.046 (1.001–1.092)].

Table 3 Relative risk for hospital mortality during off hours, the weekend and during the week
Full size table

We also analysed whether smaller sample size would have influenced our results. The 300 random sub-samples from our research dataset based on the largest sample size among studies reporting negative findings (N = 56,250) [6] showed a significant difference between office hours versus off hours in 85% of our samples. However, the analyses on 300 random sub-samples based on the smallest sample size among studies reporting negative findings (N = 6,725) [10] only showed significantly higher mortality during off hours in 15% of samples, and significantly higher mortality during office hours in 1% of samples.

Discussion

This study showed an increase in the risk of hospital mortality for patients admitted during off hours compared with patients admitted during office hours (RR 1.059), and an increase of hospital mortality risk for patients admitted during the weekend compared with patients admitted during the week (RR 1.103).

Several analyses that describe the difference in mortality between patients admitted during office hours and those admitted to the ICU during off hours have been published [5–10, 20–23]. Unfortunately, all of these studies defined office hours differently. We defined working hours as those hours when a qualified intensivist was available for direct patient care. For most ICUs in The Netherlands this is from 08:00 to 22:00 hours. This definition is in accordance with another recent Dutch publication [10]. However, our results contradict some of these more recent publications on this subject. For example, Meynaar et al. [10] analysed the difference in mortality of 6,725 patients admitted during office hours and outside office hours. They could not detect a difference in mortality after correction for case mix and illness severity. As their sample size of only 6,725 patients was much smaller than our research sample, we used 300 random sub-samples from our research dataset to replicate their sample size. We found a statistically significant difference in only 49 of 300 samples (15%). Although the sub-samples are overlapping, this suggests that their sample size might have been too small to detect these differences in mortality. However, they analysed only three ICUs located in teaching hospitals, which are possibly much more homogenous in performance over time compared with our mixed set of 70 participating ICUs.

On the other hand, our results also contradict the largest analysis thus far [6]. In a UK database of 56,250 ICU patients, Wunsch et al. found increased mortality for the weekends (Friday–Sunday) and during the evening and night. However, after correction for case mix, this difference disappeared. They concluded that there was no difference in outcome between office hours and off hours. However, they defined office hours differently, choosing three shifts (08:00–18:00, 18:00–24:00 and 24:00–08:00 hours) that best reflect ICU care in the UK. An analysis on 300 random sub-samples from our dataset sample with their sample size (n = 56,250) showed a significant difference in mortality between office hours and off hours in 256 of the 300 samples. Although the sub-samples are overlapping, this suggests that the difference in conclusion is not likely based upon limited power in the study by Wunsch et al. We corrected in a similar way for potential confounders, and therefore our analyses are comparable. This suggests that the increased mortality in our study might be based on differences in staffing or logistics between the UK and The Netherlands.

Of course, ICU performance is influenced by an intricate interplay of various factors. Besides the ICU organisation during off hours there is intensive interaction with other medical disciplines, and changes in their quality of care during off hours might influence ICU outcome as well, which is not reflected by illness severity at admission. Such unknown confounders might be stronger in smaller hospitals, which often have less staff to fill the roster and/or have less sophisticated diagnostics than larger hospitals. Furthermore, the performance of health care workers (physicians and nurses) varies during the day. Although speculative, the detrimental effect of the circadian biorhythm on human performance during the night shift and especially at the end of the night shift is a known factor [24]. We also found the highest (predicted and observed) mortality at the end of the night shift (05:00–06:00 hours), when both health care workers and patients perform at their worst.

This study has several limitations. Although the data were collected in prospective fashion during the first 24 h of admission to the ICU and the outcome (discharged alive or dead) was not influenced by subjective assessment, this remains a retrospective analysis. Therefore, true cause-and-effect relationships cannot be ascertained, and unmeasured confounders still might play a role. For example, differences of care beyond the ICU might be confounders in the association between admission timing and mortality. However, all surviving patients are discharged to the same wards, and it is reasonable to assume that they all experience the same quality of care on the wards.

Although APACHE II was used to correct for illness severity and patients were matched based upon a propensity score (age, gender, APACHE score, admission type, reason for admission), this still does not fully exclude the influence of case-mix differences. Surgical patients admitted in the middle of the night are different from patients admitted during office hours. For example, does waiting for emergency surgery during off hours result in higher acute physiology score and subsequently to higher APACHE II score (so-called lead-time bias)? Other, possibly very important, confounders might be differences in organisational aspects. Unfortunately, the data in this study could not be corrected for these organisational aspects, and we assumed office hours to be 08:00–22:00. Figure 1 shows that there is an increase in the difference between observed and predicted mortality from 16:00 hours onwards. The effect of off hours on mortality would have been stronger if off hours had been defined as, say, 20:00–08:00 hours. As of 2008, all ICUs have collected data on quality parameters, such as nurse-to-patient ratio, physician-to-patient ratio, availability of ICU beds, etc. Such information might explain the differences between ICUs and explain why some ICUs apparently have equal performance during the entire day while others perform worse during off hours.

However, a strong feature of this analysis is its size and its power to detect these differences. This is one of the largest analyses of admission timing and survival. Additionally, this analysis is performed in a large sample of the Dutch ICUs (up to 80% of the Dutch ICUs in 2008), and therefore these results can be extrapolated to represent the level of critical care in The Netherlands. Previous studies were often performed in a smaller subset of ICUs and might represent the better-performing ICUs. Such decreased external validity might explain a lack of difference between office hours and off hours in these studies.

We conclude that admission timing is associated with differences in outcome, even when mortality is corrected for illness severity by means of recalibrated APACHE II score. Patients admitted during the night (22:00–08:00 hours) or weekend days have a decreased chance of survival in comparison with patients admitted during office hours. However, the cause of this association needs further analysis that corrects for more potential confounders. This investigation has been started in The Netherlands with the registration of various quality-of-care variables.