Introduction

During the last few years there has been increasing discussion about the factors that determine the quality of care provided in hospitals during so called “off-hours”. It is known that adequate care during the first hours is of major importance for the outcome of patients admitted to intensive care units (ICUs) [1, 2]. In many ICUs, however, there is no 24-h in-house attendance of senior staff. A significant effect of admission time on the risk-adjusted mortality of patients has been found in some adult ICUs without 24-h coverage by senior staff [3–5]. This has led to an increasing trend towards 24-h ICU coverage by in-house intensivists in adult intensive care medicine [6].

Studies in pediatric intensive care medicine are sparse and have yielded contradictory results. Some authors have found off-hours admission to a pediatric ICU (PICU) to be associated with higher mortality rates, whereas other authors have found off-hours mortality rates to be equal to or even lower than office hours mortality rates [7–9].

We hypothesized that the direct on-call availability of senior staff during off-hours would not result in higher risk-adjusted PICU mortality rates when compared to in-house attendance of senior staff. Therefore, we compared the risk-adjusted mortality rates of patients admitted non-electively during office hours with those of patients admitted non-electively during off-hours in two level III PICUs without 24-h in-house attendance of senior staff.

Methods

Data from two level III PICUs in tertiary referral children’s hospitals in the Netherlands were prospectively collected from the dataset of the national audited Pediatric Intensive Care Evaluation (PICE) database. The Institutional review board waived the need for informed consent.

Both PICUs are situated in university teaching hospitals, in Utrecht and Groningen, and have multidisciplinary and comparable patient populations. The total number of beds in the PICUs was 14 and 10, respectively. Data registration was performed by trained staff and medical students. Demographics, medical history, mortality prediction score items, Australia and New Zealand Pediatric Intensive Care (ANZPIC) Registry diagnostic codes, ventilation days and admission and discharge information were anonymously collected from the database and used to calculate the risk-adjusted predicted mortality according to the Pediatric Index of Mortality-1 (PIM1) and the Pediatric Risk of Mortality-2 (PRISM2) algorithms [10–12].

All patients from 0 to 18 years of age who were non-electively admitted to one of the two PICUs between January 2003 and December 2007 were included. Non-elective admission was defined as an admission that could not be postponed for more than 6 h. Office hours were defined as week days (Monday to Friday) from 8:00 a.m. to 6:00 p.m. Off-hours were defined as week days from 6:00 p.m. to 8:00 a.m., Saturdays, Sundays and public holidays.

Both PICUs had a comparable staffing model. During office hours, senior staff were present in the units together with two residents with an average of 4 years experience in pediatrics, with no duties other than PICU-related patient care. During off-hours, one experienced resident was covering the PICU with no duties other than PICU-related patient care. The senior staff member on duty during off-hours would see all unstable patients and personally supervise all admissions and critical procedures including endotracheal intubation and insertion of central venous and arterial lines, but would not routinely be present at the PICU. The senior staff member would be able to come into the PICU within 15 min. Most of the time, the senior staff member on duty was a pediatric intensivist. Other times, a pediatrician, pediatric anesthesiologist, pediatric pulmonologist or cardiologist with experience in critical care was on duty with a pediatric intensivist on standby. An extra senior staff member was on standby in case of emergencies or when a patient needed to be transported. Throughout the study period there were no changes in the staffing models of medical or nursing staff.

The primary endpoint of this study was PICU mortality. By dividing the observed mortality by the risk-adjusted predicted mortality, standardized mortality rates (SMRs) were calculated. Secondary endpoints were ventilator days and length of stay in the PICU.

Statistical analysis was performed using SPSS version 16.0.2. (SPSS, Chicago IL). Independent t tests were performed to compare continuous variables and chi-squared tests for dichotomous variables. Skewed data are reported as medians with interquartile ranges. The Wald approximation by Rothman and Greenland [13] was used for calculation of 95% CIs for the SMRs. Multivariable logistic regression analysis was performed to evaluate the differences between the two centers as to the effect of off-hours admission on mortality. In this analysis, mortality was the dependent variable. Along with center of admission, PIM1 and PRISM2 variables were entered into two separate analyses. In the multivariable logistic regression analysis using PIM1, postoperative status and age in months were added as independent variables as well, since these variables are part of the PRISM2 algorithm, but not of the PIM1 algorithm. A p value <0.05 was considered statistically significant.

Results

Between January 2003 and December 2007, 3,212 out of a total of 5,672 patients were non-electively admitted to the PICUs of Utrecht and Groningen. Of these patients, 2,122 (66%) were admitted during off-hours. The patient characteristics are summarized in Table 1. Patients admitted during off-hours were slightly older than those admitted during office hours. More postoperative patients were admitted during office hours. During off-hours, a lower percentage of patients were admitted because of hemodynamic instability (17% vs. 20%, p = 0.02) and a higher percentage because of (multi)trauma (12% vs. 6%, p < 0.001). Both the numbers of patients admitted and the patient characteristics of the two PICUs were very similar.

Table 1 Patient characteristics
Full size table

The mortality observed in the off-hours group was 8.5% and in the office hours group 7.9%. The overall mortality predicted by the PRISM2 algorithm was higher than that predicted by the PIM1 algorithm (Table 1). The PIM1-based SMR was 1.04 (95% CI 0.89–1.20) for the off-hours group and 0.99 (95% CI 0.80–1.22) for the office hours group (Table 2). The PRISM2-based SMR was 0.69 (95% CI 0.60–0.80) for the off-hours group and 0.72 (95% CI 0.59–0.90) for the office hours group.

Table 2 SMRs and 95% CI for all non-elective patients
Full size table

Multivariable logistic regression analyses with mortality as the dependent variable showed no significant effect of off-hours admission on mortality (Table 3). A significant effect of postoperative status on mortality was seen in the PIM1 analysis as an odds ratio of 1.59 (95% CI 1.02–2.50). Apart from that, mortality was only significantly affected by the prediction scores. No center-specific effect on mortality was found in the regression analyses.

Table 3 Logistic regression analyses, mortality as dependent variable
Full size table

There was a wide variation in the secondary endpoints ventilator days and length of stay, but no difference was found between the off-hours and office hours groups (Table 1). Redoing these analyses, with Saturdays, Sundays and public holidays from 8:00 a.m. to 12:00 p.m. defined as office hours, or with inclusion of elective patients, yielded essentially identical results (data not shown).

Discussion

In this prospective 5-year study conducted in two level III PICUs, in non-electively admitted patients direct on-call availability of senior staff during off-hours did not result in higher SMRs than when senior staff were available in-house during office hours. Mortality was only affected by the mortality prediction scores PIM1 and PRISM2, and by the postoperative status of the patients. Off-hours admission did not result in an increased number of ventilator days or an increased length of PICU stay.

Previously conducted studies on the effect of off-hours in-house attendance of senior staff on pediatric intensive care mortality used various definitions of “off-hours” and yielded contradictory results. Arias et al. [7] found higher mortality rates in specific groups of patients admitted to 15 different PICUs in the US during off-hours. However, no information about availability of staff during these hours was provided. Hixson et al. [8] reported no significant effect of off-hours admission on mortality in 5,968 patients admitted to a single PICU with continuous in-house attendance of senior staff, and concluded that patients benefit from 24-h in-house attendance of senior staff. Numa et al. [9] studied 4,456 non-elective pediatric patients admitted to one PICU without 24-h coverage by an in-house intensivist and found lower risk-adjusted mortality rates and shorter lengths of stay for admissions during off-hours. Our results are in line with those reported by Numa et al. [9], and may reflect the presence of senior staff at the time of admission of a patient during off-hours. In our PICUs, when admission of a severely ill patient is being announced, the resident will immediately contact the senior staff member on call so that a senior staff member is present when patients with a medium or high risk of mortality are admitted.

In postoperative patients, PIM1-based prediction of off-hours’ mortality was significantly higher than office hours mortality. This might be have been due to the fact that, in the PIM1 algorithm, the postoperative status is not part of the scoring system, in contrast to the PRISM2 algorithm.

Our study has several limitations. In the first place, our results might have been influenced by our decision to include only non-elective patients and to treat Saturdays, Sundays and public holidays as off-hours. We assumed that the concepts of the so-called golden hour and of early goal-directed therapy would apply most to non-elective PICU admissions [1, 2, 14]. Elective admissions, on the other hand, constituted only a small proportion of all off-hours admissions; most were an overrun of procedures scheduled for office hours, and had a low predicted mortality rate. Indeed, when we included elective admissions in the analyses, the results were essentially identical (data not shown). Similarly, expanding the definition of office hours to include the morning rounds of Saturdays, Sundays and public holidays when senior staff would as a rule be present in the PICU, did not change the results (data not shown).

Secondly, we used the older PIM1 and PRISM2 algorithms to calculate SMRs, not their successors (PIM2 and PRISM3). At the beginning of the study period in 2003 the Dutch PICUs had not yet decided to include PIM2 in the PICE dataset [15, 16]. Therefore, PIM2 scores were not available for the whole study period. PRISM3 scores were not used because of the high cost of obtaining the algorithm. Although the calibration characteristics of the PIM1 and PRISM2 algorithms might be inferior to those of their successors, both algorithms have good discriminative characteristics, are still widely used, and have been validated for use in the Netherlands [17–19]. PRISM2 has been used in all Dutch PICUs from 2003 and consistently overpredicted mortality, as in other studies [20, 21]. Keeping these limitations in mind, however, the validity of our findings is supported by the fact that both PIM1 and PRISM2 yielded the same results.

Finally, the level of cover by senior staff during off-hours in our units is high, as senior staff will see all unstable patients and supervise admissions and critical procedures. Moreover, requiring senior staff to be able to come into the PICU within 15 min when on call imposes limits on home-to-work travelling time. This may limit the applicability of our findings in other settings than ours. Therefore, our results should be extrapolated to other PICUs with caution. We do not wish to imply that 24-h in-house attendance of senior staff is redundant or that on-call availability of senior staff will lead to enhanced outcome in PICUs. More studies are needed to establish the effects of different staffing models on the outcome of PICU patients. These studies should focus on other outcome measures such as complications, adverse events, patient and parent satisfaction and morbidity, since mortality is only a rough measure of outcome. Finally, other aspects of different staffing models such as cost effectiveness and feasibility should be endpoints of such studies.

Conclusion

Off-hours admission to our PICUs without 24-hour in-house attendance of senior staff was not associated with higher SMRs.