Background

Length of stay is frequently used as a measure of ICU resource use, but there is no uniform definition of what constitutes a prolonged ICU stay [1, 2]. A simplistic method uses the mean along with the standard deviation of length of stay for a population to assign a boundary of two standard deviations above the mean. But this is unsatisfactory, because the distribution of ICU length of stay is left-censored at zero and is heavily skewed to the right by a "tail" that represents patients with longer stays [3]. A prolonged ICU stay has also been subjectively defined by designating a specific duration of stay e.g., ≥ 10 days, ≥ 14 days, ≥ 21days, ≥ 30 days [4–7]. Another method for defining a prolonged stay is to visually examine the distribution of ICU stay in a population and identify a threshold for the "tail" that represents long stay patients [3, 4].

Despite differences in definition, studies have repeatedly shown that a small percentage (7% to 11%) of lengthy ICU admissions account for a large proportion (40% to 50%) of resource use [8–10]. Because patients with a prolonged ICU stay consume a disproportionate amount of resources, their early identification can assist in improving unit efficiency. This is because identifying these individuals early can improve patient throughput by signalling a need for discharge planning or exploration of care alternatives. These alternatives might include palliative care consultation [11], early mobility therapy [12, 13], transfer to an in-hospital chronic ventilator unit [14], or discharge to a long-term acute care facility [15, 16].

We have previously described models for predicting overall ICU length of stay using day 1 patient data [10, 17, 18]. Although these models provide accurate benchmarks for assessing the efficiency of ICU throughput, the accuracy of prediction is reduced for patient groups with a prolonged ICU stay. This reduced accuracy has been attributed to uncertainty about prognosis, subsequent complications, and variations in response to therapy [10]. The same factors account, at least in part, for the reduced accuracy of hospital mortality prediction using ICU day 1 data [19, 20].

Because models that include daily physiologic measures during therapy have improved the accuracy of daily mortality predictions [21–24], we examined whether a similar approach might improve predictions of ICU stay, particularly when length of stay is prolonged. Therefore, this study had the following objectives: 1) To identify a simple yet relevant clinical threshold for concern about a prolonged ICU stay. 2) To compare the characteristics of patients who stay less than or longer than the threshold for concern. 3) To describe the development and validation of a model for predicting ICU stay remaining after the threshold. 4) To compare the accuracy of ICU length of stay predictions on day 1 vs. a model that includes data collected on the threshold day.

Methods

Patient Cohort

Study data were retrospectively collected for consecutive admissions to U.S. intensive and coronary care units from January 1, 2002 to December 31, 2007. The hospitals and ICUs in this study were selected because each had installed an Acute Physiology and Chronic Health Evaluation (APACHE) computerized data collection and analysis system that recorded patient data on ICU days 1 to 7. Details about the characteristics of these hospitals, ICUs, and patients have been described elsewhere [20, 25].

ICU Length of Stay

ICU stay was measured using the exact interval (in minutes) between the day and time of ICU admission and ICU discharge, and converted back to fractional days (e.g. 3.12 days, 6.45 days). Exact ICU stay was used because previous studies have demonstrated superior accuracy compared to measurement using calendar days [10, 26, 27].

A frequency histogram of ICU length of stay was generated and a day selected to represent a clinical threshold when clinicians might be concerned that a patient will have a prolonged ICU stay. The threshold was chosen because: 1) it represented enough time to reflect complications and response to therapy for the patient; and 2) the proportion of patients still in the ICU at this point was approximately equal to the upper quintile for length of stay. Once the threshold day was selected, we developed and validated a model for predicting ICU stay remaining after the threshold day.

Patient Data Collection

All patient data were generated as a result of patient care and entered on site using a software program that included computerized pick lists, automated error checking, and calculation of physiological means. Demographic and physiological data were entered via electronic interfaces with laboratory and clinical information systems. Data collected for each patient are shown in Table 1. Detailed descriptions of these demographic, clinical and physiological items have been previously reported [10, 20, 28].

Table 1 Information collected on all patients who remained in the intensive care unit (ICU) on day 5.
Full size table

Data collection procedures were based on prior reliability studies [29] and field experience [30, 31], and details about data entry have been described elsewhere [10, 20]. Based on contractual agreements between Cerner Corporation and each hospital, data were stripped of patient identifiers in full compliance with the Health Insurance Portability and Accountability Act (HIPAA) requirements. Informed consent was not obtained because these processes were identical to those that resulted in prior Institutional Review Board waivers [28].

We excluded patients admitted to ICUs that did not collect data during days 1 through 7. We also excluded patients who had been admitted for < 4 hours, patients with burns, and patients < 16 years of age. Patients admitted from or discharged to another ICU were excluded because of inability to determine their total ICU stays. We also excluded patients admitted after coronary artery bypass surgery because their ICU stay was generally short with a small range compared to other patients.

Model Development and Validation

We developed a multivariable linear regression model to predict ICU stay remaining after the threshold day. A backwards elimination approach was used with a marginal p < 0.05 necessary to remain in the model. The model was intended for use in ICUs with advanced health information technology and the capability for automated data collection. The predictor variables in the regression model were collected on ICU day 1 and on the threshold day, and are displayed in Table 1. In addition, we used the change in Acute Physiology Score (APS) from the day before to the day of the threshold value. These variables were pre-selected based on previous research [10, 20, 22].

We extended the age, APS, and prior length of stay variables by including restricted cubic spline terms. Splines allow estimation of a non-linear relationship between a variable and ICU length of stay and replace less accurate techniques that assume a linear relationship [32, 33].

The length of ICU stay was truncated at 30 days for patients with an ICU stay >30 days. We did this because a few outliers with an extremely long ICU stay can markedly distort a length of stay analysis [4]. The use of truncation at 30 days is also supported by prior studies [10, 17, 18, 27, 34], and by visual examination of the distribution of ICU stays for the study patients.

The equation for predicting ICU stay remaining after the threshold day was developed and internally validated using admissions from 2002 through 2005; 50% of these admissions were randomly allocated to be in the development data set and the remaining 50% were used in the internal validation data set. Because predictive models tend to be over specific for the database used for their development [35, 36], we externally validated the model's accuracy using data for ICU admissions during 2006-2007. Inclusion and exclusion criteria and methods for assessing predictive accuracy were identical to those used for internal validation.

To determine which factors had the greatest impact on the model's predictive power, we separated total model variation into its components. This was accomplished by obtaining the sequential sum of squares due to each variable and then calculating their respective percentage of the total model's sums of squares.

Assessment of Model Accuracy

We used several methods to assess the accuracy of aggregate predictions of ICU stay remaining after the threshold day [37]. First, we assessed the degree of correspondence between mean observed and mean predicted remaining ICU stay. A paired Student's t-test was used to assess the null hypothesis that the mean residual between observed and predicted remaining ICU stay was zero. Second, we calculated a coefficient of determination (r2) to measure the percentage of overall variability captured by the model for individual patients and across ICUs, respectively. Given the skewness of the data, we also examined the correlation between mean observed and mean predicted values using Spearman's rho. Third, we examined calibration by graphically displaying mean observed and mean predicted remaining ICU stay across 5% quantiles of observed values. Finally, we examined the model's accuracy when stratified by vital status at ICU discharge.

To test the utility of the model for prolonged ICU stay we compared the difference between mean observed and mean predicted total ICU stay based on the APACHE IV ICU day 1 model [10] and the predicted remaining ICU stay added to the threshold number of days (threshold day + predicted remaining ICU stay).

Results

Data were collected in 138 ICUs at 54 hospitals from January 1, 2002 through December 31, 2007. When developing the predictive model we excluded 39 ICUs that did not collect daily physiological information, leaving data from 83 ICUs and 31 hospitals for analysis. Table 2 gives the characteristics of these hospitals and ICUs. For the 2002-2005 data set, 29 ICUs met inclusion criteria but did not collect data in 2006-2007. For the 2006-2007 data set, 6 ICUs met inclusion criteria but did not collect data in 2002-2005.

Table 2 Characteristics of hospitals and intensive care units that collected daily physiologic data
Full size table

There were 380,953 ICU admissions from 2002 through 2007. Of these admissions, 13,311 (3.5%) were excluded because the patient was either admitted from or discharged to another ICU, and 2,201 (0.6%) because there was no discharge location information. A further 21,886 (5.7%) patients were excluded because they were admitted after coronary artery bypass surgery. This left 343,555 patients for determining a threshold value.

Threshold for concern about a prolonged ICU stay

The distribution of ICU length of stay for the 343,555 admissions that met inclusion criteria is shown in Figure 1. The mean ICU stay was 4.21 days and the median was 2.14 days, which is indicative of substantial skewness in length of stay. We identified ICU day 5 as the clinical threshold for concern about a prolonged ICU stay, as it allowed sufficient time to reflect complications or response to therapy and represented the 80th percentile. An ICU stay > 30 days corresponded to the 99th percentile, which delineates extreme outliers and confirms our choice for using it to truncate ICU stay.

Figure 1
figure 1

Distribution of intensive care unit length of stay for 343,555 patients admitted from 2002 through 2007.

Full size image

Figure 2 displays the percentage of ICU admissions and total ICU days respectively by ICU stay ranges. Admissions staying ≤ 5 days contributed 79% of all admissions but only 37% of all ICU days. Conversely, ICU stays > 30 days occurred 1% of the time, but resulted in 12.5% of all ICU days. Ranges of ICU stay between these two boundaries show that with an increasing ICU stay the percentages of total ICU days increase.

Figure 2
figure 2

Duration of intensive care unit (ICU) stay and its association with total ICU bed days. Grey bars = % of ICU admissions. White bars = % of overall ICU days.

Full size image

Factors associated with an ICU stay < 5 days vs. ≥ 5 days

Patients with an ICU stay at or above the threshold of concern (≥ 5 days) were slightly less frequent in western U.S. and non-teaching hospitals; differences were small across ICU type. Table 3 compares the characteristics of admissions with an ICU stay <5 days compared to those with an ICU stay ≥ 5 days. Patients with an ICU stay at or above the threshold of concern (≥ 5 days) had a significantly higher day 1 mean Acute Physiology Score, APACHE IV predicted ICU stay on day 1, hospital stay before ICU admission, incidence of emergency surgery, ICU readmission, active treatment, and mechanical ventilation. Patients with a ≥ 5 day ICU stay were also more likely to have a chronic health condition, admission from the floor, other hospital, or a step down unit.

Table 3 Characteristics of patients with an intensive care unit (ICU) length of stay below (<5days) vs. above (> 5 days) a clinical threshold for concern about a prolonged ICU stay.
Full size table

Table 3 also shows the five largest diagnostic categories for patients being discharged before ICU day 5 vs. patients remaining in the ICU on day 5 or longer. With the exception of congestive heart failure, the two groups had different diagnoses with the highest incidence being bacterial pneumonia, which accounted for 5.2% of all ICU stays ≥ 5 days. In general, diagnoses associated with an ICU stay ≥ 5 days were characterized by a tendency for less rapid improvement, complications, and prognostic uncertainty. A more detailed list of these diagnoses is shown in Table 4.

Table 4 Coefficients for the main effect variables* in the APACHE IV multivariable regression model that predicts remaining intensive care unit (ICU) stay after day 5.
Full size table

Figure 3 shows the cumulative incidence of ICU length of stay for four common diagnostic categories. For a large proportion of patients admitted for upper gastrointestinal (GI) bleeding ICU stay tends to be short. In contrast, the proportion of longer ICU stays is progressively larger and more skewed for patients admitted with multiple trauma, surgery for GI perforation and pulmonary sepsis, respectively. Table 5 shows the outcomes for patients being discharged before ICU day 5 vs. patients remaining in the ICU on day 5 or longer. The latter were more likely to have adverse outcomes and less likely to be discharged home.

Table 5 Outcomes for patients with an ICU stay below (< 5 days) vs. at or above threshold for concern (> 5 days) about a prolonged ICU stay.
Full size table
Figure 3
figure 3

Cumulative percentage of admissions across intensive care unit length of stay (truncated at 30 days). The diagnoses include Gastrointestinal Bleeding (GI Bleeding), Pulmonary Sepsis, Multiple trauma (MULTITRAUM), and surgery for Gastrointestinal Perforation (SGIPERF).

Full size image

Regression model to predict remaining ICU length of stay

There were 37,447 patients with an ICU stay ≥ 5 days that met inclusion criteria and were admitted to ICUs that collected daily data. They were divided as follows: 12,640 in the development data set, 12,904 in the internal validation data set, and 11,903 in the external validation data set. A multivariable linear regression model was developed to predict the outcome "length of stay remaining after ICU day 5". In aggregate the model included 99 variables: 90 of which were collected on ICU day 1, eight on ICU day 5, and one variable that reflected physiological changes from day 4 to day 5. The predictor variables and their multivariate relationship with this outcome are shown in Table 4.

For the 12,640 patients in the development data set mean predicted remaining ICU stay after day 5 was 6.87 days and mean observed ICU stay remaining after day 5 was also 6.87 days. The model had an r2 = 20.2% across individuals and 44.3% across units. Spearman's rho was = 0.494 across individuals, analogous to an r2 = 24.4%. The relative contribution of the variables used to predict ICU stay remaining after ICU day 5 is shown in Figure 4. ICU day 5 data accounted for 81.6% of the models overall explanatory power; day 5 mechanical ventilation for 41.4%, PaO2: FiO2 for 17%, other day 5 physiologic variables included in the APS score for 15.8%, and inability to assess Glasgow coma score due to sedation or paralysis for 7.4%. The remaining variables combined contributed 18.4% of the model's variation.

Figure 4
figure 4

Percentage of model variance attributable to factors used to predict intensive care unit length of stay remaining after day 5. Physiology refers to the acute physiology score (APS) and rescaled Glasgow Coma Score. Rescaled PaO2/FiO2 ratio is defined in Appendix Table 1. GCS unable refers to inability to assess Glasgow Coma score due to sedation or paralysis. Diagnostic category includes 57 mutually exclusive diagnostic groups on day 5. Miscellaneous includes age, prior length of stay, emergency surgery, and ICU readmission.

Full size image

For the 12,904 patients in the internal validation data set mean observed remaining ICU stay was 6.87 days; mean predicted remaining ICU stay on day 5 was 6.85 days, a difference of < 1 hour. The mean residual value was not significantly different from zero (p > 0.05). For the 11,903 patients in the external validation data set, mean observed remaining ICU stay was 7.19 days and mean predicted remaining ICU stay on day 5 was 6.58 days, a difference of 14.6 hours (p < 0.001). When applied to the external data set the model had an r2 = 18.2% across individuals and 43.3% across units; Spearman's rho was 0.486 across individuals (roughly equivalent to an r2 = 23.6%). Figure 5 shows a calibration curve of observed length of stay over 5% quantiles of predicted values. Observed values deviate from predicted values mainly for the first two quantiles (lowest 10% of predicted values) and at two quantiles in the middle.

Figure 5
figure 5

Calibration curve comparing mean observed and mean predicted intensive care unit (ICU) length of stay remaining on ICU day 5. The 11,903 patients in the 2006 to 2007 external validation set are divided into 20 equal-sized groups. The dotted line indicates perfect predictive ability (i.e. the observed means match the predicted means perfectly).

Full size image

Utility of the model for predicting lengthy ICU stays

Comparison of mean observed and mean predicted total ICU stay using the APACHE IV ICU day 1 model versus the predicted remaining ICU stay added to the threshold number of days (5 days + predicted remaining ICU stay) demonstrates the usefulness of the model for predicting lengthy ICU stays. Figure 6 shows the mean ICU length of stay values for the development, internal validation, and external validation data sets, respectively. In each data set the sum of the day 5 prediction + 5 days was much closer to the observed ICU stay than the day 1 prediction. For the external validation data set, mean predicted total ICU stay was 11.58 days and mean observed ICU stay was 11.99 days, a difference of 9.7 hours (p < 0.001); using the day 1 prediction the difference between observed and predicted ICU stay was 149.3 hours (p < 0.001).

Figure 6
figure 6

Comparison of observed and predicted intensive care unit (ICU) length of stay. Mean observed (ICU) length of stay (white bar), mean predicted length of stay based on the day 5 model [5 days + predicted remaining length of stay after day 5] (gray bar), and mean predicted length of stay based on day 1 model (black bar).

Full size image

Figure 7 shows the mean observed and mean predicted remaining ICU stay stratified by ICU discharge vital status. There is close corroboration between observed and predicted mean ICU stay, regardless of whether or not the patient was discharged alive.

Figure 7
figure 7

Comparison of mean observed and mean predicted intensive care unit (ICU) length of stay remaining after day 5 among survivors and non-survivors at ICU discharge. Black, died: Observed ICU stay. Gray, died: Predicted ICU stay. Green, alive: Observed ICU stay. Yellow, alive: Predicted ICU stay.

Full size image

Discussion

This study presents a model for predicting a prolonged length of stay for ICU patients. Because there is no uniform definition for a prolonged ICU stay, we identified a clinical threshold for concern about a prolonged ICU stay. We selected a ≥ 5 day threshold based on the study population's distribution of ICU length of stay and time needed to reflect a patient's early clinical course.

There were distinct differences between patients with an ICU stay < 5 days versus those with an ICU stay ≥ 5 days. Patients with an ICU stay ≥ 5 days had significantly higher severity of illness, frequency of mechanical ventilation, emergency surgery, and ICU readmission. Patients with an ICU stay ≥ 5 days accounted for 21% of all admissions but 63% of total ICU days; and their outcomes were uniformly poorer.

Based on the above findings, we developed and externally validated a multivariable regression model to predict ICU length of stay remaining after day 5. The variables used in the model were similar to those used in the APACHE IV day 1 model, but also included information captured during day 5. This additional information proved important because day 5 data accounted for 82% of the total predictive accuracy of the equation, leaving only 18% attributable to information collected on day 1.

We believe the model for predicting ICU stay remaining after day 5 accurately identified patients at risk for a prolonged ICU stay for several reasons: First, almost half (48.8%) of the model's explanatory power is accounted for by mechanical ventilation and inability to measure Glasgow Coma score due to sedation or paralysis on day 5. Prolonged (>96 hrs) mechanical ventilation [38, 39] and sedation [40, 41] have previously been associated with increased cost and ICU length of stay. Second, a prediction based in part upon a patient's physiology on day 5 reflects the impact of response to therapy, and or the development of complications, or both during the ICU stay [42, 43]. Third, focusing on patients who remain in ICU for 5 days reduces predictive inaccuracy due to early deaths. Fourth, focusing on patients who remain in ICU for 5 days reduces inaccuracies due to differences in institutional discharge practices and length of stay variations caused by infrequent discharges between 10 pm and 7 am [44].

Previously published models of ICU length of stay developed for use in the U.S. [10, 17, 18, 27, 45], Western Europe [46], and Finland [34] all predict ICU stay on day 1 and have either not been tested or do not accurately predict a prolonged ICU stay. Inaccurate predictions of prolonged ICU stays using day 1 data alone has been attributed to prognostic uncertainty, complications, and variations in response to therapy.

Previous studies of patients with a prolonged ICU stay have also identified mechanical ventilation [6, 45–47], higher severity of illness [5,45.46,47], and persistent physiological abnormalities (multiple organ dysfunction) [6] as risk factors. Infection has also been shown to pose a significant risk for a prolonged ICU stay [5, 6, 45, 47]. Our analysis failed to identify infection because it was not included as a data element. Other previously identified variables that predict prolonged ICU stays including emergency surgery, trauma, prolonged pre-ICU hospital stay, and ICU readmission [5, 6, 45–47] were also used in our model.

Our study has several clinical implications. First, ICU clinicians who do not have access to advanced health information technology can use the most influential risk factors associated with a prolonged ICU stay to identify patients likely to have lengthy stays. Second, ICU clinicians with access to advanced electronic functionalities can use the model as a tool for improving ICU utilization. This is because patients who are likely to have a lengthy ICU stay can be referred for early mobility therapy [12, 13], early discharge planning [48], palliative care consultation [11], or assessment for transfer to a long-term acute care facility [15, 49]. Each of these interventions has been associated with a reduction in ICU stay. Third, the ability to compare case mix adjusted ICU stay can be used to compare the efficiency of resource use across ICUs. Because the APACHE IV ICU day 1 model accurately predicts length of stay across all patients [10], we recommend that it be used as a primary benchmarking tool. The model that predicts ICU stay remaining after day 5, however, provides additional information for benchmarking resource use for patients with prolonged ICU stays.

We do not recommend using this model to predict a prolonged ICU stay for individual patients. ICU day 5 prediction of a lengthy remaining ICU stay, however, can alert ICU clinicians to carefully consider patients within this group as candidates for interventions that might improve resource use.

The most important limitation of the model for predicting a prolonged ICU stay is its complexity. We believe this complexity reflects the large number of factors that determine a prolonged ICU stay. This complexity essentially mandates the use of automated data collection and calculation. Currently, the infrequent availability of advanced health information technology in most hospitals represents a major barrier to the model's widespread use [50]. As more institutions incorporate electronic medical records into their process flow, models such as the one described here can be of great value.

Our results have several additional limitations. First, the model's usefulness is probably limited to the U.S. because of international differences that impact ICU stay. These differences in ICU stay are also likely to adversely impact the use of ICU day 5 as a threshold for concern about a prolonged stay. Second, while capturing physiologic information on day 1 is too soon to account for the impact of complications and response to therapy, day 5 may still be too early to account for their effects. Previous studies indicate that more than half of the complications of ICU care occur after ICU day 5 [42, 51]. Third, despite its complexity, the model fails to account for additional factors known to influence ICU stay. These include nosocomial infection [52, 53], do not resuscitate orders [54], ICU physician staffing [54, 55], ICU acquired paralysis [56], and ICU sedation practices [57]. Fourth, the model's greatest inaccuracy is the under-prediction of remaining ICU stays of 2 days or less (see Figure 5). We speculate that these findings might be explained by discharge delays aimed at avoiding night or weekend transfers [44] or the frequency of complications on ICU days 6 to 8 [51].

Conclusions

A model that uses ICU day 1 and day 5 patient data was developed to predict ICU stay remaining after day 5. This model more accurately predicts prolonged ICU stays than a similar model that uses ICU day 1 data alone. The model can be used for performance benchmarking and as a tool for alerting clinicians to patients who may require early discharge planning.