Advertisement

Intensive Care Medicine

, Volume 41, Issue 6, pp 1029–1036 | Cite as

A clinical prediction model to identify patients at high risk of death in the emergency department

  • Michael Coslovsky
  • Jukka Takala
  • Aristomenis K. Exadaktylos
  • Luca Martinolli
  • Tobias M. MerzEmail author
Open Access
Original

Abstract

Purpose

Rapid assessment and intervention is important for the prognosis of acutely ill patients admitted to the emergency department (ED). The aim of this study was to prospectively develop and validate a model predicting the risk of in-hospital death based on all available information available at the time of ED admission and to compare its discriminative performance with a non-systematic risk estimate by the triaging first health-care provider.

Methods

Prospective cohort analysis based on a multivariable logistic regression for the probability of death.

Results

A total of 8,607 consecutive admissions of 7,680 patients admitted to the ED of a tertiary care hospital were analysed. Most frequent APACHE II diagnostic categories at the time of admission were neurological (2,052, 24 %), trauma (1,522, 18 %), infection categories [1,328, 15 %; including sepsis (357, 4.1 %), severe sepsis (249, 2.9 %), septic shock (27, 0.3 %)], cardiovascular (1,022, 12 %), gastrointestinal (848, 10 %) and respiratory (449, 5 %). The predictors of the final model were age, prolonged capillary refill time, blood pressure, mechanical ventilation, oxygen saturation index, Glasgow coma score and APACHE II diagnostic category. The model showed good discriminative ability, with an area under the receiver operating characteristic curve of 0.92 and good internal validity. The model performed significantly better than non-systematic triaging of the patient.

Conclusions

The use of the prediction model can facilitate the identification of ED patients with higher mortality risk. The model performs better than a non-systematic assessment and may facilitate more rapid identification and commencement of treatment of patients at risk of an unfavourable outcome.

Keywords

Emergency department Prognostic model Triage Vital sign instability Critically ill 

Introduction

In patients admitted to the emergency department (ED), survival is mainly determined by the severity of acute illness on admission [1, 2] and the quality of care provided throughout the entire treatment process [3]. The patient’s deterioration or unexpected death is often preceded by abnormalities in vital signs [4, 5]. The causes leading to deterioration are potentially treatable in a majority of patients and it is reasonable to assume that any intervention will be more efficient if initiated early [6, 7]. Less than optimal care in this setting is due to a failure to recognise signs of organ dysfunction, delayed alarm and subsequent failure to intervene rapidly and adequately [8, 9, 10]. As large EDs admit high numbers of patients, it is unrealistic to expect that all patients can be immediately assessed by an expert physician. In most EDs, patients are first assessed by ED nursing staff [11, 12]. Their subjective impression is used to integrate the available patient history and the first physical examination, and this serves to trigger an expedited evaluation by a physician [13, 14]. However, nurses and physicians agree only moderately when assessing the severity of a patient’s condition [15, 16].

Various scoring systems have been proposed for more systematic initial assessment and simplified identification of patients at risk [17, 18, 19, 20, 21]. The discriminant value of these scores is only moderate and varies between studies [22]. The suboptimal performance of scoring systems is multifactorial. The choice of variables and their weight may be based on clinical experience and intuition [23]. To make scores easier to calculate, simplified correlations between pathological vital signs and outcome are used. Continuous variables, such as blood pressure or heart rate, are stratified into pathological and non-pathological ranges, leading to the loss of some discriminative power. It may be wrongly assumed that there is a linear relationship between abnormal values and the risk of death. Additionally, ED scores usually do not account for the fact that vital signs have already potentially been influenced by supportive treatment—such as vasopressors or supplemental oxygen—at the time of the initial assessment.

The aim of this study was to methodically construct and validate a systematic prognostic model based on all information available at the time of arrival in the ED and to compare the discriminative performance of this newly created prediction model with a subjective risk estimate by the first ED health-care provider triaging the patient. We hypothesised that the model-based risk evaluation would allow the reliable identification of patients at greater risk of mortality and would be superior to a non-systematic, experience-based assessment.

Methods

Study design

Prospective cohort study for the development and validation of a clinical prediction model.

Setting

The study was performed in the Departments of Intensive Care Medicine and Emergency Medicine of the Bern University Hospital, a 960-bed tertiary care centre. The ED provides initial treatment of all adult emergency patients. All ED staff hold a Bachelor of Science in Nursing and around 45 % have completed postgraduate training in emergency care nursing.

Patients

This study includes all patients admitted to the hospital via the ED during 1 year starting from October 2009. Patients remaining in the ED for more than 24 h before being discharged home and patients who died in ED were rated as hospital admissions. Patients treated on an outpatient basis were excluded retrospectively.

Variables and data sources

Study data was collected by triage ED staff from paper medical records and was entered into a study database by research staff. Data included age, gender, time of admission, cause of admission (APACHE II diagnostic category [24]) as established by the ED nurse, vital sign measurements for respiratory rate (current impedance pneumography), oxygen saturation (pulse oximetry), systolic and diastolic blood pressure (sphygmomanometry), heart rate (electrocardiogram), Glasgow coma score (GCS), body temperature and peripheral perfusion (capillary refill time over/under 3 s) and the presence of a threatened airway. The ED staff noted the following interventions that had occurred before the time of ED arrival: endotracheal intubation, mechanical ventilation, ongoing cardiopulmonary resuscitation (CPR), continuous infusion of vasopressors or inotropes, fraction of inspired oxygen (FiO2) or litres per minute of supplemental oxygen and use of sedative drugs. In sedated patients, the last GCS before the start of sedation was included. For comparison purposes (nurse risk estimate), the ED nurse recorded their personal subjective estimate of the likelihood that the patient would not survive this hospital stay, on the basis of all available information on arrival, and employing a scale from 1 (very low risk of death) to 10 (very high risk of death).

Statistical methods

Several variables were transformed before inclusion in the analysis. Saturation index [25] was calculated as the oxygen saturation (SpO2) divided by FiO2. The saturation index was categorised as being under or above the median of all observations (4.2 arbitrary units). The transformation of litres of oxygen per minute to FiO2 was based on current guidelines [26, 27]. Respiratory rate was categorised according to the cut-offs of the APACHE II scoring system. Age and mean blood pressure were included in the model, together with their quadratic terms. Chained equations were used to generate a stochastic single imputed data set for missing observations, utilizing for imputation equations all potential predictors and the outcome.

The model was based on a multivariable logistic regression for the probability of death during the hospital stay. Since the data included multiple admissions for some of the patients (9.3 %), 500 distinct data sets of single patient observations were generated (permuted data sets), and one observation was randomly chosen for each patient with multiple admissions. A logistic regression was fitted to each permuted data set, including all the potential predictors. Stepwise backward elimination was performed, with retention of predictors with p < 0.1 in the Wald test. Predictors maintained in over 67 % of the resulting models were included in the final prediction model. Nagelkerke’s R 2 and the Brier score were used as measures of overall model performance [28, 29]. Model discrimination was measured as the area under the receiver operating characteristic (AUROC) curve and model calibration as the calibration slope for each permuted model and in addition assessed visually. Measures are reported for the final model as the average over all permutations. Bootstrapping was used as the validation method, since no external data source was available. The validation procedure was performed within each permuted data set by sampling—with replacement—300 samples from the permuted data set; these samples were of the same size as the original data set [30, 31]. To assess overfitting, model optimism was assessed by bootstrapping including the variable selection process. As an alternative method for internal validation, a split-sample approach was used for comparing sensitivity (Electronic Supplementary Material).

The performance of the developed model was compared to the performance of the nurse risk estimate as the sole predictor. Likelihood ratio tests were used to compare the fit of a model including the selected variables and the nurse estimation to that of a model with the nurse estimate alone.

The variable selection process was repeated, with the inclusion of the nurse estimate as a potential predictor to test whether it would be selected as being important. Additionally, the DeLong test was used to compare the developed model’s AUROC to that of a model based on the nurse estimate alone.

Continuous variables are summarised as mean ± standard deviation or median and lower (Q1) and upper (Q3) limits of interquartile range for parametric and non-parametric data, respectively. Analyses were performed using Stata 13.1 (Stata Corporation, Texas).

Results

During the study period, there were 8,607 emergency hospital admissions of 7,680 patients (38 % women). A total of 6,965 patients were admitted once and 715 patients were admitted between two and seven times. Most frequent APACHE II diagnostic categories at the time of admission were neurological (2,052, 24 %), trauma (1,522, 18 %), infection categories [1,328, 15 %; including sepsis (357, 4.1 %), severe sepsis (249, 2.9 %), septic shock (27, 0.3 %)], cardiovascular (1,022, 12 %), gastrointestinal (848, 10 %) and respiratory (449, 5 %). Few data were missing apart from breathing rate (16 % of points). Follow-up to hospital discharge was complete in all patients. The mean of patient age was 58 ± 20 years, with ages ranging from 16 to 100 years. In a total of 398 (4.6 %) of all admissions, the patient died during the subsequent hospital stay (Table 1). The median length of hospital stay (LOS) of all admissions was 5.67 days (Q1–Q3, 2.74–10.51). The mean LOS of hospital survivors and non-survivors was 5.74 (Q1–Q3, 2.83–10.57) and 2.77 (Q1–Q3, 0.73–8.95) days, respectively.
Table 1

Parameters determined by patient history, vital sign assessment and treatment at the time of ED admission, stratified by hospital survival

 

N missing

All observations (N = 8,606)

Survivors (N = 8,208)

Non-survivors (N = 398)

Intubation on admission

0 (0 %)

551 (6 %)

387 (5 %)

164 (41 %)

Defibrillation when arriving in ED

0 (0 %)

16 (0 %)

8 (0 %)

8 (2 %)

CPR when arriving in ED

0 (0 %)

51 (1 %)

9 (0 %)

42 (11 %)

Reduced peripheral circulation

0 (0 %)

298 (3 %)

170 (2 %)

128 (32 %)

Respiratory rate APACHE II

1,386 (16 %)

   

  <10

 

107 (1 %)

66 (1 %)

41 (12 %)

  10–11

 

700 (10 %)

643 (9 %)

57 (16 %)

  12–24

 

5,708 (79 %)

5,497 (80 %)

211 (60 %)

  25–34

 

651 (9 %)

611 (9 %)

40 (11 %)

  ≥35

 

54 (1 %)

51 (1 %)

3 (1 %)

MAP

64 (<1 %)

94.5 ± 19.1

95.1 ± 18.2

81.2 ± 30.2

MAP < 50

64 (<1 %)

86 (1 %)

29 (0 %)

57 (14 %)

Use of vasopressors, vasodilators, inotropes at ED admission

0 (0 %)

42 (0 %)

24 (0 %)

18 (5 %)

Heart rate

60 (<1 %)

84.9 ± 20.4

85.0 ± 19.5

82.1 ± 34.1

Heart rate < 40

60 (<1 %)

55 (1 %)

19 (0 %)

36 (9 %)

Mechanical ventilation at the time of ED arrival

0 (0 %)

419 (5 %)

265 (3 %)

154 (39 %)

Threatened airway

0 (0 %)

198 (2 %)

135 (2 %)

63 (16 %)

Saturation index (SpO2/FiO2)

0 (0 %)

   

  <4.2

 

3,088 (36 %)

2,739 (33 %)

349 (88 %)

  ≥4.2

 

5,518 (64 %)

5,469 (67 %)

49 (12 %)

SpO2

0 (0 %)

   

  <90 %

 

322 (4 %)

268 (3 %)

54 (14 %)

  ≥90 %

 

8,284 (96 %)

7,940 (97 %)

344 (86 %)

Temperature >39 °C

0 (0 %)

141 (2 %)

137 (2 %)

4 (1 %)

Temperature <35 °C

0 (0 %)

155 (2 %)

103 (1 %)

52 (13 %)

GCS

0 (0 %)

   

  14–15

 

8,146 (95 %)

7,940 (97 %)

206 (52 %)

  11–13

 

199 (2 %)

176 (2 %)

23 (6 %)

  9–10

 

55 (1 %)

42 (1 %)

13 (3 %)

  6–8

 

61 (1 %)

34 (0 %)

27 (7 %)

  3–5

 

145 (2 %)

16 (0 %)

129 (32 %)

APACHE II diagnostic category at the time of ED admission

0 (0 %)

   

  Respiratory

 

449 (5 %)

424 (5 %)

25 (6 %)

  Cardiovascular

 

1,022 (12 %)

930 (11 %)

92 (23 %)

  Neurological

 

2,052 (24 %)

1,956 (24 %)

96 (24 %)

  Gastrointestinal

 

848 (10 %)

809 (10 %)

39 (10 %)

  Trauma

 

1,522 (18 %)

1,475 (18 %)

47 (12 %)

  Other

 

2,713 (32 %)

2,614 (32 %)

99 (25 %)

Patient admitted to ED in previous 12 months

0 (0 %)

2,002 (23 %)

1,898 (23 %)

104 (26 %)

Continuous variables summarised as means and standard deviations

The vital signs low blood pressure, reduced peripheral circulation, decrease in GCS and a saturation index below 4.2 indicated an increased risk of death, as did higher age, mechanical ventilation on ED arrival and a history of ED visit in the preceding 12 months. Quadratic terms indicating a non-linear association of age and mean blood pressure and the probability of death were retained in the model. In comparison with patients with respiratory problems, patients admitted for cardiovascular or neurological problems or trauma had a lower risk of death and patients with gastrointestinal or non-specified problems had a higher risk of death. Patient gender, intubation, defibrillation and CPR before or at arrival to ED, heart rate, temperature, low or high BP, and threatened airway were not retained in the final model. Table 2 shows the parameters used in the final model.
Table 2

Coefficients of the final model

Variable

Estimate

SE

95 % CI–estimate

OR

95 % CI–OR

Intercept

−3.75

1.32

(−6.33, −1.16)

Age

0.11

0.03

(0.05, 0.17)

1.12

(1.05, 1.19)

Age2

6 × 10−4

2 × 10−4

(−1.1 × 10−3, −1 × 10−4)

0.99

(0.99, 1.00)

Reduced peripheral circulation

1.41

0.24

(0.94, 1.87)

4.09

(2.57, 6.52)

Mechanical ventilation at the time of ED arrival

0.98

0.23

(0.54, 1.43)

2.68

(1.72, 4.18)

Saturation index (SpO2/FiO2)

−1.82

0.20

(−2.21, −1.44)

0.16

(0.11, 0.24)

Patient admitted to ED in previous 12 months

0.49

0.19

(0.13, 0.86)

1.64

(1.14, 2.36)

MAP

−0.07

0.02

(−0.10, −0.04)

0.93

(0.90, 0.97)

MAP2

3 × 10−4

1 × 10−4

(1 × 10−4, 4 × 10−4)

1.00

(1.00, 1.00)

GCS

  11–13

1.31

0.28

(0.76, 1.87)

3.71

(2.13, 6.47)

  9–10

2.12

0.40

(1.33, 2.91)

8.33

(3.78, 18.33)

  6–8

3.16

0.36

(2.45, 3.87)

23.52

(11.57, 47.79)

  3–5

4.53

0.38

(3.79, 5.28)

93.19

(44.27, 196.19)

APACHE II diagnostic category at the time of ED arrival (compared to respiratory condition)

  Cardiovascular

−0.25

0.33

(−0.89, 0.39)

0.78

(0.41, 1.48)

  Neurological

−0.56

0.33

(−1.20, 0.08)

0.57

(0.30, 1.08)

  Gastrointestinal

0.63

0.34

(−0.04, 1.30)

1.88

(0.96, 3.67)

  Trauma

−0.70

0.36

(−1.41, 0.01)

0.50

(0.25, 1.02)

  Other

0.15

0.31

(−0.46, 0.75)

1.16

(0.63, 2.13)

Model performance and validation

The model appeared well calibrated by all indicators (Fig. 1a, b). The model explained approximately half of the observed variation with optimism-corrected mean Nagelkerke’s R 2 at 0.48 (reduced from 0.50, 95 % range 0.494–0.509). The overall optimism-corrected Brier score was 0.028 (corrected from sample’s 0.027, 95 % range 0.026–0.027), suggesting good calibration and high validity. The AUROC of the final model showed good discriminative ability and good internal validity at 0.920 (optimism corrected from 0.922, 95 % range 0.916–0.927; Fig. 2).
Fig. 1

Calibration plot showing a all deciles and b the lower nine deciles of predictions. Predicted probabilities using the model’s coefficients are aggregated to deciles of patients. The mean observed death rate in each decile is the percentage (and 95 % confidence intervals) of observed death from all observations in this decile, and are marked by black triangle and lines. The dashed line indicates the optimal 1:1 fit. The dotted line represents the locally weighted scatterplot smoothing (LOWESS) smoother of the predictions. The calibration slope was 0.95, indicating good calibration

Fig. 2

ROC curve of the final model. Area under the curve (AUROC) was 0.922 (95 % range 0.916–0.927), indicating good internal validity

The nurse risk estimate

The nurse risk estimate model showed inferior performance to the developed model and was by itself not a significant predictor. The nurse risk estimate exhibited low explained variability (mean Nagelkerke’s R 2 = 0.17). The overall calibration and discriminative ability was good (Brier score 0.040, mean AUROC 0.78). The optimism for all measures—AUROC, Brier score and Nagelkerke’s R 2—was negligible (<0.001), as was the shrinkage factor (0.997) calculated from the bootstrap validation. Likelihood ratio tests comparing the developed model with the model of the nurse estimation alone were always highly significant in favour of the developed model (all p < 0.001). A DeLong test of AUROC values of the nurse estimate model compared to the developed model proved highly significant in favour of the developed model (p < 0.001). The correlation between predictions of the developed model and the risk estimate model was relatively low (Spearman’s rho = 0.38, p < 0.001). The nurse risk estimate was not selected as a predictor for the final model when included in the variable selection process for model development. Compared to the developed model, the nurse risk estimate predicted a higher risk of death with a higher degree of variability in patients with mortality below 5 % and a lower risk in patients with highest mortality risk (Fig. 3).
Fig. 3

Comparison of prediction of risk of death of the nurse risk estimate model and the developed prediction model in deciles of patients according to the calibration plot. In the 8 deciles of patients with lower mortality risk the nurse risk estimate consistently predicted a higher risk of death than the developed model with considerable higher variance in prediction. In the 20 % of patients with the highest risk of death, higher mortality risk was predicted by the model than by the nurse risk estimate. N nurse risk estimate model, M prediction model

Discussion

In this prospective cohort study, we found that increased risk of hospital mortality could be predicted in the initial phase after ED arrival on the basis of routinely collected and easily available clinical information. These included patient history, preliminary diagnosis, findings of the first clinical examination and the treatment interventions implemented at the time of ED arrival. The model showed overall good discrimination in identifying patients at risk; it was superior to the subjective risk estimate of the first health-care provider triaging the patient.

The main strength of our study is the use of readily available parameters to assess the mortality risk. The analysed patients represent a cohort originating from a population covering the whole spectrum of emergencies. The sample size was large enough to offer sufficient statistical power to test all examined variables and the prospective nature of the study ensured high data quality and reduced the risk of bias due to confounders.

The main limitations of our study are related to the single-centre design and the lack of an external validation sample. This reduces the generalizability of our model and creates a risk of overfitting the model to the data at hand. In this setting, split-sample or cross validation is sometimes applied. As a result of the inherent limitations of these methods [32], we chose bootstrapping for internal validation. In a split-sample approach used for comparing sensitivity, relevant variables were very similar (Electronic Supplementary Material). A limitation of the bootstrapping method is that model development processes, such as variable selection, must occur in a programmable manner in order to run them repeatedly. Therefore, the necessary categorisation of variables can affect estimation, depending on apparently arbitrary cut-off values. To minimise these effects, we based our categorisation decisions on clinical rationales or on the analysis of the frequency distribution of the variables. We used single and not multiple imputation to account for missing observations, mainly because of the already high complexity of the computational process in creating the model. However, respiratory rate was the only predictor with more than 10 % missing values and was not included in the final model. Hospital mortality was used as the outcome parameter for model development. The use of an alternative predefined outcome assessment period such as 7 or 28 days would have provided additional relevant information. In contrast to many other European countries, such outcome data is not readily available in Switzerland for data protection reasons, and the follow-up was therefore limited to the hospital stay. The complex nature of the model mandates the use of a calculation program which can be obtained from the authors.

Several risk scoring systems for triaging of unselected emergency patients have been described [21, 33, 34, 35, 36, 37, 38, 39], but the discriminative power of the presented model appears to be superior. In contrast to previous models, we solely used information available on admission (i.e. no laboratory tests) and also integrated patient history, preliminary diagnosis and treatments on ED admission. The discriminative power of scoring systems to identify patients with higher mortality reported as AUROC ranges from 0.65 to 0.90 in the validation samples, in comparison to the AUROC of 0.92 in our model [21, 33, 34, 35, 36, 38, 40]. A comparison of AUROC values of previously published scores calculated with our patient data is provided in the Electronic Supplementary Material. Seymour et al. [39] evaluated a model based on out-of-hospital vital sign measurements to predict a combined endpoint of death and occurrence of critical illness during hospitalization. In this model the AUROC was 0.77. Goodacre et al. [21] derived several models predicting 7-day mortality in emergency medical admissions by stepwise inclusion of predictors. The AUROC increased from 0.80 for a model based on age and diagnostic code alone to 0.87 by inclusion of physiological variables and 0.90 by inclusion of blood test results. In both models non-linear associations between predictors and outcome were accounted for, either by categorizations using a priori determined cut-off points based on natural distributions or after plotting mortality against deciles of each variable. It seems conceivable that optimal modelling requires accounting for non-linear associations between predictors and outcome and—as our results show—integrating effects of treatment modalities on predictors.

In the present study, the predicted and observed risk of death was lower than 5 % in more than 80 % of patients. In these patients, the mean predicted and observed probabilities exhibit partially overlapping confidence intervals. However, model discrimination is substantial when comparing predicted and observed probabilities in patients with a mortality risk above 5 %. From a clinical point of view, it is more important to recognise patients with an increased risk of death than to exactly predict a low risk of death. Mechanical ventilation at the time of ED arrival and low saturation index, but not low SpO2 per se, were associated with higher mortality. This indicates that support by mechanical ventilation and increased FiO2 has to be taken into account when interpreting SpO2 levels in a patient with respiratory failure. A similar effect was expected but not observed for the predictive value of blood pressure measurements and the use of continuous infusions of vasopressors and/or inotropes at the time of ED arrival. Despite the high mortality (42 %) of patients receiving vasoactive drugs, this was not statistically significant. This may be related to the very small number of patients (ca. 0.5 %) receiving vasoactive drugs at the time of ED admission.

The prognostic performance of the subjective risk estimation by the triaging health-care provider was inferior to the developed model. It seems that higher mortality was predicted by triage nurses in the low-risk patients, and lower mortality in high-risk patients. This could potentially lead to misallocation of resources in low-risk situations and to delayed intervention in high-risk patients.

In conclusion, in this prospective cohort study, we found that a prediction model derived from readily available patient characteristics, physiological parameters and established treatment interventions registered during the first minutes after ED arrival was a strong indicator for the subsequent risk of death. Our data indicate that the use of the prediction model instead of an assessment based only on subjective evaluation can facilitate the identification of patients with higher mortality risk. We do not suggest that health-care providers ignore their personal experience when assessing patients and instead base their decisions solely on a scoring system. However, the model might provide a more systematic approach to the integration of all available information and facilitate the identification of patients with higher mortality risk who might be missed by an assessment based on subjective evaluation alone. This might be used to design interventions to ensure rapid commencement of treatment in patients at risk of an unfavourable outcome. The model might also serve to identify patients with a high risk of mortality in whom full treatment is not warranted. Early prediction of mortality in an end-of-life setting might prevent the use of unnecessarily aggressive treatment modalities.

Future research should be guided at external validation of this or similar prediction models using a multicentre approach and to ascertain the effectiveness in clinical impact studies.

Notes

Conflicts of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Ethical approval and patient consent

The institutional review board (IRB) waived the need for IRB and patient consent.

Supplementary material

134_2015_3737_MOESM1_ESM.docx (92 kb)
Supplementary material 1 (DOCX 91 kb)

References

  1. 1.
    Nguyen HB, Banta JE, Cho TW, Van Ginkel C, Burroughs K, Wittlake WA, Corbett SW (2008) Mortality predictions using current physiologic scoring systems in patients meeting criteria for early goal-directed therapy and the severe sepsis resuscitation bundle. Shock 30:23–28PubMedCrossRefGoogle Scholar
  2. 2.
    Jones AE, Fitch MT, Kline JA (2005) Operational performance of validated physiologic scoring systems for predicting in-hospital mortality among critically ill emergency department patients. Crit Care Med 33:974–978PubMedCrossRefGoogle Scholar
  3. 3.
    McQuillan P, Pilkington S, Allan A, Taylor B, Short A, Morgan G, Nielsen M, Barrett D, Smith G, Collins CH (1998) Confidential inquiry into quality of care before admission to intensive care. BMJ 316:1853–1858PubMedCentralPubMedCrossRefGoogle Scholar
  4. 4.
    Kause J, Smith G, Prytherch D, Parr M, Flabouris A, Hillman K (2004) A comparison of antecedents to cardiac arrests, deaths and emergency intensive care admissions in Australia and New Zealand, and the United Kingdom—the ACADEMIA study. Resuscitation 62:275–282PubMedCrossRefGoogle Scholar
  5. 5.
    Buist M, Bernard S, Nguyen TV, Moore G, Anderson J (2004) Association between clinically abnormal observations and subsequent in-hospital mortality: a prospective study. Resuscitation 62:137–141PubMedCrossRefGoogle Scholar
  6. 6.
    De Luca G, Suryapranata H, Ottervanger JP, Antman EM (2004) Time delay to treatment and mortality in primary angioplasty for acute myocardial infarction: every minute of delay counts. Circulation 109:1223–1225PubMedCrossRefGoogle Scholar
  7. 7.
    Duke G, Green J, Briedis J (2004) Survival of critically ill medical patients is time-critical. Crit Care Resusc 6:261–267PubMedGoogle Scholar
  8. 8.
    Bellomo R, Goldsmith D, Uchino S, Buckmaster J, Hart GK, Opdam H, Silvester W, Doolan L, Gutteridge G (2003) A prospective before-and-after trial of a medical emergency team. Med J Aust 179:283–287PubMedGoogle Scholar
  9. 9.
    Goldhill DR, White SA, Sumner A (1999) Physiological values and procedures in the 24 h before ICU admission from the ward. Anaesthesia 54:529–534PubMedCrossRefGoogle Scholar
  10. 10.
    Hillman KM, Bristow PJ, Chey T, Daffurn K, Jacques T, Norman SL, Bishop GF, Simmons G (2002) Duration of life-threatening antecedents prior to intensive care admission. Intensive Care Med 28:1629–1634PubMedCrossRefGoogle Scholar
  11. 11.
    Derlet RW, Richards JR (2000) Overcrowding in the nation’s emergency departments: complex causes and disturbing effects. Ann Emerg Med 35:63–68PubMedCrossRefGoogle Scholar
  12. 12.
    Lynn SG, Kellermann AL (1991) Critical decision making: managing the emergency department in an overcrowded hospital. Ann Emerg Med 20:287–292PubMedCrossRefGoogle Scholar
  13. 13.
    Gerdtz MF, Bucknall TK (2001) Triage nurses’ clinical decision making. An observational study of urgency assessment. J Adv Nurs 35:550–561PubMedCrossRefGoogle Scholar
  14. 14.
    Brillman JC, Doezema D, Tandberg D, Sklar DP, Davis KD, Simms S, Skipper BJ (1996) Triage: limitations in predicting need for emergent care and hospital admission. Ann Emerg Med 27:493–500PubMedCrossRefGoogle Scholar
  15. 15.
    Durand AC, Gentile S, Gerbeaux P, Alazia M, Kiegel P, Luigi S, Lindenmeyer E, Olivier P, Hidoux MA, Sambuc R (2011) Be careful with triage in emergency departments: interobserver agreement on 1,578 patients in France. BMC Emerg Med 11:19PubMedCentralPubMedCrossRefGoogle Scholar
  16. 16.
    Quitt J, Ryser D, Dieterle T, Luscher U, Martina B, Tschudi P (2009) Does nonmedical hospital admission staff accurately triage emergency department patients? Eur J Emerg Med 16:172–176PubMedCrossRefGoogle Scholar
  17. 17.
    Subbe CP, Slater A, Menon D, Gemmell L (2006) Validation of physiological scoring systems in the accident and emergency department. Emerg Med J 23:841–845PubMedCentralPubMedCrossRefGoogle Scholar
  18. 18.
    Rees JE, Mann C (2004) Use of the patient at risk scores in the emergency department: a preliminary study. Emerg Med J 21:698–699PubMedCentralPubMedCrossRefGoogle Scholar
  19. 19.
    Brabrand M, Folkestad L, Clausen NG, Knudsen T, Hallas J (2010) Risk scoring systems for adults admitted to the emergency department: a systematic review. Scand J Trauma Resusc Emerg Med 18:8PubMedCentralPubMedCrossRefGoogle Scholar
  20. 20.
    Prytherch DR, Smith GB, Schmidt PE, Featherstone PI (2010) ViEWS—towards a national early warning score for detecting adult inpatient deterioration. Resuscitation 81:932–937PubMedCrossRefGoogle Scholar
  21. 21.
    Goodacre S, Wilson R, Shephard N, Nicholl J (2012) Derivation and validation of a risk adjustment model for predicting 7 day mortality in emergency medical admissions: mixed prospective and retrospective cohort study. BMJ 344:e2904PubMedCentralPubMedCrossRefGoogle Scholar
  22. 22.
    Challen K, Goodacre SW (2011) Predictive scoring in non-trauma emergency patients: a scoping review. Emerg Med J 28:827–837PubMedCrossRefGoogle Scholar
  23. 23.
    Smith GB, Prytherch DR, Schmidt PE, Featherstone PI (2008) Review and performance evaluation of aggregate weighted ‘track and trigger’ systems. Resuscitation 77:170–179PubMedCrossRefGoogle Scholar
  24. 24.
    Knaus WA, Draper EA, Wagner DP, Zimmerman JE (1985) APACHE II: a severity of disease classification system. Crit Care Med 13:818–829PubMedCrossRefGoogle Scholar
  25. 25.
    Khemani RG, Patel NR, Bart RD 3rd, Newth CJ (2009) Comparison of the pulse oximetric saturation/fraction of inspired oxygen ratio and the PaO2/fraction of inspired oxygen ratio in children. Chest 135:662–668PubMedCrossRefGoogle Scholar
  26. 26.
    Dumont CP, Tiep BL (2002) Using a reservoir nasal cannula in acute care. Crit Care Nurse 22:41–46PubMedGoogle Scholar
  27. 27.
    Burton GG, Hodgkin JE, Ward JJ (1991) Respiratory care: a guide to clinical practice. Lippincott Williams and Wilkins, PhiladelphiaGoogle Scholar
  28. 28.
    Nagelkerke NJ (1991) A note on a general definition of the coefficient of determination. Biometrika 78:691–692CrossRefGoogle Scholar
  29. 29.
    Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW (2010) Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 21:128–138PubMedCentralPubMedCrossRefGoogle Scholar
  30. 30.
    Steyerberg EW (2009) Clinical prediction models. Springer, New YorkCrossRefGoogle Scholar
  31. 31.
    Harrell FE (2001) Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis. Springer, New YorkCrossRefGoogle Scholar
  32. 32.
    Labarere J, Bertrand R, Fine MJ (2014) How to derive and validate clinical prediction models for use in intensive care medicine. Intensive Care Med 40:513–527PubMedCrossRefGoogle Scholar
  33. 33.
    Olsson T, Terent A, Lind L (2004) Rapid emergency medicine score: a new prognostic tool for in-hospital mortality in nonsurgical emergency department patients. J Intern Med 255:579–587PubMedCrossRefGoogle Scholar
  34. 34.
    Duckitt RW, Buxton-Thomas R, Walker J, Cheek E, Bewick V, Venn R, Forni LG (2007) Worthing physiological scoring system: derivation and validation of a physiological early-warning system for medical admissions. An observational, population-based single-centre study. Br J Anaesth 98:769–774PubMedCrossRefGoogle Scholar
  35. 35.
    Groarke JD, Gallagher J, Stack J, Aftab A, Dwyer C, McGovern R, Courtney G (2008) Use of an admission early warning score to predict patient morbidity and mortality and treatment success. Emerg Med J 25:803–806PubMedCrossRefGoogle Scholar
  36. 36.
    van der Wulp I, Schrijvers AJ, van Stel HF (2009) Predicting admission and mortality with the emergency severity index and the Manchester triage system: a retrospective observational study. Emerg Med J 26:506–509PubMedCrossRefGoogle Scholar
  37. 37.
    Froom P, Shimoni Z (2006) Prediction of hospital mortality rates by admission laboratory tests. Clin Chem 52:325–328PubMedCrossRefGoogle Scholar
  38. 38.
    Kellett J, Deane B, Gleeson M (2008) Derivation and validation of a score based on hypotension, oxygen saturation, low temperature, ECG changes and loss of independence (HOTEL) that predicts early mortality between 15 min and 24 h after admission to an acute medical unit. Resuscitation 78:52–58PubMedCrossRefGoogle Scholar
  39. 39.
    Seymour CW, Kahn JM, Cooke CR, Watkins TR, Heckbert SR, Rea TD (2010) Prediction of critical illness during out-of-hospital emergency care. JAMA 304:747–754PubMedCentralPubMedCrossRefGoogle Scholar
  40. 40.
    Goodacre S, Turner J, Nicholl J (2006) Prediction of mortality among emergency medical admissions. Emerg Med J 23:372–375PubMedCentralPubMedCrossRefGoogle Scholar

Copyright information

© The Author(s) 2015

Open AccessThis article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.

Authors and Affiliations

  • Michael Coslovsky
    • 1
  • Jukka Takala
    • 2
  • Aristomenis K. Exadaktylos
    • 3
  • Luca Martinolli
    • 3
  • Tobias M. Merz
    • 2
    Email author
  1. 1.Clinical Trials Unit, Institute of Social and Preventive MedicineUniversity of BernBernSwitzerland
  2. 2.Department of Intensive Care MedicineUniversity Hospital and University of BernBernSwitzerland
  3. 3.Department of Emergency MedicineUniversity Hospital and University of BernBernSwitzerland

Personalised recommendations