Background

In experimental settings, scores that predict risk for clinical deterioration in hospitalised patients have evolved from vital sign based Early Warning Scores (EWS) to systems that utilise the large amount of patient centred data in Electronic Patient Records (EPRs) [1,2,3,4]. These systems are not yet in widespread use, however they represent a first step towards automatically assimilating patient data to assist clinical decision making on high risk ward patients. Each of the current, published experimental models were derived and validated on large EPR linked databases that used Intensive Care Unit (ICU) admission as one of the outcome measures. This outcome measure is commonly used (along with death and cardiac arrest) as a surrogate for confirmed clinical deterioration.

These and other prognostic models use a variety of statistical methods but multivariate regression modelling and machine learning techniques are commonly used. These methods require patient centred ‘candidate’ variables (such as vital signs or laboratory results) to form the component parts of the model [5]. The process of selecting model candidate variables is important, however there is no consensus on how best to do this. Numerous methods have been used for multivariate logistic regression, including expert opinion, forward and backward stepwise selection and machine learning techniques [6]. A logical and often used first step is evaluating univariate associations, which enables the variables to be quantified in advance of their inclusion in the model [7]. This is helpful when using EPR data where there are large number of available candidate variables [8]. Regardless of the method, the goal is to include the optimal combination of variables that maximise predictive ability, whilst avoiding unnecessary complexity [6].

In this systematic review we provide a complete summary of patient centred variables with a univariate association with unplanned ICU admission. By providing these data, we hope to aid the development of EPR based models for the prediction of ICU admission (and therefore clinical deterioration). We anticipate these data will enhance data-driven improvements in the care of deteriorating ward patients.

Methods

Search and identification of studies

The study protocol has been published [9] and follows the Preferred Reporting of Observational Studies and Meta-Analysis (PRISMA) statement [10]. An experienced medical librarian helped devise the search strategy to maximise identification of relevant studies. Studies were identified by searching Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica database (EMBASE), Cumulative Index to Nursing and Allied Health Literature (CINAHL), the Cochrane Database of Systematic Reviews and the Cochrane Central Register of controlled trials (CENTRAL). We included additional studies from the references of review articles, studies identified during screening, and from the authors’ personal libraries. We restricted the search to studies published since 2000. We did not apply any language restrictions. The search design is shown in the Additional file 1 (SDC-1).

Inclusion criteria

Included studies evaluated hospitalised, adult patients located in either the Emergency Department (ED), the general surgical or medical wards. Patients in specialist wards (such as obstetric or psychiatric) were eligible if they were evaluated as a part of the general patient population rather than disease specific sub groups of patients. Included studies required an analysis of at least two cohorts: one cohort of patients admitted to ICU (intervention) and one not admitted to ICU (control). Variables were eligible if they were patient centred and had been evaluated across both cohorts. Studies which described a univariate, statistical relationship between a patient centred variable (e.g. heart rate) and unplanned ICU admission were included. The described variables were single entities, as opposed to composites such as risk scores.

Exclusion criteria

Excluded studies did not evaluate unplanned ICU as an isolated outcome measure nor did they evaluate patients requiring ICU readmission. Additionally, studies that evaluated variables related to hospital processes or environmental risk alone (e.g. staff-to-nurse ratios), carried out multivariate analyses (without describing the univariate analyses that went into selecting variables for the model) or evaluated patient groups with a single diagnosis, were also excluded. (The studies excluded via this criterion are listed in Additional file 1 (SDC-2)). Patients admitted to ICU (or not) from high acuity areas such as HDU were excluded from the review as these hospital areas are often linked to ICUs and so are not always captured as admissions. Subgroups of illness acuity, such as needing an Rapid Response System (RRS) alert or being a high triage category, were not excluded. Finally, studies not published in peer-reviewed journals and those examining patients < 15 years old were excluded.

Study selection and data abstraction

Two authors (JM, TB) independently screened titles and abstracts of identified studies against the inclusion and exclusion criteria. (Fig. 1) They were not blinded to the journal titles or to the study authors or institutions. If there was disagreement or uncertainty regarding eligibility, the article was included in the next stage of screening. The full text was retrieved for all articles not excluded by the initial screening and re-assessed for eligibility as before. Disagreements about eligibility were resolved by discussion between the screening authors or a third party (a senior member of our research team, PW and DY). Two authors extracted independently data from the studies and supplementary material. Any uncertainties regarding data extraction were resolved by discussion amongst the study team. DistillerSR (Evidence Partners, Canada, Ottawa, Ontario) was used to manage the data and identify duplicate search results. All screening and data extraction forms were implemented within DistillerSR.

Fig. 1
figure 1

Flow diagram of the included and excluded studies

Full size image

Risk of Bias assessment

Two authors (JM, TB) independently assessed the risk of bias for included studies by using an adapted version of the Newcastle-Ottawa Scale (NOS) [11, 12]. The NOS is a scoring system designed to assess the quality of non-randomised studies in meta-analyses and systematic reviews [13]. We generated a score by assessing each study cohort for representativeness of the desired study population, the comparability of the cohorts being assessed, the size of the cohorts and correction for confounding. We adapted the NOS score to show bias in the types of studies included in this review (i.e. those showing univariate associations) whilst remaining faithful to the established NOS methodology. The details of the modified NOS scale are included in the Additional file 1 (SDC-3).

Data synthesis

We categorised patient-derived variables as comorbidities, demographics, laboratory tests, vital signs, diagnoses, medications and symptoms/signs, in general accordance with the categories used in the included studies. To synthesise and present the large number of variables included in the results in a logical way, we adopted the recently published method of Dettmer et al. (as adopted from Zaal et al.), who combined the quality of the studies investigating the variables in question (based on the NOS risk of bias assessment) with the number of times the variable was investigated [14, 15]. This semi-quantitative approach enable the assignation of a ‘weight of evidence’ to each variable (Table 1).

Table 1 Grading system for strength of evidence
Full size table

RESULTS

All included studies are shown in Table 2 [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31], The database search yielded 1520 unique studies; 1462 were removed after title and abstract review; 57 underwent full text screening; sixteen studies were included in this review (Fig. 1). Summary details are shown in Table 2 with additional study data in the Additional file 1 (SDC-4). The mean study quality score was five and the mode was seven. We graded six studies high quality [18, 20, 21, 28,29,30], four moderate quality [16, 23,24,25] and six (low quality) [17, 19, 22, 26, 27, 31]. The results of the bias assessment for each study are shown in Table 2 and the Additional file 1 (SDC-5). The quality of the studies is also reflected in the weight of evidence for any particular variable.

Table 2 Details of included studies
Full size table

Quantised data were presented as independent variables. For example, arterial oxygen saturation was evaluated six times for ranges of < 80, < 95%, 80–89%, 90–94%, mean (%) and median (%) (in each study group) across four studies and thus included six times in the initial analysis, with each of these ranges being defined as a single variable [16, 18, 20, 22]. Likewise, ‘cardiovascular disorder’ was included 14 times across six studies, as either a comorbidity, diagnosis or symptom/sign [21, 26,27,28,29]. We recorded the statistical relationship between variables and unplanned ICU admission as p values, Odds Ratios (OR), Risk Ratios (RR) [28] or Incidence Rate Ratios (IRR) [21] (with 95% confidence intervals) and did not assign preference.

Five studies (31%) were case control studies and 11 (69%) were cohort studies. Eight (50%) were prospective and eight (50%) were retrospective. The number of participants in each study ranged between 95 in a prospective cohort study and 126,826 in a retrospective case control study. The number of patients admitted to ICU ranged between 15 and 5233, while in the control group they ranged between 80 and 125,244 (Table 2). Five studies (31%) evaluated patients in emergency departments (ED) and 11 studies (69%) evaluated patients treated on hospital wards. Of the studies examining ward patients, five evaluated patients admitted via ED, two evaluated patients who had an RRS review and four evaluated patients admitted via any source (SDC-6, Additional file 1). Escobar [20] studied patients in 20 centres, Schuetz [25] in three centres and Loekiko [24] in two centres. The remaining 13 studies were single centre (Table 1).

Across the 16 studies, 189 different patient-derived variables were assessed for univariate association with unplanned ICU admission. Of these, 53 were vital signs, 42 were comorbidities, 29 were diagnosis, 26 were demographics, 25 were laboratory results, 10 were symptoms/signs and 4 were medications. One hundred and twenty-eight variables had a statistically significant positive association, two had a negative association and 59 had no association with unplanned ICU admission. Information on effect size was described as ORs, RRs or IRRs where available and is shown in Additional file 1 (SCD-7).

The semi-quantitative analysis resulted in 110 variables after repeatedly measured variables were grouped together. These are shown in Table 3 and Additional file 1 (SDC-8). Overall there were 12 variables with a strong weight of evidence (one was negative), three with a moderate weight of evidence and 33 with a weak weight of evidence for an association with unplanned ICU admission. The remaining 62 variables showed an inconclusive weight of evidence.

Table 3 Patient centred variables associated with unplanned ICU admission
Full size table

Variables associated with unplanned ICU admission

Variables with a strong, moderate and negative weight of evidence for association with unplanned ICU admission are summarised in Table 3.

Comorbidities, demographics and diagnosis

A history of congestive heart failure and diabetes were the only comorbidities in this group. These had a significant result in two high-quality studies [21, 29] of the demographics, being male [20, 21, 27, 28] and an increasing age [16, 18, 20, 21, 27, 28] had a strong weight of evidence for association. Four studies showed a significant difference in mean or median age between ICU and control groups (higher in the ICU group) [18, 20, 21, 27] and three studies showed a significant OR or IRR for increased age quantiles with the oldest quantile being 75+ years of age [21, 28]. Hepatic disease was the only diagnosis strongly associated with unplanned ICU admission [28, 30].

Vital signs

All six vital signs had a strong association. Heart rate was studied 12 times across seven studies, 10 times as quantiles (seven times for tachycardia and three times for bradycardia) and twice as a comparison of means [16, 18, 20, 22, 27, 29, 30]. Seven of the tachycardia and one of the bradycardia quantiles (< 60 beats per minute) showed a positive association. Two high-quality studies also found a significant difference in mean heart rate (higher in the ICU group) [18, 20]. Elevated respiratory rate was evaluated eight times across six studies and had a strong weight of evidence [16, 20, 22, 27, 30]. Five of the six quantiles showed a significant result [16, 27, 30] and both high-quality studies examining mean respiratory rate showed a significant difference [18, 20]. The only non-significant result was a respiratory rate of > 20 breaths per minute in a low-quality study [22]. Systolic blood pressure (SBP) was evaluated seven times across five studies [16, 18, 20, 22, 27]. Two high-quality studies showed a significant reduction in mean blood pressure [18, 20] and one moderate-quality study showed a significant OR for a SBP of 80-89 mmHg versus 90 mmHg and above [16]. Diastolic blood pressure (DBP) and temperature were evaluated in the same two high-quality studies, both showing significant differences in mean (decreased for SBP and increased for temperature) [18, 20]. Both studies had very small variations, < 0.2oCand < 2 mmHg respectively. Arterial oxygen saturation was studied six times across four studies [16, 18, 20, 22]. Lower saturation quantiles (< 80%, 80–89% and 90–94%) and lower mean/median saturations were shown to be significant.

Variables moderately and weakly associated with unplanned ICU admission are summarised in Table 3 and Additional file 1 (SDC-8) respectively.

Discussion

Statement of findings

In this systematic review of 16 observational and cohort studies evaluating ED and ward patients, we found two comorbidities (congestive cardiac failure and diabetes), two demographics (increasing age and being male), one diagnosis (hepatic disease) and six vital signs (respiratory rate, heart rate, temperature, systolic and diastolic blood pressure and arterial oxygen saturations) with a strong univariate association with unplanned ICU admission. These findings support the consensus that abnormal vital signs have significant value when predicting unplanned ICU admission. The strength of association for a history of congestive cardiac failure and diabetes and a new diagnosis of hepatic disease may reflect the high burden of care required in this patient cohort up until the terminal phase of disease. Being older and male as a risk factor for ICU admission may reflect the general hospital population as a whole. Overall this review provides a thorough summary of the candidate variables available in EPRs (and elsewhere in the clinical record) that will assist researchers to develop and evaluate predictive models for patients at risk of unplanned ICU admission.

Clinical and research implications

Progressing from vital sign based, EWS systems to EPR based, risk model systems has incrementally improved performance, both in terms of correctly identified deteriorating ward patients (sensitivity) and the number of ‘false alarms’ generated for clinical staff (specificity and positive predictive value). These performance gains have been achieved via multivariate regression models and more recently machine learning processes [1, 32,33,34,35,36]. Regardless of the statistical approach, candidate variables should be selected in a methodologically robust way. In the published literature, univariate filter methods, that rank the strength of the statistical association, are among the most common [7, 8]. It is a popular approach because the univariate analysis provides a summary of the variables most likely to enhance model performance, does not involve significant computation, is relatively simple, not time consuming and produces an easily interpretable output. It does have weaknesses however, including the potential to miss variables that have no association with the outcome when evaluated in isolation but have an association when evaluated together with another variable (e.g. age).

Despite their performance advantage, as yet no EPR based hospital model has achieved widespread adoption. In contrast, 75% of UK National Health Service hospitals monitor ward patients using the National Early Warning Score (NEWS) [37, 38]. The success of NEWS, which is a simple aggregate score that uses the univariate associations of abnormal vital signs with adverse patient outcomes, highlights the importance of interpretability and generalisability in this research and clinical domain. Advanced scoring systems that rely on complex computational processes may be difficult to interpret (and trust) for clinical staff and therefore less likely to be adopted into general use. We hope the univariate associations described will provide a convenient and intuitive reference for clinicians and researchers alike to overcome such barriers to implementation.

Strengths and limitations

The association of the variables does not infer causality. The search strategy was thorough and in accordance with current methodological guidelines but studies may have been missed. Publication bias may have affected results. The methodology of the included studies was varied, making meta-analysis inappropriate. We excluded studies examining specific sub-populations of patients only (i.e. acute liver failure) meaning the variables summarised in this review are not applicable for risk models designed for specific disease sub-groups.

There is a lack of consensus on which outcome measures to use when assessing the performance of predictive models for clinical deterioration [18]. Each of cardiac arrest, in hospital death and unplanned ICU admissions represent different populations and will, therefore, have different variable associations. We selected unplanned ICU admission as an isolated outcome measure (and excluded in hospital death and cardiac arrest) in the knowledge this would reduce the number of eligible studies and therefore potential variables for inclusion in this review. We adopted this method because we aim to advance the study of models that predict clinical deterioration, specifically in those who will most benefit from an intervention such as an ICU admission. When predicting ICU admission, some authors published the univariate relationships from within their derivation databases before including them in the multivariate analysis [20]. However we are not aware of any who have based selection on associations evaluated in external databases.

We evaluated a heterogeneous study population by including ED, ward, post-Medical Emergency Team (MET) and non-post-MET patients. This was done because we wish to better understand associations with clinical deterioration, which may occur at any time-point during hospital admission. Namely, early deterioration, which may occur soon after discharge from ED to the ward, in which case patient centred ED data is important. Or late deterioration, when the patient has been on the ward for some days. Studies examining sub-populations of patients (i.e. where specifically designed predictive algorithms have the potential to be more accurate than when used in a general patient population) were excluded on the basis that as a first step, we wish to isolate variables that will contribute to a hospital wide EPR based risk score.

We have deliberately avoided describing multivariate studies because we do not wish describe the models themselves. There are multiple examples of high performing, multivariate clinical predictive models in the literature, whose variables will have quantifiable associations with unplanned ICU admission. However, it is impossible to exclude collinearity in these instances, making obsolete our objective to individually quantify these variable associations as potential “building blocks” for future models. As a consequence, multivariate analysis was excluded unless the univariate associations were described.

Conclusion

Having abnormal vital signs, being elderly, male, having a history of heart failure or diabetes and a diagnosis of liver failure are all strongly associated with unplanned ICU admission. This systematic review is the first to comprehensively collate the evidence on patient centred variables with univariate associations with ICU admission. These results may assist the development of predictive models for hospitalised patients at risk of needing escalations in care. There is a lack of high-quality data in this field and further work is required to isolate the patient centred variables most likely to enhance model performance when predicting unplanned ICU admission.