Introduction

Many ward patients may deteriorate to the point of unexpected ICU admission or even cardiac arrest and death. About one-half of the serious adverse events are deemed to be preventable [1]. Patients often show some signs of physiological deterioration for several hours (median 6 hours) before cardiac arrest [2, 3]. Theoretically, this would appear to be sufficient time to deliver interventions that would alter the trajectory of deterioration. This ‘failure to rescue’ is the context in which rapid response systems (RRSs) have been introduced [4].

Most RRS trials have used similar criteria for activation of the team. These include various thresholds for respiratory rate, heart rate, blood pressure and mental state as well as the ‘staff worried’ criterion [5]. Unfortunately, the determination of respiratory rate and mental state on the wards is error prone and vital signs are unlikely to be measured more frequently than five times per day outside the ICU [6–8]. Variations in activation potentially influence the effectiveness of RRSs. Non-activation and delays in activation may be associated with harm and too liberal activation may cause system fatigue [9–11]. Consequently, the time spent implementing and maintaining the concept of a RRS and the rate of RRS activation may be a contributory factor to its success.

The RRS team is usually multidisciplinary in nature and tailored to meet the institutional needs and resources. In this respect there is some variation in the constitution of a RRS. In the UK the RRS team may be nurse led, and in the USA nurse or respiratory therapists may lead [12, 13]. In Australia, New Zealand and Scandinavia, a physician-led RRS is favoured [5, 14]. The optimal composition of a RRS team is unknown and may depend upon organizational structure.

Organizations such as the Institute for Healthcare Improvement, the 2009 Joint Commission’s National Patient Safety Goal in the USA, the National Institute of Clinical Excellence (NICE) in the UK as well as numerous other organizations have driven the implementation of RRS teams [15, 16]. Despite high face validity, the effectiveness of RRS teams in reducing hospital mortality remains controversial. A high-quality meta-analysis in 2010 did not find evidence of RRS teams reducing hospital mortality in adults [17]. A more recent review by Winters et al. [18] included studies of varying methodological quality and design, with inconsistent findings across outcomes. Both analyses aggregated studies of varying methodology and quality, further limiting the interpretation of the results [18].

Accordingly, a systematic review and meta-regression was undertaken to assess the effect of the RRS on hospital mortality and cardiopulmonary arrest outside the ICU and to evaluate the potential relationship between the number of RRS team activations per 1000 admissions, the presence of a physician in the RRS team and the duration of the implementation phase and the effectiveness of RRS teams.

Methods

A systematic review of studies published between 1 January 1990 and 31 December 2013 was conducted in accordance with published guidelines [19, 20]. We used the PubMed, EMBASE, CINAHL (Cumulative Index to Nursing and Allied Health Literature) and Cochrane Register of Controlled trials databases. Additionally, a hand search of bibliographies of key publications was performed. Search terms included ‘rapid response team’, medical emergency team’ and ‘critical care outreach’. Details of the electronic search are described in Fig. 1 with additional information provided in the supplementary appendix.

Study eligibility criteria and selection

The inclusion criteria for this meta-analysis was that studies had to be a study that described the effect of RRS teams in a population of hospital in-patients that included a comparison between a control cohort and intervention cohort, and provided quantitative data about mortality rates or cardiopulmonary arrests. There was no country restriction but only English language studies were included.

A total of 2935 abstracts were identified by the search strategy. The titles and abstracts were independently assessed for eligibility by two investigators (RM and IR). Eight hundred and eighty-one duplicate studies were removed and a further 1994 studies were removed because they were not relevant or did not report sufficient data (Fig. 1). In cases with multiple articles with overlapping data from the same population, we included data from the most comprehensive study and excluded the other studies [21–24].

Fig. 1
figure 1

Literature search flow diagram

Full size image

Data extraction process

Two reviewers independently, using a standardized format, extracted the data. For each study the following data elements were extracted: year of study, type of study design (randomized controlled trial (RCT), before–after, interrupted time series), type of hospital, number of centres, type of RRS, types of interventions, period pre and post implementation, demographics, number of activations per 1000 admissions, effect on do-not-resuscitate orders, non-ICU cardiopulmonary arrest, hospital mortality, ICU admissions as well as study quality indicators. Authors were contacted for additional data as required.

Risk of bias in individual studies

Reviewers worked independently to assess study quality. The Newcastle Ottawa Scale (NOS) was used for assessing non-randomized studies [25]. The NOS uses a star system to evaluate the selection of study groups, the comparability of groups and the ascertainment of either the exposure or outcome of interest. The interrupted time series, controlled before–after and cluster randomized studies were evaluated using the criteria recommended by the Cochrane Effective Practice and Organisation of Care Group [26]. This included recruitment bias, baseline imbalance, loss of clusters, incorrect analysis or selective outcome reporting. To explore variability in study results (heterogeneity) we specified the following hypotheses a priori. We hypothesized that effect size may differ according to the number of activations per 1000 admissions, the duration of implementation in months and the presence of a physician on the team. The proposed standardized method to report the dose of RRS teams is activations per 1000 patient admissions [27].

Data analysis

The primary outcome was hospital mortality. The secondary outcomes were non-ICU cardiopulmonary arrest and ICU admission rates. The meta-analyses were performed by computing the risk ratio (RR) using a random effects model. The random effects model provides more conservative estimates of treatment effects in the face of heterogeneity. The analysis was performed in subgroups of adults and paediatrics with further subgroups by study design (cluster randomized control trial (CRCT), interrupted time series and controlled before–after versus before–after studies with no contemporaneous controls and observational studies). Quantitative analysis was performed using an intention-to-treat analysis, and the RR and 95 % confidence intervals (CIs) were calculated [28]. We used the method proposed by Higgins et al. [29] to measure inconsistency between study results, reported as the I 2 statistic as well as the 95 % CI. Publication bias was evaluated using the contour funnel plot asymmetry and the Harbord modification of the Egger test [30, 31]. We performed both analyses because graphical evaluation can be subjective. Factors other than publication bias can cause asymmetry in the funnel plot. These include factors such as study quality or true study heterogeneity.

A meta-regression was undertaken to explore the association between treatment effect and study characteristics [32]. These predefined factors included the number of activations per 1000 admissions, the presence of a physician in the RRS team and the time period for implementation.

The study by Hillman et al. [5] was a cluster randomized trial. The approach recommended by the Cochrane Collaboration to approximate the effective sample size is to divide by the design effect [20]. The design effect is calculated as:

$$ \mathrm{Design}\ \mathrm{effect} = 1 + \left(M\hbox{--}\ 1\right) \times \mathrm{I}\mathrm{C}\mathrm{C} $$

where M is the average cluster size and ICC is the intracluster correlation coefficient. The ICC compares the variance within clusters with the variance between clusters. Mathematically this is the between-cluster variability divided by the sum of the within-cluster and between-cluster variabilities. An ICC value of 0 gives a design effect of 1. This value indicates that individuals within a cluster have no similarity and there is no adjustment for sample size. An ICC of 1 indicates that all individuals within a cluster are identical and the sample size is the number of clusters. Larger cluster sizes are associated with smaller ICC values. In the study by Hillman et al. [5], the number of clusters was 23 and the ICC used was 0.02.

The study by Priestley et al. [12] was described as a step wedge CRCT. This involves a sequential roll-out of and intervention to clusters over a number of time periods [33]. The order in which clusters receive the intervention is random. The measurement of data from all clusters and at each step is a key feature that distinguishes the step wedge design from a classic cluster RCT [33, 34]. This trial did not measure at each step and is actually reported more like a cluster RCT [34]. We therefore calculated the design effect according to the procedure recommended by the Cochrane Collaboration [20].

Sensitivity analyses were pre-specified. The treatment effect was reported using a cumulative meta-analysis method fixed-effects model and a graphical augmentation to the funnel plot to assess the impact of a new study on the existing meta-analysis. An influence analysis was performed in which the pooled estimates are computed omitting one study at a time. Additionally a sensitivity analysis was conducted using an ICC of 0.01 and of 0.05 to establish whether this qualitatively changed results. The details of this analysis are included in Additional file 1. The analysis was performed using STATA statistical software (version 12.0; Statacorp, College Station, TX, USA) and Revman (version 5.1; Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration).

Results

A total of 29 publications were included in the analysis (Fig. 1). No unpublished studies were obtained.

Study characteristics

The RRS studies had an effective sample size of 2,160,213 patients (1,107,492 in the intervention group and 1,108,380 in the control group) (Table 1). Nineteen studies (65.5 %) reported physicians as part of the RRS team for 24 hours per day and 7 days per week, two studies only had physician presence for office hours Monday to Friday, seven studies had no physician presence and one study did not report on the composition of the team. All of the studies have been published since 2000 and 13 studies have been published after 2008 (the end date for systematic review by Chan et al. [17]). Twenty-five studies were single centre. Twenty-one studies were conducted in academic hospitals, seven in community hospitals and one study used multiple sites that included both academic and community hospitals. The characteristics of the RRS intervention are described in Table 2. The number of RRS team activations per 1000 admissions was reported in 23 studies and varied substantially across studies. The mean and 95 % CI for the adult and paediatric activations per 1000 admissions were 16.3 (9.0–23.7) and 16.8 (6.0–27.6), respectively. About 33 % (95 % CI 23–43 %) of referrals were admitted to the ICU immediately after a RRS team consultation and 9.7 % (95 % CI 4.5–14.9 %) acquired a new designation of do not attempt resuscitation.

Table 1 Characteristics of included studies
Full size table
Table 2 Characteristics of rapid response system implementation and interventions
Full size table

Risk of bias within studies

Two investigators (RM and IR) assessed study quality independently. The risk of bias for each study is presented in Tables 3, 4 and 5. The Newcastle Ottawa Quality Assessment Scale was used to evaluate the before–after studies without contemporaneous controls [35]. The interrupted time series, controlled before–after and cluster randomized studies were evaluated using the criteria recommended by the Cochrane Effective Practice and Organisation of Care Group [26]. This included recruitment bias, baseline imbalance, loss of clusters, incorrect analysis or selective outcome reporting.

Table 3 Risk of bias table for cluster randomized control trials and controlled before–after trials
Full size table
Table 4 Risk of bias table using the Newcastle Ottawa Quality Assessment Scale for cohort studies
Full size table
Table 5 Risk of bias table for interrupted time series studies
Full size table

In general terms, the number of stars denotes study quality. A study can earn one star for each component of ‘Selection’, ‘Outcome’ and ‘Comparability’. Representativeness is awarded a star if the cohort is truly or somewhat representative of the population of interest. A star is awarded for selection of the non-exposed cohort, if it is drawn from the same population as the exposed cohort. Exposure is satisfactorily ascertained if data were acquired from secure records. A maximum of two stars can be given for ‘Comparability’. Either the exposed or non-exposed were matched in design or confounders adjusted in analysis. A maximum of three stars can be given for ‘Outcome’. Assessment of outcome is awarded a star if the outcomes were assessed by independent or blind assessment, confirmation of records by reference to secure records or record linkage. The adequacy of the duration of follow-up should be awarded a star if it was long enough for the outcomes to occur. Completeness of follow-up was considered adequate if losses were not related to the exposure or the outcome and was sufficiently low to be unlikely to introduce bias.

Syntheses of results Primary outcome

Hospital mortality

The implementation of RRS in the adult population has been associated with an overall reduction in hospital mortality (RR 0.87, 95 % CI 0.81–0.95, p <0.001) (Fig. 2). There was evidence of considerable heterogeneity (I 2 = 86 %, p <0.001). The treatment effect in the cluster randomized trials, controlled before–after and interrupted time series studies was RR 0.91 (95 % CI 0.85–0.97) with less heterogeneity (I 2 = 3 %). In the paediatric population, RRS also showed a reduction in mortality (RR 0.82, 95 % CI 0.76–0.89) with significant heterogeneity (I 2 = 78 %) (Fig. 3). There was only one study in the cluster randomized control study, controlled before–after and interrupted time series subgroup so no subgroup analysis based on study design could be performed.

Fig. 2
figure 2

Forest plot of the effect of rapid response system teams on hospital mortality in adult in-patients. Weights are calculated from random-effects analysis. CBA controlled before–after, CCO critical care outreach, CI confidence interval, ITS interrupted time series, RCT randomized controlled trial

Full size image
Fig. 3
figure 3

Forest plot of the effect of rapid response system teams on hospital mortality in paediatric in-patients. Weights are calculated from random-effects analysis. CBA controlled before–after, CI confidence interval, ITS interrupted time series, RCT randomized controlled trial

Full size image

Secondary outcomes

Cardiopulmonary arrests

The implementation of RRS in the adult population has been associated with an overall reduction in cardiopulmonary arrests (RR 0.65, 95 % CI 0.61–0.70, p <0.001) with substantial heterogeneity (I 2 = 70 %, p <0.001) (Figure S1 in Additional file 1). The treatment effect in the cluster randomized trials, controlled before–after and interrupted time series studies subgroup was RR 0.74 (95 % CI 0.56–0.98) with less heterogeneity (I 2 = 0 %). In the paediatric population, RRS also showed a reduction in cardiopulmonary arrests (RR = 0.64, 95 % CI 0.55–0.74) with minimal heterogeneity (I 2 = 7 %) (Figure S2 in Additional file 1).

ICU admissions

Only 10 of the adult studies reported the effect of RRS teams on the number ICU admissions. The implementation of RRS in the adult population has not been associated with a significant effect on the number of ICU admissions (RR 0.90, 95 % CI 0.70–1.16, p = 0.43). None of the paediatric studies reported the effect of RRS teams on the number of ICU admissions.

Assessment of publication bias

Publication bias refers to the phenomenon in which studies with less favourable results are less likely to be published than those with favourable results. Funnel plots appear asymmetric because of systematic suppression of studies. There are many factors other than publication bias that may explain funnel plot asymmetry, such as differential study quality or small study effects. A contour-enhanced funnel plot aims to disentangle these causes of funnel plot asymmetry. Generally, the level of statistical significance may drive publication bias, with studies that do not reach the perceived milestones of significance (e.g. p <0.05) less likely to be published. A contour-enhanced funnel plot overlays the contours of statistical significance on a funnel plot. This provides a novel method to assess whether the studies that exist are areas of statistical significance or whether there are areas where studies are missing that correspond to areas of low statistical significance. If studies are missing in areas of low statistical significance, then there may be publication bias. If studies are perceived to be missing, then publication bias is less likely. The contour-enhanced funnel plot was asymmetric but the perceived missing studies were in areas of high statistical significance, making publication bias a less likely cause of funnel asymmetry (Fig. 4).

Fig. 4
figure 4

Contour-enhanced funnel plot. If studies appear to be missing in areas of low statistical significance, then it is possible that the asymmetry is due to publication bias. Conversely, if the area in which studies are perceived to be missing are of high statistical significance, then publication bias is a less likely cause of the funnel asymmetry

Full size image

The Harbord modification of the Egger test was used to assess funnel plot asymmetry. The estimated intercept was −0.207 with a standard error of 0.897 and a p value of 0.819. This result suggests little evidence for small study effects.

Assessment of heterogeneity

A meta-regression was performed to explore the heterogeneity between studies. The covariates used were the number of activations per 1000 admissions, the duration of implementation in months and the presence of a physician in the RRS team. The model was adjusted for multiplicity using the Monte-Carlo permutation test for meta-regression. There was no significant relationship between any of the covariates (activation dose (p = 0.112), implementation time (p = 0.999) or physician presence (p = 0.992)) and hospital mortality. The model accounted for 18 % of the observed heterogeneity. The residual 82 % heterogeneity was probably due to persisting between study effects.

Sensitivity analysis

Assumptions about the effects of clustering were explored using an ICC of 0 and of 0.01 (Figures S3 and S4 in Additional file 1). Neither of these assumptions changed the treatment effect and is included in the supplementary data. A sensitivity analysis was performed by systematically omitting individual studies to establish the influence on outcome (Figure S5 in Additional file 1). This showed that there was no study whose removal would materially change the pooled estimate of hospital mortality. A cumulative meta-analysis was performed in which the cumulative evidence at the time of each study is calculated (Figure S6 in Additional file 1). This analysis showed that the effect estimate has been consistent over time. A further analysis examined the potential impact a new study would have on the existing meta-analysis, providing an indication of the robustness of the results to the addition of new evidence. The graph shows that all studies lie in the region with a beneficial treatment effect, which dominates the graph (Figure S7 in Additional file 1). This result suggests that the meta-analysis is relatively robust to the addition of a single new trial.

Discussion

In a systematic review and meta-analysis of 29 studies we found that a RRS team was associated with a reduction in hospital mortality in both adult and paediatric hospital populations. Our study reveals a striking degree of variation in how RRS teams were constituted, delivered and evaluated. The term ‘rapid response system’ may refer to rapid response teams, medical emergency teams or critical care outreach teams. Critical care outreach teams have the functionality of rapid response teams together with a surveillance function as well as ICU discharge follow-up.

A number of findings merit further discussion. There was a high degree of between-study heterogeneity in the included studies. The analysis included studies with different methodologies and could account for this observation. The adult CRCTs, controlled before–after and interrupted time series studies showed minimal heterogeneity (I 2 = 3 %), compared with observational and before–after studies without a contemporaneous control (I 2 = 88 %). This may be due to a small number of studies in the former subgroup.

This study did not find any dose–response relationship between the duration of the implementation phase, the presence of a physician on the team or the number of activations per 1000 and hospital mortality. These covariates were chosen because of pre-existing favourable reports [4]. Previous reports have suggested that a longer duration of implementation may lead to higher levels of support and engagement with the RRS [4, 36].

The optimal composition of the RRS team is uncertain. Two previous single-centre reports did not show the benefits of intensivist-led teams compared with registrar or resident-led teams [37, 38]. The majority of RRS interventions did not require the presence of a physician (fluids, oxygen and diuretics). Decisions around end-of-life planning may require physician involvement but would not necessarily manifest as changes in hospital mortality, although they could affect the number of cardiopulmonary resuscitation codes on the wards. It is possible that the presence of a physician in the team may have a differential effect in university hospitals compared with community hospitals, but there are insufficient data to establish this.

An increase in RRS team activations per 1000 admissions had previously been associated with reduction in cardiac arrest [39]. We were unable to show a significant relationship between RRS team activation and hospital mortality. This may suggest that the mechanism by which RSS teams reduce mortality is not through reductions in cardiac arrest. Very sensitive calling criteria may overactivate the RSS team, causing system fatigue with no tangible benefit. A final caveat is that the interpretation of meta-regression should always be undertaken cautiously. Meta-regression has limitations: the small number of studies, correlation between covariates and unmeasured characteristics, differences in the relationships that occur at a patient level and that may not be detected at a study level, and the tendency of regression to the mean.

The effect on the RRS team on the ICU workflow is important. The study was not able to show any effect on overall numbers of ICU admissions.

This study has several limitations. The vast majority of studies were observational studies without a contemporaneous control. Whilst there are several guidelines for the reporting of these studies, valuable information was often missing. The subgroup analysis did not find any significant difference in treatment effect in the different study methodologies. The outcomes of studies were reported variably. Some studies reported all hospital mortality and others reported only non-DNAR designated hospital mortality We used all hospital mortality reported because this offers the most conservative estimate of treatment. The major strength of our study is that the treatment effect has been consistent over time, is not influenced by any single study, and is robust to assumptions about clustering and to a further study being conducted.

Conclusion

This study found that RRS teams associate with a reduction in hospital mortality and cardiac arrest. These findings did not show any significant publication bias. A sensitivity analysis showed that the study findings were robust to addition of a new study. We were unable to show any benefit from the presence of a physician on the RRS team, the duration of implementation or the number of activations. Whilst RRS teams are very much part of the landscape in many health systems, further work is needed to understand the specific factors that are likely to mitigate their effectiveness in given operational contexts.

Key messages

  • RRS teams are effective in reducing hospital mortality in both adult and paediatric in-patients.

  • RRS teams also reduce hospital cardiac arrest.

  • The vast majority of rapid response interventions do not require a physician and the presence of a physician was not associated with improved outcomes.