Introduction

Stress ulcer prophylaxis (SUP) is usual practice in intensive care units (ICUs) worldwide. Across North America, Europe, and Australia, most patients with risk factors receive SUP during their ICU stay [1, 2]. A survey of 58 ICUs in North America showed that proton pump inhibitors (PPIs) are the most commonly used agents, followed by histamine-2 receptor antagonists (H2RAs); none of the participating centers used sucralfate or anti-acids [1]. A recent survey of 97 centers in Europe, Australia, and Canada yielded similar findings [3]. These practices reflect the results of systematic reviews of randomized controlled trials (RCTs) suggesting that H2RAs, compared with sucralfate or no prophylaxis, reduce the risk of gastrointestinal (GI) bleeding without increasing the risk of pneumonia, and that PPIs in comparison with H2RAs further reduce the risk of clinically important GI bleeding (CIB) without increasing pneumonia risk. The recommendations of the Surviving Sepsis Campaign guidelines are consistent with these results [4,5,6].

Nevertheless, concerns have grown regarding the magnitude of benefit of SUP, as well as the safety of acid suppressive therapy with respect to pneumonia, Clostridium difficile infection, cardiovascular events, and mortality [7]. Conventional meta-analyses are restricted to head-to-head comparisons, and therefore cannot inform on the relative merit of candidate therapies that have not been compared directly. By including indirect comparisons, network meta-analyses can not only address this limitation but also—by combining direct and indirect estimates—improve precision [8].

We therefore conducted a network meta-analysis addressing the relative impact of SUP with PPI, H2RAs, sucralfate, and placebo (or no prophylaxis) on overt CIB, pneumonia, Clostridium difficile infection, and death.

Methods

We adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Extension statement for reporting network meta-analyses [Electronic Supplemental Material (ESM) Table 1] [9].

Data sources and searches

To identify RCTs comparing PPIs, H2RAs and sucralfate with one another and with placebo or no SUP in adult critically ill patients, we searched Cochrane CENTRAL, MEDLINE, and EMBASE from inception to April 2017 (ESM Table 2). We updated the search strategy for two systematic reviews of PPIs versus H2RA, and PPI versus placebo [6, 7], and conducted a complete search of the literature for other comparisons. We applied no restriction based on dose or route of drug administration or on language of publication. Eligible studies reported on at least one of the following: CIB, overt GI bleeding, pneumonia, mortality, and Clostridium difficile infection.

Study selection

Working in pairs, six reviewers screened citations and abstracts in duplicate and independently. The same pairs of reviewers evaluated all references judged potentially relevant for full-text eligibility.

Data extraction and quality assessment

Reviewers abstracted data in duplicate using piloted forms, and collected information on population demographics (age, sex, critical illness severity measure, ICU type, risk factors for bleeding), methodology and risk of bias, intervention and comparator (drug name, dose, route of administration, and duration of exposure), and outcomes. A third reviewer adjudicated disagreements not resolved by discussion.

We predefined CIB as evidence of upper GI bleeding with any of the following: significant hemodynamic changes not explained by other causes, need for transfusion of more than two units of blood, significant decrease in hemoglobin level, evidence of bleeding on GI endoscopy, or need for surgery to control the bleeding. Overt bleeding was defined as evidence of upper GI bleeding (hematemesis, melena, hematochezia, or coffee-grounds emesis or aspirate) regardless to other clinical findings. If an RCT only reported CIB, we considered all events as overt GI bleeding events. All studies used definitions consistent with those we prespecified.

We included pneumonia events in the ICU, whether or not they were associated with mechanical ventilation, accepting the definition used in each trial. We defined Clostridium difficile infection as a combination of clinical symptoms and a positive microbiologic test.

In duplicate, for each trial, reviewers assessed the risk of bias using the instrument recommended by the Cochrane Collaboration [10]. We provided a judgment of risk of bias as low (bias is not present or unlikely to alter the results seriously), unclear, or high (bias may seriously alter the results) for each of the following items: sequence generation, allocation sequence concealment, blinding of participants and clinicians, blinding of outcome assessment, incomplete outcome data assessment, and other bias. The overall risk of bias for each included trial was categorized as low if the risk of bias was low in all domains, unclear if the risk of bias was unclear in one or more domains and with no judgment of high risk of bias, or high if the risk of bias was high in one or more domains. We resolved disagreements by discussion.

Statistical analysis

Using a frequentist framework, we performed five random effects network meta-analyses, one for each outcome, calculating odds ratios (OR) and their corresponding 95% confidence intervals (CI). To evaluate inconsistency, we fit both a consistency and an inconsistency model for each outcome [11]. Using a node-splitting procedure [12], we calculated direct and indirect estimates for each pair of treatments. We calculated the frequentist analogue of the surface under the cumulative ranking curve (SUCRA) for each treatment (hereafter referred to as the SUCRA) [13]. All analyses were performed using the package mvmeta in Stata/IC 13.1 for Windows [14]. We assessed coherence (consistency) between direct and indirect estimates by performing node splitting and test for coherence [11]. We calculated the absolute treatment effect [risk difference (RD)] using the median event rate in the placebo arm in all trials for both clinically important GI bleeding and pneumonia outcomes. The median event rate in the placebo group was 2.1 and 6% for CIB and pneumonia outcomes; respectively.

Assessment of quality of evidence

For each outcome and each direct or indirect comparison, we applied the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to assess the quality of evidence [15, 16]. For rating quality in each indirect comparison, we focused on the first-order loop with the smallest variance which, therefore, contributed the most to the estimate of effect. We assigned quality of the indirect comparison according to the contributing direct comparison (of the two) within each first-order loop with the lowest of the quality rating. We considered rating down the quality of evidence further when intransitivity (clinical heterogeneity) was present, analogous to rating down the quality of evidence in direct comparison meta-analysis for indirectness [17]. For network estimates of any paired comparison, we chose the highest quality rating amongst the direct and indirect comparisons. We rated down quality in the network estimate if incoherence between direct and indirect estimates was present. In a network meta-analysis, incoherence represents the differences between the direct and indirect estimates of effect, while heterogeneity is the differences in estimates of effect across studies that assessed the same comparison [17].

Role of the funding source

The funders of this study did not contribute to its design or conduction. The authors were entirely responsible for data collection, analysis, interpretation and reporting. The corresponding author had access to all the data and final responsibility to submit for publication.

Results

Literature search

The initial search yielded 4139 citations; 93 proved potentially eligible after reviewing abstracts, of which 57 trials from 58 reports, representing 7293 patients, ultimately proved eligible (Fig. 1, ESM Table 2) [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75].

Fig. 1
figure 1

Flow diagram of the search results. This figure shows the process of selecting eligible studies. Overall, we included 57 randomized clinical trials from 58 reports

Full size image

Study characteristics

Of the 57 eligible trials, 18 compared PPIs with H2RAs; two, PPIs with sucralfate; four, PPIs with placebo; 18, H2RAs with sucralfate; 21, H2RAs with placebo; and six, sucralfate with placebo (ESM Figs. S1–S4). Trial sample size ranged from 28 to 1200 patients. Different doses, routes of administration, and durations of prophylaxis were used for PPIs and H2RAs across the trials (ESM Tables 3, 4).

Risk of bias

The risk of bias was high in 30 trials, low in 16 trials, and unclear in 11 trials (ESM Table 5).

Quality of evidence

Direct comparisons often suffered from limitations of risk of bias and imprecision. Regarding transitivity, although the intervention dosing and route varied across trials, the variation was not large enough to warrant concerns regarding intransitivity. Details can be found in Table 1 and ESM Table 5. For all comparisons across all outcomes, there was no statistical evidence of incoherence. Although no incoherence was detected by statistical testing, the test for incoherence could be underpowered; therefore, we further assessed incoherence by visually inspecting the direct and indirect estimates for any obvious differences. We considered problematic incoherence present for two comparisons based on visual inspection of direct and indirect estimates (Table 1).

Table 1 Direct, indirect and network meta-analysis estimates of the odds ratios of the effects of different prophylaxis comparisons
Full size table

The quality of evidence of network estimates for various comparisons across all outcomes ranged from low to moderate (Table 1).

Clinical outcomes

Clinically important GI bleeding

Thirty-one RCTs (5283 patients) reported on CIB [21, 25, 26, 30,31,32, 35, 36, 38,39,40, 42,43,44, 46,47,48, 57,58,59, 61, 63,64,65,66,67, 69,70,71,72, 74]. For three comparisons, the network estimate provided moderate-quality evidence with a CI excluding 1.0: PPIs versus H2RAs (OR 0.38; 95% CI 0.20, 0.73), PPIs versus no prophylaxis or placebo (OR 0.24; 95% CI 0.10, 0.60), and PPIs versus sucralfate (OR 0.30; 95% CI 0.13, 0.69). (Table 1, Fig. 2a). The SUCRA statistic showed that PPIs ranked first, followed by H2RAs, sucralfate, and placebo (ESM Table 6, Fig. 3a).

Fig. 2
figure 2

a Clinically important bleeding outcome. b Pneumonia outcome. a Test for inconsistency: p = 0.889 (indicating not inconsistent). b Test for inconsistency: p = 0.794 (indicating not inconsistent). CI confidence interval, H2RA histamine 2 receptor antagonists, PPI proton pump inhibitors

Full size image
Fig. 3
figure 3

a Cumulative ranking curve for clinically important bleeding outcome. CIB clinically important bleeding, H2RA histamine 2 receptor antagonists, PPI proton pump inhibitors. b Cumulative ranking curve for pneumonia outcome. H2RA histamine 2 receptor antagonists, PPI proton pump inhibitors

Full size image

Pneumonia

Thirty-five RCTs (5452 patients) reported on pneumonia [18, 19, 21, 22, 25, 26, 28,29,30, 32, 36, 37, 39,40,41,42, 44, 46,47,48,49, 51,52,53,54, 57, 58, 60, 61, 65,66,67,68, 70, 71]. PPIs ranked last compared with other interventions in terms of risk of pneumonia (ESM Table 7, Fig. 3b). Network meta-analysis results showed that PPIs and H2RAs probably increase the risk of pneumonia compared with sucralfate (OR 1.65; 95% CI 1.20, 2.27; moderate quality and OR 1.30; 95% CI 1.08, 1.58; moderate quality, respectively) but not compared with placebo or no prophylaxis (OR 1.09; 95% CI 0.72, 1.66; low quality). PPIs probably increase pneumonia compared with H2RAs (OR 1.27; 95% CI 0.96, 1.68; moderate quality), and no prophylaxis (OR 1.52; 95% CI 0.95, 2.42; moderate quality). H2RAs probably have no impact on pneumonia relative to placebo (Table 1, Fig. 2b).

Mortality

Thirty-six RCTs (5498 patients) reported on mortality [18, 19, 21, 23, 25, 26, 28, 29, 31, 34,35,36,37,38,39,40, 44, 46,47,48,49,50,51, 53, 56, 59,60,61,62, 64,65,66, 68,69,70,71, 75]. The results provide moderate-quality evidence that there is no difference between any of the management options in terms of all-cause mortality (Table 1, ESM Fig. S5). Given that all estimates were similar and approximated no effect, we did not calculate SUCRA values for the mortality outcome.

Clostridium difficile infection

Only one trial reported Clostridium difficile infection; therefore, we could not provide a network estimate for this outcome [66].

Overt GI bleeding

For overt GI bleeding, 49 studies (6662 patients) proved eligible [18, 20, 21, 23,24,25,26,27,28,29,30,31,32,33,34,35,36, 38,39,40, 42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75]. PPIs were superior to H2RAs (OR 0.34; 95% CI 0.19, 0.60, moderate quality), sucralfate (OR 0.35; 95% CI 0.18, 0.71, moderate quality), and placebo or no prophylaxis (OR 0.14; 95% CI 0.07, 0.28, moderate quality). H2RAs were superior to no prophylaxis in reducing risk of overt GI bleeding (OR 0.42; 95% CI 0.28, 0.63, moderate quality), but not sucralfate (OR 1.05; 95% CI 0.69, 1.59, moderate quality) (ESM Table 8 and Fig. S6). PPIs ranked first in reducing overt bleeding compared with other agents (ESM Table 9, Fig. S7).

Discussion

In this network meta-analysis, we included 57 RCTs enrolling 7293 patients comparing various SUP strategies. SUP with either PPIs or H2RAs likely reduces CIB (moderate quality evidence) (Table 1) relative to not using prophylaxis, and PPIs are likely superior to both H2RAs and sucralfate. Given the current estimates of the frequency of CIB, the magnitude of benefit is not large.

On the other hand, PPIs likely result in a higher risk of pneumonia than other prophylaxis regimens or no prophylaxis (moderate-quality evidence) (Table 1). These results are consistent with indirect evidence from multiple observational studies that have reported an increased risk of community- and hospital-acquired pneumonia in patients using PPIs [76,77,78]. Further direct supporting evidence comes from a retrospective observational study that reported a higher risk of pneumonia in critically ill patients on PPIs than for those on H2RAs in the ICU [79].

Low-quality evidence suggests that sucralfate may reduce pneumonia in comparison with placebo. This finding—together with the moderate-quality evidence of a lower pneumonia risk with sucralfate than PPIs—is consistent with prior physiologic and microbiologic data establishing the antibacterial effect of sucralfate, independently of the gastric pH level [80,81,82]. If the point estimate for pneumonia is accurate, PPIs relative to no SUP would result in a substantial increase in pneumonia.

The use of indirect comparisons within this network meta-analysis adds additional information beyond the multiple direct comparison meta-analyses that have compared PPIs, H2RAs, and sucralfate with placebo and with one another [6, 7, 83,84,85,86,87]. First, moderate-quality indirect evidence supports the previously observed effect of PPIs decreasing CIB relative to no SUP and relative to other agents (Table 1). Second, our results provide additional moderate quality evidence in support of the hypothesis that PPIs may increase the incidence of pneumonia relative to H2RAs, to sucralfate, and to no SUP. Prior systematic reviews were limited by imprecision and by the small number of studies for some direct comparisons, such as PPIs and placebo. Network meta-analytic techniques allowed us to generate more reliable estimates for these comparisons, particularly for the pneumonia outcome. Conventional meta-analyses are restricted to direct comparisons and do not consider indirect evidence in situations where evidence is sparse.

We have thus far focused on estimates of relative effects. In considering the implications of our results for use of SUP, consideration of absolute effects is crucial. The best estimates of the impact of PPIs on CIB suggest an absolute reduction of 1.6% relative to placebo, with a CI of 0.8–1.9% (Table 2). Similar estimates for pneumonia suggest an absolute increase of 3.1%, with a CI ranging from 0.3% fewer events to 8.5% more events. These estimates raise serious questions about the net benefit of PPIs for SUP.

Table 2 Absolute treatment effect for clinically important bleeding and pneumonia outcomes
Full size table

The strengths of this work include a comprehensive search, duplicate review of trial eligibility and risk of bias, and adherence to the PRISMA guidelines for reporting network meta-analyses (ESM Table 1) [9]. We adopted a frequentist (likelihood maximization) rather than a Bayesian approach to combine the results, avoiding issues relating to pre-specification of variance [88]. We used the node-splitting technique to provide estimates of indirect evidence, and applied the GRADE approach to assess the quality of evidence for each outcome, including specification of quality of direct, indirect, and network estimates [15]. To enhance the generalizability of our results, we included studies of patients with a wide spectrum of critical illness. We used best current estimates of effect for baseline risk to derive likely absolute effects of SUP on serious bleeding and pneumonia.

This study also has limitations. Although most RCTs used microbiological, radiological and clinical criteria to diagnose pneumonia, there was considerable variation in definitions [89]. Estimates of baseline risk vary appreciably across trials. In addition, most trials examining sucralfate predate the effective pneumonia prevention strategies, attenuating the applicability of these results to current practice. Only one trial addressed Clostridium difficile infection, which is important given a recent 30,000-patient retrospective observational study suggesting that PPIs may increase the risk of Clostridium difficile colitis [90]. Furthermore, the majority of trials did not report on nutritional management, which limited our ability to examine the effect of enteral nutrition as an effect modifier.

The universal use of SUP should be reconsidered in the light of uncertain net benefit. Two large multicenter RCTs are ongoing, the SUP-ICU in Europe (clinicaltrials.gov registration NCT02467621) and the REVISE Trial in North America, Australia, and Saudi Arabia. The results of these trials will provide further information about the safety of withholding stress ulcer prophylaxis in the ICU.

Conclusion

Our results provide moderate-quality evidence that PPIs are the most effective prophylactic strategy for SUP, with an absolute risk reduction in CIB relative to no prophylaxis of 1.6%. The benefit of PPIs must, however, be weighed against the risk of pneumonia, and possibly of Clostridium difficile infection. Moderate-quality evidence provides a best estimate of the increase in pneumonia with PPIs relative to no prophylaxis of over 3%. These estimates raise serious questions regarding the net benefit of SUP and its widespread use. The findings of this network meta-analysis can inform clinicians who consider the local incidence of CIB, pneumonia, and Clostridium difficile infection when deciding on uniform, selective, or sparing use of SUP in critically ill patients.