Introduction

Goal-directed hemodynamic therapy (GDHT) is the management of global and/or regional blood flow guided by predetermined hemodynamic parameters with specified goals for intervention. Since pioneering work in 1988 [1] and 1995 [2], GDHT has revolutionized hemodynamic care in patients receiving anesthesia and surgery or admitted to the intensive care unit.

The efficacy of GDHT among different randomized controlled trials (RCTs) has been inconsistent. For example, in patients with sepsis, one GDHT protocol comprised of maintaining the central venous pressure between 8 and 12 mmHg, the mean arterial pressure between 65 and 90 mmHg, and the central venous blood oxygen saturation ≥ 70% significantly reduced mortality from 46.5% to 30.5% (n = 263) [3]; however, one decade later, three different RCTs failed to replicate this result using almost the same GDHT protocol in the same patient population with a much larger sample size (n = 1260–1600) [4,5,6]. Similar discrepancies are widespread in GDHT research [7].

The cause of these inconsistencies across GDHT research is unclear but may be related to factors such as the heterogeneity of the patient population [8, 9], the parallel use of the Enhanced Recovery After Surgery (ERAS) protocol [10, 11], the different GDHT protocols used by different studies [12,13,14], and the timing and type of antibiotics administered to septic patients [15]. However, other causes likely exist that contribute to the inconsistency of the results.

Modern GDHT is characterized by the use of innovative technologies that are noninvasive or minimally invasive and emphasize the monitoring of intravascular volume and cardiac output. These monitors are manufactured by competing companies and incur definite costs. For reasons including but not limited to the competition among different manufacturers and the costs incurred by these novel devices, different types of conflicts of interest (COI) are widespread in GDHT research.

The recent editorials published in Science [16] and Intensive Care Medicine [17] highlight the concern over the potential confounding effect of COI on biomedical research. A robust body of literature demonstrates that industry-sponsored studies tend to have proindustry results and/or conclusions [18,19,20,21]. However, these previous investigations primarily focused on research related to drugs, smoking, alcoholism, and nutrition [18,19,20], while the association between COI and research related to medical devices has not been adequately studied. Given the rapid implementation of contemporary hemodynamic monitors in acute care, an urgent need exists to understand the influence of COI on GDHT research.

We hypothesize that GDHT research is confounded by COI. RCTs that had specifically compared GDHT with usual care in adult patients under acute care were identified and analyzed to understand the association between COI and GDHT research. We herein differentiate between study results and article conclusions because results are based on objective data, while conclusions can be influenced by personal opinions.

Methods

Literature search

A systematic literature search of published RCTs comparing GDHT with usual care was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [22]. A medical librarian (A.B.) performed the systematic search of multiple databases after consultation with lead authors and a medical subject heading (MeSH) analysis of key articles provided by the research team. The formal search used relevant controlled vocabulary terms and synonymous free-text words and phrases to capture the concepts of RCT and GDHT. The electronic databases OVID Medline, OVID Embase, PubMed, and Cochrane Trials were searched on October 12, 2017 and July 6, 2018. The strategy used for the last search is presented in Supplemental file 1. Additional studies were identified nonsystematically by screening the reference lists of relevant articles and searching Google Scholar and PubMed.

Study selection

Two investigators (L.Z. and L.M.) independently screened identified references and then performed full-article reviews; conflicts were resolved by consulting a third investigator (Y.A.). The inclusion criteria were as follows: (1) adult patients (≥ 18 years old); (2) comparison between GDHT and usual care; (3) complications, mortality, or length of hospital stay reported as outcome; (4) perioperative or critical care setting; and (5) randomized controlled trial. A study was excluded if it (1) was not a randomized study, (2) was not published in a full-text article, (3) compared two different forms of GDHT instead, and (4) did not report the outcome of interest. GDHT was defined as the management of global and/or regional blood flow or oxygen delivery guided by predetermined hemodynamic parameters with specified goals for intervention. Usual care was defined as the hemodynamic management that is widely accepted as the standard of care but without guidance based on advanced volume or flow monitoring.

Definition of outcomes

Complication was defined as any deviation from the normal postoperative course or organ dysfunction [23,24,25]. Organ-specific complications include myocardial infarction, congestive heart failure, cardiac arrest, atrial fibrillation or other types of arrhythmia, pulmonary embolus, pneumonia treated with antibiotics, respiratory failure requiring intubation, respiratory insufficiency requiring physiotherapy or oxygen therapy, stroke, transient ischemic attack, postoperative delirium or cognitive decline, renal insufficiency requiring dialysis, acute kidney injury, urinary tract infection requiring antibiotics, hepatic insufficiency, gut hypoperfusion, ileus, disseminated intravascular coagulation, and sepsis. Surgery-related complications refer to surgical site bleeding, infection, anastomotic leakage, stenosis, ischemia, or tissue necrosis. In-study mortality was defined as the mortality reported by the study, referring to the rate of death in the hospital or at any time point specified by the study. The longest follow-up was used in the meta-analysis when different mortalities at different time points were reported. The length of hospital stay was defined as the total days from admission until the actual day of discharge or the day the patient was deemed fit for discharge.

Classification of conflicts of interest

All eligible articles were independently investigated by two investigators (L.Z. and L.M.) to determine the presence and type of self-reported COI in each article, with special attention focused on disclosure, acknowledgments, and the author’s work place. The COI was classified as industry sponsorship, author conflict, device loaner, none, or not reported (Table 1). If an article had different types of COI, the following priority order was used for classification: industry sponsorship > author conflict > device loaner.

Table 1 Classification of self-reported conflicts of interest (COI) in goal-directed hemodynamic therapy research
Full size table

Grading of an article’s conclusion

The conclusion of each eligible article was graded as GDHT-favorable, neutral, or unfavorable by two investigators (L.Z. and L.M.) independently. An article was determined to have a favorable conclusion if it favored GDHT over usual care, an unfavorable conclusion if it favored usual care over GDHT, and a neutral conclusion otherwise.

Data extraction

The following data were extracted from each eligible article: (1) setting of acute care, (2) number of patients, (3) classification of COI, (4) protocols involving hemodynamic parameters and specific goals, (5) complications, (6) mortality, (7) length of hospital stay, (8) article conclusion grade, (9) study origin, and (10) monitoring device used.

Quality assessment of selected studies

The risk of bias of each study was assessed by the tool established by the Cochrane Collaboration [26]. The following domains were assessed: (1) random sequence generation (selection bias), (2) allocation concealment (selection bias), (3) blinding of participants and personnel (performance bias), (4) blinding of outcome assessment (detection bias), (5) incomplete data outcome (attrition bias), (6) selective reporting (reporting bias), and (7) other bias. A study was rated as having a high risk of bias overall if more than one domain was rated as having a high risk of bias. Publication bias was assessed by visual inspection of the funnel plot, with a symmetrical plot indicating the absence of bias, and an asymmetrical plot indicating the presence of bias.

Synthesis of evidence

The effects of GDHT on complications and mortality were assessed by meta-analysis using RevMan 5.3 software (The Nordic Cochrane Centre, The Cochrane Collaboration, 2014). The I2 statistic [27] was calculated to measure the extent of heterogeneity, and the Cochrane Q test statistic was used to assess the statistical significance. A random effects model was used if significant heterogeneity was identified, and a fixed effects model was used otherwise. To quantify the magnitude of the effect size for dichotomous outcomes of complications and mortality, the risk ratio (RR) with a 95% confidence interval (CI) was calculated. A two-sided p value less than 0.05 was considered statistically significant. Forest plots were constructed to help visualize both the result of a single study and the pooled result.

The association between COI and study results, or more specifically whether the therapeutic effect of GDHT varies with different classes of COI, was investigated by both stratified meta-analysis and mixed effects meta-regression analysis. The metafor package [28] implemented in R software was used to perform meta-regression analysis, in which we investigated whether the heterogeneity of complications or mortality among eligible studies (log RR as the dependent outcome variable) is explainable by the COI classification or other study-level factors, including the study publication year, study setting, patient number, device used for hemodynamic monitoring, GDHT protocol, and origin of the study.

The association between COI and an article’s conclusion was investigated by logistic regression, in which COI classifications and other study-level factors were treated as independent variables, and the article’s conclusion was treated as the dependent outcome variable. The results are expressed in odds ratios (OR) and 95% CIs to indicate the effect of COI on the conclusion (favorable vs. neutral).

Results

Results of literature search

The systematic search yielded 2174 references, and the nonsystematic search identified 15 additional records. Following de-duplication and screening of the title/abstract, 139 studies were retained and underwent a subsequent full-text review. On the basis of the selection criteria, 82 articles were retained for the final synthesis of evidence [2,3,4,5,6, 24, 29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104]. The selection process is detailed in Fig. 1.

Fig. 1
figure 1

Article selection process

Full size image

Characteristics of eligible articles

The setting, number of patients, monitor used, intervention protocol, and main conclusion of each study are presented in Table 2. These 82 eligible articles were published between 1993 and 2018, with 71 studies being conducted in the perioperative setting, and 11 studies being conducted in the critical care setting. Most studies originated from Europe (n = 46, 56%). The risks of bias, expressed as the percentages of low, unclear, and high risks of the different domains of all studies included in the meta-analysis, are presented in Fig. 2, while the risks of bias of different domains of each study based on review authors’ judgments are detailed in Supplemental file 2. We detected no obvious evidence of publication bias in the studies included in the meta-analysis based on visual assessment of the funnel plots (Supplemental files 3 and 4).

Table 2 Randomized controlled trials (n = 82) comparing goal-directed hemodynamic therapy (GDHT) with usual care in acute care
Full size table
Fig. 2
figure 2

Risks of bias expressed as percentages of different risks (low, unclear, and high) across all studies included in the meta-analysis

Full size image

Prevalence of COI

Of the 82 eligible articles, 43 (53%) reported COI (industry sponsorship = 19; author conflict = 13; device loaner = 11), 33 (40%) declared no COI, and 6 (7%) did not include a COI statement (Supplemental file 5). None of the coauthors among these 82 articles were employed by a related industry.

Effects of GDHT on complications per COI classification

Fifty-one studies reported the number of patients with complications, with 3555 patients being managed by GDHT, and 3592 patients being managed by usual care (Supplemental file 5). Compared with usual care, GDHT significantly reduced the risk of developing complications based on these 51 studies (RR = 0.81, 95% CI 0.74–0.88; p = 0.0001; Fig. 3a), based on 31 studies with any type of COI (RR = 0.85, 95% CI 0.77–0.93; p = 0.006; Fig. 3b), based on 17 studies declaring no COI (RR = 0.76, 95% CI 0.64–0.90; p = 0.002; Fig. 3c), based on 12 industry-sponsored studies (RR = 0.83, 95% CI 0.75–0.92; p = 0.006; Fig. 3d), and based on 11 studies with author conflict (RR = 0.77, 95% CI 0.64–0.93; p = 0.007; Fig. 3e) but not based on eight studies with a device loaner (RR = 1.01, 95% CI 0.86–1.20; p = 0.90; Fig. 3f). These effects are comparable as indicated by the overlapping 95% CI ranges.

Fig. 3
figure 3

Forest plots showing the risk ratios of having a patient with complications in all eligible studies (a), studies with any type of conflict of interest (b), studies declaring no conflicts of interest (c), industry-sponsored studies (d), studies with author conflict (e), and studies with a device loaner (f). The size of the diamond reflects the weight of the trial in the pooled analysis. The horizontal bars represent the 95% confidence interval (CI)

Full size image

Effects of GDHT on mortality per COI classification

Forty-six studies reported in-study mortality, with 5942 patients being managed by GDHT, and 6003 patients being managed by usual care (Supplemental file 5). No statistically significant heterogeneity was identified among the various studies. Compared with usual care, GDHT led to a statistically significant change in mortality based on these 46 studies (RR = 0.91, 95% CI 0.85–0.99; p = 0.02; Fig. 4a) but not based on 25 studies with any type of COI (RR = 0.93, 95% CI 0.85–1.03; p = 0.15; Fig. 4b), 17 studies declaring no COI (RR = 0.89, 95% CI 0.78–1.01; p = 0.07; Fig. 4c), eight industry-sponsored studies (RR = 0.92, 95% CI 0.75–1.12; p = 0.39; Fig. 4d), seven studies with author conflict (RR = 0.77, 95% CI 0.47–1.26; p = 0.30; Fig. 4e), or 10 studies with a device loaner (RR = 0.94, 95% CI 0.85–1.03; p = 0.20; Fig. 4f). These effects are comparable as indicated by the overlapping 95% CI ranges.

Fig. 4
figure 4

Forest plots showing the risk ratios of mortality in all eligible studies (a), studies with any type of conflict of interest (b), studies declaring no conflicts of interest (c), industry-sponsored studies (d), studies with author conflict (e), and studies with a device loaner (f). The size of the diamond reflects the weight of the trial in the pooled analysis. The horizontal bars represent the 95% confidence interval (CI)

Full size image

Results of meta-regression analysis

The raw data used for meta-regression analysis are presented in Supplemental file 6. As there was no significant heterogeneity in the effect of GDHT on mortality among the 46 eligible studies (I2 = 0%, Fig. 4a), meta-regression was performed to analyze only the effects of GDHT on complications based on the pooled evidence from 51 eligible studies (I2 = 53%, Fig. 3a). The results (Supplemental file 7) did not identify a significant effect of the COI classification (p = 0.25), study setting (p = 0.55), patient number (p = 0.40), device used (p = 0.94), GDHT protocol (p = 0.99), or study origin (p = 0.20) on the observed study heterogeneity. The only factor that had a significant correlation with the RR of GDHT vs. usual care was the year of study publication (p = 0.0012, Fig. 5). As indicated by the significant p values from the tests of residual heterogeneity, it is highly likely that other study-level factors exist that were not considered in our meta-regression but influence the effect of GDHT on complications.

Fig. 5
figure 5

Bubble plot of the risk ratio of complications vs. the year of study publication

Full size image

Association between COI and article conclusions

Among the 82 eligible articles, 50 (61%) had a GDHT-favorable conclusion, 31 (38%) had a GDHT-neutral conclusion, and 1 (1%) had a GDHT-unfavorable conclusion (Supplemental file 5). The percentages of articles with specific conclusion grades per COI classification are presented in Fig. 6. Industry-sponsored studies had the highest percentage (84%) of GDHT-favorable conclusions, followed by studies with author conflict (77%), studies declaring no COI (55%), studies including no COI disclosure (50%), and studies with a device loaner (27%). Logistic regression analysis showed that only COI (p = 0.016) and the GDHT protocol (p = 0.022) were significantly associated with an article’s conclusion (GDHT-favorable vs. neutral) in GDHT research (Supplemental file 8).

Fig. 6
figure 6

Percentages of different grades of article conclusions based on the classification of conflicts of interest (COI)

Full size image

Discussion

Our study demonstrated that (1) more than half (53%) of RCTs comparing GDHT with usual care have COI; (2) GDHT reduces complications in studies with any type of COI, studies declaring no COI, industry-sponsored studies, and studies with author conflict but not in studies with a device loaner; (3) the heterogeneity of complications among eligible studies cannot be explained by COI, i.e., the therapeutic effect of GDHT on complications does not appear to vary with different classes of COI; (4) GDHT has no effect on mortality on the basis of the meta-analysis of studies with different COI; (5) COI might have a significant overall effect on the odds of having a GDHT-favorable vs. neutral conclusion; and (6) the majority of industry-sponsored studies have a GDHT-favorable conclusion (84%), while the majority of studies with a device loaner have a GDHT-neutral conclusion (73%). Taken together, the available evidence does not suggest a close relationship between COI and study results; however, it does suggest a potential association between COI and an article’s conclusion in GDHT research.

The influence of industry relationships on the outcomes of original research has been scrutinized in various fields of biomedical research [18, 19]. Although the findings diverge, these efforts do highlight concern regarding the potential confounding effect of industry relationships on biomedical research. This concern is corroborated by one recent cumulative meta-analysis concluding that compared with nonindustry-related studies, industry-related studies are more likely to have both favorable efficacy results, based on 25 papers that included 2923 studies (RR = 1.27, 95% CI 1.17–1.37), and favorable conclusions, based on 29 papers that included 4583 studies (RR = 1.34, 95% CI 1.19–1.51) [18]. Aggregation of the results of eight systematic reviews also concluded that the odds of industry-sponsored studies having a proindustry conclusion is 3.60 (95% CI 2.63–4.91) [19].

However, the majority of these previous investigations were based on drug studies, while only a few reports were based on device studies [105,106,107]. The influence of industry relationships on GDHT research, a field dependent on advanced hemodynamic monitoring devices, has not been reported. One difference between drug and device studies is that devices and reusable supplies can be loaned, which generates a COI different from those of industry sponsorship and author conflict. On the basis of this consideration, we classified COI into industry sponsorship, author conflict, and device loaner in our investigation, an approach that differs from that used in previous investigations in which studies were dichotomized into only the industry-sponsored and unsponsored categories. This differentiation is important because our findings suggest that different types of COI may have different associations with the results and conclusions of GDHT research.

Study results and article conclusions are different. Conclusions can be influenced by personal opinions and may or may not be supported by results [21], and differentiating results and conclusions is prudent when investigating the influence of COI on biomedical research. Methods for analysis also differ. In our study, the association between COI and the study results of GDHT research (i.e., complications and mortality) was assessed by stratified meta-analysis and meta-regression, whereas the association between COI and a GDHT research article’s conclusion was assessed by logistic regression, which is in accordance with the fact that study results are quantitative, while an article’s conclusions are qualitative.

Our investigation revealed that COI are widespread in GDHT research. Although GDHT is a landmark event in intensive hemodynamic care, the inconsistent results and conclusions of GDHT research as well as the associated costs hinder its wide clinical adoption [7]. In addition to industry influence, resource constraints, and the pressure of academic productivity, the urgent need for more evidence may be responsible for the high prevalence of COI in GDHT research. Our investigation found a self-reported incidence of 53%; however, the true incidence might be higher because of underreporting [108]. In our investigation, 55% of the studies declaring no COI had a GDHT-favorable conclusion, which was lower than that of industry-sponsored studies (84%) and studies with author conflict (77%) but higher than that of studies with a device loaner (27%). Although the cause of this discrepancy remains to be elucidated, the possibility of unreliable COI disclosure in GDHT research is suggested.

We found that GDHT can reduce complications but has only a marginal effect on mortality based on the overall evidence. The exact cause of this discrepancy is unknown but may be partially attributable to the difference between outcome measures (i.e., an outcome-dependent effect). The reporting of a complication not only depends on its definition but also on the accuracy and completeness of the information needed for the diagnosis. The diagnosis of a complication made by one investigator may not be made by a different investigator. This potential discrepancy does not exist when using death or survival as the end point, suggesting that the use of objective measures, such as mortality, may result in fewer inconsistencies.

Our meta-analysis did not include studies that reported complications as total events per patient or group instead of the number or percentage of patients in whom complications occurred. Moreover, we were not able to perform a meta-analysis of the length of hospital stay because of the diverse reporting methods (e.g., median vs. mean, whole range vs. interquartile range vs. 95% CI). The differing criteria for the length of hospital stay, variably defined as the time from admission to the actual day of discharge vs. the day the patient was deemed fit for discharge, added another source of heterogeneity.

There are a number of limitations in this study. It should first be noted that the cause–effect relationship between COI and the results or conclusions of GDHT research cannot be determined by this investigation. With the use of meta-analysis, we are able to calculate the pooled estimate of the therapeutic effect with improved precision compared to that of an individual study; however, we cannot guarantee that our estimates have improved accuracy (i.e., less bias) because the number of eligible studies was limited, and we had no access to the raw data from these studies. Our investigation could not determine the influence of nonfinancial COI, such as strongly held beliefs, personal relationships, and desire for career advancement, on GDHT research [109]. This factor in addition to the limited number of quality studies and the potentially missing or inaccurate disclosure of COI may confound the estimation of the association between COI and GDHT research.

It should be noted that multiple tests were performed in our investigation, and we recognize that the family-wise error rate in our study was not necessarily controlled at the 0.05 level, as we did not adjust for raw p values from multiple meta-analyses, meta-regression, or logistic regression. As a result, the statistical significance should be interpreted with caution. Nevertheless, measures such as RRs and 95% CIs should be relied upon to interpret the magnitude of any effects identified in the current study.

In summary, more than half of the RCTs comparing GDHT with usual care are related to industry in the forms of industry sponsorship, author conflict, or device loaner. The available evidence does not suggest a close relationship between COI and study results; however, it does suggest a potential association between COI and an article’s conclusion in GDHT research.