Abstract
Background
Randomized controlled trials (RCTs) and cohort studies are the most common study design types used to assess treatment effects of medical interventions. We aimed to hypothetically pool bodies of evidence (BoE) from RCTs with matched BoE from cohort studies included in the same systematic review.
Methods
BoE derived from systematic reviews of RCTs and cohort studies published in the 13 medical journals with the highest impact factor were considered. We re-analyzed effect estimates of the included systematic reviews by pooling BoE from RCTs with BoE from cohort studies using random and common effects models. We evaluated statistical heterogeneity, 95% prediction intervals, weight of BoE from RCTs to the pooled estimate, and whether integration of BoE from cohort studies modified the conclusion from BoE of RCTs.
Results
Overall, 118 BoE-pairs based on 653 RCTs and 804 cohort studies were pooled. By pooling BoE from RCTs and cohort studies with a random effects model, for 61 (51.7%) out of 118 BoE-pairs, the 95% confidence interval (CI) excludes no effect. By pooling BoE from RCTs and cohort studies, the median I2 was 48%, and the median contributed percentage weight of RCTs to the pooled estimates was 40%. The direction of effect between BoE from RCTs and pooled effect estimates was mainly concordant (79.7%). The integration of BoE from cohort studies modified the conclusion (by examining the 95% CI) from BoE of RCTs in 32 (27%) of the 118 BoE-pairs, but the direction of effect was mainly concordant (88%).
Conclusions
Our findings provide insights for the potential impact of pooling both BoE in systematic reviews. In medical research, it is often important to rely on both evidence of RCTs and cohort studies to get a whole picture of an investigated intervention-disease association. A decision for or against pooling different study designs should also always take into account, for example, PI/ECO similarity, risk of bias, coherence of effect estimates, and also the trustworthiness of the evidence. Overall, there is a need for more research on the influence of those issues on potential pooling.
Similar content being viewed by others
Background
Randomized controlled trials (RCTs) and cohort studies are the most common study designs used to assess treatment effects of medical interventions [1, 2]. RCTs, if well-designed and well-conducted, are considered as the gold standard and are widely accepted as the ideal methodology for causal inference [1,2,3].
However, RCTs may not be available for certain medical treatments due to ethical reasons or may suffer from inherent methodological limitations such as low external validity [4]. On the other hand, cohort studies may often have higher external validity, but also a higher risk of confounding. It is generally considered that systematic reviews should be based on RCTs because these studies are more likely to provide unbiased information than other study designs [5].
According to recent GRADE guidance, cohort studies can be highly valuable and provide complementary, sequential, or replacement evidence for RCTs in a systematic review or other evidence syntheses [6]. However, the potential impact of integrating evidence from cohort studies in meta-analyses of RCTs in the medical field has not been investigated yet.
To close this important research gap, this empirical study aims to conduct a pooling scenario of bodies of evidence (BoE) from RCTs with matched BoE from cohort studies. We investigate the extent of how the integration of BoE from cohort studies modifies the conclusion of BoE of RCTs, the direction of effect estimates derived from BoE of RCTs, and its impact on statistical heterogeneity. Moreover, we will evaluate the contributed aggregated weights of RCTs to the pooled estimates, use random effects and common effects models for pooling, calculate 95% prediction intervals (PIs), and test for subgroup differences between BoE from RCTs and cohort studies.
Methods
The sample of this empirical study was based on a large meta-epidemiological study [7], which was planned, written, and reported in adherence to current guidance for meta-epidemiological methodology research [8]. Eligibility criteria (PI/ECO: patient/population, intervention/exposure, comparator, and outcome) are reported in Table 1. Briefly, we included systematic reviews on medical interventions (or exposures) that included both RCTs and cohort studies for the same patient-relevant outcome and that performed meta-analyses for at least one BoE [7].
Identification of systematic reviews of RCTs and cohort studies
The original search for the meta-epidemiological study was conducted in MEDLINE on 04.05.2020 for the period between 01.01.2010 and 31.12.2019 in the 13 medical journals with the highest impact factor (according to the Journal Citation Report [JCR] 2018 category general and internal medicine). Initially, we planned to include the ten highest impact factor journals, but three journals did not publish any systematic review with an eligible BoE-pair. We therefore included the subsequent three journals according to the JCR 2018. The search strategy including the list of considered journals is given in Additional file 1: Appendix S1. Title and abstract screening was conducted by one reviewer (NB), and potentially relevant full texts were screened for eligibility by two reviewers independently (NB, LS). Any discrepancies were resolved by discussion.
For each included BoE from a systematic review, we included a maximum of three patient-relevant outcomes (e.g., mortality) and a maximum of three intermediate disease markers (e.g., blood lipids). If more than three outcomes were available for a given systematic review, we included the primary outcomes and thereafter we used a top-down approach (mentioned first). We evaluated the similarity of the PI/ECO criteria between BoE-pair from RCTs and cohort studies within each systematic review. For each BoE-pair, the similarity of each PI/ECO domain was rated as “more or less identical,” “similar but not identical,” or “broadly similar” (Additional file 1: Table S1). A detailed description of identification and evaluating similarity BoE-pairs can be found elsewhere [7].
Data extraction
Two reviewers (NB, LH) extracted the following data for each included BoE-pair into a piloted data extraction sheet: name of the first author, year of publication, type of intervention/exposure (e.g., antiretroviral therapy), description of the comparator, effect estimates (risk ratio [RR], hazard ratio [HR], odds ratio [OR], mean difference [MD], including 95% confidence interval [CI]), and number of studies. A detailed description of data extraction can be found elsewhere [7]. For the current analysis, we additionally extracted all effect estimates and corresponding 95% CI of the primary studies included for a relevant BoE (NB, LH).
Statistical analysis
For our pooling scenario, we re-analyzed the effect estimates of all eligible systematic reviews in a two-step approach: For each identified BoE-pair, we first pooled the effect estimates obtained from RCTs and cohort studies separately using a random effects model. Primary studies based on inappropriate study designs (i.e., case-control, cross-sectional, and quasi-RCTs) were excluded.
Second, we pooled the BoE from RCTs with the BoE from cohort studies with a random effects model for each BoE-pair. Binary outcomes (pooled as RRs, HRs, or ORs) and continuous outcomes (pooled as MDs on the same scale) were considered for analysis. Random effects models were used to account for potential between-study heterogeneity. For the sensitivity analysis, we used a common effects model to evaluate whether this hypothetical scenario is more conservative for pooling BoE from RCTs and cohort studies.
To explore the impact of including cohort studies on pooled effect estimates by combining BoE from RCTs and cohort studies (with or without subgroups), we compared the results and conclusions (95% CI including vs. excluding the null effect) between the BoE of RCTs only and that including both RCTs and cohort studies. Then, we evaluated the contributed weight of RCTs to the pooled estimates and conducted a statistical test for subgroup differences between the two types of BoE. A p-value < 0.05 was considered as statistical significant.
In an additional analysis, we used effect estimates of cohort studies as a reference and compared the results and conclusion between the BoE of cohort studies only and that including both, RCTs and cohort studies.
Heterogeneity in meta-analyses was tested with a standard χ2 test. We quantified any inconsistency by using the I2 parameter: I2=((Q−df))/Q × 100%, where Q is the χ2 statistic and df is its degrees of freedom [9]. An I2-value of greater than 50% was considered to represent considerable heterogeneity [10]. For binary outcomes, we additionally calculated τ2, which is independent of study size and describes variability between studies in relation to the risk estimates [11]. For continuous outcomes, we did not calculate τ2 due to the use of different scales between meta-analyses (blood pressure [mmHg] or body weight [kg]. Meta-analyses were conducted using Review Manager (RevMan) version 5.3 [12].
Whereas in a random effects meta-analysis, the focus is usually on the average treatment effect and its 95% CI, the calculation of a prediction interval (95% PI) also considers the potential treatment effect within an individual study setting, as this may differ from the average effect [11]. 95% PIs were calculated for the summary random effects for each meta-analysis since they further account for the degree of between-study heterogeneity and give a range for which we are 95% confident that the effect in a new study examining the same association lies within [11]. Calculations of 95% PI were conducted with Stata 15.
Results
Overall, 64 systematic reviews of RCTs and cohort studies were included [13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76]. Of the identified 129 outcome pairs, 118 from 59 systematic reviews were included in the present pooling scenario and re-analyzed (Additional file 1: Table S2-S3) [7, 13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76] (109 dichotomous and nine continuous outcomes) (Additional file 1: Figs. S1-118). Eleven outcome pairs from five systematic reviews [13, 26, 73,74,75] were excluded from the current analysis. Reasons for exclusion are provided in Additional file 1: Table S4.
Our sample of 118 BoE-pairs was based on 653 RCTs and 804 cohort studies. Detailed study characteristics including a description of the population, intervention/comparator, outcomes, range of study length, and risk of bias/study quality of primary studies included for each outcome pair have been described elsewhere [7].
Two of the outcome pairs were classified (PI/ECO similarity degree) as “more or less identical” and 82 as “similar but not identical,” whereas 34 were classified as “broadly similar.” Out of the 118 BoE from RCTs, for 39 (33.1%), the 95% CI excludes no effect, whereas for the BoE for cohort studies, 58 (49.2%) indicated a 95% CI excluding no effect. Twenty-four (20.3%) out of 118 BoE-pairs showed simultaneously for BoE from RCTs and BoE from cohort studies a 95% CI excluding no effect and a concordant direction of effect. The median I2 was 5% (τ2=0) across BoE from RCTs and 41% (τ2=0.03) across BoE from cohort studies, whereas the mean I2 was 23% (τ2=0.14) and 42% (τ2=0.18), respectively. Table 2 shows the summary effects of the BoE from RCTs, cohort studies, and the pooling scenario.
Pooling scenarios
By pooling BoE from RCTs and cohort studies with a random effects model, for 61 (51.7%) out of 118 BoE-pairs, the 95% CI excludes no effect. For the common effects model, for 77 (62.3%) out of 118 BoE-pairs, the 95% CI excludes no effect. Approximately half of the binary effect sizes were in the range of 0.75 to 1.25, and 64.2% reported an effect estimate <1. The test for subgroup difference comparing BoE from RCTs and BoE of cohort studies was statistically significant (p<0.05) for 25 BoE-pairs (21.2%). By pooling BoE from RCTs and cohort studies, the median I2 was 51% (τ2=0.05), whereas the mean I2 was 45% (τ2=0.11). The contributed weight of RCTs to the pooled estimates was 40% (median) and 42% (mean). As for the 95% PIs, 21.2% (n=25) of the pooled BoE from RCTs and cohort studies excluded no effect.
The direction of effect between BoE from RCTs and pooled effect estimates was mainly concordant in 94 of 118 BoE-pairs (79.7%). The difference between effect estimates was >0.25 for 4.2% (n=5) and >0.50 for 3.4% (n=4) for the dichotomous effect measures. The integration of BoE from cohort studies modified the conclusion from BoE of RCTs in 32 (27.1%) of the 118 BoE-pairs (i.e., 95% CI excludes no effect changed to 95% CI overlaps no effect or vice versa); in 28 (87.5%) of these 32 BoE, the direction of effect was concordant. In nine (28.1%) of these 32 BoE-pairs, the test of subgroup difference was statistically significant (p<0.05) comparing BoE from RCTs and BoE from cohort studies (in three of these nine associations, the direction of effect was opposite). In 12 (37.5 %) of these 32 BoE-pairs, the overall degree of PI/ECO similarity was judged as “broadly similar.” Populations (n=7, 21.9%), interventions (n=5, 15.6%), and comparators (n=4, 12.5%) rated as “broadly similar” accounted for PI/ECO dissimilarities overall. In 20 (62.5%) of the 32 BoE-pairs, the degree of PI/ECO similarity was judged as “similar but not identical.” Populations (n=18; 56.3%), interventions (n=11; 34.4%), comparators (n=7; 21.9%), and outcomes (n=1; 3.1%) rated as “similar but not identical” accounted for PI/ECO dissimilarities.
In the additional analysis with cohort studies as reference (Additional file 1: Table S5), the direction of effect between BoE from cohort studies and pooled estimates was concordant in 106 (89.8%) of the 118 BoE-pairs. The integration of BoE from RCTs modified the conclusion from BoE of cohort studies in 24 (20.3%) of the 118 BoE-pairs.
Discussion
Summary of findings
This meta-epidemiological study is the first empirical study in medical research that evaluates the impact scenario of pooling bodies of evidence from RCTs and cohort studies. Overall, 118 BoE-pairs based on 653 RCTs and 804 cohort studies were included. By pooling BoE from RCTs and cohort studies in about 50% of the BoE-pairs, the 95% CI excludes no effect, whereas in about one-third of the included BoE from RCTs, the 95% CI excludes no effect. For 21% of pooled estimates, the test for subgroup difference comparing BoE from RCTs and BoE of cohort studies was statistically significant. The median weights of BoE from RCTs to the pooled estimates were 40%, suggesting that on average the contribution weight was not dissimilar between both BoE. Overall, the degree of statistical heterogeneity was moderate (I2=51%, τ2=0.05) and higher across meta-analyses of cohort studies (I2=41%, τ2=0.03) compared to meta-analyses of RCTs (I2=5%, τ2=0.00). The integration of BoE from cohort studies modified the conclusion derived from BoE of RCTs in nearly 30% of the BoE-pairs. The direction of effect between BoE of RCTs and pooled estimates, however, was mainly concordant. This suggests that by adding evidence from cohort studies statistical precision increased substantially.
Comparison with other studies
We did not identify any similar empirical study using a pooling scenario of different study designs in the field of medical research. However, a recent methodological study investigated a similar pooling scenario in nutrition research [77]. This large pooling scenario study showed that the integration of BoE from cohort studies modified the conclusion from BoE of RCTs in nearly 50% of included diet-disease associations, although the direction of effect was mainly concordant between BoE of RCTs and pooled estimates. The median weight of RCTs to the pooled estimates was 34%, and the statistical heterogeneity was substantially higher across meta-analyses of cohort studies (I2=55%, τ2=0.01) compared to RCTs (I2=0%, τ2=0). This finding is in line with our study. However, in our study, the integration of BoE from cohort studies modified the conclusion from BoE of RCTs less often (27% vs. 44%) [77]. Two main reasons may explain this difference. First, it has been suggested that effect estimates between RCTs and cohort studies differ quite often in nutrition research [78]. A recent meta-epidemiological study, however, has shown that on average the effect-difference between both study designs was even smaller than expected [79]. Second, the median weight of RCTs to the pooled estimated was larger in our study (40% vs. 34%) [79].
A recent meta-research study investigated how RCTs and observational studies were combined in meta-analyses [80]. In nearly 40% of meta-analyses, both observational studies and RCTs were combined in a single meta-analysis, without considering the two designs as subgroups. When comparing the results of those meta-analyses with meta-analyses restricted to RCTs only, the conclusion was modified by the integration of observational studies for nearly 71%, whereas in our study this was the case for 27%. In line with our findings, the authors found that including observational studies frequently increased statistical heterogeneity.
Implications for the broader research field
In a survey investigating the rationale, perceptions, and preferences for the integration of RCTs and observational studies in evidence syntheses by Cuello-Garcia and colleagues [81], it was shown that conducting separate meta-analyses for both study designs was the most frequent approach used. However, nearly half of the experts interviewed reported that they have already, on at least one occasion, pooled RCTs and observational studies in a meta-analysis [81].
According to the recent GRADE guidance on optimizing the integration of RCT and observational studies in evidence syntheses, observational studies can provide valuable information as complementary, sequential, or replacement evidence for RCTs [6]. In our empirical scenario, evidence from cohort studies was always considered as complementary evidence for RCTs. The GRADE guidance suggests, when RCTs provide already high certainty of evidence, looking for observational evidence will be unnecessary because the high certainty will not be improved [6]. However, in our sample of 118 BoE-pairs, only six BoE of RCTs were rated as high certainty, 18 as moderate, 11 as low, and two as very low. Thus, evidence from cohort studies seems valuable in the field of medical research [7].
In line with our findings, the Cochrane Handbook indicated that authors should expect greater statistical heterogeneity in a systematic review of observational studies compared to a systematic review of RCTs. Reasons include diverse ways in which observational studies may be designed to investigate the effects of interventions/exposures, and partly due to the increased potential for methodological variation between primary studies and the resulting variation in their risk of bias. Therefore, the Cochrane Handbook recommends that RCTs and observational studies should not be combined in a meta-analysis (although the power to detect an effect may increase [82]). In contrast to the recommendations of Cochrane, a recent framework for the synthesis of observational studies and RCTs does not reject the pooling of both study designs in principle. It presents recommendations on when and how to combine evidence from different study designs, but also highlights challenges in this process [83]. Moreover, a recent scoping review summarized the methods to systematically review and meta-analyze observational studies and highlighted that existing guidance is highly conflicting for pooling if results are similar over different study designs [84]. Finally, in several high-impact factor journal meta-analyses, both study designs were pooled [21, 32, 36, 56].
Overall, it looks like further methodological research is needed to shed light into this gray area. On the one hand, further research should address the application of existing guidance in terms of utility, acceptability, and reproducibility and elaborate ways to deal with occurring challenges [83]. On the other hand, factors such as risk of bias/study quality that may contribute to the differences in effect estimates between BoE of RCTs and cohort studies and conflicting results in pooling scenarios should be further explored. Our previously conducted study analyzed disagreement of effect estimates with regard to differences by each PI/ECO domain [7]. In the meta-regression, we showed that differences of interventions were the main drivers towards disagreement. The average effect on the other pooled effect estimators, however, was not statistically significant [7].
We assume that methodological trial characteristics are other possible drivers towards disagreement, since observational studies are prone to risk of bias by confounding [5], and appropriate adjustment for confounding is thus crucial to integrate both RCTs and cohort studies (or other non-randomized studies) in a pooling scenario. In the sample provided in this study, the tools used to assess the quality/risk of bias of primary studies included across the BoE were heterogeneous, which makes the comparison of results challenging. Future studies should focus on the impact of quality characteristics on pooling scenarios by using similar appraisal tools to increase comparability between RCTs and cohort studies (e.g., ROBINS-I [85] and the Cochrane Risk of Bias Tool [86]). Moreover, attention in future studies should also be paid to the integration of other non-randomized study designs a part from cohort studies. However, overall, we assume generalizability of our findings since concordance may not be linked to study design per se, but rather on the quality/risk of bias of the studies included [1].
This paper did not aim to provide insights on how pooling results from different study designs impacts the certainty rating of results and whether it reduces or increases the amount of low or very low certainty of evidence ratings. In a recently published hypothetical scenario analysis, we could show that pooling BoE from RCTs and cohort studies for nutrition-related research questions would reduce the amount of very low and low certainty of evidence ratings [87]. We recommend that future research should examine also the impact scenario of pooling BoE of RCTs and cohort studies for medical research questions on the overall GRADE rating and on individual GRADE domains in order to inform future guidance development.
Strengths and limitations
This study has several strengths. First, we analyzed a large sample of BoE-pairs (n=118), which was based on 653 RCTs and 804 cohort studies. Second, we selected BoE-pairs from systematic reviews published in high-impact medical journals, which have shown to be of higher methodological quality [88]. Third, our study was based on a broad methodological repertoire, i.e., by including meta-analyses of binary outcomes, and also continuous outcomes, investigating different statistical measures of heterogeneity, conducting random and common effects models, and calculating 95% PI.
Limitations of this study are as follows. First, although we pooled a large sample of BoE-pairs, our sample may not be representative of all meta-analyses, and the totality of evidence of available associations might provide different results. Second, we did not consider and weighted risk of bias of primary studies in our pooling scenario. Third, only two BoE-pairs were judged as “more or less identical,” indicating that BoE of RCTs and cohort studies differ at least slightly in terms of PI/ECO criteria and caution is therefore required when pooling both BoE. Fourth, the potential for confounding in the individual cohort studies and subgroup analyses in the meta-analysis cannot be ruled out. Moreover, several subgroups also included only a small number of studies. Fifth, the methodological quality of the systematic reviews included in this study was not assessed. Although we assume that systematic reviews published in high-impact factor journals adhere to high methodological standards, this is nevertheless an important limitation. Due to these limitations, our findings need to be interpreted with caution.
Conclusions
This large pooling scenario study showed that the integration of BoE from cohort studies modified the conclusion from BoE of RCTs in 27% of included BoE, although the direction of effect was mainly concordant between BoE of RCTs and pooled estimates. The median weight of RCTs to the pooled estimates was 40%, and the statistical heterogeneity was substantially driven by integrating BoE of cohort studies. Our findings provide a first insight regarding the potential impact of pooling of both BoE in evidence syntheses. A decision for or against pooling different study designs should also always take into account, for example, PI/ECO similarity, risk of bias, coherence of effect estimates, and also the trustworthiness of the evidence. Overall, there is a need for more research on the influence of those issues on potential pooling.
Availability of data and materials
Data are based on published meta-analyses.
Abbreviations
- BoE:
-
Bodies of evidence
- CI:
-
Confidence interval
- GRADE:
-
Grading of Recommendations, Assessment, Development and Evaluation
- PI:
-
Prediction interval
- PI/ECO:
-
Patient/population, intervention/exposure, comparator, outcome
- RCT:
-
Randomized controlled trial
- τ 2 :
-
Heterogeneity value with the restricted maximum-likelihood estimation method
References
Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016;21(4):125.
Oxford Centre. EBM Levels of Evidence Working Group. In: Jeremy Howick ICJLL, Glasziou P, Greenhalgh T, Heneghan C, Liberati A, Moschetti I, Phillips B, Thornton H, Goddard O, Hodgkinson M, editors. The Oxford 2011 levels of evidence. Oxford Centre for Evidence-Based Medicine; 2011.
Kabisch M, Ruckes C, Seibert-Grafe M, Blettner M. Randomized controlled trials: part 17 of a series on evaluation of scientific publications. Dtsch Arztebl Int. 2011;108(39):663–8.
Kostis JB, Dobrzynski JM. Limitations of randomized clinical trials. Am J Cardiol. 2020;129:109–15.
JPT H, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions version 6.2 (updated February 2021). Cochrane. 2021; Available from www.training.cochrane.org/handbook.
Cuello-Garcia CA, Santesso N, Morgan RL, Verbeek J, Thayer K, Ansari MT, et al. GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines. J Clin Epidemiol. 2022;142:200–8.
Bröckelmann N, Balduzzi S, Harms L, Beyerbach J, Petropoulou M, Kubiak C, et al. Evaluating agreement between bodies of evidence from randomized controlled trials and cohort studies in medical research: a meta-epidemiological study. BMC Med. 2022;20(1):174.
Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med. 2017;22(4):139–42.
Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. Bmj. 2003;327(7414):557–60.
Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539–58.
Riley RD, Higgins JP, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549.
Review Manager (RevMan). Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration; 2014.
Abou-Setta AM, Beaupre LA, Rashiq S, Dryden DM, Hamm MP, Sadowski CA, et al. Comparative effectiveness of pain management interventions for hip fracture: a systematic review. Ann Intern Med. 2011;155(4):234–45.
Aburto NJ, Ziolkovska A, Hooper L, Elliott P, Cappuccio FP, Meerpohl JJ. Effect of lower sodium intake on health: systematic review and meta-analyses. BMJ. 2013;346:f1326.
Ahmad Y, Sen S, Shun-Shin MJ, Ouyang J, Finegold JA, Al-Lamee RK, et al. Intra-aortic balloon pump therapy for acute myocardial infarction: a meta-analysis. JAMA Intern Med. 2015;175(6):931–9.
Alipanah N, Jarlsberg L, Miller C, Linh NN, Falzon D, Jaramillo E, et al. Adherence interventions and outcomes of tuberculosis treatment: a systematic review and meta-analysis of trials and observational studies. PLoS Med. 2018;15(7):e1002595.
Anglemyer A, Rutherford GW, Horvath T, Baggaley RC, Egger M, Siegfried N. Antiretroviral therapy for prevention of HIV transmission in HIV-discordant couples. Cochrane Database Syst Rev. 2013;4:Cd009153.
Azad MB, Abou-Setta AM, Chauhan BF, Rabbani R, Lys J, Copstein L, et al. Nonnutritive sweeteners and cardiometabolic health: a systematic review and meta-analysis of randomized controlled trials and prospective cohort studies. Can Med Assoc J. 2017;189(28):E929.
Barnard S, Kim C, Park MH, Ngo TD. Doctors or mid-level providers for abortion. Cochrane Database Syst Rev. 2015;2015(7):Cd011242.
Bellemain-Appaix A, O'Connor SA, Silvain J, Cucherat M, Beygui F, Barthelemy O, et al. Association of clopidogrel pretreatment with mortality, cardiovascular events, and major bleeding among patients undergoing percutaneous coronary intervention: a systematic review and meta-analysis. JAMA. 2012;308(23):2507–16.
Bellemain-Appaix A, Kerneis M, O'Connor SA, Silvain J, Cucherat M, Beygui F, et al. Reappraisal of thienopyridine pretreatment in patients with non-ST elevation acute coronary syndrome: a systematic review and meta-analysis. BMJ. 2014;349:g6269.
Bloomfield HE, Koeller E, Greer N, MacDonald R, Kane R, Wilt TJ. Effects on health outcomes of a Mediterranean diet with no restriction on fat intake: a systematic review and meta-analysis. Ann Intern Med. 2016;165(7):491–500.
Bolland MJ, Leung W, Tai V, Bastin S, Gamble GD, Grey A, et al. Calcium intake and risk of fracture: systematic review. BMJ. 2015;351:h4580.
Brenner H, Stock C, Hoffmeister M. Effect of screening sigmoidoscopy and screening colonoscopy on colorectal cancer incidence and mortality: systematic review and meta-analysis of randomised controlled trials and observational studies. BMJ. 2014;348:g2467.
Chowdhury R, Stevens S, Gorman D, Pan A, Warnakula S, Chowdhury S, et al. Association between fish consumption, long chain omega 3 fatty acids, and risk of cerebrovascular disease: systematic review and meta-analysis. BMJ. 2012;345:e6698.
Chowdhury R, Kunutsor S, Vitezova A, Oliver-Williams C, Chowdhury S, Kiefte-de-Jong JC, et al. Vitamin D and risk of cause specific death: systematic review and meta-analysis of observational cohort and randomised intervention studies. BMJ. 2014;348:g1903.
Chung M, Tang AM, Fu Z, Wang DD, Newberry SJ. Calcium intake and cardiovascular disease risk: an updated systematic review and meta-analysis. Ann Intern Med. 2016;165(12):856–66.
Ding M, Huang T, Bergholdt HK, Nordestgaard BG, Ellervik C, Qi L. Dairy consumption, systolic blood pressure, and risk of hypertension: Mendelian randomization study. BMJ. 2017;356:j1000.
Fenton JJ, Weyrich MS, Durbin S, Liu Y, Bang H, Melnikow J. Prostate-specific antigen-based screening for prostate cancer: evidence report and systematic review for the US Preventive Services Task Force. JAMA. 2018;319(18):1914–31.
Filippini G, Del Giovane C, Clerico M, Beiki O, Mattoscio M, Piazza F, et al. Treatment with disease-modifying drugs for people with a first clinical attack suggestive of multiple sclerosis. Cochrane Database Syst Rev. 2017;4:Cd012200.
Fluri F, Engelter S, Lyrer P. Extracranial-intracranial arterial bypass surgery for occlusive carotid artery disease. Cochrane Database Syst Rev. 2010;2010(2):Cd005953.
Gargiulo G, Sannino A, Capodanno D, Barbanti M, Buccheri S, Perrino C, et al. Transcatheter aortic valve implantation versus surgical aortic valve replacement: a systematic review and meta-analysis. Ann Intern Med. 2016;165(5):334–44.
Hartling L, Dryden DM, Guthrie A, Muise M, Vandermeer B, Donovan L. Benefits and harms of treating gestational diabetes mellitus: a systematic review and meta-analysis for the U.S. Preventive Services Task Force and the National Institutes of Health Office of Medical Applications of Research. Ann Intern Med. 2013;159(2):123–9.
Henderson JT, Webber EM, Bean SI. Screening for asymptomatic bacteriuria in adults: updated evidence report and systematic review for the US Preventive Services Task Force. JAMA. 2019;322(12):1195–205.
Higgins JP, Soares-Weiser K, Lopez-Lopez JA, Kakourou A, Chaplin K, Christensen H, et al. Association of BCG, DTP, and measles containing vaccines with childhood mortality: systematic review. Bmj. 2016;355:i5170.
Hopley C, Stengel D, Ekkernkamp A, Wich M. Primary total hip arthroplasty versus hemiarthroplasty for displaced intracapsular hip fractures in older patients: systematic review. Bmj. 2010;340:c2332.
Hüpfl M, Selig HF, Nagele P. Chest-compression-only versus standard cardiopulmonary resuscitation: a meta-analysis. Lancet. 2010;376(9752):1552–7.
Jamal SA, Vandermeer B, Raggi P, Mendelssohn DC, Chatterley T, Dorgan M, et al. Effect of calcium-based versus non-calcium-based phosphate binders on mortality in patients with chronic kidney disease: an updated systematic review and meta-analysis. Lancet. 2013;382(9900):1268–77.
Jefferson T, Di Pietrantonj C, Al-Ansary LA, Ferroni E, Thorning S, Thomas RE. Vaccines for preventing influenza in the elderly. Cochrane Database Syst Rev. 2010;17(2):Cd004876.
Jefferson T, Rivetti A, Di Pietrantonj C, Demicheli V, Ferroni E. Vaccines for preventing influenza in healthy children. Cochrane Database Syst Rev. 2012;2012(8):Cd004879.
Jin H, Leng Q, Li C. Dietary flavonoid for preventing colorectal neoplasms. Cochrane Database Syst Rev. 2012;15(8):Cd009350.
Kansagara D, Dyer E, Englander H, Fu R, Freeman M, Kagen D. Treatment of anemia in patients with heart disease: a systematic review. Ann Intern Med. 2013;159(11):746–57.
Keag OE, Norman JE, Stock SJ. Long-term risks and benefits associated with cesarean delivery for mother, baby, and subsequent pregnancies: systematic review and meta-analysis. PLoS Med. 2018;15(1):e1002494.
Kredo T, Adeniyi FB, Bateganya M, Pienaar ED. Task shifting from doctors to non-doctors for initiation and maintenance of antiretroviral therapy. Cochrane Database Syst Rev. 2014;1(7):Cd007331.
Li L, Shen J, Bala MM, Busse JW, Ebrahim S, Vandvik PO, et al. Incretin treatment and risk of pancreatitis in patients with type 2 diabetes mellitus: systematic review and meta-analysis of randomised and non-randomised studies. Bmj. 2014;348:g2366.
Li L, Li S, Deng K, Liu J, Vandvik PO, Zhao P, et al. Dipeptidyl peptidase-4 inhibitors and risk of heart failure in type 2 diabetes: systematic review and meta-analysis of randomised and observational studies. Bmj. 2016;352:i610.
Matthews A, Stanway S, Farmer RE, Strongman H, Thomas S, Lyon AR, et al. Long term adjuvant endocrine therapy and risk of cardiovascular disease in female breast cancer survivors: systematic review. Bmj. 2018;363:k3845.
Menne J, Dumann E, Haller H, Schmidt BMW. Acute kidney injury and adverse renal events in patients receiving SGLT2-inhibitors: a systematic review and meta-analysis. PLoS Med. 2019;16(12):e1002983.
Mesgarpour B, Heidinger BH, Roth D, Schmitz S, Walsh CD, Herkner H. Harms of off-label erythropoiesis-stimulating agents for critically ill people. Cochrane Database Syst Rev. 2017;8:Cd010969.
Moberley S, Holden J, Tatham DP, Andrews RM. Vaccines for preventing pneumococcal infection in adults. Cochrane Database Syst Rev. 2013;23(1):Cd000422.
Molnar AO, Fergusson D, Tsampalieros AK, Bennett A, Fergusson N, Ramsay T, et al. Generic immunosuppression in solid organ transplantation: systematic review and meta-analysis. Bmj. 2015;350:h3163.
Navarese EP, Gurbel PA, Andreotti F, Tantry U, Jeong YH, Kozinski M, et al. Optimal timing of coronary invasive strategy in non-ST-segment elevation acute coronary syndromes: a systematic review and meta-analysis. Ann Intern Med. 2013;158(4):261–70.
Nelson RL, Furner SE, Westercamp M, Farquhar C. Cesarean delivery for the prevention of anal incontinence. Cochrane Database Syst Rev. 2010;2010(2):Cd006756.
Nieuwenhuijse MJ, Nelissen RG, Schoones JW, Sedrakyan A. Appraisal of evidence base for introduction of new implants in hip and knee replacement: a systematic review of five widely used device technologies. Bmj. 2014;349:g5133.
Nikooie R, Neufeld KJ, Oh ES, Wilson LM, Zhang A, Robinson KA, et al. Antipsychotics for treating delirium in hospitalized adults: a systematic review. Ann Intern Med. 2019;171:485–95.
Ochen Y, Beks RB, van Heijl M, Hietbrink F, Leenen LPH, van der Velde D, et al. Operative treatment versus nonoperative treatment of Achilles tendon ruptures: systematic review and meta-analysis. Bmj. 2019;364:k5120.
Pittas AG, Chung M, Trikalinos T, Mitri J, Brendel M, Patel K, et al. Systematic review: vitamin D and cardiometabolic outcomes. Ann Intern Med. 2010;152(5):307–14.
Raman G, Moorthy D, Hadar N, Dahabreh IJ, O'Donnell TF, Thaler DE, et al. Management strategies for asymptomatic carotid stenosis: a systematic review and meta-analysis. Ann Intern Med. 2013;158(9):676–85.
Schweizer M, Perencevich E, McDanel J, Carson J, Formanek M, Hafner J, et al. Effectiveness of a bundled intervention of decolonization and prophylaxis to decrease Gram positive surgical site infections after cardiac or orthopedic surgery: systematic review and meta-analysis. Bmj. 2013;346:f2743.
Silvain J, Beygui F, Barthelemy O, Pollack C, Cohen M, Zeymer U, et al. Efficacy and safety of enoxaparin versus unfractionated heparin during percutaneous coronary intervention: systematic review and meta-analysis. Bmj. 2012;344:e553.
Suthar AB, Lawn SD, del Amo J, Getahun H, Dye C, Sculier D, et al. Antiretroviral therapy for prevention of tuberculosis in adults with HIV: a systematic review and meta-analysis. PLoS Med. 2012;9(7):e1001270.
Te Morenga L, Mallard S, Mann J. Dietary sugars and body weight: systematic review and meta-analyses of randomised controlled trials and cohort studies. Bmj. 2013;346:e7492.
Thomas RE, Jefferson T, Lasserson TJ. Influenza vaccination for healthcare workers who work with the elderly. Cochrane Database Syst Rev. 2010;17(2):Cd005187.
Tickell-Painter M, Maayan N, Saunders R, Pace C, Sinclair D. Mefloquine for preventing malaria during travel to endemic areas. Cochrane Database Syst Rev. 2017;10:Cd006491.
Tricco AC, Zarin W, Cardoso R, Veroniki AA, Khan PA, Nincic V, et al. Efficacy, effectiveness, and safety of herpes zoster vaccines in adults aged 50 and older: systematic review and network meta-analysis. Bmj. 2018;363:k4029.
Vinceti M, Filippini T, Del Giovane C, Dennert G, Zwahlen M, Brinkman M, et al. Selenium for preventing cancer. Cochrane Database Syst Rev. 2018;1:Cd005195.
Wilson A, Gallos ID, Plana N, Lissauer D, Khan KS, Zamora J, et al. Effectiveness of strategies incorporating training and support of traditional birth attendants on perinatal and maternal mortality: meta-analysis. Bmj. 2011;343:d7102.
Wilson HA, Middleton R, Abram SGF, Smith S, Alvand A, Jackson WF, et al. Patient relevant outcomes of unicompartmental versus total knee replacement: systematic review and meta-analysis. Bmj. 2019;364:l352.
Yank V, Tuohy CV, Logan AC, Bravata DM, Staudenmayer K, Eisenhut R, et al. Systematic review: benefits and harms of in-hospital use of recombinant factor VIIa for off-label indications. Ann Intern Med. 2011;154(8):529–40.
Zhang XL, Zhu L, Wei ZH, Zhu QQ, Qiao JZ, Dai Q, et al. Comparative efficacy and safety of everolimus-eluting bioresorbable scaffold versus everolimus-eluting metallic stents: a systematic review and meta-analysis. Ann Intern Med. 2016;164(11):752–63.
Zhang XL, Zhu QQ, Yang JJ, Chen YH, Li Y, Zhu SH, et al. Percutaneous intervention versus coronary artery bypass graft surgery in left main coronary artery stenosis: a systematic review and meta-analysis. BMC Med. 2017;15(1):84.
Ziff OJ, Lane DA, Samra M, Griffith M, Kirchhof P, Lip GY, et al. Safety and efficacy of digoxin: systematic review and meta-analysis of observational and controlled trial data. Bmj. 2015;351:h4451.
Alexander DD, Miller PE, Van Elswyk ME, Kuratko CN, Bylsma LC. A meta-analysis of randomized controlled trials and prospective cohort studies of eicosapentaenoic and docosahexaenoic long-chain omega-3 fatty acids and coronary heart disease risk. Mayo Clin Proc. 2017;92(1):15–29.
Chung M, Lee J, Terasawa T, Lau J, Trikalinos TA. Vitamin D with or without calcium supplementation for prevention of cancer and fractures: an updated meta-analysis for the U.S. Preventive Services Task Force. Ann Intern Med. 2011;155(12):827–38.
Johnston BC, Zeraatkar D, Han MA, Vernooij RWM, Valli C, El Dib R, et al. Unprocessed red meat and processed meat consumption: dietary guideline recommendations from the Nutritional Recommendations (NutriRECS) Consortium. Ann Intern Med. 2019;171(10):756–64.
Chowdhury R, Warnakula S, Kunutsor S, Crowe F, Ward HA, Johnson L, et al. Association of dietary, circulating, and supplement fatty acids with coronary risk: a systematic review and meta-analysis. Ann Intern Med. 2014;160(6):398–406.
Schwingshackl L, Bröckelmann N, Beyerbach J, Werner SS, Zähringer J, Schwarzer G, et al. An empirical evaluation of the impact scenario of pooling bodies of evidence from randomized controlled trials and cohort studies in nutrition research. Adv Nutr. 2022;nmac042. https://doi.org/10.1093/advances/nmac042.
Trepanowski JF, Ioannidis JPA. Perspective: limiting dependence on nonrandomized studies and improving randomized trials in human nutrition research: why and how. Adv Nutr. 2018;9(4):367–77.
Schwingshackl L, Balduzzi S, Beyerbach J, Bröckelmann N, Werner SS, Zähringer J, et al. Evaluating agreement between bodies of evidence from randomised controlled trials and cohort studies in nutrition research: meta-epidemiological study. Bmj. 2021;374:n1864.
Bun RS, Scheer J, Guillo S, Tubach F, Dechartres A. Meta-analyses frequently pooled different study types together: a meta-epidemiological study. J Clin Epidemiol. 2020;118:18–28.
Cuello-Garcia CA, Morgan RL, Brozek J, Santesso N, Verbeek J, Thayer K, et al. A scoping review and survey provides the rationale, perceptions, and preferences for the integration of randomized and nonrandomized studies in evidence syntheses and GRADE assessments. J Clin Epidemiol. 2018;98:33–40.
Verde PE, Ohmann C. Combining randomized and non-randomized evidence in clinical research: a review of methods and applications. Res Synth Methods. 2015;6(1):45–62.
Sarri G, Patorno E, Yuan H, Guo JJ, Bennett D, Wen X, et al. Framework for the synthesis of non-randomised studies and randomised controlled trials: a guidance on conducting a systematic review and meta-analysis for healthcare decision making. Bmj. 2022;27(2):109–19.
Mueller M, D'Addario M, Egger M, Cevallos M, Dekkers O, Mugglin C, et al. Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations. BMC Med Res Methodol. 2018;18(1):44.
Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Bmj. 2016;355:i4919.
Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. Bmj. 2019;366:l4898.
Schwingshackl L, Nagavci B, Stadelmaier J, Werner SS, Cuello Garcia CA, Schünemann HJ, et al. Pooling of cohort studies and RCTs affects GRADE certainty of evidence in nutrition research. J Clin Epidemiol. 2022;147:151–9.
Fleming PS, Koletsi D, Seehra J, Pandis N. Systematic reviews published in higher impact clinical journals were of higher quality. J Clin Epidemiol. 2014;67(7):754–9.
Acknowledgements
None.
Funding
Open Access funding enabled and organized by Projekt DEAL. LS received by Deutsche Forschungsgemeinschaft (DFG): Projektnummer 459430615.
Author information
Authors and Affiliations
Contributions
NB, JJM, and LS designed the research. NB, LH, CK, JB, and LS analyzed the data and wrote the first draft of the paper. NB, JS, LH, CK, JB, MW, JJM, and LS interpreted the data. The authors read and approved the final manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable since we did not include any human subject.
Consent for publication
Not applicable since we did not include any human subject.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1: Appendix S1.
Search strategy. Tables S1-S5. Table S1. Explanation and definition for PI/ECO similarity degree. Table S2. PI/ECO similarity degree. Table S3. Differences between published (reported) effect estimates and re-calculated effect estimates. Table S4. Reason for exclusion from the pooling scenario. Table S5. Pooling results. Figures S1-S118. Fig S1. Forest plot: Low sodium (Intervention/Exposure); All-cause mortality (Outcome). Fig. S2. Forest plot: Low sodium; Cardiovascular disease. Fig. S3. Forest plot: Intra-aortic balloon pump; All-cause mortality. Fig. S4. Forest plot: Self-administered therapy; Treatment success. Fig S5. Forest plot: Self-administered therapy; Treatment completion. Fig. S6. Forest plot: Self-administered therapy; All-cause mortality. Fig S7. Forest plot: Antiretroviral therapy; HIV infection. FigS8-Forest plot: Nonnutritive sweeteners; Body Mass Index random sequence. Fig. S9. Forest plot: Surgical abortion by mid-level providers; Failure or incomplete abortion. Fig. S10. Forest plot: Surgical abortion by mid-level providers; Complications. Fig. S11. Forest plot: Surgical abortion by mid-level providers; Abortion failure and complications. Fig. S12. Forest plot: Clopidogrel pretreatment for percutaneous coronary intervention; All-cause mortality. Fig. S13. Forest plot: Clopidogrel pretreatment for percutaneous coronary intervention; Major bleeding. Fig. S14. Forest plot: Clopidogrel pretreatment for percutaneous coronary intervention; Coronary heart disease. Fig. S15. Forest plot: P2Y12 inhibitor pretreatment in non-ST elevation acute coronary syndrome; All-cause mortality. Fig. S16. Forest plot: P2Y12 inhibitor pretreatment in non-ST elevation acute coronary syndrome; Major bleeding. Fig. S17. Forest plot: P2Y12 inhibitor pretreatment in non-ST elevation acute coronary syndrome; Main composite ischemic endpoint. Fig. S18. Forest plot: Mediterranean diet; Breast cancer. Fig. S19. Forest plot: High calcium; All fractures. Fig. S20. Forest plot: High calcium; Verterbral fractures. Fig. S21. Forest plot: High calcium; Hip fracture. Fig. S22. Forest plot: Sigmoidoscopy; Colorectal cancer mortality. Fig. S23. Forest plot: Sigmoidoscopy; Colorectal cancer incidence. Fig. S24. Forest plot: High omega-3; Cerebrovascular disease. Fig. S25. Forest plot: High α-linolenic acid; Coronary heart disease. Fig. S26. Forest plot: High omega-3; Coronary heart disease. Fig. S27. Forest plot: Omega-6; Coronary heart disease. Fig. S28. Forest plot: High calcium; Cardiovascular mortality. Fig. S29. Forest plot: High dairy; Systolic blood pressure. Fig. S30. Forest plot: Radiation therapy; Erectile dysfunction. Fig. S31. Forest plot: Radical prostatectomy; Urinary incontinence. Fig. S32. Forest plot: Radical Prostatectomy; Erectile dysfunction. Fig. S33. Forest plot: Disease-modifying drugs; Conversion to clinically definite multiple sclerosis. Fig. S34. Forest plot: Extracranial-intracranial arterial bypass; All-cause mortality. Fig. S35. Forest plot: Extracranial-intracranial arterial bypass; Stroke. Fig. S36. Forest plot: Extracranial-intracranial arterial bypass; Stroke mortality or dependency. Fig. S37. Forest plot: Transcatheter aortic valve implantation; Early all-cause mortality. Fig. S38. Forest plot: Transcatheter aortic valve implantation; Mid-term all-cause mortality. Fig. S39. Forest plot: Transcatheter aortic valve implantation; Long-term all-cause mortality. Fig. S40. Forest plot: Treating gestational diabetes mellitus; High birth weight. Fig. S41. Forest plot: Treating gestational diabetes mellitus; Large-for-gestational age neonate. Fig. S42. Forest plot: Treating gestational diabetes mellitus; Shoulder dystocia. Fig. S43. Forest plot: Treating asymptomatic bacteriuria; Pyelonephritis. Fig. S44. Forest plot: Bacillus Calmette-Guérin vaccination; All-cause mortality. Fig. S45. Forest plot: Measles containing vaccines; All-cause mortality. Fig. S46. Forest plot: Total hip arthroplasty; Reoperation. Fig. S47. Forest plot: Total hip arthroplasty; Dislocation. Fig. S48. Forest plot: Total hip arthroplasty; Deep infection. Fig. S49. Forest plot: Chest-compression-only cardiopulmonary resuscitation; Survival. Fig. S50. Forest plot: Non-calcium-based phosphat binders; All-cause mortality. Fig. S51. Forest plot: Parenteral influenza vaccine; Influenza-like illness. Fig. S52. Forest plot: Parenteral influenza vaccine Influenza. FigS53-Forest plot: Inactivated influenza vaccines; Influenza. Fig. S54. Forest plot: Inactivated influenza vaccines; Influenza-like illness. Fig. S55. Forest plot: High total flavonoids; Colorectal neoplasms. Fig. S56. Forest plot: Transfusion; All-cause mortality. Fig. S57. Forest plot: Caesarean section; Urinary incontinence. Fig. S58. Forest plot: Caesarean section; Fecal incontinence. Fig. S59. Forest plot: Antiretroviral therapy by nurses; All-cause mortality. Fig. S60. Forest plot: Antiretroviral therapy by nurses; Attrition. Fig. S61. Forest plot: Nurses for maintenance of antiretroviral therapy; All-cause mortality. Fig. S62. Forest plot: Exenatide; Acute pancreatitis. Fig. S63. Forest plot: DDP-4 inhibitors; Heart failure. Fig. S64. Forest plot: DDP-4 inhibitors; Hospital admission for heart failure. Fig. S65. Forest plot: Tamoxifen; Heart failure. Fig. S66. Forest plot: SGLT-2 inhibitors; Acute kidney injury. Fig. S67. Forest plot: Erythropoiesis stimulating agents; Venous thromboembolism. Fig. S68. Forest plot: Erythropoiesis stimulating agents; All-cause mortality. Fig. S69. Forest plot: Pneumococcal polysaccharide vaccines; Invasive pneumococcal disease. Fig. S70. Forest plot: Neoral (Cyclosporin); Acute rejection of kidney transplant. Fig. S71. Forest plot: Early intervention for NSTE-ACS; All-cause mortality. Fig. S72. Forest plot: Early intervention for NSTE-ACS; Myocardial infarction. Fig. S73. Forest plot: Early intervention for NSTE-ACS; Major bleeding. Fig. S74. Forest plot: Caesarean section; Anal incontinence; feces. Fig. S75. Forest plot: Caesarean section; Anal incontinence; flatus. Fig. S76. Forest plot: Ceramic-on-ceramic bearings for total hip arthroplasty; Harris Hip Score. Fig. S77. Forest plot: High-flexion total knee arthroplasty; Flexion in degrees. Fig. S78. Forest plot: Gender-specific total knee arthroplasty; Flexion-extension range. Fig. S79. Forest plot: Second generation antipsychotics; Sedation. Fig. S80. Forest plot: Second generation antipsychotics; Neurologic outcomes. Fig. S81. Forest plot: Surgery for achilles tendon rupture; Re-rupture. Fig. S82. Forest plot: Surgery for achilles tendon rupture; Complications. Fig. S83. Forest plot: High vitamin D; Hypertension. Fig. S84. Forest plot: Carotid endarterectomy; Ipsilateral stroke. FigS85-Forest plot: Carotid endarterectomy; Stroke. Fig. S86. Forest plot: Carotid artery stenting; Periprocedural stroke. Fig. S87. Forest plot: Nasal deconolization; Surgical site infection. Fig. S88. Forest plot: Glycopeptide prophylaxis; Surgical site infection. Fig. S89. Forest plot: Enoxaparin; All-cause mortality. Fig. S90. Forest plot: Enoxaparin; Major bleeding. Fig. S91. Forest plot: Enoxaparin; All-cause mortality or myocardial infarction. Fig. S92. Forest plot: Antiretroviral therapy; Tuberculosis infection. Fig. S93. Forest plot: High sugar intake; Weight gain. Fig. S94. Forest plot: High sugar intake; Body Mass Index. Fig. S95. Forest plot: Influenza vaccines; Influenza-like illness. Fig. S96. Forest plot: Mefloquine; Discontinuation due to adverse effects. Fig. S97. Forest plot: Mefloquine; Serious adverse events or effects. Fig. S98. Forest plot: Mefloquine; Nausea. Fig. S99. Forest plot: Live-attenuated zoster vaccines; Suspected Herpes Zoster. Fig. S100. Forest plot: High selenium; Cancer. Fig. S101. Forest plot: High selenium; Cancer mortality. FigS102-Forest plot: High selenium; Colorectal cancer. FigS103-Forest plot: Training for traditional birth attendants/ assistance by traditional birth attendants; Perinatal mortality. Fig. S104. Forest plot: Training for traditional birth attendants/ assistance by traditional birth attendants; Neonatal mortality. Fig. S105. Forest plot: Unicompartimental knee arthroplasty; Venous thromboembolism. Fig. S106. Forest plot: Unicompartimental knee arthroplasty; Flexion-extension range. Fig. S107. Forest plot: Unicompartimental knee arthroplasty; Operation duration. Fig. S108. Forest plot: Recombinant factor VII; All-cause mortality. Fig. S109. Forest plot: Recombinant factor VII; Thromboembolism. Fig. S110. Forest plot: Everolimus-eluting bioresorbable vascular scaffold; Stent thrombosis. Fig. S111. Forest plot: Everolimus-eluting bioresorbable vascular scaffold; All-cause mortality. Fig. S112. Forest plot: Everolimus-eluting bioresorbable vascular scaffold; Coronary heart disease mortality. Fig. S113. Forest plot: Percutaneous coronary intervention; All-cause mortality. Fig. S114. Forest plot: Percutaneous coronary intervention; Cardiovascular mortality. Fig. S115. Forest plot: Percutaneous coronary intervention; Myocardial infarction. Fig. S116. Forest plot: Digoxin; All-cause mortality. Fig. S117. Forest plot: Digoxin; Cardiovascular mortality. Fig. S118. Forest plot: Digoxin; Hospital admission.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Bröckelmann, N., Stadelmaier, J., Harms, L. et al. An empirical evaluation of the impact scenario of pooling bodies of evidence from randomized controlled trials and cohort studies in medical research. BMC Med 20, 355 (2022). https://doi.org/10.1186/s12916-022-02559-y
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12916-022-02559-y