Current Hypertension Reports

, 19:71 | Cite as

Quality of Meta-Analyses for Randomized Trials in the Field of Hypertension: an Updated and Improved Systematic Review

  • George C Roush
  • Fiorella Perez
  • Ramy Abdelfattah
  • Andrew Prindle
  • Elie Jean
  • Tanveer Singh
  • John B. Kostis
  • William J. Kostis
  • William J. Elliott
  • Jesse A. Berlin
Guidelines/Clinical Trials/Meta-Analysis (JB Kostis, Section Editor)
Part of the following topical collections:
  1. Topical Collection on Guidelines/Clinical Trials/Meta-Analysis


Publications of hypertension-related meta-analyses (MAs) have increased exponentially in the past 25 years and now average 8/month. Theoretically, this is facilitating evidence-based management of patients. However, some practitioners and authors of guidelines have questioned the quality of published MAs. By extending a prior review, we have assessed the quality of 212 hypertension-related meta-analyses over 5 years based on systematically searching three computerized libraries. Seventeen criteria grouped into four domains of quality yielded the following results: (1) Assessment of trial quality was accomplished in 89% of MAs, and 38% analyzed trials in subgroups of trial quality where appropriate. (2) All three measures of heterogeneity (I 2, tau, and P for heterogeneity) were reported in 36%, reflecting the failure to report tau, the standard deviation of the main effect. (3) Publication bias was assessed in 75%, and 43% of MAs used a statistical test for publication bias. (4) Regarding transparency, 9 to 31% of MAs reported problems in the previous three domains in the article’s abstract. Journal impact factor reporting the MAs declined significantly over 5 years. The percent with criteria of quality in a MA was modestly correlated with journal impact factor (R 2 = 0.05, P = 0.001). False-positive results from inappropriate application of the DerSimonian-Laird model affected 25% of articles, which reported these false positives in the article’s abstract in 72%. No more than 25% of MAs had 67% or more of the criteria of quality. In conclusion, skepticism of hypertension-related MAs is justified, but their quality can be readily corrected.


Meta-analysis as topic Review (publication type) Hypertension Systematic review Randomized controlled trials (publication type) Blood pressure 


Compliance with Ethics Standards

Conflict of Interest

The authors declare that they have no conflicts of interest.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Supplementary material

11906_2017_765_Fig1_ESM.pdf (5.9 mb)
ESM 1 (PDF 6073 kb)


  1. 1.
    Greenhalgh T. How to read a paper: the basics of evidence based medicine. BMJ Books, 2001, page 54.Google Scholar
  2. 2.
    Buyse M, Piedbois P, Piedbois Y, Carlson RW. Meta-analysis: methods, strengths, and weaknesses. Oncology (Williston Park). 2000;14(3):437–43.Google Scholar
  3. 3.
    Chung KC, Burns PB, Kim HM. A practical guide to meta-analysis. J Hand Surg Am. 2006;31(10):1671–8.CrossRefPubMedGoogle Scholar
  4. 4.
    Murad MH, Montori VM, Ioannidis JP, Jaeschke R, Devereaux PJ, Prasad K, et al. How to read a systematic review and meta-analysis and apply the results to patient care: users’ guides to the medical literature. JAMA. 2014;312(2):171–9.CrossRefPubMedGoogle Scholar
  5. 5.
    Tabas GH, Ende J, Aronowitz PB, Conigliaro RL, Granieri R, Green EH et al. American College of Physicians. General Internal Medicine. MKSAP 16. 2012. Page 3.Google Scholar
  6. 6.
    National Clinical Guideline Centre. Hypertension. The clinical management of primary hypertension in adults. Clinical guideline methods, evidence, and recommendations. 2011. Page 31. Table 9.Google Scholar
  7. 7.
    U.S. Preventive Services Task Force. Procedure manual. 2008. AHRQ publication number 08-05118-EF.Google Scholar
  8. 8.
    Harbour R, Miller J. A new system for grading recommendations in evidence based guidelines. BMJ. 2001;323(7308):334–6.CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Jüni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999;282(11):1054–60.CrossRefPubMedGoogle Scholar
  10. 10.
    Berlin JA, Golub RM. Meta-analysis as evidence: building a better pyramid. JAMA. 2014;312(6):603–5.CrossRefPubMedGoogle Scholar
  11. 11.
    Dechartres A, Altman DG, Trinquart L, Boutron I, Ravaud P. Association between analytic strategy and estimates of treatment outcomes in meta-analyses. JAMA. 2014;312(6):623–30.CrossRefPubMedGoogle Scholar
  12. 12.
    Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Comparison of two methods to detect publication bias in meta-analysis. JAMA. 2006;295(6):676–80.CrossRefPubMedGoogle Scholar
  13. 13.
    Patsopoulos NA, Analatos AA, Loannidis JP. Relative citation impact of various study designs in the health sciences. JAMA. 2005;18:2362–6.CrossRefGoogle Scholar
  14. 14.
    Zanchetti A. Nat Rev Cardiol. 2011;8(5):249–51.CrossRefPubMedGoogle Scholar
  15. 15.
    Riley RD, Lambert PC, Abo-Zaid G. Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ. 2010;5:340–c221. doi: 10.1136/bmj.c2.Google Scholar
  16. 16.
    Roush GC, Amante B, Singh T, Ayele H, Araoye M, Yang D, et al. Quality of meta-analyses for randomized trials in the field of hypertension: a systematic review. J Hypertens. 2016;34(12):2305–17.CrossRefPubMedGoogle Scholar
  17. 17.
    Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias). J Clin Epidemiol. 2011;64(4):407–15.CrossRefPubMedGoogle Scholar
  18. 18.
    Als-Nielsen B, Gluud LL, Gluud C. Methdologlogical quality and treatment effects in randomized trials: a review of six empirical studies. 12th Cochrane Colloquium, Ottawa, Canada, 2004.Google Scholar
  19. 19.
    Pildal J, Hróbjartsson A, Jørgensen KJ, Hilden J, Altman DG, Gøtzsche PC. Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007;36(4):847–57.CrossRefPubMedGoogle Scholar
  20. 20.
    Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008;336(7644):601–5.CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Higgins JP, Altman DG. Chapter 8: Assessing risk of bias in included studies. In: JPT H, Green S, editors. Cochrane handbook for systematic reviews of interventions, version 5.1.0. Chichester, UK: John Wiley & Sons Ltd; 2008.CrossRefGoogle Scholar
  22. 22.
    Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001;135(11):982–9.CrossRefPubMedGoogle Scholar
  23. 23.
    Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17(1):1–12.CrossRefPubMedGoogle Scholar
  24. 24.
    Borenstein M, Hedges LV, JPT H, Rothstein HR. Introduction to meta-analysis: John Wiley & Sons; 2009.Google Scholar
  25. 25.
    Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al., GRADE Working Group. GRADE guidelines: 7. Rating the quality of evidence—inconsistency. J Clin Epidemiol. 2011;64(12):1294–302.Google Scholar
  26. 26.
    Deeks JJ, Higgins JPT, Altman DG Chapter 9: Analyzing data and undertaking met-analyses. In: Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions, version 5.1.0. John Wiley Sons Ltd.: Chichester, UK; 2008.Google Scholar
  27. 27.
    Cornell JE, Mulrow CD, Localio R, Stack CB, Meibohm AR, Guallar E, et al. Random-effects meta-analysis of inconsistent effects: a time for change. Ann Intern Med. 2014;160(4):267–70.CrossRefPubMedGoogle Scholar
  28. 28.
    IntHout J, Ioannidis JP, Borm GF. The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Med Res Methodol. 2014;14:25. doi: 10.1186/1471-2288-14-25.CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, et al. GRADE guidelines: 5. Rating the quality of evidence—publication bias. J Clin Epidemiol. 2011;64(12):1277–82.CrossRefPubMedGoogle Scholar
  30. 30.
    Green S, Higgins JPT, Alderson P, Clarke M, Mulrow CD, Oxman AD. Chapter 1: Introduction. In: JPT H, Green S, editors. Cochrane handbook for systematic reviews of interventions. Chichester, UK: John Wiley & Sons Ltd.; 2008.Google Scholar
  31. 31.
    Lefebre C, Manheimer E, Glanville J. Chapter 6: Searching for studies. In: JPT H, Green S, editors. Cochrane handbook for systematic reviews of interventions, version 5.1.0. Chichester, UK: John Wiley & Sons Ltd.; 2008.Google Scholar
  32. 32.
    Sterne JAC, Egger M, Moher D. Chapter 10: addressing reporting biases. In: JPT H, Green S, editors. Cochrane handbook for systematic reviews of interventions, version 5.1.0. Chichester, UK: John Wiley & Sons Ltd.; 2008.Google Scholar
  33. 33.
    Terrin N, Schmid CH, Lau J. In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. J Clin Epidemiol. 2005;58:894–901.CrossRefPubMedGoogle Scholar
  34. 34.
    Leimu R, Koricheva J. Cumulative meta-analysis: a new tool for detection of temporal trends and publication bias in ecology. Proc Biol Sci. 2004;271(1551):1961–6.CrossRefPubMedPubMedCentralGoogle Scholar
  35. 35.
    Rothstein HR, Sutton AJ, Borenstein M. Publication bias in meta-analysis: prevention, assessment and adjustments: John Wiley & Sons; 2004. p. 247–54.Google Scholar
  36. 36.
    Littell JH. Systematic reviews and meta-analysis: Oxford University Press; 2008. p. 116.Google Scholar
  37. 37.
    McDaniel MA. Cumulative meta-analysis as a publication bias method. Presentation at the 24th annual meeting of the Society for Industrial and Organizational Psychology, New Orleans. 2009.Google Scholar
  38. 38.
    Schmidt FL, Hunter JE. Methods of meta-analysis: correcting eror and bias in research findings: Sage Publications; 2015. p. 540.Google Scholar
  39. 39.
    Borenstein M, Higgins JP, Hedges LV, Rothstein HR. Basics of meta-analysis: I 2 is not an absolute measure of heterogeneity. Res Synth Methods. 2017;8(1):5–18.CrossRefPubMedGoogle Scholar
  40. 40.
    DerSimonian R, Laird N. Meta-analysis in clinical trials revisited. Contemp Clin Trials. 2015;45:139–45.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • George C Roush
    • 1
  • Fiorella Perez
    • 1
  • Ramy Abdelfattah
    • 1
  • Andrew Prindle
    • 1
  • Elie Jean
    • 1
  • Tanveer Singh
    • 2
  • John B. Kostis
    • 3
  • William J. Kostis
    • 3
  • William J. Elliott
    • 4
  • Jesse A. Berlin
    • 5
  1. 1.Department of Medicine, NYC Health and Hospitals/WoodhullNYU School of MedicineNew YorkUSA
  2. 2.St. Vincent’s Medical Center and Quinnipiac School of MedicineBridgeportUSA
  3. 3.Cardiovascular Institute, RutgersRobert Wood Johnson Medical SchoolNew BrunswickUSA
  4. 4.Pacific Northwest University of Health SciencesYakimaUSA
  5. 5.Johnson & JohnsonTitusvilleUSA

Personalised recommendations