Skip to main content

Quality of Meta-Analyses for Randomized Trials in the Field of Hypertension: an Updated and Improved Systematic Review

Current Hypertension Reports volume 19, Article number: 71 (2017) Cite this article

Abstract

Publications of hypertension-related meta-analyses (MAs) have increased exponentially in the past 25 years and now average 8/month. Theoretically, this is facilitating evidence-based management of patients. However, some practitioners and authors of guidelines have questioned the quality of published MAs. By extending a prior review, we have assessed the quality of 212 hypertension-related meta-analyses over 5 years based on systematically searching three computerized libraries. Seventeen criteria grouped into four domains of quality yielded the following results: (1) Assessment of trial quality was accomplished in 89% of MAs, and 38% analyzed trials in subgroups of trial quality where appropriate. (2) All three measures of heterogeneity (I 2, tau, and P for heterogeneity) were reported in 36%, reflecting the failure to report tau, the standard deviation of the main effect. (3) Publication bias was assessed in 75%, and 43% of MAs used a statistical test for publication bias. (4) Regarding transparency, 9 to 31% of MAs reported problems in the previous three domains in the article’s abstract. Journal impact factor reporting the MAs declined significantly over 5 years. The percent with criteria of quality in a MA was modestly correlated with journal impact factor (R 2 = 0.05, P = 0.001). False-positive results from inappropriate application of the DerSimonian-Laird model affected 25% of articles, which reported these false positives in the article’s abstract in 72%. No more than 25% of MAs had 67% or more of the criteria of quality. In conclusion, skepticism of hypertension-related MAs is justified, but their quality can be readily corrected.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. Greenhalgh T. How to read a paper: the basics of evidence based medicine. BMJ Books, 2001, page 54.

  2. Buyse M, Piedbois P, Piedbois Y, Carlson RW. Meta-analysis: methods, strengths, and weaknesses. Oncology (Williston Park). 2000;14(3):437–43.

    CAS  Google Scholar 

  3. Chung KC, Burns PB, Kim HM. A practical guide to meta-analysis. J Hand Surg Am. 2006;31(10):1671–8.

    Article  PubMed  Google Scholar 

  4. Murad MH, Montori VM, Ioannidis JP, Jaeschke R, Devereaux PJ, Prasad K, et al. How to read a systematic review and meta-analysis and apply the results to patient care: users’ guides to the medical literature. JAMA. 2014;312(2):171–9.

    CAS  Article  PubMed  Google Scholar 

  5. Tabas GH, Ende J, Aronowitz PB, Conigliaro RL, Granieri R, Green EH et al. American College of Physicians. General Internal Medicine. MKSAP 16. 2012. Page 3.

  6. National Clinical Guideline Centre. Hypertension. The clinical management of primary hypertension in adults. Clinical guideline methods, evidence, and recommendations. 2011. Page 31. Table 9.

  7. U.S. Preventive Services Task Force. Procedure manual. 2008. AHRQ publication number 08-05118-EF.

  8. Harbour R, Miller J. A new system for grading recommendations in evidence based guidelines. BMJ. 2001;323(7308):334–6.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  9. Jüni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999;282(11):1054–60.

    Article  PubMed  Google Scholar 

  10. Berlin JA, Golub RM. Meta-analysis as evidence: building a better pyramid. JAMA. 2014;312(6):603–5.

    CAS  Article  PubMed  Google Scholar 

  11. Dechartres A, Altman DG, Trinquart L, Boutron I, Ravaud P. Association between analytic strategy and estimates of treatment outcomes in meta-analyses. JAMA. 2014;312(6):623–30.

    CAS  Article  PubMed  Google Scholar 

  12. Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Comparison of two methods to detect publication bias in meta-analysis. JAMA. 2006;295(6):676–80.

    CAS  Article  PubMed  Google Scholar 

  13. Patsopoulos NA, Analatos AA, Loannidis JP. Relative citation impact of various study designs in the health sciences. JAMA. 2005;18:2362–6.

    Article  Google Scholar 

  14. Zanchetti A. Nat Rev Cardiol. 2011;8(5):249–51.

    CAS  Article  PubMed  Google Scholar 

  15. Riley RD, Lambert PC, Abo-Zaid G. Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ. 2010;5:340–c221. doi:10.1136/bmj.c2.

    Google Scholar 

  16. Roush GC, Amante B, Singh T, Ayele H, Araoye M, Yang D, et al. Quality of meta-analyses for randomized trials in the field of hypertension: a systematic review. J Hypertens. 2016;34(12):2305–17.

    CAS  Article  PubMed  Google Scholar 

  17. Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias). J Clin Epidemiol. 2011;64(4):407–15.

    Article  PubMed  Google Scholar 

  18. Als-Nielsen B, Gluud LL, Gluud C. Methdologlogical quality and treatment effects in randomized trials: a review of six empirical studies. 12th Cochrane Colloquium, Ottawa, Canada, 2004.

  19. Pildal J, Hróbjartsson A, Jørgensen KJ, Hilden J, Altman DG, Gøtzsche PC. Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007;36(4):847–57.

    CAS  Article  PubMed  Google Scholar 

  20. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008;336(7644):601–5.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Higgins JP, Altman DG. Chapter 8: Assessing risk of bias in included studies. In: JPT H, Green S, editors. Cochrane handbook for systematic reviews of interventions, version 5.1.0. Chichester, UK: John Wiley & Sons Ltd; 2008.

    Chapter  Google Scholar 

  22. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001;135(11):982–9.

    CAS  Article  PubMed  Google Scholar 

  23. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17(1):1–12.

    CAS  Article  PubMed  Google Scholar 

  24. Borenstein M, Hedges LV, JPT H, Rothstein HR. Introduction to meta-analysis: John Wiley & Sons; 2009.

  25. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al., GRADE Working Group. GRADE guidelines: 7. Rating the quality of evidence—inconsistency. J Clin Epidemiol. 2011;64(12):1294–302.

  26. Deeks JJ, Higgins JPT, Altman DG Chapter 9: Analyzing data and undertaking met-analyses. In: Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions, version 5.1.0. John Wiley Sons Ltd.: Chichester, UK; 2008.

    Google Scholar 

  27. Cornell JE, Mulrow CD, Localio R, Stack CB, Meibohm AR, Guallar E, et al. Random-effects meta-analysis of inconsistent effects: a time for change. Ann Intern Med. 2014;160(4):267–70.

    Article  PubMed  Google Scholar 

  28. IntHout J, Ioannidis JP, Borm GF. The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Med Res Methodol. 2014;14:25. doi:10.1186/1471-2288-14-25.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, et al. GRADE guidelines: 5. Rating the quality of evidence—publication bias. J Clin Epidemiol. 2011;64(12):1277–82.

    Article  PubMed  Google Scholar 

  30. Green S, Higgins JPT, Alderson P, Clarke M, Mulrow CD, Oxman AD. Chapter 1: Introduction. In: JPT H, Green S, editors. Cochrane handbook for systematic reviews of interventions. Chichester, UK: John Wiley & Sons Ltd.; 2008.

    Google Scholar 

  31. Lefebre C, Manheimer E, Glanville J. Chapter 6: Searching for studies. In: JPT H, Green S, editors. Cochrane handbook for systematic reviews of interventions, version 5.1.0. Chichester, UK: John Wiley & Sons Ltd.; 2008.

    Google Scholar 

  32. Sterne JAC, Egger M, Moher D. Chapter 10: addressing reporting biases. In: JPT H, Green S, editors. Cochrane handbook for systematic reviews of interventions, version 5.1.0. Chichester, UK: John Wiley & Sons Ltd.; 2008.

    Google Scholar 

  33. Terrin N, Schmid CH, Lau J. In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. J Clin Epidemiol. 2005;58:894–901.

    Article  PubMed  Google Scholar 

  34. Leimu R, Koricheva J. Cumulative meta-analysis: a new tool for detection of temporal trends and publication bias in ecology. Proc Biol Sci. 2004;271(1551):1961–6.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Rothstein HR, Sutton AJ, Borenstein M. Publication bias in meta-analysis: prevention, assessment and adjustments: John Wiley & Sons; 2004. p. 247–54.

  36. Littell JH. Systematic reviews and meta-analysis: Oxford University Press; 2008. p. 116.

  37. McDaniel MA. Cumulative meta-analysis as a publication bias method. Presentation at the 24th annual meeting of the Society for Industrial and Organizational Psychology, New Orleans. 2009.

  38. Schmidt FL, Hunter JE. Methods of meta-analysis: correcting eror and bias in research findings: Sage Publications; 2015. p. 540.

  39. Borenstein M, Higgins JP, Hedges LV, Rothstein HR. Basics of meta-analysis: I 2 is not an absolute measure of heterogeneity. Res Synth Methods. 2017;8(1):5–18.

    Article  PubMed  Google Scholar 

  40. DerSimonian R, Laird N. Meta-analysis in clinical trials revisited. Contemp Clin Trials. 2015;45:139–45.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Medicine, NYC Health and Hospitals/Woodhull, NYU School of Medicine, New York, NY, USA

    George C Roush, Fiorella Perez, Ramy Abdelfattah, Andrew Prindle & Elie Jean

  2. St. Vincent’s Medical Center and Quinnipiac School of Medicine, Bridgeport, CT, USA

    Tanveer Singh

  3. Cardiovascular Institute, Rutgers, Robert Wood Johnson Medical School, New Brunswick, NJ, USA

    John B. Kostis & William J. Kostis

  4. Pacific Northwest University of Health Sciences, Yakima, USA

    William J. Elliott

  5. Johnson & Johnson, Titusville, NJ, USA

    Jesse A. Berlin

Authors
  1. George C Roush
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fiorella Perez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramy Abdelfattah
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrew Prindle
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elie Jean
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tanveer Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. John B. Kostis
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. William J. Kostis
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. William J. Elliott
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Jesse A. Berlin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George C Roush.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflicts of interest.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

This article is part of the Topical Collection on Guidelines/Clinical Trials/Meta-Analysis

Electronic Supplementary Material

ESM 1

(PDF 6073 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Roush, G.C., Perez, F., Abdelfattah, R. et al. Quality of Meta-Analyses for Randomized Trials in the Field of Hypertension: an Updated and Improved Systematic Review. Curr Hypertens Rep 19, 71 (2017). https://doi.org/10.1007/s11906-017-0765-7

Download citation

  • Published:

  • DOI: https://doi.org/10.1007/s11906-017-0765-7

Keywords

  • Meta-analysis as topic
  • Review (publication type)
  • Hypertension
  • Systematic review
  • Randomized controlled trials (publication type)
  • Blood pressure