Skip to main content

Evidence-Based Decision-Making 2: Systematic Reviews and Meta-analysis

  • Protocol
  • First Online:
Clinical Epidemiology

Part of the book series: Methods in Molecular Biology ((MIMB,volume 1281))

Abstract

The number of studies published in the biomedical literature has dramatically increased over the last few decades. This massive proliferation of literature makes clinical medicine increasingly complex, and information from multiple studies is often needed to inform a particular clinical decision. However, available studies often vary in their design, methodological quality, populations studied and may define the research question of interest quite differently, which can make it challenging to synthesize their conclusions. In addition, since even highly cited trials may be challenged over time, clinical decision-making requires ongoing reconciliation of studies which provide different answers to the same question. Because it is often impractical for readers to track down and review all the primary studies, systematic reviews and meta-analyses are an important source of evidence on the diagnosis, prognosis, and treatment of any given disease. This chapter summarizes methods for conducting and reading systematic reviews and meta-analyses, as well as describing potential advantages and disadvantages of these publications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Protocol
USD 49.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Anonymous (2014) https://www.nlm.nih.gov/pubs/factsheets/nlm.html. Accessed 20 April 2014

  2. Umscheid CA (2013) A primer on performing systematic reviews and meta-analyses. Clin Infect Dis 57(5):725–734

    Article  PubMed  Google Scholar 

  3. Ioannidis JP (2005) Contradicted and initially stronger effects in highly cited clinical research. JAMA 294(2):218–228

    Article  CAS  PubMed  Google Scholar 

  4. Garg AX, Iansavichus AV, Kastner M, Walters LA, Wilczynski N, McKibbon KA, Yang RC, Rehman F, Haynes RB (2006) Lost in publication: half of all renal practice evidence is published in non-renal journals. Kidney Int 70(11):1995–2005

    CAS  PubMed  Google Scholar 

  5. Barrett BJ, Parfrey PS (2006) Clinical practice. Preventing nephropathy induced by contrast medium. N Engl J Med 354(4):379–386

    Article  CAS  PubMed  Google Scholar 

  6. Halloran PF (2004) Immunosuppressive drugs for kidney transplantation. N Engl J Med 351(26):2715–2729

    Article  CAS  PubMed  Google Scholar 

  7. Schrier RW, Wang W (2004) Acute renal failure and sepsis. N Engl J Med 351(2):159–169

    Article  CAS  PubMed  Google Scholar 

  8. Tonelli M, Lloyd A, Clement F, Conly J, Husereau D, Hemmelgarn B, Klarenbach S, McAlister FA, Wiebe N, Manns B (2011) Efficacy of statins for primary prevention in people at low cardiovascular risk: a meta-analysis. CMAJ 183(16):E1189–E1202

    Article  PubMed Central  PubMed  Google Scholar 

  9. Pannu N, Klarenbach S, Wiebe N, Manns B, Tonelli M (2008) Renal replacement therapy in patients with acute renal failure: a systematic review. JAMA 299(7):793–805

    Article  CAS  PubMed  Google Scholar 

  10. Oxman AD, Cook DJ, Guyatt GH (1994) Users’ guides to the medical literature. VI. How to use an overview. Evidence-based medicine working group. JAMA 272(17):1367–1371

    Article  CAS  PubMed  Google Scholar 

  11. Cook DJ, Greengold NL, Ellrodt AG, Weingarten SR (1997) The relation between systematic reviews and practice guidelines. Ann Intern Med 127(3):210–216

    Article  CAS  PubMed  Google Scholar 

  12. Bafeta A, Trinquart L, Seror R, Ravaud P (2014) Reporting of results from network meta-analyses: methodological systematic review. BMJ 348:g1741

    Article  PubMed Central  PubMed  Google Scholar 

  13. Fowkes FG, Rudan D, Rudan I, Aboyans V, Denenberg JO, McDermott MM, Norman PE, Sampson UK, Williams LJ, Mensah GA, Criqui MH (2013) Comparison of global estimates of prevalence and risk factors for peripheral artery disease in 2000 and 2010: a systematic review and analysis. Lancet 382(9901):1329–1340

    Article  PubMed  Google Scholar 

  14. Tusting LS, Willey B, Lucas H, Thompson J, Kafy HT, Smith R, Lindsay SW (2013) Socioeconomic development as an intervention against malaria: a systematic review and meta-analysis. Lancet 382(9896):963–972

    Article  PubMed  Google Scholar 

  15. Lyman GH, Kuderer NM (2005) The strengths and limitations of meta-analyses based on aggregate data. BMC Med Res Methodol 5:14

    Article  PubMed Central  PubMed  Google Scholar 

  16. Simmonds MC, Higgins JP, Stewart LA, Tierney JF, Clarke MJ, Thompson SG (2005) Meta-analysis of individual patient data from randomized trials: a review of methods used in practice. Clin Trials 2(3):209–217

    Article  PubMed  Google Scholar 

  17. Schwingshackl L, Hoffmann G (2014) Dietary fatty acids in the secondary prevention of coronary heart disease: a systematic review, meta-analysis and meta-regression. BMJ Open 4(4):e004487

    Article  PubMed Central  PubMed  Google Scholar 

  18. Sorita A, Ahmed A, Starr SR, Thompson KM, Reed DA, Prokop L, Shah ND, Murad MH, Ting HH (2014) Off-hour presentation and outcomes in patients with acute myocardial infarction: systematic review and meta-analysis. BMJ 348:f7393

    Article  PubMed Central  PubMed  Google Scholar 

  19. Threapleton DE, Greenwood DC, Evans CE, Cleghorn CL, Nykjaer C, Woodhead C, Cade JE, Gale CP, Burley VJ (2013) Dietary fibre intake and risk of cardiovascular disease: systematic review and meta-analysis. BMJ 347:f6879

    Article  PubMed Central  PubMed  Google Scholar 

  20. Kalil AC, Klompas M, Haynatzki G, Rupp ME (2013) Treatment of hospital-acquired pneumonia with linezolid or vancomycin: a systematic review and meta-analysis. BMJ Open 3(10):e003912

    Article  PubMed Central  PubMed  Google Scholar 

  21. Gagliardino JJ, Arrechea V, Assad D, Gagliardino GG, Gonzalez L, Lucero S, Rizzuti L, Zufriategui Z, Clark C Jr (2013) Type 2 diabetes patients educated by other patients perform at least as well as patients trained by professionals. Diabetes Metab Res Rev 29(2):152–160

    Article  PubMed  Google Scholar 

  22. Murthy L, Shepperd S, Clarke MJ, Garner SE, Lavis JN, Perrier L, Roberts NW, Straus SE (2012) Interventions to improve the use of systematic reviews in decision-making by health system managers, policy makers and clinicians. Cochrane Database Syst Rev 9:CD009401

    PubMed  Google Scholar 

  23. Durand MA, Carpenter L, Dolan H, Bravo P, Mann M, Bunn F, Elwyn G (2014) Do interventions designed to support shared decision-making reduce health inequalities? A systematic review and meta-analysis. PLoS One 9(4):e94670

    Article  PubMed Central  PubMed  Google Scholar 

  24. Moher D, Schulz KF, Altman DG (2001) The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials. BMC Med Res Methodol 1:2

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  25. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Moher D, Rennie D, de Vet HC, Lijmer JG (2003) The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med 138(1):W1–W12

    Article  PubMed  Google Scholar 

  26. Drummond MF, Jefferson TO (1996) Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ economic evaluation working party. BMJ 313(7052):275–283

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  27. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP (2007) The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 370(9596):1453–1457

    Article  Google Scholar 

  28. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF (1999) Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of reporting of meta-analyses. Lancet 354(9193):1896–1900

    Article  CAS  PubMed  Google Scholar 

  29. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB (2000) Meta-analysis of observational studies in epidemiology: a proposal for reporting meta-analysis. Of observational studies in epidemiology (MOOSE) group. JAMA 283(15):2008–2012

    Article  CAS  PubMed  Google Scholar 

  30. Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 151(4):264–269, W64

    Article  PubMed  Google Scholar 

  31. Sambunjak D, Franic M (2012) Steps in the undertaking of a systematic review in orthopaedic surgery. Int Orthop 36(3):477–484

    Article  PubMed Central  PubMed  Google Scholar 

  32. Bello AK, Wiebe N, Garg AX, Tonelli M (2011) Basics of systematic reviews and meta-analyses for the nephrologist. Nephron Clin Pract 119(1):c50–c60, discussion c1

    Article  PubMed  Google Scholar 

  33. Riley RD, Lambert PC, Abo-Zaid G (2010) Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ 340:c221

    Article  PubMed  Google Scholar 

  34. Wardlaw JM, Warlow CP, Counsell C (1997) Systematic review of evidence on thrombolytic therapy for acute ischaemic stroke. Lancet 350(9078):607–614

    Article  CAS  PubMed  Google Scholar 

  35. Yusuf S (1997) Meta-analysis of randomized trials: looking back and looking ahead. Control Clin Trials 18(6):594–601, discussion 61-6

    Article  CAS  PubMed  Google Scholar 

  36. Counsell C (1997) Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med 127(5):380–387

    Article  CAS  PubMed  Google Scholar 

  37. Vickers A, Goyal N, Harland R, Rees R (1998) Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials 19(2):159–166

    Article  CAS  PubMed  Google Scholar 

  38. Moher D, Pham B, Klassen TP, Schulz KF, Berlin JA, Jadad AR, Liberati A (2000) What contributions do languages other than English make on the results of meta-analyses? J Clin Epidemiol 53(9):964–972

    Article  CAS  PubMed  Google Scholar 

  39. Egger M, Davey Smith G, Schneider M, Minder C (1997) Bias in meta-analysis detected by a simple, graphical test. BMJ 315(7109):629–634

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  40. Gregoire G, Derderian F, Le Lorier J (1995) Selecting the language of the publications included in a meta-analysis: is there a Tower of Babel bias? J Clin Epidemiol 48(1):159–163

    Article  CAS  PubMed  Google Scholar 

  41. Strippoli GF, Craig MC, Schena FP, Craig JC (2006) Role of blood pressure targets and specific antihypertensive agents used to prevent diabetic nephropathy and delay its progression. J Am Soc Nephrol 17(4 Suppl 2):S153–S155

    Article  PubMed  Google Scholar 

  42. Egger M, Smith GD (1998) Bias in location and selection of studies. BMJ 316(7124):61–66

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  43. Subramanian S, Venkataraman R, Kellum JA (2002) Influence of dialysis membranes on outcomes in acute renal failure: a meta-analysis. Kidney Int 62(5):1819–1823

    Article  CAS  PubMed  Google Scholar 

  44. Jaber BL, Lau J, Schmid CH, Karsou SA, Levey AS, Pereira BJ (2002) Effect of biocompatibility of hemodialysis membranes on mortality in acute renal failure: a meta-analysis. Clin Nephrol 57(4):274–282

    Article  CAS  PubMed  Google Scholar 

  45. Teehan GS, Liangos O, Lau J, Levey AS, Pereira BJ, Jaber BL (2003) Dialysis membrane and modality in acute renal failure: understanding discordant meta-analyses. Semin Dial 16(5):356–360

    Article  PubMed  Google Scholar 

  46. Dickersin K, Scherer R, Lefebvre C (1994) Identifying relevant studies for systematic reviews. BMJ 309(6964):1286–1291

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  47. Steinbrook R (2006) Searching for the right search–reaching the medical literature. N Engl J Med 354(1):4–7

    Article  CAS  PubMed  Google Scholar 

  48. Wilczynski NL, Haynes RB (2002) Robustness of empirical search strategies for clinical content in MEDLINE. Proc AMIA Symp 904–908

    Google Scholar 

  49. Wilczynski NL, Walker CJ, McKibbon KA, Haynes RB (1995) Reasons for the loss of sensitivity and specificity of methodologic MeSH terms and textwords in MEDLINE. Proc Annu Symp Comput Appl Med Care 436–440.

    Google Scholar 

  50. Edwards P, Clarke M, DiGuiseppi C, Pratap S, Roberts I, Wentz R (2002) Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Stat Med 21(11):1635–1640

    Article  PubMed  Google Scholar 

  51. Pogue J, Yusuf S (1998) Overcoming the limitations of current meta-analysis of randomised controlled trials. Lancet 351(9095):47–52

    Article  CAS  PubMed  Google Scholar 

  52. Davidson RA (1986) Source of funding and outcome of clinical trials. J Gen Intern Med 1(3):155–158

    Article  CAS  PubMed  Google Scholar 

  53. Rochon PA, Gurwitz JH, Simms RW, Fortin PR, Felson DT, Minaker KL, Chalmers TC (1994) A study of manufacturer-supported trials of nonsteroidal anti-inflammatory drugs in the treatment of arthritis. Arch Intern Med 154(2):157–163

    Article  CAS  PubMed  Google Scholar 

  54. Biondi-Zoccai GG, Lotrionte M, Abbate A, Testa L, Remigi E, Burzotta F, Valgimigli M, Romagnoli E, Crea F, Agostoni P (2006) Compliance with QUOROM and quality of reporting of overlapping meta-analyses on the role of acetylcysteine in the prevention of contrast associated nephropathy: case study. BMJ 332(7535):202–209

    Article  PubMed Central  PubMed  Google Scholar 

  55. Berlin JA (1997) Does blinding of readers affect the results of meta-analyses? University of Pennsylvania Meta-analysis Blinding Study Group. Lancet 350(9072):185–186

    Article  CAS  PubMed  Google Scholar 

  56. Jadad AR, McQuay HJ (1996) Meta-analyses to evaluate analgesic interventions: a systematic qualitative review of their methodology. J Clin Epidemiol 49(2):235–243

    Article  CAS  PubMed  Google Scholar 

  57. Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, Lau J (2002) Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA 287(22):2973–2982

    Article  PubMed  Google Scholar 

  58. Balk EM, Lau J, Bonis PA (2005) Reading and critically appraising systematic reviews and meta-analyses: a short primer with a focus on hepatology. J Hepatol 43(4):729–736

    Article  PubMed  Google Scholar 

  59. Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A, Pham B, Klassen TP (1999) Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses. Health Technol Assess 3(12):i–iv, 1–98

    CAS  PubMed  Google Scholar 

  60. Verhagen AP, de Vet HC, de Bie RA, Boers M, van den Brandt PA (2001) The art of quality assessment of RCTs included in systematic reviews. J Clin Epidemiol 54(7):651–654

    Article  CAS  PubMed  Google Scholar 

  61. Juni P, Altman DG, Egger M (2001) Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ 323(7303):42–46

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  62. Devereaux PJ, Choi PT, El-Dika S, Bhandari M, Montori VM, Schunemann HJ, Garg AX, Busse JW, Heels-Ansdell D, Ghali WA, Manns BJ, Guyatt GH (2004) An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. J Clin Epidemiol 57(12):1232–1236

    Article  CAS  PubMed  Google Scholar 

  63. Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273(5):408–412

    Article  CAS  PubMed  Google Scholar 

  64. Laupacis A, Wells G, Richardson WS, Tugwell P (1994) Users’ guides to the medical literature. V. How to use an article about prognosis. Evidence-based medicine working group. JAMA 272(3):234–237

    Article  CAS  PubMed  Google Scholar 

  65. Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, Moher M, Moher D (1998) Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA 280(3):278–280

    Article  CAS  PubMed  Google Scholar 

  66. Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP (2006) Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol 59(7):697–703

    Article  PubMed  Google Scholar 

  67. Horton J, Vandermeer B, Hartling L, Tjosvold L, Klassen TP, Buscemi N (2010) Systematic review data extraction: cross-sectional study showed that experience did not increase accuracy. J Clin Epidemiol 63(3):289–298

    Article  PubMed  Google Scholar 

  68. Deeks JJ (2002) Issues in the selection of a summary statistic for meta-analysis of clinical trials with binary outcomes. Stat Med 21(11):1575–1600

    Article  PubMed  Google Scholar 

  69. DerSimonian R, Laird N (1986) Meta-analysis in clinical trials. Control Clin Trials 7(3):177–188

    Article  CAS  PubMed  Google Scholar 

  70. Hardy RJ, Thompson SG (1998) Detecting and describing heterogeneity in meta-analysis. Stat Med 17(8):841–856

    Article  CAS  PubMed  Google Scholar 

  71. Higgins JP, Thompson SG (2002) Quantifying heterogeneity in a meta-analysis. Stat Med 21(11):1539–1558

    Article  PubMed  Google Scholar 

  72. Haynes RB, Devereaux PJ, Guyatt GH (2002) Physicians’ and patients’ choices in evidence based practice. BMJ 324(7350):1350

    Article  PubMed Central  PubMed  Google Scholar 

  73. Sterne JA, Egger M, Smith GD (2001) Systematic reviews in health care: investigating and dealing with publication and other biases in meta-analysis. BMJ 323(7304):101–105

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  74. Simes RJ (1987) Confronting publication bias: a cohort design for meta-analysis. Stat Med 6(1):11–29

    Article  CAS  PubMed  Google Scholar 

  75. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR (1991) Publication bias in clinical research. Lancet 337(8746):867–872

    Article  CAS  PubMed  Google Scholar 

  76. Dickersin K, Min YI, Meinert CL (1992) Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA 267(3):374–378

    Article  CAS  PubMed  Google Scholar 

  77. Giatras I, Lau J, Levey AS (1997) Effect of angiotensin-converting enzyme inhibitors on the progression of nondiabetic renal disease: a meta-analysis of randomized trials. Angiotensin-converting-enzyme inhibition and progressive renal disease study group. Ann Intern Med 127(5):337–345

    Article  CAS  PubMed  Google Scholar 

  78. Guyatt G, Gutterman D, Baumann MH, Addrizzo-Harris D, Hylek EM, Phillips B, Raskob G, Lewis SZ, Schunemann H (2006) Grading strength of recommendations and quality of evidence in clinical guidelines: report from an american college of chest physicians task force. Chest 129(1):174–181

    Article  PubMed  Google Scholar 

  79. Anello C, Fleiss JL (1995) Exploratory or analytic meta-analysis: should we distinguish between them? J Clin Epidemiol 48(1):109–116, discussion 17-8

    Article  CAS  PubMed  Google Scholar 

  80. Boudville N, Prasad GV, Knoll G, Muirhead N, Thiessen-Philbrook H, Yang RC, Rosas-Arellano MP, Housawi A, Garg AX (2006) Meta-analysis: risk for hypertension in living kidney donors. Ann Intern Med 145(3):185–196

    Article  PubMed  Google Scholar 

  81. Hackam DG, Redelmeier DA (2006) Translation of research evidence from animals to humans. JAMA 296(14):1731–1732

    Article  CAS  PubMed  Google Scholar 

  82. LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian F (1997) Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med 337(8):536–542

    Article  CAS  PubMed  Google Scholar 

  83. Palma S, Delgado-Rodriguez M (2005) Assessment of publication bias in meta-analyses of cardiovascular diseases. J Epidemiol Community Health 59(10):864–869

    Article  PubMed Central  PubMed  Google Scholar 

  84. Lau J, Ioannidis JP, Schmid CH (1998) Summing up evidence: one answer is not always enough. Lancet 351(9096):123–127

    Article  CAS  PubMed  Google Scholar 

  85. Thompson SG (1994) Why sources of heterogeneity in meta-analysis should be investigated. BMJ 309(6965):1351–1355

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  86. Berlin JA (1995) Invited commentary: benefits of heterogeneity in meta-analysis of data from epidemiologic studies. Am J Epidemiol 142(4):383–387

    CAS  PubMed  Google Scholar 

  87. Davey Smith G, Egger M, Phillips AN (1997) Meta-analysis. Beyond the grand mean? BMJ 315(7122):1610–1614

    Article  PubMed Central  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Marcello Tonelli .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media New York

About this protocol

Cite this protocol

Bello, A., Wiebe, N., Garg, A., Tonelli, M. (2015). Evidence-Based Decision-Making 2: Systematic Reviews and Meta-analysis. In: Parfrey, P., Barrett, B. (eds) Clinical Epidemiology. Methods in Molecular Biology, vol 1281. Humana Press, New York, NY. https://doi.org/10.1007/978-1-4939-2428-8_24

Download citation

  • DOI: https://doi.org/10.1007/978-1-4939-2428-8_24

  • Published:

  • Publisher Name: Humana Press, New York, NY

  • Print ISBN: 978-1-4939-2427-1

  • Online ISBN: 978-1-4939-2428-8

  • eBook Packages: Springer Protocols

Publish with us

Policies and ethics