Abstract
This chapter describes umbrella reviews, overviews of reviews, and meta-epidemiologic studies focusing on their definitions, purposes, and classifications where appropriate and then elaborating on their similarities and differences. We may consider umbrella reviews as reviews integrating several types of study designs but typically randomized controlled trials and systematic reviews of such studies in a unifying fashion in order to address a content issue (e.g., whether or not a given drug is superior to another). Overviews of reviews are reviews of systematic reviews and meta-analyses which can focus on content or methodological issues. Finally, meta-epidemiologic studies focus on potentially different types of study designs but most typically on randomized trials and systematic reviews, usually across different content domains (e.g., topics or conditions), and mainly aim at addressing methodological issues.
Is there a magic method of determining when a meta-analysis is likely to be misleading?
C. David Naylor [1]
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Naylor CD. Meta-analysis and the meta-epidemiology of clinical research. Meta-analysis is an important contribution to research and practice but it’s not a panacea. BMJ. 1997;315(7109):617–9. doi:10.1136/bmj.315.7109.617.
Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. Chichester: Wiley; 2008. doi:10.1002/9780470712184.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. Chichester: Wiley; 2009. doi:10.1002/9780470743386.
Savović J, Harris RJ, Wood L, Beynon R, Altman D, Als-Nielsen B, Balk EM, Deeks J, Gluud LL, Gluud C, Ioannidis JPA, Jűni P, Moher D, Pildal J, Schulz KF, Sterne JAC. Development of a combined database for meta-epidemiological research. Res Synth Methods. 2010;1(3–4):212–25. doi:10.1002/jrsm.18.
Ioannidis JPA. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. Can Med Assoc J. 2009;181(8):488–93. doi:10.1503/cmaj.081086.
Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91–108. doi:10.1111/j.1471-1842.2009.00848.x.
Moe RH, Haavardsholm EA, Christie A, Jamtvedt G, Dahm KT, Hagen KB. Effectiveness of nonpharmacological and nonsurgical interventions for hip osteoarthritis: an umbrella review of high-quality systematic reviews. Phys Ther. 2007. doi:10.2522/ptj.20070042.
Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. doi:10.1371/journal.pmed.1000326.
Becker LA, Oxman AD. Overviews of reviews. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. Chichester: Wiley; 2008. doi:10.1002/9780470712184.ch22.
Hartling L, Vandermeer B, Fernandes RM. Systematic reviews, overviews of reviews and comparative effectiveness reviews: a discussion of approaches to knowledge synthesis. Evid Based Child Health Cochrane Rev J. 2014;9(2):486–94. doi:10.1002/ebch.1968.
Hartling L, Chisholm A, Thomson D, Dryden DM. A descriptive analysis of overviews of reviews published between 2000 and 2011. PLoS One. 2012;7(11):e49667. doi:10.1371/journal.pone.0049667.
Thomson D, Russell K, Becker L, Klassen T, Hartling L. The evolution of a new publication type: steps and challenges of producing overviews of reviews. Res Synth Methods. 2010;1(3–4):198–211. doi:10.1002/jrsm.30.
Pieper D, Buechter R, Jerinic P, Eikermann M. Overviews of reviews often have limited rigor: a systematic review. J Clin Epidemiol. 2012;65(12):1267–73. doi:10.1016/j.jclinepi.2012.06.015.
Thomson D, Foisy M, Oleszczuk M, Wingert A, Chisholm A, Hartling L. Overview of reviews in child health: evidence synthesis and the knowledge base for a specific population. Evid Based Child Health Cochrane Rev J. 2013;8(1):3–10. doi:10.1002/ebch.1897.
Harrold J, Ali S, Oleszczuk M, Lacaze-Masmonteil T, Hartling L. Corticosteroids for the prevention of bronchopulmonary dysplasia in preterm infants: an overview of Cochrane reviews. Evid Based Child Health Cochrane Rev J. 2013;8(6):2063–75. doi:10.1002/ebch.1955.
Adams NP, Jones PW. The dose–response characteristics of inhaled corticosteroids when used to treat asthma: an overview of Cochrane systematic reviews. Respir Med. 2006;100(8):1297–306. doi:10.1016/j.rmed.2006.04.015.
Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: a review of systematic reviews. Int J Technol Assess Health Care. 2005;21(3):380–5. doi:10.1017/S026646230505049X.
de Vet E, de Ridder DTD, de Wit JBF. Environmental correlates of physical activity and dietary behaviours among young people: a systematic review of reviews. Obes Rev. 2011;12(5):e130–42. doi:10.1111/j.1467-789X.2010.00784.x.
Cates CJ, Stovold E, Wieland S, Oleszczuk M, Thomson D, Becker L. The Cochrane Library and safety of regular long-acting beta2-agonists in children with asthma: an overview of reviews. Evid Based Child Health Cochrane Rev J. 2012;7(6):1798–806. doi:10.1002/ebch.1889.
Hillberg T, Hamilton-Giachritsis C, Dixon L. Review of meta-analyses on the association between child sexual abuse and adult mental health difficulties: a systematic approach. Trauma Violence Abuse. 2011;12(1):38–49. doi:10.1177/1524838010386812.
Trinquart L, Dechartres A, Ravaud P. Commentary: meta-epidemiology, meta-meta-epidemiology or network meta-epidemiology? Int J Epidemiol. 2013;42(4):1131–3. doi:10.1093/ije/dyt137.
Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408–12. doi:10.1001/jama.1995.03520290060030.
Bae J-M. Meta-epidemiology. Epidemiol Health. 2014;36:e2014019. doi:10.4178/epih/e2014019.
Zhang W. Meta-epidemiology: building the bridge from research evidence to clinical practice. Osteoarthritis Cartilage. 2010;18(Supplement 2):S1. doi:10.1016/S1063-4584(10)00293-1.
Tzoulaki I, Siontis KC, Ioannidis JP. Prognostic effect size of cardiovascular biomarkers in datasets from observational studies versus randomised trials: meta-epidemiology study. BMJ (Clin Res Ed). 2011;343:d6829. doi:10.1136/bmj.d6829.
Savović J, Jones H, Altman D, Harris R, Juni P, Pildal J, Als-Nielsen B, Balk E, Gluud C, Gluud L, Ioannidis J, Schulz K, Beynon R, Welton N, Wood L, Moher D, Deeks J, Sterne J. Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta-epidemiological studies. Health Technol Assess (Winchester, England). 2012;16(35):1–82. doi:10.3310/hta16350.
Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, Ioannidis JPA, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JAC. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med. 2012;157(6):429–38. doi:10.7326/0003-4819-157-6-201209180-00537.
Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med. 2002;21(11):1513–24. doi:10.1002/sim.1184.
Siersma V, Als-Nielsen B, Chen W, Hilden J, Gluud LL, Gluud C. Multivariable modelling for meta-epidemiological assessment of the association between trial quality and treatment effects estimated in randomized clinical trials. Stat Med. 2007;26(14):2745–58. doi:10.1002/sim.2752.
Chaimani A, Vasiliadis HS, Pandis N, Schmid CH, Welton NJ, Salanti G. Effects of study precision and risk of bias in networks of interventions: a network meta-epidemiological study. Int J Epidemiol. 2013;42(4):1120–31. doi:10.1093/ije/dyt074.
Salanti G, Dias S, Welton NJ, Ades AE, Golfinopoulos V, Kyrgiou M, Mauri D, Ioannidis JP. Evaluating novel agent effects in multiple-treatments meta-regression. Stat Med. 2010;29(23):2369–83. doi:10.1002/sim.4001.
Als-Nielsen B, Chen W, Lotte Gluud L, Siersma V, Hilden J, Gluud C. Are trial size and reported methodological quality associated with treatment effects? Observational study of 523 randomised trials. In: 12th Cochrane Colloquium: bridging the gaps. Cochrane, Ottawa, Canada 2004. 2–6.
Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, Lau J. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA. 2002;287(22):2973–82. doi:10.1001/jama.287.22.2973.
Contopoulos-Ioannidis DG, Gilbody SM, Trikalinos TA, Churchill R, Wahlbeck K, Ioannidis JPA; Project TE-P. Comparison of large versus smaller randomized trials for mental health-related interventions. Am J Psychiatry. 2005;162(3):578–84. doi:10.1176/appi.ajp.162.3.578.
Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess (Winchester, England). 2003;7(1):1–76.
Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001;135(11):982–9. doi:10.7326/0003-4819-135-11-200112040-00010.
McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356(9237):1228–31. doi:http://dx.doi.org/10.1016/S0140-6736(00)02786-0.
Moja LP, Telaro E, D’Amico R, Moschetti I, Coe L, Liberati A. Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. BMJ. 2005;330(7499):1053. doi:10.1136/bmj.38414.515938.8F.
Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. Int J Technol Assess Health Care. 2003;19(04):591–603. doi:10.1017/S0266462303000552.
Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, St. John PD, Viola R, Raina P. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol. 2003;56(10):943–55. doi:http://dx.doi.org/10.1016/S0895-4356(03)00110-0.
Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ (Clinical Res Ed). 2008;336(7644):601–5. doi:10.1136/bmj.39465.451748.AD.
Zhang W, Robertson J, Jones AC, Dieppe PA, Doherty M. The placebo effect and its determinants in osteoarthritis: meta-analysis of randomised controlled trials. Ann Rheum Dis. 2008;67(12):1716–23. doi:10.1136/ard.2008.092015.
Nüesch E, Trelle S, Reichenbach S, Rutjes AW, Burgi E, Scherer M, Altman DG, Juni P. The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study. BMJ (Clinical Research Ed). 2009;339:b3244. doi:10.1136/bmj.b3244.
Nüesch E, Trelle S, Reichenbach S, Rutjes AW, Tschannen B, Altman DG, Egger M, Juni P. Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study. BMJ (Clinical Res Ed). 2010;341:c3515. doi:10.1136/bmj.c3515.
Valdes AM, Arden NK, Tamm A, Kisand K, Doherty S, Pola E, Cooper C, Muir KR, Kerna I, Hart D, O’Neil F, Zhang W, Spector TD, Maciewicz RA, Doherty M. A meta-analysis of interleukin-6 promoter polymorphisms on risk of hip and knee osteoarthritis. Osteoarthritis Cartilage. 2010;18(5):699–704. doi:http://dx.doi.org/10.1016/j.joca.2009.12.012.
Oliver S, Bagnall AM, Thomas J, Shepherd J, Sowden A, White I, Dinnes J, Rees R, Colquitt J, Oliver K, Garrett Z. Randomised controlled trials for policy interventions: a review of reviews and meta-regression. Health Technol Assess (Winchester, England). 2010;14(16):1–165, iii. doi:10.3310/hta14160.
Herbison P, Hay-Smith J, Gillespie WJ. Different methods of allocation to groups in randomized trials are associated with different levels of bias. A meta-epidemiological study. J Clin Epidemiol. 2011;64(10):1070–5. doi:10.1016/j.jclinepi.2010.12.018.
Dechartres A, Trinquart L, Boutron I, Ravaud P. Influence of trial sample size on treatment effect estimates: meta-epidemiological study. BMJ (Clinical Res Ed). 2013;346:f2304. doi:10.1136/bmj.f2304.
Dechartres A, Boutron I, Trinquart L, Charles P, Ravaud P. Single-center trials show larger treatment effects than multicenter trials: evidence from a meta-epidemiologic study. Ann Intern Med. 2011;155(1):39–51. doi:10.7326/0003-4819-155-1-201107050-00006.
Zhang Z, Xu X, Ni H. Small studies may overestimate the effect sizes in critical care meta-analyses: a meta-epidemiological study. Crit Care (London, England). 2013;17(1):R2. doi:10.1186/cc11919.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Tsagris, M., Fragkos, K.C. (2016). Umbrella Reviews, Overviews of Reviews, and Meta-epidemiologic Studies: Similarities and Differences. In: Biondi-Zoccai, G. (eds) Umbrella Reviews. Springer, Cham. https://doi.org/10.1007/978-3-319-25655-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-25655-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25653-5
Online ISBN: 978-3-319-25655-9
eBook Packages: MedicineMedicine (R0)