Umbrella Reviews, Overviews of Reviews, and Meta-epidemiologic Studies: Similarities and Differences

  • Michail Tsagris
  • Konstantinos C. Fragkos


This chapter describes umbrella reviews, overviews of reviews, and meta-epidemiologic studies focusing on their definitions, purposes, and classifications where appropriate and then elaborating on their similarities and differences. We may consider umbrella reviews as reviews integrating several types of study designs but typically randomized controlled trials and systematic reviews of such studies in a unifying fashion in order to address a content issue (e.g., whether or not a given drug is superior to another). Overviews of reviews are reviews of systematic reviews and meta-analyses which can focus on content or methodological issues. Finally, meta-epidemiologic studies focus on potentially different types of study designs but most typically on randomized trials and systematic reviews, usually across different content domains (e.g., topics or conditions), and mainly aim at addressing methodological issues.


Evidence-based medicine Meta-analysis Meta-epidemiologic study Overview of reviews Umbrella reviews 


  1. 1.
    Naylor CD. Meta-analysis and the meta-epidemiology of clinical research. Meta-analysis is an important contribution to research and practice but it’s not a panacea. BMJ. 1997;315(7109):617–9. doi: 10.1136/bmj.315.7109.617.PubMedCentralCrossRefPubMedGoogle Scholar
  2. 2.
    Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. Chichester: Wiley; 2008. doi: 10.1002/9780470712184.Google Scholar
  3. 3.
    Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. Chichester: Wiley; 2009. doi: 10.1002/9780470743386.CrossRefGoogle Scholar
  4. 4.
    Savović J, Harris RJ, Wood L, Beynon R, Altman D, Als-Nielsen B, Balk EM, Deeks J, Gluud LL, Gluud C, Ioannidis JPA, Jűni P, Moher D, Pildal J, Schulz KF, Sterne JAC. Development of a combined database for meta-epidemiological research. Res Synth Methods. 2010;1(3–4):212–25. doi: 10.1002/jrsm.18.CrossRefPubMedGoogle Scholar
  5. 5.
    Ioannidis JPA. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. Can Med Assoc J. 2009;181(8):488–93. doi: 10.1503/cmaj.081086.CrossRefGoogle Scholar
  6. 6.
    Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91–108. doi: 10.1111/j.1471-1842.2009.00848.x.CrossRefPubMedGoogle Scholar
  7. 7.
    Moe RH, Haavardsholm EA, Christie A, Jamtvedt G, Dahm KT, Hagen KB. Effectiveness of nonpharmacological and nonsurgical interventions for hip osteoarthritis: an umbrella review of high-quality systematic reviews. Phys Ther. 2007. doi: 10.2522/ptj.20070042.Google Scholar
  8. 8.
    Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. doi: 10.1371/journal.pmed.1000326.PubMedCentralCrossRefPubMedGoogle Scholar
  9. 9.
    Becker LA, Oxman AD. Overviews of reviews. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. Chichester: Wiley; 2008. doi: 10.1002/9780470712184.ch22.Google Scholar
  10. 10.
    Hartling L, Vandermeer B, Fernandes RM. Systematic reviews, overviews of reviews and comparative effectiveness reviews: a discussion of approaches to knowledge synthesis. Evid Based Child Health Cochrane Rev J. 2014;9(2):486–94. doi: 10.1002/ebch.1968.CrossRefGoogle Scholar
  11. 11.
    Hartling L, Chisholm A, Thomson D, Dryden DM. A descriptive analysis of overviews of reviews published between 2000 and 2011. PLoS One. 2012;7(11):e49667. doi: 10.1371/journal.pone.0049667.PubMedCentralCrossRefPubMedGoogle Scholar
  12. 12.
    Thomson D, Russell K, Becker L, Klassen T, Hartling L. The evolution of a new publication type: steps and challenges of producing overviews of reviews. Res Synth Methods. 2010;1(3–4):198–211. doi: 10.1002/jrsm.30.CrossRefPubMedGoogle Scholar
  13. 13.
    Pieper D, Buechter R, Jerinic P, Eikermann M. Overviews of reviews often have limited rigor: a systematic review. J Clin Epidemiol. 2012;65(12):1267–73. doi: 10.1016/j.jclinepi.2012.06.015.CrossRefPubMedGoogle Scholar
  14. 14.
    Thomson D, Foisy M, Oleszczuk M, Wingert A, Chisholm A, Hartling L. Overview of reviews in child health: evidence synthesis and the knowledge base for a specific population. Evid Based Child Health Cochrane Rev J. 2013;8(1):3–10. doi: 10.1002/ebch.1897.CrossRefGoogle Scholar
  15. 15.
    Harrold J, Ali S, Oleszczuk M, Lacaze-Masmonteil T, Hartling L. Corticosteroids for the prevention of bronchopulmonary dysplasia in preterm infants: an overview of Cochrane reviews. Evid Based Child Health Cochrane Rev J. 2013;8(6):2063–75. doi: 10.1002/ebch.1955.CrossRefGoogle Scholar
  16. 16.
    Adams NP, Jones PW. The dose–response characteristics of inhaled corticosteroids when used to treat asthma: an overview of Cochrane systematic reviews. Respir Med. 2006;100(8):1297–306. doi: 10.1016/j.rmed.2006.04.015.CrossRefPubMedGoogle Scholar
  17. 17.
    Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: a review of systematic reviews. Int J Technol Assess Health Care. 2005;21(3):380–5. doi: 10.1017/S026646230505049X.CrossRefPubMedGoogle Scholar
  18. 18.
    de Vet E, de Ridder DTD, de Wit JBF. Environmental correlates of physical activity and dietary behaviours among young people: a systematic review of reviews. Obes Rev. 2011;12(5):e130–42. doi: 10.1111/j.1467-789X.2010.00784.x.CrossRefPubMedGoogle Scholar
  19. 19.
    Cates CJ, Stovold E, Wieland S, Oleszczuk M, Thomson D, Becker L. The Cochrane Library and safety of regular long-acting beta2-agonists in children with asthma: an overview of reviews. Evid Based Child Health Cochrane Rev J. 2012;7(6):1798–806. doi: 10.1002/ebch.1889.CrossRefGoogle Scholar
  20. 20.
    Hillberg T, Hamilton-Giachritsis C, Dixon L. Review of meta-analyses on the association between child sexual abuse and adult mental health difficulties: a systematic approach. Trauma Violence Abuse. 2011;12(1):38–49. doi: 10.1177/1524838010386812.CrossRefPubMedGoogle Scholar
  21. 21.
    Trinquart L, Dechartres A, Ravaud P. Commentary: meta-epidemiology, meta-meta-epidemiology or network meta-epidemiology? Int J Epidemiol. 2013;42(4):1131–3. doi: 10.1093/ije/dyt137.CrossRefPubMedGoogle Scholar
  22. 22.
    Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408–12. doi: 10.1001/jama.1995.03520290060030.CrossRefPubMedGoogle Scholar
  23. 23.
    Bae J-M. Meta-epidemiology. Epidemiol Health. 2014;36:e2014019. doi: 10.4178/epih/e2014019.PubMedCentralCrossRefPubMedGoogle Scholar
  24. 24.
    Zhang W. Meta-epidemiology: building the bridge from research evidence to clinical practice. Osteoarthritis Cartilage. 2010;18(Supplement 2):S1. doi: 10.1016/S1063-4584(10)00293-1.CrossRefGoogle Scholar
  25. 25.
    Tzoulaki I, Siontis KC, Ioannidis JP. Prognostic effect size of cardiovascular biomarkers in datasets from observational studies versus randomised trials: meta-epidemiology study. BMJ (Clin Res Ed). 2011;343:d6829. doi: 10.1136/bmj.d6829.CrossRefGoogle Scholar
  26. 26.
    Savović J, Jones H, Altman D, Harris R, Juni P, Pildal J, Als-Nielsen B, Balk E, Gluud C, Gluud L, Ioannidis J, Schulz K, Beynon R, Welton N, Wood L, Moher D, Deeks J, Sterne J. Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta-epidemiological studies. Health Technol Assess (Winchester, England). 2012;16(35):1–82. doi: 10.3310/hta16350.Google Scholar
  27. 27.
    Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, Ioannidis JPA, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JAC. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med. 2012;157(6):429–38. doi: 10.7326/0003-4819-157-6-201209180-00537.CrossRefPubMedGoogle Scholar
  28. 28.
    Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med. 2002;21(11):1513–24. doi: 10.1002/sim.1184.CrossRefPubMedGoogle Scholar
  29. 29.
    Siersma V, Als-Nielsen B, Chen W, Hilden J, Gluud LL, Gluud C. Multivariable modelling for meta-epidemiological assessment of the association between trial quality and treatment effects estimated in randomized clinical trials. Stat Med. 2007;26(14):2745–58. doi: 10.1002/sim.2752.CrossRefPubMedGoogle Scholar
  30. 30.
    Chaimani A, Vasiliadis HS, Pandis N, Schmid CH, Welton NJ, Salanti G. Effects of study precision and risk of bias in networks of interventions: a network meta-epidemiological study. Int J Epidemiol. 2013;42(4):1120–31. doi: 10.1093/ije/dyt074.CrossRefPubMedGoogle Scholar
  31. 31.
    Salanti G, Dias S, Welton NJ, Ades AE, Golfinopoulos V, Kyrgiou M, Mauri D, Ioannidis JP. Evaluating novel agent effects in multiple-treatments meta-regression. Stat Med. 2010;29(23):2369–83. doi: 10.1002/sim.4001.PubMedGoogle Scholar
  32. 32.
    Als-Nielsen B, Chen W, Lotte Gluud L, Siersma V, Hilden J, Gluud C. Are trial size and reported methodological quality associated with treatment effects? Observational study of 523 randomised trials. In: 12th Cochrane Colloquium: bridging the gaps. Cochrane, Ottawa, Canada 2004. 2–6.Google Scholar
  33. 33.
    Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, Lau J. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA. 2002;287(22):2973–82. doi: 10.1001/jama.287.22.2973.CrossRefPubMedGoogle Scholar
  34. 34.
    Contopoulos-Ioannidis DG, Gilbody SM, Trikalinos TA, Churchill R, Wahlbeck K, Ioannidis JPA; Project TE-P. Comparison of large versus smaller randomized trials for mental health-related interventions. Am J Psychiatry. 2005;162(3):578–84. doi: 10.1176/appi.ajp.162.3.578.
  35. 35.
    Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess (Winchester, England). 2003;7(1):1–76.Google Scholar
  36. 36.
    Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001;135(11):982–9. doi: 10.7326/0003-4819-135-11-200112040-00010.CrossRefPubMedGoogle Scholar
  37. 37.
    McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356(9237):1228–31. doi: Scholar
  38. 38.
    Moja LP, Telaro E, D’Amico R, Moschetti I, Coe L, Liberati A. Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. BMJ. 2005;330(7499):1053. doi: 10.1136/bmj.38414.515938.8F.PubMedCentralCrossRefPubMedGoogle Scholar
  39. 39.
    Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. Int J Technol Assess Health Care. 2003;19(04):591–603. doi: 10.1017/S0266462303000552.CrossRefPubMedGoogle Scholar
  40. 40.
    Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, St. John PD, Viola R, Raina P. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol. 2003;56(10):943–55. doi: Scholar
  41. 41.
    Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ (Clinical Res Ed). 2008;336(7644):601–5. doi: 10.1136/bmj.39465.451748.AD.CrossRefGoogle Scholar
  42. 42.
    Zhang W, Robertson J, Jones AC, Dieppe PA, Doherty M. The placebo effect and its determinants in osteoarthritis: meta-analysis of randomised controlled trials. Ann Rheum Dis. 2008;67(12):1716–23. doi: 10.1136/ard.2008.092015.CrossRefPubMedGoogle Scholar
  43. 43.
    Nüesch E, Trelle S, Reichenbach S, Rutjes AW, Burgi E, Scherer M, Altman DG, Juni P. The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study. BMJ (Clinical Research Ed). 2009;339:b3244. doi: 10.1136/bmj.b3244.CrossRefGoogle Scholar
  44. 44.
    Nüesch E, Trelle S, Reichenbach S, Rutjes AW, Tschannen B, Altman DG, Egger M, Juni P. Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study. BMJ (Clinical Res Ed). 2010;341:c3515. doi: 10.1136/bmj.c3515.CrossRefGoogle Scholar
  45. 45.
    Valdes AM, Arden NK, Tamm A, Kisand K, Doherty S, Pola E, Cooper C, Muir KR, Kerna I, Hart D, O’Neil F, Zhang W, Spector TD, Maciewicz RA, Doherty M. A meta-analysis of interleukin-6 promoter polymorphisms on risk of hip and knee osteoarthritis. Osteoarthritis Cartilage. 2010;18(5):699–704. doi: Scholar
  46. 46.
    Oliver S, Bagnall AM, Thomas J, Shepherd J, Sowden A, White I, Dinnes J, Rees R, Colquitt J, Oliver K, Garrett Z. Randomised controlled trials for policy interventions: a review of reviews and meta-regression. Health Technol Assess (Winchester, England). 2010;14(16):1–165, iii. doi: 10.3310/hta14160.Google Scholar
  47. 47.
    Herbison P, Hay-Smith J, Gillespie WJ. Different methods of allocation to groups in randomized trials are associated with different levels of bias. A meta-epidemiological study. J Clin Epidemiol. 2011;64(10):1070–5. doi: 10.1016/j.jclinepi.2010.12.018.CrossRefPubMedGoogle Scholar
  48. 48.
    Dechartres A, Trinquart L, Boutron I, Ravaud P. Influence of trial sample size on treatment effect estimates: meta-epidemiological study. BMJ (Clinical Res Ed). 2013;346:f2304. doi: 10.1136/bmj.f2304.Google Scholar
  49. 49.
    Dechartres A, Boutron I, Trinquart L, Charles P, Ravaud P. Single-center trials show larger treatment effects than multicenter trials: evidence from a meta-epidemiologic study. Ann Intern Med. 2011;155(1):39–51. doi: 10.7326/0003-4819-155-1-201107050-00006.CrossRefPubMedGoogle Scholar
  50. 50.
    Zhang Z, Xu X, Ni H. Small studies may overestimate the effect sizes in critical care meta-analyses: a meta-epidemiological study. Crit Care (London, England). 2013;17(1):R2. doi: 10.1186/cc11919.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of CreteHeraklionGreece
  2. 2.Division of MedicineUniversity College LondonLondonUK

Personalised recommendations