Skip to main content

Advertisement

SpringerLink
Log in

Systematic Reviews: Characteristics and Impact

  • Published:
Publishing Research Quarterly Aims and scope Submit manuscript

Abstract

Since 1990, systematic reviews are growing exponentially with thousands being published each year. The objectives of this study were to determine both their temporal characteristics in terms of growth per year, subject areas, and publishing affiliations as well as their scientific impact. In this study we used 106,038 systematic reviews collected from Web of Science in 2019. These articles were analyzed to identify topics and publishing institutions, scientific impact and more. Our data shows that while the number of systematic reviews grows each year, their scientific impact diminishes. This can be seen in both citations and usage metrics. The journals that publish the most systematic reviews are below the normal Impact Factor for journals in the medical and biomedical arenas. There are very few institutions around the world, that produce most of the systematic reviews. Topics vary from one institution to another. The sheer number of systematic reviews publications is not an indication of quality or of impact. In fact, our data show that these are on the decline. There seems to be saturation in this area, which results in less interest in and utility of systematic reviews.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Chalmers I, Hedges LV, Cooper H. A brief history of research synthesis. Eval Health Prof. 2002;25(1):12–37.

    Article  Google Scholar 

  2. Chevret S, Ferguson ND, Bellomo R. Are systematic reviews and meta-analyses still useful research? No. Berlin: Springer; 2018.

    Book  Google Scholar 

  3. Cochrane Collaboration, Higgins JPT. Cochrane handbook for systematic reviews of interventions (pp. 1 recurs en línia (1 recurs en línia (XXI, 649 pàgines))). Hoboken: Wiley-Blackwell; 2008.

  4. Ebrahim S, Bance S, Athale A, Malachowski C, Ioannidis JPA. Meta-analyses with industry involvement are massively published and report no caveats for antidepressants. J Clin Epidemiol. 2016;70:155–63. https://doi.org/10.1016/j.jclinepi.2015.08.021.

    Article  Google Scholar 

  5. Gogos C, Kodonas K, Fardi A, Economides N. Top 100 cited systematic reviews and meta-analyses in dentistry. Acta OdontolScand. 2019;78:87–97. https://doi.org/10.1080/00016357.2019.1653495.

    Article  Google Scholar 

  6. Gómez-García F, Ruano J, Aguilar-Luque M, Gay-Mimbrera J, Maestre-Lopez B, Sanz-Cabanillas JL, Carmona-Fernández PJ, González-Padilla M, VélezGarcía-Nieto A, Isla-Tejera B. Systematic reviews and meta-analyses on psoriasis: role of funding sources, conflict of interest and bibliometric indices as predictors of methodological quality. Br J Dermatol. 2017;176(6):1633–44. https://doi.org/10.1111/bjd.15380.

    Article  Google Scholar 

  7. Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJClin Res Ed. 2012;344:d7202. https://doi.org/10.1136/bmj.d7202.

    Article  Google Scholar 

  8. Hedin RJ, Umberham BA, Detweiler BN, Kollmorgen L, Vassar M. Publication bias and nonreporting found in majority of systematic reviews and meta-analyses in anesthesiology journals. AnesthAnalg. 2016;123(4):1018–25. https://doi.org/10.1213/ANE.0000000000001452.

    Article  Google Scholar 

  9. Ioannidis JPA. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Quart. 2016;94(3):485–514. https://doi.org/10.1111/1468-0009.12210.

    Article  Google Scholar 

  10. Jüni P, Holenstein F, Sterne J, Bartlett C, Egger M. Direction and impact of language bias in meta-analyses of controlled trials: empirical study. Int J Epidemiol. 2002;31(1):115–23. https://doi.org/10.1093/ije/31.1.115.

    Article  Google Scholar 

  11. Knobloch K, Yoon U, Vogt PM. Preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement and publication bias. J Cranio-Maxillo-FacSurg Off PublEurAssoc Cranio-Maxillo-FacSurg. 2011;39(2):91–2. https://doi.org/10.1016/j.jcms.2010.11.001.

    Article  Google Scholar 

  12. Lilljeborg W. Zoologische Studien: Festschrift Wilhelm Lilljeborg zum achtzigsten Geburtstag gewidmet von schwedischen Zoologen. Stockholm: Almqvist and Wiksell; 1896.

    Google Scholar 

  13. Manriquez J, Andino-Navarrete R, Cataldo-Cerda K, Harz-Fresno I. Bibliometric characteristics of systematic reviews in dermatology: a cross-sectional study through web of science and scopus. DermatologicaSinica. 2015;33(3):154–6. https://doi.org/10.1016/j.dsi.2014.12.007.

    Article  Google Scholar 

  14. Mao Z, Wang GQ, Mei XF, Chen S, Liu XX, Zeng XT, Long AH, Zhang LC, Zhang LH, Tang PF. Systematic reviews on reports of hip fractures in web of science: a bibliometric analysis of publication activity. Chin Med J. 2014;127(13):2518–22. https://doi.org/10.3760/cma.j.issn.0366-6999.20132111.

    Article  Google Scholar 

  15. Moed HF, Halevi G. On full text download and citation distributions in scientific-scholarly journals. J AssocInf Sci Technol. 2016;67(2):412–31.

    Google Scholar 

  16. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9.

    Article  Google Scholar 

  17. Møller MH, Ioannidis JP, Darmon M. Are systematic reviews and meta-analyses still useful research? We are not sure. Berlin: Springer; 2018.

    Book  Google Scholar 

  18. Morrison A, Polisena J, Husereau D, Moulton K, Clark M, Fiander M, Mierzwinski-Urban M, Clifford T, Hutton B, Rabb D. The effect of English-language restriction on systematic review-based meta-analyses:a systematic review of empirical studies. Int J Technol Assess Health Care. 2012;28(2):138–44. https://doi.org/10.1017/S0266462312000086.

    Article  Google Scholar 

  19. Page MJ, Shamseer L, Tricco AC. Registration of systematic reviews in PROSPERO: 30,000 records and counting. Syst Rev. 2018;7(1):32. https://doi.org/10.1186/s13643-018-0699-4.

    Article  Google Scholar 

  20. PlumX Metrics—Plum Analytics. https://plumanalytics.com/learn/about-metrics/ (2020). Retrieved 13 July 2020.

  21. Scudder SH. Systematic review of our present knowledge of fossil insects, including myridpods and arachnids. Washington, DC: Govt. Print. Off.; 1886.

    Google Scholar 

  22. Turner L, Boutron I, Hróbjartsson A, Altman DG, Moher D. The evolution of assessing bias in Cochrane systematic reviews of interventions: celebrating methodological contributions of the Cochrane Collaboration. Syst Rev. 2013;2:79. https://doi.org/10.1186/2046-4053-2-79.

    Article  Google Scholar 

  23. Word Counter—Count Words and Check Grammar. https://wordcounter.io/ (2020). Retrieved 13 July 2020.

  24. Zhang Y, Huang J, Du L. The top-cited systematic reviews/meta-analyses in tuberculosis research: a PRISMA-compliant systematic literature review and bibliometric analysis. Medicine. 2017;96(6):e4822. https://doi.org/10.1097/MD.0000000000004822.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Icahn School of Medicine at Mount Sinai, Gustave L. Levy Pl #11, New York, NY, 10029, USA

    Gali Halevi & Rachel Pinotti

Authors
  1. Gali Halevi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rachel Pinotti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gali Halevi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Halevi, G., Pinotti, R. Systematic Reviews: Characteristics and Impact. Pub Res Q 36, 523–537 (2020). https://doi.org/10.1007/s12109-020-09766-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12109-020-09766-6

Keywords

Search

Navigation