Systematic reviews and meta-analysis in rheumatology: a gentle introduction for clinicians

  • George A. KelleyEmail author
  • Kristi S. Kelley
Review Article


Given the plethora of studies today on the same topic, clinicians in rheumatology as well as others increasingly rely on systematic reviews, with or without meta-analysis, to aid in their evidence-based decision-making. However, given time constraints, staying up-to-date on current methods for conducting systematic reviews and meta-analyses as well as interpreting the results of these reviews for application in clinical practice can be challenging. The purpose of this paper is to try and address this gap. In this paper, a description of the different types of systematic reviews and meta-analyses is provided as well as a description of the major elements, including methodology and interpretation of systematic reviews with meta-analyses. Included is a broad, five-question checklist to aid clinicians in rheumatology for making decisions about the utility of a systematic review. It is the hopes that this paper will aid clinicians in rheumatology as well as other consumers of systematic reviews and meta-analyses with the information necessary for judging the utility of systematic reviews and meta-analyses in their own work.


Clinicians Meta-analysis guide Rheumatology Systematic reviews 


Compliance with ethical standards




  1. 1.
    Higgins JPT, Green S (eds) (2011) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane CollaborationGoogle Scholar
  2. 2.
    Pham MT, Rajic A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA (2014) A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods 5(4):371–385Google Scholar
  3. 3.
    Misra DP, Sharma A, Kadhiravan T, Negi VS (2017) A scoping review of the use of non-biologic disease modifying anti-rheumatic drugs in the management of large vessel vasculitis. Autoimmun Rev 16(2):179–191Google Scholar
  4. 4.
    Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, Moher D, Peters MDJ, Horsley T, Weeks L, Hempel S, Akl EA, Chang C, McGowan J, Stewart L, Hartling L, Aldcroft A, Wilson MG, Garritty C, Lewin S, Godfrey CM, Macdonald MT, Langlois EV, Soares-Weiser K, Moriarty J, Clifford T, Tunçalp Ö, Straus SE (2018) PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation: the PRISMA-ScR statement. Ann Intern Med 169(7):467–473Google Scholar
  5. 5.
    Kelley GA, Kelley KS (2018) Systematic reviews and cancer research: a suggested stepwise approach. BMC Cancer 18(246):9Google Scholar
  6. 6.
    McKenzie JE, Brennan SE (2017) Overviews of systematic reviews: great promise, greater challenge. Syst Rev 6(1):185Google Scholar
  7. 7.
    Christie A, Jamtvedt G, Dahm KT, Moe RH, Haavardsholm EA, Hagen KB (2007) Effectiveness of nonpharmacological and nonsurgical interventions for patients with rheumatoid arthritis: an overview of systematic reviews. Phys Ther 87(12):1697–1715Google Scholar
  8. 8.
    Fusar-Poli P, Radua J (2018) Ten simple rules for conducting umbrella reviews. Evid Based Mental Health 21(3):95–100Google Scholar
  9. 9.
    Tierney JF, Vale C, Riley R, Smith CT, Stewart L, Clarke M, Rovers M (2015) Individual participant data (IPD) meta-analyses of randomised controlled trials: guidance on their use. PLoS Med 12(7):e1001855Google Scholar
  10. 10.
    Riley RD, Simmonds MC, Look MP (2007) Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. J Clin Epidemiol 60(5):431–439Google Scholar
  11. 11.
    Kelley GA, Kelley KS (2016) Retrieval of individual participant data for exercise meta-analyses may not be worth the time and effort. Biomed Res Int 2016(5059041):1–5Google Scholar
  12. 12.
    Riley RD, Lambert PC, Staessen JA, Wang J, Gueyffier F, Thijs L, Boutitie F (2008) Meta-analysis of continuous outcomes combining individual patient data and aggregate data. Stat Med 27(11):1870–1893Google Scholar
  13. 13.
    Steinberg KK, Smith SJ, Stroup DF, Olkin I, Lee NC, Williamson GD, Thacker SB (1997) Comparison of effect size estimates from a meta-analysis of summary data from published studies and from a meta-analysis using individual patient data for ovarian cancer studies. Am J Epidemiol 145:917–925Google Scholar
  14. 14.
    Cooper H, Patall EA (2009) The relative benefits of meta-analysis conducted with individual participant data versus aggregated data. Psychol Methods 14(2):165–176Google Scholar
  15. 15.
    Stewart LA, Clarke M, Rovers M, Riley RD, Simmonds M, Stewart G, Tierney JF (2015) Preferred reporting items for systematic review and meta-analyses of individual participant data: the PRISMA-IPD statement. JAMA 313(16):1657–1665Google Scholar
  16. 16.
    Higgins J, Green S (2008) Cochrane handbook for systematic reviews of interventions (version 5.0.1)Google Scholar
  17. 17.
    Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med 151(4):W65–W94Google Scholar
  18. 18.
    Dekkers OM, Vandenbroucke JP, Cevallos M, Renehan AG, Altman DG, Egger M (2019) COSMOS-E: guidance on conducting systematic reviews and meta-analyses of observational studies of etiology. PLoS Med 16(2):e1002742Google Scholar
  19. 19.
    Petticrew M, Rehfuess E, Noyes J, Higgins JPT, Mayhew A, Pantoja T, Shemilt I, Sowden A (2013) Synthesizing evidence on complex interventions: how meta-analytical, qualitative, and mixed-method approaches can contribute. J Clin Epidemiol 66(11):1230–1243Google Scholar
  20. 20.
    Alfonso-Cristancho R, Armstrong N, Arjunji R, Riemsma R, Worthy G, Ganguly R, Kleijnen J (2017) Comparative effectiveness of biologics for the management of rheumatoid arthritis: systematic review and network meta-analysis. Clin Rheumatol 36(1):25–34Google Scholar
  21. 21.
    Riley RD, Jackson D, Salanti G, Burke DL, Price M, Kirkham J, White IR (2017) Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples. Br Med J 358Google Scholar
  22. 22.
    Mills EJ, Ioannidis JP, Thorlund K, Schunemann HJ, Puhan MA, Guyatt GH (2012) How to use an article reporting a multiple treatment comparison meta-analysis. JAMA 308(12):1246–1253Google Scholar
  23. 23.
    Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, Ioannidis JP, Straus S, Thorlund K, Jansen JP, Mulrow C, Catala-Lopez F, Gotzsche PC, Dickersin K, Boutron I, Altman DG, Moher D (2015) The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med 162(11):777–784Google Scholar
  24. 24.
    Brittain EH, Fay MP, Follmann DA (2012) A valid formulation of the analysis of noninferiority trials under random effects meta-analysis. Biostatistics 13(4):637–649Google Scholar
  25. 25.
    Sacks HS, Berrier J, Reitman D, Ancona-Berk VA, Chalmers TC (1987) Meta-analysis of randomized controlled trials. N Engl J Med 316:450–455Google Scholar
  26. 26.
    Mohammed AT, Khattab M, Ahmed AM, Turk T, Sakr N, Khalil AM, Abdelhalim M, Sawaf B, Hirayama K, Huy NT (2017) The therapeutic effect of probiotics on rheumatoid arthritis: a systematic review and meta-analysis of randomized control trials. Clin Rheumatol 36(12):2697–2707Google Scholar
  27. 27.
    Ioannidis JPA (2016) The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q 94(5):485–514Google Scholar
  28. 28.
    Page MJ, Shamseer L, Tricco AC (2018) Registration of systematic reviews in PROSPERO: 30,000 records and counting. Syst Rev 7(1):32Google Scholar
  29. 29.
    Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA (2015) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. Br Med J 349:g7647Google Scholar
  30. 30.
    Kelley GA, Kelley KS, Callahan LF (2018) Community-deliverable exercise and anxiety in adults with arthritis and other rheumatic diseases: a systematic review with meta-analysis of randomised controlled trials. BMJ Open 8(2):18Google Scholar
  31. 31.
    Dickersin K, Scherer R, Lefebvre C (1994) Identifying relevant studies for systematic reviews. Br Med J 309:1286–1291Google Scholar
  32. 32.
    Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH (2017) Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev 6:245Google Scholar
  33. 33.
    Vine R (2006) Google Scholar. J Med Libr Assoc 94(1):97–99Google Scholar
  34. 34.
    Burnham JF (2006) Scopus database: a review. Biomed Digit Libr 3(1):1–8Google Scholar
  35. 35.
    Pedder H, Sarri G, Keeney E, Nunes V, Dias S (2016) Data extraction for complex meta-analysis (DECiMAL) guide. Syst Rev 5(1):212Google Scholar
  36. 36.
    Sanderson S, Tatt ID, Higgins JP (2007) Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol 36(3):666–676Google Scholar
  37. 37.
    Seehra J, Pandis N, Koletsi D, Fleming PS (2016) Use of quality assessment tools in systematic reviews was varied and inconsistent. J Clin Epidemiol 69:179–184Google Scholar
  38. 38.
    Higgins JPT, Sterne JAC, Savović J, Page MJ, Hróbjartsson A, Boutron I, Reeves B, Eldridge S (2016) A revised tool for assessing risk of bias in randomized trials. Cochrane Database Syst Rev 10(Supplement 1)Google Scholar
  39. 39.
    Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I, Carpenter JR, Chan A-W, Churchill R, Deeks JJ, Hróbjartsson A, Kirkham J, Jüni P, Loke YK, Pigott TD, Ramsay CR, Regidor D, Rothstein HR, Sandhu L, Santaguida PL, Schünemann HJ, Shea B, Shrier I, Tugwell P, Turner L, Valentine JC, Waddington H, Waters E, Wells GA, Whiting PF, Higgins JP (2016) ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 355Google Scholar
  40. 40.
    Kelley GA, Kelley KS, Hootman JM (2015) Effects of exercise on depression in adults with arthritis: a systematic review with meta-analysis of randomized controlled trials. Arthritis Res Ther 17:21Google Scholar
  41. 41.
    Zeng D, Lin DY (2015) On random-effects meta-analysis. Biometrika 102(2):281–294Google Scholar
  42. 42.
    Dersimonian R, Laird N (1986) Meta-analysis in clinical trials. Control Clin Trials 7:177–188Google Scholar
  43. 43.
    Doi SA, Barendregt JJ, Khan S, Thalib L, Williams GM (2015) Advances in the meta-analysis of heterogeneous clinical trials I: the inverse variance heterogeneity model. Contemp Clin Trials 45(Pt A):130–138Google Scholar
  44. 44.
    Doi SA, Barendregt JJ, Khan S, Thalib L, Williams GM (2015) Advances in the meta-analysis of heterogeneous clinical trials II: the quality effects model. Contemp Clin Trials 45(Pt A):123–129Google Scholar
  45. 45.
    Higgins JP, Thompson SG, Spiegelhalter DJ (2009) A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A 172(1):137–159Google Scholar
  46. 46.
    Partlett C, Riley RD (2017) Random effects meta-analysis: coverage performance of 95% confidence and prediction intervals following REML estimation. Stat Med 36(2):301–317Google Scholar
  47. 47.
    Ioannidis JP, Patsopoulos NA, Evangelou E (2007) Uncertainty in heterogeneity estimates in meta-analyses. BMJ 335(7626):914–916Google Scholar
  48. 48.
    Sterne JA, Sutton AJ, Ioannidis JP, Terrin N, Jones DR, Lau J, Carpenter J, Rucker G, Harbord RM, Schmid CH, Tetzlaff J, Deeks JJ, Peters J, Macaskill P, Schwarzer G, Duval S, Altman DG, Moher D, Higgins JP (2011) Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. Br Med J 343:d4002Google Scholar
  49. 49.
    Furuya-Kanamori L, Barendregt JJ, Doi SAR (2018) A new improved graphical and quantitative method for detecting bias in meta-analysis. Int J Evid Based Healthc 16(4):195–203Google Scholar
  50. 50.
    Mallett S, Clarke M (2002) The typical Cochrane review. How many trials? How many participants? Int J Technol Assess Health Care 18(4):820–823Google Scholar
  51. 51.
    Clarke M, Brice A, Chalmers I (2014) Accumulating research: a systematic account of how cumulative meta-analyses would have provided knowledge, improved health, reduced harm and saved resources. PLoS One 9(7):e102670Google Scholar
  52. 52.
    Higgins J, Thompson S, Deeks J, Altman D (2002) Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice. J Health Serv Res Policy 7(1):51–61Google Scholar
  53. 53.
    Cohen J (1988) Statistical power analysis for the behavioral sciences. Academic Press, New YorkGoogle Scholar
  54. 54.
    Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ (2008) GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. Br Med J 336(7650):924–926Google Scholar
  55. 55.
    Fiest KM, Hitchon CA, Bernstein CN, Peschken CA, Walker JR, Graff LA, Zarychanski R, Abou-Setta A, Patten SB, Sareen J, Bolton J, Marrie RA, Burden CTDt (2017) Managing the effects of psychiatric comorbidity in chronic immunoinflammatory D (2017) systematic review and meta-analysis of interventions for depression and anxiety in persons with rheumatoid arthritis. J Clin Rheumatol 23(8):425–434Google Scholar

Copyright information

© International League of Associations for Rheumatology (ILAR) 2019

Authors and Affiliations

  1. 1.Meta-Analytic Research Group, School of Public Health, Department of Biostatistics, Robert C. Byrd Health Sciences CenterWest Virginia UniversityMorgantownUSA

Personalised recommendations