Skip to main content

Network Meta-Analysis Techniques for Synthesizing Prevention Science Evidence

Prevention Science volume 23, pages 415–424 (2022)Cite this article

Abstract

Network meta-analysis is a popular statistical technique for synthesizing evidence from studies comparing multiple interventions. Benefits of network meta-analysis, over more traditional pairwise meta-analysis approaches, include evaluating efficacy/safety of interventions within a single framework, increased precision, comparing pairs of interventions that have never been directly compared in a trial, and providing a hierarchy of interventions in terms of their effectiveness. Network meta-analysis is relatively underutilized in prevention science. This paper therefore presents a primer of network meta-analysis for prevention scientists who wish to apply this method or to critically appraise evidence from publications using the method. We introduce the key concepts and assumptions of network meta-analysis, namely, transitivity and consistency, and demonstrate their applicability to the field of prevention science. We then illustrate the method using a network meta-analysis examining the comparative effectiveness of brief alcohol interventions for preventing hazardous drinking among college students. We provide data and code for all examples. Finally, we discuss considerations that are particularly relevant in network meta-analyses in the field of prevention, such as including non-randomized evidence.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2

References

  • Achana, F. A., Sutton, A. J., Kendrick, D., Wynn, P., Young, B., Jones, D. R., et al. (2015). The effectiveness of different interventions to promote poison prevention behaviours in households with children: A network meta-analysis. PloS One, 10, e0121122.

  • Baker, S. G., & Kramer, B. S. (2002). The transitive fallacy for randomized trials: If A bests B and B bests C in separate trials, is A better than C? BMC Medical Research Methodology, 2, 1–5.

    Article  Google Scholar 

  • Barth, J., Munder, T., Gerger, H., Nüesch, E., Trelle, S., Znoj, H., et al. (2016). Comparative efficacy of seven psychotherapeutic interventions for patients with depression: A network meta-analysis. Focus, 14, 229–243.

    Article  Google Scholar 

  • Cameron, C., Fireman, B., Hutton, B., Clifford, T., Coyle, D., Wells, G., et al. (2015). Network meta-analysis incorporating randomized controlled trials and non-randomized comparative cohort studies for assessing the safety and effectiveness of medical treatments: Challenges and opportunities. Systematic Reviews, 4, 1–8.

    Article  Google Scholar 

  • Chaimani, A., Caldwell, D. M., Li, T., Higgins, J. P., & Salanti, G. (2017). Additional considerations are required when preparing a protocol for a systematic review with multiple interventions. Journal of Clinical Epidemiology, 83, 65–74.

    Article  Google Scholar 

  • Chaimani, A., & Salanti, G. (2015). Visualizing assumptions and results in network meta-analysis: The Network Graphs Package. The Stata Journal, 15, 905–950. https://doi.org/10.1177/1536867x1501500402

    Article  Google Scholar 

  • Chaimani, A., Vasiliadis, H. S., Pandis, N., Schmid, C. H., Welton, N. J., & Salanti, G. (2013). Effects of study precision and risk of bias in networks of interventions: A network meta-epidemiological study. International Journal of Epidemiology, 42, 1120–1131. https://doi.org/10.1093/ije/dyt074

    Article  PubMed  Google Scholar 

  • Chandler, J., Cumpston, M., Li, T., & Page, M. J. (2019). & Welch (Vol. A). John Wiley & Sons.

    Google Scholar 

  • Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) Checking consistency in mixed treatment comparison meta-analysis. Statistics in Medicine, 29, 932–944.

  • Dias, S., Sutton, A. J., Welton, N. J., & Ades, A. (2016). Heterogeneity: Subgroups, meta-regression, bias and bias-adjustment.

  • Donegan, S., Williamson, P., Gamble, C., & Tudur-Smith, C. (2010). Indirect comparisons: A review of reporting and methodological quality. PloS One, 5.

  • Efthimiou, O., Mavridis, D., Debray, T. P., Samara, M., Belger, M., Siontis, G. C., et al. (2017). Combining randomized and non-randomized evidence in network meta-analysis. Statistics in Medicine, 36, 1210–1226.

    Article  Google Scholar 

  • Hennessy, E. A., & Tanner-Smith, E. E. (2015) Effectiveness of brief school-based interventions for adolescents: A meta-analysis of alcohol use prevention programs. Prevention Science, 16, 463–474.

  • Hennessy, E. A., Tanner-Smith, E. E., Mavridis, D., & Grant, S. P. (2019). Comparative effectiveness of brief alcohol interventions for college students: Results from a network meta-analysis. Prevention Science, 20, 715–740.

    Article  Google Scholar 

  • Henry, D., Tolan, P., Gorman-Smith, D., & Schoeny, M. (2017). Alternatives to randomized control trial designs for community-based prevention evaluation. Prevention Science, 18(6), 671–680.

    Article  Google Scholar 

  • Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., & White, I. R. (2012). Consistency and inconsistency in network meta-analysis: Concepts and models for multi-arm studies. Research Synthesis Methods, 3, 98–110.

    Article  CAS  Google Scholar 

  • Higgins, J. P., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, D., Oxman, A. D., et al. (2011). The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ, 343, d5928.

  • Hubbard, S., Cooper, N., Kendrick, D., Young, B., Wynn, P. M., He, Z., et al. (2015). Network meta-analysis to evaluate the effectiveness of interventions to prevent falls in children under age 5 years. Injury Prevention, 21, 98–108.

    Article  Google Scholar 

  • Hutton, B., Salanti, G., Caldwell, D. M., Chaimani, A., Schmid, C. H., Cameron, C., et al. (2015). The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: Checklist and explanations. Annals of Internal Medicine, 162, 777–784.

    Article  Google Scholar 

  • Kanters, S., Park, J. J., Chan, K., Socias, M. E., Ford, N., Forrest, J. I., et al. (2017). Interventions to improve adherence to antiretroviral therapy: A systematic review and network meta-analysis. The Lancet HIV, 4, e31–e40.

    Article  Google Scholar 

  • König, J., Krahn, U., & Binder, H. (2013). Visualizing the flow of evidence in network meta-analysis and characterizing mixed treatment comparisons. Statistics in Medicine, 32, 5414–5429.

    Article  Google Scholar 

  • Lin, L., Chu, H., & Hodges, J. S. (2016). Sensitivity to Excluding Treatments in Network Meta-analysis. Epidemiology, 27, 562–569. https://doi.org/10.1097/ede.0000000000000482

    Article  PubMed  PubMed Central  Google Scholar 

  • Lu, G., & Ades, A. (2004). Combination of direct and indirect evidence in mixed treatment comparisons. Statistics in Medicine, 23, 3105–3124.

    Article  CAS  Google Scholar 

  • Lu, G., & Ades, A. (2006). Assessing evidence inconsistency in mixed treatment comparisons. Journal of the American Statistical Association, 101, 447–459.

    Article  CAS  Google Scholar 

  • Luangasanatip, N., Hongsuwan, M., Limmathurotsakul, D., Lubell, Y., Lee, A. S., Harbarth, S., et al. (2015). Comparative efficacy of interventions to promote hand hygiene in hospital: Systematic review and network meta-analysis. BMJ, 351.

  • Lumley, T. (2002). Network meta-analysis for indirect treatment comparisons. Statistics in Medicine, 21, 2313–2324.

    Article  Google Scholar 

  • Mavridis, D., Giannatsi, M., Cipriani, A., & Salanti, G. (2015). A primer on network meta-analysis with emphasis on mental health. Evidence-Based Mental Health, 18, 40–46.

    Article  Google Scholar 

  • Mavridis, D., Palmer, S. C., & Strippoli, G. F. (2016). Comparative superiority of ACE inhibitors over angiotensin receptor blockers for people with CKD: Does it matter? American Journal of Kidney Diseases, 67, 713–715.

    Article  Google Scholar 

  • Mavridis, D., Porcher, R., Nikolakopoulou, A., Salanti, G., & Ravaud, P. (2020). Extensions of the probabilistic ranking metrics of competing treatments in network meta-analysis to reflect clinically important relative differences on many outcomes. Biometrical Journal, 62, 375–385. https://doi.org/10.1002/bimj.201900026

    Article  PubMed  Google Scholar 

  • Mills, E. J., Kanters, S., Thorlund, K., Chaimani, A., Veroniki, A. A., & Ioannidis, J. P. (2013). The effects of excluding treatments from network meta-analyses: Survey. BMJ, 347, f5195. https://doi.org/10.1136/bmj.f5195

    Article  PubMed  PubMed Central  Google Scholar 

  • Nikolakopoulou, A., Chaimani, A., Veroniki, A. A., Vasiliadis, H. S., Schmid, C. H., & Salanti, G. (2014a). Characteristics of networks of interventions: A description of a database of 186 published networks. PloS One, 9.

  • Nikolakopoulou, A., Higgins, J. P., Papakonstantinou, T., Chaimani, A., Del Giovane, C., Egger, M., et al. (2020). CINeMA: An approach for assessing confidence in the results of a network meta-analysis. PLoS Medicine, 17, e1003082.

  • Nikolakopoulou, A., Mavridis, D., Furukawa, T. A., Cipriani, A., Tricco, A. C., Straus, S. E., et al. (2018). Living network meta-analysis compared with pairwise meta-analysis in comparative effectiveness research: Empirical study. BMJ, 360, k585.

  • Nikolakopoulou, A., Mavridis, D., & Salanti, G. (2014b). Demystifying fixed and random effects meta-analysis. Royal College of Psychiatrists.

  • Petropoulou, M., Nikolakopoulou, A., Veroniki, A. A., Rios, P., Vafaei, A., Zarin, W., et al. (2017). Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. Journal of Clinical Epidemiology, 82, 20–28.

    Article  Google Scholar 

  • Rücker, G. (2012). Network meta-analysis, electrical networks and graph theory. Research Synthesis Methods, 3, 312–324.

    Article  Google Scholar 

  • Rücker, G., & Schwarzer, G. (2015). Ranking treatments in frequentist network meta-analysis works without resampling methods. BMC Medical Research Methodology, 15, 58.

    Article  Google Scholar 

  • Rücker, G., Schwarzer, G., Krahn, U., König, J., & Schwarzer, M. G. (2015). Package ‘netmeta’. Network Meta-Analysis using Frequentist Methods (Version 0.7–0).

  • Salanti, G. (2012). Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: Many names, many benefits, many concerns for the next generation evidence synthesis tool. Research Synthesis Methods, 3, 80–97.

    Article  Google Scholar 

  • Salanti, G., Ades, A., & Ioannidis, J. P. (2011). Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: An overview and tutorial. Journal of Clinical Epidemiology, 64, 163–171.

    Article  Google Scholar 

  • Salanti, G., Higgins, J. P., Ades, A., & Ioannidis, J. P. (2008). Evaluation of networks of randomized trials. Statistical Methods in Medical Research, 17, 279–301.

    Article  Google Scholar 

  • Sarri, G., Patorno, E., Yuan, H., Guo, J., Bennett, D., Wen, X., et al. (2020). Framework for the synthesis of non-randomised studies and randomised controlled trials: A guidance on conducting a systematic review and meta-analysis for healthcare decision making. BMJ Evidence-Based Medicine, bmjebm-2020–111493, https://doi.org/10.1136/bmjebm-2020-111493

  • Schwingshackl, L., Dias, S., & Hoffmann, G. (2014). Impact of long-term lifestyle programmes on weight loss and cardiovascular risk factors in overweight/obese participants: A systematic review and network meta-analysis. Systematic Reviews, 3, 130.

    Article  Google Scholar 

  • Sharma, M., Singh, S., Desai, V., Shah, V. H., Kamath, P. S., Murad, M. H., et al. (2019). Comparison of therapies for primary prevention of esophageal variceal bleeding: A systematic review and network meta-analysis. Hepatology, 69, 1657–1675.

    Article  Google Scholar 

  • Simpson, E., Martyn-St James, M., Hamilton, J., Wong, R., Gittoes, N., Selby, P., et al. (2020). Clinical effectiveness of denosumab, raloxifene, romosozumab, and teriparatide for the prevention of osteoporotic fragility fractures: A systematic review and network meta-analysis. Bone, 130, 115081.

  • Thompson, S. G., & Higgins, J. P. (2002). How should meta-regression analyses be undertaken and interpreted? Statistics in Medicine, 21, 1559–1573.

    Article  Google Scholar 

  • Tricco, A. C., Thomas, S. M., Veroniki, A. A., Hamid, J. S., Cogo, E., Strifler, L., et al. (2017). Comparisons of interventions for preventing falls in older adults: A systematic review and meta-analysis. JAMA, 318, 1687–1699.

    Article  Google Scholar 

  • Turner, R. M., Davey, J., Clarke, M. J., Thompson, S. G., & Higgins, J. P. (2012). Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews. International Journal of Epidemiology, 41, 818–827. https://doi.org/10.1093/ije/dys041

    Article  PubMed  PubMed Central  Google Scholar 

  • Turner, R. M., Domínguez-Islas, C. P., Jackson, D., Rhodes, K. M., & White, I. R. (2019). Incorporating external evidence on between-trial heterogeneity in network meta-analysis. Statistics in Medicine, 38, 1321–1335.

    Article  Google Scholar 

  • Veroniki, A. A., Vasiliadis, H. S., Higgins, J. P., & Salanti, G. (2013). Evaluation of inconsistency in networks of interventions. International Journal of Epidemiology, 42, 332–345.

    Article  Google Scholar 

  • White, I. R. (2015). Network meta-analysis. The Stata Journal, 15, 951–985.

    Article  Google Scholar 

  • Wilson, D., Tanner-Smith, E., & Mavridis, D. (2016). Campbell methods policy note on network meta-analysis (Version 1.0, Updated September 2015). Oslo: The Campbell Collaboration.

  • Yamaoka, K., Nemoto, A., & Tango, T. (2019). Comparison of the effectiveness of lifestyle modification with other treatments on the incidence of type 2 diabetes in people at high risk: A network meta-analysis. Nutrients, 11, 1373.

    Article  CAS  Google Scholar 

  • Zhang, J., Yuan, Y., & Chu, H. (2016). The impact of excluding trials from network meta-analyses – an empirical study. PLoS One, 11, e0165889. https://doi.org/10.1371/journal.pone.0165889

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Primary Education, School of Education, University of Ioannina, Ioannina, Greece

    G Seitidis, S Nikolakopoulos & D Mavridis

  2. Department of Biostatistics, University Medical Centre Utrecht, Utrecht, Netherlands

    S Nikolakopoulos

  3. Recovery Research Institute, Center for Addiction Medicine, Massachusetts General Hospital & Harvard Medical School, Boston, MA, USA

    EA Hennessy

  4. Department of Counseling Psychology and Human Services, University of Oregon, Eugene, OR, USA

    EE Tanner-Smith

  5. Prevention Science Institute, University of Oregon, Eugene, OR, USA

    EE Tanner-Smith

  6. Faculté de Médecine, Université Paris Descartes, Paris, France

    D Mavridis

Authors
  1. G Seitidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S Nikolakopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. EA Hennessy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. EE Tanner-Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. D Mavridis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G Seitidis.

Ethics declarations

Ethics Approval

Ethical approval is not applicable, as this is an NMA tutorial.

Consent to Participate

Informed consent is not applicable, as this is an NMA tutorial.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 869 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Seitidis, G., Nikolakopoulos, S., Hennessy, E. et al. Network Meta-Analysis Techniques for Synthesizing Prevention Science Evidence. Prev Sci 23, 415–424 (2022). https://doi.org/10.1007/s11121-021-01289-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-021-01289-6

Keywords

  • Consistency
  • Network Meta-Analysis
  • Prevention science
  • Ranking
  • Transitivity
  • Tutorial