Skip to main content
Log in

Network vs. Pairwise Meta-Analyses: A Case Study of the Impact of an Evidence-Synthesis Paradigm on Value of Information Outcomes

  • Original Research Article
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

Abstract

Objective

To evaluate the impact of using two evidence-synthesis paradigms, pairwise meta-analysis (PMA) vs. network meta-analysis (NMA), on the expected value of information (EVI) outcomes, using pharmacotherapy of chronic obstructive pulmonary disease as a case study.

Methods

Bayesian random-effects PMAs were performed for each pharmacotherapy vs. placebo, and a Bayesian random-effects NMA was performed combining both placebo-controlled and head-to-head trials. Both provided comparative rate ratio (RR) estimates between each pharmacotherapy vs. placebo. A Markov model was developed to project costs and quality-adjusted life-years of five commonly used treatments for chronic obsructive pulmonary disorder. RRs for the treatment effect compared with placebo derived using PMA and NMA were used alongside values from the literature to populate the model. In addition to standard cost-effectiveness outputs, we calculated and compared the expected value of perfect information (EVPI) and the expected value of partial perfect information (EVPPI) for treatment effects, for comparisons that included all or a subset of treatments.

Results

The network of evidence included five different treatments, compared in 19 randomized controlled trials (RCTs), which in total included 28,172 individuals. The cost-effectiveness outcomes were similar between the two evidence-synthesis paradigms. The individual EVPI for all treatments was Can$1,262 for PMA-based analyses and Can$572 for NMA-based analyses. For all comparisons involving two, three, or four treatments, the comparison with the highest EVPI was different between the two methods. Similarly, the choice of PMA or NMA had resulted in substantially different EVPPI rankings.

Conclusion

Our case study shows that the choice of PMA or NMA can have significant effects on the EVI results. Under comparable conditions, the incorporation of more evidence in the NMA most likely increases the precision of estimates and therefore is likely to result in lower EVI outcomes. As our study demonstrates, the difference in EVI outcomes can be substantial, potentially affecting the decision to conduct research and the design of future research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Drummond M, O’Brien B, Stoddart G, Torrance G. Methods for the economic evaluation of health care programmes. UK: Oxford University Press; 2005.

    Google Scholar 

  2. Claxton K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. J Health Econ. 1999;18(3):341–64.

    Article  PubMed  CAS  Google Scholar 

  3. Eckermann S, Willan AR. Expected value of information and decision making in HTA. Health Econ. 2007;16(2):195–209.

    Article  PubMed  Google Scholar 

  4. Sculpher M, Claxton K, Drummond M, McCabe C. Whether trial-based economic evaluation for health care decision making? Health Econ. 2006;15(7):677–87.

    Article  PubMed  Google Scholar 

  5. Buxton MJ, Drummond MF, Van Hout BA, Prince RL, Sheldon TA, Szucs T, et al. Modelling in economic evaluation: an unavoidable fact of life. Health Econ. 1997;6(3):217–27.

    Article  PubMed  CAS  Google Scholar 

  6. Caldwell DM, Ades AE, Higgins JPT. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ. 2005;331(7521):897–900.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Thorlund K, Mills EJ. Sample size and power considerations in network meta-analysis. Syst Rev. 2012;1(1):41.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Dias S, Sutton AJ, Ades AE, Welton NJ. Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials. Med Decis Mak. 2013;33(5):607–17.

    Article  Google Scholar 

  9. Cooper NJ, Peters J, Lai MCW, Juni P, Wandel S, Palmer S, et al. How valuable are multiple treatment comparison methods in evidence-based health-care evaluation? Value Health. 2011;14(2):371–80.

    Article  PubMed  Google Scholar 

  10. Thorlund K, Zafari Z, Druyts E, Mills EJ, Sadatsafavi M. The impact of incorporating Bayesian network meta-analysis in cost-effectiveness analysis—a case study of pharmacotherapies for moderate to severe COPD. Cost Eff Resour Alloc. 2014;12(1):8.

  11. Mills EJ, Druyts E, Ghement I, Puhan MA. Pharmacotherapies for chronic obstructive pulmonary disease: a multiple treatment comparison meta-analysis. Clin Epidemiol. 2011;28(3):107–29.

    Article  Google Scholar 

  12. Lunn D, Thomas A, Best N, Spiegelhalter D. WinBUGS: a Bayesian modelling framework: concepts, structure, and extensibility. Stat Comput. 2000;10:325–37.

    Article  Google Scholar 

  13. Buist AS, Vollmer WM, Sullivan SD, Weiss KB, Lee TA, Menezes AM, et al. The burden of obstructive lung disease initiative (BOLD): rationale and design. COPD. 2005;2(2):277–83.

    Article  PubMed  Google Scholar 

  14. Global Initiative for Chronic Obstructive Lung Disease (GOLD). Global strategy for diagnosis, management, and prevention of COPD [Internet]. [cited 2011 Dec]. http://www.goldcopd.org/uploads/users/files/GOLD_Report_2011_Feb21.pdf.

  15. R Development Core Team. R: A language and environment for statistical computing [Internet]. Vienna, Austria: R Foundation for Statistical Computing; 2011 [cited 2009 Nov 16]. http://www.r-project.org/.

  16. Barton GR, Briggs AH, Fenwick EAL. Optimal cost-effectiveness decisions: the role of the cost-effectiveness acceptability curve (CEAC), the cost-effectiveness acceptability frontier (CEAF), and the expected value of perfection information (EVPI). Value Health. 2008;11(5):886–97.

    Article  PubMed  Google Scholar 

  17. Brennan A, Kharroubi S, O’Hagan A, Chilcott J. Calculating partial expected value of perfect information via Monte Carlo sampling algorithms. Med Decis Mak. 2007;27(4):448–70.

    Article  Google Scholar 

  18. Jalal H, Dowd B, Sainfort F, Kuntz KM. Linear regression metamodeling as a tool to summarize and present simulation model results. Med Decis Mak. 2013;33(7):880–90.

    Article  Google Scholar 

  19. Suissa S, Dell’Aniello S, Ernst P. Long-term natural history of chronic obstructive pulmonary disease: severe exacerbations and mortality. Thorax. 2012;67(11):957–63.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Mills EJ, Ioannidis JPA, Thorlund K, Schünemann HJ, Puhan MA, Guyatt GH. How to use an article reporting a multiple treatment comparison meta-analysis. JAMA. 2012;308(12):1246–53.

    Article  PubMed  CAS  Google Scholar 

  21. Jansen JP, Fleurence R, Devine B, Itzler R, Barrett A, Hawkins N, et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value Health. 2011;14(4):417–28.

    Article  PubMed  Google Scholar 

  22. Ades A, Claxton K, Sculpher M. Evidence synthesis, parameter correlation and probabilistic sensitivity analysis. Health Econ. 2006;15(4):373–81.

    Article  PubMed  CAS  Google Scholar 

  23. Cooper N, Sutton A, Abrams K, Turner D, Wailoo A. Comprehensive decision analytical modelling in economic evaluation: a Bayesian approach. Health Econ. 2004;13(3):203–26.

    Article  PubMed  Google Scholar 

  24. Comparing multiple interventions methods group, a network meta-analysis toolkit, [Internet]. [cited 2014 May 16]. http://cmimg.cochrane.org/network-meta-analysis-toolkit.

  25. Hoogendoorn M, Rutten-van Mölken MPMH, Hoogenveen RT, van Genugten MLL, Buist AS, Wouters EFM, et al. A dynamic population model of disease progression in COPD. Eur Respir J. 2005;26(2):223–33.

    Article  PubMed  CAS  Google Scholar 

  26. Spencer M, Briggs A, Grossman R, Rance L. Development of an economic model to assess the cost effectiveness of treatment interventions for chronic obstructive pulmonary disease. Pharmacoeconomics. 2005;23(6):619–37.

    Article  PubMed  CAS  Google Scholar 

  27. Weiss KB, Sullivan SD, Lee TA. BOLD economic burden model [Internet]. http://www.boldstudy.org.

  28. Najafzadeh M, Marra CA, Lynd LD, Sadatsafavi M, FitzGerald JM, McManus B, et al. Future impact of various interventions on the burden of COPD in Canada: a dynamic population model. PLoS One. 2012;7(10):e46746.

    Article  PubMed  CAS  PubMed Central  Google Scholar 

  29. Chapman KR, Bourbeau J, Rance L. The burden of COPD in Canada: results from the Confronting COPD survey. Respir Med. 2003;97(Suppl C):S23–31.

    Article  PubMed  Google Scholar 

  30. MIMS. MIMS: Prescription drug database and drug prescription guide. 2012.

Download references

Acknowledgments

ZZ, KT, and MS developed the research idea. ZZ performed the literature review and created the decision-analytic model. KT performed the network meta-analysis. ZZ and MS wrote the first draft of the manuscript. MS and KT provided methodological support. JMF and CM provided clinical insight. All authors revised the manuscript and approved the final version. Mohsen Sadatsafavi is the guarantor of this work.

Zafar Zafari has declared no conflicts of interest. Kristian Thorlund has declared no conflicts of interest. J Mark Fitzgerald has declared no conflicts of interest. Carlo Marra has declared no conflicts of interest. Mohsen Sadatsafavi has declared no conflicts of interest.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mohsen Sadatsafavi.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zafari, Z., Thorlund, K., FitzGerald, J.M. et al. Network vs. Pairwise Meta-Analyses: A Case Study of the Impact of an Evidence-Synthesis Paradigm on Value of Information Outcomes. PharmacoEconomics 32, 995–1004 (2014). https://doi.org/10.1007/s40273-014-0179-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-014-0179-1

Keywords

Navigation