Use of Indirect and Mixed Treatment Comparisons for Technology Assessment


Indirect and mixed treatment comparison (MTC) approaches to synthesis are logical extensions of more established meta-analysis methods. They have great potential for estimating the comparative effectiveness of multiple treatments using an evidence base of trials that individually do not compare all treatment options. Connected networks of evidence can be synthesized simultaneously to provide estimates of the comparative effectiveness of all included treatments and a ranking of their effectiveness with associated probability statements.

The potential of the use of indirect and MTC methods in technology assessment is considerable, and would allow for a more consistent assessment than simpler alternative approaches. Although such models can be viewed as a logical and coherent extension of standard pair-wise meta-analysis, their increased complexity raises some unique issues with far-reaching implications concerning how we use data in technology assessment, while simultaneously raising searching questions about standard pair-wise meta-analysis. This article reviews pair-wise meta-analysis and indirect and MTC approaches to synthesis, clearly outlining the assumptions involved in each approach. It also raises the issues that the National Institute for Health and Clinical Excellence (NICE) needed to consider in updating their 2004 Guide to the Methods of Technology Appraisal, if such methods are to be used in their technology appraisals.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Table I
Table II
Table III
Fig. 3
Table IV


  1. 1.

    Hierarchy of evidence and grading of recommendations. Thorax 2004; 59 (Suppl. 1): 13

  2. 2.

    Egger M, Davey Smith G, Altman DG. Systematic reviews in health care: meta-analysis in context. London: BMJ Books, 2000

    Google Scholar 

  3. 3.

    Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions, 4.2.5 [updated May 2005]. In: The Cochrane Library. Issue 3. Chichester: John Wiley & Sons, Ltd, 2005

  4. 4.

    Higgins JPT, Whitehead A. Borrowing strength from external trials in a meta-analysis. Stat Med 1996; 15: 2733–2749

    PubMed  Article  CAS  Google Scholar 

  5. 5.

    Lumley T. Network meta-analysis for indirect treatment comparisons. Stat Med 2002; 21: 2313–2324

    PubMed  Article  Google Scholar 

  6. 6.

    Hasselblad V. Meta-analysis of multitreatment studies. Med Decis Making 1998: 18: 37–43

    PubMed  Article  CAS  Google Scholar 

  7. 7.

    Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004; 23: 3105–3124

    PubMed  Article  CAS  Google Scholar 

  8. 8.

    Caldwell DM, Ades AE, Higgins JPT. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ 2005; 331: 897–900

    PubMed  Article  Google Scholar 

  9. 9.

    Glenny AM, Altman DG, Song F. Indirect comparisons of competing interventions. Health Technol Assess 2005; 9 (26): 1–148

    PubMed  CAS  Google Scholar 

  10. 10.

    National Institute for Health and Clinical Excellence (NICE). Guide to the methods of technology appraisal. London: NICE 2004

    Google Scholar 

  11. 11.

    Salanti G, Higgins JPT, Ades AE, et al. Evaluation of networks of randomized trials. Stat Methods Med Res 2007; 17: 279–301

    PubMed  Article  Google Scholar 

  12. 12.

    Sutton AJ, Abrams KR, Jones DR, et al. Methods for meta-analysis in medical research. London: John Wiley, 2000

    Google Scholar 

  13. 13.

    Furberg CT, Morgan TM. Lessons from overviews of cardiovascular trials. Stat Med 1987; 6: 295–303

    PubMed  Article  CAS  Google Scholar 

  14. 14.

    Gillies CL, Abrams KR, Lambert PC, et al. Pharmacological and lifestyle interventions to prevent or delay type 2 diabetes in people with impaired glucose tolerance: systematic review and meta-analysis. BMJ 2007; 334: 299–302

    PubMed  Article  Google Scholar 

  15. 15.

    Stewart LA, Clarke MJ. Practical methodology of meta-analyses (overviews) using updated individual patient data. Cochrane Working Group. Stat Med 1995; 14: 2057–2079

    PubMed  Article  CAS  Google Scholar 

  16. 16.

    Sculpher M. Subgroups and heterogeneity in cost-effectiveness analysis. Pharmacoeconomics 2008; 26 (9): 799–806

    PubMed  Article  Google Scholar 

  17. 17.

    Deeks JJ. Issues in the selection of a summary statistic for meta-analysis of clinical trials with binary outcomes. Stat Med 2002; 21: 1575–1600

    PubMed  Article  Google Scholar 

  18. 18.

    DerSimonian R, Kacker R. Random-effects model for meta-analysis of clinical trials: an update. Contemp Clin Trials 2007; 28: 105–114

    PubMed  Article  Google Scholar 

  19. 19.

    Greenland S. Can meta-analysis be salvaged? Am J Epidemiol 1994; 140: 783–787

    PubMed  CAS  Google Scholar 

  20. 20.

    Ades AE, Lu G, Higgins JPT. The interpretation of random-effects meta-analysis in decision models. Med Decis Making 2005; 25: 646–654

    PubMed  Article  CAS  Google Scholar 

  21. 21.

    Welton NJ, White JR, Lu G, et al. Correction: interpretation of random effects meta-analysis in decision models. Med Decis Making 2007; 27: 212–214

    PubMed  Article  CAS  Google Scholar 

  22. 22.

    Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med 1999; 18: 2693–2708

    PubMed  Article  CAS  Google Scholar 

  23. 23.

    Lambert P, Sutton AJ, Abrams KR, et al. A comparison of summary patient level covariates in meta-regression with individual patient data meta-analyses. J Clin Epidemiol 2002; 55: 86–94

    PubMed  Article  CAS  Google Scholar 

  24. 24.

    Turner RM, Spiegelhalter DJ, Smith GCS, et al. Bias modelling in evidence synthesis. J R Stat Soc Ser A Stat Soc. In press

  25. 25.

    Bucher HC, Guyatt GH, Griffith LE, et al. The results of direct and indirect treatment comparisons in meta-analysis of randomised controlled trials. J Clin Epidemiol 1997; 50: 683–691

    PubMed  Article  CAS  Google Scholar 

  26. 26.

    Barrio V. Actual methodological controversies on the controlled clinical trials and on meta-analysis. Nefrologia 1998; 18: 32–39

    Google Scholar 

  27. 27.

    Song F, Harvey I, Lilford R. Adjusted indirect comparison may be less biased than direct comparison for evaluating new pharmaceutical interventions. J Clin Epidemiol 2008 May; 61 (5); 455–463

    PubMed  Article  CAS  Google Scholar 

  28. 28.

    Lu G, Ades AE. Assessing evidence inconsistency in mixed treatment comparisons. J Am Stat Assoc 2006; 101: 447–459

    Article  CAS  Google Scholar 

  29. 29.

    Song F, Altman DG, Glenny M-A, et al. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ 2003; 326: 472–476

    PubMed  Article  Google Scholar 

  30. 30.

    Spiegelhalter DJ, Thomas A, Best NG. WinBUGS version 1.2 user manual. Cambridge (UK): MRC Biostatistics Unit, 1999

    Google Scholar 

  31. 31.

    Ades AE, Sculpher M, Sutton A, et al. Bayesian methods for evidence synthesis in cost-effectiveness analysis. Pharmacoeconomics 2006; 24 (1): 1–19

    PubMed  Article  CAS  Google Scholar 

  32. 32.

    Efron B, Tibshirani RJ. An introduction to the bootstrap. 1st ed. New York: Chapman & Hall, 1993

  33. 33.

    Li ZH, Begg CB. Random effects models for combining results from controlled and uncontrolled studies in a metaanalysis. J Am Stat Assoc 1994; 89: 1523–1527

    Article  Google Scholar 

  34. 34.

    Lu G, Ades AE, Sutton AJ, et al. Meta-analysis of multiple treatment comparisons at multiple follow-up times. Stat Med 2007; 20: 3681–3699

    Article  Google Scholar 

  35. 35.

    Nixon RM, Bansback N, Brennan A. Using mixed treatment comparisons and meta-regression to perform indirect comparisons to estimate the efficacy of biologic treatments in rheumatoid arthritis. Stat Med 2007; 26: 1237–1254

    PubMed  Article  CAS  Google Scholar 

  36. 36.

    Welton N, Cooper NJ, Ades A, et al. Mixed treatment comparison with multiple outcomes reported inconsistently across trials: evaluation of antivirals for treatment of influenza A and B. Stat Med. In press

  37. 37.

    Moher D, Cook DJ, Eastwood S, et al. Improving the quality of reporting of meta-analysis of randomised controlled trials: the QUOROM statement. Lancet 1999; 354: 1896–1900

    PubMed  Article  CAS  Google Scholar 

  38. 38.

    Cooper NJ, Sutton AJ, Lu G, et al. Mixed comparison of stroke prevention treatments in individuals with non-rheumatic atrial fibrillation. Arch Intern Med 2006; 166 (12): 1269–1275

    PubMed  Article  Google Scholar 

Download references


This paper was initially prepared as a briefing paper for NICE as part of the process of updating the Institute’s 2004 Guide to the Methods of Technology Appraisal. The work was funded by NICE through its Decision Support Unit, which is based at the universities of Sheffield, Leicester, York, Leeds and at the London School of Hygiene and Tropical Medicine.

K.R. Abrams, A.E. Ades, N.J. Cooper and A.J. Sutton have all delivered fee-paying courses on indirect and mixed treatment comparisons (MTC), and have had research projects developing and using MTC methods funded by the Medical Research Council (MRC), NHS Health Technology Appraisal and the healthcare industry. A.J. Sutton is an applicant on an MRC grant investigating the validity of indirect comparisons. K.R. Abrams, A.E. Ades, N.J. Cooper and A.J. Sutton are all applicants on grant applications using MTC methods. K.R. Abrams and A.E. Ades have also acted as paid consultants to consultancy companies in the healthcare industry, specifically on MTC methods. K.R. Abrams also receives royalities for Bayesian Approaches to Clinical Trials and Health-care Evaluation.

The authors would like to thank Louise Longworth (NICE) for input into the original briefing and for insightful comments on an earlier draft of this paper; Karl Claxton for his insightful comments on an earlier draft of the paper; and Deborah Caldwell for allowing us to reproduce tables I, III and IV for the paper. Although this paper has its roots in the document prepared for a workshop on MTCs hosted by NICE as part of the process of updating their 2004 guidance, it has evolved considerably since then and includes numerous substantive changes.

Author information



Corresponding author

Correspondence to Dr Alex Sutton.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Sutton, A., Ades, A.E., Cooper, N. et al. Use of Indirect and Mixed Treatment Comparisons for Technology Assessment. Pharmacoeconomics 26, 753–767 (2008).

Download citation


  • Technology Assessment
  • Indirect Comparison
  • Decision Context
  • Technology Appraisal
  • Mixed Treatment Comparison