, Volume 26, Issue 9, pp 753–767 | Cite as

Use of Indirect and Mixed Treatment Comparisons for Technology Assessment

  • Alex SuttonEmail author
  • A. E. Ades
  • Nicola Cooper
  • Keith Abrams
Briefing Paper


Indirect and mixed treatment comparison (MTC) approaches to synthesis are logical extensions of more established meta-analysis methods. They have great potential for estimating the comparative effectiveness of multiple treatments using an evidence base of trials that individually do not compare all treatment options. Connected networks of evidence can be synthesized simultaneously to provide estimates of the comparative effectiveness of all included treatments and a ranking of their effectiveness with associated probability statements.

The potential of the use of indirect and MTC methods in technology assessment is considerable, and would allow for a more consistent assessment than simpler alternative approaches. Although such models can be viewed as a logical and coherent extension of standard pair-wise meta-analysis, their increased complexity raises some unique issues with far-reaching implications concerning how we use data in technology assessment, while simultaneously raising searching questions about standard pair-wise meta-analysis. This article reviews pair-wise meta-analysis and indirect and MTC approaches to synthesis, clearly outlining the assumptions involved in each approach. It also raises the issues that the National Institute for Health and Clinical Excellence (NICE) needed to consider in updating their 2004 Guide to the Methods of Technology Appraisal, if such methods are to be used in their technology appraisals.


Technology Assessment Indirect Comparison Decision Context Technology Appraisal Mixed Treatment Comparison 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This paper was initially prepared as a briefing paper for NICE as part of the process of updating the Institute’s 2004 Guide to the Methods of Technology Appraisal. The work was funded by NICE through its Decision Support Unit, which is based at the universities of Sheffield, Leicester, York, Leeds and at the London School of Hygiene and Tropical Medicine.

K.R. Abrams, A.E. Ades, N.J. Cooper and A.J. Sutton have all delivered fee-paying courses on indirect and mixed treatment comparisons (MTC), and have had research projects developing and using MTC methods funded by the Medical Research Council (MRC), NHS Health Technology Appraisal and the healthcare industry. A.J. Sutton is an applicant on an MRC grant investigating the validity of indirect comparisons. K.R. Abrams, A.E. Ades, N.J. Cooper and A.J. Sutton are all applicants on grant applications using MTC methods. K.R. Abrams and A.E. Ades have also acted as paid consultants to consultancy companies in the healthcare industry, specifically on MTC methods. K.R. Abrams also receives royalities for Bayesian Approaches to Clinical Trials and Health-care Evaluation.

The authors would like to thank Louise Longworth (NICE) for input into the original briefing and for insightful comments on an earlier draft of this paper; Karl Claxton for his insightful comments on an earlier draft of the paper; and Deborah Caldwell for allowing us to reproduce tables I, III and IV for the paper. Although this paper has its roots in the document prepared for a workshop on MTCs hosted by NICE as part of the process of updating their 2004 guidance, it has evolved considerably since then and includes numerous substantive changes.


  1. 1.
    Hierarchy of evidence and grading of recommendations. Thorax 2004; 59 (Suppl. 1): 13Google Scholar
  2. 2.
    Egger M, Davey Smith G, Altman DG. Systematic reviews in health care: meta-analysis in context. London: BMJ Books, 2000Google Scholar
  3. 3.
    Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions, 4.2.5 [updated May 2005]. In: The Cochrane Library. Issue 3. Chichester: John Wiley & Sons, Ltd, 2005Google Scholar
  4. 4.
    Higgins JPT, Whitehead A. Borrowing strength from external trials in a meta-analysis. Stat Med 1996; 15: 2733–2749PubMedCrossRefGoogle Scholar
  5. 5.
    Lumley T. Network meta-analysis for indirect treatment comparisons. Stat Med 2002; 21: 2313–2324PubMedCrossRefGoogle Scholar
  6. 6.
    Hasselblad V. Meta-analysis of multitreatment studies. Med Decis Making 1998: 18: 37–43PubMedCrossRefGoogle Scholar
  7. 7.
    Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004; 23: 3105–3124PubMedCrossRefGoogle Scholar
  8. 8.
    Caldwell DM, Ades AE, Higgins JPT. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ 2005; 331: 897–900PubMedCrossRefGoogle Scholar
  9. 9.
    Glenny AM, Altman DG, Song F. Indirect comparisons of competing interventions. Health Technol Assess 2005; 9 (26): 1–148PubMedGoogle Scholar
  10. 10.
    National Institute for Health and Clinical Excellence (NICE). Guide to the methods of technology appraisal. London: NICE 2004Google Scholar
  11. 11.
    Salanti G, Higgins JPT, Ades AE, et al. Evaluation of networks of randomized trials. Stat Methods Med Res 2007; 17: 279–301PubMedCrossRefGoogle Scholar
  12. 12.
    Sutton AJ, Abrams KR, Jones DR, et al. Methods for meta-analysis in medical research. London: John Wiley, 2000Google Scholar
  13. 13.
    Furberg CT, Morgan TM. Lessons from overviews of cardiovascular trials. Stat Med 1987; 6: 295–303PubMedCrossRefGoogle Scholar
  14. 14.
    Gillies CL, Abrams KR, Lambert PC, et al. Pharmacological and lifestyle interventions to prevent or delay type 2 diabetes in people with impaired glucose tolerance: systematic review and meta-analysis. BMJ 2007; 334: 299–302PubMedCrossRefGoogle Scholar
  15. 15.
    Stewart LA, Clarke MJ. Practical methodology of meta-analyses (overviews) using updated individual patient data. Cochrane Working Group. Stat Med 1995; 14: 2057–2079PubMedCrossRefGoogle Scholar
  16. 16.
    Sculpher M. Subgroups and heterogeneity in cost-effectiveness analysis. Pharmacoeconomics 2008; 26 (9): 799–806PubMedCrossRefGoogle Scholar
  17. 17.
    Deeks JJ. Issues in the selection of a summary statistic for meta-analysis of clinical trials with binary outcomes. Stat Med 2002; 21: 1575–1600PubMedCrossRefGoogle Scholar
  18. 18.
    DerSimonian R, Kacker R. Random-effects model for meta-analysis of clinical trials: an update. Contemp Clin Trials 2007; 28: 105–114PubMedCrossRefGoogle Scholar
  19. 19.
    Greenland S. Can meta-analysis be salvaged? Am J Epidemiol 1994; 140: 783–787PubMedGoogle Scholar
  20. 20.
    Ades AE, Lu G, Higgins JPT. The interpretation of random-effects meta-analysis in decision models. Med Decis Making 2005; 25: 646–654PubMedCrossRefGoogle Scholar
  21. 21.
    Welton NJ, White JR, Lu G, et al. Correction: interpretation of random effects meta-analysis in decision models. Med Decis Making 2007; 27: 212–214PubMedCrossRefGoogle Scholar
  22. 22.
    Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med 1999; 18: 2693–2708PubMedCrossRefGoogle Scholar
  23. 23.
    Lambert P, Sutton AJ, Abrams KR, et al. A comparison of summary patient level covariates in meta-regression with individual patient data meta-analyses. J Clin Epidemiol 2002; 55: 86–94PubMedCrossRefGoogle Scholar
  24. 24.
    Turner RM, Spiegelhalter DJ, Smith GCS, et al. Bias modelling in evidence synthesis. J R Stat Soc Ser A Stat Soc. In pressGoogle Scholar
  25. 25.
    Bucher HC, Guyatt GH, Griffith LE, et al. The results of direct and indirect treatment comparisons in meta-analysis of randomised controlled trials. J Clin Epidemiol 1997; 50: 683–691PubMedCrossRefGoogle Scholar
  26. 26.
    Barrio V. Actual methodological controversies on the controlled clinical trials and on meta-analysis. Nefrologia 1998; 18: 32–39Google Scholar
  27. 27.
    Song F, Harvey I, Lilford R. Adjusted indirect comparison may be less biased than direct comparison for evaluating new pharmaceutical interventions. J Clin Epidemiol 2008 May; 61 (5); 455–463PubMedCrossRefGoogle Scholar
  28. 28.
    Lu G, Ades AE. Assessing evidence inconsistency in mixed treatment comparisons. J Am Stat Assoc 2006; 101: 447–459CrossRefGoogle Scholar
  29. 29.
    Song F, Altman DG, Glenny M-A, et al. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ 2003; 326: 472–476PubMedCrossRefGoogle Scholar
  30. 30.
    Spiegelhalter DJ, Thomas A, Best NG. WinBUGS version 1.2 user manual. Cambridge (UK): MRC Biostatistics Unit, 1999Google Scholar
  31. 31.
    Ades AE, Sculpher M, Sutton A, et al. Bayesian methods for evidence synthesis in cost-effectiveness analysis. Pharmacoeconomics 2006; 24 (1): 1–19PubMedCrossRefGoogle Scholar
  32. 32.
    Efron B, Tibshirani RJ. An introduction to the bootstrap. 1st ed. New York: Chapman & Hall, 1993Google Scholar
  33. 33.
    Li ZH, Begg CB. Random effects models for combining results from controlled and uncontrolled studies in a metaanalysis. J Am Stat Assoc 1994; 89: 1523–1527CrossRefGoogle Scholar
  34. 34.
    Lu G, Ades AE, Sutton AJ, et al. Meta-analysis of multiple treatment comparisons at multiple follow-up times. Stat Med 2007; 20: 3681–3699CrossRefGoogle Scholar
  35. 35.
    Nixon RM, Bansback N, Brennan A. Using mixed treatment comparisons and meta-regression to perform indirect comparisons to estimate the efficacy of biologic treatments in rheumatoid arthritis. Stat Med 2007; 26: 1237–1254PubMedCrossRefGoogle Scholar
  36. 36.
    Welton N, Cooper NJ, Ades A, et al. Mixed treatment comparison with multiple outcomes reported inconsistently across trials: evaluation of antivirals for treatment of influenza A and B. Stat Med. In pressGoogle Scholar
  37. 37.
    Moher D, Cook DJ, Eastwood S, et al. Improving the quality of reporting of meta-analysis of randomised controlled trials: the QUOROM statement. Lancet 1999; 354: 1896–1900PubMedCrossRefGoogle Scholar
  38. 38.
    Cooper NJ, Sutton AJ, Lu G, et al. Mixed comparison of stroke prevention treatments in individuals with non-rheumatic atrial fibrillation. Arch Intern Med 2006; 166 (12): 1269–1275PubMedCrossRefGoogle Scholar

Copyright information

© Adis Data Information BV 2008

Authors and Affiliations

  • Alex Sutton
    • 1
    Email author
  • A. E. Ades
    • 2
  • Nicola Cooper
    • 1
  • Keith Abrams
    • 1
  1. 1.Department of Health Sciences and NICE Decision Support UnitUniversity of LeicesterLeicesterUK
  2. 2.Department of Community Based MedicineUniversity of BristolBristolUK

Personalised recommendations