, Volume 24, Issue 1, pp 1–19 | Cite as

Bayesian methods for evidence synthesis in cost-effectiveness analysis

  • A. E. Ades
  • Mark SculpherEmail author
  • Alex Sutton
  • Keith Abrams
  • Nicola Cooper
  • Nicky Welton
  • Guobing Lu
Leading Article


Recently, health systems internationally have begun to use cost-effectiveness research as formal inputs into decisions about which interventions and programmes should be funded from collective resources. This process has raised some important methodological questions for this area of research. This paper considers one set of issues related to the synthesis of effectiveness evidence for use in decision-analytic cost-effectiveness (CE) models, namely the need for the synthesis of all sources of available evidence, although these may not ‘fit neatly’ into a CE model.

Commonly encountered problems include the absence of head-to-head trial evidence comparing all options under comparison, the presence of multiple endpoints from trials and different follow-up periods. Full evidence synthesis for CE analysis also needs to consider treatment effects between patient subpopulations and the use of nonrandomised evidence.

Bayesian statistical methods represent a valuable set of analytical tools to utilise indirect evidence and can make a powerful contribution to the decision-analytic approach to CE analysis. This paper provides a worked example and a general overview of these methods with particular emphasis on their use in economic evaluation.


Markov Chain Monte Carlo Pelvic Inflammatory Disease Evidence Synthesis Relative Treatment Effect Mixed Treatment Comparison 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Tony Ades and Mark Sculpher receive funding from the UK Medical Research Council as part of the Health Services Research Collaboration. Mark Sculpher is also funded through a Public Health Career Scientist Award from the UK NHS Research and Development Programme.

The authors have no conflicts of interest.


  1. 1.
    Hjelmgren J, Berggren F, Andersson F. Health economic guidelines: similarities, differences and some implications. Value Health 2001; 4: 225–50PubMedCrossRefGoogle Scholar
  2. 2.
    Sullivan S, Lyles A, Luce B, et al. AMCP guidance for submission of clinical and economic evaluation data to support formulary listing in US health plans and pharmacy benefits management organisations. J Manag Care Pharm 2001; 7: 272–82Google Scholar
  3. 3.
    National Institute for Clinical Excellence (NICE). Guide to the methods of technology appraisal. London: NICE, 2004Google Scholar
  4. 4.
    National Institute for Clinical Excellence (NICE). Guide to technology appraisal process. London: NICE, 2004Google Scholar
  5. 5.
    Drummond MF, Davies L. Economic analysis alongside clinical trials. Int J Technol Assess Health Care 1991; 7: 561–73PubMedCrossRefGoogle Scholar
  6. 6.
    Claxton K, Sculpher M, Drummond M. A rational framework for decision making by the National Institute for Clinical Excellence. Lancet 2002; 360: 711–5PubMedCrossRefGoogle Scholar
  7. 7.
    Barton P, Jobanputra P, Wilson J, et al. The use of modelling to evaluate new drugs for patients with a chronic condition: the case of antibodies against tumour necrosis factor in rheumatoid arthritis. Health Technol Assess 2004; 8 (11): 1–91Google Scholar
  8. 8.
    Briggs AH, Goeree R, Blackhouse G, et al. Probabilistic analysis of cost-effectiveness models: choosing between treatment strategies for gastroesophageal reflux disease. Med Decis Making 2002; 22: 290–308PubMedGoogle Scholar
  9. 9.
    Spiegelhalter DJ, Abrams KR, Myles JP. Bayesian approaches to clinical trials and health-care evaluation. Chichester: Wiley, 2004Google Scholar
  10. 10.
    Fryback DG, Chinnis JO, Ulviva JW. Bayesian cost-effectiveness analysis: an example using the GUSTO trial. Int J Technol Assess Health Care 2001; 17: 83–97PubMedCrossRefGoogle Scholar
  11. 11.
    O’Hagan A, Luce B. A primer on Bayesian statistics in health economics and outcomes research. Bethesda (MD): Medtap International, 2003Google Scholar
  12. 12.
    Felli JC, Hazen G. A Bayesian approach to sensitivity analysis. Health Econ 1999; 8: 263–8PubMedCrossRefGoogle Scholar
  13. 13.
    Sutton AJ, Abrams KR. Bayesian methods in meta-analysis and evidence synthesis. Stat Methods Med Res 2001; 10: 277–303PubMedCrossRefGoogle Scholar
  14. 14.
    Cooper NJ, Sutton AJ, Abrams KR. Decision analytical economic modelling within a Bayesian framework: application to prophylactic antibiotics use for caesarean section. Stat Methods Med Res 2002; 11: 491–512PubMedCrossRefGoogle Scholar
  15. 15.
    Cooper NJ, Sutton AJ, Abrams KR, et al. Comprehensive decision analytical modelling in economic evaluation: a Bayesian approach. Health Econ 2004; 13: 203–26PubMedCrossRefGoogle Scholar
  16. 16.
    Parmigiani G. Modeling in medical decision making: a Bayesian approach. Chichester: Wiley, 2002Google Scholar
  17. 17.
    Spiegelhalter DJ, Thomas A, Best N, et al. WinBUGS user manual: version 1.4. Cambridge (UK): MRC Biostatistics Unit, 2001Google Scholar
  18. 18.
    Doubilet P, Begg CB, Weinstein MC, et al. Probabilistic sensitivity analysis using Monte Carlo simulation. Med Decis Making 1985; 5: 157–77PubMedCrossRefGoogle Scholar
  19. 19.
    Critchfield GC, Willard KE, Connelly DP. Probabilistic analysis of decision trees using Monte Carlo simulation. Med Decis Making 1986; 6: 85–92PubMedCrossRefGoogle Scholar
  20. 20.
    Bailey KR. Inter-study differences: how should they influence the interpretation and analysis of results. Stat Med 1987; 6: 351–60PubMedCrossRefGoogle Scholar
  21. 21.
    Lambert PC, Sutton AJ, Abrams KR, et al. A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. J Clin Epidemiol 2002; 55: 86–94PubMedCrossRefGoogle Scholar
  22. 22.
    Shadish WR, Haddock CK. Combining estimates of effect size. In: Cooper H, Hedges LV, editors. The handbook of research synthesis. New York: Russell Sage Foundation, 1994: 261–81Google Scholar
  23. 23.
    DerSimonian R, Laird N. Meta-analysis of clinical trials. Control Clin Trials 1986; 7: 177–88PubMedCrossRefGoogle Scholar
  24. 24.
    Prevost TC, Abrams KR, Jones DR. Hierarchical models in generalised synthesis of evidence: an example based on studies of breast cancer screening. Stat Med 2000; 19: 3359–76PubMedCrossRefGoogle Scholar
  25. 25.
    Mandelblatt JS, Fryback DG, Weinstein MC, et al. Assessing the effectiveness of health interventions. In: Gold MR, Siegel JE, Russell LB, et al., editors. Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996: 135–75Google Scholar
  26. 26.
    van Valkengoed IGM, Morré SA, van den Brute AJC, et al. Over-estimation of complication rates of chlamydia trachomatis screening programmes: implications for cost-effectiveness analyses. Int J Epidemiol 2004; 33: 416–25PubMedCrossRefGoogle Scholar
  27. 27.
    Eddy DM, Hasselblad V, Shachter R. Analysis of breast cancer screening: seven controlled studies. In: Eddy DM, Hasselblad V, Shachter R, editors. Meta-analysis by the confidence profile method. Boston: Academic Press, 1982: 223–9Google Scholar
  28. 28.
    Hasselblad V, McCrory DC. Meta-analytic tools for medical decision making: a practical guide. Med Decis Making 1995; 15: 81–96PubMedCrossRefGoogle Scholar
  29. 29.
    Ades AE. A chain of evidence with mixed comparisons: models for multi-parameter evidence synthesis and consistency of evidence. Stat Med 2003; 22: 2995–3016PubMedCrossRefGoogle Scholar
  30. 30.
    Ades AE, Cliffe S. Markov Chain Monte Carlo estimation of a multi-parameter decision model: consistency of evidence and the accurate assessment of uncertainty. Med Decis Making 2002; 22: 359–71PubMedGoogle Scholar
  31. 31.
    Bucher H, Hengstler P, Schindler C, et al. Percutaneous transluminal coronary angioplasty versus medical treatment for non-acute coronary heart disease: meta-analysis of randomised controlled trials. BMJ 2000; 321: 73–7PubMedCrossRefGoogle Scholar
  32. 32.
    Yazdanpanah Y, Sissoko D, Egger M, et al. Clinical efficacy of antiretroviral combination therapy based on protease inhibitors or non-nucleoside analogue reverse transcriptase inhibitors: indirect comparison of controlled trials. BMJ 2004; 328: 249–53PubMedCrossRefGoogle Scholar
  33. 33.
    Song F, Altman DG, Glenny A-M, et al. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ 2003; 326: 472–5PubMedCrossRefGoogle Scholar
  34. 34.
    Antithrombotic Trialists’ Collaboration. Collaborative metaanalysis of randomised trials of antiplatelet therapy for prevention of death, myocardial infarction, and stroke in high risk patients. BMJ 2002; 324: 71–86CrossRefGoogle Scholar
  35. 35.
    Chalmers I, Altman DG. Systematic reviews. London: BMJ Publishing Group, 1995Google Scholar
  36. 36.
    Hasselblad V. Meta-analysis of multi-treatment studies. Med Decis Making 1998; 18: 37–43PubMedCrossRefGoogle Scholar
  37. 37.
    Wilby J, Kainth K, Hawkins N, et al. A rapid and systematic review of the clinical effectiveness, tolerability and cost effectiveness of newer drugs for epilepsy in adults. London: National Institute for Clinical Excellence, 2003Google Scholar
  38. 38.
    Bridle C, Palmer S, Bagnall A-M, et al. A rapid and systematic review and economic evaluation of the clinical and costeffectiveness of newer drugs for treatment of mania associated with bipolar affective disorder. Health Technol Assess 2004; 18 (19): 1–187Google Scholar
  39. 39.
    Higgins JPT, Whitehead J. Borrowing strength from external trials in meta-analysis. Stat Med 1996; 15: 2733–49PubMedCrossRefGoogle Scholar
  40. 40.
    Eddy DM, Hasselblad V, Shachter R. Meta-analysis by the confidence profile method. London: Academic Press, 1992Google Scholar
  41. 41.
    Stinnett A, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analyses. Med Decis Making 1998; 18: S68–80PubMedCrossRefGoogle Scholar
  42. 42.
    Palmer S, Sculpher M, Philips Z, et al. Management of non-STelevation acute coronary syndromes: how cost-effective are glycoprotein IIb/IIIa antagonists in the UK National Health Service? Int J Cardiol 2005 Apr 20; 100 (2): 229–40PubMedCrossRefGoogle Scholar
  43. 43.
    Abrams KR, Sutton AJ, Cooper NJ, et al. Populating economic decision models using meta-analysis of heterogeneously reported studies augmented with expert beliefs. Developing Economic Evaluation Methods (DEEM) workshop; 2003 Sep 4-5; BristolGoogle Scholar
  44. 44.
    Berkey CS, Anderson JJ, Hoaglin DC. Multiple-outcome metaanalysis of clinical trials. Stat Med 1996; 15: 537–57PubMedCrossRefGoogle Scholar
  45. 45.
    Berkey CS, Hoaglin DC, Antczak-Bouckoms A, et al. Metaanalysis of multiple outcomes by regression with random effects. Stat Med 1998; 17: 2537–50PubMedCrossRefGoogle Scholar
  46. 46.
    Nam I-S, Mengerson K, Garthwaite P. Multivariate meta-analysis. Stat Med 2003; 22: 2309–33PubMedCrossRefGoogle Scholar
  47. 47.
    Prentice RK. Surrogate endpoints in clinical trials: definition and operational criteria. Stat Med 1998; 8: 431–40CrossRefGoogle Scholar
  48. 48.
    Psaty BM, Weiss NS, Furberg CD, et al. Surrogate end points, health outcomes, and the drug-approval process for the treatment of risk factors for cardiovascular disease. JAMA 1999; 282: 786–90PubMedCrossRefGoogle Scholar
  49. 49.
    Pahor M, Psaty BM, Alderman MH et al. Health outcomes associated with calcium antagonists compared with other firstline hypertensive therapies: a meta-analysis of randomised controlled trials. Lancet 2000; 356: 1949–54PubMedCrossRefGoogle Scholar
  50. 50.
    Fleming TR, de Mets DL. Surrogate end-points in clinical trials: are we being misled? Ann Intern Med 1996; 125: 605–13PubMedGoogle Scholar
  51. 51.
    Daniels MJ, Hughes MD. Meta-analysis for the evaluation of potential surrogate markers. Stat Med 1997; 16: 1965–82PubMedCrossRefGoogle Scholar
  52. 52.
    Gail MH, Pfeiffer R, van Houwelingen HC, et al. On metaanalytic assessment of surrogate outcomes. Biostatistics 2000; 1: 231–46PubMedCrossRefGoogle Scholar
  53. 53.
    Molenberghs G, Burzykowski T, Alonso A, et al. A perspective on surrogate endpoints in controlled clinical trials. Stat Methods Med Res 2004; 13: 177–206PubMedGoogle Scholar
  54. 54.
    Byar DP. Assessing apparent treatment-covariate interactions in randomized clinical trials. Stat Med 1985; 4: 255–63PubMedCrossRefGoogle Scholar
  55. 55.
    Manca A, Rice N, Sculpher MJ, et al. Assessing generalisability by location in trial-based cost-effectiveness analysis: the use of multilevel models. Health Econ 2005 May; 14 (5): 471–85PubMedCrossRefGoogle Scholar
  56. 56.
    Dixon DO, Simon R. Bayesian subset analysis. Biometrics 1991; 47: 871–81PubMedCrossRefGoogle Scholar
  57. 57.
    Dixon DO, Simon R. Bayesian subset analysis in a colorectal cancer clinical trial. Stat Med 1992; 11: 13–22PubMedGoogle Scholar
  58. 58.
    Sharp SJ, Thompson SG. Analysing the relationship between treatment effect and underlying risk in meta-analysis: comparison and development of approaches. Stat Med 2000; 19: 3251–74PubMedCrossRefGoogle Scholar
  59. 59.
    Vandenbroucke JP. When are observational studies as credible as randomised trials? Lancet 2004; 363: 1728–31PubMedCrossRefGoogle Scholar
  60. 60.
    Centre for Reviews and Dissemination. CRD report 4. Undertaking systematic reviews of research on effectiveness: CRD’s guidance for carrying out or commissioning reviews. 2nd ed. York: CRD, University of York, 2001Google Scholar
  61. 61.
    Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. JAMA 2000; 342: 1887–92Google Scholar
  62. 62.
    Benson BA, Hartz AJ. A comparison of observational studies and randomized controlled trials. N Engl J Med 2000; 342: 1878–86PubMedCrossRefGoogle Scholar
  63. 63.
    Britton A, McKee M, Black N, et al. Choosing between randomised and non-randomised studies: a systematic review. Health Technol Assess 1998; 2 (13): i–iv, 1-124PubMedGoogle Scholar
  64. 64.
    Reeves BC, MacLehose RR, Harvey IM, et al. Comparisons of effect size estimates derived from randomised and nonrandomised studies. In: Black N, Brazier J, Fitzpatrick R, et al., editors. Health services research methods: a guide to best practice. London: BMJ Publishing Group, 1998: 73–85Google Scholar
  65. 65.
    DuMouchel W. Bayesian data mining in large frequency tables, with an application to the FDA Spontaneous Reporting System (with discussion). Am Stat 1999; 53: 177–202Google Scholar
  66. 66.
    Sutton A, Abrams K, Jones D. Generalised synthesis of evidence and the threat of publication bias: the example of electronic fetal heart rate monitoring (EFM). J Clin Epidemiol 2002; 55: 1013–24PubMedCrossRefGoogle Scholar
  67. 67.
    Droitcour F, Silberman G, Chelimsky E. Cross design synthesis: a new form of meta-analysis for combining the results from randomised clinical trials and medical-practice databases. Int J Technol Assess Health Care 1993; 9: 440–9PubMedCrossRefGoogle Scholar
  68. 68.
    Spiegelhalter DJ, Best NG. Bayesian approaches to multiple sources of evidence and uncertainty in complex cost-effectiveness modelling. Stat Med 2003; 22: 3687–709PubMedCrossRefGoogle Scholar
  69. 69.
    Schulz KF, Chalmers I, Hayes RJ, et al. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273: 408–12PubMedCrossRefGoogle Scholar
  70. 70.
    Li Z, Begg CB. Random effects models for combining results from controlled and uncontrolled studies in meta-analysis. J Am Stat Assoc 1994; 89: 1523–7CrossRefGoogle Scholar
  71. 71.
    Carlin B. Analysis of local decisions using hierarchical modeling, applied to home radon measurement and remediation: comment. Stat Sci 1999; 14: 328–30Google Scholar
  72. 72.
    Samsa GP, Renter RA, Parmigiani G, et al. Performing costeffectiveness analysis by integrating randomised trial data with a comprehensive decision model: application to treatment of ischemic stroke. J Clin Epidemiol 1999; 52: 259–71PubMedCrossRefGoogle Scholar
  73. 73.
    Parmigiani G, Ancukiewicz M, Matchar D. Decision models in clinical recommendations development: the stroke prevention policy model. In: Berry DA, Stangl DK, editors. Bayesian biostatistics. New York: Marcel Dekker, 1996: 207–33Google Scholar
  74. 74.
    Craig BA, Fryback DG, Klein R, et al. A Bayesian approach to modelling the natural history of a chronic condition from observations with intervention. Stat Med 1999; 18: 1355–71PubMedCrossRefGoogle Scholar
  75. 75.
    Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004 Oct 30; 23 (20): 3105–24PubMedCrossRefGoogle Scholar
  76. 76.
    Gilks WR, Richardson S, Spiegelhalter DJ. Markov chain Monte Carlo in practice. London: Chapman & Hall/CRC, 1996Google Scholar
  77. 77.
    Welton NJ, Ades AE. A model of toxoplasmosis incidence in the UK: evidence synthesis and consistency of evidence. Appl Stat 2005; 54: 385–404Google Scholar
  78. 78.
    Lambert PC, Sutton AJ, Burton PR, et al. How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS. Stat Med 2005 Aug 15; 24 (15): 2401–28PubMedCrossRefGoogle Scholar
  79. 79.
    Whitehead A. Meta-analysis of controlled clinical trials. Chichester: Wiley, 2002CrossRefGoogle Scholar

Copyright information

© Adis Data Information BV 2006

Authors and Affiliations

  • A. E. Ades
    • 1
  • Mark Sculpher
    • 2
    Email author
  • Alex Sutton
    • 3
  • Keith Abrams
    • 3
  • Nicola Cooper
    • 3
  • Nicky Welton
    • 1
  • Guobing Lu
    • 1
  1. 1.Medical Research Council Health Services Research CollaborationUniversity of BristolBristolEngland
  2. 2.Centre for Health EconomicsUniversity of YorkHeslington, YorkEngland
  3. 3.Centre for Biostatistics & Genetic Epidemiology, Department of Health SciencesUniversity of LeicesterLeicesterEngland

Personalised recommendations