, Volume 26, Issue 9, pp 781–798 | Cite as

Exploring Uncertainty in Cost-Effectiveness Analysis

  • Karl ClaxtonEmail author
Briefing Paper


This paper describes the key principles of why an assessment of uncertainty and its consequences are critical for the types of decisions that a body such as the UK National Institute for Health and Clinical Excellence (NICE) has to make. In doing so, it poses the question of whether formal methods may be useful to NICE and its advisory committees in making such assessments. Broadly, these include the following: (i) should probabilistic sensitivity analysis continue to be recommended as a means to characterize parameter uncertainty; (ii) which methods should be used to represent other sources of uncertainty; (iii) when can computationally expensive models be justified and is computation expense a sufficient justification for failing to express uncertainty; (iv) which summary measures of uncertainty should be used to present the results to decision makers; and (v) should formal methods be recommended to inform the assessment of the need for evidence and the consequences of an uncertain decision for the UK NHS?


Probabilistic Sensitivity Analysis Decision Uncertainty Positive Guidance Uncertain Decision Expensive Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This paper was initially prepared as a briefing paper for NICE as part of the process of updating the Institute’s 2004 Guide to the Methods of Technology Appraisal. The work was funded by NICE through its Decision Support Unit (DSU), which is based at the universities of Sheffield, Leicester, York, Leeds and at the London School of Hygiene and Tropical Medicine.

The author has no conflicts of interest that are directly related to the contents of this article.

The author thanks members of the DSU who commented on the briefing document that forms the basis of this paper as well as Iain Chalmers, Alex Sutton, Alan Brennan, Louise Longworth and Carole Longson, who provided helpful comments on earlier drafts of this paper. All errors and omissions are the responsibility of the author.


  1. 1.
    Culyer AJ, McCabe C, Briggs A, et al. Searching for a threshold, not setting one: the role of the National Institute for Health and Clinical Excellence. J Health Serv Res Pol 2007; 12: 56–58CrossRefGoogle Scholar
  2. 2.
    McCabe C, Claxton K, Culyer AJ. The NICE cost-effectiveness threshold: what it is and what that means. Pharmacoeconomics 2008; 26 (9): 733–744PubMedCrossRefGoogle Scholar
  3. 3.
    Phelps CE, Mushlin Al. On the (near) equivalence of cost-effectiveness and cost-benefit analyses. Int J Technol Assess Health Care 1991; 7 (1): 12–21PubMedCrossRefGoogle Scholar
  4. 4.
    Stinnett A, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making 1998; 18: S68–S80PubMedCrossRefGoogle Scholar
  5. 5.
    Sculpher MJ. Subgroups and heterogeneity in cost-effectiveness analysis. Pharmacoeconomics 2008; 26 (9): 799–806PubMedCrossRefGoogle Scholar
  6. 6.
    Claxton K. The irrelevance of inference: a decision making approach to the stochastic evaluation of health care technologies. J Health Econ 1999; 18: 342–364CrossRefGoogle Scholar
  7. 7.
    Briggs A, Sculpher M. An introduction to Markov modelling for economic evaluation. Pharmacoeconomics 1998; 13: 397–409PubMedCrossRefGoogle Scholar
  8. 8.
    Briggs A, Claxton K, Sculpher MJ. Decision analytic modelling for the evaluation of health technologies. Oxford: Oxford University Press, 2006Google Scholar
  9. 9.
    Eckermann S, Willan A. Expected value of information and decision making in HTA. Health Econ 2007; 16: 195–209PubMedCrossRefGoogle Scholar
  10. 10.
    Palmer S, Smith PC. Incorporating option values into the economic evaluation of health care technologies. J Health Econ 2000; 19: 755–766PubMedCrossRefGoogle Scholar
  11. 11.
    Griffin S, Claxton K, Palmer S, et al. Dangerous omissions: the consequences of ignoring decision uncertainty. Med Dec Making 2007; 27 (1): E8Google Scholar
  12. 12.
    Chalkidou K, Hoy A, Littlejohns P. Making a decision to wait for more evidence: when the National Institute for Health and Clinical Excellence recommends a technology only in the context of research. J R Soc Med 2007; 100 (10): 453–460PubMedCrossRefGoogle Scholar
  13. 13.
    National Institute for Health and Clinical Excellence. NICE guide to the methods of health technology appraisal. London: NICE, 2004Google Scholar
  14. 14.
    Cooksey D. A review of UK health research funding. London: Stationery Office, 2006Google Scholar
  15. 15.
    Office of Fair Trading. The pharmaceutical price regulation scheme: an OFT market study. London: Office of Fair Trading, 2007Google Scholar
  16. 16.
    Claxton K, Briggs A, Buxton MJ, et al. Value-based pricing for NHS drugs: an opportunity not to be missed? BMJ 2008; 336: 251–254PubMedCrossRefGoogle Scholar
  17. 17.
    Claxton K, Sculpher MJ, McCabe C, et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ 2005; 14: 339–347PubMedCrossRefGoogle Scholar
  18. 18.
    Thompson K, Graham J. Going beyond the single number: using probabilistic risk assessment to improve risk management. Human Ecologic Risk Assess 1996; 2: 1008–1034CrossRefGoogle Scholar
  19. 19.
    O’Hagan A, Luce BR. A primer on Bayesian statistics in health economics and outcomes research. Bethesda (MD): Bayesian initiative in health economic and outcomes research, MEDTAP International, 2003Google Scholar
  20. 20.
    O’Hagan A, Buck CE, Daneshkhah A, et al. Uncertain judgements: eliciting expert probabilities. Chichester: John Wiley and Sons, 2006CrossRefGoogle Scholar
  21. 21.
    Sutton A, Ades AE, Cooper N, et al., on behalf of the NICE Decision Support Unit. Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics 2008; 26 (9): 753–767PubMedCrossRefGoogle Scholar
  22. 22.
    Ades AE, Sutton AJ. Multiparameter evidence synthesis in epidemiology and medical decision-making: current approaches. JRSS (A) 2005; 16 (1): 5–35Google Scholar
  23. 23.
    Ades AE, Claxton K, Sculpher MJ. Evidence synthesis, parameter correlation, and probabilistic sensitivity analysis. Health Econ 2006; 15: 373–381PubMedCrossRefGoogle Scholar
  24. 24.
    Bojke L, Claxton K, Palmer S, et al. Defining and characterising structural uncertainty in decision analytic models [CHE research paper 9]. York: Centre for Health Economics, University of York, 2006Google Scholar
  25. 25.
    McCabe C, Dixon S. Testing the validity of cost-effectiveness models. Pharmacoeconomics 2000; 17 (5): 501–513PubMedCrossRefGoogle Scholar
  26. 26.
    Draper D. Assessment and propagation of model uncertainty. J Royal Stat Soc 1995; 57: 45–97Google Scholar
  27. 27.
    Chatfield C. Model uncertainty, data mining and statistical inference (with discussion). J Royal Stat Soc 1995; 158: 419–466CrossRefGoogle Scholar
  28. 28.
    Bojke L, Claxton K, Bravo Vergel Y. Using expert elicitation to resolve issues of structural uncertainty in decision analytic models. Med Dec Making 2007; 27 (1): E8Google Scholar
  29. 29.
    White IR, Pocock SJ, Wang D. Eliciting and using expert opinions about dropout bias in randomised controlled trials. Clin Trials 2007; 4 (2): 125–139PubMedCrossRefGoogle Scholar
  30. 30.
    Leal J, Wordsworth S, Legood R, et al. Eliciting expert opinion for economic models. Value Health 2007; 10 (3): 195–203PubMedCrossRefGoogle Scholar
  31. 31.
    Griffin S, Claxton K, Hawkins N, et al. Probabilistic analysis and computationally expensive models: necessary and required? Value Health 2006; 9: 244–252PubMedCrossRefGoogle Scholar
  32. 32.
    Edmunds WJ, Medley GF, Nokes DJ. Evaluating the cost-effectiveness of vaccination programmes: a dynamic perspective. Stat Med 1999; 18 (23): 3263–3282PubMedCrossRefGoogle Scholar
  33. 33.
    O’Hagan A, Stevenson M, Madan J. Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA. Health Econ 2007; 16 (10): 1009–1023PubMedCrossRefGoogle Scholar
  34. 34.
    Stevenson M, Oakley J, Chilcott J. Gaussian process modelling in conjunction with individual patient simulation modelling: a case study describing the calculation of cost-effectiveness ratios for the treatment of established osteoporosis. Med Dec Making 2004: 24: 89–100CrossRefGoogle Scholar
  35. 35.
    Ades AE, Lu G, Claxton K. Expected value of sample information in medical decision modelling. Med Dec Making 2004; 24: 207–227CrossRefGoogle Scholar
  36. 36.
    Van Hout B A, Al MJ, Gordon GS, et al. Costs, effects and c/e-ratios alongside a clinical trial. Health Econ 1994; 3: 309–319CrossRefGoogle Scholar
  37. 37.
    Fenwick E, Claxton K, Sculpher MJ. Representing uncertainty: the role of cost-effectiveness acceptability curves. Health Econ 2001; 10: 779–789PubMedCrossRefGoogle Scholar
  38. 38.
    Fenwick E, O’Brien B, Briggs AH. Cost-effectiveness acceptability curves: facts, fallacies and frequently asked questions. Health Econ 2004; 13: 405–415PubMedCrossRefGoogle Scholar
  39. 39.
    Colbourn T, Asseburg C, Bojke L, et al. Preventive strategies for group B streptococcal and other bacterial infections in early infancy: cost effectiveness and value of information analyses. BMJ 2007; 335: 655–662PubMedCrossRefGoogle Scholar
  40. 40.
    Pratt J, Raiffa H, Schlaifer R. Statistical decision theory. Cambridge (MA): MIT Press, 1995Google Scholar
  41. 41.
    Yokota F, Thompson KM. Value of information literature analysis: a review of applications in health risk management. Med Dec Making 2004; 24: 287–298CrossRefGoogle Scholar
  42. 42.
    Poynard T, Munteanu M, Ratziu V, et al. Truth survival in clinical research: an evidence-based requiem? Ann Intern Med 2002; 136: 888–895PubMedGoogle Scholar
  43. 43.
    Philips Z, Claxton K, Palmer P. The half-life of truth: what are appropriate time horizons for research decisions? Med Dec Making 2008; 28: 287–299CrossRefGoogle Scholar
  44. 44.
    Brennan A, Kharroubi S, O’Hagan A, et al. Calculating partial expected value of perfect information via Monte Carlo sampling algorithms. Med Dec Making 2007; 27: 448–470CrossRefGoogle Scholar
  45. 45.
    Felli JC, Hazen GB. Sensitivity analysis and the expected value of perfect information. Med Dec Making 1998; 18: 95–109CrossRefGoogle Scholar
  46. 46.
    Griffin S, Claxton K. Interpreting the expected value of perfect information about parameters. Med Dec Making 2007; 27 (1): E8Google Scholar
  47. 47.
    Brennan A, Kharroubi S. Expected value of sample information for Weibull survival data. Health Econ 2007: 16: 1205–1225PubMedCrossRefGoogle Scholar
  48. 48.
    Conti S, Claxton K. Dimensions of design space: a decision theoretic approach to optimal research design [CHE research paper 38]. York: Centre for Health Economics, University of York, 2008Google Scholar

Copyright information

© Adis Data Information BV 2008

Authors and Affiliations

  1. 1.Centre for Health Economics, Department of Economics and NICE Decision Support UnitUniversity of YorkHeslington, YorkUK

Personalised recommendations