, Volume 17, Issue 5, pp 479–500 | Cite as

Handling Uncertainty in Cost-Effectiveness Models

Consensus Conference Papers


The use of modelling in economic evaluation is widespread, and it most often involves synthesising data from a number of sources. However, even when economic evaluations are conducted alongside clinical trials, some form of modelling is usually essential. The aim of this article is to review the handling of uncertainty in the cost-effectiveness results that are generated by the use of decision-analytic-type modelling. The modelling process is split into a number of stages: (i) a set of methods to be employed in a study are defined, which should include a ‘reference case’ of agreed methods to enhance the comparability of results; (ii) the clinical and demographic characteristics of the patients the model relates to should be specified as carefully as in any experimental study; and (iii) the data requirements of the model should be estimated using the principles of Bayesian statistics, such that prior distributions are specified for unknown model parameters. Monte Carlo simulation can then be employed to sample from these prior distributions to obtain a distribution of the cost effectiveness of the intervention. Such probabilistic analyses are related to parameter uncertainty. In addition, modelling uncertainty is likely to add a further layer of uncertainty to the results of particular analyses.



I am grateful to Dr Alastair Gray for comments and suggestions on earlier versions of this work, the participants of the Expert Workshop on ‘Validating Cost-Effectiveness Models’, 22 to 23 April 1999, held in Sheffield, England, and particularly to Dr Dennis Fryback for his insightful comments on a previous draft of this article, and finally to 2 anonymous referees for their comments. Of course, the responsibility for errors and inaccuracies in this article is all my own.


  1. 1.
    Posnett J, Jan S. Indirect cost in economic evaluation: the opportunity cost of unpaid inputs. Health Econ 1996; 5 (1): 13–23PubMedCrossRefGoogle Scholar
  2. 2.
    Buxton MJ, Drummond MF, van Hout BA, et al. Modelling in economic evaluation: an unavoidable fact of life. Health Econ 1997; 6: 217–27PubMedCrossRefGoogle Scholar
  3. 3.
    Association of British Pharmaceutical Industries. Pharmaceutical industry and Department of Health agree guidelines for the economic analysis of medicines [press release]. London: Association of British Pharmaceutical Industries, 1994Google Scholar
  4. 4.
    Australian Commonwealth Department of Human Services and Health. Guidelines for the pharmaceutical industry on preparation of submissions to the Pharmaceutical Benefits Advisory Committee: including major submissions involving economic analysis. Canberra: Australian Government Publishing Service, 1995Google Scholar
  5. 5.
    Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ 1996; 313: 275–83PubMedCrossRefGoogle Scholar
  6. 6.
    Canadian Coordinating Office for Health Technology Assessment. Guidelines for the economic evaluation of pharmaceuticals: Canada. 2nd ed. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA), 1997Google Scholar
  7. 7.
    Manning WG, Fryback DG, Weinstein MC. Reflecting uncertainty in cost-effectiveness analysis. In: Gold MR, Siegel JE, Russell LB, et al., editors. Cost-effectiveness in health and medicine. New York (NY): Oxford University Press, 1996: 247–75Google Scholar
  8. 8.
    Drummond M, Brandt A, Luce B, et al. Standardizing methodologies for economic evaluation in health care: practice, problems, and potential. Int J Technol Assess Health Care 1993; 9 (1): 26–36PubMedCrossRefGoogle Scholar
  9. 9.
    Lipscomb J. Time preference for health in cost-effectiveness analysis. Med Care 1989; 27 (3 Suppl.): S233–53PubMedCrossRefGoogle Scholar
  10. 10.
    Parsonage M, Neuburger H. Discounting and health benefits. Health Econ 1992; 1 (1): 71–6PubMedCrossRefGoogle Scholar
  11. 11.
    Cairns J. Discounting and health benefits: another perspective [comment]. Health Econ 1992; 1 (1): 76–9PubMedCrossRefGoogle Scholar
  12. 12.
    Coyle D, Tolley K. Discounting of health benefits in the pharmacoeconomic analysis of drug therapies. Pharmacoeconomics 1992; 2 (2): 153–62PubMedCrossRefGoogle Scholar
  13. 13.
    Katz DA, Welch HG. Discounting in cost-effectiveness analysis of healthcare programmes. Pharmacoeconomics 1993; 3 (4): 276–85PubMedCrossRefGoogle Scholar
  14. 14.
    Lipscomb J. The proper role for discounting: search in progress. Med Care 1996; 34 (12 Suppl.): DS119-23Google Scholar
  15. 15.
    Lipscomb J, Weinstein M, Torrance, et al. Time preference. In: Gold MR, Siegel JE, Russell LB, et al., editors. Cost-effectiveness in health and medicine. New York (NY): Oxford University Press, 1996: 214–46Google Scholar
  16. 16.
    HM Treasury. Appraisal and evaluation in central government. London: H.M. Stationery Office, 1997Google Scholar
  17. 17.
    Donaldson C. Willingness to pay for publicly-provided goods: a possible measure of benefit. J Health Econ 1990; 9: 103–18PubMedCrossRefGoogle Scholar
  18. 18.
    Llewellyn TH, Sutherland HJ, Tibshirani R, et al. The measurement of patients’ values in medicine. Med Decis Making 1982; 2 (4): 449–62CrossRefGoogle Scholar
  19. 19.
    Mehrez A, Gafni A. Quality-adjusted life years, utility theory, and healthy-years equivalents. Med Decis Making 1989; 9: 142–9PubMedCrossRefGoogle Scholar
  20. 20.
    Torrance GW. Measurement of health state utilities for economic appraisal: a review. J Health Econ 1986; 5: 1–30PubMedCrossRefGoogle Scholar
  21. 21.
    Conroy RM, O’Brien E, O’Malley K, et al. Measurement error in the Hawksley random zero sphygmomanometer: what damage has been done and what can we learn? BMJ 1993; 306 (6888): 1319–22PubMedCrossRefGoogle Scholar
  22. 22.
    Gold MR, Siegel JE, Russell LB, et al. Cost-effectiveness in health and medicine. New York (NY): Oxford University Press, 1996Google Scholar
  23. 23.
    Meltzer David, Accounting for future costs in medical cost-effectiveness analysis, J. Health Econ 1997; 16 (1): 33–64CrossRefGoogle Scholar
  24. 24.
    Drummond MF, O’Brien B, Stoddart GL, et al. Methods for the economic evaluation of health care programmes. 2nd ed. Oxford: Oxford University Press, 1997Google Scholar
  25. 25.
    Koopmanschap MA, Rutten FFH. Indirect costs: the consequence of production loss or increased costs of production. Med Care 1996; 34 (12 Suppl.): DS59–68Google Scholar
  26. 26.
    Koopmanschap MA, Rutten FFH, Van Ineveld BM, et al. The friction cost method for measuring indirect costs of disease. J Health Econ 1995; 14: 171–89PubMedCrossRefGoogle Scholar
  27. 27.
    Russell LB. Is prevention better than cure? Washington, DC: The Brookings Institution, 1986Google Scholar
  28. 28.
    Briggs AH, Gray AM. Handling uncertainty when performing economic evaluation of health care interventions. Health Technol Assess 1999; 3 (2): 1–134PubMedGoogle Scholar
  29. 29.
    Anderson MH, Camm AJ. Implications for present and future applications of the implantable cardioverter-defibrillator resulting from the use of a simple model of cost efficacy. Br Heart J 1993; 69 (1): 83–92PubMedCrossRefGoogle Scholar
  30. 30.
    Pharoah PD, Hollingworth W. Cost effectiveness of lowering cholesterol concentration with statins in patients with and without pre-existing coronary heart disease: life table method applied to health authority population. BMJ 1996; 312 (7044): 1443–8PubMedCrossRefGoogle Scholar
  31. 31.
    Eddy DM. Screening for cervical cancer. Ann Intern Med 1990; 113 (3): 214–26PubMedGoogle Scholar
  32. 32.
    Johannesson M, Weinstein MC. On the decision rules of cost-effectiveness analysis. J Health Econ 1993; 12: 459–67PubMedCrossRefGoogle Scholar
  33. 33.
    Weinstein M, Fineberg HV. Clinical decision analysis. Philadelphia (PA): WB Saunders Company, 1980Google Scholar
  34. 34.
    Williams A. The economics of coronary artery bypass grafting. BMJ 1985; 291: 326–9PubMedCrossRefGoogle Scholar
  35. 35.
    Pauker SG, Kassirer J. Decision analysis. N Engl J Med 1987; 316 (5): 250–8PubMedCrossRefGoogle Scholar
  36. 36.
    Stinnett AA, Paltiel AD. Estimating CE ratios under second order uncertainty: the mean ratio versus the ratio of means. Med Decis Making 1997; 17 (4): 483–9PubMedCrossRefGoogle Scholar
  37. 37.
    Eddy DM, Hasselblad V, Shachter R. A Bayesian method for synthesizing evidence: the Confidence Profile Method. Int J Technol Assess Health Care 1990; 6 (1): 31–55PubMedCrossRefGoogle Scholar
  38. 38.
    Eddy DM, Hasselblad V, Shachter R. An introduction to a Bayesian method for meta-analysis: the confidence profile method. Med Decis Making 1990; 10 (1): 15–23PubMedCrossRefGoogle Scholar
  39. 39.
    Zhou XH, Melfi CA, Hui SL. Methods for comparison of cost data. Ann Intern Med 1997; 127 (8 Pt 2): 752–6PubMedGoogle Scholar
  40. 40.
    Briggs A, Gray A. The distribution of health care costs and their statistical analysis for economic evaluation. J Health Serv Res Policy 1998; 3: 233–45PubMedGoogle Scholar
  41. 41.
    Gelman A, Carlin J, Stern H, et al. Bayesian data analysis. London: Chapman & Hall, 1995Google Scholar
  42. 42.
    Wennberg J, Gittelsohn A. Variations in medical care in small areas. Sci Am 1982; 4: 120–4CrossRefGoogle Scholar
  43. 43.
    Andersen TF, Mooney G. The challenge of medical practice variations. Basingstoke: Macmillan, 1990Google Scholar
  44. 44.
    Cleary PD, Greenfield S, Mulley AG, et al. Variations in length of stay and outcomes for six medical and surgical conditions in Massachusetts and California. JAMA 1991; 266 (1): 73–9PubMedCrossRefGoogle Scholar
  45. 45.
    Draper D. Assessment and propagation of model uncertainty. J R Stat Soc Br 1995; 57 (1): 45–97Google Scholar
  46. 46.
    Briggs A, Sculpher M. An introduction to Markov models for economic evaluation. Pharmacoeconomics 1998; 13 (4): 397–409PubMedCrossRefGoogle Scholar
  47. 47.
    Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Med Decis Making 1993; 13: 322–38PubMedCrossRefGoogle Scholar
  48. 48.
    SMLTREE [computer program]. 2.9. Hollenberg J, 1989Google Scholar
  49. 49.
    Decision Analysis by TreeAge (DATA) [computer program]. Tree Age Software Inc. v3.5. Williamstown (MA): Tree Age, 1998Google Scholar
  50. 50.
    Briggs AH, Fenn P. Confidence intervals or surfaces? Uncertainty on the cost-effectiveness plane. Health Econ 1998: 7 (8): 723–40PubMedCrossRefGoogle Scholar
  51. 51.
    Sharples LD, Briggs A, Caine N, et al. A model for analyzing the cost of main clinical events after cardiac transplantation. Transplantation 1996; 62 (5): 615–21PubMedCrossRefGoogle Scholar
  52. 52.
    Hunink MG, Bult JR, de Vries J, et al. Uncertainty in decision models analyzing cost-effectiveness: the joint distribution of incremental costs and effectiveness evaluated with a nonparametric bootstrap method. Med Decis Making 1998; 18 (3): 337– 46PubMedCrossRefGoogle Scholar
  53. 53.
    Pratt JW, Raiffa H, Schlaifer R. Introduction to statistical decision theory. Cambridge (MA): MIT Press, 1995Google Scholar
  54. 54.
    Stinnett AA, Mullahy J. The negative side of cost-effectiveness analysis [letter]. JAMA 1997; 277 (24): 1931–2PubMedGoogle Scholar
  55. 55.
    Stinnett AA, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making 1998; 18 (2 Suppl.): S65–80Google Scholar
  56. 56.
    van Hout BA, Al MJ, Gordon GS, et al. Costs, effects and C/Eratios alongside a clinical trial. Health Econ 1994; 3 (5): 309–19PubMedCrossRefGoogle Scholar
  57. 57.
    Briggs AH. A Bayesian approach to stochastic cost-effectiveness analysis. Health Econ 1999; 8 (3): 257–61PubMedCrossRefGoogle Scholar
  58. 58.
    Doubilet P, Begg CB, Weinstein MC, et al. Probabilistic sensitivity analysis using Monte Carlo simulation: a practical approach. Med Decis Making 1985; 5: 157–77PubMedCrossRefGoogle Scholar
  59. 59.
    Critchfield GC, Willard KE, Connelly DP. Probabilistic sensitivity analysis methods for general decision models. Comput Biomed Res 1986; 19: 254–65PubMedCrossRefGoogle Scholar
  60. 60.
    Berwick DM, Cretin S, Keeler E. Cholesterol, children, and heart disease: an analysis of alternatives. Pediatrics 1981; 68 (5): 721–30PubMedGoogle Scholar
  61. 61.
    Hornberger JC. The hemodialysis prescription and cost effectiveness: Renal Physicians Association Working Committee on Clinical Guidelines. J Am Soc Nephrol 1993; 4 (4): 1021–7PubMedGoogle Scholar
  62. 62.
    Gabriel SE, Campion ME, O’Fallon WM. A cost-utility analysis of misoprostol prophylaxis for rheumatoid arthritis patients receiving nonsteroidal antiinflammatory drugs. Arthritis Rheum 1994; 37 (3): 333–41PubMedCrossRefGoogle Scholar
  63. 63.
    Fiscella K, Franks P. Cost-effectiveness of the transdermal nicotine patch as an adjunct to physicians’ smoking cessation counseling. JAMA 1996; 275 (16): 1247–51PubMedCrossRefGoogle Scholar
  64. 64.
    Oh PI, Maerov P, Pritchard D, et al. A cost-utility analysis of second-line antibiotics in the treatment of acute otitis media in children. Clin Ther 1996; 18 (1): 160–82PubMedCrossRefGoogle Scholar
  65. 65.
    O’Brien BJ, Drummond MF, Labelle RJ, et al. In search of power and significance: issues in the design and analysis of stochastic cost-effectiveness studies in health care. Med Care 1994; 32 (2): 150–63PubMedCrossRefGoogle Scholar
  66. 66.
    Sheldon TA. Problems of using modelling in the economic evaluation of health care [editorial]. Health Econ 1996; 5 (1): 1–11PubMedCrossRefGoogle Scholar
  67. 67.
    Sculpher MJ, Drummond MF, Buxton MJ. The iterative use of economic evaluation as part of the process of health technology assessment. J Health Services Res Policy 1997; 2: 26–30Google Scholar
  68. 68.
    Sonnenberg FA, Roberts MS, Tsevat J, et al. Toward a peer review process for medical decision analysis models. Med Care 1994; 32 (7 Suppl.): JS52–64Google Scholar
  69. 69.
    Claxton K, Posnett J. An economic approach to clinical trial design and research priority-setting. Health Econ 1996; 5 (6): 513–24PubMedCrossRefGoogle Scholar
  70. 70.
    Felli JC, Hazen GB. Sensitivity analysis and the expected value of perfect information. Med Decis Making 1998; 18 (1): 95–109PubMedCrossRefGoogle Scholar
  71. 71.
    Fenwick E, Claxton K, Sculpher M, et al. Improving the efficiency and relevance of health technology assessment: the role of decision analytic modelling. Health Economists’ Study Group Conference; 1999 Jan 6–8; BirminghamGoogle Scholar

Copyright information

© Adis International Limited 2000

Authors and Affiliations

  1. 1.Health Economics Research CentreUniversity of Oxford, Institute of Health SciencesHeadington, OxfordEngland

Personalised recommendations