, Volume 17, Issue 5, pp 501–513

Testing the Validity of Cost-Effectiveness Models

Consensus Conference Papers


A growing body of recent work has identified several problems with economic evaluations undertaken alongside controlled trials that can have potentially serious impacts on the ability of decision makers to draw valid conclusions. At the same time, the use of cost-effectiveness models has been drawn into question, due to the alleged arbitrary nature of their construction. This has led researchers to try and identify ways of improving the quality of cost-effectiveness models through identifying ‘best practice’, producing guidelines for peer review and identifying tests of validity.

This paper investigates the issue of testing the validity of cost-effectiveness models or, perhaps more appropriately, whether it is possible to objectively measure the quality of a cost-effectiveness model. A review of the literature shows that there is much confusion over the different aspects of modelling that should be assessed in respect to model quality, and how this should be done.

We develop a framework for assessing model quality in terms of: (i) the structure of the model; (ii) the inputs of the model; (iii) the results of the model; and (iv) the value of the model to the decision maker. Quality assessment is investigated within this framework, and it is argued that it is doubtful that a set of objective tests of validity will ever be produced, or indeed that such an approach would be desirable. The lack of any clearly definable and objective tests of validity means that the other parts of the evaluation process need to be given greater emphasis. Quality assurance forms a small part of a broader process and is best implemented in the form of good practice guidelines. A set of key guidelines are presented.


  1. 1.
    Kassirer JP, Angell M. The journal’s policy on cost effectiveness analyses [editorial]. N Engl J Med 1994; 331 (10): 669–70PubMedCrossRefGoogle Scholar
  2. 2.
    O’Brien B. Economic evaluation of pharmaceuticals: Frankenstein’s monster or vampire of trials? Med Care 1996; 34 (12): DS99–108Google Scholar
  3. 3.
    Drummond MF, Stoddart GL. Economic analysis and clinical trials. Control Clin Trials 1984; 5: 115–28PubMedCrossRefGoogle Scholar
  4. 4.
    Drummond MF, Davies L. Economic analysis alongside clinical trials: revisiting the methodological issues. Int J Technol Assess Health Care 1991; 7: 561–73PubMedCrossRefGoogle Scholar
  5. 5.
    Adams ME, McCall NT, Gray DT, et al. Economic analysis in randomised controlled trials. Med Care 1992; 30 : 231–43PubMedCrossRefGoogle Scholar
  6. 6.
    Morris J, Goddard M. Economic evaluation and quality of life assessments in cancer clinical trials: the CHART trial. Eur J Cancer 1993; 5: 766–70CrossRefGoogle Scholar
  7. 7.
    Bonsel GJ, Rutten FFH, Uyl-de Groot CA. Economic evaluation alongside cancer trials: methodological and practical aspects. Eur J Cancer 1993; 29A: S10–4PubMedCrossRefGoogle Scholar
  8. 8.
    Drummond MF. Economic analysis alongside clinical trials: problems and potential. J Rheumatol 1995; 22: 1403–7PubMedGoogle Scholar
  9. 9.
    Powe NE, Griffiths RI. The clinical-economic trial: promise, problems, and challenges. Control Clin Trials 1995; 16: 377–94PubMedCrossRefGoogle Scholar
  10. 10.
    Mauskopf J, Schulman K, Bell L, et al. A strategy for collecting pharmacologic data during phase II/III clinical trials. Pharmacoeconomics 1996; 9: 264–77PubMedCrossRefGoogle Scholar
  11. 11.
    Ellwein LB, Drummond MF. Economic analysis alongside clinical trials: bias in the assessment of economic outcomes. Int J Technol Assess Health Care 1996; 12: 691–7PubMedCrossRefGoogle Scholar
  12. 12.
    Rigby K, Silagy C, Crockett A. Can resource use be extracted from randomized controlled trials to calculate costs? Int J Technol Assess Health Care 1996; 12: 714–20PubMedCrossRefGoogle Scholar
  13. 13.
    Rittenhouse BE. Exorcising protocol-induced spirits: making the clinical trial relevant for economists. Med Decis Making 1997; 17: 331–9PubMedCrossRefGoogle Scholar
  14. 14.
    Sheldon T. Problems of using modelling in the economic evaluation of health care. Health Econ 1996; 5 (1): 1–11PubMedCrossRefGoogle Scholar
  15. 15.
    Eddy D. Technology assessment: the role of mathematical modelling. In: Mosteller F, editor. Assessing medical technologies. Washington, DC: National Academy Press, 1985: 144–60Google Scholar
  16. 16.
    Buxton MJ, Drummond MF, van Hout BA, et al. Modelling in economic evaluation: an unavoidable fact of life. Health Econ 1997; 6 (3): 217–28PubMedCrossRefGoogle Scholar
  17. 17.
    Mandelblatt JS, Fryback DG, Weinstein MC, et al. Assessing the effectiveness of health interventions. In: Gold MR, Siegel JE, Russell LB, et al., editors. Cost-effectiveness analysis in health and medicine. New York (NY): Oxford University Press, 1996: 135–64Google Scholar
  18. 18.
    Sonnenberg FA, Roberts MS, Tsevat J, et al. Toward a peer review process for medical decision analysis models. Med Care 1994; 32 (7): JS52–64Google Scholar
  19. 19.
    Ellis J, Mulligan I, Rowe J, et al. Inpatient general medicine is evidence based. Lancet 1995 Aug 12; 346 (8972): 407–10PubMedCrossRefGoogle Scholar
  20. 20.
    Morris S, McGuire A, Caro J, et al. Strategies for the management of hypercholesterolaemia: a systematic review of the cost-effectiveness literature. J Health Serv Res Policy 1997; 2: 231–50PubMedGoogle Scholar
  21. 21.
    Deverill M, Brazier J, Green C, et al. The use of QALY and non-QALY measures of health related quality of life: assessing the state of the art. Pharmacoeconomics 1998; 13 (4): 411–20PubMedCrossRefGoogle Scholar
  22. 22.
    Dolan P. Valuing health related quality of life: issues and controversies. Pharmacoeconomics 1999; 15 (2): 119–27PubMedCrossRefGoogle Scholar
  23. 23.
    Rittenhouse B. Use of models in economic evaluations of medicines and other technologies. London: Office of Health Economics, 1996Google Scholar
  24. 24.
    Halpern MT, Luce BR, Brown RE, et al. Health and economic outcomes modeling practices: a suggested framework. Value Health 1998; 1 (2): 131–47PubMedCrossRefGoogle Scholar
  25. 25.
    Halpern MT, McKenna M, Hutton J. Modeling in economic evaluation: an unavoidable fact of life [letter]. Health Econ 1998; 7 (8): 741–2PubMedCrossRefGoogle Scholar
  26. 26.
    Nuijten MJC. The selection of data sources for use in modelling studies. Pharmacoeconomics 1998; 13 (3): 305–16PubMedCrossRefGoogle Scholar
  27. 27.
    Tosteson AHA, Rosenthall DI, Melton LJ, et al. Cost effectiveness of screening perimenopausal white women for osteoporosis: bone densitometry and hormone replacement therapy. Ann Intern Med 1990; 113: 594–603PubMedGoogle Scholar
  28. 28.
    Sculpher M, Michaels J, McKenna M, et al. A cost utility analysis of laser-angioplasty for peripheral arterial occlusions. Int J Technol Assess Health Care 1996; 12 (1): 104–25PubMedCrossRefGoogle Scholar
  29. 29.
    Schulpher M, Drummond M, Buxton M. The iterative use of economic evaluation as part of the process of health technology assessment. J Health Serv Res Policy 1997; 2 (1): 26–30Google Scholar

Copyright information

© Adis International Limited 2000

Authors and Affiliations

  1. 1.Sheffield Health Economics Group, School of Health and Related ResearchUniversity of SheffieldSheffieldEngland

Personalised recommendations