Summary
This article examines the twin concepts of the statistical significance and quantitative importance of observed differences in studies comparing medicines in terms of economic parameters such as cost-effectiveness and measures of health-related quality of life (HRQOL). Central to the design and interpretation of any comparative study, such as a randomised controlled trial, is some prior judgement about the order of magnitude of a difference that would make one switch from one therapy to another. Starting with current definitions of clinically important differences we argue by analogy that the importance of differences in HRQOL require a shift of focus from the physician to the patient for preferences and judgements concerning what is important to them. Whether an intervention offers sufficient value for money (cast effectiveness or cost utility) to warrant resources being reallocated to it is a collective decision requiring the input of public preferences about the relative importance of alternative therapies and health benefits. Ultimately, the importance of the health benefits offered by a new drug is revealed by societal willingness-to-pay. This may be stated implicitly through committees using cost-effectiveness ‘league tables’ for decision making, or explicitly by consumer surveys of willingness-to-pay in the context of cost-benefit analysis and stemming from the theoretical foundation of welfare economics.
Similar content being viewed by others
References
Aaronson NK. Quality of life assessment in clinical trials: methodologic issues. Controlled Clinical Trials 10: 195S–208S, 1989
Adams ME, McCall NT, Gray DT, et al. Economic analysis is randomized control trials. Medical Care 30: 231–243, 1992
Braitman LE. Confidence intervals assess both clinical significance and statistical significance. Annals of Internal Medicine 114: 515–517, 1991
Burnand B, Kernan WN, Feinstein AR. Indexes and boundaries for ‘quantitative significance’ in statistical decisions. Journal of Clinical Epidemiology 43: 1273–1284, 1990
Cohen J. Statistical Power analysis for the behavioural sciences, Academic Press, New York, 1969
Commonwealth of Australia, Guidelines for the pharmaceutical industry on preparation of submissions to the Pharmaceutical Benefits Advisory Committee: including submissions involving economic analyses, Woden(ACT) Department of Health, Housing and Community Services, 1990
Croog SH, Levine S, Testa MA, et al. The effects of anti-hypertensive therapy on quality of life. New England Journal of Medicine 314: 157–164, 1986
Detsky AS. Are clinical trials a cost-effective investment? Journal of the American Medical Association 262: 1795–1800, 1989
Detsky A. Guidelines for preparation of economic analysis of pharmaceutical products: a draft document for Ontario and Canada Pharmacoeconomics 3: 354–361, 1993
Doubilet P, Weinstein MC, McNeil BJ. Use and misuse of the term ‘cost-effective’ in medicine. New England Journal of Medicine 314: 253–255, 1986
Drummond MF, Davics LM. Economic analysis alongside clinical trials: revisiting the methodological issues. International Journal of Technology Assessment in Health Care 7: 561–573, 1991
Drummond MF, O’Brien BJ. Clinical importance, statistical significance and the assessment of economic and quality-of-life outcomes. Health Economics 2: 205–212, 1993
Drummond MF, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes, Oxford University Press, Oxford, 1987
Eddy DM. Oregon’s methods. Did cost-effectiveness analysis fail? Journal of the American Medical Association 266: 2135–2141, 1991
Efron B, Gong G. A leisurely look at the Bootstrap, Jackknife and cross-validation. American Statistician 1: 36–48, 1983
Elixhauser A, Luce BR, Taylor WR, et al. Health care CSA/CEA: an update in the the growth and composition of the literature. Medical Care 31: JS1–JS11, 1993
Fletcher A, Gore S, Jones D, et al. Quality of life measures in health care: II. Design, analysis and interpretation. British Medical Journal 305: 1145–1148, 1992
Furlong W, Feeny D, Torrance GW, et al. Guide to design and development of health-state utility instrumentation. Centre for Health Economics and Policy Analysis, McMaster University, Working Paper #90-9, 1990
Gafni A, Birch S. Guidelines for the adoption of new technologies: a prescription for the uncontrolled growth in expenditures and how to avoid the problem. Canadian Medical Association Journal 148: 913–917, 1993
Gardner MJ, Altman DG. Statistics with confidence - confidence intervals and statistical guidelines. British Medical Journal, London, 1989
Guyatt GH, Feeny DH, Patrick DL. Measuring health-related quality of life. Annals of Internal Medicine 118: 622–629, 1993
Henry D. Economic analysis as an aid to subsidisation: the development of Australian guidelines for pharmaceuticals. PharmacoEconomics 1: 54–67, 1992
Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Controlled Clinical Trials 10: 407–415, 1989
Jaeschke R, Guyatt GH, Cook D. Quality of life instruments in the evaluation of new drugs. PharmacoEconomics 1: 84–93, 1992
Johannesson M, Jonsson B. Economic evaluation in health care: is there a role for cost-benefit analysis? Health Policy 17: 1–23, 1991
Kaplan RM, Anderson JP. A general health policy model: update and applications. Health Services Research 23: 203–235, 1988
Laupacis A, Feeny D, Detsky A, Tugwell P. How attractive does a new technology have to be to warrant adoption and utilization? Tentative guidelines for using clinical and economic evaluations. Canadian Medical Association Journal 146: 473–481, 1992
Mason J, Drummond M, Torrance G. Some guidelines on the use of cost-effectiveness league tables. British Medical Journal 306: 570–572, 1993
O’Brien BJ, Drummond MF, Labelle RJ, Willan A. In search of power and significance: issues in the design and analysis of stochastic cost-effectiveness studies in health care. Medical Care 32: 150–163, 1994
O’Brien BJ, Viramontes JL. Willingness to pay: a valid and reliable measure of health state preference? Medical Decision Making, in press, 1994
Phelps CE, Mushlin AI. On die (near) equivalence of cost-effectiveness and cost-benefit analyses, International Journal of Technology Assessment in Healthcare. 7: 12–21, 1991
Sackett DL, Haynes RB, Tugwell P. Clinical epidemiology: a basic science for clinical medicine, Little, Brown and Company, Boston, 1985
Slovic P. Perception of risk. Science 236: 280–285, 1987
Spilker H, Tilson H. Quality of life bibliography. Medical Care 28(Suppl. 12): 1990
Sugden R, Williams A. The principles of practical cost-benefit analysis, Oxford University Press, Oxford, 1978
Testa MA, Anderson RB, Nackley JF, et al. Quality of life and antihypertensive therapy in men. New England Journal of Medicine 328: 907–913, 1993
Thompson MS, Read JL, Hutchings HC, Harris ED. The cost-effectiveness of aurofarin: results of a randomized clinical trial. Journal of Rheumatology 15: 35–42, 1989
Torrance GW. Measurement of health state utilities for economic appraisal: a review. Journal of Health Economics 5: 1–30, 1986
Ware JE, Sherbourne CD. The MOS 36-item Short-Form Health Survey (SF-36): I. Conceptual framework and item selection. Medical Care 30: 473–483, 1992
Ware JE, Snow KK, Kosinski M, Gardek B. SF-36 Health Survey: manual and interpretation guide (The Health Institute, New England Medical Center), Nimrod Press, Boston, 1993
Weinstein MC, Stason WB. Foundations of cost-effectiveness analysis for health and medical practices. New England Journal of Medicine 296: 716–721, 1977
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
O’Brien, B.J., Drummond, M.F. Statistical Versus Quantitative Significance in the Socioeconomic Evaluation of Medicines. Pharmacoeconomics 5, 389–398 (1994). https://doi.org/10.2165/00019053-199405050-00005
Published:
Issue Date:
DOI: https://doi.org/10.2165/00019053-199405050-00005