Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Costing and Cost Analysis in Randomized Controlled Trials

Caveat Emptor

Abstract

This article provides an overview of the central issues regarding cost valuation and analysis for a decision maker’s evaluation of costing performed within randomized controlled trials (RCTs). Costing involves specific choices for valuation and analysis that involve trade-offs. Understanding these choices and their implications is necessary for proper evaluation of how costs are valued and analyzed within an RCT and cannot be assessed through a checklist of adherence to general principles.

Resource costing, the most common method of costing, involves measuring medical service use in study case report forms and translating this use into a cost by multiplying the number of units of each medical service by price weights for those services. A choice must be made as to how detailed the measurement of resources will be. Micro-costing improves the specificity of the cost estimate, but it is often impractical to precisely measure resources at this level and the price weights for these micro-units may not be available. Gross-costing may be more practical, and price weights are often easier to find and are more reliable, but important resource differences between treatment groups may be lost in the bundling of resources. Price weights can either be nationally determined or centre specific, but the appropriate price weight will depend on perspective, convenience, completeness and accuracy. Identifying the resource types and the appropriate price weights for these resources are the essential elements to an accurate valuation of costs.

Once medical services are valued, the resulting individual patient cost estimates must be analysed. The difference in the mean cost between treatment groups is the important summary statistic for cost-effectiveness analysis from both the budgetary and the social perspectives. The statistical challenges with cost data typically stem from its skewed distribution and the resulting possibility that the sample mean may be inefficient and possibly inappropriate for statistical inference. Multivariable analysis of cost is useful, even if the data come from an RCT, but the same distributional problems that affect univariate tests of cost also affect use of cost as a dependent variable in a multivariable regression analysis. The generalized linear model (GLM) overcomes many of the problems of more common cost models, but caution must be used when applying this model because it is prone to mis-specification and precision losses in data with a heavy-tailed log error term.

Attention to the appropriate cost valuation and analysis techniques reviewed here will help bring the same level of rigor and attention to the methodological issues in cost valuation as is currently applied to clinical evidence within RCTs.

This is a preview of subscription content, log in to check access.

Fig. 1

Notes

  1. 1.

    Economic efficiency, or Kaldor-Hicks efficiency, defines an efficient outcome as one where no one can be made worse off and at least one person can be made better off after allowing for those that can be made better off to compensate those that are made worse off. If a cost-effectiveness ratio is based on medians or other summary measures that are not means, the ratio could not be used to allocate resources according to the principles of economic efficiency.

  2. 2.

    When we refer to sample mean in general we refer to the arithmetic mean.

  3. 3.

    OLS models tend to be more sensitive to the idiosyncrasies of cost data rather than the predominant pattern of the data, which is what leads to over-fitting.

References

  1. 1.

    Gold MR, Siegel JE, Russel LB, et al., editors. Cost-effectiveness in health and medicine. Oxford: Oxford University Press, 1996

  2. 2.

    Drummond MF, Sculpher MJ, Torrance GW, et al. Methods for the economic evaluation of health care programmes. Oxford: Oxford University Press, 2005

  3. 3.

    Glick HA, Doshi JA, Sonnad SS, et al. Economic evaluation in clinical trials. Oxford: Oxford University Press, 2007

  4. 4.

    Finkler S. The distinction between costs and charges. Ann Int Med 1982; 96: 102–9

  5. 5.

    Medical Research Council Laparoscopic Groin Hernia Trial Group. Cost-utility analysis of open versus laparoscopic groin hernia repair: results from a multicentre randomized clinical trial. Br J Surg 2001; 88 (5): 653–61

  6. 6.

    Fleming MF, Mundt MP, French MT, et al. Benefit-cost analysis of brief physician advice with problem drinkers in primary care settings. Med Care 2000; 38 (1): 7–18

  7. 7.

    Halpern MT, Palmer CS, Zodet M, et al. Cost-effectiveness of gemifloxacin: results from the GLOBE study. Am J Health Sys Pharm 2002; 59 (14): 1357–65

  8. 8.

    Simon GE, Manning WG, Katzelnick DJ, et al. Cost-effectiveness of systematic depression treatment for high utilizers of general medical care. Arch Gen Psychiatry 2001; 58 (2): 181–7

  9. 9.

    Beck EJ, Beecham J, Mandalia S, et al. What is the cost of getting the price wrong? J Public Health Med 1999; 21: 311–7

  10. 10.

    Heerey A, McGowan B, Ryan M, et al. Microcosting versus DRGs in the provision of cost estimates for use in pharmacoeconomic evaluation. Expert Rev Pharmacoeconomics Outcomes Res 2002; 2: 29–33

  11. 11.

    Shwartz M, Young DW, Siegrist R. The ratio of costs to charges: how good a basis for estimating costs? Inquiry 1995; 32: 476–81

  12. 12.

    Rittenhouse BE, Dulissec B, Stinnett AA. At what price significance? The effect of price estimates on statistical inference in economic evaluation. Health Econ 1999; 8: 213–19

  13. 13.

    Reed SD, Friedman JY, Gnanasakthy A, et al. Comparison of hospital costing methods in an economic evaluation of a multinational clinical trial. Int J Technol Assess Health Care 2003; 19: 395–406

  14. 14.

    Chumney ECG, Biddle AK, Simpson KN, et al. The effect of cost construction based on either DRG or ICD-9 codes or risk group stratification on the resulting cost-effectiveness ratios. Pharmacoeconomics 2004; 22 (18): 1209–16

  15. 15.

    Taira DA, Seto TB, Siegrist R, et al. Comparison of analytic approaches for the economic evaluation of new technologies alongside multicenter clinical trials. Am Heart J 2003;145: 452–58

  16. 16.

    Goeree R, Gafni A, Hannah M, et al. Hospital selection for unit cost estimates in multicentre economic evaluations: does the choice of hospitals make a difference? Pharmacoeconomics 1999; 15 (6): 561–72

  17. 17.

    Schulman K, Burke J, Drummond M, et al. Resource costing for multinational neurologic clinical trials: methods and results. Health Econ 1998; 7: 629–38

  18. 18.

    Glick HA, Orzol SM, Tooley JF, et al. Design and analysis of unit cost estimation studies: how many hospital diagnoses? How many countries? Health Econ 2003; 12: 517–27

  19. 19.

    Thompson SG, Barber JA. How should cost data in pragmatic randomised trials be analysed? BMJ 2000; 320: 1197–200

  20. 20.

    Ramsey S, Willke R, Briggs A, et al. Best practices for economic analysis alongside clinical trials: an ISPOR RCT–CEA Task Force report. Value Health 2005; 8: 521–33

  21. 21.

    Hurley J. An overview of normative economics of the health sector. In: Culyer AJ, Newhouse JP, editors. Handbook of health economics, vol. 1A. Amsterdam: Elsevier Science BV, 2000; 61: 55–118

  22. 22.

    Duan N. Smearing estimate: a nonparametric retransformation method. J Am Stat Assoc 1983; 78: 605–10

  23. 23.

    Gayan AK. Significance of difference between the means of two non–normal samples. Biometrika 1950; 37: 399–408

  24. 24.

    Pearson ES, Please NW. Relation between the shape of population distribution and the robustness of four simple test statistics. Biometrika 1975; 62: 223–41

  25. 25.

    Geary RC. Testing for normality. Biometrika 1947; 36: 353–69

  26. 26.

    Doshi JA, Glick HA, Polsky D. Analyses of cost data in economic evaluations conducted alongside randomized controlled trials. Value Health 2006; 9 (5): 334–40

  27. 27.

    Mullahy J. Much ado about two: reconsidering retransformation and the two-part model in health econometrics. J Health Econ 1998; 17 (3): 247–81

  28. 28.

    Barber JA, Thompson SG. Analysis of cost data in randomized trials: an application of the non–parametric bootstrap. Stat Med 2000; 19: 3219–36

  29. 29.

    Manning WG. The logged dependent variable, heteroscedasticity, and the retransformation problem. J Health Econ 1998; 17 (3): 283–95

  30. 30.

    Davison AC, Hinkley DV. Bootstrap methods and their application. Cambridge: Cambridge University Press, 1997

  31. 31.

    O’Hagan A, Stevens JW. Assessing and comparing costs: how robust are the bootstrap and methods based on asymptotic normality? Health Econ 2003; 12: 33–49

  32. 32.

    Conigliani C, Tancredi A. Semi-parametric modelling for costs of health care technologies. Stat Med 2005; 24 (20): 3171–84

  33. 33.

    Briggs A, Nixon R, Dixon S, et al. Parametric modelling of cost data: some simulation evidence. Health Econ 2005; 11: 421–28

  34. 34.

    Gail MH, Wieand S, Pianados S. Biased estimates of treatment effect in randomized experiments with nonlinear regressions and omitted covariates. Biometrika 1984; 71: 431–44

  35. 35.

    Senn SJ. Covariate imbalance and random allocation in clinical trials. Stat Med 1989; 8: 467–75

  36. 36.

    Vazquez-Polo FJ, Negrin Hernanez MA, Gonzalez Lopez-Valcarcel B. Using covariates to reduce uncertainty in the economic evaluation of clinical trial data. Health Econ 2005; 14: 545–57

  37. 37.

    Newhous JM. Estimation efficiency with omitted covariates in generalized linear models. J Am Statist Assoc 1998; 93 (443): 1124–29

  38. 38.

    Nixon RM, Thompson SG. Methods for incorporating covariate adjustment, subgroup analysis and between-centre differences into cost-effectiveness evaluations. Health Econ 2005; 14: 1217–29

  39. 39.

    Newhous JM, Jewell NP. A geometric approach to assess bias due to omitted covariates in generalized linear models. Biometrika 1993; 80: 807–16

  40. 40.

    Manning WG, Mullahy J. Estimating log models: to transform or not to transform? J Health Econ 2001; 20: 461–94

  41. 41.

    Nelder JA, Wedderburn RWM. Generalized linear models. J Roy Statist Soc A 1972; 135: 370–84

  42. 42.

    McCullagh P, Nelder JA. Generalized linear models. 2nd ed. London: Chapman & Hall, 1989

  43. 43.

    Blough DK, Madden CW, Hornbrook MC. Modeling risk using generalized linear models. J Health Econ 1999; 18: 153–71

  44. 44.

    Barber J, Thompson S. Multiple regression of cost data: use of generalized linear models. J Health Serv Res Policy 2004; 9: 197–9

  45. 45.

    Basu A, Rathouz PJ. Estimating marginal and incremental effects on health outcomes using flexible link and variance function models. Biostatistics 2005; 6: 93–109

Download references

Acknowledgements

Support for this paper was received from NIH/NIDA grant R01 DA017221. No other sources of funding were used to assist in the preparation of this article. The authors have no conflicts of interest that are directly relevant to the content of this article.

Author information

Correspondence to Dr Daniel Polsky.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Polsky, D., Glick, H. Costing and Cost Analysis in Randomized Controlled Trials. Pharmacoeconomics 27, 179–188 (2009). https://doi.org/10.2165/00019053-200927030-00001

Download citation

Keywords

  • Ordinary Little Square
  • Medical Service
  • Ordinary Little Square Regression
  • Resource Type
  • Ordinary Little Square Model