Rethinking How We Measure Costs in Implementation Research



Hospitals and other health care delivery organizations are sometimes resistant to implementing evidence‐based programs, citing unknown budgetary implications.


In this paper, I discuss challenges when estimating health care costs in implementation research.


A case study with intensive care units highlights how including fixed costs can cloud a short‐term analysis.





Main Measures

Health care costs, charges and payments.

Key Results

Cost data should accurately reflect the opportunity costs for the organization(s) providing care. Opportunity costs are defined as the benefits foregone because the resources were not used in the next best alternative. Because there is no database of opportunity costs, cost studies rely on accounting data, charges, or payments as proxies. Unfortunately, these proxies may not reflect the organization’s opportunity costs, especially if the goal is to understand the budgetary impact in the next few years.


Implementation researchers should exclude costs that are fixed in the time period of observation because these assets (e.g., space) cannot be used in the next best alternative. In addition, it is common to use costs from accounting databases where we implicitly assume health care providers are uniformly efficient. If providers are not operating efficiently, especially if there is variation in their efficiency, then this can create further problems. Implementation scientists should be judicious in their use of cost estimates from accounting data, otherwise research results can misguide decision makers.

This is a preview of subscription content, access via your institution.


  1. 1.

    Owens DK, Qaseem A, Chou R, Shekelle P. High-value, cost-conscious health care: concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Ann Intern Med. 2011;154(3):174-180.

    Article  Google Scholar 

  2. 2.

    Neumann P, Sanders G, Russell L, Siegel J, Ganiats T, eds. Cost-effectiveness in Health and Medicine. Oxford University Press; 2016.

  3. 3.

    Tan-Torres Edejer T, Baltussen R, Adam T, et al., eds. WHO Guide to Cost Effectiveness Analysis. Geneva, Switzerland: World Health Oranization; 2003.

    Google Scholar 

  4. 4.

    Shwartz M, Young DW, Siegrist R. The ratio of costs to charges: how good a basis for estimating costs? Inquiry. 1995;32(4):476-481.

    PubMed  Google Scholar 

  5. 5.

    Palmer S, Raftery J. Economic Notes: opportunity cost. Br Med J. 1999;318(7197):1551-1552.

    CAS  Article  Google Scholar 

  6. 6.

    Gold MR, Siegel JE, Russell LB, Weinstein MC, eds. Cost-Effectiveness in Health and Medicine. Oxford: Oxford University Press; 1996.

    Google Scholar 

  7. 7.

    Luce BR, Manning WG, Siegel JE, Lipscomb J. Estimating Costs in Cost Effectiveness Analysis. In: Gold MR, Siegel JE, Russell LB, Weinstein MC, eds. Cost-Effectiveness in Health and Medicine. New York: Oxford University Press; 1996:176-213.

    Google Scholar 

  8. 8.

    Basu A. Estimating Costs and Valuations of Non-Health Benefits in Cost-Effectiveness Analysis. In: Neuman P, Sanders G, Russell LB, Siegel JE, Ganiats TG, eds. Cost-Effectiveness in Health and Medicine. New York: Oxford University Press; 2016:201-235.

    Google Scholar 

  9. 9.

    Halpern NA, Pastores SM. Critical care medicine in the United States 2000-2005: an analysis of bed numbers, occupancy rates, payer mix, and costs. Crit Care Med. 2010;38(1):65-71.

    Article  Google Scholar 

  10. 10.

    Gooch RA, Kahn JM. ICU bed supply, utilization, and health care spending: an example of demand elasticity. JAMA. 2014;311(6):567-568.

    CAS  Article  Google Scholar 

  11. 11.

    Chen LM, Render M, Sales A, Kennedy EH, Wiitala W, Hofer TP. Intensive care unit admitting patterns in the Veterans Affairs health care system. Arch Intern Med. 2012;172(16):1220-1226.

    Article  Google Scholar 

  12. 12.

    Chang DW, Dacosta D, Shapiro MF. Priority Levels in Medical Intensive Care at an Academic Public Hospital. JAMA Intl Med. 2017;177(2):280-281.

    Article  Google Scholar 

  13. 13.

    Dasta JF, McLaughlin TP, Mody SH, Piech CT. Daily cost of an intensive care unit day: the contribution of mechanical ventilation. Crit Care Med. 2005;33(6):1266-1271.

    Article  Google Scholar 

  14. 14.

    Chiou H, Jopling JK, Scott JY, et al. Detecting organisational innovations leading to improved ICU outcomes: a protocol for a double-blinded national positive deviance study of critical care delivery. BMJ Open. 2017;7(6):e015930.

    Article  Google Scholar 

  15. 15.

    Arrow K. Uncertainty and the welfare economics of medical care. AER. 1963;53(5):941-973.

    Google Scholar 

  16. 16.

    Garber AM, Phelps CE. Economic foundations of cost-effectiveness analysis. J Health Econ. 1997;16(1):1-31.

    CAS  Article  Google Scholar 

  17. 17.

    Weinstein MC, Stason WB. Foundations of cost-effectiveness analysis for health and medical practices. N Engl J Med. 1977;296(13):716-721.

    CAS  Article  Google Scholar 

  18. 18.

    Drummond MF, O’Brien B, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. Second Edition ed. Oxford: Oxford University Press; 1997.

    Google Scholar 

  19. 19.

    Bryan S, Mitton C, Donaldson C. Breaking the addiction to technology adoption. Health Econ. 2014;23(4):379-383.

    Article  Google Scholar 

  20. 20.

    Hwang J, Christensen CM. Disruptive innovation in health care delivery: a framework for business-model innovation. Health Aff. 2008;27(5):1329-1335.

    Article  Google Scholar 

  21. 21.

    Humphreys K, Wagner TH, Gage M. If substance use disorder treatment more than offsets its costs, why don’t more medical centers want to provide it? A budget impact analysis in the Veterans Health Administration. J Subst Abuse Treat. 2011;41(3):243-251.

    Article  Google Scholar 

  22. 22.

    Ettner SL, Huang D, Evans E, et al. Benefit-cost in the California treatment outcome project: does substance abuse treatment “pay for itself”? Health Serv Res. 2006;41(1):192-213.

    Article  Google Scholar 

  23. 23.

    Basu A. Estimating Costs and Valuations of Non-Health Benefits in Cost-Effectiveness Analysis In: Neumann P, Sanders G, Russell LB, Siegel JE, Ganiats TG, eds. Cost-Effectiveness in Health and Medicine. Oxford University Press; 2016.

  24. 24.

    Barnett PG. An improved set of standards for finding cost for cost-effectiveness analysis. Med Care. 2009;47(7 Suppl 1):S82-88.

    Article  Google Scholar 

  25. 25.

    Kaplan RS, Porter ME. How to solve the cost crisis in health care. Harv Bus Rev. 2011;89(9):46-52, 54, 56-61 passim.

    PubMed  Google Scholar 

  26. 26.

    Adang EM, Wensing M. Economic barriers to implementation of innovations in health care: is the long run-short run efficiency discrepancy a paradox? Health Policy. 2008;88(2-3):236-242.

    Article  Google Scholar 

  27. 27.

    Valdmanis VG, Rosko MD, Mutter RL. Hospital quality, efficiency, and input slack differentials. Health Serv Res. 2008;43(5 Pt 2):1830-1848.

    Article  Google Scholar 

  28. 28.

    Bloom N, Van Reenen J. Measuring and Explaining Management Practices Across Firms and Countries. Q J Econ. 2007;122(4):1351-1408.

    Article  Google Scholar 

  29. 29.

    Neumann PJ, Stone PW, Chapman RH, Sandberg EA, Bell CM. The quality of reporting in published cost-utility analyses, 1976-1997. Ann Intern Med. 2000;132(12):964-972.

    CAS  Article  Google Scholar 

  30. 30.

    Mauskopf J, Sullivan S, Annemans L. Principles of Good Practice for Budget Impact Analysis: Report of the ISPOR Task Force on Good Research Practices – Budget Impact Analysis. Value Health. 2007;10:336-347.

    Article  Google Scholar 

  31. 31.

    Sullivan SD, Mauskopf JA, Augustovski F, et al. Budget impact analysis-principles of good practice: report of the ISPOR 2012 Budget Impact Analysis Good Practice II Task Force. Value Health. 2014;17(1):5-14.

    Article  Google Scholar 

  32. 32.

    Roberts SLE, Healey A, Sevdalis N. Use of health economic evaluation in the implementation and improvement science fields-a systematic literature review. Implement Sci. 2019;14(1):72.

    Article  Google Scholar 

  33. 33.

    Reeves P, Edmunds K, Searles A, Wiggers J. Economic evaluations of public health implementation-interventions: a systematic review and guideline for practice. Public Health. 2019;169:101-113.

    CAS  Article  Google Scholar 

  34. 34.

    Dopp AR, Mundey P, Beasley LO, Silovsky JF, Eisenberg D. Mixed-method approaches to strengthen economic evaluations in implementation research. Implement Sci. 2019;14(1):2.

    Article  Google Scholar 

  35. 35.

    Smith M, Saunders R, Stuckhardt L, McGinnis JM, eds. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, D.C.: The National Academies Press; 2012. Institute of Medicine, ed.

    Google Scholar 

Download references


I thank Angela So for her comments and suggestions. This paper was based on a presentation made on June 3, 2019, to the Cost and Implementation working group.


Funding was provided by the VA Department of Veterans Affairs to Todd Wagner through a Research Career Scientist Award (RCS 17-154).

Author information



Corresponding author

Correspondence to Todd H. Wagner PhD.

Ethics declarations

Conflict of Interest

The author declares that he does not have a conflict of interest

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wagner, T.H. Rethinking How We Measure Costs in Implementation Research. J GEN INTERN MED 35, 870–874 (2020).

Download citation


  • costs and cost analysis
  • budgets
  • cost-benefit analysis
  • economic models