Skip to main content
SpringerLink
Log in

Incorporating quality into the measurement of hospital efficiency: a double DEA approach

  • Published:
Journal of Productivity Analysis Aims and scope Submit manuscript

Abstract

This paper uses “double DEA” to assess how accounting for quality influences DEA technical efficiency scores of a sample of 1,074 US hospitals. In the first use of DEA, quality indices are estimated using a variety of process and outcome measures of quality. In the second use of DEA, technical efficiency is assessed while controlling for quality. A variety of DEA quality indices and a DEA variety of efficiency models are compared to determine how the treatment of quality influences findings regarding technical efficiency. Controlling for efficiency does matter, with outcome measures having an apparently greater impact than process measures. Given the call for improved quality and better cost containment, controlling for quality is an important contribution to efficiency analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. There is some empirical evidence to support this; for example, Maniadakis et al. (1999) found that following the reform of the National Health Service, the productivity and quality of a sample of Scottish hospitals followed opposite trends. Others, however (e.g., Clement et al. 2008), have found that technical efficiency and quality are positively related.

  2. Historically, CMS hospital payments reflected processes; under the P4P program outcomes also affect payments.

  3. Given appropriate data, measures of access as well as quality could be included in the efficiency estimates. At this we are not aware of any reliable measures of access to care or a way to observe non-patients who do not receive care for lack of access. Improvements in efficiency while controlling for quality should reduce costs and improve access to care.

  4. Medicaid is a means-tested health care program funded for individuals and families. It is jointly by the U.S. federal and state governments and is administered by the states. It was created in 1965.

  5. A similar system was implemented for outpatient services in 2000.

  6. The payment amounts are adjusted for regional input price differences and complex cases that require extreme amounts of resources to treat.

  7. Alternatively, providers could choose the treatment option with highest reimbursement relative to costs incurred when an illness has multiple treatment options (i.e., “upcoding”).

  8. The outcome data were graciously provided by HealthGrades, Inc. for this research; the data cover the twelve month period June 2004–May 2005.

  9. http://www.hospitalcompare.hhs.gov/hospital-search.aspx?AspxAutoDetectCookieSupport=1; 2005 measures were used.

  10. The Hospital Quality Alliance, a public–private group that collects and shares information on hospital quality, includes information on these three conditions as their “starter kit” for measuring hospital quality. CMS’s Hospital Quality Initiative initially focused on these three conditions as well.

  11. At the time HealthGrades, Inc. provided the data for this study predicted and observed mortality and complication rates were available to consumers on their website. Since that time they have converted to a star system to report the relative quality of outcomes for hospital patients and do not show actual mortality and/or complication rates. For more information see www.healthgrades.com.

  12. It is also possible that higher quality care could result in greater efficiency and lower costs. For example, reducing adverse drug events would increase the quality of care and could save hospitals millions of dollars (Agency for Health care Research and Quality [AHRQ], 2001).

  13. Changing the weights in DEA will change the values of the quality indices. Weight restrictions (see, e.g., Dyson and Thanassoulis, 1988, or Wong and Beasley, 1990) could be to examine how sensitive quality indices are to the choice of weights. Such an exercise is beyond the scope of this paper.

  14. Factor analysis was run on the full sample and again on each half when the sample was randomly divided. Results of the factor analysis are available from the authors on request.

  15. This approach has also been used to construct quality of life indicators (e.g., Hashimoto et al., 1993; González et al., 2011).

  16. As noted above, changing or restricting weights would change DEA scores. One reason for restricting weights would be to incorporate value judgments into the analysis (Allen et al., 1997).

  17. The correlations between the “raw” DEA scores and the bias corrected scores were very high.

  18. Though not reported, Pearson and Spearman correlations were calculated separately for each quartile of the data; in all cases, the correlations remained statistically significant.

  19. All three labor categories (excluding residents) were proportionally adjusted for hospitals that operate a long-term care or skilled nursing facility in conjunction with the hospital. AHA data report skilled nursing beds separately, but labor is aggregated. The analysis was run using adjusted and unadjusted labor values with results being qualitatively equivalent. The results reported here use adjusted labor inputs.

  20. The CMI reflects the severity of patients seen by a hospital throughout the year relative to the “average” patient for all hospitals. Adjusting for case mix creates a “level playing field” when evaluating hospital performance. For more information, see Grosskopf and Valdmanis (1993).

  21. “Compliance competition” can create undesirable consequences. For example, Metersky et al. (2006) documented that some patients receive antibiotics before a conclusive diagnosis is made to ensure the hospital complies with care guidelines that all patients ultimately diagnosed with pneumonia receive antibiotics within 4 h of arrival. This maximizes compliance rates, but is a concern of hospitals with public disclosure of compliance rates.

  22. The Medicare claims data researchers use to calculate mortality and complication rates are also publicly available; however, relatively few potential hospital patients are likely to analyze the claims data to determine quality.

  23. If the quality of care for fairly serious conditions is not closely correlated with the quality of more minor procedures, which would be consistent with the findings of DesHarnais et al. (1991), then the quality indices used here would be more appropriate for inpatient services and less appropriate for outpatient services.

  24. The relative inefficiency of teaching hospitals may be due to the fact that “teaching” output of these hospitals is not included in the analysis.

  25. For teaching status, similar results, both qualitatively and in terms of statistical significance, were obtained when the ratio was based on QI2 or QI3 rather than QI1. Quantitatively, the results were very similar when QI2 was used to form the ratio rather than QI1. When QI3 was used to form the ratio, the means for both teaching and non-teaching hospitals were smaller, but still above 1.

  26. For ownership status, similar results, both qualitatively and in terms of statistical significance, were obtained when the ratio was based on QI2 rather than QI1. Quantitatively, the results were very similar when QI2 was used to form the ratio rather than QI1. When QI3 was used to form the ratio, the means were smaller for all three ownership forms but still above 1. In the case of using QI3, the ratio was statistically significantly higher for the two non-for-profit (private and public) than for the (private) for-profit hospitals.

  27. Physician quality is likely related to how they are compensated; i.e., physicians responds to incentives. For a thorough discussion of payment incentives offered to physicians, see Robinson (2001).

  28. For example, the Medicare Prescription Drug, Improvement, and Modernization Act of 2003 includes a provision that hospitals failing to provide data for ten specific quality indicators will receive a 0.4 percentage point smaller annual payment increase compared to hospitals that do submit the data. Increasingly, third party payers are including financial incentives to encourage higher quality. For example, to learn about Blue Cross Blue Shield’s hospital pay-for-performance plans see http://www.bcbsm.com/provider/value_partnerships/hpp/.

References

  • Agency for Health care Research and Quality (2001) Reducing and preventing adverse drug events to decrease hospital costs. Res Action (1) (AHRQ Publication Number 01-0020). http://www.ahrq.gov/qual/aderia/aderia.htm

  • Allen R, Athanassopoulos AD, Dyson RG, Thanassoulis E (1997) Weights restrictions and value judgements in DEA: evolution, development and future directions. Ann Oper Res 73:13–34

    Article  Google Scholar 

  • Allison JJ, Kiefe CI, Weissman NW, Person SD, Rousculp M, Canto JG, Sejong Bae O, Williams D, Farmer R, Centor RM (2000) Relationship of hospital teaching status with quality of care and mortality for medicare patients with acute MI. J Am Med Assoc 284:1256–1262

    Article  Google Scholar 

  • Banker RD, Charnes A, Cooper WW (1984) Models for estimating technical and scale inefficiencies in data envelopment analysis. Manage Sci 30:1078–1092

    Article  Google Scholar 

  • Carey K (2003) Hospital cost efficiency and system membership. Inquiry 40:25–38

    Article  Google Scholar 

  • Carey K, Burgess JF Jr (1999) On measuring the hospital cost-quality trade-off. Health Econ 8:509–520

    Article  Google Scholar 

  • Cherchye L (2001) Using data envelopment analysis to assess macroeconomic policy performance. Appl Econ 33:407–416

    Article  Google Scholar 

  • Chirikos TN, Sear AM (2000) Measuring hospital efficiency: a comparison of two approaches. Health Serv Res 34:1389–1408

    Google Scholar 

  • Clement JP, Valdmanis VG, Bazzoli GJ, Zhao M, Chukmaitov A (2008) Is more better? An analysis of hospital outcomes and efficiency with a DEA model of output congestion. Health Care Manag Sci 11:67–77

    Article  Google Scholar 

  • Deily ME, McKay NL (2006) Cost inefficiency and mortality rates in florida hospitals. Health Econ 15:419–431

    Article  Google Scholar 

  • DesHarnais S, McMahon LF Jr, Wroblewski R (1991) Measuring outcomes of hospital care using multiple risk-adjusted indexes. Health Serv Res 26:425–445

    Google Scholar 

  • Donabedian A (1978) The quality of medical care: methods for assessing and monitoring the quality of care for research and for quality assurance programs. Science 200:856–864

    Article  Google Scholar 

  • Doyle JR, Green RH (1991) Comparing products using data envelopment analysis. Omega 19:631–638

    Article  Google Scholar 

  • Dyson RG, Thanassoulis E (1988) Reducing weight flexibility in data envelopment analysis. J Operat Res Soc 39:563–576

    Google Scholar 

  • Farsi M (2004) Changes in hospital quality after conversion in ownership status. Int J Health Care Finance Econ 4:211–230

    Article  Google Scholar 

  • Ferrier GD, Valdmanis V (1996) Rural hospital performance and its correlates. J Prod Anal 7:63–80

    Article  Google Scholar 

  • Fuchs VR (1999) Health care for the elderly: how much? Who will pay for it? Health Aff 18:11–21

    Article  Google Scholar 

  • González E, Cárcaba A, Ventura J, García J (2011) Measuring quality of life in Spanish municipalities. Local Gov Stud 37:171–197

    Article  Google Scholar 

  • Grosskopf S, Valdmanis V (1993) Evaluating hospital performance with case-mix-adjusted outputs. Med Care 31:525–532

    Article  Google Scholar 

  • Halsteinli V, Kittlesen SA, Magnussen J (2010) Productivity growth in outpatient child and adolescent mental health services: the impact of case-mix adjustment. Soc Sci Med 70:439–446

    Article  Google Scholar 

  • Hartz AJ, Gottlieb MS, Kuhn EM, Rimm AA (1993) The relationship between adjusted hospital mortality and the results of peer review. Health Serv Res 27:765–777

    Google Scholar 

  • Hashimoto A, Ishikawa H (1993) Using DEA to evaluate the state of society as measured by multiple social indicators. Socio-Econ Plann Sci 27:257–268

    Article  Google Scholar 

  • HealthGrades (2006) The ninth annual Healthgrades hospital quality in America study. www.healthgrades.com

  • Hines S (2003) Using publicly reported data to choose a hospital: do different websites and different measures give consistent answers?. Delmarva Insights, Delmarva Foundation

    Google Scholar 

  • Hollingsworth B (2003) Non-parametric and parametric applications measuring efficiency in health care. Health Care Manag Sci 6:203–218

    Article  Google Scholar 

  • Hollingsworth B (2008) The measurement of efficiency and productivity of health care delivery. Health Econ 17:1107–1128

    Article  Google Scholar 

  • Hollingsworth B, Smith P (2003) Use of ratios in data envelopment analysis. Appl Econ Lett 10:733–735

    Article  Google Scholar 

  • Hollingsworth B, Dawson PJ, Maniadakis N (1999) Efficiency measurement of health care: a review of non-parametric methods and applications. Health Care Manag Sci 2:161–172

    Article  Google Scholar 

  • Jacobs R, Goddard M, Smith P (2005) How robust are hospital ranks based on composite performance measures? Med Care 43:1177–1184

    Article  Google Scholar 

  • Jha AK, Li Z, Orav EJ, Epstein AM (2005) Care in U.S. hospitals—the hospital quality alliance program. N Engl J Med 353:265–274

    Article  Google Scholar 

  • Keeler EB, Kahn KL, Draper D, Sherwood MJ, Rubenstein LV, Reinisch EJ, Kosecoff J, Brook RH (1990) Changes in sickness at admission following the introduction of the prospective payment system. J Am Med Assoc 264:1962–1968

    Article  Google Scholar 

  • Keeler EB, Rubenstein LV, Kahn KL, Draper D, Harrison ER, McGinty MJ, Rogers WH, Brook RH (1992) Hospital characteristics and quality of care. J Am Med Assoc 268:1709–1714

    Article  Google Scholar 

  • Kosecoff J, Kahn KL, Rogers WH, Reinisch EJ, Sherwood MJ, Rubenstein LV, Draper D, Roth CP, Chew C, Brook RH (1990) Prospective payment system and impairment at discharge: the ‘quicker-and-sicker’ story revisited. J Am Med Assoc 264:1980–1983

    Article  Google Scholar 

  • Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, Bratzler DW (2007) Public reporting and pay for performance in hospital quality improvement. N Engl Med 356:486–496

    Article  Google Scholar 

  • Lovell CAK (1995) Measuring the macroeconomic performance of the Taiwanese economy. Int J Prod Econ 39:165–178

    Article  Google Scholar 

  • Lovell CAK, Pastor JT, Turner JA (1995) Measuring macroeconomics performance in the OECD: a comparison of European and non-european countries. Eur J Oper Res 87:507–518

    Article  Google Scholar 

  • Magnussen J (1996) Efficiency measurement and the operationalization of hospital production. Health Serv Res 31:21–37

    Google Scholar 

  • Maniadakis N, Hollingsworth B, Thanassoulis E (1999) The impact of the internal market on hospital efficiency, productivity and service quality. Health Care Manag Sci 2:75–85

    Article  Google Scholar 

  • Martin AB, Lassman D, Washington B, Catlin A, the National Health Expenditure Accounts Team (2012) Growth in US Health spending remained slow in 2010; health share of gross domestic product was unchanged from 2009. Health Aff 31:208–219

    Article  Google Scholar 

  • MedPAC (2010), A Data Book: health care spending and the medicare program. http://www.medpac.gov/documents/Jun10DataBookEntireReport.pdf

  • Metersky ML, Sweeney TA, Getzow MB, Siddiqui F, Nsa W, Bratzler DW (2006) Antibiotic Timing and diagnostic uncertainty in medicare patients with pneumonia: is it reasonable to expect all patients to receive antibiotics within 4 hours? Chest 130:16–21

    Article  Google Scholar 

  • Morey RC, Fine DJ, Loree SW, Retzlaff-Roberts DL, Tsubakitani S (1992) The trade-off between hospital cost and quality of care: an exploratory empirical analysis. Med Care 30:677–698

    Article  Google Scholar 

  • Mutter RL, Rosko MD, Wong HS (2008) Measuring hospital inefficiency: the effects of controlling for quality and patient burden of illness. Health Serv Res 43:1992–2013

    Article  Google Scholar 

  • Mutter R, Valdmanis V, Rosko M (2010) High versus lower quality hospitals: a comparison of environmental characteristics and technical efficiency. Health Serv Outcomes Res Method 10:134–153

    Article  Google Scholar 

  • Nayar P, Ozcan Y (2008) Data envelopment analysis comparison of hospital efficiency and quality. J Med Syst 32:193–199

    Article  Google Scholar 

  • Oh I, Lee J-D, Hwang S, Heshmati A (2010) Analysis of product efficiency in the Korean automobile market from a consumer’s perspective. Empirical Economics 38:119–137

    Article  Google Scholar 

  • Olesen OB, Petersen NC (1995) Incorporating quality into data envelopment analysis: a stochastic dominance approach. Int J Prod Econ 39:117–135

    Article  Google Scholar 

  • Olesen OB, Petersen NC (2002) The use of data envelopment analysis with probabilistic assurance regions for measuring hospital efficiency. J Prod Anal 17:83–109

    Article  Google Scholar 

  • Pear R (2006) A.M.A. to develop measure of quality of medical care. New York Times (February 22, 2006), A12

  • Pine M, Norusis M, Jones B, Rosenthal GE (1997) Predictions of hospital mortality rates: a comparison of data sources. Ann Intern Med 126:347–354

    Article  Google Scholar 

  • Robinson JC (2001) Theory and practice in the design of physician payment incentives. Milbank Q 79:149–177

    Article  Google Scholar 

  • Rogers WH, Draper D, Kahn KL, Keeler EB, Rubenstein LV, Kosecoff J, Brook RH (1990) Quality of care before and after implementation of the DRG-based prospective payment system: a summary of effects. J Am Med Assoc 264:1989–1994

    Article  Google Scholar 

  • Rosko MD (2001) Cost efficiency of US hospitals: a stochastic frontier approach. Health Econ 10:539–551

    Article  Google Scholar 

  • Shleifer A (1985) A theory of yardstick competition. RAND J Econ 16:319–327

    Article  Google Scholar 

  • Simar L, Wilson PW (1998) Sensitivity Analysis of Efficiency Scores: how to Bootstrap in Nonparametric Frontier Models. Manage Sci 44:49–91

    Article  Google Scholar 

  • Sloan FA, Picone GA, Taylor DH Jr, Chou S-Y (2001) Hospital ownership and cost and quality of care: is there a dime’s worth of difference? J Health Econ 20:1–21

    Article  Google Scholar 

  • Thomas J William, Hofer TP (1999) Accuracy of risk-adjusted mortality rate as a measure of hospital quality of care. Med Care 37:83–92

    Article  Google Scholar 

  • Thomasson M (2002) From sickness to health: the twentieth-century development of U.S. health insurance. Explor Econ Hist 39:233–253

    Article  Google Scholar 

  • Thompson RG, Singleton FD Jr, Thrall RM, Smith BA, Wilson M (1986) Comparative site evaluations for locating a high-energy physics lab in texas. Interfaces 16:35–49

    Article  Google Scholar 

  • Valdmanis VG, Rosko MD, Mutter RL (2008) Hospital quality, efficiency, and input slack differentials. Health Serv Res 43:1830–1848

    Article  Google Scholar 

  • Williams SC, Schmaltz SP, Morton DJ, Koss RG, Leob JM (2005) Quality of care in the U.S. hospitals as reflected by standardized measures, 2002–2004. N Engl J Med 353:255–264

    Article  Google Scholar 

  • Wong Y-HB, Beasley JE (1990) Restricting weight flexibility in data envelopment analysis. J Operat Res Soc 41:829–835

    Google Scholar 

  • Zweifel P, Breyer F, Kifmann M (2009) Health economics, 2nd edn. Springer, Berlin

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Economics, University of Arkansas, Fayetteville, AR, USA

    Gary D. Ferrier & Julie S. Trivitt

Authors
  1. Gary D. Ferrier
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Julie S. Trivitt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gary D. Ferrier.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ferrier, G.D., Trivitt, J.S. Incorporating quality into the measurement of hospital efficiency: a double DEA approach. J Prod Anal 40, 337–355 (2013). https://doi.org/10.1007/s11123-012-0305-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11123-012-0305-z

Keywords

JEL Classification

Search

Navigation