Skip to main content

Something borrowed, something new: measuring hospital performance in the context of value based health care

Introduction

Following the publication of Porter and Teisberg’s “Redefining health care” in 2006 [1] the concept of value-based health care (VBHC) has become widely spread, albeit perhaps not fully understood in health care management circles [2]. A central component of the theory is that the overarching goal for health care delivery should be to improve the value to patients, with value here being defined as the health outcomes per dollar spent [3]. It is argued that this goal is something that all stakeholder could unite around. This is often expressed in the form of an equation (the value equation), with v being the value, o the outcomes and c the costs:

$$v=\frac{o}{c}$$
(1)

It should be noted that within the framework of VBHC, the intention has never been to actually calculate this ratio (as we shall see below, this is in fact not possible because we are not dealing with a single outcome)—it serves merely as an illustration of the fact that we can increase the value in different ways: We can decrease costs while maintaining the same outcomes and we can improve outcomes while keeping costs constant. Both of these scenarios would lead to value being increased.

There has been considerable attention paid to the numerator of the value equation, i.e. outcomes measurement. One area in particular is the developments of standardized outcomes sets in different diseases to allow for comparison between clinics or hospitals. Some recent examples include heart failure, chronic kidney disease and pregnancy and childbirth [4,5,6].Footnote 1 The standardized outcome sets contain very different types of outcome measures within a single set. Examples from coronary artery disease include among other things all-cause mortality, readmission to hospital, procedures undertaken and patient-reported quality of life [8]. It can be observed that the outcomes are therefore multidimensional and not seen as a single measure of health which is also explicit in the theoretical work [3]. A better way of expressing the value equation may therefore be utilizing a vector notation where o is a vector of patient relevant outcomes:

$${\varvec{v}}=\frac{{\varvec{o}}}{c}$$
(2)

Mathematically this would give us a vector of values (v) of vastly different interpretations depending on the outcomes used which probably makes no-one happy. This again emphasizes why the value equation is a theoretical construct for illustrative purposes and not something that we are meant to actually estimate.

In practice, the multidimensional nature of the outcomes unfortunately makes both determining if more value is being produced over time, and which of a set of providers deliver the highest value difficult to determine just considering the value equation. The results for some outcomes may have improved over time while others have worsened, and some providers may be better at a subset of outcomes while underperforming in a different subset. These may be important questions for a health care payer wishing to compare different providers for example. To answer these questions, we need to move from the (relatively) new area of value-based health care, to the old and well-established area of efficiency and productivity research.

Key efficiency and productivity measurement concepts

To measure how well someone is doing in the production of health care we need something to compare it with, i.e. we need a representation of the best practice technology to measure the performance against. This best practice technology, or frontier, can be constructed from data on inputs producing outputs. Inputs could e.g. be number of hours worked by physicians, nurses, and other staff. It could also be medical equipment, pharmaceuticals, premises, or any other resource needed to produce the outputs. Outputs could e.g. be the number of patients treated, surgery procedures, life years gained, quality of life, and/or any other quantifiable medical procedures or outcomes, or indeed standardized value sets as defined within the context of VBHC. In addition to inputs and outputs the model can include input prices, output prices, and quality indicators to address different aspects of efficiency and productivity.

A Decision-Making Unit’s (DMU) ability to produce outputs from inputs will be measured against other DMUs. DMUs could e.g. be countries, hospitals, specialist care units, primary care units, or patients. The important thing for a DMU is that it should have control of the way it produces outputs. And there need to be other DMUs to compare it with.

Efficiency can be calculated using data envelopment analysis (DEA) based on distance functions. Distance functions are frequently used in production and utility theory and were independently introduced in different forms by Debreu, Koopmans, Malmquist and Shephard [9,10,11,12]. See Shephard or Färe for definitions and properties of the distance functions [13, 14].

The distance functions could be either input based, or output based. An input-based distance function approach tries to minimize the use of inputs given the produced outputs, and an output-based approach tries to maximize the outputs produced with the given inputs. Here the distance function is exemplified using an input-based approach to demonstrate how the performance of a DMU could be measured.

The distance function has many appealing characteristics, such as e.g. being able to simultaneously handle many inputs producing many outputs. It is not necessary to assume a specific functional form and it is independent of unit of measurement which makes it possible to keep both inputs and outputs in their natural units of measurement. Changing the unit of measurement from e.g. hours to days will have no effect on the calculated result. Not having to convert the unit of measurements into a single monetary unit, as e.g. when computing the incremental cost effectiveness ratio, is a major advantage in the European health care sector where market prices often are nonexistent. It is also attractive from a practical stand-point within the context of VBHC: Fundamental to the theory of VBHC is that in order to measure value we need to capture all costs that have impact on the outcomes. This means that we need to measure the costs of the full care delivery value chain. This is possible, e.g. through time-driven activity based costing, but may require a significant effort even if the entire care chain is located at a single facility [15]. Measuring resources independent of their valuation may make this a more accurate and less cumbersome task.

Still, there is nothing preventing having inputs, or outputs, in monetary units, but if prices are available it is often preferable to keep input and output quantities in their original units and add the corresponding prices as separate variables. This will allow for the calculation of allocative efficiency in addition to the technical efficiency.

The Farrell input-based measure of technical efficiency (TE), is the reciprocal of the input distance function and is illustrated in Fig. 1 [16]. The figure shows the concept of radial efficiency measurement using two inputs \(\left({x}_{1},{x}_{2}\right)\) producing a given output vector \(\left(y={y}_{1},\dots ,{y}_{M}\right)\) with a piecewise linear technology estimation. Input \({x}_{1}\) could e.g. be number of hours worked by physicians, and \({x}_{2}\) number of hours worked by nurses. The frontier of the input set \(L\left(y\right)\) is bounded by the isoquant \(I-\acute{I}\) and constructed by e.g. hospital (DMU) a and hospital b. These two hospitals are said to be technically efficient since they are on the frontier with an efficiency score of one. They are on the frontier since they use the least possible inputs to produce the same outputs. They do this with a different mix of inputs (physician hours, and nurse hours), but there is no other hospital doing this more technically efficient.

Fig. 1
figure1

The input set, and the measurement of input based technical and allocative efficiency

Hospital c on the other hand is located inside the input set \(L\left(y\right)\) and is less efficient in producing the outputs than hospitals a and b. The degree of technical efficiency of hospital c is measured as the deviation from the frontier corresponding to \(L\left(y\right)\) as the ratio 0ĉ/0c, where point ĉ represents a hypothetical hospital made up from a convex combination of the actual hospitals a and b.

Each hospital receives a technical efficiency score between zero and one, where a score of one indicates that this hospital is producing technically efficient compared to the other hospitals in the sample. A score below one indicates inefficiency and the magnitude of the score provides the degree of inefficiency.

If input prices are available, as illustrated by the price line Ṕ–P (representing the minimum cost) in Fig. 1, the allocative efficiency (AE) could be calculated in addition to the technical efficiency. The input based allocative efficiency is measured as the ratio 0ć/0ĉ and gauges to what extent hospital c uses a cost minimizing mix of inputs to produce the outputs. As with the input based technical efficiency score the measure is bounded between zero and one, where a score of one indicates an efficient observation.

The distance 0ć/0c in Fig. 1 represents the input based overall efficiency (OE) and equals the ratio between minimum cost, \({C}^{*}\), and actual observed cost, \(px={p}_{1}{x}_{1},\dots ,{p}_{N}{x}_{N}\). The overall efficiency can be calculated as the product of the technical and allocative efficiency scores, i.e. \(OE=TE\bullet AE\). The measure is bounded between zero and one and a cost-efficient use of inputs yields a score of one.

Linear programming (LP) can be used to construct the piecewise linear technology frontier and to compute the efficiency scores relative to this technology. As mentioned before, the result includes an efficiency score between zero and one for each DMU, where a score of one indicates an efficient DMU, and a score below one an inefficient one. Since the degree of inefficiency is measured radially it means that a technical efficiency score of e.g. 0.8 for hospital c in Fig. 1, is interpreted as 80% efficient and if hospital c would produce as well as its efficient pairs—it could reduce its use of inputs by 20% without any reduction of the produced outputs. In addition to producing in a technically inefficient way, hospital c is producing in an allocative inefficient way since it is using a too large \({\raise0.7ex\hbox{${x_{1} }$} \!\mathord{\left/ {\vphantom {{x_{1} } {x_{2} }}}\right.\kern-\nulldelimiterspace} \!\lower0.7ex\hbox{${x_{2} }$}}\) mix of inputs given the observed input prices illustrated by Ṕ–P. By moving to the same input mix as hospital b, hospital c would eliminate its allocative inefficiency. Tying this back to the reasoning within VBHC, we now have a straightforward way of ranking DMUs based on the value generated—a higher efficiency score indicates higher value generated.

What about changes over time then? The efficiency measures discussed so far, all deals with cross sectional data from one time period. In contrast productivity measurement is a multi-period measure used to measure the change in productivity from one period to the next and can be measured by a Malmquist productivity index [11]. Moreover, following Färe et al., the input-based Malmquist productivity index can be decomposed into an efficiency change component and a technology change component [17].

That is, the index makes it possible to measure how the efficiency of production changes over time, as well as how the technology (frontier) changes over time, see Fig. 2 for an illustration with an input-based Malmquist productivity measurement approach.

Fig. 2
figure2

The input-based Malmquist productivity index

The technology (frontier) shift is illustrated with a front in time t (generated with input data x1 and x2, and output data y from 2017), and a corresponding front in time t + 1 generated with data from 2018). The shift in technology (what is possible) is shown as the difference between the two frontiers, and in this example it can be seen that there is an improvement in technology (the frontier shifting inwards) so that the same output levels produced in 2017 can be produced with less resource use in 2018. The second component of the index, the possible change in efficiency between 2017 and 2018, is not illustrated in Fig. 2 but is included in all Malmquist productivity indices, see Färe et al. for a thoroughly exposition of the index characteristics [17].

Concluding remarks

In the preceding sections we have argued for the use of distance functions and the Malmquist productivity index to answer questions about value (in the terminology used in VBHC) when comparing different health care providers. A key feature is that this approach is independent of unit of measurement. The fact that we in any outcomes set have several potentially very different outcome measures does therefore not constitute a problem. Another attractive feature is that no explicit weighting of the outcomes is necessary—we can be agnostic as to their relative importance. The same independence of the unit of measurement that applied to outputs is true for inputs as well, which means that in the absence of reliable estimates of costs it is possible to use primary measures of resource quantities instead—with the important caveat that these need to be measured in a consisten way. This approach therefore alleviates some of the challenges arising when trying to measure costs by making it possible to rely on more easily attained measures of resource quantities.

Looked at it this way, it turns out that the concept of value corresponds well with the traditional concepts of efficiency and productivity There are two distinguishing features that could differentiate an analysis rooted in the theory of value-based health care from the traditional efficiency analysis. First, it is very explicit in the theoretical underpinnings of value-based health care that the outcomes of interest should be patient relevant ones. In reflection of this, there has always been a role for the patient voice in the development of standardized outcomes sets in this context. Second, to maximize value it is necessary that the entire care delivery value chain is captured when costs are estimated. Traditional analysis of efficiency and productivity is thus a wider concept, for which analyses also tied to VBHC would constitute a subset. Hopefully this combination of the two traditions could help inform future discussion about value when put into practice.

Notes

  1. 1.

    There has of course many other initiatives dealing with standard outcomes sets that are not tied to value-based health care. One example is COMET, focusing on effectiveness in clinical trials [7].

References

  1. 1.

    Porter, M.E., Teisberg, E.O.: Redefining health care: creating value-based competition on results. Harvard Business School Press, Boston (2006)

    Google Scholar 

  2. 2.

    Fredriksson, J.J., Ebbevi, D., Savage, C.: Pseudo-understanding: an analysis of the dilution of value in healthcare. BMJ Qual. Saf. 24(7), 451–457 (2015). https://doi.org/10.1136/bmjqs-2014-003803

    Article  PubMed  Google Scholar 

  3. 3.

    Porter, M.E.: What is value in health care? N. Engl. J. Med. 363(26), 2477–2481 (2010). https://doi.org/10.1056/NEJMp1011024

    CAS  Article  PubMed  Google Scholar 

  4. 4.

    Burns, D.J.P., Arora, J., Okunade, O., Beltrame, J.F., Bernardez-Pereira, S., Crespo-Leiro, M.G., Filippatos, G.S., Hardman, S., Hoes, A.W., Hutchison, S., Jessup, M., Kinsella, T., Knapton, M., Lam, C.S.P., Masoudi, F.A., McIntyre, H., Mindham, R., Morgan, L., Otterspoor, L., Parker, V., Persson, H.E., Pinnock, C., Reid, C.M., Riley, J., Stevenson, L.W., McDonagh, T.A.: International Consortium for Health Outcomes Measurement (ICHOM): standardized patient centered outcomes measurement set for heart failure patients. JACC Heart Fail. (2019). https://doi.org/10.1016/j.jchf.2019.09.007

    Article  PubMed  Google Scholar 

  5. 5.

    Nijagal, M.A., Wissig, S., Stowell, C., Olson, E., Amer-Wahlin, I., Bonsel, G., Brooks, A., Coleman, M., Karalasingam, S.D., Duffy, J.M., Flanagan, T., Gebhardt, S., Greene, M.E., Groenendaal, F., Jeganathan, J.R.J., Kowaliw, T., Lamain-de-Ruiter, M., Main, E., Owens, M., Petersen, R., Reiss, I., Sakala, C., Speciale, A.M., Thompson, R., Okunade, O., Franx, A.: Standardized outcome measures for pregnancy and childbirth, an ICHOM proposal. BMC Health Serv. Res. 18(1), 953 (2018). https://doi.org/10.1186/s12913-018-3732-3

    Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Verberne, W.R., Das-Gupta, Z., Allegretti, A.S., Bart, H.A., Van Biesen, W., García-García, G., Gibbons, E., Parra, E., Hemmelder, M.H., Jager, K.J., Ketteler, M., Robert, C., Al Rohani, M., Salt, M.J., Stopper, A., Terkivatan, T., Tuttle, K.R., Yang, C.W., Wheeler, D.C., Bos, W.J.W.: Development of an international standard set of value-based outcome measures for patients with chronic kidney disease: a report of the International Consortium for Health Outcomes Measurement (ICHOM) CKD Working Group. Am. J. Kidney Dis. 73(3), 372–384 (2019). https://doi.org/10.1053/j.ajkd.2018.10.007

    Article  PubMed  Google Scholar 

  7. 7.

    Kirkham, J.J., Davis, K., Altman, D.G., Blazeby, J.M., Clarke, M., Tunis, S., Williamson, P.R.: Core outcome set-STAndards for development: the COS-STAD recommendations. PLoS Med. 14(11), e1002447 (2017). https://doi.org/10.1371/journal.pmed.1002447

    Article  PubMed  PubMed Central  Google Scholar 

  8. 8.

    McNamara, R.L., Spatz, E.S., Kelley, T.A., Stowell, C.J., Beltrame, J., Heidenreich, P., Tresserras, R., Jernberg, T., Chua, T., Morgan, L., Panigrahi, B., Rosas Ruiz, A., Rumsfeld, J.S., Sadwin, L., Schoeberl, M., Shahian, D., Weston, C., Yeh, R., Lewin, J.: Standardized outcome measurement for patients with coronary artery disease: consensus from the International Consortium for Health Outcomes Measurement (ICHOM). J. Am. Heart Assoc. (2015). https://doi.org/10.1161/JAHA.115.001767

    Article  PubMed  PubMed Central  Google Scholar 

  9. 9.

    Debreu, G.: The coefficient of resource utilization. Econom. J. Econom. Soc. 19(3), 273–292 (1951)

    Google Scholar 

  10. 10.

    Koopmans, T. C.: An analysis of productions an efficient combination of activities. In: Koopmans TC (ed) Activity Analysis of Production and Allocation, Cowles Commission for Research in Economics, Monograph, no. 13. Willey, New York (1951)

  11. 11.

    Malmquist, S.: Index numbers and indifference surfaces. Trabajos de Estadistica y de Jnvestigacion Operativa 4(2), 209–242 (1953)

    Article  Google Scholar 

  12. 12.

    Shephard, R.W.: Cost and production functions. Princeton University Press, Princeton (1953)

    Google Scholar 

  13. 13.

    Färe, R.: Fundamentals of production theory, vol. 311. Lecture notes in economics an dmathematical systems. Springer, Berlin (1988)

  14. 14.

    Shephard, R.W.: Theory of cost and production functions. Princeton studies in mathematical economics, vol. 4. Princeton University Press, Princeton, NJ (1970)

  15. 15.

    Keel, G., Savage, C., Rafiq, M., Mazzocato, P.: Time-driven activity-based costing in health care: a systematic review of the literature. Health Policy 121(7), 755–763 (2017). https://doi.org/10.1016/j.healthpol.2017.04.013

    Article  PubMed  Google Scholar 

  16. 16.

    Farrell, M.J.: The measurement of productive efficiency. J. R. Stat. Soc. Ser. A 120, 253–281 (1957)

    Article  Google Scholar 

  17. 17.

    Färe, R., Grosskopf, S., Lindgren, B., Roos, P.: Productivity developments in Swedish Hospitals: a Malmquist output index approach. In: Charnes, A., Cooper, W.W., Lewin, A.Y., Seiford, L.M. (eds.) Data envelopment analysis: theory, methodology, and applications. Springer, Dordrecht (1989)

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Peter Lindgren.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lindgren, P., Althin, R. Something borrowed, something new: measuring hospital performance in the context of value based health care. Eur J Health Econ 22, 851–854 (2021). https://doi.org/10.1007/s10198-020-01209-5

Download citation

JEL Classification

  • I10
  • D24