Abstract
Accounting for risk attitudes in medical decision making under uncertainty has attracted little research. A recent proposal recommended using the results of a costeffectiveness analysis to construct a costeffectiveness riskaversion curve (CERAC) to inform riskaverse decision makers choosing among healthcare programs with uncertain costs and effects. The CERAC is based on a riskadjusted performance measure widely used in financial economics called the Sortino ratio. This paper evaluates the CERAC based on the Sortino ratio, derives its various properties, discusses the implications of using it to inform decision making under uncertainty, and compares it with the expectedutility approach. Analytic formulae for the CERAC, relating it to the means and standard deviations of costs and effects of a healthcare program, are derived for both approaches. Compared with the expectedutility approach, the CERAC based on the Sortino ratio implicitly assumes that the decision maker is highly risk averse.
A few approaches have been suggested for accounting for risk attitudes in medical decision making under uncertainty. 
This paper evaluates the use of the Sortinoratiobased riskadjustment measure to inform decision making under uncertainty. 
Compared with the theoretically grounded expectedutility approach, the Sortinoratiobased riskadjustment measure implicitly assumes that the decision maker is highly risk averse. 
If the level of risk aversion of the decision maker is substantially lower than what is implicitly assumed, use of the Sortinoratiobased riskadjustment measure will result in recommending suboptimal healthcare programs. 
Introduction
The widely used framework of costeffectiveness analysis (CEA) assumes that decision makers are neutral toward risk in health and medicine [1]. Accounting for risk attitudes in medical decision making under uncertainty may be important [2]. Recognizing this, a few approaches have been suggested, some based on theoretical grounds, while others can be considered ad hoc recommendations. For example, a recommendation was made to the Joint Committee on Vaccination and Immunisation, an independent expert committee that advises the government of the United Kingdom on immunization programs, to require an immunization program to be deemed costeffective only if 90% of scenarios in a Monte Carlo simulation fall below a £25,000 per qualityadjusted lifeyear (QALY) threshold [3]. Such somewhat arbitrary and ad hoc approaches can be problematic, therefore risking adoption of suboptimal healthcare programs [4].
Recently, Sendi [5] introduced the costeffectiveness riskaversion curve (CERAC) to inform choices under uncertainty for riskaverse decision makers and policy makers. Sendi [5] proposed using the results of a CEA to construct the CERAC based on a riskadjusted performance measure widely used in financial economics called the Sortino ratio [6].
The purpose of this paper is to evaluate the Sortinoratiobased CERAC as a riskadjustment measure, investigate its various properties, discuss the implications of using it to inform decision making under uncertainty, and compare it with the expectedutility approach. This will be achieved by deriving analytic formulae for both the Sortinoratiobased CERAC and the expectedutilitybased CERAC, relating the CERAC to the moments (e.g., means and standard deviations) of the distribution of costs and effects of a healthcare program. The paper also discusses the level of risk aversion implicitly assumed in the Sortinoratiobased CERAC.
CERAC Based on the Sortino Ratio
The financial economics literature includes several riskadjusted performance measures that are frequently used to evaluate investment opportunities. Sendi [5] discussed the Sharpe ratio [7, 8], considered to be the most popular performance ratio, and proposed its derivative, the Sortino ratio, to construct the CERAC. The net benefittorisk ratio S_{NMB}, analogous to the Sortino ratio, is calculated as the average net monetary benefit (NMB) of a program adjusted by its associated downside risk (defined as the risk of achieving an NMB below the expected NMB). Given the joint distribution of costs (denoted by C) and effects (denoted by E), the NMB of a given program is defined as
where \(\lambda\) is the decision maker’s maximum willingness to pay (WTP) per health outcome E (e.g., QALYs). Thus, S_{NMB} is calculated as
where \({\mu }_{\mathrm{NMB}}\) is the expected NMB of a program and \({\mathrm{DD}}_{\mathrm{NMB}}\) is the root mean square of the downside deviations of the program’s NMB realizations from the expected NMB. The expected value of NMB of a program (Eq. 1) is given by
where \({\mu }_{\mathrm{E}}\) denotes the expected effects and \({\mu }_{\mathrm{C}}\) denotes the expected costs of the program. The formula for \({\mathrm{DD}}_{\mathrm{NMB}}\) of a continuous random variable NMB with probability density function \(f\left(x\right)\) is defined as:
where
It should be mentioned that the Sharpe ratio penalizes all deviations, not just downside deviations only, as in the Sortino ratio. Thus, in deriving the Sharpe ratio, the \(g\left(x\right)\) function in the above formula becomes \(g\left(x\right)=1 \, \mathrm{for \, all} \, x\). For both riskadjustment measures, it is assumed that a riskaverse decision maker prefers a program with a higher net benefittorisk ratio.
Although the approach does not require making distributional assumptions, Sendi [5] assumed that both costs and effects are normally distributed, with standard deviation of costs denoted by \({\sigma }_{C}\), standard deviation of effects denoted by \({\sigma }_{E}\), and the correlation between the costs and effects of each program denoted by \(\rho\). Thus, it follows from the bivariate normal distribution of costs and effects \(\mathcal{N}\left[\left({\mu }_{\mathrm{E}},{\mu }_{\mathrm{C}}\right),\left({\sigma }_{\mathrm{E}},{\sigma }_{\mathrm{C}}\right),\rho \right]\) that the NMB of a program is also normally distributed, with the mean given by Eq. (3) and variance \({\mathrm{var}}_{\mathrm{NMB}}\) by
Because the normal distribution is symmetric around the mean, the total of upside deviations and the total of downside deviations are equal. Thus, considering only downside deviations from the mean requires only dividing the variance by 2 to obtain \({\mathrm{DD}}_{\mathrm{NMB}}\). Therefore, the root mean square of the downside deviations is
Combining Eqs. (2), (3), and (6), the net benefittorisk ratio S_{NMB} based on the Sortino ratio under normally distributed costs and effects is given by
For a normally distributed NMB, the benefittorisk ratio based on the Sharpe ratio is
It is clear from Eq. (8) that, under the assumption of a normal distribution of NMB, the Sharpe ratio is proportional to the Sortino ratio. Consequently, use of either the Sharpe or the Sortino ratio will result in the same ranking of healthcare programs when the distribution of NMB is symmetric.
As an illustration of the use of the net benefittorisk ratio S_{NMB}, Sendi [5] provided three examples; two of them (example 1 and 2) are shown in Table 1. In example 1, in Table 2, assuming λ = $50,000/QALY, program B has a greater expected NMB (i.e., \({\mu }_{\mathrm{NMB}}\) of $850,000 vs $700,000) and hence is preferred by a riskneutral decision maker to program A. However, with greater uncertainties in both costs and effects, program B has a lower net benefittorisk ratio (i.e., \({S}_{\mathrm{NMB}}\) 3.1 vs 10.1) and should not be preferred by a riskaverse decision maker. The decision is reversed in example 2. Program D is now preferred (i.e., \({S}_{\mathrm{NMB}}\) 12.3 vs 2.5), because program D has less uncertainty in both costs and effects compared with program C.
Properties of CERAC Based on the Sortino Ratio
It is important to understand the mathematical properties of the CERAC given by Eq. (7). It should be noted that all these properties are derived under the assumption that both costs and effects are normally distributed. As a function of WTP \(\lambda\), the CERAC:

1.
Cuts the xaxis where the expected NMB is zero (i.e., \({S}_{\mathrm{NMB}}=0\) when \(\lambda ={\mu }_{\mathrm{C}}/{\mu }_{\mathrm{E}}\)).

2.
Asymptotically approaches the ratio of the expected effects to the square root of half the variance of effects (i.e., \({\text{lim}}_{{\lambda \to \infty }} S_{{{\text{NMB}}}} = \mu _{{\text{E}}} /\sqrt {\sigma _{{\text{E}}}^{2} /2}\))

3.
Is not always monotonically increasing in \(\lambda\) (Fig. 1 shows cases of both increasing and decreasing CERACs).
Although he used simulation rather than an analytical approach and only a limited number of examples, Sendi [5] correctly described properties (1) and (2). Also, in all the three examples provided by Sendi [5], the CERAC is depicted to be monotonically increasing as \(\lambda\) increases. As shown in Fig. 1, using inputs \({\mu }_{\mathrm{E}}=10,{\mu }_{\mathrm{C}}=\mathrm{60{,}000},{\sigma }_{\mathrm{E}}=1.1,{\sigma }_{\mathrm{C}}=\mathrm{20{,}000},\mathrm{and } \; \rho =0.9\), the CERAC of a given program is not always monotonically increasing in \(\lambda\). As a function of \(\lambda\), the net benefittorisk ratio of the program increases initially, reaches a peak, and declines thereafter, reaching its asymptotic value from above.
In general, the shape of the CERAC, including nonmonotonicity and the position of the peak, is ambiguous and is determined by the interplay of many factors. For example, when the correlation between costs and effects approaches zero, the CERAC is always monotonically increasing. However, when there is correlation between costs and effects, CERAC can be monotonically increasing or has an invertedU shape.
For a given value of WTP \(\lambda\), the net benefittorisk ratio increases with higher mean values of effects \({\mu }_{\mathrm{E}}\) and lower values of mean costs \({\mu }_{\mathrm{C}}\), and higher values of the correlation between costs and effects \(\rho\) for all positive values of NMB. The positive association between net benefittorisk ratio and \(\rho\) follows from the inverse relationship between the variance of NMB and the correlation between costs and effects \(\rho\). The net benefittorisk ratio can increase or decrease with higher values of the standard deviation of costs \({\sigma }_{\mathrm{C}}\) or effects \({\sigma }_{\mathrm{E}}\). As a function of the overall mean and variance of NMB, the netbenefit ratio increases with increases in the mean and decreases, albeit at a lower rate, with lower variance of NMB. Mathematically speaking, the net benefittorisk ratio is homogenous of degree zero in the mean and standard deviation of NMB. That is, the net benefittorisk ratio does not change if both the mean and standard deviation of NMB change at the same rate. For example, doubling the values of the mean and standard deviation of costs and effects leaves the ratio unchanged. Sendi [5] did not describe this very important property of the net benefittorisk ratio.
The last property of the net benefittorisk ratio is more general according to Eq. (2): the ratio does not change if both the mean and the root mean square of the downside deviation of NMB change at the same rate. In other words, proportional changes in the overall mean and the root mean square of the downside deviation of NMB are valued equivalently. This raises the following question: can CERAC correctly account for the tradeoff between risk and expected returns? Because the net benefittorisk ratio as proposed by Sendi [5] is not based on theoretical grounds, it can be considered somewhat arbitrary. For example, there is no justification for implicitly assuming a fixed risk aversion that cannot be varied across decision makers. More importantly, what is the risk attitude implicit in the use of the Sortinobased net benefittorisk ratio performance measures?
Example 3 illustrates how much expected value a riskaverse decision maker will be willing to forgo in order to avoid risk (Table 1). The inputs in example 3 are the same as those in example 1, except that the mean effect of program B’ is now almost close to three times the original mean (20 vs 59 QALYs). Example 3 also includes a new scenario B” where the mean effect of program B” is 17 QALYs. Recall that in example 1 program B is preferred by a riskneutral decision maker whereas program A is preferred by a riskaverse decision maker. In example 3, the incremental costeffectiveness ratio (ICER) of program B’ compared with program A is $2273/QALY (Table 2). When the mean effect of program B” is 17 QALYs, the ICER of B’’ compared with A is equal to $50,000/QALY, and a riskneutral decision maker is indifferent between program A and B’’. At mean effect of program B’ of 59 QALYs, both programs have a net benefittorisk ratio of 10.1, and a riskaverse decision maker is indifferent between program A and B’. This implies that, compared with a riskneutral decision maker, a riskaverse decision maker will only be willing to accept an additional 42 (= 59 − 17) QALYs per person (i.e., equivalent to an incremental NMB of $2,100,000) in order to become indifferent between program A and the riskier program B’.
Example 4 is the same as example 2, except that the mean effect of program A’ is now 69 instead of 15 QALYs (Table 1). Recall that in example 2, program D is preferred over program C by either a riskneutral or a riskaverse decision maker. However, assuming λ = $50,000/QALY, example 4 illustrates that program C’ is preferred over program D by a riskaverse decision maker but is no longer preferred by a riskneutral decision maker as long as the mean effects of program C’ is greater than 18 QALYs (Table 2). In fact, according to the net benefittorisk ratio approach, even though it is strongly being dominated by program C’, program D is always preferred over program C’ by a riskaverse decision maker as long as program C’ offers mean effects of less than 69 QALYs. In other words, compared with a riskneutral decision maker (indifferent between the two programs if the mean effect is 18 QALYs), a riskaverse decision maker will be willing to accept at least 51 (= 69 − 18) additional QALYs (equivalent to an incremental NMB of $2,550,000) in order to prefer program C’ over the less risky program D.
Both example 3 and 4 in Table 1 illustrate a very important property of the Sortinoratiobased CERAC: the preference of a riskaverse decision maker using the Sortinoratiobased CERAC implies a strong risk–return tradeoff. For example, suppose a hypothetical program E has double the standard deviation (i.e., quadruple the variance) of NMB compared with another program F. The two programs are valued equally by a riskaverse decision maker using the Sortinoratiobased CERAC only if the mean NMB of program E is also double that of program F.
In a more recent paper, Sendi et al. [9] claim that the downside deviation can be modified to reflect different levels of risk aversion. The authors suggest that this can be done by defining a different minimally acceptable return (MAR) for the downside deviation of NMB. The authors were not clear on how to select an MAR, nor on what is an appropriate MAR, but discussed a few options—all seem arbitrary. Focusing on their first alternative choice of defining the downside deviation relative to the 25% percentile of the NMB distribution, denoted as \({\eta }_{25\mathrm{NMB}}\), instead of the mean, the denominator of the ratio, Eq. (4), becomes
where
Applying this to the inputs of example 3 resulted in higher ratios because the downside deviation is lower now ($37,920.3 for program A and $151,681 for program B’). Most importantly, both programs have the same net benefittorisk ratio of 18.5, signifying that a riskaverse decision maker is still indifferent between program A and B’. In other words, the level of riskaversion did not change whether MAR was defined as the mean or the 25% percentile of the NMB. This example suggests that, regardless of the choice of MAR, using the Sortinoratiobased CERAC implicitly assumes that the decision maker is highly risk averse.
The result that changing MAR does not modify the degree of risk aversion can be shown to be more general than the inputs of example 3 by deriving a general formula for the Sortino ratio \({S\alpha }_{\mathrm{NMB}}\) of a normally distributed NMB for any percentile \(\alpha\) (where \(0<\alpha <1\)):
where \({\mu }_{\mathrm{NMB}}\) and \({\mathrm{var}}_{\mathrm{NMB}}\) are as defined before by Eqs. (3) and (5), respectively. The welldefined function \(f\left(\alpha \right)\) is independent of the mean and variance of the NMB distribution. When MAR is given by the mean (\(\alpha =50\%)\), \(f\left(\alpha \right)=\sqrt{2}\). The value of \(f\left(25\%\right)=1.4142\) when MAR is equal to the 25% percentile of the NMB. It is clear from Eq. 10 that the choice among healthcare programs is the same regardless of the value of \(\alpha .\) Consequently, whether MAR is defined as the mean or any percentile of the NMB, use of the Sortinoratiobased CERAC will result in the same ranking of healthcare programs.
CERAC Based on ExpectedUtility Approach and Certainty Equivalence
The health economics literature includes a few attempts at incorporating decision makers’ risk preferences when making decisions under uncertainty using the standard expectedutility maximization methods [10,11,12,13]. For example, Elbasha [13] suggested using the exponential utility–momentgenerating function (mgf) approach for incorporating risk preferences and the tradeoff between risk and returns to investment in health and medicine. This approach assumes that the utility function of each individual is represented by an exponential function, a convenient functional form allowing the expected value of utility function to be expressed in terms of only the mgf of the random variables. From welfare economics, a utilitarian welfare function assumes equal individual weights (since individuals are assumed to be ex ante identical) and linearly aggregates individual exponential utility functions. The exponential, also known as the constant absolute risk aversion (CARA), utility function over NMB is given by
where r is the absolute riskaversion parameter. Applying expectation, the expected utility is
where \({\mathbb{E}}\) is the expectation operator and \(M\left(r\right)\) is the mgf of the distribution of the random variable NMB. The mgf of the normal distribution of NMB \(M\left(r\right)\) is
It is wellknown that maximizing expected utility is the same as maximizing the certainty equivalent (CE) version of the expected utility [13]. The CE is defined as the fixed sure amount that one is willing to accept to be indifferent to a risky position. Thus, the CE of the NMB is defined as
Using Eqs. (11) and (12), the CE of the expected negative exponential utility function satisfies the following equation:
which can be solved for CE as
Substituting Eq. (13) into Eq. (14), the CE of a normally distributed NMB simplifies to the wellknown meanvariance CE
Utilizing Eqs. (3), (5), and (15), the CE can be expressed in terms of the parameters of the bivariate normal distribution of costs and effects as
Thus, if costs and effects follow a joint normal distribution, assuming that the value of the riskaversion parameter r is known, Eq. (16) can be used to calculate the CE of each program. A riskaverse decision maker prefers a program with a larger CE. Hence, a riskaverse decision maker should choose the program with the highest CE value.
In example 1 in Table 2, assuming λ = $50,000/QALY and r = 0.00001, program A is preferred by a riskaverse decision maker because it has a higher CE value (i.e., \({\mathrm{CE}}_{\mathrm{NMB}}\) of $651,875 vs $80,000). As with the net benefittorisk ratio approach, a riskaverse decision maker prefers program A in example 1 in Table 1 according to the CE approach as long as the value of r is greater than 0.0000021. However, a less riskaverse decision maker (e.g., r < 0.0000021) would prefer program B to program A. By varying the values of λ, the CERACs based on the CE can also be constructed. For example, the CE of program B increases initially with WTP, reaches a peak, and declines thereafter (Fig. 2).
In example 2, according to the CE approach, program D is preferred over program C (Table 2). In fact, any riskaverse decision maker (i.e., r is nonnegative) would always prefer program D over program C as long as λ ≥ $20,000/QALY. Similarly, according to the CE approach, program B’, in example 3, is preferred over program A (Table 2). Moreover, for a riskaverse decision maker to be indifferent between program A and program B’ in example 3 according to the CE approach, the riskaversion parameter r must be 0.0000291 (Fig. 4). Recall that the level of risk in example 1 and example 3 are the same. Thus, comparing the riskaversion parameters (0.0000291 vs 0.0000021), by requiring higher expected return to be willing to be indifferent between program A and program B’ in example 3, the decision maker following the netbenefit ratio approach is 14 times more risk averse compared with the riskaverse decision maker in example 1.
To better understand the tradeoff between risk and return as the WTP varies and the role of the decision maker’s attitude to risk, it is instructive to draw a costeffectiveness riskaversion map (CERAM) that depicts combinations of the riskaversion parameter r and WTP λ for which the riskaverse decision maker is indifferent between two programs and areas where the program is being preferred over another. In example 1 (Table 1), the blue curve in Fig. 3 depicts combinations of λ and r where program A has the same CE as program B, and the blue area represents combinations where program A has larger CE than program B. Similarly, the orange area represents combinations where program A has lower CE than program B. For example, when WTP is $50,000/QALY and r is 0.00001, a riskaverse decision maker would prefer program A over program B. However, when WTP is $50,000/QALY, a less riskaverse decision maker (e.g., r < 0.0000021) would prefer program B over the less risky program A.
A similar CERAM can be constructed using the data from example 3, in Table 1 (Fig. 4). With WTP = $50,000/QALY and r = 0.00001, a clear choice for a riskaverse decision maker is the riskier program B’ with higher expected return. Program B’ remains the optimal choice unless the level of risk aversion for the decision maker is higher than 0.0000291. If the WTP = $50,000/QALY and r > 0.0000291, a riskaverse decision maker would prefer the less risky program A.
Properties of CERAC Based on the Certainty Equivalence
It is important to understand the mathematical properties of the CERAC given by Eq. (14). All other things being equal, the CE increases with mean levels of effects and higher values of the correlation between costs and effects \(\rho\) and decreases with mean levels of costs, and risk aversion. The CE can increase or decrease with higher values of the standard deviation of costs \({\sigma }_{\mathrm{C}}\) or effects \({\sigma }_{\mathrm{E}}\). As a function of the overall mean and variance of NMB, the CE increases with increases in the mean and decreases, albeit at a lower rate, with lower variance of NMB. CE is an invertedU shape as function of \(\lambda\), being negative for low values of \(\lambda\), increases as \(\lambda\) increases until it reaches a maximum (when \(\lambda =\frac{{\mu }_{\mathrm{E}}+r\rho {\sigma }_{\mathrm{C}}{\sigma }_{\mathrm{E}}}{r{\sigma }_{\mathrm{E}}^{2}}\)), and decreases thereafter.
An Alternative Utility Function
The CEbased CERAC assuming an exponential utility function affords the opportunity to derive analytic results for any distribution of costs and effects since most known distributions have mgf. Nonetheless, the CEbased CERAC can be constructed using other utility functions. Some of these functional forms, such as the quadratic utility function, also allow derivation of analytic results. However, the quadratic utility function has some theoretically unsatisfactory properties (it decreases as NMB increases over some ranges, and it exhibits increasing absolute risk aversion). Another popular functional form, considered to be the most widely used for empirical analysis [14], is the constant relative risk aversion (CRRA) utility function [15]
where \(\gamma\) is the coefficient of CRRA. The CE derived from CRRA is given by
If NMB is lognormally distributed, with log (NMB) being normally distributed with mean m and standard deviation s, an analytic formula for CE can be derived as
As with CARA, a riskaverse decision maker should choose the program with the highest \({\mathrm{CE}}_{\mathrm{NMB}}\) derived using a CRRA utility function. To illustrate, the parameters in example 1, in Table 1, assuming λ = $50,000/QALY, result in a mean NMB of $700,000 and $850,000 for program A and B, respectively (Table 2). The respective standard deviation is 98,107.1 and 392,428. These values are related to the parameters of the lognormal distribution \({\mu }_{\mathrm{NMB}}\) and \({\sigma }_{\mathrm{NMB}}\) according to
This implies values of \(\left(m,s\right)=\left(13.449, 0.139\right)\) for program A and \(\left(m,s\right)=\left(\mathrm{13.556,0.439}\right)\) for program B. Substituting these values into Eq. 17 and assuming a CRRA coefficient of \(\gamma =\) 2.5, program A has a CE value of $683,184, whereas program B has a CE value of $667,615. Therefore, according to the CE approach based on a CRRA utility function, a riskaverse decision maker prefers program A in example 1 in Table 1. In fact, assuming λ = $50,000/QALY, program A is always preferred over program B as long as the value of \(\gamma\) is greater than 2.23467. However, a less riskaverse decision maker (e.g., \(\gamma <2.23467\)) would prefer program B over program A.
Discussion
Standard CEA frameworks do not consider the role of risk attitudes in informing coverage and reimbursement decisions of new medical technologies under uncertainty. There has been little research that has emphasized the importance of incorporating risk preferences and the tradeoff between the mean and the risk of returns to investment in health and medicine [9, 10, 16,17,18,19]. This paper presents two approaches for extending CEA to incorporate attitudes toward risk and contrasted them. The first approach applies principles of portfolio theory of how individual investors select among risky financial investments to CEA of healthcare programs. Specifically, the paper analyzes the approach introduced by Sendi [5] and Sendi et al. [9] that uses the net benefitstorisk ratio derived from the popular Sortino financial riskadjustment measure. Some of the results regarding the CERAC based on a Sortino ratio depend on the assumption of normality. It is not clear what would be the implications of deviations from this assumption. However, it is expected that major conclusions would still be valid under other distributional assumptions. For example, whether a simulation or an analytical approach is used, or whether a normal distribution is assumed or not, the conclusion that the CERAC based on the Sortino ratio riskadjustment measure implicitly assumes that the decision maker is highly risk averse remains valid.
The second approach draws from principles of welfare economics and expected utility theory in the evaluation of and selection from risky healthcare programs. The approach is illustrated using two frequently used utility functions: CARA or CRRA utility functions. The CARA utility function has several advantages, including being a secondorder local approximation to any wellbehaved utility function. Likewise, the CRRA utility function provides flexible functional forms for modeling risk aversion that can fit the data well.
A Comparison of the SortinoRatio and ExpectedUtility Approaches
Each approach has strengths and limitations (Table 3). The major strengths of the Sortinoratiobased approach to constructing the net benefitstorisk ratio include [5]:

S1
There is no need to explicitly specify a preference function over expected return and risk.

S2
There is no need to make assumptions about the distribution of costs and effects. Instead, it is possible to use empirical NMB distribution obtained from the probabilistic sensitivity analysis (PSA).

S3
Superior performance measure when the distribution of NMB is skewed and the riskaverse decision maker penalizes expected NMB for only downside deviation without penalizing upside deviation.

S4
Straightforward to calculate and widely applied in finance.

S5
Has several desirable qualitative properties (rises with mean and falls with variance of NMB).
The major weaknesses of the Sortinoratiobased approach include:

W1
It is an ad hoc approach to characterizing the risk–return tradeoff among healthcare programs. Although the risk preferences, implicit in using riskadjusted performance measures, are not well understood, rankings based on the Sortino ratio may not be consistent with preferences of a decision maker with low or moderate levels of risk aversion [20].

W2
A CERAC constructed based on a riskadjusted performance measure like the Sortino ratio includes an implied fixed risk attitude and does not afford the decision maker the opportunity to impose her or his own individual risk attitudes.

W3
By including only downside deviation, the approach excludes upside deviation, which may be important to a large class of riskaverse decision makers. To illustrate, consider the following simple hypothetical example of a riskaverse decision maker choosing between two risky investment opportunities. The first option can result in a loss of − 200 with probability 20% or a gain of 50 with probability 80%. The second option can also result in a loss of − 200 with probability 20%, but the gain is more uncertain: 20 with probability 20% and 60 with probability 60%. Both options have expected values of zero and the same downside deviation of 8000 (= 0.2 * (− 200)^{2}). According to the Sortino ratio, the decision maker should be indifferent between the two options because the downside deviations (i.e., uncertainty in losses) are equivalent. However, the first option has an upside deviation (i.e., uncertainty in gains) of 2000, whereas the upside deviation of the second option of 2240 is higher. For a riskaverse decision maker, the first option is clearly better because the upside risk is lower compared with that of the second option. It should be mentioned that this is consistent with how risk aversion is commonly defined in economics and finance as “the tendency of people to prefer outcomes with low uncertainty to those outcomes with high uncertainty, even if the average outcome of the latter is equal to or higher in monetary value than the more certain outcome” [21]. In this example, the average gains are equal (0.8 * 50 = 0.2 * 20 + 0.6 * 60 = 40) across the two options, but the uncertainty in gain is lower for the first option compared with that of the second option (2000 vs 2240).

W4
It is difficult to interpret a negative net benefitstorisk ratio resulting from a negative expected NMB. Also, when the expected NMB is zero, the net benefitstorisk ratio is zero regardless of the level of risk of a healthcare program.
The expectedutilitybased approach shares with the Sortinoratiobased approach major strengths S2 and S5. In addition, the most important strength of this approach to decision making under uncertainty is that its theoretical foundation is rooted in traditional welfare economics [12]. However, despite their theoretical appeal (e.g., being based on the principles of welfare economics and expectedutility theory), methods to analyze the results of a CEA under uncertainty and risk aversion have not been widely used. Among the limitations of the CE approach are:
W1. CE typically require the analyst to explicitly characterize preferences over risk and expected return by specifying an explicit utility function. For example, the momentgenerating approach assumes an exponential utility function, which implies CARA. This excludes an important class of riskaverse decision makers that have preferences that exhibit decreasing absolute risk aversion or CRRA. Although it is possible to use the approach with a CRRA utility function to derive analytic results, the required assumption of lognormally distributed NMB may not be reasonable given that NMB can be negative for low values of WTP.
W2. CE requires knowing the decision maker’s risk attitude by estimating the degree of risk aversion of the decision maker, which might be difficult to elicit in practice.
Instead of implicitly assuming a fixed degree of risk aversion as in the case of financial riskadjusted measures, the CE approach explicitly incorporates risk preferences of decision makers. It is true that understanding and eliciting the decision maker’s risk attitude in health is difficult. Despite the availability of methods of estimation of risk attitude coefficients such as the Arrow–Pratt measure of absolute riskaversion parameter r, there are no accepted estimates of r [22,23,24]. One solution suggested in this paper is to use a range of riskaversion coefficients extending from risk neutrality to moderate risk aversion and depict that against WTP in a twoway sensitivity approach. For example, using the inputs of example 1 (Table 1), the blue shared area in Fig. 3 shows the combinations of (λ, r) where a riskaverse decision maker would prefer program A over program B. It should be noted that to overcome the difficulty of directly eliciting the decision maker’s maximum WTP for per health outcome λ, the costeffectiveness acceptability curve is constructed by plotting the probability that the joint distribution of incremental costs and effects where the intervention is costeffective for a range of values of λ. Analogously, the CEbased CERAC can be constructed by plotting pairs of riskaversion parameter r and λ for which the decision maker is indifferent between two healthcare programs (Figs. 3, 4).
For most practical applications of CEA, there may be no need for choosing an appropriate joint distribution for costs and effects or a distribution of NMB. Instead, the analyst can use the results of the PSA from CEA directly to estimate the mean utility using the equation of the chosen utility function (e.g., Eq. 9 or Eq. 15).
In conclusion, adapting financial riskadjustment measures such as the Sortino ratio to construct a CERAC can be a helpful tool for highlighting the level of risk to riskaverse decision makers when making choices among healthcare interventions with uncertain costs and effects. However, compared with the expectedutility approach, riskadjustment measures implicitly assume levels of risk aversion that can be considered high for a class of decision makers. If the level of risk aversion of the decision maker is substantially lower than what is implicitly assumed, it is not recommended to use the Sortinoratiobased riskadjustment measure because it will result in suboptimal decisions. In addition to being theoretically grounded in the principles of welfare economics, the expectedutility approach accommodates wide ranges of risk preferences of decision makers. By presenting a CERAM depicting combinations of risk aversion and WTP where each health program is considered costeffective, the expectedutility approach has the added advantage of allowing the risk attitudes of the decision maker to be explicitly incorporated.
References
Weinstein MC, Stason WB. Foundations of costeffectiveness analysis for health and medical practices. N Engl J Med. 1977;296(13):716–21.
BenZion U, Gafni A. Evaluation of public investment in health care. Is the risk irrelevant? J Health Econ. 1983;2:161–5.
Immunisation and High Consequence Infectious Diseases Team, Global and Public Health Group. Costeffectiveness methodology for vaccination programmes: consultation on the CostEffectiveness Methodology for Vaccination Programmes and Procurement (CEMIPP) Report. Department of Health and Social Care; 2018. pp. 1–19.https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/707847/cemippconsultationdocument.pdf. Accessed 5 Mar 2021.
O’Mahony JF, Paulden M. Appraising the costeffectiveness of vaccines in the UK: insights from the Department of Health Consultation on the revision of methods guidelines. Vaccine. 2019;37(21):2831–7.
Sendi P. Dealing with bad risk in costeffectiveness analysis: the costeffectiveness riskaversion curve. Pharmacoeconomics. 2021;39:161–9.
Sortino FA, Van Der Meer R. Downside risk. J Portf Manag. 1991;17:27–31.
Sharpe WF. Mutual fund performance. J Bus. 1966;39:119–38.
Sharpe WF. The sharpe ratio. J Portf Manag. 1994;21:49–58.
Sendi P, MatterWalstra K, Schwenkglenks M. Handling uncertainty in costeffectiveness analysis: budget impact and risk aversion. Healthcare. 2021;9:1419.
O’Brien BJ, Sculpher MJ. Building uncertainty into costeffectiveness rankings: portfolio riskreturn tradeoffs and implications for decision rules. Med Care. 2000;38:460–8.
Graff Zivin J. Costeffectiveness analysis with risk aversion. Health Econ. 2001;10:499–508.
Graff Zivin J, Bridges JF. Addressing risk preferences in costeffectiveness analyses. Appl Health Econ Health Policy. 2002;1:135–9.
Elbasha EH. Risk aversion and uncertainty in costeffectiveness analysis: the expectedutility, momentgenerating function approach. Health Econ. 2005;14:457–70.
Abellan JM, Pinto JL, Mendez I, Badıa X. Towards a better QALY model. Health Econ. 2006;15:665–76.
Wakker PP. Explaining the characteristics of the power (CRRA) utility family. Health Econ. 2008;17(12):1329–44.
Bridges J, Stewart M, King M, van Gool K. Adapting portfolio theory for the evaluation of multiple investments in health with multiplicative extension for treatment synergies. Eur J Health Econ. 2002;3:47–53.
Sendi P, Al MJ, Zimmermann H. A riskadjusted approach to comparing the return on investment in health care programs. Int J Health Care Finance Econ. 2004;4:199–210.
Córdoba JC, Ripoll M. Risk aversion and the value of life. Rev Econ Stud. 2016;84:1472–509.
Lakdawalla DN, Phelps CE. Health technology assessment with risk aversion in health. J Health Econ. 2020;72:102346.
Plantinga A, de Groot JS. Riskadjusted performance measures and implied risk attitudes. J Perform Meas. 2002;6:9–20.
Werner J. Risk aversion. In: Palgrave M (eds) The new Palgrave dictionary of economics. Palgrave Macmillan, London; 2008. pp. 1–6. https://doi.org/10.1057/9781349951215_27411.
BarShira Z, Just R, Zilberman D. Estimation of farmers’ risk attitude: an econometric approach. Agric Econ. 1997;17:211–22.
Szpiro G. Measuring risk aversion: an alternative approach. Rev Econ Stat. 1986;68:156–9.
Cher D, Miyamoto J, Lenert L. Incorporating risk attitude into Markovprocess decision models. Med Decis Mak. 1997;17:340–50.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
This work was supported by Merck Sharp & Dohme Corp., a subsidiary of Merck & Co., Inc., Kenilworth, NJ, USA; Merck Sharp and Dohme.
Conflict of interest
The author declares no competing interests.
Availability of data and material
Not applicable.
Code availability
Code is available from the author upon request.
Authors' contributions
Not applicable.
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Rights and permissions
Open access This article is licensed under a Creative Commons AttributionNonCommercial 4.0 International License, which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/bync/4.0/.
About this article
Cite this article
Elbasha, E.H. Cost‑Effectiveness Risk‑Aversion Curves: Comparison of RiskAdjusted Performance Measures and ExpectedUtility Approaches. PharmacoEconomics 40, 497–507 (2022). https://doi.org/10.1007/s40273021011235
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40273021011235