Skip to main content
Log in

Fuzzy approach to decision analysis with multiple criteria and uncertainty in health technology assessment

  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Decision making in health technology assessment (HTA) involves multiple criteria (clinical outcomes vs. cost) and risk (criteria measured with estimation error). A survey conducted among Polish HTA experts shows that opinions how to trade off health against money should be treated as fuzzy. We propose an approach that allows to introduce fuzziness into decision making process in HTA. Specifically, in the paper we (i) define a fuzzy preference relation between health technologies using an axiomatic approach; (ii) link it to the fuzzy willingness-to-pay and willingness-to-accept notions and show the survey results in Poland eliciting these; (iii) incorportate uncertainty additionally to fuzziness and define two concepts to support decision making: fuzzy expected net benefit and fuzzy expected acceptability (the counterparts of expected net benefit and cost-effectiveness acceptability curves, CEACs, often used in HTA). Illustrative examples show that our fuzzy approach may remove some problems with other methods (CEACs possibly being non-monotonic) and better illustrate the amount of uncertainty present in the decision problem. Our framework can be used in other multiple criteria decision problems under risk where trade-off coefficients between criteria are subjectively chosen.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Defined in the Reimbursement Act to be equal to triple annual gross domestic product per capita.

  2. There are other criteria that may be used in HTA, e.g., the decision maker may want to give priority to treatments being offered to particularly vulnerable populations like patients with rare or ultra-rare diseases.

  3. Another issue arises here, beyond the main interest of this paper: based on what grounds can we aggregate utilities between patients, neglecting the problem of inter-personal utility comparisons? Briefly, due to ethical issues we do not want to differentiate treatment provided to individual patients because of differences in their preferences. That is why in HTA we typically measure the clinical improvements in single patients objectively: longevity of life is totally objective, and we might use some definition of health-related quality of life, like EQ-5D. We can then translate health states into utilities using a single population-based value set. Hence, we do not aggregate utilities but rather apply a single utility function to all the patients.

  4. It is sometimes claimed that higher drug prices and higher WTP ratios are rationalized in these situations, as the only way to stimulate the R&D process, when the demand is small.

  5. Throughout the paper we use the notation that if X is a name of a fuzzy set, then \(X(\cdot )\) denotes a membership function defining this set, and \(X_\alpha \) denotes the \(\alpha \)-cut. Obviously, the membership function and \(\alpha \)-cuts can be derived from each other, and both can serve equally well to define a fuzzy set.

  6. By \({\mathbf {1}}_A(x)\) we denote the standard indicator function, whose value is 1 if \(x\in A\), and 0 if \(x\notin A\).

  7. In Sect. 2 we denoted by \(E_i\) and \(C_i\) random variables describing the uncertainty of the effect and cost assessment of technology i. To clarify, here \(E_{(i)}\) and \(C_{(i)}\) denote the sequence of random variables, with each element of this sequence (denoted by i) measuring the uncertainty of the difference between the effect and cost of two alternatives being compared.

  8. Whenever the underlying first-order uncertainty permits the central limit theorem to hold, i.e., the inter-patient variability is such that increasing the sample size allows to estimate the average effect and cost more precisely.

  9. In standard Euclidean metric in \({\mathbb {R}}^2\), denoted by \(d(\cdot ,\cdot )\).

References

  • Arrow, K. (1963). Uncertainty and the welfare economics of medical care. The American Economic Review, 53(5), 141–149.

    Google Scholar 

  • Arrow, K., & Lind, R. (1970). Uncertainty and the evaluation of public intervention decisions. The American Economic Review, 60, 364–378.

    Google Scholar 

  • Billingsley, P. (1999). Convergence of probability measures (2nd ed.). New York: Wiley.

    Book  Google Scholar 

  • Black, W. (1990). The CE Plane: A graphic representation of cost-effectiveness. Medical Decision Making, 10, 212–214.

    Article  Google Scholar 

  • Bleichrodt, H., Wakker, P., & Johannesson, M. (1997). Characterizing QALYs by risk neutrality. Journal of Risk and Uncertainty, 15, 107–114.

    Article  Google Scholar 

  • Briggs, A., Claxton, K., & Sculpher, M. (2006). Decision modelling for health economic evaluation. Oxford: Oxford University Press.

    Google Scholar 

  • Briggs, A., & Fenn, P. (1998). Confidence intervals or surfaces? Uncertainty on the cost-effectiveness plane. Health Economics, 7(8), 723–740.

    Article  Google Scholar 

  • Briggs, A., Weinstein, M. C., Fenwick, E. A., Karnon, J., Sculpher, M. J., & Paltiel, A.D., on behalf of the ISPOR-SMDM Modeling Good Research Practices Task Force. (2012). Model parameter estimation and uncertainty analysis: A report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Medical Decision Making, 32(5), 722–732.

  • Claxton, K. (1999). The irrelevance of inference: A decision-making approach to the stochastic evaluation of health care technologies. Journal of Health Economics, 18(3), 341–364.

    Article  Google Scholar 

  • Devlin, N., & Parkin, D. (2004). Does NICE have a cost-effectiveness threshold and what other factors influence its decisions? A binary choice analysis. Health Economics, 13, 437–452.

    Article  Google Scholar 

  • Eckermann, S., & Willan, A. (2011). Presenting evidence and summary measures to best inform societal decisions when comparing multiple strategies. Pharmacoeconomics, 29(7), 563–577.

    Google Scholar 

  • Eichler, H. G., Kong, S., Gerth, W., Mavros, P., & Jönsson, B. (2004). Use of cost-effectiveness analysis in health-care resources allocation decision-making: How are cost-effectiveness thresholds expected to emerge? Value in Health, 7(5), 518–528.

    Article  Google Scholar 

  • Fenwick, E., Claxton, K., & Sculpher, M. (2001). Representing uncertainty: The role of cost-effectiveness acceptability curves. Health Economics, 10(8), 779–787.

    Article  Google Scholar 

  • Fenwick, E., O’Brien, B., & Briggs, A. (2004). Cost-effectiveness acceptability curves facts, fallacies and frequently asked questions. Health Economics, 13, 405–415.

    Article  Google Scholar 

  • Gafni, A., & Birch, S. (2006). Incremental cost-effectiveness ratios (ICERs): The silence of the lambda. Social Science & Medicine, 62, 2091–2100.

    Article  Google Scholar 

  • Garber, A. (2000). Advances in cost-effectiveness analysis of health interventions. In A. J. Culyer (Ed.), Handbook of health economics (Vol. 1A, pp. 181–221). Amsterdam: North-Holland.

    Google Scholar 

  • Gold, M., Siegel, J., Russell, L., & Weinstein, M. (Eds.). (1996). Cost-effectiveness in health and medicine. Oxford: Oxford University Press.

    Google Scholar 

  • Hunink, M. G., Bult, J. R., de Vries, J., & Weinstein, M. C. (1998). Uncertainty in decision models analyzing cost-effectiveness: The joint distribution of incremental costs and effectiveness evaluated with a nonparametric bootstrap method. Medical Decision Making, 18(3), 337–346.

    Article  Google Scholar 

  • Jakubczyk, M., & Kamiński, B. (2010). Cost-effectiveness acceptability curves-caveats quantified. Health Economics, 19, 955–963.

    Article  Google Scholar 

  • Kahneman, D., Knetsch, J., & Thaler, R. (2009). Experimental tests of the endowment effect and the coase theorem. In E. L. Khalil (Ed.), The new behavioral economics. Volume 3. Tastes for endowment, identitiy and the emotions (Vol. 3, pp. 119–142). London: Elgar.

    Google Scholar 

  • Klir, G., & Yuan, B. (1995). Fuzzy sets and fuzzy logic: Theory and applications. Englewood Cliffs NJ: Prentice Hall.

    Google Scholar 

  • Lee, K. (2005). First course on fuzzy theory and applications. Berlin: Springer.

    Google Scholar 

  • Löthgren, M., & Zethraeus, N. (2000). Definition, interpretation and calculation of cost-effectiveness acceptability curves. Health Economics, 9, 623–630.

    Article  Google Scholar 

  • Moreno, E., Girón, F., Vázquez-Polo, F., & Negrín, M. (2010). Optimal healthcare decisions: Comparing medical treatments on a cost-effectiveness basis. European Journal of Operational Research, 204, 180–187.

    Article  Google Scholar 

  • Moreno, E., Girón, F., Vázquez-Polo, F., & Negrín, M. (2013). Optimal treatments in cost-effectiveness analysis in the presence of covariates: Improving patient subgroup definition. European Journal of Operational Research, 226, 173–182.

    Article  Google Scholar 

  • Obenchain, R. (1997). Issues and algorithms in cost-effectiveness inference. Biopharmaceutical Report, 5(2), 1–7.

    Google Scholar 

  • Obenchain, R. (2008). ICE preference maps: Nonlinear generalizations of net benefit and acceptability. Health Services and Outcomes Research Methodology, 8, 31–56.

    Article  Google Scholar 

  • O’Brien, B., Gertsen, K., Willan, A., & Faulkner, L. (2002). Is there a kink in consumers’ threshold value for cost-effectiveness in health care? Health Economics, 11, 175–180.

    Article  Google Scholar 

  • Pliskin, J., Shepard, D., & Weinstein, M. (1980). Utility functions for life years and health status. Operations Research, 28(1), 206–224.

    Article  Google Scholar 

  • Sadatsafavi, M., Najafzadeh, M., & Marra, C. (2008). Acceptability curves could be misleading when correlated strategies are compared. Medical Decision Making, 28(3), 306–307.

    Article  Google Scholar 

  • Severens, J., Brunenberg, D., Fenwick, E., O’Brien, B., & Joore, M. (2005). Cost-effectiveness acceptability curves and a reluctance to lose. Pharmacoeconomics, 23(12), 1207–1214.

    Article  Google Scholar 

  • van Hout, B., Al, M., Gordon, G., & Rutten, F. (1994). Costs, effects and C:E-ratios alongside a clinical trial. Health Economics, 3, 309–319.

    Article  Google Scholar 

  • Weinstein, M., & Zeckhauser, R. (1973). Critical ratios and efficient allocation. Journal of Public Economics, 2, 147–157.

    Article  Google Scholar 

  • Zaric, G. (2010). Cost-effectiveness analysis, health-care policy, and operations research models. In Wiley Encyclopedia of operations research and management science, Wiley. doi:10.1002/9780470400531.eorms0202.

  • Zivin, J., & Bridges, J. (2002). Addressing risk preferences in cost-effectiveness analysis. Applied Health Economics and Health Policy, 1(3), 135–139.

    Google Scholar 

Download references

Acknowledgments

It would not have been possible to collect the survey results presented in the paper without the help from M. Niewada, who facilitated the contact with the respondents. We would like to acknowledge the help of HTA experts who participated in the survey: K. J. Filipiak, K. Jahnz-Różyk, and the others, who opted to remain anonymous. We also express our gratitude to D. Golicki, T. Macioch, W. Wrona, and again M. Niewada, who commented on the first version of the survey.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Michał Jakubczyk.

Appendix

Appendix

Proof of Proposition 1

For \(e=0\) we have \(\text {fNB}(x)=\mu (0,c+x)\) and so is equal to 1 for \(x\ge -c\), and to 0 for \(x<-c\). Hence \(\text {fNB}_\alpha =]-\infty ,-c]\) for \(\alpha \in ]0,1]\),while \(\tau _\alpha =-c\).

For \(e>0\):

$$\begin{aligned}&\sup (\text {fNB}_\alpha )=\sup (\left\{ x\in {\mathbb {R}}:\, \mu (e,c+x)\ge \alpha \right\} )=\sup \left( \left\{ x\in {\mathbb {R}}:\, \mu \left( 1,\frac{c+x}{e}\right) \ge \alpha \right\} \right) \\&\quad =\sup (\left\{ ye-c:\, y\in {\mathbb {R}}\wedge \mu (1,y)\ge \alpha \right\} )\\&\quad =e\times \sup (\left\{ y\in {\mathbb {R}}:\, \mu (1,y)\ge \alpha \right\} )-c\\&\quad =e\times \sup (\text {fWTP}_\alpha )-c, \end{aligned}$$

where the first equality comes from the definition of fNB; the second from scale invariance, the third from substitution \(x=ye-c\), the fourth from taking irrelevant terms out of \(\sup \) function, the fifth from the definition of fWTP.

For \(e<0\) we proceed analogously (we replace \(\sup \) with \(\inf \) as we take \(e<0\) out):

$$\begin{aligned}&\sup (\text {fNB}_\alpha )=\sup (\left\{ x\in {\mathbb {R}}:\, \mu (e,c+x)\ge \alpha \right\} )\\&\quad =\sup \left( \left\{ x\in {\mathbb {R}}:\, \mu \left( -1,\frac{-c-x}{e}\right) \ge \alpha \right\} \right) \\&\quad =\sup (\left\{ ye-c:\, y\in {\mathbb {R}}\wedge \mu (-1,-y)\ge \alpha \right\} )\\&\quad =e\times \inf (\left\{ y\in {\mathbb {R}}:\, \mu (-1,-y)\ge \alpha \right\} )-c\\&\quad =e\times \inf (\text {fWTA}_\alpha )-c. \end{aligned}$$

Obviously, if for \(\alpha \in ]0,1]\) the sets \(\left\{ (e,c)\in {\mathbb {R}}^2:\, \mu (\cdot ,\cdot )\ge \alpha \right\} \) are closed, then \(\text {fNB}_\alpha (e,c)\) is closed. \(\square \)

Proof of Proposition 2

Due to monotonicity axiom \(\text {fNB}(\cdot )\) is non-increasing. Therefore \(\forall \alpha _1,\alpha _2\in ]0,1],\alpha _1<\alpha _2\), we have \(\sup (\text {fNB}_{\alpha _2})\le \sup (\text {fNB}_{\alpha _1})\), and this inequality is preserved under integration in Definition 4. Regarding fEA, obviously \({\mathbf {1}}_{[\alpha ,1]}\left( \mu (e,c)\right) \) is non-increasing with \(\alpha \), and this (weak) inequality is preserved under integration. \(\square \)

Proof of Proposition 3

When WTP and WTA are crisp then \(\mu (\cdot ,\cdot )\) only takes values in \(\left\{ 0,1 \right\} \). Therefore, for any \(\alpha >0\) all intervals \(\text {fENB}_\alpha \) are identical, and the same holds for \(\text {fEA}_\alpha \). Thus it suffices to calculate \(\text {fENB}_1\) and \(\text {fEA}_1\).

We first focus on \(\text {fENB}_1\). From Eq. (2) we get \(\phi _1=\int _\varOmega \sup \left( {\text {fNB}}_1\left( e,c\right) \right) \,{\mathrm {d}}{\fancyscript{P}}\). Now we remember that WTP and WTA are crisp and have the same value t, so by applying Proposition 1 we get:

$$\begin{aligned} \phi _1=\int _\varOmega (et-c)\,{\mathrm {d}}{\fancyscript{P}} = {\mathbb {E}}(Et-C). \end{aligned}$$

This means that \(\text {fENB}(x)\) is equal to 1 for \(x\le {\mathbb {E}}(Et-C)\), and drops to 0 if x is larger. This is exactly the result given in the theorem. Now we move to \(\text {fEA}_1\). From Eq. 3 we have:

$$\begin{aligned} \text {fEA}_1=\left[ 0,\int _\varOmega {\mathbf {1}}_{\{1\}}(\mu (e,c)) \,{\mathrm {d}}{\fancyscript{P}}\right] . \end{aligned}$$

However, because WTP and WTA have exactly the same crisp value t, we see that \(\mu (e,c)=1\) when \(et-c\ge 0\) which leads to \(\text {fEA}_1=\left[ 0,{\mathbb {P}}(Et-C)\right] \). This means that \(\text {fEA}(x)\) is equal to 1 for \(x\le {\mathbb {P}}(Et-C)\), and drops to 0 if x is larger. This is exactly the result given in the theorem. \(\square \)

Proof of Proposition 4

We start with \(\text {fEA}_{(i)}\). We will show that for \(\beta >\mu (e^*,c^*)\) we have \(\sup \left( \text {fEA}_{(i),\beta }\right) \rightarrow 0\) and for \(\beta <\mu (e^*,c^*)\) we have \(\sup \left( \text {fEA}_{(i),\beta }\right) \rightarrow 1\). This will show that for fixed \(x\in ]0,1[\) we have \(\sup \{\alpha \in [0,1]:\, x\in \text {fEA}_{(i),\alpha }\}\rightarrow \mu (e^*,c^*)\). We start with \(\beta >\mu (e^*,c^*)\). We have:

$$\begin{aligned} \text {fEA}_{i,\beta }=\left[ 0,\int _\varOmega {\mathbf {1}}_{[\beta ,1]}(\mu (e,c)) \,{\mathrm {d}}{\fancyscript{P}}_i\right] . \end{aligned}$$

As \(\mu \) is continuous in \((e^*,c^*)\), there exists \(\delta >0\) such that for all (ec) closerFootnote 9 to \((e^*,c^*)\) than \(\delta \) we have \(\mu (e,c)<\beta \). Then:

$$\begin{aligned} \int _\varOmega {\mathbf {1}}_{[\beta ,1]}(\mu (e,c)) \,{\mathrm {d}}{\fancyscript{P}}_i&= \int _{\varOmega ,d((e,c),(e^*,c^*))<\delta } {\mathbf {1}}_{[\beta ,1]}(\mu (e,c)) \,{\mathrm {d}}{\fancyscript{P}}_i \,+\\&\quad +\,\int _{\varOmega ,d((e,c),(e^*,c^*))\ge \delta } {\mathbf {1}}_{[\beta ,1]}(\mu (e,c)) \,{\mathrm {d}}{\fancyscript{P}}_i. \end{aligned}$$

The first term is simply equal to 0, because in the region of integration \(\mu \) is less than \(\beta \). The second term converges to 0 with \(i\rightarrow +\infty \) because \((E_{(i)},C_{(i)}\) converges in probability to \((e^*,c^*)\) and \({\mathbf {1}}_{[\beta ,1]}(\mu (e,c))\) is bounded. Thus \(\sup (\text {fEA}_{(i),\beta })\rightarrow 0\).

The proof for \(\beta <\mu (e^*,c^*)\) is analogous, but then the first term is equal to

$$\begin{aligned} {\mathbb {P}}(d((E_{(i)},C_{(i)}),(e^*,c^*))<\delta ), \end{aligned}$$

and this value converges to 1 because \((E_{(i)},C_{(i)})\) converges in probability to \((e^*,c^*)\).

Now we move to the proof of convergence of \(\text {fENB}_{(i),\alpha }\). We have to show that \(\phi _{i,\alpha }=\int _\varOmega \sup \left( {\text {fNB}}_\alpha \left( e,c\right) \right) \,{\mathrm {d}}{\fancyscript{P}}_i\) from Eq. (2) converges to \(\sup \left( {\text {fNB}}_\alpha \left( e^*,c^*\right) \right) \). Because we assume that \((E_{(i)},C_{(i)})\) converges in probability to \((e^*,c^*)\) then by the continuous mapping theorem (Billingsley 1999) it is enough to show that \(g(e,c)=\sup \left( {\text {fNB}}_\alpha \left( e,c\right) \right) \) is continuous at \((e^*,c^*)\). Observe from Eq. (1) and the proof of Proposition 1 that \(\sup (fNB_\alpha (e,c))=\tau _\alpha (e,c)\), and is obviously continuous in the whole \({\mathbb {R}}^2\). \(\square \)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jakubczyk, M., Kamiński, B. Fuzzy approach to decision analysis with multiple criteria and uncertainty in health technology assessment. Ann Oper Res 251, 301–324 (2017). https://doi.org/10.1007/s10479-015-1910-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-015-1910-9

Keywords

Navigation