, Volume 37, Issue 3, pp 293–299 | Cite as

Determining Value in Health Technology Assessment: Stay the Course or Tack Away?

  • J. Jaime CaroEmail author
  • John E. Brazier
  • Jonathan Karnon
  • Peter Kolominsky-Rabas
  • Alistair J. McGuire
  • Erik Nord
  • Michael Schlander
Open Access
Current Opinion


The economic evaluation of new health technologies to assess whether the value of the expected health benefits warrants the proposed additional costs has become an essential step in making novel interventions available to patients. This assessment of value is problematic because there exists no natural means to measure it. One approach is to assume that society wishes to maximize aggregate health, measured in terms of quality-adjusted life-years (QALYs). Commonly, a single ‘cost-effectiveness’ threshold is used to gauge whether the intervention is sufficiently efficient in doing so. This approach has come under fire for failing to account for societal values that favor treating more severe illness and ensuring equal access to resources, regardless of pre-existing conditions or capacity to benefit. Alternatives involving expansion of the measure of benefit or adjusting the threshold have been proposed and some have advocated tacking away from the cost per QALY entirely to implement therapeutic area-specific efficiency frontiers, multicriteria decision analysis or other approaches that keep the dimensions of benefit distinct and value them separately. In this paper, each of these alternative courses is considered, based on the experiences of the authors, with a view to clarifying their implications.

Key Points for Decision Makers

Valuing new health technologies is difficult because choices must be made regarding which effects to consider and how to reflect their value in monetary terms.

A commonly implemented approach involves putting a price to a quality-adjusted life-year. This may be considered in light of other factors, such as society may give priority to more severe illnesses and ensuring fair access to resources, but the processes for doing so are neither systematic nor transparent.

Approaches to address broader societal values remain rudimentary but deserve significant efforts to operationalize them.

1 Introduction

In most countries, clinicians and patients do not have unfettered access to all interventions that can help prevent, diagnose, or treat illness. Even if an intervention is effective and reasonably safe, a decision must be made on whether to pay for it. If the outlays required are too high, its use may be restricted or funding withheld [1]. Although these decisions are taken routinely, they are difficult because assessing what is ‘too high’ involves determining what is reasonable to pay for a given benefit, and this requires putting a price on health gains and other intangibles for which there is no clear monetary valuation, as well as making comparisons across individuals. Ultimately, the value of health effects has to do with what one gives up to obtain them, but that ‘opportunity’ cost is rarely directly evident.

Where the trade-off can be gauged (for example, a hospital), it is feasible to approach the problem directly without explicitly valuing the consequences (e.g. to pay for this intervention we must close a bed) [2]. By contrast, determining value at the healthcare system level is extremely challenging because it is difficult to identify what will be displaced. This is even less clear in systems without a fixed budget, or where insurance premiums might be increased.

A common approach involves estimating the outlays and gains, and assigning them value without invoking opportunity cost directly. Since most outlays are measured in monetary units, their valuation is straightforward, but health gains are not naturally assessed this way. This poses a problem for weighing consequences measured in different clinical units. A common approach is to convert them into duration of life, modified by its quality, or quality-adjusted life-years (QALYs) [3], and then apply a price per QALY gained (i.e. ‘cost-effectiveness threshold’). If this gain exceeds the outlay [4], then the intervention is deemed worthwhile [5]. This determination is made without identifying what to displace or assessing the opportunity costs, although there have been recent attempts to do so [6, 7, 8].

The objective of our work was to produce a white paper to help further deliberations on determining value in the UK and elsewhere. The first author selected the other six experts to represent a wide range of viewpoints. The seven met in London, UK, in 2016, where each participant was asked to address: (1) What should health technology assessment (HTA) achieve? (2) What should be the approach to value-based pricing? Based on these deliberations, a draft paper was circulated for comment. Further telephone discussions led to additional versions, and four of the authors participated in an Issues Panel at an International Society for Pharmacoeconomics and Outcomes Research (ISPOR) meeting in Glasgow. There was substantial disagreement but no pressure to reach consensus. Instead, in this paper, we examine these challenging aspects to consider possible improvements to value determination in HTA. We begin by contemplating the aims of healthcare and then set out the requirements for value assessment. An examination of the options follows, ranging from staying the course1 with evidence-based reinforcement, through extending the cost per QALY approach to address some limitations, to abandoning it altogether and tacking to an alternative. Based on this examination, we make some recommendations for consideration by healthcare systems when reforming their HTA processes.

2 The Societal Aims for the Healthcare System

A healthcare system funded by citizens’ contributions must aim to meet the needs that arise due illness. It should strive to restore and maintain health, providing palliation when necessary. As the needs are vast and ways of meeting them ever-expanding, no system can fully achieve these goals, implying choices when deploying resources and the additional aim of ensuring these are consistent with society’s objectives. Various theories have emerged as to what these are.

One theory is that society wishes to maximize the aggregate health of the population [3], with the aggregate defined as the sum of health across all citizens, regardless of who bears what illness or benefit. An alternative view is that fairness matters [9] and that assuming a QALY is a QALY regardless of who receives it is not entirely fair because other aspects, such as severity of illness, are relevant [10]. A key criterion is that everyone should have equal access to necessary care [11], including not prioritizing those with greater capacity to benefit, either because more can be done for them or because they were healthier beforehand [12]. This means that it is acceptable to commit resources to manage an illness that is given higher societal priority. This alternative view is supported by studies indicating that citizens are concerned not only with their own treatment but also that of others [13]. This appeal to communitarian values [14] implies sharing scarce resources even if this reduces aggregate population health gain because more QALYs could have been obtained via some other use and is supported by empirical research [15, 16]. However, regardless of the criteria adopted, some interpersonal comparison is involved, even if implicitly.

If societal healthcare objectives expand beyond health gain, any number of other aspects might be considered, such as well-being, burden of illness, degree of unmet need, improved productivity, impact on caregivers, improved organizational efficiency, public dislike for all-or-nothing decisions that may discriminate (for example, against those with orphan diseases), maintenance of hope, and even the intervention’s innovativeness [15, 17, 18]. Pursuit of such objectives is not necessarily inconsistent with evaluation of cost effectiveness based on thresholds, but they augment or even constrain the cost-effective outcomes to be delivered.

3 Stay the Course—‘Efficiency First’, as Exemplified by National Institute for Health and Care Excellence (NICE) in the UK

The common approach, benefitting from extensive experience, judges health gains in terms of QALYs against a predetermined cost per QALY threshold, taken to represent the opportunity cost [19] or willingness-to-pay for new interventions [20]. The QALY, proposed as a health status index to aid clinical decisions involving trade-offs between duration and quality of life [21], is not itself a measure of value. Tariffs based on trade-offs among health states have to be applied but these only represent preference valuations if exceptional and, to many, unreasonable assumptions hold [22]. These tariffs consider the gain as having a linear value: zero QALYs equals no value, one QALY equals one unit of value, two QALYs equal double the value, etc. Health gains can only be considered if they can be measured in QALYs and the nature of the gain, whether in duration or quality of life, is not considered. No other factors are accorded value.

Recently, concerted efforts have been made to provide an evidence base for the threshold [6, 7, 8]. An average was computed in Australia and Spain using national mortality data and quality-of-life data from surveys. In the UK study, health authority data on cost increases and mortality reductions from one year to the next were used to compute a cost per QALY threshold in each of 23 broadly defined therapeutic areas (in some it could not be calculated), with substantial variation across resulting estimates (Fig. 1). Although the original theory implies that the threshold is the highest cost-effectiveness ratio [23], the researchers instead took the mean across areas, weighting more efficient practice areas higher. A recent systematic review of 120 empirical economic studies, reporting estimates of the value of a statistical life-year using a variety of approaches, found the median to be nearly US $200,000 among European studies, and even higher among American studies [24, 25].
Fig. 1

Empirical estimates of cost effectiveness relative to total cost, both axes on a logarithmic scale. The number beside each label is the total cost effectiveness, in thousands. Note, three categories—Social Care, Trauma, and Other—plotted on the horizontal axis have unknown cost effectiveness and only total cost. GBP Great British Pound, GI gastrointestinal, GU genitourinary, ID infectious disease, MSK musculoskeletal, UNK unknown, QALY quality-adjusted life-year

4 Broadening the Scope of the Cost per Quality-Adjusted Life-Year (QALY)

One possibility for broadening the cost per QALY approach is to incorporate other domains, such as the impact on relationships, autonomy and meaning [26], for example in a measure termed a WELBY (see Brazier and Tsuchiya [27]). Another idea is to consider that efficiency relative to the threshold is but one input into determining an intervention’s value [28], and other inputs can be incorporated by deliberation [29]. The methods for these deliberations are not well codified, including how to resolve disputes and how to weigh factors. Although most HTA agencies allow for discussion, it has been recognized that the numerical nature of the cost-effectiveness ratio and its clear juxtaposition to a threshold tends to hold sway over more qualitative aspects [30].

Adjusting the threshold [31] to take severity of illness into account is used with ‘end-of-life’ interventions in the UK [32]. This tactic can be extended to consider the ‘burden of illness’, as is done in Sweden [33], that is the degree to which an illness decreases QALYs [34] and applying different threshold prices accordingly. Indeed, an even higher threshold has been advocated for the treatment of rare diseases [35]. Apart from difficulties forecasting the QALYs lost [36, 37], it is unresolved if burden should be considered in proportional or absolute terms (thus favouring those with longer life expectancy or better quality of life beforehand), and how to take this into account in valuing an intervention.

5 Tacking Away from the Cost per QALY

The adjusted threshold approach could be further relaxed by eliminating the imperative to measure benefit in terms of QALYs. If separate thresholds are contemplated for different contexts, then this can be extended to each therapeutic area where the clinical measures of benefit are well established, and can include whatever aspects are relevant, even if they do not map well onto either duration or quality of life. The implied willingness to pay for benefits in a therapeutic area can be established using the ‘efficiency frontier,’ as is done in Germany [38]. However, if an intervention is the first in a therapeutic area, there is no established price, requiring use of a frontier from a similar area as a gauge.

Multicriteria decision analysis (MCDA) [39] provides for addressing a range of effects against a criteria set, which can include duration and quality of life, but also extend to whatever other aspects are relevant, scored and aggregated by quantitatively weighting their importance. For a choice among interventions, MCDA provides the necessary ranking without requiring further valuation, but it is unclear how this applies when deciding whether to fund a particular intervention. The MCDA score by itself does not address this, nor does it consider opportunity costs, and it presents its own technical difficulties [40].

Rather than aggregating multiple dimensions into a single score, the components can be kept distinct and each one valued separately [41]. Gradients of value (Fig. 2) are proposed to accord with the premise that value increases with severity, but there is diminishing marginal value as health is restored (also noted empirically in the study by Rowen et al. [41]). Different gradients are applied to each domain and other criteria affecting them can be incorporated. One point is priced and this used as an anchor. The maximum reimbursable price for any intervention is established by summing the monetary values of each type of gain (under the strong assumption that they are independent).
Fig. 2

Proposed relative value gradients according to quality-of-life improvement and initial severity

(data taken from Nord [48])

Many stated preference studies may have been misspecified because they were restricted to use value [42], which excludes option value (e.g. access in case of need; risk aversion) and altruistic motivations, which can be captured by choosing increments in private or public insurance premiums as the payment vehicle [43, 44]. These, or the Relative Social Willingness to Pay [45], imply a focus on social transfer payments and a greater role for the budgetary impact of healthcare programs [17, 46]. Fairness-based evaluation frameworks, sometimes collectively referred to as (social) ‘cost-value analysis’ [47], have not reached sufficient maturity to be recommended for formal implementation in HTA.

6 Discussion

This paper provides a summary of alternative approaches to assessing value to inform the selection of societally preferred healthcare options. Ultimately, everyone would like to see that funds expended on healthcare provide sufficient value to offset the opportunity costs. Nonetheless, positions diverge on what the healthcare system’s aims should be and these largely determine whether the preference is to stay the course or tack away. If maximizing aggregate health is the aim, then sticking with cost per QALY gained gauged against a threshold is appealing. Nevertheless, this implies society must be willing to tolerate the many known problems with the QALY [48, 49, 50]. However, there is growing dissatisfaction with the QALY’s constrained and somewhat dominating scope [51, 52] and thus with the inability of the cost per QALY approach to sufficiently capture the benefits valued by societies.

Several countries have already decided not to use a cost per QALY approach. It has been rejected in Germany [53] and Spain, and, despite increasing interest, remains largely unused in the United States [54]. In France, it is a minor component of HTA [55], and some Latin American countries have also shown reluctance to implement it [56]. These jurisdictions have opted largely for incorporating other aspects beyond clinical health in a deliberative approach, although the efficiency frontier and MCDA have both played an occasional role. While HTA in these healthcare systems functions, some will remain uncomfortable with such an approach, preferring to adjust the threshold. Indeed, relaxing the constraint on the threshold appears to be the direction of travel of even NICE in the UK [42]. An in-depth comparison of countries applying alternative approaches with respect to equity and efficiency, as well as transparency and legitimacy, should be high on the research agenda.

Although this tacks away from the foundation of maximizing aggregate health, it complicates providing an empirical basis for decision making. By how much should the threshold be adjusted and for which other benefits? Who makes these modifications and on what basis? What are the implications for considering opportunity cost? Tacking away requires answers to these questions.

Regardless of the manner by which the cost per QALY approach is extended—qualitative deliberations, quantitative adjustments, or both—why stick with the QALY if maximizing aggregate health is not the sole aim? One reason is convenience. Many societies have invested considerable efforts in implementing it, and awareness of the concept has already extended somewhat beyond the specialized confines of HTA, and the introduction of completely new concepts will face substantial hurdles. Another justification is that staying with the threshold provides a benchmark for moving towards other criteria for which explicit arguments have to be made. As with staying the course, a society that moves towards an extended QALY approach is still subject to its violations of the fundamental assumptions and the resulting uncertainties in determining a justifiable reimbursement price. If a single threshold is replaced with a sliding scale, the other criteria and justifications for supporting them have to be explicit.

Relying on an efficiency frontier is one way to move away from both the QALY and the single threshold. However, the efficiency frontier allows for ever-higher prices per unit of benefit as the absolute benefit grows (i.e. decreasing efficiency), with no inherent guidance on the minimum efficiency required. The efficiency frontier approach also assumes that established expenditures reflect societal priorities, but there is no reason to believe this is reliably so in the absence of explicit valuations across therapeutic areas.

MCDA is another way to extend HTA beyond the single dimension of the QALY, but this method has not been developed for recurring decisions at a societal level. The score that results has no inherent meaning, although it can be used in an efficiency frontier as an integrated measure of benefit, thus removing one difficulty with this frontier approach [57].

Separating the valuation of life-extension from quality-of-life improvements avoids difficult-to-sustain assumptions involving a single integrating index, but requires the untenable assumption that there are no interactions between the domains. Moreover, establishing the anchor point is challenging, particularly if the domains are not independently evaluable. This also raises the question of whether a universal quality-of-life index applies in disparate illnesses. Can an index such as the EQ-5D [58] be leveraged in this regard [59]? However, there is nothing preventing extension of the method to other dimensions. By scaling the relative value of gains in disparate domains and anchoring these to specific monetary amounts, the approach implies cross-domain relative values.

Establishing an acceptable price to pay for a given set of benefits is a problem that goes beyond the remit of the HTA into political and other spheres because, in the end, it has to do with how much of society’s resources are allocated to the healthcare budget and what priorities are set therein. Perhaps then, HTA should return to the foundation of opportunity cost, and, rather than appraise whether an intervention is ‘worth it’, provide instead an estimate of what would have to be given up (with all its consequences) or how much additional funding would be needed if nothing is displaced. This would provide the trade-off consequences explicitly, without the need for a cost-effectiveness threshold or even a QALY. The decision-making body thus stands to be swayed less by a single threshold-driven decision rule and can accord due weight to all relevant inputs. This deliberative decision making is arguably much more difficult, and, without explicit well-defined processes and decision rules, may be less transparent and consistent.

7 Conclusion

Given the substantial implications of these decisions to patients, families and society in general, it behoves our field to strive for an approach that addresses as fully as possible the healthcare system’s aims, is methodologically sound, and is feasible to implement within the available timelines and resource constraints. None of the alternatives examined yet achieve this.


  1. 1.

    When two boats at sea are on a collision course, the skipper on the burdened vessel (because, for example, it is on port tack and approaching a starboard tack boat) radios or yells ‘stay the course’ or ‘hold your course’ to the stand-on vessel if she thinks she can safely cross in front; otherwise the burdened vessel ‘tacks away’ to avoid a collision.


Compliance with Ethical Standards

Conflict of interest

J Jaime Caro, John Brazier, Jonathan Karnon, Peter Kolominsky-Rabas, Alistair McGuire, Erik Nord and Michael Schlander declare no financial conflicts of interest. All authors are well known for their views on the subject matter discussed in this paper.


A meeting in London and subsequent deliberations were supported by an unrestricted grant from Janssen Pharmaceutica. Two senior staff from Janssen were present as observers during the panel meeting; they have had access to the draft versions of this paper and have provided very helpful comments, but neither they nor the company they represent had any editorial control. The agreement with Janssen ensured the authors’ independence in study design, data interpretation, writing, and submitting the manuscript for publication. All decisions regarding the content of this paper and the decision to publish were made independently by the authors.

Overall Guarantor

J. Jaime Caro will serve as overall guarantor for this study and manuscript. All authors participated in the discussions that led to this paper and in the revisions of all drafts. All authors approved the final version submitted for publication.


  1. 1.
    Dranove D. What’s your life worth: healthcare rationing … Who lives? Who dies? And who decides?. Upper Saddle River: FT Press; 2003. p. 1–191.Google Scholar
  2. 2.
    McGregor M, Brophy JM. End-user involvement in health technology assessment (HTA) development: a way to increase impact. Int J Technol Assess Health Care. 2005;21:263–7.Google Scholar
  3. 3.
    Weinstein M, Torrance G, McGuire A. QALYs: the basics. Value Health. 2009;12:S5–9.Google Scholar
  4. 4.
    Meltzer DO, Basu A, Sculpher MJ. Ch 2 theoretical foundations of cost-effectiveness analysis in health and medicine. In: Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG, editors. Cost-effectiveness in health and medicine. Oxford: Oxford University Press; 2017.Google Scholar
  5. 5.
    Dix Smith M, Drummond M, Brixner D. Moving the QALY forward: rationale for change. Value Health. 2009;12:S1–4.Google Scholar
  6. 6.
    Claxton K, Martin S, Soares M, et al. Methods for the estimation of the NICE cost-effectiveness threshold. Health Technol Assess. 2015;19:1–7.Google Scholar
  7. 7.
    Edney LC, Haji AAH, Cheng TC, Karnon J. Estimating the reference incremental cost-effectiveness ratio for the Australian health system. PharmacoEconomics. 2018;36:239–52.Google Scholar
  8. 8.
    Vallejo-Torres L, Garcia-Lorenzo B, Serrano-Aguilar P. Estimating a cost-effectiveness threshold for the Spanish NHS. Health Econ. 2018;27(4):746–61.Google Scholar
  9. 9.
    Nord E, Pinto JL, Richardson J, Menzel P, Ubel P. Incorporating societal concerns for fairness in numerical valuations of health programmes. Health Econ. 1999;8:25–39.Google Scholar
  10. 10.
    Nord E. The trade-off between severity of illness and treatment effect. Health Policy. 1993;24:227–38.Google Scholar
  11. 11.
    Daniels N. Just healthcare. Cambridge: Harvard University Press; 1985.Google Scholar
  12. 12.
    McKie J, Richardson J. Social preferences for prioritizing the treatment of severely ill patients: the relevance of severity, expected benefit, past health and lifetime health. Health Policy. 2017;121:913–22.Google Scholar
  13. 13.
    McKie J, Shrimpton B, Richardson J, Hurworth R. The monetary value of a life year: evidence from a qualitative study of treatment costs. Health Econ. 2011;20:945–57.Google Scholar
  14. 14.
    Mooney G. Is it not time for health economists to rethink equity and access? Health Econ Policy Law. 2009;4:209–21.Google Scholar
  15. 15.
    Dolan P, Tsuchiya A. Health priorities and public preferences: the relative importance of past health experience and future health prospects. J Health Econ. 2005;24:703–14.Google Scholar
  16. 16.
    Richardson J, Iezzi A, Maxwell A. How important is severity for the evaluation of health services: new evidence using the relative social willingness to pay instrument. Eur J Health Econ. 2017;18:671–83.Google Scholar
  17. 17.
    Culyer AJ. Ethics, priorities and cancer. J Cancer Policy. 2017;11:6–11.Google Scholar
  18. 18.
    Lakdawalla DN, Romley JA, Sanchez Y, Maclean JR, Penrod JR, Philipson T. How cancer patients value hope and the implications for cost-effectiveness assessments of high-cost cancer therapies. Health Aff. 2012;31:676–82.Google Scholar
  19. 19.
    McCabe C, Claxton K, Culyer AJ. The NICE cost-effectiveness threshold. Pharmacoeconomics. 2008;26:733–44.Google Scholar
  20. 20.
    Gyrd-Hansen D. Willingness to pay for a QALY. Pharmacoeconomics. 2005;23:423–32.Google Scholar
  21. 21.
    Pliskin JS, Shepard DS, Milton W. Utility functions for life years and health status. Oper Res. 1980;28:206–24.Google Scholar
  22. 22.
    Broome J. QALYs. J Public Econ. 1993;50:149–67.Google Scholar
  23. 23.
    Culyer AJ. Cost-effectiveness thresholds in healthcare: a bookshelf guide to their meaning and use. Health Econ Pol Law. 2016;11:415–32.Google Scholar
  24. 24.
    Viscusi WK. Pricing lives: guideposts for a safer society. Princeton: Princeton University Press; 2018.Google Scholar
  25. 25.
    Schlander M, Schwarz O, Hernandez D, Schaefer R. New estimates of the willingness-to-pay for a statistical life year: a systematic review of the empirical economic literature. Value Health. 2018;21:S11.Google Scholar
  26. 26.
    Peasgood T, Ward SE, Brazier J. Health-state utility values in breast cancer. Expert Rev Pharmacoecon Outcomes Res. 2010;10:553–66.Google Scholar
  27. 27.
    Brazier J, Tsuchiya A. Improving cross-sector comparisons: going beyond the health-related QALY. Appl Health Econ Health Policy. 2015;13:557–65.Google Scholar
  28. 28.
    Garrison L, Kamal-Bahl S, Towse A. Toward a broader concept of value: identifying and defining elements for an expanded cost-effectiveness analysis. Value Health. 2017;20:213–6.Google Scholar
  29. 29.
    Baltussen R, Jansen MP, Mikkelsen E, Tromp N, Hontelez J, Bijlmakers L, et al. Priority setting for universal health coverage: we need evidence-informed deliberative processes, not just more evidence on cost-effectiveness. Int J Health Policy Manag. 2016;5:615–8.Google Scholar
  30. 30.
    Ghijben P, Gu Y, Lancsar E, Zavarsek S. Revealed and stated preferences of decision makers for priority setting in health technology assessment: a systematic review. PharmacoEconomics. 2018;36:323–40.Google Scholar
  31. 31.
    Stolk EA, van Donselaar G, Boruwer WBF, Busschbach JJV. Reconciliation of economic concerns and health policy: illustration of an equity adjustment procedure using proportional shortfall. PharmacoEconomics. 2004;22:1097–107.Google Scholar
  32. 32.
    Collins M, Latimer N. NICE’s end of life decision making scheme: impact on population health. BMJ. 2013;346:22–3.Google Scholar
  33. 33.
    Persson U. Value based pricing in Sweden: lessons for design?
  34. 34.
    Rowen D, Brazier J, et al. Eliciting societal preferences for weighting QALYs for burden of illness and end of life. Med Decis Making. 2016;36:210–22.Google Scholar
  35. 35.
    Changes to NICE drug appraisals: what you need to know. Accessed 18 Jul 2018.
  36. 36.
    Brazier J, Ratcliffe J. Measurement and valuation of health for economic evaluation. In: Quah SR, editor. International encyclopedia of public health. Amsterdam: Elsevier; 2016. p. 586–93.Google Scholar
  37. 37.
    Drummond MF, Iglesias CP, Cooper NJ. Systematic reviews and economic evaluations conducted for the National Institute for Health and Clinical Excellence in the United Kingdom: a game of two halves? Int J Technol Assess Healthc. 2008;24:146–50.Google Scholar
  38. 38.
    Caro JJ, Nord E, Siebert U, McGuire A, McGregor M, Henry D, et al. The efficiency frontier approach to economic evaluation of health-care interventions. Health Econ. 2010;19:1117–27.Google Scholar
  39. 39.
    Marsh K, Lanitis T, Neasham D, Orfanos P, Caro J. Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. Pharmacoeconomics. 2014;32:345.Google Scholar
  40. 40.
    Marsh K, Sculpher M, Caro JJ, Tervonen T. The use of MCDA in HTA: great potential but more effort is needed. Value Health. 2018;21:394–7.Google Scholar
  41. 41.
    Nord E. Beyond QALYs: multi-criteria-based estimation of maximum willingness to pay for health technologies. Eur J Health Econ. 2018;19:267–75.Google Scholar
  42. 42.
    Smith RD. Use, option and externality values: are contingent valuations studies in healthcare mis-specified? Health Econ. 2007;16:861–9.Google Scholar
  43. 43.
    Smith RD, Sach TC. Contingent valuation: what needs to be done? Health Econ Policy Law. 2010;5:91–111.Google Scholar
  44. 44.
    Gyrd-Hansen D. Using the stated preference technique for eliciting valuations: the role of the payment vehicle. Pharmacoeconomics. 2013;31:853–61.Google Scholar
  45. 45.
    Richardson J, Iezzi A, Sinha K, Khan MA, Mckie J. An instrument for measuring the social willingness to pay for health state improvement. Health Econ. 2014;23:792–805.Google Scholar
  46. 46.
    Richardson J, McKie J. Economic evaluation of services for a National Health scheme: the case for a fairness-based framework. J Health Econ. 2007;26:785–99.Google Scholar
  47. 47.
    Nord E. Cost-value analysis of health interventions: introduction and update on methods and preference data. PharmacoEconomics. 2014;33:89–95.Google Scholar
  48. 48.
    Schlander M. Measures of efficiency in healthcare: QALMs about QALYs? Z Evid Fortbild Qual Gesundhwes. 2010;104:214–26.Google Scholar
  49. 49.
    Beresniak A, Medina-Lara A, Auray JP, et al. Validation of the underlying assumptions of the quality-adjusted life-years outcome: results from the ECHOUTCOME European Project. PharmacoEconomics. 2015;33:61–9.Google Scholar
  50. 50.
    Gafni A, Birch S. Preferences for outcomes in economic evaluation: an economic approach to addressing economic problems. Soc Sci Med. 1995;40:767–76.Google Scholar
  51. 51.
    Birch S, Gafni A. Information created to evade reality (ICER): things we should not look to for answers. Pharmacoeconomics. 2006;24:1121–31.Google Scholar
  52. 52.
    Neumann PJ, Cohen JT. QALYs in 2018: advantages and concerns. JAMA. 2018;319:2473–4.Google Scholar
  53. 53.
    Institute for Quality and Efficiency in Healthcare (IQWiG). Methods for Assessment of the Relation of Benefits to Costs in the German Statutory Healthcare System. Accessed 16 Jun 2009.
  54. 54.
    Neumann PJ, Weinstein MC. Legislating against use of cost-effectiveness information. N Engl J Med. 2010;363:1495–7.Google Scholar
  55. 55.
    Rumeau-Pichon C, Harousseau JL. Analysis of cost-effectiveness assessments in France by the French National Authority for Health (HAS). Value Health. 2014;17:A414.Google Scholar
  56. 56.
    Brixner D, Maniadakis N, Kalo Z, et al. Applying multi-criteria decision analysis (MCDA) simple scoring as an evidence-based HTA methodology for evaluating off-patent pharmaceuticals (OPPs) in emerging markets. Value Health Region Issues. 2017;13:1–6.Google Scholar
  57. 57.
    Marsh K, IJzerman M, Thokala P, et al. Multiple criteria decision analysis for healthcare decision making—emerging good practices: report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health. 2016;19:125–37.Google Scholar
  58. 58.
    Brooks R, Rabin R, de Charro F, editors. The measurement and valuation of health status using EQ-5D: a European perspective. Dordrecht: Kluwer; 2003.Google Scholar
  59. 59.
    Devlin NJ, Parkin D, Browne J. Patient-reported outcome measures in the NHS: new methods for analysing and reporting EQ-5D data. Health Econ. 2010;19:886–905.Google Scholar

Copyright information

© The Author(s) 2018

Open AccessThis article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (, which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • J. Jaime Caro
    • 1
    • 2
    • 8
    Email author
  • John E. Brazier
    • 3
  • Jonathan Karnon
    • 4
  • Peter Kolominsky-Rabas
    • 5
  • Alistair J. McGuire
    • 1
  • Erik Nord
    • 6
  • Michael Schlander
    • 7
  1. 1.London School of Economics and Political ScienceLondonUK
  2. 2.EvideraWalthamUSA
  3. 3.School of Health and Related Research (ScHARR), University of SheffieldSheffieldUK
  4. 4.School of Public HealthUniversity of AdelaideAdelaideAustralia
  5. 5.Interdisciplinary Centre for Health Technology Assessment and Public Health, Friedrich-Alexander University of Erlangen-NürnbergErlangenGermany
  6. 6.Norwegian Institute of Public HealthOsloNorway
  7. 7.Division of Health EconomicsGerman Cancer Research Center (DKFZ), University of HeidelbergHeidelbergGermany
  8. 8.LincolnUSA

Personalised recommendations