Ten years have passed since the publication of the EQ-5D-5L descriptive system  and 5 years since a value set for England was first published . These years have been squandered by a protracted and misguided debate in the UK, culminating in the National Institute for Health and Care Excellence (NICE) rejecting the published value set . In this editorial, I provide one outsider's view of the debacle and present some ideas for discussion that might inform future decisions about new methods.
2 A Little History
Publication of the final (peer-reviewed) version of the value set in August 2017  triggered a flurry of editorials and commentaries [4,5,6]. Contributors to the debate varied in their perspectives of whether or not NICE should recommend the use of the 5L value set, but all were broadly supportive of a ‘pause’  and the ‘understandable care’  and ‘caution’  of NICE's approach .
A team was appointed to develop a new value set for the UK, and there is, as far as I understand it, a clear roadmap to the adoption of the EQ-5D-5L. In the interim, NICE has recommended methods for mapping from 5L data to the old 3L value set. But sticking plasters can be replaced; NICE recently proposed an alternative mapping function to the earlier recommendation [8, 9].
The new valuation study and the various mapping function recommendations are second-best solutions to a problem of NICE's making. But NICE is not the only culprit here. The events highlight some troubling tendencies in our collective attitude to methodological progress in health technology assessment (HTA). Based on the advice of numerous academics, NICE has adopted a pernicious kind of status quo bias borne of a misunderstanding of what it means to deliver consistent decision making.
3 Conceptual Consistency
NICE publishes methodological guidance for the conduct of technology appraisals ; it will soon publish a new manual. Health economists in the UK (and further afield) take this seriously. Guidance facilitates consistency. Without consistency in the approach used by analysts, results would be less comparable and NICE's job of supporting an efficient allocation of resources would become more difficult.
The scope of NICE's decision making should determine the appropriate methods to be employed. It would be an error to conflate the underlying objectives and scope of decision making with the specific methods. The old distinction between normative and positive economics may be helpful here; “the positive deals with evaluating means while the normative deals with evaluating ends” . EQ-5D value sets represent means, not ends.
When NICE reviews its methods guidance, as it has been doing for the past year or so , it considers matters less trivial than the latest technical developments in methodology. Certain aspects of methodological distinction may be more appropriately framed as conceptual or normative. Examples relevant to health state valuation include the sources of values (e.g. ‘public’ vs ‘patient’ values ) and the objects of value (e.g. whether to consider outcomes ‘beyond health’ ).
Let us consider the example of patient values or preferences. NICE recommends the use of public values in the estimation of health state utility values. Other HTA agencies make other recommendations. There are sound reasons to favour either approach (or both, or an alternative). Importantly, these differences do not arise from mere measurement error or bias but have fundamentally different conceptual bases and ethical justifications. A shift from one to the other is rightly contentious and contested in the literature; no approach can ever be proven superior.
HTA agencies may reasonably shift from one approach to another. However, this would render appraisals under each conceptual regime incomparable. For example, if NICE recommended using patient preferences to value health states, judging the value of technologies historically evaluated based on societal values would not be possible. Past decisions could be deemed to be ‘wrong’ based on the current scope of decision making. Thus, there is merit in HTA agencies supporting conceptual consistency through time. There may be a need to make changes in light of new thinking, but conceptual consistency ensures that decisions about care provided by the National Health Service (NHS) can be justified based on current priorities. If the conceptual basis for the HTA process changes, then the provision of technologies evaluated based on retired concepts is (to a greater or lesser extent) undermined.
This has little to do with what I would typically call ‘methodology’.
4 Methodological Fluidity
Quantitative and qualitative methods are subject to constant development. Undoubtedly, new methods require testing; researchers must assess the extent to which methods are valid in identifying the truth that we seek. This applies to methods for health state valuation and HTA more broadly. Recent decades have seen extensive development in methods for discrete choice experiments , eliciting values for states ‘worse than dead’ , and various other tweaks specific to EQ-5D valuation .
It is reasonable to make methodological recommendations based on the latest research and developments. Changes in such recommendations need not be considered in the same way that a conceptual shift, such as a change in the source of valuation, must be considered. Relevant methods, in this case, may relate to preference elicitation, including data collection strategies and methods for modelling data. It is on these grounds that the EQ-5D-5L value set for England faced criticism .
The favourable properties of the EQ-5D-5L descriptive system, compared with the 3L, have been demonstrated extensively . This should not come as a surprise; researchers developed the 5L precisely because a long history of methodological work suggests that this would be the case. NICE’s apprehensions lie in the value set. Here too, and for similar reasons, we see substantial advantages of more modern methods for preference elicitation [17, 20, 21].
5 A Different Kind of Decision Problem
When assessing technologies, a necessary step is to identify a reasonable comparator. In most cases, alternatives should be compared with the status quo. NICE has failed to recognise the pertinence of this principle in the adoption of new methods.
The decision to reject the 5L value set seemingly gave no consideration to the alternative, which is to continue using the Measurement and Valuation of Health (MVH) study 3L value set developed in the early nineties . Any criticism of the 5L value set for England is irrelevant unless the same criticism cannot be levelled at the MVH value set. Show me one defence of the 3L value set in comparison with the 5L value set! And yet, here we find ourselves, using the 3L value set.
Normative statements cannot be tested. The validity of methods to derive a value set can be tested. However, they cannot—or at least should not—be tested against some unattainable perfection. In lieu of a methodological ‘gold standard’ (as noted by Werner Brouwer and Denzil Fiebig in their expert reviews of the value set [23, 24]), perfection became the comparator. Show me a quantitative study without error or room for improvement! This was a predictable problem for the ‘quality assurance’ process .
The methods that we employ must be evaluated against the next best alternative. Perversely, methodological developments since the 5L value set for England study was conducted have been used against it. On this basis, as methods are continually refined, newly published value sets will always fall victim to future improvements.
The 5L saga is a textbook example of perfect being the enemy of the good. In the context of an HTA system that is heavily driven by cost per quality-adjusted life-year (QALY) analyses, it has likely cost us dearly.
6 Concluding Remarks
The NHS is habitually mocked for using fax machines and Windows XP because better technologies are available. Could this be a distinctly British problem?
NICE will soon publish a new manual for HTAs and, based on the most recent draft, it is likely to maintain its recommendation against using the EQ-5D-5L . The guidance that NICE provides for methods of technology assessment, and the process for revising these recommendations, must recognise that conceptual consistency and methodological fluidity are compatible principles. And researchers should cease resting on their laurels.
NICE’s reference case should be revised regularly, including explicit reconsideration of the recommended measurement instrument to estimate QALYs. Where there have been methodological advances within a conceptually consistent framework, new measures should be recommended. In some cases, it may be appropriate for NICE to reconsider the conceptual basis for outcome measurement, which would require a normative case to be made.
That the EQ-5D-5L represents methodological developments should not act as a barrier to use by NICE. Results derived using the 5L value set will differ from the 3L value set , but this is not relevant to the decision problem so long as both seek to measure the same thing and are conceptually consistent. While we await the new UK value set, NICE should recommend the 5L value set for England. Axe the fax! To hell with the 3L!
Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20:1727–36.
Devlin N, Shah K, Feng Y, Mulhern B, Van Hout B. Valuing health-related quality of life: an EQ-5D-5L value set for England. London: the Office of Health Economics. 2016. https://www.ohe.org/publications/valuing-health-related-quality-life-eq-5d-5l-value-set-england.
Devlin NJ, Shah KK, Feng Y, Mulhern B, van Hout B. Valuing health-related quality of life: an EQ-5D-5L value set for England. Health Econ. 2018;27:7–22.
Brazier J, Briggs A, Bryan S. EQ-5D-5L: smaller steps but a major step change? Health Econ. 2018;27:4–6.
Devlin N, Brazier J, Pickard AS, Stolk E. 3L, 5L, what the L? A NICE Conundrum. Pharmacoeconomics. 2018;36:637–40.
Round J. Once bitten twice shy: thinking carefully before adopting the EQ-5D-5L. Pharmacoeconomics. 2018;36:641–3.
National Institute for Health and Care Excellence. Position statement on use of the EQ-5D-5L valuation set. 2017. https://www.nice.org.uk/Media/Default/About/what-we-do/NICE-guidance/NICE-technology-appraisal-guidance/eq5d5l_nice_position_statement.pdf. Accessed 4 Apr 2018.
National Institute for Health and Care Excellence. NICE health technology evaluations: the draft manual. 2021. https://www.nice.org.uk/Media/Default/About/what-we-do/our-programmes/nice-guidance/chte-methods-and-processes-consultation/health-technology-evaluations-manual.docx.
Hernandez Alava M, Wailoo A, Pudney S. Methods for mapping between the EQ-5D-5L and the 3L for technology appraisal. Sheffield: University of Sheffield: Decision Support Unit; 2017. p. 34.
National Institute for Health and Care Excellence. Guide to the methods of technology appraisal 2013. In: London: National Institute for Health and Care Excellence; 2013.
Dugger WM, Waller W. Radical institutionalism: from technological to democratic instrumentalism. Rev Soc Econ. 1996;54:169–90.
National Institute for Health and Care Excellence. Changes we’re making to health technology evaluation. 2021. https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance/changes-to-health-technology-evaluation. Accessed 14 Nov 2021.
Versteegh MM, Brouwer WBF. Patient and general public preferences for health states: a call to reconsider current guidelines. Soc Sci Med. 2016;165:66–74.
Brazier J, Tsuchiya A. Improving cross-sector comparisons: going beyond the health-related QALY. Appl Health Econ Health Policy. 2015;13:557–65.
Bahrampour M, Byrnes J, Norman R, Scuffham PA, Downes M. Discrete choice experiments to generate utility values for multi-attribute utility instruments: a systematic review of methods. Eur J Health Econ. 2020;21:983–92.
Devlin N, Buckingham K, Shah K, Tsuchiya A, Tilling C, Wilkinson G, et al. A comparison of alternative variants of the lead and lag time TTO. Health Econ. 2013;22:517–32.
Stolk E, Ludwig K, Rand K, van Hout B, Ramos-Goñi JM. Overview, update, and lessons learned from the international EQ-5D-5L valuation work: version 2 of the EQ-5D-5L valuation protocol. Value Health. 2019;22:23–30.
van Hout B, Mulhern B, Feng Y, Shah K, Devlin N. The EQ-5D-5L value set for England: response to the “Quality Assurance.” Value Health. 2020;23:649–55.
Feng Y-S, Kohlmann T, Janssen MF, Buchholz I. Psychometric properties of the EQ-5D-5L: a systematic review of the literature. Qual Life Res. 2021;30:647–73.
Janssen MF, Bonsel GJ, Luo N. Is EQ-5D-5L better than EQ-5D-3L? A head-to-head comparison of descriptive systems and value sets from seven countries. Pharmacoeconomics. 2018;36:675–97.
Feng Y, Devlin NJ, Shah KK, Mulhern B, van Hout B. New methods for modelling EQ-5D-5L value sets: an application to English data. Health Econ. 2018;27:23–38.
MVH Group. The measurement and valuation of health: final report on the modelling of valuation tariffs. York: Centre for Health Economics, University of York; 1995.
Brouwer W. The EQ-5D-5L valuation set for England: a view from the side. 2019. https://www.nice.org.uk/Media/Default/About/what-we-do/NICE-guidance/NICE-technology-appraisals/Brouwer-advice-EQ-5D-5L-valuation.pdf.
Fiebig DG. Expert evaluation of EQ-5D-5L. 2019. https://www.nice.org.uk/Media/Default/About/what-we-do/NICE-guidance/NICE-technology-appraisals/Fiebig-advice-EQ-5D-5L-valuation.pdf.
Sampson C. Bad reasons not to use the EQ-5D-5L. The Academic Health Economists’ Blog. 2018. https://aheblog.com/2018/03/23/bad-reasons-not-to-use-the-eq-5d-5l/. Accessed 5 Nov 2021.
Mulhern B, Feng Y, Shah K, Janssen MF, Herdman M, van Hout B, et al. Comparing the UK EQ-5D-3L and English EQ-5D-5L value sets. Pharmacoeconomics. 2018;36:699–713.
Hernandez Alava M, Pudney S, Wailoo A. Quality review of a proposed EQ-5D-5L value set for England. EEPRU. 2018. http://www.eepru.org.uk/wp-content/uploads/2017/11/eepru-report-eq-5d-5l-27-11-18-final.pdf. Accessed Oct 2018.
Conflicts of Interest
At the time of writing, I was employed by the Office of Health Economics (OHE), which was the lead organisation in the research to develop the value set for England. I did not work for OHE at the time of the research and none of the original authors were consulted in the drafting of this editorial or the ideas within. Two of the authors of the independent review  (Hernandez Alava and Wailoo) supervised me as a Masters student. Ours is a small world. I have received research funding and benefits in kind from the EuroQol Group but I am not a EuroQol member. The views expressed are solely those of the author and do not necessarily represent the views of OHE, the EuroQol Group, or anyone else.
No funding or other support was received for the writing of this editorial.
About this article
Cite this article
Sampson, C. NICE and the EQ-5D-5L: Ten Years Trouble. PharmacoEconomics Open 6, 5–8 (2022). https://doi.org/10.1007/s41669-021-00315-1