Could or Should We Use MCDA in the French HTA Process?
Over the last decade, an increasing literature has been highlighting the benefits of introducing multi-criteria decision analysis (MCDA) in healthcare decision processes , including the emerging ISPOR Good Practices Task Force [2, 3]. Based on decision theory dating back to the 1960s [4, 5, 6, 7], MCDA constitutes a tool to address complex decisions involving multiple criteria, especially where other methods fail to consider more than the outcome of interest. A set of criteria is first established; weights and scores are combined with performance data for each alternative. This allows for the generation of an estimate of the relative value of alternative courses of action. The approach has advantages including (1) exhaustiveness: MCDA can take into account the multiplicity of outcomes; (2) flexibility: scores can be expressed in ordinal or numerical terms; and (3) inclusiveness: MCDA involves both experts and stakeholders in the evaluation process. It also has drawbacks, in particular (1) controversy: some MCDA methods may lack theoretical foundations, and the rigor of some applications of MCDA has been questioned ; however, these concerns can be addressed by adopting MCDA methods founded on expected utility theory and following established good practice ; (2) subjectivity: the weighting framework is generally based on human judgment and, as such, can be subject to bias; nonetheless, the subjective judgment is inherent in the decisions being made and is not an artefact of MCDA; and (3) non-comparability: the ad-hoc aggregation of outcomes makes results context-dependent and difficult to transfer to other situations or cases.
In France, the conceptual framework of MCDA was founded by Bernard Roy  and has since been applied to industrial and infrastructure investments [9, 10]. Despite French academics’ role in the development of MCDA, there has been little use in health technology assessment (HTA), except for a few attempts aiming at an assessment of medical devices in hospital-based HTA such as drug-eluting beads for trans-catheter arterial chemoembolization . The objective of this editorial is to discuss the possible implementation of MCDA in the French HTA process. In addition, it could serve as a case study highlighting the challenges of introducing MCDA in other countries’ HTA.
As stated above, one downside of MCDA is that it covers a wide range of approaches, which differ mainly in the treatment of performance data and the elicitation of preferences. A recent classification suggested for use by HTA decision makers  distinguishes three categories—qualitative MCDA: the deliberating committee uses a performance matrix to derive a ranking of alternatives; quantitative MCDA: before deliberation, a value-measurement model based on explicit criteria weights is applied to the performance matrix; and MCDA with decision rules: the committee uses pre-defined decision rules to be applied to the performance matrix. This typology is used to examine the potential implementation of MCDA in the French HTA context.
2 A Qualitative Multi-Criteria Decision Analysis (MCDA) is Implicitly Considered in Reimbursement and Price Negotiation of Drugs and Medical Devices
The French National Authority for Health (HAS) does not formally include MCDA in its HTA. Reimbursement is based on clinical benefit as assessed by the Transparency Committee (CT) for drugs and the Medical Device and Health Technology Evaluation Committee for medical devices (CNEDIMTS), respectively. The Committee of Economic Evaluation and Public Health (CEESP) evaluates submissions (cost-effectiveness and budget impact analyses) by manufacturers for innovative drugs and medical devices with a claimed added clinical benefit (major, important or moderate) and an expected sales revenue above €20 million over 2 years . In the absence of a willingness-to-pay threshold, CEESP provides the pricing committee of the Ministry of Health (CEPS) with opinions on the level of the incremental cost-effectiveness ratios (ICERs), their uncertainty, and the budget impact in relation to the price claimed by the manufacturer.
The assessment of clinical benefit by CT and CNEDIMTS can be considered as a form of qualitative MCDA with time-consistent and transparent criteria. For example, the clinical benefit of a drug comprises (1) severity of the disease and its impact on morbidity and mortality; (2) clinical efficacy, effectiveness and safety; (3) the type of drug—preventive, symptomatic or curative; (4) position in treatment strategy compared with alternatives; and (5) impact on public health (e.g., burden of disease). However, the deliberative process resulting in a recommendation for reimbursement does not use pre-defined criteria weights and does not involve any inputs beyond those from health professionals of the two committees.
3 Using Quantitative MCDA in Price Negotiations for Drugs and Medical Devices is Challenging
There are several quantitative MCDA frameworks that could potentially support HTA. Nobody denies the benefits of quantitative MCDA in enhancing transparency and in integrating preference dimensions that are not usually captured by quality-adjusted life-years (QALYs) through a framework based on explicit and pre-specified criteria. However, the current implementation of quantitative MCDA raises several issues.
In price negotiations with manufacturers, economic evaluations (called efficiency opinions in the French HTA context) are being used by CEPS as one criterion among other predefined criteria such as added clinical benefit, prices of comparators, expected sales volume, and the European reference price . A quantitative MCDA could, in principle, fit in but is difficult to implement in practice as long as negotiations regarding discounts and rebates remain confidential. Moreover, policy makers may not want to commit to a common set of weights for these criteria, seeing their decisions as context dependent and reflecting the heterogeneous preferences of the public stakeholders involved.
Beyond these institutional considerations, the implementation of quantitative MCDA in HTA [15, 16, 17, 18, 19, 20] has met two major criticisms related to the way it has been used to date [1, 2, 8, 21, 22, 23]. First, the additive value model simply combines criteria and their weights. It cannot accommodate overlapping criteria and non-independence of preferences. For example, the EVIDEM (Evidence and Value: Impact on Decision Making) framework  provides a performance matrix that includes overlapping economic criteria such as budget impact, cost effectiveness, and impact on other expenditures. Furthermore, these criteria cannot be perceived as benefit attributes . Second, the connection between the scaling of criteria and weights is ambiguous . This is particularly problematic for innovative interventions (e.g., tumor-agnostic treatments), where the range of treatment benefits (e.g., overall survival) and key safety parameters (e.g., adverse reaction) is often unknown.
These criticisms can be overcome by implementing an elicitation process that reflects the multiplicative nature of preferences. However, other questions will emerge: do policy makers really need to know the preferences of stakeholders regarding trade-offs between benefits and costs of health technologies? And if the answer is yes, how can one estimate the opportunity cost that supports a funding decision in such a framework? More specifically, how can one measure the ICER threshold? Today, to the best of our knowledge, there are no MCDA applications addressing these issues for reimbursement and pricing decisions.
4 Combining MCDA with Decision Rules to Improve the HTA Deliberation Process
Without explicitly using MCDA terminology, HTA agencies may apply decision-making rules designed to better structure their deliberation process. Specifically, NICE uses a threshold of £20,000 per QALY gained but may adopt a higher threshold for special conditions and therapies (e.g., end-of-life treatments).
In the case of France, the inclusion of decision rules permitting adjustment of the cost-effectiveness and budget impact analyses is certainly a promising perspective. It may enhance the consistency of efficiency assessments providing improved guidance in price negotiations. However, it is challenging to define decision rules in the absence of a willingness-to-pay threshold. Even in the presence of such a threshold, the rules governing adjustments to specific contexts (e.g., innovation) or disease characteristics (e.g., orphan disease, end-of-life treatment) need to be specified. This may call for a pilot project aiming to re-structure the deliberation process concerning drugs and medical devices, in particular by clarifying the role of economic evaluation.
This contribution examined how MCDA could be included in the French HTA process. Despite its expected benefits, implementing quantitative MCDA seems premature at this point. Too many issues have to be resolved to be able to derive consistent estimates of the trade-offs between benefits and costs. However, MCDA with decision rules may constitute a promising compromise once a cost-effectiveness threshold has been adopted. Yet, MCDA with decision rules would call for a transparent and consistent framework to avoid controversy and to make economic evaluation comparable across therapies. Such an investigation exploring its feasibility may be initiated by academic researchers and French healthcare authorities.
We are grateful to Dr. Kevin Marsh, from Evidera, for his comments on previous drafts of the manuscript. We would like to thank Prof. Peter Zweifel, from the University of Zurich, for his review of the last version of the manuscript and the editor for providing suggestions on how to improve the manuscript.
Compliance with Ethical Standards
Conflict of interest
The authors have no conflict of interest to declare. No funding was received for performing the study. Salah Ghabri is employed by HAS. Jean-Michel Josselin and Benoît le Maux are employed by University of Rennes 1.
- 4.Roy B. Classement et choix en présence de points de vue multiples (la méthode ELECTRE)”. La Revue d’Informatique et de Recherche Opérationelle. 1968;8:57–75.Google Scholar
- 5.Saaty T. The analytic hierarchy process. New York: McGraw Hill; 1980. p. 1980.Google Scholar
- 10.Hugonnard JC, Roy B. Le plan d’extension du métro en banlieue parisienne, un cas type d’application de l’analyse multicritère. Les Cahiers Scientifiques de la Revue Transports. 1982;6:77–108.Google Scholar
- 12.Baltussen R. The use of MCDA for HTA agencies—options, limitations and the way forward. https://www.ispor.org/conferences-education/education-training/virtual/webinars/. Accessed 25 Sept 2019.
- 13.Legifrance. Loi n° 2016-41 du 26 janvier de modernisation de notre système de santé. https://www.legifrance.gouv.fr/eli/loi/2016/1/26/AFSX1418355L/jo. Accessed 25 Sept 2019.
- 19.Castro H, Tringali M, Cleemput I, et al. Advancing MCDA and HTA into coverage decision-making. In: Marsh K, Goetghebeur M, Thokala P, et al., editors. Multi criteria decision analysis to support healthcare decisions. Heidelberg: Springer; 2017.Google Scholar
- 21.Claxton K. Three questions to ask when examining MCDA. Value and Outcomes Spotlight. 2015. https://www.ispor.org/docs/default-source/publications/value-outcomes-spotlight/january-february-2015/vos-three-questions-to-ask-when-examining-mcda.pdf?sfvrsn=3b3b650c_2. Accessed 25 Sept 2019.