Abstract
Background
The multiplicity of issues, including uncertainty and ethical dilemmas, and policies involved in appraising interventions for rare diseases suggests that multicriteria decision analysis (MCDA) based on a holistic definition of value is uniquely suited for this purpose. The objective of this study was to analyze and further develop a comprehensive MCDA framework (EVIDEM) to address rare disease issues and policies, while maintaining its applicability across disease areas.
Methods
Specific issues and policies for rare diseases were identified through literature review. Ethical and methodological foundations of the EVIDEM framework v3.0 were systematically analyzed from the perspective of these issues, and policies and modifications of the framework were performed accordingly to ensure their integration.
Results
Analysis showed that the framework integrates ethical dilemmas and issues inherent to appraising interventions for rare diseases but required further integration of specific aspects. Modification thus included the addition of subcriteria to further differentiate disease severity, disease-specific treatment outcomes, and economic consequences of interventions for rare diseases. Scoring scales were further developed to include negative scales for all comparative criteria. A methodology was established to incorporate context-specific population priorities and policies, such as those for rare diseases, into the quantitative part of the framework. This design allows making more explicit trade-offs between competing ethical positions of fairness (prioritization of those who are worst off), the goal of benefiting as many people as possible, the imperative to help, and wise use of knowledge and resources. It also allows addressing variability in institutional policies regarding prioritization of specific disease areas, in addition to existing uncertainty analysis available from EVIDEM.
Conclusion
The adapted framework measures value in its widest sense, while being responsive to rare disease issues and policies. It provides an operationalizable platform to integrate values, competing ethical dilemmas, and uncertainty in appraising healthcare interventions.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
The adapted EVIDEM framework provides an operationalizable platform to integrate individual and social values, competing ethical dilemmas, and uncertainty, which are particularly challenging in appraising interventions for rare diseases. |
The addition of subcriteria to further differentiate disease severity, disease-specific treatment outcomes and economic consequences of interventions, inclusion of negative scoring scales for all comparative criteria, and integration of a methodology to further incorporate context-specific population priorities and policies makes the EVIDEM framework particularly responsive to rare diseases, while maintaining its applicability across disease areas. |
Additionally, a method was developed to address variability in country or institution policies with respect to prioritization of specific disease areas. |
1 Introduction
Appraising the value of interventions for rare diseases raises a number of difficult issues. Many of these conditions are severe, chronic, progressive, and life-threatening, with multiple medical, psychological, and social consequences [1, 2]. Their rareness is a major barrier to developing targeted therapeutic interventions due to both economic and scientific constraints. As a consequence, unmet needs remain significant and widespread [3]. Uncertainty is also a critical issue in rare diseases, and includes limitations in our understanding of natural history [4] as well as challenges in obtaining high-quality epidemiological [5–8], clinical [9, 10], and economic data. In addition to uncertainty on evidence, there is variability in country or institution policies and across stakeholders on the values at stake. Regulatory policies to support the development of interventions for rare diseases are in place [11, 12] but, once approved, appraising these often highly-priced products for reimbursement is a challenge for health technology assessment (HTA) [13–15]. Some agencies acknowledge that their standard appraisal approaches need to be modified for orphan products [13], and a few have developed explicit approaches, including the National Institute for Health and Care Excellence (NICE) Interim Process for Highly Specialized Technologies (HST) [16] and the Ontario Public Drugs Program framework [17].
The multiplicity of issues involved in appraising interventions for rare diseases, as highlighted in the NICE interim HST process [16], suggests that multicriteria decision analysis (MCDA)-based approaches that apply a holistic definition of value are uniquely suited for this purpose. MCDA makes explicit which criteria are used and how they are balanced in decision making. Several multicriteria approaches designed for appraising interventions for rare diseases have been proposed, suggesting different sets of decision criteria that were selected based on various approaches [18–20]. To be justifiable, decisions regarding healthcare interventions, and thus the criteria included in a multicriteria framework, must be based on reasons that reflect society’s ethical principles and substantive values, concepts such as fairness and efficiency [21, 22], while the decision-making process must reflect procedural values such as transparency, accountability, and participation [21] to help legitimize decisions by upholding procedural justice [21, 23, 24]. Thus, there is a need for a comprehensive appraisal framework that features decision criteria that are selected based on ethical principles (i.e. social substantive values) and reflects social procedural values.
EVIDEM is an open-source, collaboratively-developed [25–30] MCDA framework designed to appraise the holistic value of healthcare interventions [31]. It has been tested [26, 27, 32, 33] and implemented [34, 35] in various real-world decision-making settings. The EVIDEM framework v3.0 is structured around the key objectives (domains) that govern healthcare decision making and define the holistic value of interventions, i.e. to optimize the health of patients, populations, and healthcare systems. Attainment of these objectives is assessed using operationalizable criteria, each of which is rooted in established ethical positions [36, 37], including deontology (imperative to help, beneficence, nonmaleficence) [38–41], utilitarianism (greatest good for the greatest number) [42, 43], distributive justice and fairness (prioritize those who are worst off) [43, 44], and virtue ethics and practical wisdom [45]. The framework was designed to meet MCDA design principles, i.e. each criterion must make a unique contribution to the value of an intervention, while the criteria set needs to be complete, operationalizable, and free of redundancies and mutual dependencies [46]. Objectives are classified into normative and feasibility domains. In the quantitative part of the framework, criteria are weighted and scored to obtain a numerical measure of the value of an intervention (Value Estimate). The qualitative part captures the impact on value of those criteria that are difficult to quantify. Key procedural values underpinning EVIDEM are transparency, pragmatism, and participatory decision making [25–27].
Because of its holistic definition of value, explicit rootedness in ethical principles and real-world application, the EVIDEM framework was selected for this study to be analyzed and further developed into a comprehensive MCDA approach that addresses rare disease issues and policies, while maintaining its applicability across disease areas.
2 Methodology
2.1 Identification of Specific Issues Raised by Rare Diseases
A systematic literature review was carried out to identify issues in rare diseases that potentially impact on value appraisal as well as relevant current or proposed regulatory and reimbursement policies. Full-text publications related to disease impact (including size of population), context of interventions, clinical outcomes, costs, ethical issues, and licensing and reimbursement policies were included. Case reports and articles of clinical focus describing the natural history, clinical presentation, or treatment of specific rare diseases were excluded. The principal sources of information were peer-reviewed publications identified from the PubMed/MEDLINE databases (published from January 2003 to September 2013). Using keywords such as ‘orphan disease’ and ‘rare disorder, the PubMed/MEDLINE searches yielded 6525 citations, 115 of which were reviewed in full text, and 41 [2–10, 17–20, 47–74] included in the analysis, including five publications describing current or proposed appraisal frameworks for rare diseases [17–20, 68]. Additionally, websites of major regulatory and HTA agencies, rare disease organizations and networks (e.g. ORPHANET, EUROPLAN, Office for Rare Disease Research), and bibliographies of key publications were searched for relevant information.
The selected publications were analyzed to identify issues pertaining to value appraisal. The relevant issues were classified by the decision criterion to which they pertain, using the criteria definitions and the structure of the EVIDEM framework as an analytical tool.
2.2 Analysis and Adaptation of the EVIDEM Framework from the Perspective of Issues Raised by Rare Diseases
Ethical and methodological foundations of the EVIDEM framework v3.0 [36, 37] were analyzed on a criterion-by-criterion basis to assess their relevance and implications from the perspective of issues raised by rare diseases. This analysis was used to determine whether a modification of the framework was required to enhance its usefulness for appraising interventions for rare diseases, while preserving its applicability across disease areas. Potential modifications included:
-
definition of subcriteria to allow a more differentiated assessment of specific criteria;
-
further development of scoring scales;
-
structural transformation of the framework by moving criteria that are assessed qualitatively into the quantitative part of the framework to integrate context-specific policies and priorities.
3 Results
The adapted EVIDEM framework is shown in Fig. 1 (see Online Resource 1 for the full framework). For the quantitative part of the framework (a), hierarchical point allocation was selected as the primary weight elicitation technique, a method that allows direct expression of personal values and makes explicit the trade-offs that need to be made across the criteria [75]. As in the original framework, weighting is performed independently of the intervention appraised (generic weighting), with the exception of the subcriteria defining intervention outcomes, for which disease-specific weights are elicited to define the relative weight of each outcome as part of the Comparative Effectiveness criterion. Evaluators score the intervention with respect to each criterion using constructed, cardinal scoring scales, designed to measure their judgments on the evidence presented. These scales include scores of zero, corresponding to situations where the intervention has no value with respect to a criterion. The framework was adapted to include negative scores for all comparative criteria to reflect worse outcomes or economic consequences than comparators, which are relevant to rare diseases, but also in general for fair appraisal of interventions. The impacts of contextual criteria on the value of the intervention are considered qualitatively using a separate tool (b).
Criteria-specific analyses and adaptations are summarized in Table 1 and are reported below.
3.1 Normative Universal Objectives and Quantitative Criteria
These criteria are assessed quantitatively because they are universally quantifiable in the sense that their achievement can be assessed across healthcare systems on scales for which the high and low ends are a priori agreed upon [37].
The objective of addressing areas of high therapeutic need covers three criteria:
-
Disease Severity: Grounded in the ethical imperative to alleviate suffering in those who are worst off (theory of justice), interventions for more severe diseases have greater value than those for less severe diseases [36]. Rare diseases can have multiple impacts on patients, causing morbidity, disability, reduced quality of life (QoL), and shorter life expectancy [1, 2, 69]. Approximately half of these conditions begin in childhood and many cause disabilities [1], thus posing a high burden on caregivers, usually family members [69], which may have a detrimental impact on their QoL [1–3]. To allow a more differentiated assessment of the multiple domains of disease severity potentially impacted by various conditions, four subcriteria were introduced, including Effect of Disease on Caregivers’ QoL, which is consistent with other assessment frameworks, such as the NICE Interim Process for HST [16].
-
Size of Population: Based on utility theory, this criterion aims at alleviating suffering in as many individuals as possible [36, 37]. Accordingly, a large population of patients affected represents a high therapeutic need, which is in agreement with surveys of the general public revealing that rareness in itself, apart from other considerations such as disease severity, is not considered a healthcare priority [51, 52, 63, 66]. Capturing an important aspect of social value, this criterion needs to be retained in an appraisal framework that is to be applied across different disease areas. Review of epidemiological data on rare diseases [76] suggested an adaptation of the scoring scale to better reflect the wide range of possible disease incidence/prevalence rates, extending from very common (prevalence >5 in 100) to very rare diseases (<5 in 100,000).
-
Unmet Needs, whereby a condition for which current interventions have many and serious limitations regarding their effectiveness, safety, tolerability, and QoL impact represents high need [37]. This criterion is highly relevant for rare diseases, where major therapeutic limitations persist and few interventions targeted for a specific condition are available [3].
The objective of providing large improvements in health outcomes is rooted in deontology (i.e. duty- or rule-based ethics) and outlined in the Hippocratic Oath as well in the principles of beneficence, nonmaleficence, and respect of autonomy (principlism) [21, 36, 41]. This is reflected in three criteria:
-
Comparative effectiveness;
-
Comparative safety/tolerability;
-
Comparative patient-reported outcomes (PROs)
One of the issues in rare diseases pertaining to these criteria is uncertainty or lack of consensus on the outcomes that need to be assessed in clinical studies. This issue can be partially addressed by dividing these criteria into subcriteria to represent outcomes specific to the disease, which will allow appraisers to discuss and differentially weight the importance (relevance) of each outcome.
The objective of delivering important types of health benefit is rooted in the ethical imperative to prevent and eliminate suffering (deontology), and covers two criteria [36]:
-
Type of preventive benefit, whereby preventing a disease entirely (eradication) is the most important type of preventive benefit (risk reduction).
-
Type of therapeutic benefit, whereby alleviating suffering entirely (cure) is the most important type of therapeutic benefit.
Although the vast majority of rare diseases are of genetic origin [2], and interventions are therefore generally therapeutic rather than preventive, this design acknowledges the value of both prophylactic and therapeutic interventions to ensure compatibility of the framework across therapeutic areas.
Both maximization of health benefits and minimization of costs are legitimate objectives of healthcare decision making, rooted in the ethical imperative of making wise use of scarce resources (practical wisdom) to maximize health under resource constraints (utilitarianism) [36, 37]. Thus, an intervention’s economic consequences represent value attributes and are therefore incorporated into a framework intended to measure value from a holistic perspective, as recommended by the UK Government manual conducting multicriteria decision making [46]. The objective of producing favorable economic consequences is addressed with three criteria; for each of these, cost reduction (or release of resources) is seen as a favorable economic consequence:
-
Comparative cost consequences—cost of intervention: Net cost of an intervention to the healthcare system in terms of its acquisition, implementation, maintenance, and replacement of existing treatments or interventions.
-
Comparative cost consequences—other medical costs: All medical cost consequences of an intervention, apart from the cost of the intervention itself (e.g. hospitalization, specialized care, primary care, long-term care, adverse event costs).
-
Comparative cost consequences—nonmedical costs: All cost consequences outside the scope of medical care.
Interventions can affect patients’ medical or nonmedical expenses through co-payments, travel, and paid caregivers, as well as impact patients’ and caregivers’ ability to work, all of which are highly relevant to rare diseases [2, 3, 70], as well as to many other conditions. Therefore, the adapted framework includes subcriteria, defined by payer (i.e. healthcare system, patients), allowing differentiation of diverse economic consequences and adjustments for appraisals from different perspectives.
Cost effectiveness combines several criteria already covered and is therefore not included in the framework, in line with basic design principles of MCDA to avoid double-counting and with other rare disease frameworks [16, 17, 19].
The objective of reducing uncertainty by grounding decisions in solid knowledge is covered under two criteria, both related to practical wisdom [36]:
-
Quality of Evidence: This criterion ensures that higher value is placed on interventions that have a statistically solid and unbiased scientific evidence base [36, 37, 77]. Responding to the imperative of evidence-based decision making, this criterion explicitly rewards high-quality research and development. The rareness of a condition affects data quality in multiple areas, including the burden and natural history of the disease, and the health and economic consequences of a proposed therapy [4–8]. For example, designing well-powered, double-blind, randomized controlled trials (RCTs) is challenging for rare diseases because patients are few [9] and effective comparators are often absent [10]. Therefore, real-life studies and modeling can play an important role for assessing rare disease therapies [17]. Thus, the adapted framework includes subcriteria representing different fields of research, allowing a differential assessment of data quality.
-
Expert Consensus/Clinical Practice Guidelines (CPGs): This criterion captures the strength of recommendations for optimal clinical practice based on expert interpretation of current scientific knowledge as well as bedside expertise [37]. A strong recommendation in a CPG developed following AGREE principles [78] raises confidence in the value of an intervention. For rare diseases, the availability and quality of CPGs is often limited [55], thus it is particularly important to capture expert knowledge, bearing in mind the scarcity of clinical experts and clinical data [1, 64].
3.2 Transformation of a Generic Qualitative Criterion into a Quantitative Context-Specific Criterion: Population Priority and Access
The objective of Addressing Priorities to Increase Fairness/Justice reflects a society’s sense of justice, which may entail giving priority to certain populations, as defined in specific societies and healthcare systems [36, 37]. Although there are various models of distributive justice (e.g. libertarian, communitarian, egalitarian, and utilitarian) [79], solidarity with those most in need is a key concept [56], as expressed in surveys of the general European population [51, 63]. Patients with rare diseases may be seen as a particularly disadvantaged population as they are often affected by social isolation [1, 57], delayed diagnosis [1, 3, 72], and inequalities in access to adequate treatment and care [1]. The disadvantaged status of these patients is acknowledged in regulatory policies that support research in orphan diseases [11, 12]. In addition, a number of public initiatives were launched to improve delivery of healthcare and social services to patients with rare diseases [80–83].
Although most decision makers consider population priorities [28], specific priorities may or may not be explicitly established in the context of a given healthcare system. Thus, one can distinguish two types of contexts:
-
If no population priorities are explicitly established, this objective cannot be quantitatively operationalized and must remain in the qualitative part for the framework for qualitative, general (case-by-case) consideration.
-
If specific population priorities are clearly established, the framework is adapted to integrate the objective of addressing priorities into the quantitative part of the framework, bearing in mind that multiple priorities may exist in a given context. To operationalize this integration, each population priority is defined as a criterion:
-
priority 1 (e.g. rare diseases)
-
priority 2 (e.g. HIV)
-
priority 3 (e.g. diabetes)
-
etc.
This design allows relative weighting of each priority versus other, potentially competing priorities that may have been established. Scoring is performed based on how well the intervention to be appraised is aligned with the respective priority. The output of this design can thus quantitatively capture the contribution of priorities (in contexts where such have been established) to the value of an intervention. In addition, this design allows addressing variability in country or institution policies with respect to prioritization of specific disease areas.
3.3 Normative Contextual Objectives and Qualitative Criteria
These criteria cannot be operationalized on a universal basis and are therefore appraised qualitatively in the generic framework [37]. As illustrated in the section above, when adapting the framework to a given context, objectives can be more closely defined and made operationalizable into quantitative criteria, thereby allowing their integration into the quantitative appraisal.
The objective of Aligning with the Mandate and Scope of the Healthcare System rests upon the principle of beneficence and utility, and its inclusion in the framework ensures that this is explicitly considered in each appraisal [36, 37]. The principal mandate of healthcare is to restore and maintain normal functioning [68], and interventions for rare diseases with serious health effects are generally aligned with this mandate.
This objective of Aligning with the Common Goal (i.e. the health of patients, populations, and healthcare systems) rather than with special interests corresponds to the reasonableness condition of the Accountability for Reasonableness framework [23, 24], which demands that decisions be based on relevant and mutually acceptable reasons [36]. Awareness of stakeholder pressures and barriers helps ensure that decisions are fair-minded and not unduly influenced by special interests [37]. For rare diseases, several organizations exist that inform, support, and advocate for patients [47, 48, 54, 62, 65]. While these are important to bring real-life patient perspectives to decision making, a balanced approach needs to be exercised for fair allocation of resources across populations in need.
Although reducing Environmental Impact is a universal normative objective, it currently does not play a significant role in healthcare decision making (although many decision makers think that it should [28]), and is thus appraised qualitatively, but could be incorporated into the quantitative (universal) part of the framework in a given context [37]. Due to their rarity, the environmental impact of rare disease treatments is expected to be low.
3.4 Contextual Feasibility Objectives and Qualitative Criteria
Some interventions may be desirable from a normative point of view, but not feasible in a given context.
Consideration of Affordability and Opportunity Costs, i.e. resources or existing interventions that may be forgone if a new intervention is adopted, is aligned with the principle of efficiency and utilitarianism (maximize health resources) [43], which comes into play at both the patient and societal levels [36, 37]. While the normative aspect of economic considerations (i.e. favorable economic consequences) is included in the quantitative (universal) part of the framework, affordability and opportunity cost considerations require a financial/budgeting exercise to determine feasibility in a given context [37]. For rare diseases, this reflects on whether resources foregone in other disease areas (e.g. more common diseases) are significant in the context of system-wide healthcare economics [56].
The ability of a healthcare system to ensure appropriate use of a new intervention and realize its potential benefit within the system’s capacity, depends on infrastructure, organization, skills, legislation, barriers, and risk of inappropriate use [37]. Relevant for rare diseases is to ensure that specific skills, infrastructure, and surveillance requirements (e.g. patient registries, monitoring) and the ability to reach remote populations are systematically considered [3, 58, 60].
The political, historical and cultural context is important to consider in appraisals. This includes the legal framework, such as, for example, the Equality Act, under which the NICE is required to avoid discrimination based on protected characteristics, and the Human Rights Act, under which it must consider implications for human rights [84]. This objective also covers the impact of the intervention on innovation and research, an important aspect for rare diseases, as research in this area was shown to advance understanding of pathophysiology, broadly contributing to the development of knowledge [56]. Precedence is another important consideration in coverage decision making in the rare disease field, as in other therapeutic areas.
3.5 Hypothetical Example for Application of the Adapted Framework
Table 2 shows an example of the application of the adapted framework in two appraisal contexts: if population priorities have not been established (context A), they are not part of the quantitative model and are considered qualitatively. In contrast, in a context in which population priorities have been explicitly defined (context B), these are incorporated into the quantitative model. In this case, the appraising committee must decide how much relative weight they will trade off from the other domains to ‘population priorities’ and how they will weigh the priority ‘rare diseases’ against other priorities established in their context (Table 2a).
Now let us consider assessment of a hypothetical intervention X for a rare condition Y by the same committee (Table 2b). Committee members assign a performance score for each criterion that expresses their judgment of the available evidence, as shown in the example. Scores are standardized and multiplied by the normalized weights (in Table 2a) to calculate the contribution of each criterion to the value of intervention X (Table 2b). The overall value estimate can then be used to rank interventions across disease areas. In this hypothetical example, inclusion of ‘population priorities’ in the quantitative part of the model increases the value estimate of X due to its full alignment with the rare disease priority and the relatively high weight the committee has assigned to this criterion. The committee also qualitatively considers what impact contextual criteria may have on the value of intervention X, which may impact its ranking of intervention X. If priorities have not been included in the quantitative value estimate, their potential impact can be captured qualitatively and may affect ranking.
4 Discussion
In this study, we analyzed the EVIDEM framework from the perspective of issues raised by rare diseases and developed it further in response to these issues. The adapted EVIDEM framework provides an operationalizable platform to integrate individual and social values, competing ethical dilemmas, and uncertainty, which are particularly challenging in appraising interventions for rare diseases. The addition of subcriteria to further differentiate disease severity, disease-specific treatment outcomes and economic consequences of interventions, inclusion of negative scoring scales for all comparative criteria, and integration of a methodology to further incorporate context-specific population priorities and policies makes the EVIDEM framework particularly responsive to rare diseases, while maintaining its applicability across disease areas.
Development of the EVIDEM framework involves continuous research and development on handling of uncertainty, which is particularly pertinent to the context of rare diseases where ethical dilemmas and variations in judgments and perspectives are challenging for the healthcare community [25–27]. Uncertainty in judgments on evidence can be explored using score ranges [85]; uncertainty in weights can be gauged through application of different weighting techniques (e.g. pair-wise comparison) [30, 46, 75]; uncertainty due to variability in individual perspectives (weights) and judgments (scores) can be assessed through standard measures of statistical dispersion [25–27, 32, 33]; uncertainty due to model structure can be explored by modifying the structure, e.g. eliminating the lowest weighted criteria [86]; and reproducibility can be examined by repeating the appraisal exercise (test–retest) [33, 87]. In this study, we additionally developed a method to address variability in country or institution policies with respect to prioritization of specific disease areas. This method allows adapting the framework according to whether or not explicit prioritization policies exist and to examine the relative importance of specific priorities.
In a survey of the Norwegian population, the notion that “Patients with rare diseases should have the same right to treatment as others even if more expensive” found strong support [51]. Similarly, in a UK survey, concern for fairness exceeded concern for population health maximization (i.e. utility theory), the guiding principle of the cost-effectiveness-based approach to prioritization [63]. Cost effectiveness is not considered in current [16, 17] or proposed rare disease frameworks [18–20, 68], and several HTA agencies waive their requirement for cost-effectiveness analysis for orphan drugs [13, 88]. Some propose modifying the current cost-effectiveness paradigm by, for example, assigning quality-adjusted life-year (QALY) weights depending on disease prevalence [9] or different willingness-to-pay thresholds for diseases with high ‘social value’ [89]. However, people do not prioritize rareness but severe, life-threatening diseases, with high unmet needs and interventions that reduce reliance on informal caregivers [63, 66]. These attributes, while characteristic of many rare diseases [1], also apply to other disease areas. Implicit in these proposals is the realization that there is a need for a holistic definition of value, beyond the cost per QALY, to guide appraisal of healthcare interventions.
In comparison to other proposed MCDA frameworks for rare diseases, where selection of decision criteria was based on a literature review, consultation with stakeholders, or rationales [18–20], selection of decision criteria for the EVIDEM MCDA framework is additionally explicitly rooted in a comprehensive set of ethical principles as well as MCDA design principles, which promotes thorough exploration of the values underpinning appraisal. In addition, due to alignment with the ultimate common goal of all healthcare stakeholders, i.e. identifying interventions that are beneficial to the health of patients as well as to populations and healthcare systems (definition of most valuable interventions), the design proposed here can be applied system-wide across interventions and diseases, in distinction to other MCDA rare disease frameworks. System-wide operationalization is further supported through fully developed implementation methods, particularly the generic design of the scoring scales, which measure evaluators’ judgment of the available evidence across types of interventions and outcomes.
This approach is also in contrast to MCDA models that are designed adhoc to address a specific decision problem, with a discrete set of known options (i.e. interventions). Such ad hoc models can be designed around these options and their attributes, in terms of criteria selection and scoring scales [46, 90–92]. Ad hoc models, while useful for a circumscribed decision problem, are not adapted to the resource allocation context in which decision makers need to decide, in a consistent manner, whether any proposed intervention provides sufficient value to be funded and adopted into practice. In such contexts, decision makers need a tool to measure value grounded in fundamental principles that reflect the goals of the healthcare system [93]. For example, such an MCDA tool, including decision criteria such as incremental benefits, incremental total cost, and quality of evidence, was proposed for Israel’s Public National Advisory Committee [77]. In addition, EVIDEM has been adapted and implemented in several jurisdictions as a system-wide approach that encompasses all the aspects of decisions for resource allocation [34, 35].
Decision making at its core involves the balancing of ethical dilemmas. Aiming to maximize the overall utility of society (utilitarianism), few resources should be allocated to therapies benefiting few individuals [56]. From a rights-based perspective, access to appropriate healthcare is a right, which is constitutionally established in some jurisdictions, although its scope is open to interpretation [56]. Obviously, a rights-based approach needs to be counterbalanced with considerations of healthcare efficiency and sustainability. The multicriteria framework proposed here makes this balancing act explicit by defining decision criteria, each providing a distinct contribution towards the goal of healthcare. Prioritization of rare diseases is made explicit and weighted against other priorities and the goal of benefiting as many people as possible, while taking into account disease severity, unmet medical needs and economic consequences. This MCDA design incorporates the moral principles of beneficence, nonmaleficence, respect for autonomy, and distributive justice, set forth by Beauchamp and Childress [39, 40] and explicitly subscribed to by the NICE [94], and ensures that these principles are actively considered in decision making. Such approaches facilitate communication to achieve workable resolution of ethical dilemmas across stakeholders, which is necessary to promote the most valuable healthcare interventions to optimize health of patients, populations, and healthcare systems.
5 Conclusion
The adapted framework measures value in its widest sense and is responsive to rare diseases issues and policies. It provides an operationalizable platform to explore values, competing ethical dilemmas, and uncertainty in appraising healthcare interventions.
References
EURORDIS. Rare diseases: understanding this public health priority. EURORDIS. 2005. Available at: http://www.eurordis.org/publication/rare-diseases-understanding-public-health-priority. Accessed 21 Oct 2013.
Barrera LA, Galindo GC. Ethical aspects on rare diseases. Adv Exp Med Biol. 2010;686:493–511.
Kole A, Faurisson F. Rare diseases social epidemiology: analysis of inequalities. Adv Exp Med Biol. 2010;686:223–50.
Beutler E. Lysosomal storage diseases: natural history and ethical and economic aspects. Mol Genet Metab. 2006;88(3):208–15.
Gomez-Rubio V, Lopez-Quilez A. Statistical methods for the geographical analysis of rare diseases. Adv Exp Med Biol. 2010;686:151–71.
de la Paz MP, Villaverde-Hueso A, Alonso V, Janos S, Zurriaga O, Pollan M, et al. Rare diseases epidemiology research. Adv Exp Med Biol. 2010;686:17–39.
Groft SC, de la Paz MP. Rare diseases—avoiding misperceptions and establishing realities: the need for reliable epidemiological data. Adv Exp Med Biol. 2010;686:3–14.
Fuller M, Meikle PJ, Hopwood JJ. Epidemiology of lysosomal storage diseases: an overview. In: Mehta A, Beck M, Sunder-Plassmann G, editors. Fabry disease: perspectives from 5 years of FOS. Oxford: Oxford PharmaGenesis; 2006.
Hughes DA, Tunnage B, Yeo ST. Drugs for exceptionally rare diseases: do they deserve special status for funding? QJM. 2005;98(11):829–36.
Kanters TA, de Sonneville-Koedoot C, Redekop WK, Hakkaart L. Systematic review of available evidence on 11 high-priced inpatient orphan drugs. Orphanet J Rare Dis. 2013;8(1):124.
Liu BC, He L, He G, He Y. A cross-national comparative study of orphan drug policies in the United States, the European Union, and Japan: towards a made-in-China orphan drug policy. J Public Health Policy. 2010;31(4):407–20.
Loorand-Stiver L. Drugs for rare diseases: evolving trends in regulatory and health technology assessment perspectives. Environ Scan. 2013;2013(42):1–40.
Roll K, Stargardt T, Schreyogg J. Authorization and reimbursement of orphan drugs in an international comparison [in German]. Gesundheitswesen. 2011;73(8–9):504–14.
Tambuyzer E. Rare diseases, orphan drugs and their regulation: questions and misconceptions. Nat Rev Drug Discov. 2010;9(12):921–9.
Herder M. When everyone is an orphan: against adopting a U.S.-styled orphan drug policy in Canada. Acc Res. 2013;20(4):227–69.
National Institute for Health and Care Excellence. Interim process and methods of the Highly Specialised Technologies programme. 2013. Available at: http://www.nice.org.uk/aboutnice/howwework/devhst/DevelopingNICEHighlySpecialisedTechnologies.jsp. Accessed 30 Oct 2013.
Winquist E, Bell CM, Clarke JT, Evans G, Martin J, Sabharwal M, et al. An evaluation framework for funding drugs for rare diseases. Value Health. 2012;15(6):982–6.
Hughes-Wilson W, Palma A, Schuurman A, Simoens S. Paying for the Orphan Drug System: break or bend? Is it time for a new evaluation system for payers in Europe to take account of new rare disease treatments? Orphanet J Rare Dis. 2012;7:74.
Paulden M, Stafinski T, Menon D, McCabe C. Value-based reimbursement decisions for orphan drugs: a scoping review and decision framework. Pharmacoeconomics. 2014;33(3):255–69.
Sussex J, Rollet P, Garau M, Schmitt C, Kent A, Hutchings A. A pilot study of multicriteria decision analysis for valuing orphan medicines. Value Health. 2013;16(8):1163–9.
Clark S, Weale A. Social values in health priority setting: a conceptual framework. J Health Organ Manag. 2012;26(3):293–316.
Biron L, Rumbold B, Faden R. Social value judgments in healthcare: a philosophical critique. J Health Organ Manag. 2012;26(3):317–30.
Daniels N, Sabin J. Limits to health care: fair procedures, democratic deliberation, and the legitimacy problem for insurers. Philos Public Aff. 1997;26(4):303–50.
Daniels N. Decisions about access to health care and accountability for reasonableness. J Urban Health. 1999;76(2):176–91.
Goetghebeur M, Wagner M, Khoury H, Levitt RJ, Erickson LJ, Rindress D. Evidence and value: impact on DEcisionMaking: the EVIDEM framework and potential applications. BMC Health Serv Res. 2008;8(1):270.
Goetghebeur M, Wagner M, Khoury H, Rindress D, Gregoire JP, Deal C. Combining multicriteria decision analysis, ethics and health technology assessment: applying the EVIDEM decision making framework to growth hormone for Turner syndrome patients. Cost Eff Resour Alloc. 2010;8(1):4.
Goetghebeur MM, Wagner M, Khoury H, Levitt RJ, Erickson LJ, Rindress D. Bridging health technology assessment (HTA) and efficient health care decision making with multicriteria decision analysis (MCDA): applying the EVIDEM framework to medicines appraisal. Med Decis Mak. 2012;32(2):376–88.
Tanios N, Wagner M, Tony M, Baltussen R, van Til J, Rindress D, et al. Which criteria are considered in healthcare decisions? Insights from an international survey of policy and clinical decision makers. Int J Technol Assess Health Care. 2013;29(4):456–65.
Guindo LA, Wagner M, Baltussen R, Rindress D, van Til J, Kind P, et al. From efficacy to equity: review of decision criteria used in resource allocation and healthcare decisionmaking. Cost Eff Resour Alloc. 2012;10(1):9.
van Til J, Groothuis-Oudshoorn C, Lieferink M, Dolan J, Goetghebeur M. Does technique matter: a pilot study exploring weighting techniques for a multi-criteria decision support framework. Cost Eff Resour Alloc. 2014;12:22.
EVIDEM Collaboration website . EVIDEM Collaboration. 2015. Available at: http://www.evidem.org. Accessed 12 June 2015.
Miot J, Wagner M, Khoury H, Anderson AN, Rindress D, Goetghebeur MM. Field testing of a framework for coverage of a diagnostic test for cervical cancer in South Africa. Cost Eff Resour Alloc. 2011;10(1):2.
Tony M, Wagner M, Khoury H, Rindress D, Papastavros T, Oh P, et al. Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada. BMC Health Serv Res. 2011;11:329.
Radaelli G, Lettieri E, Masella C, Merlino L, Strada A, Tringali M. Implementation of EUnetHTA Core Model(R) in Lombardia: the VTS framework. Int J Technol Assess Health Care. 2014;30(1):105–12.
MCDA for HTA in action: leveraging experience from Europe and Latin America. Panel presented at the Health Technology Assessment International (HTAi) 12th Annual Conference, 15–17 June 2015: Oslo; 2015.
Goetghebeur M, Wagner M. Defining value: the ethical foundations of MCDA in healthcare decision making. Manuscript in preparation. In: Marsh K, Thokala P, Goetghebeur M, Baltussen R, editors. Healthcare decisions supported by multi-criteria decision analysis. Cham: Springer International Publishing AG; 2015.
EVIDEM Collaboration. Decision criteria of the framework: conceptual background, definitions, design & instructions. EVIDEM Collaboration. 2015. Available at: https://www.evidem.org/components-decision.php. Accessed 29 July 2015.
Alexander L, Moore M. Deontological ethics. In: Zalta EN, editor. The Stanford Encyclopedia of Philosophy (Spring 2015 edition). Available at: http://plato.stanford.edu/archives/spr2015/entries/ethics-deontological/.
Beauchamp TL, Childress JF. Principles of biomedical ethics. Oxford; New York: Oxford University Press; 2001.
Beauchamp TL. The ‘four principles’ approach. Principles of health care ethics. Chichester: Wiley; 1994. p. 3–12.
Lawrence DJ. The four principles of biomedical ethics: a foundation for current bioethical debate. J Chiropr Humanit. 2007;14:34–40.
Driver J. The history of utilitarianism. In: Zalta EN, editor. The Stanford Encyclopedia of Philosophy (Winter 2014 edition). Available at: http://plato.stanford.edu/archives/win2014/entries/utilitarianism-history/.
World Health Organization. Guidance on ethics and equitable access to HIV treatment and care. World Health Organization. 2004. Available at: http://www.who.int/ethics/Guidance%20on%20Ethics%20and%20HIV.pdf. Accessed 6 Feb 2009.
Hoedemaekers R, Dekkers W. Justice and solidarity in priority setting in health care. Health Care Anal. 2003;11(4):325–43.
Hursthouse R. Virtue ethics. In: Zalta EN, editor. The Stanford Encyclopedia of Philosophy (Fall 2013 edition). Available at: http://plato.stanford.edu/archives/fall2013/entries/ethics-virtue.
Department for Communities and Local Government. Multi-criteria analysis: a manual. London: Department for Communities and Local Government; 2009.
Black AP, Baker M. The impact of parent advocacy groups, the Internet, and social networking on rare diseases: the IDEA League and IDEA League United Kingdom example. Epilepsia. 2011;52(Suppl 2):102–4.
Boon W, Broekgaarden R. The role of patient advocacy organisations in neuromuscular disease R&D: the case of the Dutch neuromuscular disease association VSN. Neuromuscul Disord. 2010;20(2):148–51.
Cornu C, Kassai B, Fisch R, Chiron C, Alberti C, Guerrini R, et al. Experimental designs for small randomised clinical trials: an algorithm for choice. Orphanet J Rare Dis. 2013;8:48.
Denis A, Mergaert L, Fostier C, Cleemput I, Simoens S. Budget impact analysis of orphan drugs in Belgium: estimates from 2008 to 2013. J Med Econ. 2010;13(2):295–301.
Desser AS, Gyrd-Hansen D, Olsen JA, Grepperud S, Kristiansen IS. Societal views on orphan drugs: cross sectional survey of Norwegians aged 40 to 67. BMJ. 2010;341:c4715.
Desser AS. Prioritizing treatment of rare diseases: a survey of preferences of Norwegian doctors. Soc Sci Med. 2013;94:56–62.
Dimichele DM, Blanchette V, Berntorp E. Clinical trial design in haemophilia. Haemophilia. 2012;18(Suppl. 4):18–23.
Dunkle M, Pines W, Saltonstall PL. Advocacy groups and their role in rare diseases research. Adv Exp Med Biol. 2010;686:515–25.
Eidt D, Frank M, Reimann A, Wagner TOF, Mittendorf T. Maánahmen zur Verbesserung der gesundheitlichen Situation von Menschen mit Seltenen Erkrankungen in Deutschland. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2009. Available at: http://www.bmg.bund.de/uploads/publications/BMG-G-09050-Bericht-Massnahmen-seltene-Krankheiten_200908.pdf. Accessed 28 Nov 2013.
Gericke CA, Riesberg A, Busse R. Ethical issues in funding orphan drug research and development. J Med Ethics. 2005;31(3):164–8.
Henderson SL, Packman W, Packman S. Psychosocial aspects of patients with Niemann-Pick disease, type B. Am J Med Genet A. 2009;149A(11):2430–6.
Hollak CE, Aerts JM, Ayme S, Manuel J. Limitations of drug registries to evaluate orphan medicinal products for the treatment of lysosomal storage disorders. Orphanet J Rare Dis. 2011;6:16.
Joppi R, Bertele’ V, Garattini S. Orphan drugs, orphan diseases. The first decade of orphan drug legislation in the EU. Eur J Clin Pharmacol. 2013;69(4):1009–24.
Kenny TD, Jessop EG, Gutteridge WH. Monitoring clinical quality in rare disease services: experience in England. Orphanet J Rare Dis. 2008;3:23.
Kesselheim AS, Myers JA, Avorn J. Characteristics of clinical trials to support approval of orphan vs nonorphan drugs for cancer. JAMA. 2011;305(22):2320–6.
Lavery C. Role of patient support groups in lysosomal storage diseases. In: Mehta A, Beck M, Sunder-Plassmann G, editors. Fabry disease: perspectives from 5 years of FOS. Oxford: Oxford PharmaGenesis; 2006.
Linley WG, Hughes DA. Societal views on NICE, cancer drugs fund and value-based pricing criteria for prioritising medicines: a cross-sectional survey of 4118 adults in Great Britain. Health Econ. 2013;22(8):948–64.
MacLeod S. Optimal therapy for rare disorders and genetic diseases: ethical and political challenges. Proc West Pharmacol Soc. 2007;50:21–3.
Mavris M, Le CY. Involvement of patient organisations in research and development of orphan drugs for rare diseases in europe. Mol Syndromol. 2012;3(5):237–43.
Mentzakis E, Stefanowska P, Hurley J. A discrete choice experiment investigating preferences for funding drugs used to treat orphan diseases: an exploratory study. Health Econ Policy Law. 2011;6(3):405–33.
Orofino J, Soto J, Casado MA, Oyaguez I. Global spending on orphan drugs in France, Germany, the UK, Italy and Spain during 2007. Appl Health Econ Health Policy. 2010;8(5):301–15.
Pinxten W, Denier Y, Dooms M, Cassiman JJ, Dierickx K. A fair share for the orphans: ethical guidelines for a fair distribution of resources within the bounds of the 10-year-old European Orphan Drug Regulation. J Med Ethics. 2012;38(3):148–53.
Rajmil L, Perestelo-Perez L, Herdman M. Quality of life and rare diseases. Adv Exp Med Biol. 2010;686:251–72.
Rosselli D, Rueda JD, Solano M. Ethical and economic considerations of rare diseases in ethnic minorities: the case of mucopolysaccharidosis VI in Colombia. J Med Ethics. 2012;38(11):699–700.
Schey C, Milanova T, Hutchings A. Estimating the budget impact of orphan medicines in Europe: 2. Orphanet J Rare Dis. 2011;6:62.
van der Kloot WA, Hamdy NA, Hafkemeijer LC, den Dulk FM, Chotkan SA, van Emmerik AA, et al. The psychological burden of an initially unexplained illness: patients with sternocostoclavicular hyperostosis before and after delayed diagnosis. Health Qual Life Outcomes. 2010;8:97.
Wyatt K, Henley W, Anderson L, Anderson R, Nikolaou V, Stein K, et al. The effectiveness and cost-effectiveness of enzyme and substrate replacement therapies: a longitudinal cohort study of people with lysosomal storage disorders. Health Technol Assess. 2012;16(39):1–543.
Drummond M, Towse A. Orphan drugs policies: a suitable case for treatment. Eur J Health Econ. 2014;15(4):335–40.
Dolan JG. Multi-criteria clinical decision support. A primer on the use of multiple-criteria decision-making methods to promote evidence-based, patient-centered healthcare. Patient. 2010;3(4):229–248.
Orphanet Report Series. Prevalence of rare diseases: bibliographic data. 2014. Available at: http://www.orpha.net/consor/cgi-bin/Education_Home.php?lng=EN#REPORT_RARE_DISEASES.
Golan OG, Hansen P. Which health technologies should be funded? A prioritization framework based explicitly on value for money. Isr J Health Policy Res. 2012;1(1):44.
Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010;182(18):E839–42.
Leget C, Hoedemaekers R. Teaching medical students about fair distribution of healthcare resources. J Med Ethics. 2007;33(12):737–41.
EUROPLAN Project. EUROPLAN 2012-2015. EUROPLAN Project. 2015. http://www.europlanproject.eu/Content?folder=1&content=1. Accessed 23 Oct 2015.
Taruscio D, Vittozzi L, Stefanov R. National plans and strategies on rare diseases in Europe. Adv Exp Med Biol. 2010;686:475–91.
EUCERD. EUCERD Joint Action: working for rare diseases. EUCERD. 2014. http://www.eucerd.eu/?page_id=54. Accessed 23 Oct 2015.
Molster C, Youngs L, Hammond E, Dawkins H. Key outcomes from stakeholder workshops at a symposium to inform the development of an Australian national plan for rare diseases. Orphanet J Rare Dis. 2012;7:50.
National Institute for Health and Care Excellence. NICE’s equality objectives and equality programme 2013–2016. 2013. Available at: http://www.nice.org.uk/about/nice-communities/public-involvement/equality-scheme#. Accessed 28 Jan 2015.
Goetghebeur M, Wagner M, Bec M, Nikodem M, Zyla A, Micaleff A et al. IMI work package 6: report. Benefit-risk balance re-evaluation case study Raptiva® (efalizumab). European Medicines Agency. 2015. v.2.6:1–99.http://www.imi-protect.eu/. Accessed 23 Oct 2015.
Cox R, Sanchez J, Revie CW. Multi-criteria decision analysis tools for prioritising emerging or re-emerging infectious diseases associated with climate change in Canada. PLoS One. 2013;8(8):e68338.
Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420–8.
Denis A, Mergaert L, Fostier C, Cleemput I, Hulstaert F, Simoens S. Critical assessment of Belgian reimbursement dossiers of orphan drugs. Pharmacoeconomics. 2011;29(10):883–93.
Drummond M, Evans B, LeLorier J, Karakiewicz P, Martin D, Tugwell P, et al. Evidence and values: requirements for public reimbursement of drugs for rare diseases: a case study in oncology. Can J Clin Pharmacol. 2009;16(2):e273–81.
Bettinger TL, Shuler G, Jones DR, Wilson JP. Schizophrenia: multi-attribute utility theory approach to selection of atypical antipsychotics. Ann Pharmacother. 2007;41(2):201–7.
Singh S, Dolan JG, Centor RM. Optimal management of adults with pharyngitis: a multi-criteria decision analysis. BMC Med Inform Decis Mak. 2006;6:14.
Aenishaenslin C, Hongoh V, Cisse HD, Hoen AG, Samoura K, Michel P, et al. Multi-criteria decision analysis as an innovative approach to managing zoonoses: results from a study on Lyme disease in Canada. BMC Public Health. 2013;13(1):897.
Tromp N, Baltussen R. Mapping of multiple criteria for priority setting of health interventions: an aid for decision makers. BMC Health Serv Res. 2012;12:454.
National Institute for Health and Care Excellence. Social value judgements. Principles for the development of NICE guidance. 2nd ed. 2013. Available at: http://www.nice.org.uk/proxy/?sourceurl=http://www.nice.org.uk/aboutnice/howwework/socialvaluejudgements/socialvaluejudgements.jsp. Accessed 28 Jan 2015.
Australian Government. Therapeutic goods regulations 1990: statutory rules No. 394, 1990 as amended. Australian Government. 2013. https://www.comlaw.gov.au/Details/F2013C00670. Accessed 23 Oct 2015.
Office of Legislative and Regulatory Modernization. Initial draft discussion document for a Canadian orphan drug regulatory framework. Orphanet. 2012. 1-25. http://www.orpha.net/national/data/CA-EN/www/uploads/Initial-Draft-Discussion-Document-for-A-Canadian-Orphan-Drug--Regulatory-Framework.doc. Accessed 30 Oct 2013.
Acknowledgments
The authors acknowledge the contribution of the EVIDEM collaboration for the open-source development of the mcda framework on which this study was based.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
This study was made possible by a research grant from Genzyme, Cambridge, MA, USA.
Monika Wagner, Hanane Khoury, Jacob Willet, Donna Rindress, and Mireille Goetghebeur are employees of LASER Analytica, which has received funding from Genzyme (a subsidiary of Sanofi) for the conduct of this study and preparation of the manuscript.
Monika Wagner and Mireille Goetghebeur designed the study and drafted the manuscript. All authors contributed to the collection, analysis and interpretation of data, revised the manuscript, and approved the final version.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Wagner, M., Khoury, H., Willet, J. et al. Can the EVIDEM Framework Tackle Issues Raised by Evaluating Treatments for Rare Diseases: Analysis of Issues and Policies, and Context-Specific Adaptation. PharmacoEconomics 34, 285–301 (2016). https://doi.org/10.1007/s40273-015-0340-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40273-015-0340-5