Introduction

In the European Union (EU), a disease is considered ‘orphan’ if it is a life-threatening or seriously debilitating disorder that affects fewer than 1 per 2 000 (or less than 0.05 %) of the population [1]. To date, 7 000 rare diseases have been identified, affecting 30–40 million patients in the EU [2]. Although no legal definition of ‘ultra-orphan’ diseases has been established, this subcategory was introduced by the National Institute for Health and Care Excellence (NICE). It is suggested to be applied to diseases with an estimated prevalence of <1:50 000 [3]. Initially, the small consumers’ market of orphan drugs restrained the pharmaceutical industry from developing medicines for rare diseases. For this reason, European legislation was introduced in 2000 to stimulate the development of orphan drugs, following the example of the United States who introduced the Orphan Drug Act in 1983. This legislation implies that the pharmaceutical industry has a right to i) obtain protocol assistance at a reduced charge, ii) gain access to the centralized authorization procedure, iii) get reduction of registration costs, and iv) benefit from 10 years of market exclusivity after registration [4]. This has led to the authorization of 124 new orphan drugs in the EU from 2000 until 2015 by the European Medicines Agency (EMA), of which about one-third is indicated for an ultra-orphan disease (http://www.ema.europa.eu/ema/). Authorization of these ultra-orphan drugs is often based on studies in small patient groups not representing the entire patient population. Moreover, biomarkers and intermediate endpoints are often used [5]. Because of this, longer term effectiveness on clinically relevant outcome measures is often unclear at the time of authorization. If, at time of authorization, insufficient evidence on efficacy and safety exists, the regulatory authorities can register an orphan drug by approving it ‘under exceptional circumstances’. This applies to drugs for which the applicant can demonstrate that comprehensive data cannot be provided (due to specific reasons foreseen in the legislation). At the same time, costs of orphan drugs have increased over the past years, in particular costs of ultra-orphan drugs. This rise has to do with the rapid technical advances including unravelling of molecular mechanisms that underlie some diseases and require a more personalised approach [6]. About 250 new rare diseases are described annually [7]. For member states to make decisions on reimbursement, it is important to gain a better insight into the balance between expenses and health gains for a specific drug in order to determine the ‘value for money’ for orphan drugs. Consequently, economic evaluations play an important role in current decision making about the reimbursement of drugs. In some countries in the EU, reimbursement of drugs depends on the incremental costs per quality-adjusted life year (QALY). In England and Wales, incremental costs exceeding £20 000 to £30 000 per QALY gained are in principle not reimbursed [8–10]. Conversely, other countries in the EU recommend against any pre-set threshold. It is, however, a scientific challenge to evaluate the cost-effectiveness or cost-utility of orphan drugs. With this review we aim to assess the economic evaluations ultra-orphan drugs marketed in the EU that have been performed so far with specific focus on the methods used and the quality of the studies.

Methods

The EMA created the ‘List of medicinal products for rare diseases in Europe’ (http://ec.europa.eu/health/documents/community-register/html/index_en.htm), which describes orphan medicinal products (OMPs) in Europe with European orphan designation and marketing authorization. We used this list, which was updated in April 2015, to identify all OMPs in Europe. Subsequently, www.orpha.net was searched to select those drugs that were developed to treat ultra-orphan diseases (prevalence < 1:50 000).

For each OMP included in this study, we conducted a literature search in the databases of MEDLINE and EMBASE (both via OvidSP) for economic evaluations published prior to April 2015. Key terms used included the disease indication, the drug’s generic name and EU brand name, and the terms cost-benefit, cost-effectiveness, health economics, pharmaco-economics, economic evaluation and quality-adjusted life years, including alternative notations. Additional reports were identified by hand searching the reference lists in the retrieved papers. For each literature search, one author (YS) read the titles and abstracts of all identified studies. If considered relevant, full text was read and analysed. Papers that met the following inclusion criteria were included: I) reporting on an original cost-effectiveness or cost-utility analysis, or at least aiming at such analysis, of an orphan drug for its approved orphan indication, II) orphan drug is marketed in the EU and is indicated for a condition with a prevalence of <1:50 000 (ultra-orphan drug), and III) article is published in the English language. OMPs withdrawn from the EU market or discontinued from the community register of orphan medicinal products (at the end of the 10-year period of market exclusivity) were also included. Studies in which data on costs and/or utilization were collected but not related to a measure of benefit were excluded. To convert different currencies used in the included studies, we used a currency converter (May 2015).

We used the 19-item Consensus on Health Economic Criteria (CHEC)-list to assess the quality of economic evaluations (see Fig. 1) [11]. The CHEC-list has been developed using a Delphi method and focuses on the methodological quality of economic evaluation aspects. It was developed for systematic reviews of full economic evaluations based on effectiveness studies (cohort studies, case–control studies, randomized controlled trials) and is recommended by the Cochrane Handbook for Systematic Reviews of Interventions [12]. The CHEC rating and data extraction was conducted by two authors (YS and MB).

Fig. 1
figure 1

Consensus on Health Economic Criteria (CHEC) list

Full size image

Results

Literature search

We identified 32 ultra-orphan drugs from the EMA and Orphanet website. The cost-utility study for type 1 Gaucher disease compares ERT (imiglucerase [Cerezyme®] and velaglucerase alfa [Vpriv®]) with standard medical care [13]. Although imiglucerase is not registered as an (ultra) orphan drug due to the fact that the EU orphan legislation was installed at a later stage, we decided to include this study in our review since Gaucher is considered to be an ultra-orphan disease. The same applies to the study by Beutler et al., which compares alglucerase (Ceredase®) for Gaucher disease to standard of care [14]. The literature search resulted in 251 articles, of which 16 met our inclusion criteria (see Fig. 2). The majority of the studies considered treatments of lysosomal storage disorders (LSDs) and pulmonary arterial hypertension (PAH). Two of the 16 articles reported on more than 1 ultra-orphan drug resulting in 19 economic evaluations in total (see Table 1). The table summarizes the methods used, comparators, base-case incremental cost-effectiveness ratios (ICERs)/cost-utility ratios (ICURs), and CHEC-list sum scores. To estimate the ICER/ICUR of ultra-orphan drugs marketed for LSDs (alglucosidase alfa [Myozyme®], agalsidase alfa [Replagal®], algasidase beta [Fabrazyme®], alglucerase, imiglucerase, velaglucerase alfa and mercaptamine [Procysbi®]) and for paroxysmal nocturnal haemoglobinuria (eculizumab [Soliris®]), these drugs have been compared to standard medical care, while ultra-orphan drugs for PAH (sildenafil [Revatio®], iloprost [Ventavis®], bosentan [Tracleer®] and ambrisentan [Volibris®]) have been compared with each other. Agalsidase alfa, agalsidase beta, laronidase, sildenafil, iloprost and bosentan were approved under exceptional circumstances. The majority of the studies included were conducted in the USA [14–18], followed by the UK [19–23], and the Netherlands [13, 24, 25]. The remaining studies have been conducted in Spain [26], Australia [27] and Canada 28].

Fig. 2
figure 2

Study flow diagram

Full size image
Table 1 Results of included studies
Full size table

Economic models

A study on enzyme replacement and substrate reduction therapies for LSDs did not conduct a full economic evaluation since a cost-effectiveness or cost-utility analysis was considered unfeasible due to the high drug costs and the lack of a measurable effect on either clinical or health-related quality of life outcomes [22]. In spite of this, the authors calculated that 3.6–17.9 discounted QALYs would need to be generated for each year of being treated in order for them to be cost-effective. Likewise, a cost-effectiveness analysis of laronidase [Aldurazyme®] for mucopolysaccharidosis type 1 was not performed as a consequence of limited evidence of effectiveness [21]. In the same study, a crude decision model was used to estimate cost-utility of agalsidase alfa and beta for Fabry disease. A perfect drug scenario was assumed (i.e., treatment with agalsidase alfa or beta was assumed to restore patients to full health), which undoubtedly led to an underestimation of the ICUR (€351 622 per QALY). The same holds true for another study on enzyme replacement therapy (ERT) for Fabry disease [16]. This study roughly estimated health gains based on only 2 studies, of which 1 reported utilities of Fabry patients in the pre-ERT era and the other used data from the Fabry Outcome Survey (FOS, a Shire sponsored post-marketing drug registry) 10 years later. It is known, however, that the availability of ERT prompted awareness programs and screening studies which ultimately led to an expansion of the disease’s phenotype. Consequently, the FOS cohort consists of patients who have on average a milder disease course than the patients in the pre-ERT study, making it incorrect to ascribe the higher utility scores to the effect of ERT. The reported 80 % probability that ERT has a positive net benefit at a willingness-to-pay of €266 520 per QALY is therefore an overestimation. Cost-utility of alglucerase for Gaucher disease was roughly estimated without using a comprehensive economic model. The authors assumed the best effect a drug can have (immediate death versus survival with perfect quality of life) which certainly led to an underestimation of the ICUR (€43 532–€130 595 per QALY depending on the dose used) [14]. Finally, Connock et al. conducted three crude economic evaluations. Under the assumptions that eculizumab returns survival to normal, the ICER varied from €697 500–€1 953 000per life year gained [23]. In all 4 studies an imprecise estimate of costs was made [14, 16, 21, 23].

The remaining 11 studies used more sophisticated pharmacoeconomic models to evaluate costs and effects of a certain treatment, of which 7 used a Markov-state-transition model [13, 15, 18, 19, 25, 26, 28]. Other models used were patient-level simulation models (N = 3) [20, 24, 27] and decision trees (N = 1) [17]. In only 1 study the scientific basis of the model was explained [20]. In this study, a patient-level simulation model was used because the authors found it more computationally efficient and it removed the need for establishing arbitrary durations for the time cycles as required by a Markov model. In general, pharmacoeconomic models use data from different sources such as literature, own patient cohorts and clinical experience, and are therefore able to give more precise estimates than the studies mentioned above.

Quality assessment

Quality scoring of studies that conducted a cost-effectiveness or cost-utility analysis varied from 7 to 19 out of 19 points on the CHEC-list, with studies using a very simple model having lower scores (7–13) [14, 16, 21, 23] than studies using a Markov-state-transition or patient-level simulation model (14–19) [13, 15, 18–20, 24–28]. The study that used a decision tree scored 15 points [17].

Data on costs and effects were mostly obtained from literature, hospital databases, and health and labour questionnaires. The studies that used a patient-level simulation model restricted events to clinical deterioration and/or death [20, 24, 27], while Markov-state-transition models used up to 47 disease states to describe the course of the disease under study [28]. All studies measured outcomes in QALYs (cost-utility analyses), except for 2 studies in which costs per life year gained was calculated [23, 27]. Four studies considered a societal perspective [13, 17, 24, 25], while the other studies considered only direct medical costs (i.e., a health care perspective) [14–16, 18–21, 23, 26–28]. In all but 2 [16, 17] studies costs and effects were discounted appropriately. The studies by Highland and Garin did not require discounting as the time horizon was 1 year [15, 18]. In most studies the ICER/ICUR was calculated. If a drug is both clinically superior and less expensive, the drug under study is referred to as a ‘dominant’ drug. The opposite is a ‘dominated’ drug, which applies to ambrisentan in the study by Garin et al. [15]. In both cases of dominancy, the ICER would have reached a negative value and is usually not calculated. Thirteen studies applied a sensitivity analysis [13, 15–21, 24–28]. Eight studies discussed the issue of generalizability for application in a different context [13, 15–19, 21, 27]. CHEC-list scores of individual studies are shown in Additional file 1.

Incremental cost-effectiveness/cost-utility ratios

The ICERs and ICURs of individual ultra-orphan drugs are depicted in Table 1. In general, drugs for metabolic diseases appeared to be significantly less cost-effective than drugs indicated for PAH. This can be explained by the lower costs of drugs for PAH, and the short life expectancy (median survival of 2.8 years) of PAH patients if they remain untreated. Effective treatment options can thus result in a substantial number of life years gained, in contrast to the drugs for the slowly progressive, chronic metabolic diseases.

Remarkable differences are observed when comparing the ICURs from different economic evaluations in Fabry disease (€351 622 to €3 282 252 per QALY) [21, 25], Pompe disease (€153 405 to €1 043 868 per QALY) [19, 24], and Gaucher disease (€43 532 to €432 540 per QALY) [13, 14]. These differences can be attributed to the assumptions made; studies assuming survival with perfect quality of life [14, 21], or a huge increase in survival rate [19] upon therapy result in lower but less realistic ICURs.

Discussion

We conducted a systematic review to identify cost-effectiveness and cost-utility studies for ultra-orphan drugs marketed in the EU and applied the CHEC-list to assess the quality of studies included. The challenges to perform adequate economic evaluations in the very small populations of ultra-orphan drugs are expected to be similar to the challenges in the broader group of orphan drugs and for drugs that are used for rare indications but have not received orphan drug designation. Only a limited number (N = 16) of studies on the cost-effectiveness or cost-utility of ultra-orphan drugs have been published, most of which were performed in the USA, the UK and the Netherlands. The large representation of studies from the UK and the Netherlands can be explained by the obligation to perform post-marketing economic evaluations in order to gain reimbursement. In the USA health technology assessments are not systematically conducted, but to an increasing extent orphans are part of the debate on pricing and cost-effectiveness [29].

Five of the 16 studies did not succeed to perform a real economic evaluation, or used an approach that was too simplistic to lead to a realistic estimate of the ICER/ICUR [14, 16, 21–23]. The latter either assumed the maximum possible benefit from therapy (immediate death in case of no treatment versus survival with perfect quality of life with treatment) [14, 21, 23], or based their estimate of effectiveness on only a very limited number of studies which probably led to an overestimation of treatment effect [16]. These studies highlight the fact that economic evaluations of ultra-orphan drugs are faced with many difficulties and limitations. The low number of patients and often slowly progressive, chronic character of the disease make a randomized clinical trial to estimate cost-effectiveness or cost-utility unfeasible. Pharmacoeconomic models using data from different sources such as literature, own patient cohorts and clinical experience are therefore more suitable [30]. Indeed, the majority of studies (N = 11) used models to estimate the cost-effectiveness or cost-utility ratio, including Markov-state-transition models in 7 studies, patient-level simulation models in 3 studies, and a decision tree in 1 study. A decision tree is the most simple model of disease progression. This type of model is primarily useful for short time horizon decisions where the likelihood of an event's occurrence is constant over time. It is in general not suitable for slowly progressive, chronic diseases although it has been used in the assessment of mercaptamin for the treatment of cystinosis [17]. The authors assumed that treatment would delay renal failure for 5 years. After renal failure all patients were assumed to follow the same disease course with dialysis and transplants, irrespective of previous treatment. Such a simplification of the disease course can be analysed with a decision tree, but the modelling choice for a chronic disease usually comes down to a choice between a cohort-level modelling approach such as a Markov-state-transition model, and a patient-level simulation model such as discrete event simulation (DES), which both allow for a more detailed representation of the disease course.

In patient-level simulation models, each patient is represented individually and the final outcomes are calculated by aggregating over all individuals [31]. This type of model has greater flexibility and allows for a better representation of heterogeneous patient cohorts than a Markov-state-transition model, since it allows for irregular time intervals between events. Also, the possibility to make the time to future events dependent on patient attributes including their history of previous events may result in a better reflection of the actual disease course. However, this type of model is in general more complex and needs more patient data to be able to reliably estimate specific effects and interactions of symptoms and complications on the progression of the disease. Kanters et al. used such a model to calculate costs and effects of alglucosidase alfa for the treatment of the infantile form of Pompe disease [24]. Only survival was modeled as event, making it a relatively simple model without the need for large patient numbers. Cost-effectiveness of bosentan has also been studied with patient-level simulation models [20, 27]. Again, only a limited number of events (clinical deterioration and/or death) were modelled. All 3 studies scored between 14 and 17 points. The question is whether a model with a very limited number of events accurately reflects the actual disease course. Complications that arise during the course of the disease were not taken into account while they probably result in a decrease in quality of life and an increase in costs. The model would become more complicated if these events were to be incorporated. The very small patient numbers in the case of ultra-orphan diseases may then limit the use of a patient-level simulation model, since it will be hard to gather sufficient information for reliable estimates of effects and interactions of symptoms and complications. Therefore, economic evaluations of ultra-orphan drugs will often use a cohort-simulation model, more specifically a Markov-state-transition model. This type of model has been used to estimate costs and effects of ERT for Gaucher, Fabry and Pompe disease [13, 19, 25], as well as drugs indicated for paroxysmal nocturnal haemoglobinuria [28] and PAH [15, 18, 26]. All studies scored between 14 and 19 points. In contrast to the patient-level simulation model, the Markov model represents the study population as a homogeneous cohort. It is particularly suitable to reflect the continuous risk of a disease over a long time period [30]. However, the suitability might be limited by the Markovian assumption of ‘no memory’, which means that the probability of transitioning from one state to the other entirely depends on the state the patient is in, regardless of the states the patient passed through the past.

In summary, the choice of which type of economic model to use is dependent on the type of disease that is described, the amount of data available, but also on the time and experience of the researcher. A Markov-state-transition model seems to be most suitable model for economic evaluations of ultra-orphan drugs considering the need for the incorporation of several disease states and the small patient groups. It should be noted here that we found a wide range of ICURs for the same drug even if the same type of model was used. This is at least partly caused by the many assumptions that had to be made due to limited data sets. To be able to build a model that comes as close as possible to the actual course of the disease under study, and to reduce as much as possible the uncertainty of the ICER/ICUR, it is of utmost importance to gather sufficient data. A crucial step in generating data is to establish disease-specific registries which include longitudinal data on all affected patients in the EU. Ideally, registries would start before drugs are marketed, to be able to generate data about the natural history of the disease. For this purpose, EU countries need to work in close collaboration. Existence of reliable data on the natural course and effectiveness of therapy enables researchers to conduct sound economic evaluations.

On the other hand, it has been argued that cost-effectiveness or cost-utility studies are inappropriate in the case of (ultra) orphan drugs since conventional criteria for cost-effectiveness will never be met. Indeed, considering studies using either a Markov-state-transition or patient-level simulation model, ICURs for different types of ERT ranged from €432 540 (Gaucher disease) to €3 282 252 (Fabry disease). Wyatt et al. calculated the number of QALYs to be generated each year in order for different types of ERT to be cost-effective which resulted in the impossible amount of 3.6-17.9 QALYs [22]. Consequently, most ultra-orphan drugs are not ‘cost-effective’ if standard health technology assessment procedures were to be applied to them, which was already stated in a previous paper by Drummond et al. [32]. It is highly unlikely that the ICUR could be decreased by the orders of magnitude required to make ultra-orphan drugs cost-efficient by current standards in the future. Although there is no clear consensus on what constitutes an acceptable cost-utility ratio, in the Netherlands a value of €80 000 per QALY for illnesses associated with a considerable burden has been suggested. However, so far, no drugs have been withdrawn from being reimbursed because of their unfavourable ICUR in the Netherlands. The National Institute for Health and Clinical Excellence (NICE) in England and Wales uses a threshold ICUR of £20 000 - £30 000 per QALY for non-orphan drugs [33]. The value of ultra-orphan drugs is assessed in a ‘Highly specialised technology (HST)’ programme, which is similar to the Single Technology Appraisal process [34]. The ICUR of £20 000 - £30 000 per QALY is not strictly adhered to when evaluating orphan drugs, and greater emphasis is applied to other factors such as ethics and lack of alternative treatments. In other countries in the EU (such as Ireland) there is no threshold ICUR value [35]. The lack of consensus on the threshold of the ICUR and absence of consequences for reimbursement in some countries may imply that cost-utility is not the only principle that has to be taken into account when assessing the value of orphan drugs. From a societal perspective, other (ethical) considerations may matter, such as the rarity and seriousness of the disease, the availability of alternative therapies and the cost to the patient if the drug would not be reimbursed [36]. These may be reasons to accept higher costs per QALY. Possibilities to deal with this are to apply variable cost-utility thresholds with a higher threshold for orphan drugs, or to value the health gain in a patient with a rare disease with a higher weight [5]. It is questionable, however, whether the QALY in itself is a suitable outcome measure in the case of ultra-orphan diseases. In most studies the EQ-5D (EuroQoL five dimensions) quality of life questionnaire is used to determine health status. Each observed health score profile on this questionnaire can be converted to a utility score ranging from - 0.594 (i.e., serious health problems with mobility, self-care, usual activities, pain/discomfort, and mood) to 1 (i.e., no problems at all). Patients with a slowly progressive, chronic disease will not be cured from one day to the other. In other words, the utility score will mostly not increase to perfect quality of life upon therapy. Instead, patients may feel less tired and better capable of working which is less well represented in the EQ-5D questionnaire. Consequently, the use of the QALY in its current form for the assessment of cost-utility of ultra-orphan drugs has been constantly discussed (see for example: http://www.rarediseaseblogs.net/author/cees-smit/). Others have argued that the QALY measure is even invalid [37]. Beresniak et al. asked more than 1300 subjects to express their preferences regarding combinations of different health states and time durations, which were subsequently compared to the results of the QALY formula (quality of life x number of life-years). They showed that observed and calculated utility values were significantly different. This might explain why costs/QALY estimates vary greatly. The authors argued that other approaches for health care decisions should be considered. Indeed, the costs/QALY approach has been rejected by the USA and Germany for ethical and methodological reasons [38, 39]. However, to remove an imperfect measure without replacing it with another might be not advisable [40].

Aside from using more suitable, representative ways to determine health status, bringing down costs would also lead to more acceptable ICERs/ICURs. The extremely high prices charged for new orphan drugs may cause an unsustainable pricing market that threatens the health care structure [41].

Conclusions

Altogether, economic evaluations of ultra-orphan drugs are feasible if pharmacoeconomic modelling is used. The most suitable type of model seems to be a Markov-state-transition model. It should be realised, however, that most ultra-orphan drugs will not meet conventional criteria for cost-effectiveness. Still, ultra-orphan drugs are often reimbursed. Further discussion on the application of economic evaluations and their consequences in case of ultra-orphan drugs is therefore called for. The challenges described in this study may also apply to the broader group of (non-ultra) orphan drugs. However, this needs further study.