FormalPara Key Points

Multi-criteria decision analysis (MCDA) can improve the healthcare decision-making process by considering an explicit set of criteria and their relative importance under a fully transparent process.

This study approximated the overall estimated value of two innovative drugs for chronic inflammatory skin diseases (atopic dermatitis and psoriasis) from a broad and systematic view, while incorporating local multi-disciplinary views to express a societal perspective.

This exercise allows us to better understand where the value of dupilumab and secukinimab lies for the different stakeholders, providing useful information that could help to make better decisions on the assessment, pricing and public reimbursement of these interventions.

1 Introduction

In healthcare, deciding the most appropriate and efficient allocation of resources is a difficult but necessary task that requires consideration of multiple factors. The process often utilises decision-supporting tools, such as economic evaluation. However, cost-effectiveness analysis has some limitations: it has been criticized for its rigidity and for not adequately capturing the social value of a drug [1,2,3,4].

Given the need for a more systematic and explicit approach that takes into account factors relevant to society simultaneously, multi-criteria decision analysis (MCDA) represents an emerging tool in health decision-making that aims to complement the cost-effectiveness analysis [5, 6]. MCDA allows participants to reflect and express clearly which criteria they are considering and how they are valued, introducing greater transparency, consistency and accountability [7]. Also, MCDA helps people to face ethical dilemmas, and it provides a context for the promotion of multi-disciplinary dialogue based on evidence [6, 8,9,10].

MCDA is a useful methodology that guides the making of complex decisions, such as the funding or reimbursement of healthcare interventions. Increasingly, it has been used in real practice to prioritise different healthcare interventions [7,8,9,10,11,12,13,14,15,16,17] or to assess the value of drugs [18,19,20]. Chronic disabling diseases, such as the most serious dermatological pathologies, constitute an area in which MCDA can be especially useful. However, very few MCDA have focused on chronic inflammatory diseases [21].

Atopic dermatitis (AD) and psoriasis are two of the main dermatological diseases [22]; they are systemic, inflammatory, chronic and immunomediated dermatoses that substantially reduce the quality of life of patients [23,24,25]. Biological drugs are effective in the management and control of severe forms of both diseases. In the case of psoriasis, therapeutic perspectives and clinical experience is much more extensive, thanks to the existence of various approved drugs. In contrast, there is currently only one officially approved biologic drug for AD, which is available in a few countries.

Dupilumab (Dupixent®, Sanofi-Aventis Group, La Boetie, Paris, France), approved in 2017, is indicated to treat moderate to severe AD in adults who are candidates for systemic therapy. Prior to its authorisation, the only treatment with approved indication for severe AD was ciclosporin (cyclosporine), which is effective in the management of the disease in the short-term but has a high risk of cumulative toxicity; thus, it is not advisable to administer for prolonged periods, limiting its use for the treatment of any chronic pathology [26].

Secukinumab (Cosentyx®, Novartis Europharm Limited, Camberley, United Kingdom), authorised in 2015, is a biological drug indicated to treat moderate to severe plaque psoriasis in adults who are candidates for systemic treatments. Its relevant comparators are other biological treatments, such as etanercept, infliximab, adalimumab and ustekinumab.

The main objective of this study was to assess the value of dupilumab in the treatment of severe AD compared with the value of secukinumab in the treatment of moderate to severe plaque psoriasis in Spain, using the MCDA methodology. The advantage of comparing these drugs indirectly is better interpretation of the result for dupilumab, contextualising it with that of secukinumab, a drug that was authorised some years ago and is perceived as a high value-added medicine.

2 Method

This study was designed following the Evidence and Value: Impact on DEcision Making (EVIDEM) framework (version 3.1) for MCDA, a methodology widely used and based on the application and evaluation of a standard set of 13 quantitative criteria, obtaining an estimated overall value of the intervention [27]. The EVIDEM framework also includes a contextual tool for qualitative criteria.

The steps taken to carry out the MCDA (Fig. 1) were based on previously published good methodological practices [7, 28]. A committee of ten experts was formed, comprising three clinicians (dermatologists), four patients (two with severe psoriasis and two with severe AD), two regional payers and one health economist. Members were selected based on their expertise, while trying to achieve a balanced geographical representation. The consultancy firm WEBER (Madrid, Spain) reached out to the panellists, except for the patients, who were selected by their dermatologists—also experts on this committee. WEBER was responsible for training the experts committee on MCDA, reviewing and sharing the evidence with the panellists prior to the meeting, coordinating the committee, and analysing and presenting the results.

Fig. 1
figure 1

Study design

Full size image

Following the EVIDEM methodology, the first step consisted of assigning weights and was carried out online before the experts knew the interventions to be evaluated. Each expert assigned a weight between 1 and 5 to each quantitative criterion according to its importance, 1 being the least relevant and 5 the most.

The second step was carried out through a face-to-face meeting of the committee. Based on the available scientific evidence (previously validated by the committee’s clinicians) (Electronic Supplementary Material 1), the debate generated, and individual experience and perception, each expert assigned a score to each quantitative criterion for the two interventions evaluated. The scores for the absolute criteria (those that did not involve comparisons with alternative interventions) could range from 0 to 5, 0 being the lowest value and 5 the highest. For relative criteria (those involving comparison with an alternative intervention), the scale ranged from − 5 to 5 to reflect the full range of comparative effects (Electronic Supplementary Material 2). The comparators were those contained in the pivotal clinical trials of the respective interventions. Thus, dupilumab was evaluated only against placebo, whereas secukinumab was evaluated threefold: against placebo, etanercept and ustekinumab. The impact of the seven EVIDEM qualitative criteria was also discussed at the meeting, based on the available evidence (Electronic Supplementary Material 3) and the stakeholders’ perception.

The estimated value for each intervention was obtained through a linear additive model, combining the relative weighting of each criterion with the reported score for each intervention in each criterion. Each estimated value was transformed into a 0–1 scale to facilitate its interpretation. In particular, the following formula was used [8]:

$$V = \sum\limits_{x = 1}^{n} {V_{x} = \sum\limits_{x = 1}^{n} {\left( {\frac{{w_{x} }}{{\sum {W_{n} } }}S_{x} } \right)} } ,$$

where V is the total estimated value, V x the value contribution of criterion x, W x the weighting of criterion x, ΣW n the sum of all weights, and S x the scoring of criterion x.

To check the degree of consistency and replicability of the analysis, a retest was carried out several weeks after the meeting. The experts re-evaluated the criteria, online, for both interventions, assigning weights and scores. The degree of agreement between the responses made at the two timepoints was evaluated through the intra-rater correlation coefficients (ICC 3,1) using STATA® version 14 (STATA Corp., LP, College Station, TX, USA).

3 Results

3.1 Panellists’ Weights

The panellists considered that all criteria had a high relative importance, giving all of them an average weight above 3 (out of 5) (Fig. 2). According to the committee, the most relevant criteria should be quality of evidence (4.80 ± 0.42), comparative effectiveness/efficacy of the intervention (4.60 ± 0.52) and disease severity (4.40 ± 1.07). The least relevant criteria were the type of preventive benefit (3.10 ± 1.29) and the size of the affected population (3.30 ± 0.67). The greatest variability in the responses was given to the type of benefit of the intervention, both preventive (standard deviation [SD] 1.29) and therapeutic (SD 1.20).

Fig. 2
figure 2

Mean weights for decision criteria by the advisory committee. A 5-point weighting scale was used, with 1 for the lowest weight and 5 for the highest weight

Full size image

In general, the clinicians assigned the highest average weights (between 3.67 and 5.00), while those of decision makers (between 2.33 and 4.67) and patients (between 2.75 and 4.75) were lower and more similar to each other. The greatest discrepancies occurred between clinicians and decision makers in the type of preventive benefit and the quality of evidence, and between clinicians and patients in the three variables of relative cost.

3.2 Scores for Dupilumab

Regarding the mean scores for dupilumab for severe AD (Fig. 3), the committee gave the highest score (4.8 out of 5) and with the strongest consensus (SD 0.42) to the unmet needs of the disease, highlighting the problems faced by patients with severe AD, as they currently lack long-term effective and safe systemic therapies [23,24,25]. The severity of the disease obtained the second highest score (4.50 ± 0.85). The committee assessed the severe AD in the context of all the diseases that a patient may suffer, specifying that the pruritus is usually intense and influences the mood, rest, relationships and daily activities of the patients, substantially reducing their quality of life [29,30,31]. Evidence indicating that moderate to severe AD has a greater impact on quality of life than other diseases, such as psoriasis, especially in the field of mental health, was presented [32,33,34,35].

Fig. 3
figure 3

Mean scores for decision criteria for the appraisal of dupilumab (vs. placebo) by the advisory committee. Criteria 1–3, 7–8 and 12–13 were evaluated in absolute terms (with a score between 0 and 5), while the rest were evaluated in terms relative to placebo (with a score between – 5 and 5)

Full size image

According to the committee, dupilumab presents an improvement in clinical efficacy (4.40 ± 0.84) and patient-reported outcomes (PROs) (4.10 ± 1.20) versus placebo, as well as a higher therapeutic benefit (4.10 ± 0.74). In contrast, the score associated with the preventive benefit of the treatment was 0.80 (SD 1.69). The greatest variability occurred in the relative safety profile (SD 2.87). According to the prevalence considered, the size of the population potentially affected by the intervention is estimated to be between 4.9 and 9.8 of every 10,000 people in Spain. Experts scored this criterion with an average score of 2.50 ± 0.53. The number of people indicated for treatment with dupilumab, that is, non-responder patients with severe AD or who have contraindications or intolerance to systemic immunosuppressive therapies, was taken into account.

In terms of costs, given that there were still no real data available for Spain, the committee relied on its own perception and personal experience. Because dupilumab is a first-in-class biological, the annual treatment cost per patient was assumed to be the same as for secukinumab. This criterion was the only one where dupilumab obtained a negative average score (− 2.8 ± 1.55), which implies that it was more expensive than its comparator (placebo). The other two cost criteria were scored positively, suggesting that the treatment could produce savings on other healthcare costs (1.10 ± 1.29), especially non-healthcare costs (2.70 ± 1.64), thanks to the presumable reduction of losses of working or teaching hours due to better control of the disease [36, 37].

The quality of the evidence received a mean score of 3.70 (SD 1.34), meaning that the pivotal trials from which the evidence came were deemed adequate [38, 39]. In contrast, the degree of alignment of dupilumab with current clinical practice guidelines (CPG) was considered low (1.40 ± 2.07), given that, due to its recent approval, it has not yet been included in the guidelines.

The analysis highlighted certain differences in the opinions of the expert committee members. While patients tended to assign the highest scores, decision makers were the least benevolent in the scoring of most of the criteria and the clinicians were the strictest in scoring safety and cost of acquisition. For example, clinicians and patients unanimously scored the severity of the disease with the highest score, while it was not a chief concern for decision makers (3.33 ± 0.58). Efficacy scores showed large differences between the three groups (3.33 for decision makers vs. 4.67 for clinicians and 5 for patients), as did the type of therapeutic benefit (3.33 vs. 4.33 and 4.5, respectively).

3.3 Scores for Secukinumab

It is important to note that the scoring of the absolute criteria was common for the three comparisons (vs. placebo, etanercept and ustekinumab). The absolute criteria that obtained higher scores were severity of the disease (4.30 ± 0.82) and type of therapeutic benefit (4.30 ± 0.67), followed by quality of the evidence (4.00 ± 0.94) (Fig. 4). Given the symptoms, co-morbidities and impairment of patients’ quality of life [40,41,42], the committee rated moderate to severe plaque psoriasis as highly severe but lower than severe AD (4.3 vs. 4.5). The scientific evidence on secukinumab was assessed as relevant and well reported (4.00 ± 0.94), despite some limitations regarding the choice of comparators, their doses, the selected subpopulations and the effectiveness measurement bias [43,44,45].

Fig. 4
figure 4

Mean scores for decision criteria for the appraisal of secukinumab (vs. placebo, etanercept and ustekinumab) by the advisory committee. Criteria 1–3, 7–8 and 12–13 (blue markers) were evaluated in absolute terms (with a score between 0 and 5) and were applied equally to the three comparisons made, while criteria 4–6 and 9–11 (orange markers) were evaluated in relative terms to the different comparators (with a score between − 5 and 5). ETN etanercept, PBO placebo, UTK ustekinumab

Full size image

The intervention would affect a greater population size than dupilumab (27 per 10,000 people [46,47,48]), with this criterion scored as 3.50 ± 0.85. Regarding the unmet needs at the time of the release of secukinumab, these were quantified as lower than for dupilumab, thanks to the existence of other highly effective biological drugs (2.1 vs. 4.8 for dupilumab). In view of the fact that the CPG place secukinumab as another treatment option, recommending it at the same level as the rest of the options [48,49,50], the committee assessed this criterion as having a score of 1.00 ± 0.82. It was clear that the treatment did not offer a marked preventive benefit (0.10 ± 0.32).

The relative criteria were scored against the three available comparators, resulting in a clear gradient. The most favourable scores were given versus placebo and the least favourable versus ustekinumab. When compared with placebo, efficacy and PROs were assessed with very high scores and a high degree of consensus (5.00 ± 0.00 and 4.90 ± 0.32, respectively). These scores decreased when compared with etanercept (3.70 ± 0.82 and 3.20 ± 1.40, respectively) and ustekinumab (1.90 ± 1.45 and 1.70 ± 1.83, respectively), as the difference in efficacy and PROs between treatments decreased [39, 43,44,45]. Concerning tolerability, secukinumab had an acceptable safety profile, with similar adverse events than other biologic drugs indicated for psoriasis [44, 45].

Regarding the economic consequences, the committee distinguished the direct costs of the intervention (with a negative score vs. the comparators) from the rest of the resulting direct and indirect costs (with a positive relative score). The cost of the intervention was estimated based on the defined daily dose (according to the technical specifications), the official drug price and the mandatory discount applied in Spain [51, 52]. The annual cost per patient treated with secukinumab was assumed to be slightly lower than that of ustekinumab and 36% higher than that of etanercept (which already had a biosimilar approved). The average scores were − 3.00 ± 2.40 versus placebo, − 0.90 ± 1.60 versus etanercept and – 0.30 ± 2.20 versus ustekinumab. The committee considered that the treatment with secukinumab would avoid some healthcare costs with respect to alternatives and reduce labour productivity losses [45, 53, 54].

The patients were also the group who gave the highest mean scores to the evaluated intervention. The greatest differences between expert groups were found for the relative criteria and for unmet needs (SD 1.91).

3.4 Value Estimates for Dupilumab and Secukinumab

The overall estimated value integrated the weights and scores of each panellist on a scale between 0 and 1. Regarding the contribution of each criteria to the overall value estimate for dupilumab and secukinumab (Fig. 5), both interventions were considered to provide a high added value versus their comparators. Specifically, the estimated value for dupilumab in severe AD, versus placebo, was 0.51 (95% confidence interval [CI] 0.42–0.61)]. This value was higher than that obtained for secukinumab in moderate to severe plaque psoriasis, independent of the comparators considered: 0.48 (95% CI 0.37–0.59) versus placebo, 0.45 (95% CI 0.34–0.56) versus etanercept and 0.39 (95% CI 0.27–0.52) versus ustekinumab. The greatest differences between the value of dupilumab and secukinumab were due to the compared efficacy (0.08 vs. 0.03 when using ustekinumab as a comparator), followed by the cost of the intervention (− 0.05 vs. − 0.01), unmet needs (0.07 vs. 0.03) and PROs (0.06 vs. 0.03).

Fig. 5
figure 5

Mean value contributions of each quantitative criterion and overall value estimates for dupilumab and secukinumab. Average of the committee of experts and for each group (clinicians, decision makers and patients). \({\text{Value contribution}}\; = \;{\text{normalised weight}} \times {\text{normalised weight}} = \sum {{\text{value contribution of all }}13{\text{ criteria}}.}\) ETN etanercept, PBO placebo, UTK ustekinumab

Full size image

Approximately half of the estimated value of dupilumab was due to the positive contribution of four criteria: comparative effectiveness/efficacy (13.0%), disease severity (12.8%), unmet needs (11.7%) and quality of evidence (11.5%). The comparative cost of the intervention was the only criterion that contributed negatively, reducing the estimated value of dupilumab by 7.8% (Fig. 5a).

For secukinumab, 65.8% of the estimated value versus placebo was due to the positive contribution of five criteria: comparative effectiveness/efficacy (15.5%), quality of evidence (13.0%), comparative PROs (12.9%), severity of the disease (12.8%) and type of therapeutic benefit (11.6%) (Fig. 5b). The compared cost also contributed negatively to the overall estimated value (− 8.7%). When secukinumab was compared with alternative biologics, the relative criteria contributed less to the final value. In both cases, the criteria that contributed most to the estimated value were the quality of evidence (15.7% in the etanercept scenario and 18.6% in the ustekinumab scenario), severity of the disease (15.4 and 18.3%) and type of therapeutic benefit (14.0 and 16.6%). In contrast, the negative contribution of the compared costs of the intervention had a smaller impact on the value estimate for secukinumab (3.3% vs. etanercept and 1.4% vs. ustekinumab) (Fig. 5c, d).

3.5 Replicability and Consistency

The retest showed good reproducibility, with a relatively high degree of agreement and consistency of the results. Between test and retest, 48% of weights were identical, 38% differed by 1 point and 14% differed by 2 or 3 points. The greatest inconsistency was recorded for the patients’ subgroup. The ICC for weights was 0.52. Scores showed a slightly better consistency than weights: 59% were identical in the case of dupilumab and 53, 57 and 60% in the case of secukinumab (vs. placebo, etanercept and ustekinumab, respectively). The greatest divergences occurred in type of preventive benefit, severity of the disease and compared cost of the intervention. The intra-rater correlation between scores was slightly higher for dupilumab (0.802) than for secukinumab (0.763 vs. placebo, 0.750 vs. etanercept and 0.735 vs. ustekinumab).

The value estimate obtained in the retest for the two interventions was consistent with that of the primary analysis. The value for dupilumab was 0.55 (95% CI 0.44–0.66) (6.4% higher than the original value), thanks to the greater relative contribution of unmet needs, PROs, type of therapeutic benefit, other healthcare costs and quality of evidence (Table 1). The improvement in the overall value estimate was significant in the group of decision makers (+ 27%), compared with a lesser improvement among patients (+ 5.3%) and a decrease among clinicians (− 8.8%).

Table 1 Estimated value for dupilumab and secukinumab in the test and the retest
Full size table

The retest performed for secukinumab also showed consistent figures with the original analysis. The estimated value remained lower than that obtained for dupilumab, again showing a clear gradient between comparators: 0.50 (95% CI 0.36–0.63) versus placebo, 0.44 (95% CI 0.31–0.58) versus etanercept and 0.40 (95% CI 0.25–0.54) vs. ustekinumab (Table 1). The degree of correlation between the overall values at both timepoints for secukinumab was higher than for dupilumab.

Despite the relatively high degree of agreement and consistency between test and retest, the differences between the values of both drugs increased from 7 to 11% in the placebo scenario, from 14 to 23% in the scenario with etanercept and from 30 to 38% in the scenario with ustekinumab.

3.6 Qualitative Criteria

The panellists also assessed the impact of each qualitative criterion on both interventions (Electronic Supplementary Material 4). Given the lack of accurate and relevant evidence, together with some difficulties of interpretation, the contextual tool represents complementary information that supports a wider overview of the MCDA framework.

4 Discussion

The aim of our study was to assess the value of dupilumab for severe AD compared with the value of secukinumab for moderate to severe plaque psoriasis in Spain, using an MCDA. To our knowledge, this is the first MCDA applied in the field of dermatology in Spain and the first MCDA that estimates the value of a treatment for AD. Our work has made it possible to numerically approximate an overall value estimate for two innovative medicines from a broader perspective than usual, taking into account fundamental additional elements. Although there is still no evidence on indirect comparisons between both drugs, the juxtaposition of AD and psoriasis is not new [55, 56]. The availability of therapeutic alternatives was a relevant differential element. Currently, severe AD is a chronic disease with many unmet needs. By contrast, the psoriasis context is radically different because there are different biological drugs that can manage the disease effectively and safely. In addition, severe AD was deemed to have a greater impact on quality of life than psoriasis.

Our study was able to compare, to a certain extent, the results obtained for both drugs, given that both AD and psoriasis are chronic inflammatory diseases, the two drugs evaluated are biologics that were commercialized recently and the assessment was performed by the same committee.

The results suggest that the two evaluated biologicals constitute interventions of high added value. The innovative drug selected for severe AD (dupilumab) obtained an overall value estimate of 0.51 points. This figure was similar to that of the drug selected for moderate to severe plaque psoriasis (secukinumab) when compared with placebo (0.48 points) and slightly higher than secukinumab when compared with its two alternatives (0.45 and 0.39). As expected, the estimated value for secukinumab decreased as the performance of its comparator increased. The retest revealed fair to good consistency of the results. Despite the multi-disciplinary debate, differences among stakeholders persisted.

To contextualize, the estimated values are slightly above the range of values obtained in the literature for other innovative drugs to which the MCDA–EVIDEM framework was applied. For example, obinutuzumab for rituximab-refractory indolent non-Hodgkin lymphoma obtained a value of 0.45 in Italy, tramadol for chronic non-cancer pain had a value of 0.44 in Canada, growth hormone for Turner syndrome had a value of 0.41 in Canada, and lenvatinib for radio-iodine-refractory differentiated thyroid cancer had a value between 0.33 and 0.38 in Spain [8, 57,58,59].

The panellists noted that the exercise was useful and interesting, and that the exchange of opinions between them enriched the analysis and individual assessments. However, they also stressed that it required some familiarity with the assessment of medications, the amount of evidence provided, and the type of concepts and language used. In this regard, it is important to emphasise the importance of providing the best possible understanding, especially to patients, the group least familiar with this type of exercise. On the other hand, the differences between the types of experts that made up the committee is worth noting. Both their weightings and scores inevitably conformed to their own perceptions, experience, training and value judgements, and therefore shows the importance of having this multi-disciplinary vision.

The study is not exempt from certain limitations. First, it carries the inherent limitations of the EVIDEM approach, which includes a fixed set of criteria and, therefore, may exclude other relevant criteria. However, this methodology allows for the use of validated criteria and is in continuous review and evolution. Secondly, it is a subjective exercise, the results of which depend, to a large extent, on the composition of the evaluation committee, and the value judgements, experience and training of its members. However, the fact that the same committee evaluated both drugs facilitates the comparison of the results. Thirdly, MCDA can be complex for some agents, which can lead to misinterpretation of some of the evidence and/or scoring scales. We tried to alleviate this limitation by sending each member complete information on the methodology and evidence, in an understandable language, prior to the meeting. Lastly, the representativeness of the results at a national level is limited by the small number of experts that made up the committee. Nevertheless, the number of experts included is in line with other MCDA studies carried out for a specific intervention, between eight and 19 experts [8, 57, 58].

We should also note the influence of both the comparator used and the time of execution on the final result, as the valuation obtained is not generalizable to other comparators, nor does it last over time. Finally, the value estimates of the MCDA are not intended to be used in a prescriptive manner, but rather as a further element of discussion that serves as a basis for ordering interventions and complementing pharmacoeconomic analyses, which usually lack this broader perspective [8, 60].

5 Conclusions

This study applied an MCDA to the evaluation of two innovative biological drugs aimed at treating chronic inflammatory diseases in Spain. The results show the high added value posed by both drugs from a multi-disciplinary perspective, under a methodological umbrella that takes into account a wide spectrum of value attributes. Assuming a similar annual economic cost per patient for both treatments, the results confirm that dupilumab as a therapy for severe AD has a final estimated value similar to or slightly higher than secukinumab for moderate to severe plaque psoriasis. This exercise allows us to understand better where the value of health interventions lies for the different agents. It is presented as a tool aimed to help agents to make decisions on the assessment, pricing and public reimbursement of health interventions. In the future, it would be desirable to continue developing MCDA and to extend its use, so that decision-making takes place in a framework of greater transparency, consistency and efficiency, serving as a complementary tool to economic evaluation.