The systematic review identified 660 records, and 131 underwent full-text review . Fifteen academic studies, published between 2000 and 2018, were eligible for inclusion (Appendix 1, Table A2, see ESM) [35,36,37,38,39,40,41,42,43,44,45,46,47,49]. Eight studies were from the US, five from European countries, one from Canada and one from Australia. No study was adjudicated to have perfect compliance with the reporting checklists. From the 15 included studies, 35 distinct pairwise comparisons were identified.
EUnetHTA Transferability Assessment
The outcomes of the transferability assessment are shown in Table 2. Most of the identified CEAs had the required perspective (payer, n = 10), and therefore the required approach to indirect costs [35, 37, 38, 41, 42, 44,45,46,47, 49]. Two CEAs adopted the societal perspective [36, 43], while three stated a societal perspective was taken but did not document the inclusion of indirect costs and were categorised as payer perspective [39, 40, 48]. Utility values were derived using either health state vignettes and the standard gamble method (n = 5), or from a generic utility instrument (EQ-5D, n = 7), while one study used a mixture of both . Three studies did not apply or did not specify a discount rate [36, 39, 43]. One study applied a different rate to costs and QALYs . Rates applied varied from 3 to 6%; only three studies applied the same discount rate as the Irish reference case of 5% [35, 42, 47]. The technological context for the management of advanced melanoma has moved swiftly in recent times. However, studies were considered transferable if they considered comparators that were relevant at the time of publication, and if the pre- and post-intervention care modelled in the CEA were considered a likely representation of care at that time in Ireland. Thus, no studies were excluded due to differences in the technological context or pre- and post-intervention care.
Treatment acquisition costs were generally based on the publicly available price, with three exceptions where mark-down prices were used, which limits transferability to the Irish context [38, 46, 48]. Three studies did not provide a cost per QALY outcome [36, 43, 48], and were not considered transferable to the Irish context, as there is no specified cost-effectiveness threshold for alternative outcomes.
The EUnetHTA tool includes several health system and national factors when considering transferability: personnel characteristics, the integration of technology into the healthcare system, factors affecting disease incidence and prevalence and the epidemiological and demographic context. The reference case for HTA in Ireland does not provide specific directions relating to these topics, except to highlight that the inputs should be reflective of the relevant target population. The epidemiological context and factors that influence incidence and prevalence were considered likely to be different in the identified cost-effectiveness models, as Ireland has a relatively high incidence of advanced melanoma and a lower incidence of melanoma with the BRAF mutation compared with international norms . Ireland also has a younger population than many other OECD countries, and a relatively high life expectancy . However, in the absence of explicit modelling of baseline risk or national population demographics, these differences were considered unlikely to impact on the outcomes of the included cost-effectiveness models. Insufficient information was provided in the reviewed studies to determine if personnel characteristics or integration of technology into the health service were similar to Ireland, but it was considered unlikely that there would be significant differences. These factors were all marked as unlikely to affect transferability in this assessment.
Incentives were considered likely to be the same in five CEAs with publicly funded health systems, as they provide health care in a similar model to Ireland [38, 42, 46, 47, 49]. Studies conducted within the US health system were considered to have different incentive structures, due to differences in the model of health care provision [36, 37, 39,40,41, 43, 45, 48]. The incentive structure was considered unclear in two studies, set in Australia and Switzerland, due to the interplay of private and public provision and compulsory private health insurance, respectively [35, 44]. However, the incentive structure was considered unlikely to have a significant impact overall on the costs and QALY gain associated with each treatment, and so did not preclude any studies from the specific transferability assessment.
Where deterministic sensitivity analyses were published, the outcomes were reviewed in light of the specified parameters of the Irish reference case. Discount rates, when included, were not found to be important drivers of the ICER in any of the CEAs [35, 38, 47]. Time horizon was an important driver of the ICER in three studies [35, 37, 42]. In two of these studies, the time horizons used were significantly shorter than the recommended lifetime horizon of the Irish reference case, which could potentially limit the transferability of the outcomes to the Irish context [35, 42]. In both cases, the decision was unlikely to change with an increase in the time horizon. Health state costs were important in four studies in determining the ICER [35, 37, 38, 41], but would only have changed the decision in one study when varied to the upper limit in the deterministic sensitivity analysis . Utility was an important determinant of the ICER in seven of the included studies, but only in two of those would it be likely to change the decision based on the tornado plots provided in the published manuscript [45, 48].
None of the identified CEAs were entirely generically transferable to the Irish setting, as no model was fully aligned with the Irish reference case. Alignment with the reference case was most commonly seen for perspective, treatment of indirect costs and the technological context. Disagreement was most commonly seen for the discount rate (n = 12). It was considered highly likely that the results would change or be irrelevant if transferred to the Irish setting for five of the CEAs, two of which did not provide a cost per QALY outcome, and three of which used mark-down prices to estimate the ICER. The remaining studies (n = 10) were included in the consideration of specific transferability.
Transferability of Conclusions in Accordance with Cost-Effectiveness Threshold
For the assessment of specific transferability, incremental costs, QALYs and cost-effectiveness thresholds were extracted from the included CEAs (n = 10) (Table 3). Twenty-five pairwise comparisons were considered from these studies. The NMB was calculated at the cost-effectiveness thresholds specified in the published studies, which varied from US$35,000 to US$200,000 per QALY (Table 3). Of the 25 comparisons, nine had a positive NMB at the cost-effectiveness threshold specified in the published study (36%) (Table 3), indicating that they were cost effective at the study threshold, and 16 (64%) had a negative NMB, indicating that they were not considered to be cost effective.
Secondly, the NMB for these pairwise comparisons was recalculated using the Irish cost-effectiveness thresholds. When the higher Irish threshold (US$60,000 per QALY) was applied to the calculation of NMB, only four comparisons estimated a positive NMB (16%), a reduction of 20% compared with using the CEA-specific threshold (Table 3). Using the lower Irish threshold (US$26,667 per QALY), only three comparisons (12%) retained a positive NMB.
Data on incremental costs and QALYs for the pairwise comparisons were extracted from the published NCPE summary HTA reports and converted to US dollars (Table 4). The NCPE had not evaluated six of the pairwise comparisons for the Irish drug reimbursement process (mainly comparisons of the newest agents versus chemotherapy, and unlicensed regimens). Of the 19 pairwise comparisons that could be compared with the identified CEAs, in 14 (73.7%) instances the NCPE conclusion based on the ICER was in agreement with the study conclusions, and five were in disagreement (26.3%).
When the study NMB for each pairwise comparison was re-estimated at the higher Irish threshold, 17 of 19 (89.5%) pairwise comparisons were in agreement with the NCPE conclusions based on the ICER. In the five comparisons where the conclusions changed from positive at the CEA-specific threshold to negative at the Irish threshold, three changed to be in agreement with the NCPE conclusions, whilst two that had previously agreed with the NCPE were now different [41, 47].
When the study NMB for each pairwise comparison was re-estimated using the lower Irish threshold, only three comparisons (12%) retained a positive NMB. The single additional study that changed conclusion at this threshold compared with the higher Irish threshold shifted to be in agreement with the NCPE conclusion, resulting in a total of 18 out of 19 comparisons (95%) being in agreement with the NCPE conclusions.