FormalPara Key Points for Decision Makers

Nivolumab is expected to have a better survival profile than observation.

The full range of overall survival uncertainty should be tested (i.e. using more than one modelling approach). This is difficult if patient-level data are not available.

Decisions need to be made based on the most recently available data because clinical pathways are always changing.

1 Background

Melanoma is the most dangerous form of skin cancer, caused mainly by ultraviolet exposure-induced mutations leading to rapid multiplication of skin cells and the formation of malignant tumours [1]. For early-stage melanoma, surgical resection is the standard treatment and is associated with good long-term survival prognosis for stage I and II disease [2]. However, patients with stage III disease (who have regional involvement of lymph nodes at diagnosis) or metastatic disease are at higher risk of recurrence after loco-regional resections [2]. Melanoma classified as stage III is described as disease that has spread locally or through the lymphatic system to a regional lymph node or on the way to a lymph node (in-transit/satellite/microsatellite disease) [3]. In stage IV, the melanoma has spread through the bloodstream to other parts of the body and is mostly considered unresectable [3]. The risk of recurrence increases with increasing disease stage. The overall 5-year recurrence-free survival (RFS) for patients with stage IIIA, IIIB and IIIC is approximately 63%, 32% and 11%, respectively [4].

Until recently, adjuvant treatment options for stage III and IV resectable melanoma were limited and included interferon and ipilimumab in the USA. Nivolumab (Opdivo®, Bristol-Myers Squibb) is a human immunoglobulin G4 (IgG4) monoclonal antibody that disrupts programmed cell death 1 (PD-1) signalling between T cells and tumour cells, restoring T cell anti-tumour immunity. Nivolumab is currently licensed by the European Medicines Agency (EMA) and the US FDA for many therapeutic indications and has recently been approved as an adjuvant treatment for adults with melanoma with involvement of lymph nodes or metastatic disease who have undergone complete resection [5, 6]. Pembrolizumab, and dabrafenib, in combination with trametinib, have also been recently approved for adjuvant treatment for adults with stage III melanoma.

The phase III randomised controlled trial CheckMate-238 (NCT02388906) was conducted in patients with resected stage III or IV melanoma and investigated adjuvant nivolumab compared with ipilimumab [7]. The 24-month minimum follow-up data showed a significant RFS benefit for nivolumab compared with ipilimumab (hazard ratio (HR) 0.66; 95% confidence interval (CI) 0.54–0.81; p < 0.0001) [8]. The overall survival (OS) data in CheckMate-238 were too immature to report any results.

Recently, a network-meta analysis (NMA) was conducted combining randomised controlled trials in the adjuvant melanoma setting to produce one evidence network comparing nivolumab with other comparators: interferon, observation/placebo, bio-chemotherapy, pembrolizumab, and dabrafenib plus trametinib [9]. In addition to the NMA, indirect treatment comparisons (ITCs) comparing nivolumab with placebo alone were conducted [10].

However, information on the expected long-term impact of nivolumab in the adjuvant setting on both RFS and OS is lacking. Cost-effectiveness analysis models require the projection of outcomes for a patient’s lifetime. These outcomes can be hard to validate since the treatment landscape of later-stage melanoma has rapidly changed. The FDA alone has approved 12 different treatments in 16 indications for melanoma since 2011, and this situation is similar in other countries [11, 12].

A targeted literature review of adjuvant treatments submitted to the UK National Institute for Health and Care Excellence (NICE) found that the main topics discussed concerning difficulties in modelling in the adjuvant setting were the use of model structure to accurately model a disease pathway, immature data, validating post-recurrence survival due to changes in clinical pathways, and accounting for longer-term survivors within model projections [13].

This study presents the estimated survival of patients with melanoma with involvement of lymph nodes or metastatic disease who have undergone complete resection followed by adjuvant nivolumab or observation only. To explore the impact of model selection on predicted life-years in the adjuvant melanoma setting, we present alternative options for estimating OS and explore the difference each option makes to the extrapolated outcomes.

2 Methods

2.1 Data Sources

A de novo model was built to estimate the survival associated with nivolumab compared with observation in the absence of OS data from CheckMate-238. Alternative data sources were required because head-to-head trial data for nivolumab compared with observation were lacking and OS data from CheckMate-238 were unavailable. Data sources were identified through a systematic literature review of adjuvant melanoma studies, and additional searches were conducted for long-term melanoma data. These searches identified several data sources that were used to model the effectiveness of adjuvant nivolumab compared with observation (Table 1). A phase III randomised controlled trial comparing ipilimumab with placebo for patients with resected stage III melanoma, CA184-029 (NCT00636168), was used along with CheckMate-238 to form an ITC between nivolumab and placebo [14, 15]. As patient-level data were available for CA184-029, OS and post-recurrence survival (PRS) data were extracted from this trial and used as options to estimate nivolumab OS.

Table 1 Data sources used in the model
Full size table

Long-term melanoma data were required to either estimate long-term outcomes within the model or validate model projections. For this analysis, long-term melanoma datasets were limited to registry data and interferon studies [16,17,18].

2.2 Model Structure

The two modelling options were developed in Microsoft Excel® for predicting OS. Each model option used the same three-health-state structure comprising RFS, PRS and death (Fig. 1). For simplicity, recurrence type (i.e. local/regional or distant) was grouped into one health state. This was not expected to affect outcomes given that the proportions of patients with each type of recurrence were similar across treatment arms.

Fig. 1
figure 1

Model structure

Full size image

A lifetime horizon of 60 years was used in the base case to account for the age distribution in the CheckMate-238 and CA184-029 trials, which ranged from 18 to 86 years (median 54 and 51, respectively; Table 1), which were used to inform the age of patients in the model.

2.3 Recurrence-Free Survival

An ITC was performed between CheckMate-238 and CA184-029 to compare nivolumab with placebo, in which placebo outcomes were used as a proxy for observation. A patient-level meta-regression was performed between the two trials, which is considered the ‘gold-standard’ in population adjustment between studies and was the most robust ITC possible between nivolumab and placebo given the availability of patient-level data for both CheckMate 238 and CA184-029 [19]. In addition, this method does not rely on the proportional hazards assumption, which is not required when patient-level data are available [20]. Covariate-adjusted parametric curves were produced from the ITC, and these informed the proportion of patients who were recurrence free over time for each model option. The first assessment for recurrence in both CheckMate 238 and CA184-029 was at 12 weeks, therefore parametric survival curves were re-based at 12 weeks to improve the fit to the Kaplan–Meier (KM) data, and the KM data were used directly for the first 12 weeks. Other survival extrapolation techniques were considered (e.g. cure-based models [21, 22]), but, at the time of this analysis, the data from CheckMate 238 were considered immature to execute these techniques effectively, given the importance of the ‘cure-rate fraction’ on the extrapolated outcomes [23]. The log-logistic distribution was considered the most appropriate curve to extrapolate RFS based on guidance given in NICE technical support document 14 (Fig. 2) [20]. The process of curve selection is presented in the Appendix in the Electronic Supplementary Material (ESM). As neither CheckMate-238 nor CA184-029 covered the full patient population, patient characteristics in the model were taken from both trials covering all patients with lymph node involvement (stage III/IV). Survival curves for the full licensed patient population were then simulated using the corrected group prognosis (CGP) method [24]. The CGP method calculates a survival curve for each unique combination of covariates, which are then weighted based on the proportion of patients in each group. The weightings used to inform the final curves and groups within the CGP method are presented in the Appendix.

Fig. 2
figure 2

Indirect treatment comparison results—recurrence-free survival using log-logistic distribution. RFS recurrence-free survival

Full size image

In addition to the parametric survival curves, an adjusted indirect comparison between nivolumab and placebo was performed using the Bucher method [25].

2.4 Post-Recurrence Survival

2.4.1 Partitioned Survival Model: Surrogacy Model

The first model option used a partitioned survival model (PSM) structure in which PRS was calculated as the difference between OS and RFS. RFS for each treatment arm was taken directly from the patient-level meta-regression (Fig. 2). The estimated OS for observation used covariate-adjusted parametric survival curves fitted to the placebo arm in the CA184-029 trial. The CGP method was used to adjust the final curve to apply to the model population that was consistent with the approved indication. The OS for nivolumab was estimated using a surrogacy relationship between the RFS HR and OS HR derived from previous adjuvant melanoma studies including two recent trials [15, 26,27,28]. We assumed the proportional hazards assumption is appropriate between nivolumab and observation because this was a reasonable assumption between nivolumab and ipilimumab for RFS and between ipilimumab and placebo for RFS and OS in CA184-029. The process of curve selection is presented in the Appendix. This model option is referred to as ‘PSM surrogacy’.

2.4.2 State-Transition Model: Literature Post-Recurrence Survival Model

The second model option used a state-transition model (STM) and is referred to as ‘STM Literature PRS’. This structure was designed to capture the effect on survival of the subsequent metastatic melanoma therapies that have become available in the last few years. The proportions of patients in the recurrence-free health state in each treatment arm were taken from the patient-level meta-regression. The transition to either post-recurrence or death was split by the proportion of patients who had a recurrence or died after being recurrence free in the CheckMate-238 trial. PRS was split by type of recurrence: local/regional only or distant. The overall PRS curve was weighted by the proportion of patients who had a local/regional recurrence only (30.2%) or a distant recurrence at any point (69.8%) out of the patients who had a recurrence in CheckMate-238.

Survival of patients who had only experienced a local/regional recurrence used a generalised gamma curve fitted to the post local/regional recurrence data from CA184-029; this was considered appropriate because there were not many systemic treatment changes for patients who had a local/regional recurrence from the time of CA184-029 initiation. The survival of patients who had a distant recurrence at any point used a weighted survival curve from different treatment options for metastatic melanoma. A survival curve was produced for each metastatic treatment using clinical trial patient-level data from CheckMate-067 or reported outcomes from the literature. (CheckMate-067 was a phase III trial comparing the efficacy of nivolumab plus ipilimumab, nivolumab monotherapy, and ipilimumab monotherapy in previously untreated advanced melanoma). For other treatments, the efficacy was based on reported trial outcomes, and data sources were selected to keep the comparative efficacy between treatments clinically plausible. Further details of the sources used to estimate the survival of each treatment are presented in Table 3 in the ESM. This approach relies on the assumption that the different data sources are suitably comparable. Furthermore, it was assumed that the effect of each subsequent treatment was the same regardless of adjuvant therapy and that the patient populations from each trial were generalisable to our patient population.

The proportion of patients who received each treatment and informed the weighted distant recurrence curve was taken from CheckMate-238: for nivolumab, subsequent treatment data from the nivolumab arm were used and, for observation, subsequent treatment data from the ipilimumab arm were used as a proxy. These data are reported in the Appendix.

We also conducted a scenario using PRS from another adjuvant trial (CA184-029).

2.5 Long-Term Survival

Long-term data using the latest melanoma-specific registry data [17] were incorporated into the model as an option to inform long-term estimates at a certain cut-off.

Stage III data from the American Joint Committee on Cancer (AJCC) version 8 were digitised, and pseudo-patient-level data were created and fitted with a generalised gamma curve. To estimate long-term RFS, an HR was calculated from the long-term data in the adjuvant melanoma trial E1697 [18] comparing RFS and OS using a Cox regression model. The log-cumulative hazard plot showed that the proportional hazard assumption between RFS and OS was reasonable. This HR was then applied to the RFS curves after 10 years (HRRFS vs. OS: 1.98; 95% CI 1.63–2.41). Background general population mortality data were also used to ensure that the probability of death was never less than that of the general population [29].

2.6 Validation

The model projections were internally validated by comparing the projected curves to the clinical trial data (by changing the model patient characteristics to match the trial characteristics through the CGP method). For validation of the RFS meta-regression, the results of the Bucher indirect treatment comparison were compared with those of a treatment with a similar mechanism of action, using the phase III Keynote 054 trial, which investigated adjuvant pembrolizumab compared with placebo for patients with stage III melanoma [30]. External validation of the OS involved comparing the AJCC version 8 [17] and version 7 [16] long-term data with the extrapolated observation OS values and checking plausibility. Additionally, given the lack of relevant long-term data, clinical opinion was sought to verify the model predictions. Ten healthcare professionals with extensive experience treating patients with melanoma from different countries were shown the model-extrapolated outcomes. An STM was also developed that used the same inputs as the PSM surrogacy model to test the structural uncertainty between a PSM and STM structure.

3 Results

3.1 Recurrence-Free Survival

The RFS outcomes from the patient-level meta regression demonstrated that patients receiving adjuvant nivolumab were expected to have better RFS than patients receiving observation alone (Fig. 2).

A Bucher comparison was performed for nivolumab versus observation for RFS using the ipilimumab arms of the CheckMate-238 and CA184-029 trials. Nivolumab had a significant RFS benefit over observation (HR 0.54; 95% CI 0.41–0.69) [10]. This was similar to the results of the Keynote 054 trial, which showed a significant RFS benefit for pembrolizumab versus placebo (HR 0.57; 98.4% CI 0.43–0.74). The results of the patient-level meta-regression were also consistent with those of the Bucher ITC (HR 0.51; 95% CI 0.38–0.68 using the exponential curve for comparison).

3.2 Overall Survival

Table 2 and Fig. 3 present the OS projections for the different modelling options in undiscounted life-years. The extrapolated 60-year survival estimates produced from the trial data for observation were not consistent with survival outcomes from the long-term data sources. Ten-year OS from the CA184-029 placebo arm was estimated to be approximately 40%, compared with 69% in melanoma-specific registry data from the AJCC version 8 [17], 25–68% in AJCC version 7 [16] and 75% from trial E1697 for stage III melanoma [18]. Because the model produced lower survival estimates, long-term registry data were applied to each treatment arm after 10 years using the latest melanoma-specific registry data [17]. A 10-year period was used based on historical data, with different cut-offs tested in scenario analyses [16].

Table 2 Undiscounted life-year projections using different modelling options and time horizons
Full size table
Fig. 3
figure 3

Overall survival: different modelling options a PSM surrogacy, b STM literature. Nivo nivolumab, Obs observation, OS overall survival, PSM partitioned survival model, STM state-transition model

Full size image

Each of the different modelling options predicted different estimated survival for both nivolumab and observation; however, for both options, nivolumab was predicted to have at least a 32% increase in life-years gained compared with observation.

The model survival projections for observation were validated against the long-term melanoma registry data. The model population was adjusted to reflect the data sources for a like-to-like comparison. All the model options seemed to underpredict survival compared with registry data. Clinicians also agreed that the survival projections appeared pessimistic. The survival projections for observation from each model option were more in line with the AJCC version 7 data [16] than the version 8 data [17] but were still slightly lower than would be expected. Baseline characteristics were unavailable for the registry datasets; therefore, it is difficult to explain the differences in survival projections. AJCC version 8 [17] survival was very optimistic but may have considered patients who had adjuvant therapy, and the mean age of the patients is unknown. Plausible model projections would have been between the version 7 and 8 AJCC datasets, given that newer metastatic treatments should have improved survival since the 2009 dataset. The PSM and STM using the same data sources produced different long-term estimates, indicating that—in this case—structure had an impact on survival estimates (21.0 life-years for nivolumab and 14.0 life-years for observation vs. 18.4 life-years for nivolumab and 11.1 life-years for observation over the 60-year time horizon, respectively).

STM literature PRS produced different PRS between observation and nivolumab because of the different weightings of subsequent treatments used to inform the curves. Based on the data from CheckMate-238, a higher proportion of patients received palliative chemotherapy after a distant recurrence after treatment with nivolumab compared with those treated with ipilimumab (9.6 vs. 3.8%). Additionally, the ipilimumab arm had a higher usage of more effective subsequent treatments such as pembrolizumab (19.4%) and nivolumab monotherapy (12.3%) compared with the nivolumab arm (1.8 and 4.2%, respectively). Despite the lower PRS estimate, nivolumab was still predicted to lead to more life-years gained because of the longer RFS.

Furthermore, a proportion of patients in the CheckMate-238 trial did not receive any subsequent treatment after a distant recurrence (31.1% for nivolumab and 30.3% for ipilimumab). A scenario assuming that all patients have subsequent treatment after a distant recurrence was therefore explored in both treatment arms, but this had minimal impact on results, showing incremental life-years of 2.8. We also explored a scenario wherein the same PRS was assumed between treatment arms; this resulted in incremental life-years of 4.5.

The scenario using PRS data from CA184-029 showed an incremental difference of 4.9 life-yers between nivolumab and observation, which was higher than the STM literature PRS incremental difference (3.8 life-years). This was expected to be the most pessimistic option given that it used PRS from an older trial.

4 Discussion

A number of issues relating to adjuvant disease were encountered during this study; specifically, unavailability of OS data because of its immaturity, and lack of appropriate external long-term survival data to inform model survival projections. One of the key issues with the long-term extrapolation of outcomes in the adjuvant melanoma setting is generalisability to current practice due to the recent changes. Different modelling options were created to quantify the structural uncertainty around modelling of OS using the most relevant data sources.

The models brought together the most relevant efficacy data and used robust statistical techniques to establish the comparative efficacy of nivolumab and observation. The key limitation is the lack of availability of OS data; however, the model explored different survival outcomes using the range of data available when actual OS data were not available. The surrogacy relationship used in the PSM used a wealth of data from adjuvant trials, updated with data from recent trials, including CA184-029 and COMBI-AD. A limitation of this method is that the majority of trials that informed the surrogacy relationship included interferon studies, which may not reflect post-recurrence treatment with newer metastatic therapies. The STM literature PRS explored the effect of the changing melanoma pathway by using weighted curves informed by subsequent treatment usage. This allowed exploration of change to clinical practice based on adjuvant therapy. This option split patients by recurrence type (local/regional or distant), but a transition between patients having a local recurrence then a distant recurrence could not be accounted for in the three-health-state model, and these patients consequently had to be grouped into their latest recurrence group.

The mean long-term survival is predicted to lie within the range of 15–21 life-years for nivolumab, with an incremental benefit of 4–7 life-years gained compared with observation alone. Therefore, nivolumab is expected to have higher OS gains compared with observation irrespective of the modelling approach used. The predicted OS benefit for nivolumab is consistent with what has been demonstrated in the metastatic setting. Data from CheckMate-067 demonstrated that nivolumab led to longer OS than ipilimumab, with a progression-free survival HR of 0.55 (95% CI 0.45–0.66), translating to an OS benefit of 0.63 (98% CI 0.48–0.81). Patients in CheckMate-067 went on to receive further treatments upon progression, including re-treatment with immunotherapies (46% of patients on nivolumab received subsequent systemic therapy and 31% received subsequent immunotherapy; 63% of ipilimumab-treated patients received subsequent systemic therapy, and 44% received immunotherapy). This shows that early treatment with nivolumab is likely to provide greater OS benefit than treatment with ipilimumab [31].

It has also been shown in the metastatic setting that using techniques to model OS without OS data can reliably predict incremental survival; however, patient-level data were available to do this [32]. In this case, patient-level data were available for CA184-029 to produce a robust meta-regression for RFS with adjustments of patient characteristics.

Use of different modelling options helps test more fully the impact of uncertainty within the survival projections using different methods and data sources. The additional benefit of the STM approach in this case was the greater flexibility to test different scenarios and assumptions post-recurrence, so this could be considered the preferred option where multiple data sources are available to inform PRS.

All the different tested model options demonstrated that nivolumab was expected to have survival gains over observation, but the magnitude of this benefit is uncertain. Longer follow-up data from the CheckMate-238 trial are required to determine which of the model option projections was the most accurate. However, given the effectiveness of adjuvant therapy, OS data are unlikely to mature for a long time. Median OS from the models was between 4.5 and 5.0 years for observation and between 8.7 and 18.1 years for nivolumab (Table 2). Waiting for these data would result in delaying access for patients to life-changing treatments. Alternative ways of determining the value of adjuvant treatments, placing greater emphasis on RFS, should be considered, particularly in the long term in a setting in which the aim of treatment is to remain disease free. Moreover, because the clinical pathway for melanoma is continually evolving, the impact of this on OS estimates would need to be considered in any future analyses. In addition to newer therapies in the metastatic setting, the availability of adjuvant treatments in melanoma clinical practice may change the usage of subsequent treatments, further changing long-term patient outcomes.

Decisions on the reimbursement of therapies need to be made based on the available data. Waiting for further data from clinical trials does not always resolve the underlying uncertainty in projections, and it is not always realistic to wait for such data to mature. Thorough exploration of the uncertainty and understanding of the clinical assumptions behind modelling projections assists in appropriate decision making.