FormalPara Key Points

Health technology assessments of chimeric antigen receptor T cell therapies have been associated with a high uncertainty. Outcome-based payment arrangements (OBAs) aim to address uncertainty associated with the translation of trial outcomes to longer-term clinical outcomes. The extent to which OBAs alleviate uncertainty over the longer term is largely unknown.

There was a modest impact on reducing cost-effectiveness uncertainty under each OBA scenario compared with other sources of uncertainty in the model, whereas the potential financial uncertainty of an OBA was high. This emphasizes the need for careful consideration in making recommendations for OBAs, particularly given the resources and complexities associated with real-world data collection and payment arrangements.

If the main concern of governments and payors is the financial impact of high-cost therapies, then cost containment measures such as financial caps or price-volume agreements may be a more efficient approach to managing the high financial burden associated with cell and gene therapies compared with OBAs.

1 Introduction

Chimeric antigen receptor T cell (CAR-T) therapies were the first cell and gene therapies to undergo health technology assessment (HTA) globally. At the time of assessment, clinical evidence was from single-arm studies and data on their long-term benefits and safety were immature [1,2,3,4]. Evidence limitations, together with the high cost of a once-off treatment meant that HTA agencies grappled with a high level of decision uncertainty [1, 2, 5]. Consequently, public funding for CAR-T therapy was often conditional on Managed Access Programs (MAPs) [6].

MAPs have been proposed as potential solutions for addressing uncertainty where there is high clinical need but limited clinical evidence together with high upfront costs and consequently elevated financial risk [7,8,9,10,11,12,13,14]. A key recommendation from the National Institute of Health and Care Excellence's (NICE’s) assessment of methods of review, economic evaluation and appraisal of cell and gene therapies was the importance of innovative payment methodologies for technologies where there is both a high level of uncertainty and high expected patient benefits [15]. Outcome-based payment arrangements (OBAs) are a way of addressing clinical uncertainties and financial risk associated with cell and gene therapies [7, 10, 16,17,18,19,20], although their ability to address uncertainty in cost-effectiveness has not been assessed. OBAs link the net price of a medical technology to clinically relevant endpoints at the patient level [8, 14, 16, 21, 22] (the taxonomy of OBAs in relation to MAPs has been described extensively [8, 11, 14]).

A review of HTAs of CAR-T therapies globally showed substantial variation in modeled, long-term benefit, particularly in young patients [6]. This variation was attributed to limited clinical evidence and the application of different extrapolation approaches beyond the observed period in the trials [6]. OBAs are tasked with addressing uncertainty over the longer term. But the extent to which this occurs in practice, either for cost-effectiveness or total financial exposure, remains largely unknown. The main objective of this research was to assess the value of OBAs in dealing with uncertainty in cost-effectiveness, by linking complete remission (CR) to survival in young patients with acute lymphoblastic leukemia (ALL) treated with tisagenlecleucel. This will assist in determining whether decisions to publicly fund cell and gene therapies should be conditional on OBAs. CR was the outcome of interest because it is a short-term objective outcome, allowing an OBA to be enacted over a reasonable timeframe, and CR has been linked to improved survival [23]. The impact of the timing of outcomes assessment and distribution of payments on cost-effectiveness was also assessed.

2 Methods

2.1 Study Setting and Patient Population

Australia was the setting for the economic analysis from a healthcare system perspective. The population was children and young adults (3–23 years of age) with relapsed or refractory ALL, based on evidence for tisagenlecleucel from two phase II, single-arm, multicenter studies, ELIANA [24] and ENSIGN [25].

2.2 Intervention and Comparator

The CAR-T therapy of interest was tisagenlecleucel. The intervention pathway commenced with the intention to treat with tisagenlecleucel, from the point of leukapheresis, followed by lymphodepleting chemotherapy before receiving a single infusion of tisagenlecleucel [26]. In accordance with the clinical trials, a small proportion of patients did not proceed to infusion due to an adverse event (AE), disease progression or death [24]. The comparator was blinatumomab, the standard of care in Australia prior to tisagenlecleucel being available and as accepted by the Australian Medical Services Advisory Committee (MSAC) in its consideration of tisagenlecleucel [27]. Blinatumomab is administered as a continuous infusion over a 28-day treatment cycle, and patients receive up to five cycles of treatment [28]. Evidence for blinatumomab was from a single-arm phase I/II study in young patients with relapsed/refractory ALL [29]. Another immunotherapy, inotuzumab ozogamicin, is also funded in Australia for ALL, for use after blinatumomab, although it was not considered a relevant comparator because it is not registered for use in children (< 18 years) [30].

2.3 Model Structure

The model contained treatment specific structures. For tisagenlecleucel, a partitioned survival model (PSM) was preceded by a decision tree to model the pathway from leukapheresis to assessment of response, similar to previous approaches [31,32,33] (Fig. 1). Patients infused with tisagenlecleucel were assessed for response at 3 or 12 months (reflecting possible OBA approaches).

Fig. 1
figure 1

Decision-tree and partitioned survival model structure. OS overall survival, PFS progression-free survival, SCT stem cell transplant, t time

Full size image

Response was defined as a CR or CR with incomplete blood count (CRi), consistent with the primary outcome of the tisagenlecleucel clinical studies [24, 34]. Patients who did not achieve a CR/CRi or were lost to follow-up were considered non-responders. Because tisagenlecleucel may be used as a bridge to allogeneic stem cell transplant (SCT), responders were separated according to whether they received subsequent SCT or no SCT. SCT was considered separately in the model due to its high cost and the uncertainty around the proportion of patients who will receive an SCT following tisagenlecleucel. Patients who died before a response assessment were assigned costs and outcomes for the preceding time period. Patients unable to receive an infusion due to an AE or disease progression proceeded to treatment with the comparator and were assigned the same costs and outcomes as blinatumomab in addition to associated pre-infusion costs for tisagenlecleucel.

Following the decision tree, patients moved to a PSM corresponding to the outcome achieved at the assessment point: response (no SCT), response (with SCT) or no response. Patients who achieved a response could either remain progression-free, progress or die, with the proportion of progression-free patients derived directly from the event-free survival (EFS) curve, the proportion dead as 1 minus the overall survival (OS) curve, and the proportion progressed as the difference between the OS and EFS curves [35]. It was assumed that non-responders had progressive disease, and therefore could only progress or die. The results for the progression-free, progressed or dead in each sub-group were combined to calculate the total costs and benefits of the entire cohort.

Treatment initiation with blinatumomab was considered from the point of infusion; hence, the entire patient cohort was modeled using a single PSM consisting of three health states—progression-free, progressed and dead—without considering response status (as there is no known OBA associated with public funding for blinatumomab on the Pharmaceutical Benefits Scheme [PBS]). A 1-month cycle length with half-cycle correction was applied, and costs and benefits were measured over a lifetime horizon due to the potential curative benefit of tisagenlecleucel. Costs and outcomes were discounted at a rate of 5%, as recommended by MSAC guidelines [36] (Table 1).

Table 1 Key parameter summary
Full size table

2.4 Outcome Measures

Modeled benefits were measured in life years (LYs) and quality-adjusted life years (QALYs) derived from  OS and EFS. EFS was measured from the time of first tisagenlecleucel infusion to relapse, death due to any cause or treatment failure. OS was measured from the time of first tisagenlecleucel infusion to the time of death due to any cause. For blinatumomab, OS was measured from the start of treatment until death or date of last evaluation for all treated patients. EFS was not reported in the blinatumomab clinical study [29]. EFS for blinatumomab was estimated assuming a constant cumulative hazard ratio (HR) of 0.83 between OS and EFS over time, based on the relationship observed between OS and EFS Kaplan-Meier (KM) curves from a published study of mitoxantrone in pediatric ALL [23]. The impact of this assumption was tested in sensitivity analyses using the lower and upper range of HRs for EFS:OS from the mitoxantrone study [23] (Table 1).

2.5 Data Analysis

Visual inspection of the KM curves for OS for the ELIANA and ENSIGN studies showed the curves to be similar (Figure S1, see the electronic supplementary material), without statistically significant difference in OS between the studies (HR 0.62, 95% confidence interval [CI] 0.36–1.0; P = 0.098), allowing data from the studies to be pooled. OS was grouped according to response status; non-responders were patients not in CR/CRi at the specified time point and included patients who had never achieved a CR/CRi or had achieved a CR/CRi prior to the specified time point but had subsequently relapsed (Fig. 2a). The EFS analysis only included patients who achieved a CR/CRi at the specified time point (Fig. 2b).

Fig. 2
figure 2

Kaplan–Meier analysis of tisagenlecleucel for (A) overall survival by responders and non-responders and (B) event-free survival for responders

Full size image

The approach to the sub-group analysis by response for EFS followed the same approach to that described for OS. Analyses used data censored for SCT to allow for sensitivity analysis on different rates of subsequent SCT post infusion with tisagenlecleucel. Data for patients who received SCT following infusion with tisagenlecleucel were not analyzed separately due to small patient numbers, and SCT censored data were applied to this patient group.

A 30-day window either side of each response time point was included to reflect the variability in timing of response assessments in clinical practice. In the studies, clinical assessments were conducted monthly for the first 6 months; consequently, at the 3-month time point (61- to 122-day period), there was potential for duplicate patient entries as patients could have been assessed up to three times. To eliminate duplicate patient entries, only the last assessment during the 61- to 122-day period was included. A summary of the number of patients in each sub-group and by time point relative to the overall cohort is presented in supplementary Table S1.

For blinatumomab, time-to-event data were reconstructed from the published KM OS curve [29] using the software DigitizeIt. Pseudo-patient level data were generated using the approach by Guyot et al. (2012) [37] and KM analysis performed following the methods described by Wei and Royston (2017) [38]. The statistical software packages R [44], RStudio [39] and STATA [40] were used for the analysis of individual patient data (IPD) and generating KM data.

2.6 Extrapolation of Time-to-Event Data

In selecting the most appropriate approach to extrapolating survival data, duration of patient follow-up, completeness of the data, and level of censoring of the observed data were taken into account [35]. HTA agencies generally recommend the use of parametric models over Cox proportional hazard regression models where data are immature [35, 41]. Additionally, because the model used sub-group data for tisagenlecleucel, an assumption of proportional hazards for the overall population was not appropriate. Instead, independent parametric models were fitted to each patient group in the model and the survival estimates weighted by the proportion of patients in each group (Figures S3 and S4, see the electronic supplementary material). Although not used in the extrapolation, the HR and CI for OS of tisagenlecleucel versus blinatumomab were derived using a Cox proportional hazard regression model, showing a significant improvement in OS for tisagenlecleucel for the overall cohort (supplementary text).

Selection of the parametric model was based on whether the model was statistically a good fit according to the Akaike information criterion (AIC) and Bayesian information criterion (BIC), and also whether the extrapolated portion was clinically and biologically plausible [42] (Table 1). Extrapolations were applied from the point on the KM curve where patient numbers were small (< 12 patients) due to a high level of censoring [43] (Table 1). EFS curves were constrained by the OS curve, so that EFS did not exceed OS at any time point. For blinatumomab, the extrapolated EFS curve was based on the HR of EFS:OS as described above.

2.7 Long-Term Survival

Extrapolation of the observed data using parametric models continued until end of year 4 (“cure point”), after which a standardized mortality ratio (SMR) was applied to age-adjusted all-cause mortality from Australian life tables [44]. An SMR of 9.05 was used based on a Canadian cohort study in childhood cancer patients who had survived at least 5 years [45]. SMR adjusted all-cause mortality was applied to both OS and EFS from year 5 onwards based on the observed relationship between OS and EFS [39], leading to a convergence of EFS and OS over time.

2.8 Utility Values

Utility values for the PD and EFS health states of the model were calculated from patient-level EQ-5D data from the ELIANA study using UK preference weights [46] and applied to both arms of the model due to the absence of published EQ-5D data for blinatumomab. Additionally, a general disutility associated with serious adverse events (SAEs), sourced from an economic analysis in chronic lymphocytic leukaemia [47], was applied to each arm, weighted by the duration and frequency of SAEs from the clinical studies [24, 29] with the exception of grade 3/4 cytokine release syndrome (CRS), where a higher disutility was assumed due to its potentially high severity. An additional disutility was also applied to the proportion of patients who went on to receive an SCT, assuming a duration of disutility of 1 year [48]. The utility and disutility values are summarized in Table 1, with further information on the calculation of the disutilities in Table S2 (see the electronic supplementary material).

2.9 Resource Use and Costs

Cost inputs sourced from prior publications were adjusted for inflation using the Reserve Bank of Australia’s (RBA’s) inflation calculator [49], and when sourced from international publications, converted to Australian dollars using RBA exchange rates [50]. For the purposes of publication, costs are reported in US dollars [50].

The base case assumed a single upfront payment of $375,000 for tisagenlecleucel based on the NICE published price [4]. The cost of blinatumomab was calculated from the Australian PBS price [51] using an average number of treatment cycles from the clinical study [29], noting that the net price may be lower due to confidential pricing arrangements (Table 1; Table S4 in the electronic supplementary material). Ancillary costs associated with the administration of each treatment included costs of infusion, length of hospital stay and management of SAEs including use of tocilizumab for CRS and intravenous immunoglobulin (IVIg) for B cell aplasia (supplementary Tables S3 and S5). For tisagenlecleucel, administrative costs included leukapheresis and bridging chemotherapy (supplementary Table S3). Subsequent SCT costs for tisagenlecleucel and blinatumomab were estimated from a local costing study [52]. Ancillary costs were calculated by multiplying the percentage utilization of each ancillary service from ELIANA [24] by the cost for each service estimated from Australian-relevant cost data (supplementary Tables S3 and S5). Other disease management costs associated with EFS and PD were applied using an average cost from a cost-effectiveness analysis for pixantrone in adult non-Hodgkin lymphoma [53]. Resource use and costs for tisagenlecleucel and blinatumomab are summarized in supplementary Tables S3–S6.

2.9.1 OBA Scenario Analyses

Two different response-based payment structures were considered: (1) split payment; payment 1 on infusion and payment 2 on response, or a smaller payment where patients could not be assessed for response (lost to follow-up); (2) single payment on response only. The amount per payment was weighted by the proportion of responders and non-responders to equal a weighted price of $375,000 for each scenario (Table 1). An equal weighted price was maintained across each OBA scenario because the purpose of the analysis was to test the impact of different payment structures, as opposed to the impact of a lower net price. To assess the impact of the OBA scenarios on the incremental cost-effectiveness ratio (ICER), response rate was varied by ± 20% and rates of non-response, dead and lost to follow-up were varied proportionally (Table 1). Additionally, the differential cost of each scenario compared with the base case was calculated assuming a population size of 100 patients, to assess the budget impact of each payment structure.

2.9.2 Sensitivity Analyses

Variables tested in sensitivity analyses were those that had previously been identified to have a substantial impact on cost-effectiveness of tisagenlecleucel in ALL [6, 54, 55]. These included discount rate, type of parametric extrapolation, long-term survival, SCT rate, infusion rate and IVIg use, as well as response rate (Table 1).

3 Results

3.1 Base Case

Applying a single, upfront payment of $375,000 for tisagenlecleucel, resulted in a total cost of $585,890, 7.13 LYs and 5.36 QALYs. The corresponding results for blinatumomab were costs of $206,294, 1.80 LYs and 1.09 QALYs. This gave an incremental cost of $379,595 and incremental benefit of 5.32 and 4.27 for LYs gained and QALYs gained, respectively. The ICER was $71,318 per LY gained and $88,979 per QALY gained over a patient’s lifetime (Table 2). Results were very similar when assessed using the 12-month response rate for tisagenlecleucel; $90,129 per QALY gained over a patient’s lifetime (Table 2). The 12-month analysis is not discussed further due to the similarity in results to those of the 3-month analysis.

Table 2 Base-case results
Full size table

3.2 OBA Scenario Analyses

A change of ± 20% in response rate had different impacts on the ICER depending on the type of payment structure applied (Figs. 3, 4; Table S7 in the electronic supplementary material). For the split payment scenario, the ICER range was $82,650–$99,154 per QALY gained. There was only minor variation in the ICER for the responder-only payment scenario, from $88,249 to $89,434 per QALY gained. For comparison, variation in the ICER was greatest with a single upfront payment (base case) ranging from $77,599–$107,273 per QALY. The reverse effect was seen when the financial impact was calculated for each scenario; the greatest variability in differential lifetime financial cost was associated with a responder only payment (± $9,082,757), followed by a split payment arrangement (± 5,514,492), with the least variation (± $2,857,757) associated with the base-case single upfront payment when the response rate was varied ± 20% (Fig. 4).

Fig. 3
figure 3

Tornado diagram of cost-effectiveness for OBA scenarios (boxed) compared with other sensitivity analyses. EFS event-free survival, HR hazard ratio, ICER incremental cost-effectiveness ratio, IVIg intravenous immunoglobulin, OBA outcome-based payment arrangement, OS overall survival, QALY quality-adjusted life year, RR response rate, SCT stem cell transplant, SMR standardized mortality ratio, yrs years

Full size image
Fig. 4
figure 4

Change in financial impact and cost-effectiveness (incremental cost per QALY) under varying RRs for the base case and each OBA scenario. Financial impact was calculated as the difference from base case in lifetime incremental cost of tisagenlecleucel versus blinatumomab for 100 patients. OBA outcome-based payment arrangement, QALY quality-adjusted life year, RR response rate

Full size image

3.3 Sensitivity Analyses

The ICER was most sensitive to discount rate, “cure point” and type of parametric extrapolation (Fig. 3). A discount rate of 1.5% had the biggest impact on improving cost-effectiveness, resulting in a decrease in the ICER to $57,660 per QALY. Conversely, extending the cure point to 10 years had the biggest impact on reducing cost-effectiveness by increasing the ICER to $116,206 per QALY. The type of parametric extrapolation also had a substantial impact on the ICER, varying from $76,018 per QALY with an exponential equation to $94,049 per QALY with Gompertz. An assumption of lifetime IVIg duration and an increase in SCTs post infusion with tisagenlecleucel increased the ICER considerably ($105,351 and $95,509 per QALY, respectively). Other variables tested in sensitivity analyses had a relatively minor impact.

4 Discussion

The base-case ICER ($88,979 per QALY) was within the range of cost-effectiveness previously considered by MSAC in their original evaluation of tisagenlecleucel for ALL ($69,280–$98,450 per QALY [92,373–131,267 Australian dollars]) [6]. Although the base-case ICER was the same under each OBA scenario, each payment structure had a different impact on cost-effectiveness when response rates were varied, which is what the analysis sought to test. The responder-only payment was effective in maintaining a consistent ICER, although under a split payment arrangement, the ICER changed by − 7.1 to 11.4%. The biggest variation in the ICER occurred using a single upfront payment when response rates varied (− 12.8% to 20.6% change).

The responder-only OBA was effective at maintaining a consistent ICER because under this arrangement the cost of tisagenlecleucel was directly linked to the benefit, so that when response rates altered, costs altered proportionately. Although intuitive, this requires a direct relationship between response and longer-term survival, which this analysis demonstrates. Furthermore, a responder-only OBA resulted in high overall financial variability which may, in the eyes of the payor offset the benefit of a responder-only OBA in terms of maintaining cost-effectiveness.

Considering the high cost of cell and gene therapies, government may consider financial certainty more important than maintaining cost-effectiveness, particularly under a pay-for-responder arrangement which could see total financial costs increase substantially. Other considerations for a pay-for-responder arrangement include the reluctance of manufacturers to enter into an agreement where payment is entirely contingent on response and there is no recovery of costs for patients who do not respond.

In this analysis, the price per responder was determined by the proportion of patients in response in the clinical trials, to give an overall weighted price of $375,000. This meant that for the responder-only OBA, there was no cost for patients who did not achieve a response, but the cost was inflated for patients who responded (Table 1). Consequently, when response rates varied, cost-effectiveness, set at the response rate seen in the clinical trials, was maintained. This approach does not adjust for a change in cost-effective price where response rates vary, highlighting the importance in setting the weighted price at the point of the initial cost-effectiveness assessment.

Not surprisingly, cure point was also a key driver of cost-effectiveness, as this determines the point at which the extrapolation of the survival curve is informed by disease-adjusted general population mortality. Interestingly, cure point and the type of parametric extrapolation were more important drivers of cost-effectiveness than response rate, highlighting the limitations of a surrogate outcome as the basis for an OBA.

The assessment of response to tisagenlecleucel at different time points had little impact on cost-effectiveness. Incremental costs, QALYs and the resulting ICERs were similar regardless of whether patients were assessed for response at 3 or 12 months post infusion. Patients who were assessed at 3 months experienced higher rates of progression and mortality compared with patients who remained in response at 12 months, although this was offset by improved survival for the non-responder group at 3 months compared with 12 months. Therefore, the time point for response assessment for an OBA may not be important, where the overall price is weighted by response rate.

To our knowledge, this is the first analysis to link response to survival to assess the impact of an OBA on both costs and benefits in an analysis of cost-effectiveness of tisagenlecleucel in ALL. The Institute of Clinical and Economic Review in their assessment of effectiveness and value of CAR-T therapies considered an option for an OBA, which was replicated in subsequent publications, although was only considered in relation to the distribution of payments and did not link response status to survival [31,32,33]. Other studies have looked at the impact on the cost-effectiveness of tisagenlecleucel of a response-only payment using clinical remission, although none accounted for any associated change in benefit [54,55,56,57,58].

A recent review of OBAs highlighted an increase in their use in Australia, Italy, Sweden and the USA, noting that publicly available information was limited and, perhaps more importantly, that there was even less information on whether the objectives of these OBAs, in terms of realizing value for money, had been achieved [59].

Accelerated registration pathways have created a dilemma for HTA agencies that rely on value-based assessments for public funding decisions, as conditional registrations do not require the level of evidence preferred by HTA agencies [8]. Arguably, this has led to an increased interest in OBAs. If OBAs are to be used as a tool for addressing uncertainty in HTA, then their utility in addressing clinical and economic uncertainty should be weighed against the costs and resources required to administer an OBA at the point of assessment of cost-effectiveness, and importantly their financial impact. Although the administrative costs of OBAs were not explored as part of this research, previous studies have highlighted substantial costs associated with their implementation [60, 61]. Decision makers may wish to consider alternative mechanisms of dealing with cost-effectiveness uncertainty, such as adopting conservative modeling assumptions, a higher cost-effectiveness threshold or lower levels of discounting. Cost containment could be achieved through financial caps or price-volume agreements that are easier to administer and do not rely on the collection of outcomes data.

4.1 Limitations

A limitation of the analysis was the small patient numbers by sub-group of the tisagenlecleucel data, which may reduce the reliability of extrapolation of the data beyond the observed period. Analysis of the data by sub-group required access to IPD, which means that application of this approach is likely to be limited to sponsor-led analyses. An inherent limitation was the lack of long-term, comparative data, although the included studies were considered the best sources of evidence by HTA agencies [6]. Although the analysis was limited by the level of evidence available, this is the issue that we sought to address and is one faced by decision makers on a daily basis.

5 Conclusion

In this analysis, a pay-for-responder arrangement was effective at maintaining a consistent ICER, although overall financial uncertainty was high. Compared with other variables in the model, a split-payment OBA had a modest impact on reducing uncertainty in cost-effectiveness. Although the greatest variability in cost-effectiveness was seen with a conventional single, upfront payment, this also resulted in the least financial variability. These results suggest that the value of implementing OBAs should be weighed against the additional resources needed to administer such arrangements, and importantly the overall cost to government of the relevant technology. This analysis used a weighted pricing approach so that the overall price for each OBA was equivalent to the base-case single, upfront price at the response rates seen in the clinical trials. However, price is not readjusted for cost-effectiveness at different response rates, highlighting the importance in determining the weighted price at the point of initial cost-effectiveness assessment. Further work could consider measuring the impact of OBAs retrospectively to evaluate whether objectives, in terms of ensuring value for money, were met.