FormalPara Key Points for Decision Makers

The minimal modeling approach to estimating real option value (ROV) does not require constructing a formal, full cost-effectiveness model, and evaluates the impact of uncertainty qualitatively.

ROV may interact with the conventional value, the value of hope, productivity effects, and insurance value, and the impact of ROV on cost effectiveness can be evaluated via threshold analysis.

The 15-item checklist can standardize the reporting of ROV in value assessment.

1 Introduction

There has been significant progress in recent years on defining and measuring the real option value (ROV) of health technologies [1, 2]. Valuing this aspect of innovation is important because it supports a reward system that incentivizes the magnitude and types of innovation. In the words of the Professional Society for Health Economics and Outcomes Research (ISPOR) Special Task Force on US Value Frameworks, “… it is critical to investigate these value frameworks because of the signals they send to innovators. Value-based approaches can encourage firms to produce more of what is being optimized in the frameworks, and discourage them from bringing to market products that do not produce good value” [3].

ROV, which is not routinely measured in conventional cost-effectiveness analysis (CEA), may be created when current treatment (1) increases or decreases a patient’s chance of accessing future innovations by affecting survival; (2) expands or restricts a patient’s eligibility for future innovations by affecting quality of life; or (3) enhances or attenuates efficacy of future innovations through interaction of mechanisms of action [4]. Thus far in value assessment, two methods have been developed to quantify the ex ante ROV of health technologies and its uncertainty. The first method leverages drug pipeline data to forecast future approvals and their effects on survival and quality of life, and the second method uses patient registry data to directly forecast future mortality trends [5,6,7,8]. Case studies in metastatic melanoma, non-small cell lung cancer, renal cell carcinoma, and chronic myeloid leukemia found that the ex ante ROV of several targeted therapies and immunotherapies ranges from 5 to 18% of their conventional value (i.e., the incremental life-years gained or incremental quality-adjusted life-years (QALYs) gained) [5,6,7,8]. In addition to ex ante modeling, there is also empirical evidence of ROV, resulting in tangible survival gain and playing a role in treatment decision making in metastatic cancer [9,10,11].

The growing body of research on ROV has drawn interest from stakeholders to assess the implications of incorporating ROV into the health technology assessment (HTA) of new medicines [12]. To date, empirical work on ROV has mostly been limited to life extension in a few metastatic cancers. The impact of other types of ROV on value outside of life extension, and the methods and data required for quantifying such ROV, are under development. Furthermore, little attention has been paid to the costs of the future innovation or potential interactions between ROV and other elements of value. These limitations have created the impression that ROV only adds value to treatments and that ROV cannot be estimated without significant risk of double-counting, which is a common argument for not including ROV as a quantitative element in value assessment [13].

In this paper, we first present a theoretical framework for ROV that delineates how it is created, its types, and its estimation. We describe how parameters for estimating ROV can be derived. Then, based on that framework, we developed a ‘minimal modeling’ approach for estimating ROV that does not require constructing a full, formal cost-effectiveness model. We examine the potential impact of ROV on the incremental cost-effectiveness ratio (ICER) as well as the potential interactions between ROV and other elements of value. Finally, we developed a checklist for reporting ROV for inclusion in value assessment. We conducted an empirical exercise using the minimal modeling approach and report the findings using the checklist.

2 Theoretical Framework

For a simple illustration, we use \({S}_{j}(t)\) to represent the survival function under treatment j, and \({w}_{j}(t)\) to represent the utility weight of the health state on treatment j. Health state utility is a function of time, as disease may progress over time, leading to a varying utility value. \(T\) represents the maximum remaining life expectancy of a representative patient. The conventional incremental QALYs of current treatment \(trt\) relative to control \(ctrt\) is:

$$\Delta QALY_{trt} = \mathop \int \limits_{0}^{T} w_{trt} \left( t \right)S_{trt} \left( t \right){\text{d}}t - \mathop \int \limits_{0}^{T} w_{ctrt} \left( t \right)S_{ctrt} \left( t \right){\text{d}}t.$$

The integral of a survival function is the area under the survival curve, which is also the mean survival time. Survival functions of most new medical technologies are derived from pivotal trials where future innovations are not incorporated. As a result, the incremental QALYs calculated conventionally using efficacy from pivotal trials do not usually include ROV.

Assume a future innovation \(future\) is expected to arrive at time \({t}_{future}\) with probability \(p\). The ROV is the incremental value of \(trt\) enabling patients to benefit from \(future\):

$$\begin{aligned} \Delta ROV & = \left[ {S_{trt} \left( {t_{future} } \right)u_{trt} - S_{ctrt} \left( {t_{future} } \right)u_{ctrt} } \right] \times p \times \left[ {\mathop \int \limits_{0}^{{T - t_{future} }} w_{future} \left( t \right)S_{future} \left( t \right){\text{d}}t - \mathop \int \limits_{0}^{{T - t_{future} }} w_{cfuture} \left( t \right)S_{cfuture} \left( t \right){\text{d}}t} \right] \\ & = \left[ {S_{trt} \left( {t_{future} } \right)u_{trt} - S_{ctrt} \left( {t_{future} } \right)u_{ctrt} } \right] \times p \times \Delta QALY_{future} , \\ \end{aligned}$$

where \({S}_{j}\left(t\right)\) is the proportion of patients alive at time \(t\) under treatment \(j\), and \({u}_{j}\) is the percentage of survivors under treatment \(j\) who are eligible for the future innovation when it arrives. The reason for including \({u}_{j}\) is because when future innovation arrives, not all survivors may be eligible for it. For instance, some patients’ disease may have progressed for a long time, making them ineligible to receive additional lines of treatment. \({S}_{trt}\left({t}_{future}\right){u}_{trt}-{S}_{ctrt}\left({t}_{future}\right){u}_{ctrt}\) is therefore the difference in the proportion of patients who can receive future innovation when it arrives between \(trt\) and \(ctrt\). \({\int }_{0}^{T-{t}_{future}}{{w}_{future}\left(t\right)S}_{future}(t){\text{d}}t-{\int }_{0}^{T-{t}_{future}}{{w}_{cfuture}\left(t\right)S}_{cfuture}(t){\text{d}}t\) is the difference in the utility-weighted area under the survival curve between future innovation and the alternative where there is no technology advancement. Like the conventional value, ROV is also an incremental value. Another useful metric for measuring ROV is expressed as a percentage of the conventional incremental QALYs gained:

$$\frac{\Delta ROV}{{\Delta QALY_{trt} }} = \left[ {S_{trt} \left( {t_{future} } \right)u_{trt} - S_{ctrt} \left( {t_{future} } \right)u_{ctrt} } \right] \times p \times \frac{{\Delta QALY_{future} }}{{\Delta QALY_{trt} }}.$$

As reflected in these equations, ROV can be created through life extension (i.e., \({S}_{trt}>{S}_{ctrt}\)), improved eligibility for future innovations conditional upon survival (i.e., \({u}_{trt}>{u}_{ctrt}\)), or both. Examples of ROV from life extension include many cancer treatments, where longer survival on current treatment means patients have a greater chance of living to see the arrival of future innovations. Improved eligibility for future innovations can be through current treatment slowing down disease progression or interaction of mechanisms of action between current treatment future innovations. Treatment for early-stage Alzheimer’s disease is an example of delayed disease progression keeping patients eligible for future innovations for mild cognitive impairment. Antibiotic resistance is an example where current treatment precludes patients from using future innovations from the same class, resulting in negative ROV. Some treatments may create multiple types of ROV. For instance, antibiotics can create positive ROV through life extension, as well as negative ROV through drug resistance.

3 Quantifying Real Option Value (ROV): Estimation of Parameters

3.1 Estimating \({\mathbf{t}}_{\mathbf{f}\mathbf{u}\mathbf{t}\mathbf{u}\mathbf{r}\mathbf{e}}\), the Expected Time to Approval of Future Innovation

This can be achieved by a search of the ClinicalTrials.gov database for investigational new treatments in clinical development for the target population [5, 8]. Time of potential US FDA approval is calculated by adding the average length of drug development time (length of each phase of a clinical trial and FDA review time) to the start date of the pivotal trial, which is documented on ClinicalTrials.gov. \({t}_{future}\)—expected time to approval—is then calculated by taking the difference between the expected time of approval and time of ROV analysis. Wong et al. analyzed 406,308 entries of clinical trial data for over 21,143 compounds from 2000 to 2015 and estimated that the median clinical trial durations were 1.6, 2.9, and 3.8 years, for trials in phases I, II, and III, respectively [14]. Other studies also estimated the length of new drug research and development (R&D), using different samples of drugs from different time frames [15, 16]. When possible, analysts should select the most recent estimates from the same disease and therapeutic class.

3.2 Estimating \(p\), the Probability of Approval

The probability of approval \(p\) can be derived using estimates from the literature on phase transition probabilities of new drug development. There are several studies that provided estimates on these statistics. In the study by Wong et al. mentioned above, it is estimated that the phase transition probabilities from phase I to phase II, from phase II to phase III, and from phase III to FDA approval was 38.8%, 38.2%, and 59.0%, respectively [14]. In another study that included investigational cancer drugs from 1990 to 2005, drugs that had a response rate of ≤13.8% in phase II had a 2.5% probability of being approved by the FDA, while those that had a response rate of >13.8% in phase II had a 77% probability of being approved [17]. Ideally, analysts should use the most recent estimates from the same disease and therapeutic class when possible.

3.3 Estimating \({t}_{future}\) and \(\mathbf{p}\) when there are multiple Investigational New Treatments

Oftentimes, there are multiple investigational new treatments under development for a specific disease at the time of ROV analysis. If that is the case, the probability of any approval is 1 minus no approval. For example, if there is one new treatment in phase II and one new treatment in phase III, using the estimates from Wong et al. [14], the probability of the treatment in phase II obtaining approved is approximately 28.8%, and the probability of the treatment in phase III being approved is approximately 59.0%. The probability of any approval is therefore 1 – [(1–28.8%)*(1–59.0%)] = 70.8%. Average time to approval among multiple investigational compounds can be calculated by weighting time to approval for each compound by its expected probability of approval. For example, if the treatment in phase II is expected to be approved in 40 months from the ROV analysis and the treatment in phase III is expected to be approved in 20 months, the weighted average for time to approval is approximately [20*59.0%+40*28.8%*(1-59.0%)]/70.8%=18.3 months.

3.4 Estimating \({S}_{j}\left({t}_{future}\right)\), the Proportion of Patients Alive at the Time of Approval of Future Innovation

With \({t}_{future}\), the proportions of patients alive when future innovation is expected to arrive can be read directly from the Kaplan–Meier curves from the pivotal trials.

3.5 Estimating \({u}_{j}\), the Proportion of Surviving Patients Eligible for Future Innovation

Several factors may influence the proportion of surviving patients eligible for future innovation. For example, some innovations only work for subsets of patients with specific genetic mutations. Some innovations are only recommended for patients who remain progression-free or have progressed recently. Therefore, estimating \({u}_{j}\) should be on a case-by-case basis. Opinions from domain experts can be used and/or market research on the potential uptake of future innovation can be used to help triangulate an estimate of \({u}_{j}\) [8].

3.6 Estimating \({\Delta QALY}_{future}\), the Incremental Quality-Adjusted Life-Years of Future Innovation

Projecting the incremental QALYs of future innovations requires data on their expected efficacy. At the time of an ROV analysis, future innovation has not yet been approved, but preliminary data on its efficacy are usually available from early-stage trials, interim analyses, etc. [18]. If the survival function and time in each health state on future innovation are known, the expected incremental QALYs of future innovation can be projected in a disease progression model such as a Markov state transition model.

However, early-stage efficacy data may lack important details, such as the shape of the survival curve, for constructing a disease progression model. For example, only median survival or response rate may be reported in the press release of the phase II result of an investigational cancer drug. Moreover, analysts may wish to quickly assess the potential size of ROV without constructing a full, formal CEA, in order to determine if ROV will be formally integrated into the HTA. In the following section, we develop a ‘minimal modeling’ approach that uses only a few parameters to project an approximation of the potential size of ROV.

4 Minimal Modeling Approach

Based on the theoretical framework, we know that:

$$\Delta ROV = \left[ {S_{trt} \left( {t_{future} } \right)u_{trt} - S_{ctrt} \left( {t_{future} } \right)u_{ctrt} } \right] \times p \times \Delta QALY_{future} .$$

Estimation of \({t}_{future}\), \(p\), \({S}_{j}\left({t}_{future}\right)\), and \({u}_{j}\) is described above. The value of \(\Delta {QALY}_{future}\) can be approximated by assuming that survival function follows the exponential distribution where:

$$median=mean\times ln(2)$$

In a simple oncology example where survival can be categorized into two health states, i.e. preprogression (PP) and progressive disease (PD), \(\Delta {QALY}_{future}\) can be rewritten as:

$$\begin{aligned} \Delta QALY_{future} & = \left( {med_{PP, future} w_{PP} + med_{PD, future} w_{PD} - med_{PP, cfuture} w_{PP} - med_{PD, cfuture} w_{PD} } \right)/\ln \left( 2 \right) \\ & = \left[ {w_{PP} \times \left( {med_{PP, future} - med_{PP, cfuture} } \right) + w_{PD} \times \left( {med_{PD, future} - med_{PD, cfuture} } \right)} \right]/\ln \left( 2 \right), \\ \end{aligned}$$

where \({med}_{PP, future}\) and \({med}_{PD, future}\) are the median PP and PD time with future innovation, respectively; \({med}_{PP, cfuture}\) and \({med}_{PD, cfuture}\) are the median PP and PD time without future innovation, respectively; and \({w}_{PP}\) and \({w}_{PD}\) are health state utility weights for PP and PD, respectively. The incremental QALYs of future innovation can be approximated by the sum of the increase in the median PP time and the increase in the median PD time, weighted by the utility weights of PP and PD, divided by the natural logarithm of 2. This formula can be expanded to account for more health states. If the projected \(\Delta ROV\) is small in relation to conventional value \(\Delta {QALY}_{trt}\), the analyst may conclude that a more precise estimate is not worth the additional effort.

5 Assessing the Level of Uncertainty in ROV

As outlined previously, ROV is driven by the likelihood and expected time of arrival of future innovations, the proportion of patients who survive to, and are eligible for, future innovations, as well as the expected efficacy of future innovations. The proportion of patients who survive to, and are eligible for, future innovations is largely a measure of how current treatment impacts survival and quality of life, and the impact of its uncertainty is evaluated thoroughly in probabilistic sensitivity analysis of the conventional value. Forecasted characteristics of future innovations are unique to ROV calculation and are more usefully explored via a qualitative assessment of uncertainty.

Uncertainty in the likelihood and expected time of arrival of future innovations can be evaluated by whether disease- and class-specific estimates of success rate and length of R&D are used. When disease- and class-specific estimates are used to approximate [8], uncertainty is likely lower than using industry-wide estimates [5].

Uncertainty in the efficacy of future innovations can be qualitatively assessed using the following metrics: whether the efficacy has been established using surrogate endpoints (i.e., endpoints that are not survival or quality of life), and whether the efficacy has been established based on single-arm or randomized trials. Uncertainty is likely greater when efficacy is evaluated using surrogate endpoints and/or in single-arm trials [8].

6 Assessing Potential Interactions Between ROV and Other Elements of Value

The possibility of accessing future innovations may modify other elements of value, both standard and novel. The problem of potential overlap or double-counting is not unique to ROV. For example, analysts have long recognized that impacts on patient productivity could affect both the numerator (e.g., lost output from absenteeism) and patient utility in the denominator of the ICER. Some care in measurement is required along with some caution in interpretation, but ignoring joint channels of influence would also be an error.

6.1 With Conventional Value

A patient knowing that he or she can access effective new treatments on the horizon for their condition may increase or decrease their adherence to their current treatment and thus the effectiveness and conventional value of the current treatment. For example, if a patient with a terminal illness knows that a promising new treatment in late-stage clinical development demonstrates improved efficacy and is therefore likely to be approved by the FDA, they may be more adherent to their current life-prolonging treatment to increase their chance of surviving to access the new treatment. For progressing diseases with some risk of death, such as hepatitis C, if current treatment is only moderately beneficial and with some risk of toxicity, patients may poorly adhere or postpone treatment in anticipation of the arrival of a new, effective treatment.

6.2 With the Value of Hope

ROV may also interact with the so-called ‘value of hope’ [19]. The theory here posits that in some situations, particularly at the end of life, patients may become risk-seeking when choosing their treatment. They may prefer a risky gamble—a treatment with a long tail of the survival curve (or a substantial ‘cure fraction’)—over a safe bet that has the same mean efficacy. This is because risk-seeking patients place a disproportionately high value on large health gains, illustrated technically as a convex utility curve. Potential access to future innovations may increase the value of this ‘hope’ even further as a large health gain at the tail of the curve may enable patients to benefit from future innovations, leading to additional health gain and higher utility in the future.

6.3 With Productivity

Current treatment enabling patients to benefit from future innovations can lead to not only additional gain in patient health but also additional gains in productivity for patients and their caregivers. This is because if future innovations are more effective at improving health, they may be able to avert more productivity loss from the illness. Consequently, patients and their caregivers may be able to live a more productive life once these future innovations become available. It is also possible that future innovations have a negative impact on productivity despite better efficacy, possibly due to the high complexity of the regimen leading to high time costs.

6.4 With insurance Value

A treatment can provide ‘physical health risk protection’ to risk-averse healthy individuals by reducing the expected health loss if they were to become ill [20, 21]. A treatment with positive ROV can further reduce health loss in the ill state compared with a similar treatment without ROV, thereby creating greater insurance value for the healthy.

7 Impact on Costs and the Incremental Cost-Effectiveness Ratio (ICER)

Accounting for future innovation in value assessment will have implications for total costs.

The conventional ICER of \(trt\) relative to its control \(ctrt\) is:

$$ICER_{conventional} = \frac{{\Delta Cost_{trt} }}{{\Delta QALY_{trt} }}$$

ICER that incorporates future innovations:

$$ICER_{ROV} = \frac{{\Delta Cost_{trt} + \left[ {S_{trt} \left( {t_{future} } \right)u_{trt} - S_{ctrt} \left( {t_{future} } \right)u_{ctrt} } \right] \times p \times \Delta Cost_{future} }}{{\Delta QALY_{trt} + \left[ {S_{trt} \left( {t_{future} } \right)u_{trt} - S_{ctrt} \left( {t_{future} } \right)u_{ctrt} } \right] \times p \times \Delta QALY_{future} }},$$

where \(\frac{\Delta {Cost}_{future}}{\Delta {QALY}_{future}}\) is the ICER of future innovation. This equation implies that when future innovation \(future\) is priced at the same ICER as the current treatment \(trt\), including ROV will not have any impact on the current ICER. If future innovation is less cost effective than current treatment (\(\frac{\Delta {Cost}_{future}}{\Delta {QALY}_{future}}>\frac{\Delta {Cost}_{trt}}{\Delta {QALY}_{trt}})\), including ROV will increase the current ICER. Conversely, if future innovation is more cost effective than current treatment (\(\frac{\Delta {Cost}_{future}}{\Delta {QALY}_{future}}<\frac{\Delta {Cost}_{trt}}{\Delta {QALY}_{trt}})\), including ROV will lower the ICER.

Only one study thus far has incorporated the price and cost offsets of future innovation in ROV analysis [5]. It is challenging to forecast the price of future innovation as it may be influenced by many factors, including efficacy, competition, innovativeness, unmet need, etc. Empirical research on determinants of drug prices that includes both market dynamics and drug characteristics is lacking. Therefore, threshold analyses can be useful in calculating what the cost effectiveness of future innovation would need to be in order to change the judgment on the cost effectiveness of current treatment. For example, if the ICER for the current treatment is $200,000 per QALY, a threshold analysis could calculate the ICER of the future innovation needed to bring the ICER of the current treatment down to $150,000 per QALY. Alternatively, if the ICER for the current treatment is $125,000 per QALY, a threshold analysis could calculate an ICER for the future innovation that brings the ICER of the current treatment down to $100,000 per QALY or up to $150,000 per QALY. Based on these threshold analyses, analysts can then provide their qualitative evaluation on whether including ROV would likely have an impact on the overall assessment of cost effectiveness.

8 Reporting ROV in Health Technology Assessment

We developed a 15-item checklist for reporting ROV in HTA for inclusion in every value assessment (Table 1). We reviewed and simplified existing guidelines on reporting CEA and added items that are specific to ROV [22, 23]. There are five sections in the checklist, each focusing on the types of ROV, the size of ROV, level of uncertainty of ROV estimate, interaction between ROV and other elements of value, and impact of ROV on ICER. The first section includes three questions, each on one type of ROV. If the answer is ‘no Effect’ for all three questions, the analyst can stop here and conclude that there is no ROV. If section 1 indicates that there is ROV, then in section 2 the analyst is asked to estimate the size of ROV, both in QALYs and as a percentage of the conventional incremental QALYs. In section 3, the analyst reports the potential level of uncertainty around the ROV estimate, by indicating whether class-specific, disease-specific, or industry-wide estimates were used to estimate the probability and time of arrival of future innovation and whether efficacy of future innovation was established using surrogate endpoints and in randomized or single-armed trials. In section 4, the analyst is asked to assess whether ROV can potentially interact with the conventional value, value of hope, productivity, and insurance value. Finally, in section 5, the analyst indicates whether inclusion of ROV will likely change the conclusion on cost effectiveness.

Table 1 Checklist for reporting real option value in value assessment
Full size table

9 An Empirical Example Using Minimal Modeling

In this section, we used ipilimumab for first-line treatment of metastatic melanoma as an example to illustrate the estimation of ROV using minimal modeling and reporting [24]. We previously estimated the ROV of ipilimumab in an augmented CEA model that incorporated the arrival of future innovation [5]. Here, we use minimal modeling instead of building a formal CEA model, as well as improved the estimation of some parameters. Ipilimumab was first approved to treat non-resectable or metastatic melanoma in 2011. Efficacy of ipilimumab in previously untreated patients was demonstrated in a phase III trial that randomized 502 patients to ipilimumab plus dacarbazine or dacarbazine plus placebo [24]. The trial showed that patients in the ipilimumab arm had significantly longer overall survival and progression-free survival compared with those in the placebo arm [24]. We conducted this analysis as if it were performed at launch in 2011 to capture the ex ante ROV of ipilimumab at the time of launch by using information that was available by 2011.

9.1 Types of ROV

In this case, ipilimumab may create ROV through both prolonging survival and delaying disease progression (Table 2, section 1).

Table 2 Checklist for reporting real option value—ipilimumab for first-line treatment of metastatic melanoma
Full size table

9.2 Size of ROV

Table 3 summarizes the four investigational new drugs in phase III testing in 2011 for advanced or metastatic melanoma that could have potentially been the next line of treatment after first-line ipilimumab or placebo. These four products were identified from a search on ClinicalTrials.gov for investigational new drugs under clinical development for metastatic melanoma by March 2011 [25]. We ignored products that were in phase II testing at the time as their chances of obtaining approval were much lower than those in phase III. For the four investigational new drugs, we extracted information on their efficacy from their phase II studies, their phase II study design, and start date of phase III (Table 2).

Table 3 Eligible phase III trials for previously treated metastatic melanoma registered on ClinicalTrials.gov between January 2001 and March 2011
Full size table

To estimate the probability of FDA approval, we used estimates from DiMasi et al., that investigational cancer drugs that had a response rate of ≤ 13.8% in phase II had a 2.5% probability of being approved, while those that had a response rate of >13.8% in phase II had a 77% probability of being approved [17]. Therefore, GSK1120212, T-VEC, and GSK2118436 each had a 77% chance of being approved, while Allovectin-7 had a 2.5% chance of approval.

To estimate the expected time of FDA approval, we added the mean length of phase III trials, i.e. 31 months, and the mean FDA review time for antineoplastic products, i.e. 10 months, to the phase III start date of each of the investigational new treatments [26, 27]. The projected FDA approval dates for GSK1120212, T-VEC, GSK2118436, and Allovectin-7, if approved, were April 2014, September 2012, March 2014, and March 2010, respectively (Fig. 1).

Fig. 1
figure 1

Timelines for phase III testing and US FDA approval for investigational new drugs. mo months

Full size image

After reviewing the probability and the expected time of approval of each of the four investigational products, we decided to exclude Allovectin-7, as the likelihood of approval was extremely low, and it would have been approved before ipilimumab if it had demonstrated significant efficacy. The probability of any approval \(p\) among GSK1120212, T-VEC, and GSK2118436 is then \(1-{\left(1-77\%\right)}^{3}=98.8\%\). The expected time of approval, if any approval, is the average among the three products, which is September 2013. Time to approval \({t}_{future}\) is 30 months, which is the difference between September 2013 and March 2011.

Using OS and PFS curves from ipilimumab’s phase III study [24], at 30 months after the start of the first-line treatment, 7.5% of ipilimumab patients and 2.5% of control patients remained progression-free (difference: 5 percentage points) and 17.5% of ipilimumab patients and 12.5% of control patients were alive but had disease progression (difference: 5 percentage points). Based on these data, we estimated that an additional 5–10 percentage points of patients would survive and be eligible for future innovation on ipilimumab versus control. Moreover, two of the three investigational new drugs are indicated for BRAF mutation-positive patients, which account for approximately 50% of all melanoma cases. Therefore, we multiplied a factor of (0.5+0.5+1)/3=0.67 to the range of 5–10 percentage points.

Finally, the OS estimates of GSK1120212, T-VEC, and GSK2118436 from phase II were 14.2 months, 15.9 months, and 13.0 months, respectively, with an average of 14.5 months [27,28,29]. In the phase III study of ipilimumab, the median OS for patients who progressed on their first-line treatment was approximately 8 months [24]; therefore, if any of the three above-mentioned investigational new drugs was approved, patients who did not respond to their first-line treatment may be able to take advantage of this new medicine and live approximately 14.5 months instead of 8 months. In its phase III study, ipilimumab and control had the same median PFS (3 months) [24]. We made another simplifying and conservative assumption that the future innovation also does not improve median PFS. Therefore, all of the projected survival gain was from a longer time in the PD state. The utility weight for PD for metastatic melanoma was assumed to be 0.52 [30]. Using the minimal modeling approach, the incremental QALYs of future innovation versus control (assuming an exponential survival function) was therefore:

$$\Delta QALY_{future} = 0.52 \times \frac{{\left( {14.5 - 8} \right)}}{\ln \left( 2 \right)} = 4.9.$$

Multiplying \(\Delta {QALY}_{future}\) with the difference in the proportion of patients who survived and were eligible, as well as the likelihood of approval, the ROV of ipilimumab enabling patients to benefit from future innovation in this case was approximately 0.16–0.32 QALYs (Table 2, section 2). We calculated ROV as a percentage of the conventional value by approximating \(\frac{\Delta {QALY}_{future}}{\Delta {QALY}_{trt}}\):

$$\frac{{\Delta QALY_{future} }}{{\Delta QALY_{trt} }} = \frac{{w_{2L, PD} \times \left( {med_{PD, future} - med_{PD, cfuture} } \right)}}{{w_{1L, PD} \times \left( {med_{PD, trt} - med_{PD, ctrt} } \right)}} \approx \frac{{med_{PD, future} - med_{PD, cfuture} }}{{med_{PD, trt} - med_{PD, ctrt} }} = \frac{14.5 - 8}{{11.2 - 9.1}} = 3.1.$$

In the ipilimumab trial, median OS for first-line ipilimumab was 11.2 months and 9.1 months for control [24]. Multiplying \(\frac{\Delta {QALY}_{future}}{\Delta {QALY}_{trt}}\) with the difference in the proportion of patients who survived and were eligible, as well as the likelihood of approval, the ROV of ipilimumab as a percentage of ipilimumab’s conventional value was 10–20% (Table 2, section 2). This method assumes that survival curves of first-line ipilimumab and control both follow the exponential distribution.

9.3 Level of Uncertainty

In this empirical analysis, cancer-specific estimates were used to estimate the probabilities of arrival, while industry-wide estimates were used to estimate the expected times of arrival (Table 2, section 3). The phase II studies of investigational new drugs included in this study all reported median OS and were all single-arm trials (Table 2, section 3).

9.4 Interaction with Other Value Elements

In this case, ROV is likely to interact with the value of hope and insurance value, while unlikely to influence the conventional value through adherence or to interact with productivity effects (Table 2, section 4). Prior research in metastatic melanoma suggested that patients were risk-loving and placed a high value on large health gain at the tail of the curve [19]. ROV can further extend the tail of the curve, as large health gain on ipilimumab enables patients to benefit from future innovations, leading to additional health gain and even higher valuation. For risk-averse healthy individuals, ROV of ipilimumab can further reduce health loss if individuals develop metastatic melanoma in the future, thereby creating greater insurance value. Since both ipilimumab and chemotherapy are administered intravenously in the physician’s office and there are well-established criteria for stopping treatment (e.g., disease progression, intolerable adverse effects, etc.), adherence is unlikely influenced by potential access to future innovation. Finally, the majority of metastatic melanoma patients at the time had a remaining life expectancy of only a few months. Therefore, it is unlikely that any treatment would have much effect on productivity, or that ROV would interact with productivity effects.

9.5 Impact on the ICER

The ICER for first-line ipilimumab without ROV was $229,175 per QALY [5]. To bring the current first-line ICER down to $150,000 per QALY, the future innovation would need to be net cost-saving. Therefore, it is unlikely that including future innovation will change the conclusion of the cost effectiveness of ipilimumab (Table 2, section 5). Note however that this is from a conventional healthcare perspective that does not consider eventual genericization over the product life cycle [31].

10 Discussion

This paper builds upon and integrates several threads in the existing literature related to ROV. The approach presented here is a relatively straightforward extension of conventional CEA, by embedding it in the longer-term development context for a specific disease area.

The minimal modeling approach is designed to simplify and standardize the estimation of ROV in value assessment. Analysts can use this method to quickly assess the potential size of ROV without a formal CEA, to determine if ROV should be formally integrated in the HTA. In many cases, this minimal modeling approach will be sufficient for formal quantification of ROV in HTA, as detailed data on health state transition probabilities required for constructing a CEA are often not available in reports on the preliminary efficacy for future innovations. Since this minimal modeling approach is independent of model structure, it is applicable to all types of ROV across disease areas.

Estimating ROV requires predicting the future, which is highly uncertain in nature. Previous studies have used one-way, two-way, and probabilistic sensitivity analyses to characterize uncertainty in ROV estimates in augmented CEAs [32, 33]. However, early efficacy data of potential future arrivals, with which ROV is estimated, are often based on surrogate endpoints from non-randomized trials. Estimates for success rates and development time are often not available for the specific drug class or even the disease of interest. These create challenges for deriving distributions for these parameters for conventional sensitivity analysis. The qualitative approach to assessing uncertainty in ROV proposed in this paper can to some extent address this limitation, but this approach alone does not capture all the uncertainty around ROV. The 15-item checklist for reporting ROV will standardize the consideration and reporting of ROV in value assessment. For treatments without ROV, a simple three-question qualitative evaluation will be sufficient. Systematically including and reporting ROV in value assessment will minimize bias and improve transparency, which will help improve the credibility of ROV research and acceptance by stakeholders, as well as use.

The checklist also provides a useful framework for considering ROV in pricing and coverage decisions. The size of ROV in incremental QALYs gained and the impact of ROV on the ICER provide useful starting points for decision making regarding the ‘option premium’ in value-based price and whether a technology should be covered by a health plan. However, these should not be viewed alone but in light of the uncertainty in the ROV estimate, as well as potential interactions with other elements of value.

Despite significant recent progress on ROV theory and methods, more work is needed. An area that this paper did not delve into is the interaction between ROV and other elements of value. Although we explained the intuition of some of the interactions, we did not include mathematical proofs or provide empirical methods for estimation. For example, there has been significant progress on incorporating risk aversion and diminishing returns to health in HTA using the Generalized Risk-Adjusted Cost-Effectiveness (GRACE) approach [34,35,36]. A more comprehensive assessment of ROV could explicitly incorporate risk preference and diminishing return by incorporating ROV into GRACE [37].

A type of value that is related to ROV is from a future cure completely halting disease progression. In this case, if the current treatment can slow down disease progression, when the future cure arrives, the current treatment would leave patients permanently in a state with a higher quality of life. Similar to ROV, this value requires accounting for future innovations. Unlike ROV, no opportunities for future treatment are created or exercised by current treatment [38]. Nevertheless, it is important to account for this value from slowing down disease progression, especially with the advent of gene therapies and other potential cures on the horizon.

Some cures may also reverse disease progression. In this case, patients with a larger deficit in health would benefit more from the arrival of a future cure as there is more room for improvement. As a result, a current treatment that can improve health would actually create negative ROV as it would reduce how much a patient can benefit from a future cure. Future research should examine cases of negative ROV.

As emphasized at the outset, a more accurate assessment of value will provide more nuanced signals to innovators that will help them to target innovations to the disease areas of greatest unmet need and thus greater value, thereby promoting the overall dynamic efficiency of pharmaceutical research and development efforts.