1 Value of Information Analysis Principles and Applications

To enhance early access to novel health technologies, reimbursement decisions are increasingly made when the evidence base to support those decisions is lacking or far from mature [1, 2]. Making decisions with substantial evidence uncertainty may lead to suboptimal recommendations and costly consequences (e.g. listing a drug that is not cost effective). Intuitively, decision uncertainty can be reduced by collecting more evidence; however, the additional benefits from a new research study may not justify its costs in terms of the direct research costs (e.g. site and recruitment costs) and the opportunity costs from the benefits forgone when the adoption of a promising health technology is deferred while the additional information is being collected [3, 4].

Value of information (VOI) analysis quantifies the expected value of research in reducing decision uncertainty to inform whether a decision can be made based on existing evidence or if additional evidence is required and worthwhile [3, 4]. There is a range of VOI measures to inform research and reimbursement decisions. The most common measure is the expected value of perfect information (EVPI), which is the value of additional information to resolve uncertainty in all decision parameters. Another measure is the expected value of perfect parameter information (EVPPI), which estimates the value of resolving uncertainty in a parameter or a subset of parameters. Both EVPI and EVPPI measure the maximum (i.e. upper bound) value of research, allowing for a rapid screening for the need and potential value of additional evidence. For instance, a negligible EVPI indicates that there is little value from additional research and a decision can be made based on existing evidence [5]. However, if additional research is potentially worthwhile (i.e. EVPI is significant), the value of reducing uncertainty through collecting data in a study of a specific sample size can be estimated using the expected value of sample information (EVSI). By comparing the expected monetary benefits and costs of research studies, VOI analysis informs various types of decisions including (i) reimbursement decisions to adopt, reject, or ask for additional evidence (e.g. coverage with evidence development), (ii) efficient trial design by selecting sample sizes that maximise monetary benefits and (iii) prioritising research studies with the highest returns on research investments [4, 6, 7].

Despite the value of VOI analysis as a decision tool, its application in practice remains limited. The two main barriers against a wider VOI application have been the complexity of the calculations required to estimate VOI measures, especially the EVSI, and the lack of awareness among researchers and policy makers about this approach [8, 9].

2 Current Developments

Two professional groups have been recently formed to promote VOI analysis and enhance its application in practice: (i) VOI Task Force under the International Society for Pharmacoeconomics and Outcomes Research (ISPOR), which aims to improve the accessibility of VOI analysis for all stakeholders through the development of good practice guidance [10, 11]; and (ii) the Collaborative Network for Value of Information (ConVOI), which is an international network of VOI experts that aims to improve the calculation, adoption and application of VOI methods in clinical and public health research [12]. ISPOR VOI Task Force has recently released two reports: the first report provides general recommendations for good practice when planning, undertaking or reviewing the results of VOI analyses [10]; the second report is directed at analysts, providing detailed algorithms and step-by-step guidance to VOI calculations [11].

To enhance the feasibility of VOI analysis, efficient calculation methods have been recently developed; these include a regression-based method [13, 14], importance sampling method [15], Gaussian approximation method [16] and moment matching method [17]. Some of these methods come with web-based tools to facilitate VOI calculation. Examples of these tools include Sheffield Accelerated Value of Information (SAVI) [18] and BCEAweb, an online version of the BCEA (Bayesian Cost-Effectiveness Analysis) R package [19]. Furthermore, the curve of optimal sample size (COSS) has been proposed as an approach to present sensitivity analyses on VOI-specific parameters, providing summary data for decision makers to determine the sample size that optimises research funding over different willingness-to-pay thresholds [20].

3 The Way Forward

With the recent methodological developments, efforts should be focused on supporting the application of VOI in practice and facilitating its incorporation into decision-making frameworks. This may be achieved by effective communication with stakeholders (i.e. decision makers, researchers) to improve their understanding of the approach and its value. Notwithstanding, it is vitally important to understand and address the needs, expectations and concerns of the different stakeholders.

Researchers may consider VOI a barrier that limits their access to research funding; however, assessing the value for money of research studies would help researchers and research organisations make early stop–go decisions about their research projects and make better use of their limited resources [2, 21]. Moreover, VOI analysis can be used to support investigator-initiated grant applications by demonstrating the potential value for money of funding applications. Expectedly, many researchers would find it difficult to accept a new method for sample size calculation, which is different from the traditional hypothesis-testing approach. Nevertheless, VOI analysis can support, but not necessarily replace, the current approach by helping researchers economically calculate sample sizes and explore additional aspects of research design such as the optimum follow-up duration and allocation of participants in trial arms. To build capacity, more health economists and analysts need to be trained on using VOI methods and online tools through hands-on workshops and courses.

From a public research funding organisations’ perspective, VOI analysis should promote a transparent, quantitative and objective approach to allocate limited research budgets and set research priorities. VOI has been successfully applied to inform research prioritisation in pilot studies in the UK, US and Australia [5, 7, 22]. It is important to keep in mind, however, that research funding organisations may not have the time and/or capacity to build decision analytic models to evaluate all research proposals they receive during a funding round [23]. In this case, a rapid approach to VOI estimation such as the use of minimal or no modelling should be considered [24]. Minimal modelling might be appropriate when the outcomes of interest and the uncertainty around these estimates are sufficiently reported in prior evidence (e.g. a meta-analysis) to inform a decision about the benefits of alternative interventions [24, 25]. Two web-based tools are now available to help research organisations, and researchers, estimate VOI without full economic modelling: Rapid Assessment of Need for Evidence (RANE) [26], which requires only an expression of uncertainty in a primary outcome (e.g. odds ratio) and other inputs such as baseline risk and incident population; and the Value of Information for Cardiovascular Trials and Other Comparative Research (VICTOR) [27], a web-based platform that can help researchers plan a comparative cardiovascular disease study using clinical trial or other research designs. To further facilitate the use of these tools when there is a large number of funding applications, VOI estimation may be conducted for a short list of proposals (e.g. the top 10 percentile of applications), or for proposals that request large budgets [23, 28].

For health technology assessment agencies, VOI may appear as a complex academic exercise. They may also be sceptical about the robustness of the findings of VOI with the associated modelling and assumptions. Nevertheless, incorporating this approach into decision frameworks is intuitive if it is introduced as a decision support tool to reduce the chance and costly consequences of sub-optimal decisions [1, 2, 29]. Many decision-making organisations embrace evidence-based approaches in making decisions using cost-effectiveness analysis; furthermore, handling and presenting uncertainty is already embedded in the current decision frameworks in many jurisdictions. Therefore, there is no reason why an additional necessary step to evaluate the consequences (i.e. opportunity losses) of uncertainty should not be considered to optimise decision making. Links could be established between reimbursement and research decisions whereby research is commissioned and funded to address decision making needs [1, 7, 29].

Industry may also consider VOI a barrier for reimbursement and market access. Clearly, the expected benefit from the industry perspective is the expected profits, which is different from the societal benefits from the public organisations’ perspective. Nevertheless, VOI can support flexible funding schemes such as coverage with evidence development to allow the early adoption of promising interventions while further research is underway [1, 2, 7, 29]. Moreover, VOI can inform the ‘research and development’ decision; that is, to continue to investigate in a given product or not. Even if the future product does not appear to be cost effective but the uncertainty is high, the product may become cost effective with additional research. Further, the price could be reviewed to make the product cost effective and to reduce uncertainty [1, 2, 7, 29].

VOI analysis is a powerful approach to inform reimbursement decisions, optimise trial design and set research priorities. Despite its potential, the application of VOI in practice is limited. Major methodological advances have taken place over the past few years and best practice guidelines have been developed. To further facilitate the application of VOI analysis in practice, it is essential to understand and address the needs, expectations and concerns of different stakeholders, and to consider the barriers and facilitators to a wider adoption of these methods.