A Critical Appraisal and Recommendations for Cost-Effectiveness Studies of Poly(ADP-Ribose) Polymerase Inhibitors in Advanced Ovarian Cancer

Abstract

Background

Ovarian cancer is the fifth leading cause of cancer death in women in the US. With poly(ADP-ribose) polymerase (PARP) inhibitors having shown promising results in ongoing trials, there is interest in better understanding their economic value.

Objective

This study aimed to review and evaluate the quality of published cost-effectiveness analyses (CEAs), and provide recommendations for CEAs in this setting.

Methods

A systematic literature review of the MEDLINE and EMBASE databases was conducted in June 2019 to identify CEAs of PARP inhibitors in treating advanced ovarian cancer from peer-reviewed journals and conferences. Key information from the identified publications were extracted and reviewed. The quality of full-text studies was assessed using the Quality of Health Economic Studies instrument. Recommendations for future CEAs were developed based on the findings from the literature review.

Results

Eighteen CEAs (five in full texts) met the inclusion criteria. Most adopted a US healthcare or societal perspective. The majority of the studies did not clearly display the economic model structure. No studies reported the validation of model projections based on internal or external data. Surrogate outcomes such as incremental costs per progression-free life-year gained were the most common outcomes reported. The majority of studies drew their conclusions based on surrogate outcomes, even with no theoretical or empirical threshold for cost effectiveness. All five full-text studies included some type of sensitivity or scenario analyses. The key drivers of the incremental cost-effectiveness ratio were treatment duration, effects, and costs, health utility, and prevalence of BRCA mutations.

Conclusion

In the existing CEAs for PARP inhibitors, there were uncertainties and challenges leading to variation in quality. We provided recommendations to improve consistency and quality of CEAs in this setting, which will help to better understand the value of PARP inhibitors, improve decision making, and reduce potential misallocation of resources.

FormalPara Key Points for Decision Makers
With poly(ADP-ribose) polymerase (PARP) inhibitors having shown promising results in ongoing trials, there is interest in better understanding their economic value.
The key drivers of the incremental cost-effectiveness ratio were treatment duration, effects, and costs, health utility, and prevalence of BRCA mutations.
Recommendations specific to PARP inhibitor economic evaluations are related to outcomes, model validation, and conclusions on value.

Introduction

Approximately 21,000 women in the US are diagnosed with ovarian cancer each year, with the majority of patients diagnosed in the late stages [1]. The 5-year survival rate among patients with ovarian cancer is estimated at 47%, and with nearly 14,000 deaths each year, the cancer is the fifth most deadly among women in the US [2]. Germline mutations in the BRCA1 and/or BRCA2 genes, which code for deoxyribonucleic acid (DNA) repair, are an important risk factor for ovarian cancer. A recent study showed that 44% of women with a BRCA1 mutation and approximately 17% of women with a BRCA2 mutation will develop ovarian cancer by the age of 80 years [3].

Treatment of ovarian cancer typically consists of surgery and platinum chemotherapy with or without bevacizumab. Following primary therapy, maintenance with poly(ADP-ribose) polymerase (PARP) inhibitors can help manage the disease and delay disease progression, especially among women with BRCA mutations. PARP inhibitors, including olaparib, rucaparib, and niraparib, block the PARP pathway and make it hard for tumor cells with BRCA mutations to repair DNA, causing cell death [4]. In the US, olaparib was initially approved as a fourth-line option in ovarian cancer patients with a BRCA mutation [5]. In recent studies, olaparib monotherapy has demonstrated benefit as a maintenance therapy for adult patients with a BRCA mutation who are in a complete or partial response to primary therapy [6], and also as a maintenance therapy for patients with platinum-sensitive relapsed (PSR) disease [7, 8]. Olaparib has also shown benefit when used in combination with bevacizumab as a maintenance therapy in newly diagnosed ovarian cancer patients [9]. Rucaparib and niraparib are currently recommended as options for PSR disease [10]. Niraparib also demonstrated benefits in progression-free survival (PFS) among patients with newly diagnosed advanced ovarian cancer who had a response to platinum-based chemotherapy in a recent study [11].

Cost-effectiveness analysis (CEA) is a commonly used and well-established method of evaluating the comparative effectiveness and costs of alternative treatment options. CEA is becoming increasingly important in the US, as stakeholders further use it to determine drug value and access in recent years. Given the emergence of the PARP inhibitor class, there is much interest in their economic value, and many CEAs have been published or presented. For example, the Institute for Clinical and Economic Review estimated the cost effectiveness of PARP inhibitors as maintenance therapy for women with relapsed ovarian cancer and found that the PARP inhibitors would provide gains in quality-adjusted survival and overall survival (OS) over alternative therapies, and only olaparib is priced in alignment with these benefits [12]. CEA is a reliable reference for stakeholders only if it is of sufficient quality and unbiased; otherwise, it may lead to the misallocation of healthcare resources [13, 14]. In this study, we aimed to review and evaluate the quality of existing CEAs for PARP inhibitors in advanced ovarian cancer and identify the challenges in conducting high-quality CEAs. Based on the key drivers, issues, and uncertainties found in the literature review, this study also aimed to provide recommendations for the proper conduct of high-quality CEAs in this setting to better understand the value of PARP inhibitors, improve future CEAs and decision making, and reduce potential misallocation of resources.

Methods

Systematic Literature Review

A systematic literature review was conducted to identify CEAs of PARP inhibitors in treating advanced ovarian cancer, according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Relevant disease and study design terms were compiled for searches in the MEDLINE and EMBASE databases (Online Appendix 1). In addition, proceedings of 2016–2019 annual meetings of the American Society of Clinical Oncology (ASCO), Society of Gynecologic Oncology (SGO), International Society for Pharmacoeconomics and Outcomes Research (ISPOR), and the European Society for Medical Oncology (ESMO) were screened. The systematic literature search was conducted in June 2019. The titles and abstracts of all citations returned from the systematic literature search were screened using a predefined set of inclusion criteria (Online Appendix 2), i.e. CEAs of PARP inhibitors in the treatment of adult patients with advanced ovarian cancer that reported incremental cost-effectiveness ratio (ICER) results and were published in English. Studies that assessed the cost effectiveness of other ovarian cancer treatments (e.g. bevacizumab) were eligible for inclusion if they also reported the cost-effectiveness of PARP inhibitors. Other types of studies, such as clinical trials, observational studies, reviews, or meta-analyses were excluded. The full texts of studies and posters/presentations that passed title and abstract screening were retrieved (if available) and were reviewed using the same criteria.

Data from included publications were extracted into a prespecified extraction grid available in Online Appendix 3. Extracted information included study designs (e.g. model structure, model perspective, model outputs, and target population), efficacy inputs (e.g. OS and PFS data source, extrapolation methods, model fit evaluation, and model validation), utility and cost inputs (e.g. utility tool and data source, costs components and data source), and study results (e.g. base-case results and sensitivity analysis results). All screening and data extraction steps were conducted independently by two reviewers in parallel, and any disagreements were resolved via independent arbitration by a third reviewer.

Quality Assessment of Economic Analysis

The quality of identified full-text studies was assessed using the Quality of Health Economic Studies (QHES) instrument [15]. This instrument evaluates the appropriateness of study methods as reported in the papers, and the validity, transparency, and comprehensiveness of study results. It has 16 items with dichotomous response (yes/no), and each item has a weighted point value. The quality of a study is measured by the sum of points across items with ‘yes’ responses. The quality score ranges from 0 to 100 and a higher score indicates better quality. Two reviewers independently reviewed each study using the QHES checklist. If the score differed between reviewers, co-authors discussed the score in order to reach a consensus.

Studies that were published as abstracts, and their corresponding posters (if available), were reviewed and summarized by the same reviewers. Specifically, we reviewed the abstracts in reference to the domains listed in Table 1 and the more detailed extraction grid available in the Online Supplementary Material (Appendix 3). These studies were not scored using the QHES instrument, as many components of reporting cannot be fairly assessed from abstracts.

Table 1 Identified studies from the systematic literature review, and their study design

Results

Overview

Of 193 publications returned from the search, 18 cost-effectiveness studies met the inclusion criteria (Fig. 1) [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33]. Five of the 18 studies were published as full-text articles [17, 19, 22, 26, 33] and the remaining were abstracts presented at conferences. The 18 studies identified by the systematic literature review and the key elements of their study designs are summarized in Table 1. The study population was patients with newly diagnosed ovarian cancer for two studies [31, 32], and patients with relapsed/recurrent advanced ovarian cancer for the remaining 16 studies. Additionally, subsets of ovarian cancer patients with BRCA mutations were considered in 6 of the 18 studies [18, 21, 24,25,26, 32]. Most of the studies adopted a US healthcare or societal perspective.

Fig. 1
figure1

PRISMA flow diagram for cost-effectiveness analyses of PARP inhibitors. (1) Study population: studies in non-human populations (level 1: n = 13) and non-ovarian cancer populations (level 1: n = 38) were excluded. (2) Study treatment: studies that did not include PARP inhibitors (level 1: n = 41) were excluded. (3) Publication type: guidelines and recommendations (level 1: n = 1), reviews (level 1: n = 24), and letters (level 1: n = 5) were excluded. (4) Study design: clinical trials (level 1: n = 2) and observational studies (level 1: n = 4; level 2: n = 1) were excluded. (5) Outcome of interest: studies that did not report the incremental cost-effectiveness ratio (level 1: n = 1), budget impact analyses (level 1: n = 13), and cost–benefit analyses (level 1: n = 1) were excluded. ASCO American Society of Clinical Oncology, ESMO European Society for Medical Oncology, ISPOR International Society for Pharmacoeconomics and Outcomes Research, PARP poly(ADP-ribose) polymerase, PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses, SGO Society of Gynecologic Oncology

The majority (n = 14) of the included abstracts and full-text studies reported the incremental costs per progression-free life-year (PF LY) gained or per progression-free quality-adjusted life-year (PF QALY) gained [16, 17, 19, 20, 23, 24, 26,27,28,29,30,31,32,33]. Five studies reported incremental costs per LY or QALY gained [18, 21, 22, 25, 29]. The incremental cost per PF LY gain reported for olaparib versus placebo ranged from €63,046 [25] to $600,000 [26], depending on the country, line of therapy, and BRCA mutation status. Similarly, the incremental cost per PF LY gain reported ranged from $248,992 [23] to $496,157 [24] for rucaparib versus placebo, and from $197,000 [33] to $304,775 [23] for niraparib versus placebo.

Objective, Perspective, Subgroups, and Time Horizon for Cost-Effectiveness Models

Of the five full-text studies, QHES scores were 71 [26], 77 [19], 79 [17], 85 [33], and 92 [22] points of a possible 100 (Table 2). All five full-text studies clearly stated their objective (QHES item 1) and perspective of analysis (QHES item 2). All studies used evidence from randomized controlled trials (RCTs); however, there was variation in evidence used as inputs to cost-effectiveness estimation (QHES item 3). Specifically, the majority (four of five) of the full-text studies relied on the surrogate measure of PFS and did not include OS evidence or any attempt at extrapolation beyond the trial time horizon [17, 19, 26, 33]. As expected, studies that relied solely on PFS evidence were, by definition, restricted to shorter time horizons (QHES item 8) of < 24 months or not reported [17, 19, 26, 33], whereas the one full-text study that generated OS from a PFS/OS relationship assessed outcomes over a 20-year time horizon [22]. This trend was consistent in the abstracts we reviewed but did not score; of the abstracts assessed, two studies included a longer time horizon, with the remaining studies focused on shorter time horizons [18, 25]. Only 3 of 18 studies reported discounting outcomes, with two studies reporting a 3% discounting rate and one study reporting a 3.5% discounting rate (QHES item 8) [18, 22, 25]. All full-text studies met QHES items 4 and 6 (incremental analysis between alternatives). QHES item 4 (prespecified subgroups) was particularly important, given that economic evaluations were informed by clinical evidence specific to patients’ BRCA status.

Table 2 Quality of Health Economic Studies assessment for six full-text studies

Model Structure

In 16 of the 18 identified studies, model types were described as decision-analytic models [19, 20, 22, 23, 26, 33], Markov models [16, 17, 25, 28,29,30,31,32], and a partitioned survival model [18]. For the remaining two studies, one did not report the model structure [27] and the other relied on reported median OS for the CEA and did not estimate life expectancy through simulation or evidence synthesis [21]. Health states considered in the models were reported in six studies; two of the studies considered three states (progression-free, progressed disease, and death) [18, 22], while the other four studies considered five states (response, hematological complications, non-hematological complications, next-line therapy or progression, and death) [28,29,30, 32].

Among the five full-text studies, only two (40%) clearly displayed the economic model structure, study methods, and components of the numerator and denominator (QHES item 12) [17, 33]. It was unclear how components of the numerator and denominator were calculated in the remaining three full-text studies. Across all full-text articles and abstracts, four studies extrapolated results beyond the trial time horizon to report LYs and/or QALYs gained [18, 21, 22, 25]. However, the majority of studies did not clearly define model structures through text or figure descriptions. For example, two studies did not include a model figure or explain in detail the model structure [19, 26]. Three of the full-text studies included an upfront decision tree on BRCA testing or BRCA prevalence for targeting therapy to subgroups by BRCA status [17, 19, 26]. Two full-text studies stratified results by BRCA status [22, 33].

Data Abstraction, Cost Measurement, and Outcome Measures

All five full-text studies stated the methodology for data abstraction (QHES item 7) and cost sources and estimation (QHES item 9). Five studies (abstracts) compared PARP inhibitors with therapies that are not approved by the US FDA for the treatment of ovarian cancer [21, 28,29,30,31]. These studies, along with all full-text studies and abstracts, did not attempt any indirect treatment comparisons or meta-analyses when comparing PARP inhibitors with other active comparators. While all outcome measures were stated across the full-text studies, one study did not include negative outcomes such as adverse events (QHES item 10) [26]. Of the five full-text studies, two reported utilities using the EuroQoL 5-dimensions [17, 22], one reported deriving scores from the Functional Assessment of Cancer Therapy–Ovarian (FACT-O) scale [33], and the remaining full-texts did not use any scales (QHES item 11).

Model Uncertainty, Validation and Limitations

All five full-text studies included some type of sensitivity or scenario analyses (QHES item 5) [17, 19, 22, 26, 33]. Within the full-text papers, only two performed a probabilistic analysis [22, 33], and of the abstracts assessed, four reported a one-way or probabilistic sensitivity analysis [16, 18, 25, 27]. Within the full-text papers, only two performed a probabilistic analysis [22, 33]. All five full-text studies listed limitations (QHES item 13), but only four of the five studies discussed magnitude and/or direction of bias (QHES item 14) in the cost-effectiveness findings [19, 22, 26, 33]. We found no studies that provided comprehensive validation, as suggested by good practice guidelines, using both internal and external data [34]. Guy et al. provided a description of validation procedures with partial validation information such as Kaplan–Meier plots with parametric distribution fits for visual inspection [22]. There was no reported validation of modeled OS, which is instrumental to the prediction of both LYs and QALYs.

Key Drivers of Cost and Cost Effectiveness

Key drivers of cost and cost-effectiveness results are input parameters that produce the largest impact on model outputs (e.g. costs, QALYs, ICERs) from sensitivity analyses, scenario analyses, or modeling assumptions. Across all reviewed model-based economic evaluations, key drivers of ICERs for PARP inhibitors against comparators include inputs to the denominator such as treatment effect assumptions related to PFS and OS [18, 22, 26, 33] and health-related quality of life (HRQoL) utility scores for health states [18, 22]. Other key drivers of cost-effectiveness ratios include treatment discontinuation [22], prevalence of germline BRCA mutations [19, 26], and acquisition price [16, 19, 26, 33].

As reported in sensitivity and scenario analyses, increases (decreases) in PFS, while holding the comparator PFS and other parameters fixed, decreased (increased) ICER estimates [22, 26, 33]. Smith et al. and Zhong et al. both reported cost per PF LY ratios above $100,000 per PF LY gained for PARP inhibitors, even with considerable increases in PFS over routine surveillance in both patients with and without BRCA mutations [26, 33]. Alternatively, Guy et al. found that niraparib was cost effective at $100,000 per QALY in the base-case versus olaparib, rucaparib, and routine surveillance [22]. In this analysis, increases in PFS for niraparib were assumed to translate to increases in OS, thus reducing the ratios below $50,000 per QALY versus routine surveillance. Conversely, decreases in PFS increased the ratios above $140,000 per QALY versus routine surveillance. Time on treatment, which may be related to PFS, was associated with higher ICERs, and shorter time on treatment was associated with lower ICERs, as compared with the base-case ICER [22]. While this seems counterintuitive, it is plausible that as patients stay alive they may continue to take maintenance therapy, resulting in higher overall costs that may not necessarily be outweighed by the benefit of therapy.

The four studies that extrapolated results beyond the trial time horizon to report LYs and QALYs gained, found that increases (decreases) in OS were also associated with decreases (increases) in ICERs [18, 21, 22, 25]. Chin and Carlson varied the OS hazard ratio for rucaparib versus placebo, resulting in a range of ICERs from $325,000 per QALY at upper values and over $500,000 per QALY at lower values, compared with a base-case ICER of $361,000 per QALY [18]. Through scenario analyses, Guy et al. found that ICERs were most sensitive to the PFS/OS relationship, as well as distributional assumptions for OS extrapolation [22]. Assuming a 1:1 relationship between PFS and OS, compared with the 1:2 relationship in the base-case for niraparib versus routine surveillance, increased ICERs from $68,287 to $126,500.

Variation in utility scores had a small impact on ICERs across four studies that used utility scores to estimate a QALY or PF QALY gained [18, 19, 22, 33]. However, incorporation of HRQoL utility scores varied by study, including differences in utility scores by health state and differences in utility scores within health state, by treatment. For example, one study added utility values as a scenario analysis and found an increase from $226,000 per PF LYG in the base-case to $297,000 per PF QALY gained for niraparib versus placebo [33], whereas Chin and Carlson [18] and Guy et al. [22] found small ranges of ICER per QALY estimates (i.e. less than $25,000 change from base-case) after varying utility scores by their upper and lower bounds. Other small nuances led to differences in cost per QALY estimates. For example, Guy et al. applied a utility of 0.849 for niraparib, 0.820 for routine surveillance, and 0.769 for olaparib, all within the same PFS health state. Therefore, the study found that olaparib and niraparib had equivalent LYs but differences in total QALYs [13].

Trial evidence has suggested differences in response to PARP inhibitors by BRCA mutation status. Model evaluations found differences in ICER estimates by BRCA mutation status [22, 33]. Given improvements in PFS and OS were larger in patients with BRCA mutations, ICER estimates across studies were lower compared with patients without BRCA mutations [22, 33].

As expected, the price of therapy was also a significant driver of ICER estimates across studies [17, 19, 26, 33]. For studies that reported suggested price ranges, prices per month to align with benefit ranged from $1000 [17] up to $5000 [26]. There are two caveats to these price suggestions [17, 26]: the studies used PF LYs or PF QALYs gained to calculate threshold prices, and some studies used prior dosing strategies that have since been updated.

Input parameters that did not have a considerable impact on ICERs were adverse events and their associated costs [17, 19, 22, 33], hospice care [17], relapse chemotherapy [17, 22], and cost of BRCA testing [17, 19].

Study Conclusions and Disclosures

Four full-text studies drew their conclusions based on incremental costs per PF LY or PF QALY gained [17, 19, 26, 33]. Given there is no theoretical or empirical threshold for cost effectiveness using PFS as an effectiveness measure, coming to conclusions on whether a PARP inhibitor was or was not cost effective was not appropriate based on reviewer assessment (QHES item 15). None of the abstracts reviewed reported QALY or LY gained results. All full-text studies stated the funding source, with one study funded by industry and all others either unfunded or government funded (QHES item 16).

Discussion

The majority of studies identified in the review reported on the cost effectiveness of PARP inhibitors from a US healthcare or societal perspective, despite there being reluctance in the US to the widespread acceptance of CEAs and ICERs. It is not our intention to wade into that debate, but rather to focus on quality issues that should be embraced as a prerequisite to the use of CEAs more broadly. The QHES was designed as a minimum benchmark of quality to improve comparability across CEAs. Cost-effectiveness modeling approaches involve nuanced analytic decisions that can be the source of much debate, which at minimum should be reported clearly, and this is what the QHES judges. Therefore, the QHES is necessary but not sufficient for assessing the quality of cost-effectiveness studies. Here, we discuss overall findings on the strengths and limitations in the quality of published CEAs of PARP inhibitors in advanced ovarian cancer, together with recommendations for future studies.

Evidence Synthesis, Model Structure, and Extrapolations

The quality of evidence synthesis includes analytic decisions surrounding appropriate model comparators, inputs, model structure, and PFS/OS relationships. Appropriate model comparators include therapies indicated for the population with head-to-head or indirect treatment comparison evidence. However, in numerous studies, non-indicated therapies for ovarian cancer were included in model comparisons [21, 28,29,30,31]. Furthermore, comparisons across PARP inhibitors were made without a network meta-analysis, and using potentially biased, naive cross-trial comparisons of effect [22]. Including non-indicated therapies may be relevant given wide use in practice; however, without appropriate head-to-head evidence in practice or through a network meta-analysis may result in biased cost-effectiveness estimates.

Additionally, inputs should be clearly defined and use a consistent approach across therapies. For example, one paper assigned differential utility values for the same health state of PFS [22], which led to the same extrapolation estimate of OS across therapies, but different QALY findings. Given significant differences in tolerability or safety profiles, assigning differential utility values may be appropriate, but in this case, there was no clear justification or evidence provided in support of this assumption.

There were inconsistencies in both the reporting and application of model structures that reflect the disease course of advanced ovarian cancer. For example, seven of the studies provided generic descriptions of model structure, such as ‘decision-analytic model’ [19, 20, 22,23,24, 26, 33]. While the remaining studies were more specific, with reporting model structures such as Markov model [16, 17, 25, 28,29,30,31,32] or partitioned survival model [18], there were further inconsistencies on reporting the number of health states among those same studies [16, 17, 25, 31]. Of the remaining studies that reported the number and description of health states, the majority included five health states that included complications, next lines of therapy, and death [28,29,30,31], while others included three health states that included progression-free, progressed, and death [18, 22]. Clear descriptions of model structure that reflect the disease course given available data would improve the comparability and replicability of modeling studies for the treatment of ovarian cancer with PARP inhibitors.

Other evidence synthesis and input inconsistences were observed for the PFS/OS relationship. PFS is a key surrogate outcome for OS and is generally defined as the length of time from random assignment in a trial to disease progression or death from any cause [35]. Where OS is uncertain or immature, the assumed relationship between PFS and OS is crucial for cost-effectiveness modeling because these analyses are attempting to establish the time spent in each health state, which contributes to incremental differences in quality and quantity of life between comparators. However, there are multiple challenges to describing this relationship, including, but not limited to, trials often being powered for PFS and not OS, leading to greater uncertainty in OS extrapolations; PFS includes death as an event within survival analyses, leading to difficulties in assigning death in models at the appropriate time point without patient-level data; patients may receive multiple lines of therapy, making it difficult to estimate survival based on one therapy; and patients in the placebo arm may switch to the treatment being evaluated after disease progression, making the OS benefit of the treatment underestimated [36].

Of the four studies that extrapolated results beyond the trial time horizon to report LYs and QALYs gained, two included the source of the relationship between PFS and OS [18, 22]. Both studies used data from the phase II Study 19 trial [7, 8, 37] of olaparib versus placebo for OS data, along with PFS sourced from niraparib and rucaparib trials. However, the rationale behind the relationship between the PFS of the treatment being modeled (niraparib or rucaparib) and OS from another treatment (olaparib) was not clearly delineated in either study, and no evidence of the validity of the predicted OS curve from this relationship was reported. Results from both studies are therefore difficult to validate against other clinical and economic evidence [13]. This underscores the importance of modeling treatment effects on the correlated outcomes of PFS and OS in a way that adequately accounts for uncertainty related to parameters describing the surrogate relationship (e.g. a multivariate meta-analysis [38]), in order to fully describe the range of possible LY and QALY estimates.

Outcome Measures

Without extrapolation of OS, the majority of full-text studies were left with reporting PF LYs or PF QALYs gained [16, 19, 26, 33]. Studies that present PFS outcomes alone will be missing important changes in quality and quantity of life over time that contribute to the benefits and risks of therapy. While QALYs and LYs gained also require assumptions such as extrapolation with the relationship between PFS and OS, QALYs and LYs are easily comparable with other oncologic and non-oncologic conditions. An ISPOR Task Force on defining elements of value in healthcare noted that despite their limitations, QALYs remain the most accepted measure for capturing the incremental benefit of a treatment, although QALYs may not reflect all elements of a treatment’s value [39].

Drawing Conclusions on Value

The majority of studies reported PF LYs or PF QALYs gained, and drew surprisingly strong conclusions on value, given no widely accepted threshold for cost effectiveness exists for these effectiveness outcomes. Even with limitations on extrapolated evidence to estimate QALYs and LYs gained, many countries around the world make coverage and reimbursement decisions based on cost per QALY (or similar metric) ratios. Even in the US, value assessment bodies use cost per QALYs or LYs, alongside benefits, disadvantages and contextual considerations, to identify high- and low-value interventions [40]. Regardless of use in coverage and reimbursement decisions, modelers should focus on what they can control through their analytic decisions and not draw conclusions as to the cost effectiveness of a modeled intervention where no threshold is stated or exists for the ICER metric employed in the study. Modelers who extrapolate beyond trial horizons can attempt multiple uncertainty analyses that vary the relationship between PFS and OS. Where extrapolation may not be possible, we recommend discussing clinical benchmarks on OS that would meet commonly cited cost-effectiveness thresholds.

Since running the searches, we identified a relevant study published in full text by Wolford et al. [41]. This study evaluated the cost effectiveness of PARP inhibitors for platinum-resistant ovarian cancer from a US patient and third-party payer perspective in terms of cost per median PFS month, through a five-state Markov model. The relationship of this study and the five conference abstracts authored by Wolford that were included in this review is unclear [28,29,30,31,32]. The QHES score for the full-text study is 78 (Table 2), within the range of full-text studies included in the review (71–92). The conclusions of our critical appraisal and recommendations for future studies are unaltered by our additional consideration of this publication.

Conclusion and Key Recommendations for Future Cost-Effectiveness Analyses for Poly(ADP-Ribose) Polymerase Inhibitors in Ovarian Cancer

The key discussion points above translate into key recommendations for future CEAs (Table 3). Recommendations generic to all CEAs include appropriate analytic decisions around time horizon, population, model structure, and input parameters. In order to capture full benefits and costs, lifetime horizons (i.e. life expectancy of the population) should be used in the base-case analysis. This requires extrapolation beyond trial duration, and we provide additional recommendations to validate these steps below. There is strong evidence that supports differential response to therapy by mutation status. Subgroup analyses and appropriate comparators within those subgroups should be clearly defined.

Table 3 Key recommendations for future CEAs for PARP inhibitors in ovarian cancer

Future studies should clearly describe the model structure and number of health states, along with justifying the structure chosen based on the disease course and available data. Specifically, modelers should consider the role of intermediate clinical endpoints, such as second PFS and time to second subsequent treatment, in supporting such relationships [42]. Accurate descriptions of model structure would improve the comparability and replicability of model-based outcomes across studies. Further efforts to improve comparability and replicability of models can be informed by open-source modeling efforts [34, 43,44,45]. All inputs should be consistent with modeling assumptions, clinical evidence, and country-specific healthcare system dynamics. For example, if a lifetime horizon is chosen, costs and outcomes related to subsequent treatment should be included. Inputs such as drug price should include both the appropriate price that is paid by public and/or private payers depending on the perspective chosen [46]. Negative inputs such as adverse events and the cost of those adverse events for active comparators and the standard of care should be included in all model analyses. Additionally, utility inputs should be consistent across health states and treatments unless there is evidential support for them to vary. In other words, health states such as PFS or progressed disease should have the same mean starting utility across therapies, and any reductions in utility would come from tolerability or safety considerations.

Recommendations specific to PARP inhibitor evaluations are related to outcomes, model validation, uncertainty, and conclusions on value. Generic outcomes (QALYs) and cancer-specific outcomes (PF LYs) should both be included to increase comparability across past and future studies. Model validation should follow good research practice guidelines that include both internal and external validation [47]. Where uncertainty exists in the relationships between PFS and OS, it is important to validate modeled OS at relevant time intervals against RCT and/or real-world data. Finally, conclusions from the cost effectiveness of PARP inhibitors should align with uncertainty in the evidence. We recommend two paths, given that mature OS data are not available: attempt multiple uncertainty analyses that link different assumptions between PFS and OS; and/or estimate clinical benchmarks on OS that would meet commonly cited cost-effectiveness thresholds.

References

  1. 1.

    Center for Disease Control and Prevention, US Department of Health and Human Services. Ovarian cancer. Inside knowledge about gynecologic cancer 99-9124. 2019. https://www.cdc.gov/cancer/ovarian/pdf/ovarian_facts.pdf. Accessed June 2019.

  2. 2.

    Siegel RL, Miller KD, Jemal A. Cancer statistics, 2019. CA Cancer J Clin. 2019;69(1):7–34.

    Article  Google Scholar 

  3. 3.

    National Cancer Institute. BRCA mutations: cancer risk and genetic testing. 2018. https://www.cancer.gov/about-cancer/causes-prevention/genetics/brca-fact-sheet#how-much-does-having-a-brca1-or-brca2-gene-mutation-increase-a-womans-risk-of-breast-and-ovarian-cancer. Accessed July 2019.

  4. 4.

    O’Connor MJ. Targeting the DNA damage response in cancer. Mol Cell. 2015;60(4):547–60.

    PubMed  Article  Google Scholar 

  5. 5.

    US FDA. FDA approves olaparib tablets for maintenance treatment in ovarian cancer. 2017. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-olaparib-tablets-maintenance-treatment-ovarian-cancer. Accessed June 2019.

  6. 6.

    Moore K, Colombo N, Scambia G, Kim BG, Oaknin A, Friedlander M, et al. Maintenance olaparib in patients with newly diagnosed advanced ovarian cancer. N Engl J Med. 2018;379(26):2495–505.

    CAS  Article  Google Scholar 

  7. 7.

    Ledermann J, Harter P, Gourley C, Friedlander M, Vergote I, Rustin G, et al. Olaparib maintenance therapy in platinum-sensitive relapsed ovarian cancer. N Engl J Med. 2012;366(15):1382–92.

    PubMed  PubMed Central  CAS  Article  Google Scholar 

  8. 8.

    Pujade-Lauraine E, Ledermann JA, Selle F, Gebski V, Penson RT, Oza AM, et al. Olaparib tablets as maintenance therapy in patients with platinum-sensitive, relapsed ovarian cancer and a BRCA1/2 mutation (SOLO2/ENGOT-Ov21): a double-blind, randomised, placebo-controlled, phase 3 trial. Lancet Oncol. 2017;18(9):1274–84.

    PubMed  PubMed Central  CAS  Article  Google Scholar 

  9. 9.

    Ray-Coquard I, Pautier P, Pignata S, Perol D, Gonzalez-Martin A, Berger R, et al. Olaparib plus bevacizumab as first-line maintenance in ovarian cancer. N Engl J Med. 2019;381(25):2416–28.

    PubMed  CAS  Article  Google Scholar 

  10. 10.

    National Comprehensive Cancer Network. Ovarian cancer (version 2.2019). https://www.nccn.org/professionals/physician_gls/pdf/ovarian.pdf. Accessed Oct 2019.

  11. 11.

    Gonzalez-Martin A, Pothuri B, Vergote I, DePont Christensen R, Graybill W, Mirza MR, et al. Niraparib in patients with newly diagnosed advanced ovarian cancer. N Engl J Med. 2019;381(25):2391–402.

    PubMed  CAS  Article  Google Scholar 

  12. 12.

    Institute for Clinical and Economic Review. Poly ADP-ribose polymerase (PARP) inhibitors for ovarian cancer: effectiveness and value. 2017. https://icer-review.org/wp-content/uploads/2017/02/MWCEPAC_OVARIAN_FINAL_EVIDENCE_REPORT_10112017-1.pdf. Accessed June 2019.

  13. 13.

    McQueen RB, Whittington MD, Chapman RH, Kumar VM, Campbell JD. Comment on “Cost-effectiveness of niraparib versus routine surveillance, olaparib and rucaparib for the maintenance treatment of patients with ovarian cancer in the United States”. PharmacoEconomics. 2019;37(7):963–4.

    PubMed  Article  Google Scholar 

  14. 14.

    Wallace K, Goble S, Isaacson J, Maloney L, Cameron T, Bedel J. Comment on: “Cost-effectiveness of niraparib versus routine surveillance, olaparib and rucaparib for the maintenance treatment of patients with ovarian cancer in the United States”. PharmacoEconomics. 2019;37(8):1065–7.

    PubMed  PubMed Central  Article  Google Scholar 

  15. 15.

    Ofman JJ, Sullivan SD, Neumann PJ, Chiou C-F, Henning JM, Wade SW, et al. Examining the value and quality of health economic analyses: implications of utilizing the QHES. J Manag Care Pharm. 2003;9(1):53–61.

    PubMed  Google Scholar 

  16. 16.

    Alvarez-Secord A, Barnett J, Ledermann J, Peterson B, Myers E, Havrilesky LJ. Cost-effectiveness of homologous recombination defect testing to target PARP inhibitor use in platinum-sensitive recurrent ovarian cancer. Gynecol Oncol. 2012;125(Suppl 1):S92–3.

    Article  Google Scholar 

  17. 17.

    Alvarez-Secord A, Barnett JC, Ledermann JA, Peterson BL, Myers ER, Havrilesky LJ. Cost-effectiveness of BRCA1 and BRCA2 mutation testing to target PARP inhibitor use in platinum-sensitive recurrent ovarian cancer. Int J Gynecol Cancer. 2013;23(5):846–52.

    Article  Google Scholar 

  18. 18.

    Chin LM, Carlson JJ. Cost-effectiveness analysis of rucaparib maintenance treatment for recurrent ovarian carcinoma after response to platinum therapy. Value Health. 2018;21(Suppl 1):S30.

    Article  Google Scholar 

  19. 19.

    Dottino JA, Moss HA, Lu KH, Alvarez-Secord A, Havrilesky LJUS. Food and drug administration-approved poly(ADP-ribose) polymerase inhibitor maintenance therapy for recurrent ovarian cancer: a cost-effectiveness analysis. Obstet Gynecol. 2019;133(4):795–802.

    PubMed  PubMed Central  CAS  Article  Google Scholar 

  20. 20.

    Dottino JA, Moss HA, Lu KH, Alvarez-Secord A, Havrilesky LJ. Are FDA-approved PARPi cost-effective as maintenance treatment of platinum-sensitive recurrent ovarian cancer? Gynecol Oncol. 2018;149(Suppl 1):11–2.

    Article  Google Scholar 

  21. 21.

    Fernandez-Santos JM, Andreas GS. Relapsed ovarian cancer treatment costs comparison in Spain. Value Health. 2017;20(Suppl 5):A100.

    Google Scholar 

  22. 22.

    Guy H, Walder L, Fisher M. Cost-effectiveness of niraparib versus routine surveillance, olaparib and rucaparib for the maintenance treatment of patients with ovarian cancer in the United States. PharmacoEconomics. 2019;37(3):391–405.

    PubMed  Article  Google Scholar 

  23. 23.

    Liu AY, Cohen JG, Walsh CS, Holschneider CH, Sinno AK. A cost-effectiveness analysis of three PARP inhibitors for maintenance therapy in platinum-sensitive recurrent ovarian cancer. Gynecol Oncol. 2018;149(Suppl 1):9.

    Article  Google Scholar 

  24. 24.

    Liu AY, Cohen JG, Walsh CS, Holschneider CH, Sinno AK. A cost-effectiveness analysis of three PARP inhibitors for maintenance therapy in platinum-sensitive recurrent ovarian cancer. Gynecol Oncol. 2017;147(1):196.

    Article  Google Scholar 

  25. 25.

    Mylonas C, Aravantinos G, Bamias A, Kourlaba G, Maniadakis N. Economic evaluation of olaparib in patients with BRCA-mutated Psroc in Greece. Value Health. 2016;19(Suppl 7):A732.

    Article  Google Scholar 

  26. 26.

    Smith HJ, Walters Haygood CL, Arend RC, Leath CA 3rd, Straughn JM Jr. PARP inhibitor maintenance therapy for patients with platinum-sensitive recurrent ovarian cancer: a cost-effectiveness analysis. Gynecol Oncol. 2015;139(1):59–62.

    PubMed  CAS  Article  Google Scholar 

  27. 27.

    Smith HJ, Walters Haygood CL, Arend RC, Leath CA III, Straughn JM Jr. PARP inhibitors as maintenance therapy for patients with platinum-sensitive recurrent ovarian cancer: can we afford it? Gynecol Oncol. 2015;137(Suppl 1):9.

    Article  Google Scholar 

  28. 28.

    Wolford JE, Bai J, Eskander RH, Keller R, Minion L, Chan JK, et al. A Markov model to evaluate cost-effectiveness of the PARP inhibitor, olaparib, for fourth-line treatment of recurrent ovarian cancer. J Clin Oncol. 2016;34(15 Suppl):5563.

    Article  Google Scholar 

  29. 29.

    Wolford JE, Bai J, Eskander RH, Keller R, Minion LE, Chan JK, et al. Evaluating the cost-effectiveness of current FDA-approved PARP inhibitors for the treatment of recurrent ovarian cancer. J Clin Oncol. 2017;35(15 Suppl):5516.

    Article  Google Scholar 

  30. 30.

    Wolford JE, Bai J, Minion LE, Keller R, Eskander RN, Chan JK, et al. Cost-effectiveness of maintenance therapy in advanced ovarian cancer: paclitaxel, bevacizumab, niraparib, rucaparib, olaparib, and pembrolizumab. J Clin Oncol. 2018;36(15 Suppl):5508.

    Article  Google Scholar 

  31. 31.

    Wolford JE, Liang SY, Tewari KS, Eskander RN, Kapp DS, Mann AK, et al. An economic analysis: examining the cost-effectiveness of bevacizumab and olaparib as upfront maintenance treatment of advanced ovarian cancer. Gynecol Oncol. 2019;154(Suppl 1):25.

    Article  Google Scholar 

  32. 32.

    Wolford JE, Tewari KS, Liang SY, Bai J, Mann AK, Moore KN, et al. SOLO1 versus SOLO2: cost-effectiveness of olaparib as maintenance therapy for newly diagnosed and platinum-sensitive recurrent ovarian carcinoma among women with germline BRCA mutations (gBRCAmut). J Clin Oncol. 2019;37(15 Suppl):5545.

    Article  Google Scholar 

  33. 33.

    Zhong L, Tran AT, Tomasino T, Nugent E, Smith JA. Cost-effectiveness of niraparib and olaparib as maintenance therapy for patients with platinum-sensitive recurrent ovarian cancer. J Manag Care Spec Pharm. 2018;24(12):1219–28.

    PubMed  Google Scholar 

  34. 34.

    Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Med Decis Mak. 2012;32(5):733–43.

    Article  Google Scholar 

  35. 35.

    Gutman SI, Piper M, Grant MD, Basch E, Oliansky DM, Aronson N. AHRQ methods for effective health care. Progression-free survival: what does it mean for psychological well-being or quality of life?. Rockville: Agency for Healthcare Research and Quality; 2013.

    Google Scholar 

  36. 36.

    Davis S, Tappenden P, Cantrell A. NICE decision support unit methods development. A review of studies examining the relationship between progression-free survival and overall survival in advanced or metastatic cancer. London: National Institute for Health and Care Excellence; 2012.

    Google Scholar 

  37. 37.

    Ledermann JA, Harter P, Gourley C, Friedlander M, Vergote I, Rustin G, et al. Overall survival in patients with platinum-sensitive recurrent serous ovarian cancer receiving olaparib maintenance monotherapy: an updated analysis from a randomised, placebo-controlled, double-blind, phase 2 trial. Lancet Oncol. 2016;17(11):1579–89.

    PubMed  CAS  Article  Google Scholar 

  38. 38.

    National Institute for Health and Care Excellence. Multivariate meta-analysis of summary data for combining treatment effects on correlated outcomes and evaluating surrogate endpoints (NICE DSU Technical Support Document 20). 2019. http://nicedsu.org.uk/wp-content/uploads/2019/10/TSD-20-mvmeta-final.pdf. Accessed Oct 2019.

  39. 39.

    Lakdawalla DN, Doshi JA, Garrison LP Jr, Phelps CE, Basu A, Danzon PM. Defining elements of value in health care-a health economics approach: an ISPOR special task force report [3]. Value Health. 2018;21(2):131–9.

    PubMed  Article  Google Scholar 

  40. 40.

    Institute for Clinical and Economic Review. Value assessment framework for 2020–2023. 2019. https://icer-review.org/wp-content/uploads/2019/05/ICER_2020_2023_VAF_013120-4.pdf. Accessed July 2019.

  41. 41.

    Wolford JE, Bai J, Moore KN, Kristeleit R, Monk BJ, Tewari KS. Cost-effectiveness of niraparib, rucaparib, and olaparib for treatment of platinum-resistant, recurrent ovarian carcinoma. Gynecol Oncol. 2020;157(2):500–7.

    PubMed  PubMed Central  CAS  Article  Google Scholar 

  42. 42.

    Hettle R, Posnett J, Borrill J. Challenges in economic modeling of anticancer therapies: an example of modeling the survival benefit of olaparib maintenance therapy for patients with BRCA-mutated platinum-sensitive relapsed ovarian cancer. J Med Econ. 2015;18(7):516–24.

    PubMed  Article  Google Scholar 

  43. 43.

    Carlson JJ, Walton SM, Basu A, Chapman RH, Campbell JD, McQueen RB, et al. Achieving appropriate model transparency: challenges and potential solutions for making value-based decisions in the United States. PharmacoEconomics. 2019;37(11):1321–7.

    PubMed  PubMed Central  Article  Google Scholar 

  44. 44.

    Sampson CJ, Arnold R, Bryan S, Clarke P, Ekins S, Hatswell A, et al. Transparency in decision modelling: what, why, who and how? PharmacoEconomics. 2019;37(11):1355–69.

    PubMed  Article  Google Scholar 

  45. 45.

    Wu EQ, Zhou ZY, Xie J, Metallo C, Thokala P. Transparency in health economic modeling: options, issues and potential solutions. PharmacoEconomics. 2019;37(11):1349–54.

    PubMed  Article  Google Scholar 

  46. 46.

    Mattingly TJ, Levy JF, Slejko JF, Onwudiwe NC, Perfetto EM. Estimating drug costs: how do manufacturer net prices compare with other common US price references? PharmacoEconomics. 2018;36(9):1093–9.

    PubMed  PubMed Central  Article  Google Scholar 

  47. 47.

    Weinstein MC, O’Brien B, Hornberger J, Jackson J, Johannesson M, McCabe C, et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices—Modeling Studies. Value Health. 2003;6(1):9–17.

    PubMed  Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to R. Brett McQueen.

Ethics declarations

Author Contributions

Two investigators (WG and RBM) performed the initial search and reviewed each article for quality using the QHES tool. If the quantitative score differed between reviewers, all co-authors (DM, MM, KM, RH, ES, ES, SZ, IK, and JS) discussed the score in order to reach a consensus. WG and RBM prepared the first draft of the manuscript. DM, MM, KM, RH, ES, ES, SZ, IK, and JS provided written text in the manuscript and edited all versions of the manuscript.

Disclosures

This study was supported by AstraZeneca and Merck Sharp & Dohme Corp (MSD), a subsidiary of Merck & Co, Inc, Kenilworth, NJ, USA. Dominic Muston, Matthew Monberg, and Elizabeth Szamreta are employees of Merck & Co., Inc.; Wei Gao, Elyse Swallow, Su Zhang, Iden Kalemaj, and James Signorovitch are employees of Analysis Group; and Kimmie McLaurin and Robert Hettle are employees of AstraZeneca. Robert Brett McQueen received consulting fees from Analysis Group to develop recommendations for cost-effectiveness studies of PARP inhibitors in advanced ovarian cancer.

Data Sharing Statement

Please see Appendix 1, Appendix 2, and Appendix 3 in the Electronic Supplementary Material for the search strategy inclusion criteria, and the full data extraction grid.

Electronic supplementary material

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gao, W., Muston, D., Monberg, M. et al. A Critical Appraisal and Recommendations for Cost-Effectiveness Studies of Poly(ADP-Ribose) Polymerase Inhibitors in Advanced Ovarian Cancer. PharmacoEconomics 38, 1201–1218 (2020). https://doi.org/10.1007/s40273-020-00949-9

Download citation