FormalPara Key Points for Decision Makers

Published budget-impact analyses do not always estimate changes in the treated population size and related changes in the disease-related costs of drugs for chronic conditions that reduce mortality rates or slow disease progression, which may result in over- or underestimates of the budget impact of a new drug.

Treatment switching or discontinuation may not be appropriately included in many published budget-impact analyses.

Sensitivity and scenario analyses presented in published budget-impact analyses are typically too limited to allow a budget holder to assess the likely budget impact for their health plan.

1 Introduction

Decisions about the reimbursement and use of new drugs take place in an environment where there is an aging population and new healthcare technologies with increased efficacy but often high prices. At the same time, government and private payer decision makers are increasingly concerned about healthcare expenditures. Given this environment, before healthcare decision makers recommend reimbursement of a new drug in many jurisdictions, they review its economic impact in addition to its safety and efficacy. There are two key components of a comprehensive evaluation of the economic impact for a new drug: (1) a cost-effectiveness analysis, which estimates the incremental costs and benefits of the new drug compared with the current standard of care for the condition of interest for the period over which changes in costs and benefits are expected; and (2) a budget-impact analysis, which forecasts the treatment shares for the new drug and changes in rates of use of all current treatments when the new drug is added to the treatment mix and the associated effects on disease outcomes, resource use, and costs [1].

Budget-impact analyses are used by healthcare decision makers either before adding a new drug to the formulary to determine its affordability given budget constraints or as a tool to use once a new drug has been added to the formulary to determine by how much annual budgets are likely to increase and to plan for these changes. Some health technology assessment (HTA) agencies make reimbursement recommendations based on the predicted budget impact of a new drug. For example, an analysis of reimbursement recommendations by the Pharmaceutical Benefits Advisory Committee (PBAC) indicated a correlation between the predicted budget impact and whether restrictions for use were recommended or the submission was rejected [2]. In contrast, the National Institute for Health and Care Excellence (NICE) makes its reimbursement recommendations based on unmet need, efficacy, safety, and cost effectiveness. NICE then provides regional decision makers in the UK with budget-impact analyses (a ‘costing template’), programmed using Microsoft Excel® spreadsheets and posted on a public website, so that each region can determine the impact of the new drug reimbursement recommendation for their annual healthcare expenditures [3]. A third use for a budget-impact analysis for a decision maker is to provide the basis for requests for increased funding either from public funds or through increased insurance premiums. For example, the US AIDS Drug Assistance Programs used budget-impact analysis to support requests for additional funding from state governments in the USA when new HIV drugs were introduced to the market during the 1990s [4]. Budget-impact analyses for any of these uses have been developed by HTA agencies, manufacturers and their consultants, and healthcare budget holders.

Many jurisdictions worldwide have developed guidelines for performing budget-impact analyses, including the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) [5, 6], Canada [7], Australia [8], Poland [9], the USA [10], England and Wales [11], Belgium [12, 13], and Brazil [14]. Some of these guidelines provide jurisdiction-specific preferences for model design, assumptions, and input data sources [7, 8, 11], while others provide more general guidance on the framework for estimation and preferred model design and input data source categories [6, 9, 10]. In addition to formal guidelines, several publications have presented recommendations for the design of budget-impact models over the last 20 years [15–18].

Three published reviews of budget-impact analyses have assessed the quality of published budget-impact analyses. Mauskopf et al. [19] noted the limited number of published studies. They reviewed ten studies from multiple countries and found that the methods used in the studies were variable; they concluded that a comprehensive and standardized approach was needed. In a review by Orlewska and Gulácsi [20], 34 published studies from multiple countries were included, 18 of which were for drugs. The authors concluded that there was fairly good agreement with published guidelines for most elements, with the exceptions of reporting, uncertainty analysis, and discounting. Most recently, van de Vooren et al. [21] reviewed 17 published budget-impact analyses for drug introductions in European Union countries. They concluded from this review that budget-impact analysis is not a well-established technique and many studies are not of acceptable quality. In their review, quality was assessed according to adherence to the latest published criteria for what should be included in a budget-impact analysis.

In this review, we focus on budget-impact analyses published for drugs in the USA. We present seven key elements of budget-impact analysis study design and describe recommended practice for each of these elements as proposed in published guidelines and methods studies. We then review the US studies identified in a targeted literature search to assess whether their design was consistent with our recommended-practice definition for each of the key elements of budget-impact model design.

2 Methods

Based on published guidelines for budget-impact analysis and other published methodological studies [5–18], we identified seven key elements in the design of a budget-impact analysis and described recommended practice for completing each element. We then performed targeted keyword searches from 1990 through to 16 October 2015, of MEDLINE, EMBASE, EconLit, and the Cochrane Library for US studies reporting estimates of the budget impact of the introduction of a new drug to the treatment mix. The search keywords used for the searches are shown in the Electronic Supplementary Material. The titles and abstracts identified in these searches were screened to identify articles likely to include budget-impact analyses of adding a new drug to the treatment mix in US health plans. After the initial screen, full-text articles were obtained for the selected studies and screened a second time to ensure that they met the inclusion criteria. The final set of selected budget-impact analyses was subdivided into three categories: drugs for acute diseases, drugs for chronic diseases, and combined cost-effectiveness and budget-impact analyses. For each category, we prepared a table presenting a summary of how each study addressed the seven key elements and a qualitative text synthesis showing the extent to which the key elements were not included or recommended practice was not followed.

2.1 Key Elements in a Budget-Impact Model

In our review, we focused on seven key elements in the design of a budget-impact analysis: model structure, population size and characteristics, time horizon, treatment mix, treatment costs, disease-related costs, and uncertainty analysis.

2.1.1 Model Structure

The first key element, the model structure, depends on the disease and treatments being considered and can be a simple cost calculator where the costs with and without the new drug in the treatment mix are calculated using steady-state inputs and simple formulas. Alternatively, when changes in treatment sequences and/or treatment-related or disease-related outcomes are anticipated within the model time horizon, a more complex, decision-analytic model structure can be used, such as decision trees, cohort or patient-based Markov models, or discrete-event simulation models. The ISPOR guidelines recommend using the simplest model structure that will provide credible estimates of the budget impact of adding the new drug to the formulary [6].

2.1.2 Population Size and Characteristics

One of the most important elements in a budget-impact analysis is the estimate of the population size and mix of disease severity or other characteristics such as the age and sex of patients currently being treated for the disease and who will be eligible for treatment with the new drug. If the new drug slows or reverses disease progression or reduces mortality within the model time horizon, then its impact on the treated population size and disease severity mix should also be estimated directly or be otherwise accounted for in the budget-impact model estimates. This is because changes in the number of individuals treated and/or in the number of individuals with a specific disease severity will impact the payer’s budgets. Estimates are also needed of currently untreated patients who might decide to seek treatment when the new drug is added to the formulary, thus increasing the size of the treated population. For chronic conditions, the population estimates should also account for those who became eligible for treatment with the new drug in previous years (prevalent cases) who might switch to the new treatment as well as those newly diagnosed and/or newly eligible for treatment (incident cases) who might enter the treated population during the model time horizon.

2.1.3 Time Horizon

The third key element, the time horizon for the budget-impact analysis, is generally chosen based on the financial information requirements of the budget holder and is not related to the duration of the disease for which the new drug is indicated. Thus, the time horizons for an acute disease and a chronic disease might be the same. However, the duration of the illness determines the number of people being treated at any one time. Time horizons are typically short, from 3 to 5 years, because of short planning horizons by the budget holders. These short time horizons do not allow for estimates of changes in health outcomes and associated costs that might only occur over longer time horizons when the full benefit of a new drug might be experienced. Model time horizons that correspond to a ‘steady state’ after introduction of the new drug could be included in a budget-impact analysis, as recommended in the Belgian guidelines [13]. Costs are presented for each year of the model time horizon separately and are not discounted, since budgets are determined for each year in nominal currency.

2.1.4 Treatment Mix

The fourth key element in a budget-impact analysis is the mix of treatments currently used for the indicated and eligible population, and the predicted change in the treatment mix if the new drug is added to the formulary. The predicted change in the treatment mix depends both on the uptake of the new drug each year over the model time horizon as well as whether the new drug is added to current treatments or replaces them. If the new drug replaces a current treatment, credible assumptions about the treatments from which treatment share is taken must be made. The resulting budget impact is likely to be sensitive to these assumptions. For example, the budget impact will be higher if the treatment share for the new drug is taken from generic drugs rather than from branded drugs or surgical procedures.

2.1.5 Treatment Costs

The fifth key element, the costs associated with drug treatment, may include acquisition, administration, monitoring, and adverse event costs, and all should be included in the budget-impact analysis or a rationale for their exclusion provided. The costs of interest to budget holders might differ according to their perspectives. A published budget-impact analysis should include all of the cost categories and present them separately in the publication to provide information that is useful for all perspectives. For the USA, the analysis should also be designed so that discounts, co-insurance, and co-payments can be subtracted from the drug acquisition costs to provide an estimate of the budget holder’s expected costs.

2.1.6 Disease-Related Costs

The sixth key element, an estimate of the impact of the new drug on other disease-related costs, is not always included in budget-impact analyses. For example, these costs should not be included if the impact on disease-related costs will not occur during the model time horizon. If included, they should be based on clinical trial or observational data. These costs are simple to include for acute conditions and for chronic conditions when the full impact on disease-related costs can be assumed to occur immediately and stay constant over time. Where changes in disease-related costs for a chronic illness occur more gradually over the model time horizon, changes in disease-related costs may be estimated by running a disease-progression model (similar to that used to estimate the cost-effectiveness ratios) in ‘prevalence’ mode, where the model tracks the prevalent (those already with the disease of interest) and incident or newly treated cohorts over the model time horizon.

2.1.7 Uncertainty Analysis

For any budget-impact analysis, different decision makers might vary in their agreement with the model assumptions and in their perception of the input parameter values that are most relevant for their health plan. Most budget-impact models are developed as interactive computer programs that allow decision makers to change key assumptions and input values to reflect their health plan. However, when budget-impact models are published, this flexibility for the user is not available. The recommended approach in a published model, and our seventh key element, is to present a series of one-way sensitivity analyses and/or scenario analyses that might be of interest to different decision makers. The values tested in these analyses should include alternative values for inputs that might vary across health plans but be known with certainty by the budget holder, as well as inputs for which the values are not known with certainty. The latter inputs include those for which there are known uncertainty bounds (e.g., efficacy from a clinical trial) as well as those for which the uncertainty bounds are unknown (e.g., future estimates of new drug uptake). The alternative values tested should be either data driven or based on likely ranges for each parameter. The impacts on the results of input parameters for which there are known uncertainty bounds are especially important to include in the sensitivity or scenario analyses. The use of the same arbitrary percentage range (e.g., ±20 %) for all parameters is generally not recommended practice. Most budget-impact guidelines do not recommend a probabilistic sensitivity analysis, with one exception being the Belgian guidelines [13].

3 Results

After completing the keyword searches and the screening of titles and abstracts of 230 unique records and then the screening of 79 selected full-text articles, we identified eight budget-impact analyses for drugs for acute conditions, 27 budget-impact analyses for drugs for chronic conditions, and ten combined cost-effectiveness and budget-impact analyses for drugs either for acute or chronic conditions in the USA.

3.1 Acute Conditions

Table 1 presents the seven key elements of the design of a budget-impact model included in the eight budget-impact analyses for acute conditions. Of the eight analyses, four used a simple cost-calculator model structure [22–25] and four used more complex decision-modeling techniques [26–29]. The cost-calculator approach is appropriate for acute conditions. However, the use of the decision models in four studies allowed the inclusion of titration, augmentation, or switching to a second-line treatment explicitly in the analysis when the first-line drug was not effective.

Table 1 Acute diseases
Full size table

All eight models used an incident population that they correctly assumed to remain constant over time since none of the new drugs were estimated to change the number of people with the condition or to impact disease progression. Three of the models [23, 24, 29] estimated the incident population starting from the population for a hypothetical health plan and applying incidence and diagnosis rates. Four models for inpatients used estimated or hypothetical numbers of inpatients with the specific condition [22, 25, 27, 28], and one model just assumed a hypothetical number of patients with the condition of interest who might be treated on an inpatient or outpatient basis [26].

Since these studies all focused on acute conditions, the time horizon assumed was 1 year, implicitly making the assumption that the budget impact would be the same each year.

Six of the eight studies compared the current treatment mix with a new treatment mix including the new drug. In these studies, the new drug took treatment share from all drugs [24, 28], one drug [25, 29], or a subset of the drugs [26, 27] in the current treatment mix. One study [23] just compared two of the three drugs used for the indication with each other, and a second study [22] compared a combination regimen made up of a current drug plus a mix of backbone treatments with the new drug plus a different mix of backbone treatments.

Only one of the eight studies [28] included the four drug cost categories of acquisition, administration, monitoring, and adverse events. One study included acquisition, administration, and monitoring [27]; three studies included acquisition and adverse events [22, 25, 29]; one study included acquisition and administration [23]; one study included acquisition and monitoring [26]; and one study included acquisition only [24]. The studies that did not include administration costs were either for oral drugs [24, 26, 29] or reported no difference in these costs [22, 25]. Only four studies included adverse events.

The eight studies varied as to whether they included changes in disease-related costs. Three of the studies did not include these costs [23–25], with two of them stating that they were excluded because there would be no difference in disease outcomes [23, 25] and the third hospital-based study assuming days on treatment did not change with addition of the new drug regimen [24]. Three of the four hospital-based studies included changes in costs attributable to changes in hospital length of stay [22, 27, 28], with one study also including hospital procedures [22]. Two of the decision-analytic models included changes in disease-related costs [26, 29].

Uncertainty analyses were variable among the studies. Seven studies included one-way sensitivity analyses [22–26, 28, 29], three studies included scenario analyses [25–27], and one study [22] included a probabilistic sensitivity analysis, which is not generally recommended for budget-impact analyses. The one-way sensitivity analyses used either arbitrary percentage ranges, data-driven ranges, or a mix of the two. In addition, the three studies including scenario analyses were variable in the extent to which alternative scenarios were tested, limiting the value of these studies to budget holders with different patient populations and practice patterns.

3.2 Chronic Conditions

Table 2 presents the seven key elements of the design of a budget-impact model for the 27 budget-impact analyses for chronic conditions. The table presents these published studies in two main sections: those analyses where the clinical data indicated that the new drug would not change mortality or disease progression rates within the model time horizon (17 studies) and those analyses where clinical data indicated that such changes would take place (ten studies).

Table 2 Chronic disease
Full size table

3.2.1 Constant Mortality or Disease Progression Rates

Of the 17 analyses in which the new drug would not change the mortality or disease progression rates within the model time horizon, 12 used a simple cost-calculator model structure [30–41] and five used formal decision-analytic model structures [42–46]. The cost-calculator approach can be appropriate for chronic conditions when the mortality and/or disease progression rates remain constant, and this approach generally has the advantage of being simpler and more transparent for the user. However, the use of a decision-analytic model structure allows the user to account for switching, titration, and discontinuation explicitly in the analysis when the first-line drug was not effective or tolerated.

All 12 models that used a static cost-calculator or database approach used a prevalent population who had the condition for which the new drug was indicated. Only one of the studies [31] allowed the number being treated to increase after the addition of the new drug. With a new effective drug added to the formulary, more patients with the disease might enter treatment, which could increase the budget impact of the new drug. The cost-calculator models varied according to whether diagnosed and treated prevalence rates were applied to the US population [38]; a hypothetical health plan [30–33, 35, 36, 39, 40]; and specific populations, including a prison population [34], people filling prescriptions for glaucoma lipid therapy [37], or people with acute myeloid leukemia [41].

Three of the five studies that used a decision-analytic model structure used only an incident population, those newly starting treatment in the first year of the study, and did not consider whether any patients currently being treated would also switch to the new drug [42, 44, 45]. Of these three, only one model included a new incident cohort starting treatment each year of the model time horizon [42]. The other two models followed a single incident cohort over multiple years [44, 45]. Following only a single incident cohort and/or not accounting for a prevalent population (those already with the indication) does not provide the decision maker with an accurate annual budget-impact estimate for all those being treated for the condition and may underestimate the budget impact. One decision-analytic model did include both multiple incident cohorts and the prevalent population for the 3-year model time horizon [46]. The fifth decision-analytic model [43] estimated the treated prevalent population size and used that as a cohort to follow for 7 years using a Markov model. This approach allows those already being treated with the current drugs to switch to the new drug but assumes that the number of people being treated each year is constant and does not allow efficacy to change with time on treatment.

Although these studies focused on chronic conditions, the time horizon assumed was 1 year in seven of the 12 cost-calculator models [31, 32, 34, 37, 38, 40, 41] and only 6 months in one of the cost-calculator models using data from a database analysis [30], implicitly making the assumption that the budget impact would be the same for subsequent periods. In the other four cost-calculator models [33, 35, 36, 39] and the five decision-analytic models [42–46], time horizons varied from 2 to 7 years.

Thirteen of the 17 studies compared the current or old treatment mix for the population of interest with at least one new treatment mix including the new drug [32–35, 37, 38, 40–42, 44–46]. In these studies, the drug for which the budget impact was estimated took treatment share from all drugs (six studies) [33, 35, 36, 42, 45, 46] or one drug (four studies) [31, 38, 40, 43]. In one study, the drug was added to current therapy [39], and the treatment shares before and after introduction of a new drug were taken from a database analysis in two studies [30, 32]. Two of the studies that took treatment shares from all drugs assumed equi-proportionate contributions of the current drugs to the treatment share for the new drug [36, 42]. In addition, the base case in the Berenson et al. [43] study compared treatment with the drug of interest for all patients to treatment with one of two other drugs for all patients. Treatment mix with the three drugs was only considered in a scenario analysis.

Of the other four studies that did not consider treatment mixes, one study [34] compared no treatment with a mix of new drugs. A second study [44] compared no treatment with several alternative drugs considered one at a time, a third study [37] compared treatment with one current drug with treatment with the new drug, and a fourth study [41] compared different dosing for a single drug with and without genotyping.

None of the 17 studies included the four drug cost categories of acquisition, administration, monitoring, and adverse events. Ten of the studies included acquisition only [30, 32–35, 37, 38, 40, 42, 43]; two studies included acquisition, administration, and monitoring [31, 44]; one study included acquisition, adverse events, and monitoring [36]; two studies included acquisition and administration [39, 46]; and two studies included acquisition and monitoring [41, 45]. Of those studies that did not include administration costs, nine were studies of oral drugs [30, 32–35, 38, 40, 42, 45] and three were studies with the same administration method [37, 41, 43], but one study [36] compared a subcutaneously administered drug with oral drugs and should have included administration costs. Only one study included adverse events [36], and five studies included monitoring [31, 36, 41, 44, 45].

The 17 studies varied as to whether they included changes in disease-related costs. Three of the studies did not include these costs [31, 34, 37]. In one of these studies (for pulmonary hypertension [31]), changes in disease outcomes were expected within the model time horizon based on the clinical data cited in the article. As a result, these should have been included in the budget-impact analysis. In the other two studies, changes in clinical outcomes would most likely occur beyond the model time horizon (for chronic hepatitis C [34] and glaucoma lipid therapy [37]). The other 14 studies, including the five decision-analytic models, did include changes in disease-related costs either in the base case or in scenario analysis [30, 32, 33, 35, 36, 38–46]. In one cost-calculator model for Lennox–Gastaut syndrome [35], the patients’ disease was assumed to remain uncontrolled after discontinuation, with lower treatment costs but higher disease-related costs. In practice, these patients would likely be switched to another treatment, with the associated costs and benefits.

Uncertainty analyses were variable among the studies, with 14 studies including one-way sensitivity analyses [30–34, 36, 39–46], nine studies including scenario analyses [32, 34–36, 38, 39, 44–46], and one study not including any uncertainty analyses [37]. The one-way sensitivity analyses used either arbitrary percentage ranges, data-driven ranges, or a mix of the two. In addition, the nine studies including scenario analyses were variable in the extent to which alternative scenarios were tested, with three of the studies including best-case and worst-case scenarios only [32, 38, 39] and one study including disease stage only.

3.2.2 Decreased Mortality or Disease Progression Rates

For ten of the models of chronic illness, the clinical data indicated that a decrease in mortality or disease progression rates during the model time horizon would be expected with use of the new drug [47–56]. Eight of these models used a simple cost-calculator model structure [47–54], and only two used a decision-analytic model structure [55, 56]. Seven of the eight cost-calculator models were for advanced or metastatic cancers [47–51, 53, 54], and one was for congestive heart failure [52]. The two decision-analytic models were for prevention of cardiovascular disease [56] or localized cancer [55].

All of the models estimated the eligible population size for a hypothetical health plan, although the assumed size of the hypothetical health plan varied widely, ranging from 0.25 to 10 million. Of the eight models that used the static cost-calculator approach, four used a prevalent population with the condition for which the new drug was indicated and assumed no change in population size during the model time horizon [50–53]. The other four models used an incident population capturing only those newly indicated for treatment with the new drug [47–49, 54]. The size of the incident population would not be expected to change for any of the diseases being modeled, but the treated population size would be expected to increase over time with new drugs that decrease mortality or slow disease progression, allowing for a longer time on treatment. However, such changes were not accounted for in these models.

The two studies that used a decision-analytic model structure [55, 56] used only an incident population, those newly starting treatment in the first year of the study, and did not consider whether any patients currently being treated for the same indication would switch to the new drug. Also, only one decision-analytic model [55] included a new incident cohort starting treatment each year of the model time horizon. The other model [56] followed a single incident cohort over multiple years. Neither of these studies provided the decision maker with annual budget-impact estimates for all those being treated for the condition during the model time horizon.

Although these studies all focused on chronic conditions, seven of the eight cost-calculator models assumed a time horizon of 1 year [47–49, 51–54], either assuming that the mortality rates were sufficiently high that no patients would be alive in subsequent years for those with advanced cancer or that the budget impact would be the same in subsequent periods for those with congestive heart failure. In the other cost-calculator model [50] and the two decision-analytic models [55, 56], time horizons varied from 3 to 10 years.

Five of the ten studies compared the current or old treatment mix with a new treatment mix including the new drug [47, 50, 51, 53, 54]. In all five of these studies, the drug for which the budget impact was estimated took treatment share from all drugs. Two studies compared no maintenance or adjuvant therapy in patients with cancer (current standard of care) with maintenance or adjuvant therapy with the new drug [48, 55], one study compared current use of prevention with guideline-recommended use of prevention with low-dose aspirin in patients with cardiovascular disease [56], one study compared current standard of care with the new drug plus current standard of care [52], and one study compared monotherapy with a treatment mix of monotherapy or combination therapy formed by adding the new drug [49]. The comparison used in the Danese et al. [49] study is problematic because the budget-impact model did not include other possible combination therapies in the treatment mix.

Only one of the ten studies included the four drug cost categories of acquisition, administration, monitoring, and adverse events [47]. Two of the studies included only acquisition [50, 52]; four studies included acquisition, administration, and adverse events [48, 49, 51, 53]; two studies included acquisition and adverse events [54, 56]; and one study included acquisition, monitoring, and adverse events [53]. Thus, eight of the ten studies included adverse events [47–49, 51, 53–56]. Monitoring and the associated costs for the treatment of regimen adverse effects may be significant for chemotherapy regimens but were only included in two studies [47, 55]. In the seven cost-calculator models of advanced or metastatic cancer, the longer duration of treatment with more effective drugs because of treatment to progression was captured in the drug acquisition costs [47–51, 53, 54]. Five studies included administration costs [47–49, 51, 53]. Of those that did not include administration costs, four were for oral drugs, but one was for chemotherapy for metastatic breast cancer [50] in which there might have been differences in administration costs among the regimens, but these were not reported.

The ten studies varied as to whether they included changes in disease-related costs. Six of the studies did not include these costs even though changes in disease outcomes were expected within the model time horizon based on the clinical data cited in the articles [47–51, 53]. All six of these studies were in advanced cancer and assumed treatment would occur until progression, per label, or as observed in trials. These studies did not account for the changes in monitoring and symptom-related treatment for treated patients over the model time horizon. These changes could have been captured by increasing the size and changing disease state mix (progressing or stable disease) for the treated population. The other four studies, including the two decision-analytic models, included changes in disease-related costs [52, 54–56]. However, one of the cost-calculator models that included disease-related costs did not include the costs for monitoring and symptom-related treatment of the treated population that increased in size because of the reduction in mortality [52]. Thus, this study likely overestimated the decrease in disease-related costs with the new drug on the formulary.

Uncertainty analyses were very variable among the studies, with nine studies including one-way sensitivity analyses [47–50, 52–56] and seven studies including scenario analyses [48, 50–54]. The one-way sensitivity analyses used either arbitrary percentage ranges, data-driven ranges, or a mix of the two. In addition, the seven studies including scenario analyses were variable in the extent to which alternative scenarios were tested, with most of the studies presenting very few different scenarios.

3.3 Combined Cost-Effectiveness and Budget-Impact Analyses

Table 3 presents the seven key elements of the design of a budget-impact analysis when it is presented alongside a cost-effectiveness model. An overall finding for the ten studies that presented the results of both a cost-effectiveness analysis and a budget-impact model in the same article is that the information provided for the budget-impact model design, assumptions, inputs, and results was sparse and not sufficient to completely characterize the model [57–66]. More detailed information was provided for the cost-effectiveness analysis, some of which was relevant for the budget-impact analysis. However, detailed information on the estimated population size and characteristics and changes in the treatment mix was not provided.

Table 3 Combined cost-effectiveness and budget-impact analyses
Full size table

Four of the studies were for acute events [57–60] and six were for chronic diseases or conditions [61–66]. However, none of the treatments for the chronic conditions were likely to change mortality or disease progression rates during the model time horizon. The model structures used varied: one chronic disease model did not indicate the structure used [61], one model for a chronic condition [63] used a cost calculator, and the other eight studies used a decision-analytic model structure—either decision trees for acute events [57–60] or a Markov or hybrid [52] decision tree and Markov models for chronic diseases [64–66].

The study that used the cost calculator [63] and the study without a model structure specified [61] both used a prevalent treated population and assumed no change in population size during the model time horizon. Of the eight models with a decision-analytic model structure, three acute disease models included either a prevalent population or an incident cohort with the condition for the 1-year time horizon [58–60]. The fourth acute disease model followed a prevalent population for 2 or 3 years, assuming no change in population size [57]. One of the chronic disease studies [66] used a current prevalent population initiating treatment and followed this population for 3 and 10 years. Two chronic disease studies followed a prevalent population that did not change in size or characteristics over time for time horizons of 1 [64] and 5 [62] years. Finally, one of the chronic disease studies using a decision-analytic model structure [65] included estimates of both the treated prevalent population and five incident cohorts for the 5-year model time horizon, which is recommended practice.

The time horizons assumed were 1 year in three of the four acute disease studies, 2 or 3 years in the other acute disease study, and ranged from 1 to 10 years in the chronic condition studies.

Because a combination cost-effectiveness and budget-impact analysis was presented in these ten studies, five of them compared the budget impact of treating all patients with the new drug compared with treating all patients with another drug [57, 60–62], or with the current mix of drugs [65]. One study [66] compared a mix of new drugs with no drug treatment. These comparisons do not provide an estimate of the actual budget impact that will be expected, since they assume that the treatment share for the new drug(s) will be 100 %, which is unrealistic in most cases. Five studies included a comparison of the current mix of treatments with a new mix including the new drug [58, 59, 61, 63, 64]. Two of these studies provided no information about which current drugs the treatment share of the new drug was taken from [58, 64], two studies took the treatment shares for the new drug equi-proportionately from all current drugs [61, 63], and one study took the treatment shares for the new drug from two specific drugs [59].

Only one of the ten studies included the four drug cost categories of acquisition, administration, monitoring, and adverse events [57]. Five studies included only acquisition costs [58, 59, 64–66], two studies included acquisition and administration costs [60, 62], one study included acquisition and monitoring costs [61], and one study included acquisition, administration, and monitoring costs [63]. In particular, only one of the studies included adverse events [59] and a justification for their omission was not provided by the other studies. In addition, of the six studies that did not include administration costs, three were for oral drugs while three were for indications where the treatment mix included both oral and injectable or intravenously administered drugs and so should have included administration costs [61, 64, 66].

Nine of the ten studies included changes in disease-related costs [57–63, 65, 66]. In the one study that excluded these costs in the budget-impact analysis [64], they were included in the cost-effectiveness analysis; however, since this was for diabetes mellitus, the disease-related costs would probably not change within the model time horizon. The other nine studies included changes in disease-related costs based on those estimated for the cost-effectiveness models. However, in the Chhatwal et al. [65] study, it is not clear whether the offsetting disease-related costs estimated for the budget-impact analysis are those that would occur during the model time horizon or in the remaining lifetime of the treated patients, as estimated using the Markov cost-effectiveness model. In the other eight studies, the offsetting disease-related costs included in the models would be expected to occur during the budget-impact model time horizon.

Uncertainty analyses were not included for the budget-impact model in four of the ten studies [59–61, 65]. In the other six studies, only very limited one-way sensitivity or scenario analyses were included [52–64, 66]. Most of the studies included more extensive uncertainty analyses for the cost-effectiveness analyses.

4 Discussion

The primary finding from this review of the design of published budget-impact models is that, despite published guidelines for budget-impact analysis, there is substantial variability in the extent to which these studies use recommended practice as described in our review for the seven key elements of budget-impact model design. A summary of recommended design and common design flaws identified in this review is presented in Table 4. The finding of variability in the inclusion of key design elements was also made in the previous reviews of budget-impact analyses, including the review by Orlewska and Gulácsi [20] and the recent review of budget-impact analyses for European Union countries by van de Vooren et al. [21]. Our review has provided additional details about deviations from recommended practice for each of the seven key elements. Our review recognized that the design for a budget-impact analysis will vary according to the type of disease (i.e., acute or chronic) as well as the likely impact of the new treatment (e.g., whether it changes mortality or disease progression within the model time horizon for a chronic disease).

Table 4 Summary of recommended approaches and review findings for the seven key elements of budget-impact analysis study design
Full size table

An important recommendation from the recent ISPOR guidelines [6] and exemplified by the methods used by NICE for their costing templates is to keep the model structure as simple as possible. Thus, a cost-calculator approach should always be considered first when designing a budget-impact model. Only if this approach cannot credibly capture the impact of the new drug on the budget for the specific disease and treatment characteristics would a more complex decision-analytic modeling approach be preferred. Of the 45 studies included in this review, 25 used a cost-calculator approach. One study did not specify the modeling approach. The other studies used more formal decision-analytic techniques to account for treatment sequencing and/or disease progression similar to those used for cost-effectiveness analyses (e.g., decision trees, Markov models, discrete-event simulations). The use of a formal decision-analytic model structure allows the modeler to account for changes in treatment over time such as switching, titration, and discontinuation when the first-line drug is not effective or well-tolerated, as well as to account for changes in disease outcomes over time. A cost-calculator model structure can indirectly account for these changes in treatment over time through the evolution of treatment shares over time and the related clinical impacts. However, derivation of credible estimates of these effects is more challenging when using a cost-calculator approach. Both types of model structure can estimate the budget impacts for chronic diseases for both the prevalent population already eligible for the new drug and multiple incident populations becoming eligible for the new drug over the model time horizon. This is important since the different population characteristics, uptakes, and effectiveness for the incident and prevalent populations may significantly affect the resulting estimates of budget impact. However, in our review for chronic diseases, none of the cost-calculator models included both prevalent and incident populations, and most of the models using formal decision-analysis techniques to account for treatment sequencing and/or disease outcomes only included incident or prevalent populations.

The current size and characteristics of the treated population and any changes that the new treatment may have on the treated population are critical determinants of the budget impact of a new treatment and need to be considered explicitly in the model design. Many of the studies reviewed did not include all of the elements that determine the population dynamics with the introduction of the new drug. Recommended practice for estimating the current population size and characteristics uses a ‘funnel-down’ approach starting from the total covered population size. In this approach, the starting covered population is reduced by the disease incidence and/or prevalence, then further reduced by the diagnosis rate, the percentage of those seeking treatment, and the percentage of those eligible for the new drug and receiving active treatment. This approach was used in some but not all of the studies reviewed. The value of including all of these steps for the estimation of the current population size is that changes to any step of the funnel-down process with the introduction of a new drug can readily be applied, and its impact can be examined in sensitivity analysis. For example, a new drug might increase the number of people who seek treatment or who accept active treatment, or the incidence of the disease might change over time.

Another aspect of the population dynamics that should be included in budget-impact analyses is the growth in the treated prevalence for more effective treatments in diseases/conditions such as advanced or metastatic cancer, congestive heart failure, or HIV infection. In these diseases/conditions, the size and/or characteristics of the treated population are likely to change during the model time horizon because of reduced mortality rates, lower rates of disease progression, or reduced disease severity. If these increases in treated population size or changes in population characteristics (such as the percentage with stable vs. progressing disease or the percentage in different disease severity states) are not accounted for, the resulting budget-impact estimates are likely to be biased. In a cost-calculator model, these effects can be included by programming the model to allow for the treated population size and characteristics to be different without and with the new drug. Disease-progression models that are run using a starting prevalent population and adding incident populations each year will pick up the changes in the treated population size and characteristics through changes in clinical outcomes that are included in the model.

The time horizon selected for the model is based on the information needs of the budget holder. However, the use of a 1-year time horizon in many of the published studies we reviewed is not ideal unless it is accompanied by extensive sensitivity analyses. Even in instances where the budget holder is only interested in a 1-year time horizon because of the availability of reliable of data, projections beyond 1 year are recommended. This is because population size and characteristics, treatment shares for the new drug and current treatments, and treatment costs (e.g., because of generic entry) for different treatments might change over time. A 3- to 5-year time horizon provides additional information about the possible impact of these changes over time. A time horizon to steady state might be of interest in some jurisdictions.

The fourth element we reviewed was treatment mix. Many of the analyses examined the impact of adding the new drug to the current mix of treatments. In some of these studies, the current mix and new mix of treatments (i.e., x % of patients on drug A, y % of patients on drug B, z % of patients on drug C) were not presented. The uptake of the new drug over time was included in all studies but without a rationale for the values used. Sources for this information might include expert opinion, market research with practicing physicians, or observed or modeled data based on previous new drug launches. Generally, the new drug will not be used in all patients, though this might occur in some cases. In addition, the rationale for how treatment shares were redistributed from the current treatments to the new drug was rarely provided. Redistribution to the new drug was frequently assumed to be taken equi-proportionately from all current drugs but without a rationale for this choice. Possible sources for this information could be from market research with practicing physicians or assuming redistribution from specific drug classes for which use is most likely to be less with the new drug (e.g., reduction in treatment share for current interferons with the introduction of a new, more convenient interferon for patients with multiple sclerosis). The treatment mix should also take into account titration, switching, and discontinuation. These impacts can be explicitly included in a decision tree or disease-progression model and need to be approximated when using a cost-calculator model. The impact of the new drug on the current treatment mix can be a major determinant of the eventual impact on the budget. This is especially true when the current mix includes many brand and generic drugs and/or a variety of dosing combinations. Thus, transparency for the base-case assumptions and multiple scenario analyses are important to help readers understand how the published analysis might represent their jurisdiction. Despite the importance of these inputs, only a few of the reviewed studies included scenario analyses testing the impact of different assumptions about the new treatment mix.

The models in the studies reviewed also varied in which treatment-related costs were included. In general, it would be better if all models included drug acquisition, diagnosis, monitoring, administration, and adverse event costs, as these may all be affected due to the introduction of the new treatment. All models included the acquisition costs. However, the studies varied as to whether these costs included or excluded patient co-payments, co-insurance, or supplier discounts off the published prices or dispensing fees. About half of the studies did not state whether patient payments or manufacturer discounts were accounted for in the acquisition cost estimates. In the US context, these factors can have a major impact on the actual drug costs to the budget holder. However, it is probably best to include these factors only in sensitivity analyses for published budget impacts, as consideration of these issues can become quite complex since there is great variability in how US health plans are administered. Many models omitted one or more of administration, monitoring, and adverse event costs. Although administration costs may not be relevant in all studies (e.g., for oral drugs or where they are likely to be similar for all drugs in the treatment mix), the other two cost categories generally will be relevant and should be included or a rationale provided for why they were not included. Such a rationale for not including monitoring or adverse event costs might be that they were similar for the new drug and for the treatments it would replace or that the incidence rates and/or costs would be very low.

There was also variability among the studies as to whether disease-related costs were included. Published guidelines generally recommend including the disease-related costs. However, the changes in treatment-related and disease-related costs should be presented separately to address the different perspectives of different budget holders. In the NICE costing templates, disease-related costs are generally only included when there is evidence from head-to-head clinical trials or credible indirect treatment comparison analyses [67]. In the studies reviewed, the changes in disease-related costs were taken from published clinical trials but mostly from one or a few studies rather than from a systematic review and meta-analysis of all published studies. For acute diseases, both cost calculators and more complex decision models can capture the changes in disease outcomes. For chronic diseases, it is challenging for a cost calculator to capture the effect on disease outcomes of titration, discontinuation, and switching to another drug in the treatment pathway using credible assumptions, while a more complex decision model can more readily capture these effects on disease outcomes based on clinical trial and observational data.

Since budget-impact estimates depend significantly on the budget holder’s perspective and health plan variables, it is very important in a published study to show how the budget impact varies with different population characteristics, treatment mix, treatment efficacy and costs, and cost categories. It is critical for publications of budget-impact analyses to include extensive sensitivity and scenario analyses to reflect these different perspectives and give a balanced picture of what the budget impact might look like in different situations, since the reader does not have access to the computer program. Possible sources for this information are interviews with likely budget holders or published studies or national statistics of jurisdiction population characteristics. Scenario analyses allow budget holders to see the results for alternative combinations of input parameters, but these should represent alternatives likely to be experienced rather than best-case or worst-case scenarios. One-way sensitivity analyses allow budget holders to understand the impact that changes in a single input parameter might have on the results. The ranges used in the one-way analyses should be data driven for input parameters where such data are available or based on suggestions from budget holders in the jurisdiction of interest. The use of arbitrary ranges either ±10 or ±50 % is not appropriate and can give a distorted picture of the likely budget impact.

In our review, we included ten combined cost-effectiveness and budget-impact analysis articles. A rationale for including both together in a single model might be that they share many common input parameter values and to present a comprehensive economic evaluation of a new drug in one model. Since we feel that budget-impact models should be kept as simple as possible, separate budget-impact models may be better since cost-effectiveness models that are designed to meet HTA requirements are generally very complex. On the other hand, if the cost-effectiveness model is simple or if a simple budget-impact model would not be credible to decision makers, then a combined model may be preferable. In this case, we believe that it is critical for the structure, assumptions, and input values for both models to be described in detail in the published study or in its online appendices. This was not the case for most of the reviewed combined cost-effectiveness and budget-impact analysis articles.

All previous reviews of budget-impact models have presented information on model design and have concluded that it is variable [19–21]. As found in this review, this variability persists. In our review, we selected seven key design elements for a budget-impact analysis and describe recommended practice for each design element based on methods guidelines as well as presenting a thorough examination of the extent to which recommended practice is followed for each design element. Because disease and treatment characteristics vary widely, some variability in the model design is inevitable. In our review, we recognize that the recommended design for a budget-impact analysis will vary according to the type of disease (acute or chronic) as well as the likely impact of the new drug (whether or not it affects mortality or disease progression). Based on our review of the published US budget-impact analyses, researchers and reviewers could benefit from asking a series of questions about the model design to increase the likelihood that recommend practice is followed, common pitfalls are not present, and the published results are useful for many different budget holders. These might include the following questions:

  • Does the model design appropriately consider changes in treatment-related and disease-related costs attributable to changes in treated population size and characteristics over time?

  • Does the model design consider all of the treatment-related and disease-related costs associated with changes in the treatment mix with the new drug over time?

  • Does the model design include treatment-related acquisition, administration, monitoring, and adverse event costs or give a rationale for their exclusion?

  • Does the study provide results from comprehensive one-way sensitivity analyses including data-driven or budget holder-driven ranges?

  • Does the study provide results from multiple alternative scenarios that might be encountered by different budget holders?

The van de Vooren et al. [21] review also expressed a concern that most of the published budget-impact models for European Union countries were sponsored by the drug manufacturer and that this might introduce bias into the estimates. They specifically state that this bias could be introduced either by the model design or the input parameter values. In our review of 45 US budget-impact analyses, 40 were sponsored by industry or had industry authors. Our review focused on model design and did not review the specific data sources used to derive the input parameter values for each study. However, it was apparent from our review that there are no standard sources in the USA for many of the input parameter values used in budget-impact models. These include the predicted treatment shares for the new and current drugs over the model time horizon; the size of the eligible population who are taking active treatment; possible changes in those seeking treatment with the new drug; drug acquisition costs net of patient payments and discounts; and ranges for the sensitivity analyses. Although issues around recommended practice in model design have been found in our review as in the previous reviews, we believe these issues may be due to the relative newness of these types of analyses in the published literature and associated lack of guidance for researchers and peer-reviewers rather than the sponsor of the study. To determine whether the results of a specific analysis are biased would require a detailed evaluation of both the model design and the input parameter sources and assumptions, which was beyond the scope of this review and was not attempted in the van de Vooren et al. [21] review.

5 Conclusion

Even though guidelines for budget-impact analyses have existed for some time, there still seems to be great variability in the design of these analyses, even for those analyses performed for a new drug for the same type of disease (acute or chronic) and the same type of impact of the new drug on the population size or characteristics. This variability occurs in all seven key elements of budget-impact model design identified in our review. It is important for researchers and peer-reviewers to critically evaluate these different elements of the analyses targeted for publication to ensure they follow recommended practice in order for these analyses to be useful for assisting in health plan decision making.