FormalPara Key Points for Decision Makers

This work describes the National Centre for Pharmacoeconomics Rapid Review, a pivotal part of the health technology assessment (HTA) process in Ireland that allows for appropriate resource prioritisation within a national HTA agency. It is considered to be an efficient way of determining the requirement for a full HTA and targeting resources for those drugs where there is most value in conducting an HTA.

There has been a general year-on-year increase in the number of submissions received since the process was introduced.

Drugs for cancer indications and first-in-class drugs are more likely to require a full HTA. When economic factors and clinical evidence-related factors were included in the model, drugs for orphan indications were not found to be associated with an increased likelihood of requiring a full HTA. The analysis was limited by the ability of the included variables to capture features such as ‘uncertainty’ in the corresponding clinical data.

The Rapid Review process has resulted in appreciable reductions in NCPE appraisal time over the 10-year study period.

1 Introduction

In Ireland, similar to other jurisdictions, a health technology assessment (HTA)—a multidisciplinary process used to systematically evaluate the costs and outcomes associated with a health technology—is used to inform decisions around drug reimbursement. In line with Irish legislation, clinical effectiveness, cost effectiveness and affordability (amongst other criteria) must be considered when a drug reimbursement decision is made [1]. The National Centre for Pharmacoeconomics (NCPE) has conducted the HTA of drugs in Ireland since its establishment in 1998 [2]. As set out in the European Union Transparency Directive, such assessments to support pricing and reimbursement decisions should be conducted in a timely manner [3]. Since 2006, successive agreements between the Health Service Executive (HSE; the government agency that manages the provision of publicly funded healthcare in Ireland) and the Irish Pharmaceutical Healthcare Association (a body representing the pharmaceutical industry in Ireland) have specified that drugs for which reimbursement is sought must be subject to an assessment by the NCPE [4]. The 2006 agreement stated that assessments were only required for high-cost drugs or those associated with a large budget impact. Since 2009, all new drugs were considered for assessment. In 2009, to meet the demands of both volume and timeliness of assessments, the NCPE introduced the Rapid Review process.

The objective of the Rapid Review is to provide a recommendation to the HSE (the decision maker) on the need for a full HTA and, in some cases, on reimbursement. Each Rapid Review pertains to one drug for one indication. Following the Rapid Review, a full HTA will subsequently be required for those drugs for which additional information and/or analysis is required to inform a reimbursement recommendation. Factors that are evaluated during a Rapid Review and thus inform the requirement for a full HTA include: the cost of the drug relative to potential comparators; comparative clinical effectiveness, and the degree of uncertainty relating to this; anticipated cost effectiveness, and the degree of uncertainty relating to this; and the potential budget impact [5]. In order to facilitate completion of the assessment within a 4-week timeframe, a number of features of the NCPE’s full HTA process are beyond the scope of the Rapid Review process, most notably evidence synthesis analyses and formal cost-effectiveness analyses. Key differences between the Rapid Review and full HTA processes are outlined in Table 1.

Table 1 Summary of key differences between the Rapid Review and full HTA processes
Full size table

There have been attempts to characterise the factors that may be associated with a Rapid Review recommendation; however, these analyses have been subject to limitations. A previous analysis of drugs for orphan indications indicated that such drugs were significantly more likely to require a full HTA following a Rapid Review than drugs for non-orphan indications, but did not account for other factors that may be associated with the likelihood of requiring a full HTA, such as economic or clinical evidence-related considerations [9]. Murphy and Redmond reported that drugs for certain therapeutic areas (including cancer), drugs for orphan indications, and drugs designated ‘first-in-class’ status were more likely to require a full HTA [10]. The analysis was limited by the availability of Irish cost data, and did not account for any factors relating to clinical evidence [11]. The Rapid Review was introduced with the aim of directing the NCPE’s resources to areas where a full HTA would be of greatest value, resulting in an overall reduction in NCPE appraisal time. To date, no empirical analysis of the potential differences in NCPE appraisal time arising as a result of the Rapid Review process has been reported.

This paper provides an overview of the NCPE Rapid Review process, with a specific focus on the perspective of the HTA agency. It evaluates trends in Rapid Review submissions and recommendations over time, assesses factors associated with Rapid Review recommendation using data uniquely available to the NCPE and explores the potential savings in NCPE appraisal time resulting from the process.

1.1 Rapid Review Process

The applicant pharmaceutical company (herein ‘the Applicant’) can submit a reimbursement application for the drug under consideration to the HSE at any time once the European Union Committee on Human Medicinal Product’s positive opinion has been granted. Upon receipt of a reimbursement application, the HSE commissions the NCPE to undertake a Rapid Review assessment. The Rapid Review dossier is completed by the Applicant in accordance with the NCPE’s pre-specified template, and encompasses various domains related to the intervention including the licensed indication, the indication for which reimbursement is sought, the target population, clinical efficacy and safety data, any potential comparators, price and budget impact, information on on-going clinical investigations and HTAs published in other jurisdictions [7]. The NCPE review process involves a targeted review of the relevant literature, and an appraisal of the dossier submitted. Depending on the outcome of the Rapid Review appraisal, the NCPE makes a recommendation to the HSE. The recommendation specifies if a further assessment in the form of a full HTA is required; if not, a reimbursement recommendation may be made at this stage. Generally, each Rapid Review is undertaken by a NCPE Review Group assessor, with reporting completed in line with a standardised NCPE format. Following an initial appraisal, the Rapid Review recommendation is made on the basis of a consultation process in accordance with internal protocols. The outcome of the Rapid Review, including information on the timelines, is made publicly available on the NCPE’s website (www.ncpe.ie). At present, the NCPE makes one of five possible recommendations (Table 2). Over time, the recommendations have evolved, primarily to provide stakeholders with more transparency on the decisions made. For the purpose of this analysis, Rapid Review recommendations have been categorised into one of two outcomes: ‘full HTA not required’ and ‘full HTA required’ (Table 2).

Table 2 Rapid review recommendations and interpretation
Full size table

2 Methods

A retrospective analysis of all Rapid Review submissions made to the NCPE from January 2010 to December 2019, inclusive, was conducted. The year 2010 was the first full year where the Rapid Review process was instituted. The cut-off date ensured a final recommendation had been made at the time of data collection. All Rapid Reviews commissioned by the HSE and appraised by the NCPE over this time period were eligible for inclusion. Data were sourced from the NCPE internal records and the European Medicines Agency website. The date the Rapid Review was commissioned by the HSE was considered as the starting date for each of the Rapid Reviews. The number of Rapid Review submissions received annually and over the full 10-year study period was recorded. For each Rapid Review, the Rapid Review recommendation was recorded (i.e. full HTA required or full HTA not required), and the proportion of submissions requiring a full HTA was calculated.

For each Rapid Review, additional data were collected for the variables outlined in Table 3. Because of changes in internal process for the reporting of Rapid Reviews over time, only data pertaining from 2012 onward were collected to ensure completeness of the dataset. Variables are categorised under the following: drug-related factors, economic factors and clinical evidence-related factors. All were hypothesised to be associated with the likelihood of requiring a full HTA. Drugs for orphan indications, drugs for cancer indications and drugs designated first-in-class have previously been found to be more likely to require a full HTA [9, 10]. The year in which the Rapid Review was conducted was also included, as anecdotally it has been proposed that the increasingly complex nature of the clinical evidence supporting submissions may mean a full HTA is more likely. As outlined in the description of the Rapid Review process, clinical evidence-related factors and economic factors are explicitly recognised as being associated with the likelihood of requiring a full HTA. The specific variables collected under these factors were informed by the nature and availability of data routinely reported in the NCPE’s Rapid Review report. Data were collected on whether a patient access scheme (PAS) was currently in place for the drug under assessment (where it was reimbursed at an earlier stage for a different indication), and if in place for the most relevant comparator(s). A PAS is typically implemented as the result of confidential negotiations between the Applicant and HSE. Summary statistics were calculated.

Table 3 Variable definitions
Full size table

Multivariable logistic regression models were estimated to evaluate the factors associated with the requirement for a full HTA following a Rapid Review. The dependent variable was the binary variable that indicated if a full HTA was required following a Rapid Review (as categorised in Table 2). A model including all variables relating to drug-specific factors was first specified (Model 1). An additional model (Model 2) was then estimated, which included all variables relating to economic factors and clinical evidence-related factors, in addition to those specified in Model 1. Marginal effects at the means were calculated post-model estimation.

An exploratory analysis was performed to estimate the additional NCPE appraisal time that theoretically would have been required to evaluate all drugs submitted to the NCPE over the study period as full HTAs, had the Rapid Review process not been established. The cumulative NCPE appraisal time for two separate scenarios was estimated, with the difference representing the additional NCPE appraisal time that would otherwise have been required. Timelines for the Rapid Review (mean 32.2 days) and full HTA (mean 133.3 days) processes were sourced from a previous analysis of the NCPE’s HTA timelines [6].Footnote 1 The first scenario modelled a ‘world with’ the Rapid Review process, where all drugs underwent a Rapid Review and, where recommended, a full HTA. For the ‘world without’, it was assumed that all drugs for which reimbursement was sought over the 10-year study period underwent a full HTA. A number of additional sensitivity analyses were performed to explore uncertainty around the assumptions used in this analysis.

  • Our base-case analysis was based on a previous study that measured timelines for submissions appraised between 2015 and 2017 [6]; assumptions regarding the generalisability of these timelines to the wider context were examined. Scenarios where mean appraisal timelines both increased and decreased with respect to time were evaluated. Longer appraisal times could occur as a result of increases in both the volume and complexity of submissions received, whereas reductions in timelines might reflect additional recruitment within the NCPE in recent years.

  • Our base-case analysis assumed that the time taken for a full HTA in the ‘world without’ the Rapid Review process would equal that in the ‘world with’, implying that the Rapid Review does not contribute to the efficiency of the subsequent full HTA process. Timelines were varied by + 20% and − 20% to examine the assumption. The Rapid Review is used to inform pre-submission meetings held between the NCPE and Applicant prior to submission of a full HTA, meaning it is likely that a full HTA in a ‘world without’ would require additional appraisal time.

  • In our base-case analysis, it is assumed that all drugs for which a full HTA is recommended after a Rapid Review proceed to submit a full HTA dossier. It is the NCPE’s experience that not all Applicants elect to proceed with a full HTA submission, meaning a reimbursement recommendation cannot be made. When testing this assumption, only reductions in the rate of Applicants’ submissions were examined (as it is not possible to ‘over-submit’).

  • We examined a scenario where, in the ‘world without’ the Rapid Review process a number of Applicants would elect to not proceed with a reimbursement application. It may be that a number of Applicants would not submit a full HTA (owing to the increased workload associated), in the absence of the opportunity to first submit a Rapid Review and potentially receive a reimbursement recommendation at this stage. As for the previous scenario, only reductions were examined.

Data collection and analysis were completed using Microsoft Excel® and Stata® Version 14 (StataCorp)

3 Results

A total of 446 Rapid Review submissions were assessed by the NCPE from January 2010 to December 2019, inclusive. The number per year increased from 19 in 2010 to 57 in 2019, with a peak of 63 in 2017 (Fig. 1). In terms of Rapid Review recommendations, a full HTA was deemed to be required in 221 (49.6%) of the Rapid Reviews. The proportion requiring a full HTA varied annually, from 68.4% in 2010 to 38.9% in 2014. Of the Rapid Reviews analysed, 390 were conducted between January 2012 and December 2019, inclusive. Summary statistics for additional data collected for these Rapid Reviews are presented in Table 4.

Fig. 1
figure 1

Annual Rapid Review submissions 2010–19. HTA health technology assessment

Full size image
Table 4 Summary statistics: Rapid Reviews 2012–19 (N = 390)
Full size table

Results of the multivariable logistic regression models estimated are presented in Table 5. Results are presented as marginal effects at means, with a positive coefficient indicating that a variable is associated with an increased likelihood of requiring a full HTA and a negative coefficient indicating a full HTA is less likely following Rapid Review. Based on the results of Model 1, drugs for cancer indications (47.3 percentage points [ppts], p < 0.01), drugs for orphan indications (22.8 ppts, p < 0.01) and drugs designated first-in-class (42.9 ppts, p < 0.01) were all associated with a statistically significant increase in the likelihood of requiring a full HTA. However, when variables relating to cost and clinical evidence were included, the association was no longer significant for drugs for orphan indications (Model 2: 0.1 ppts). As compared to Rapid Reviews completed in 2012, those conducted in 2019 were less likely to require a full HTA (− 37.0 ppts, p < 0.05). Otherwise, no clear trend was observed between the year the Rapid Review was completed and the requirement for a full HTA. Economic factors were significantly associated with a higher likelihood of requiring a full HTA. As compared to drugs priced on par or lower than comparators, drugs that were more expensive than comparators were associated with an 80.2 ppts increase in the likelihood of requiring a full HTA (p < 0.01). For every €1000 increase in the cost per treatment course of the drug, the likelihood of requiring a full HTA increased by 0.3 ppts (p < 0.01). The pre-existence or offer of a PAS for the drug under assessment was associated with a lower likelihood of requiring a full HTA (− 19.0 ppts, p < 0.05); similarly, a lower likelihood was reported where a PAS was known to be in place for a comparator (− 27.0 ppts, p < 0.05).

Table 5 Results of multivariable logistic regression models: Rapid Reviews 2012–19 (n = 390)
Full size table

In terms of clinical evidence-related factors, drugs that were supported by clinical trials where results of the final analysis were available at the time of the Rapid Review were less likely to require a full HTA (− 27.6 ppts, p < 0.01). Compared to submissions supported by evidence from a pivotal phase III trial, those that related to phase II pivotal trials (− 30.0 ppts, p < 0.05) or ‘other’ types of trial (− 45.3 ppts, p < 0.01) were less likely to require a full HTA. There was no association found between the concealment used in the pivotal trial and the likelihood of requiring a full HTA. Placebo-controlled pivotal trials were more likely to require a full HTA than those where the comparator involved an active control (24.6 ppts, p < 0.01).

The theoretical additional NCPE appraisal time required, had the Rapid Review process not been instituted, was estimated at 15,631 days, cumulative over the 10-year period assessed. Compared to a world without the Rapid Review process, the time taken in the ‘world with’ represents a 26.3% reduction in overall NCPE appraisal time. Assuming a mean NCPE appraisal time for a full HTA of 133.3 days, the difference in appraisal time represents the equivalent of 117 full HTAs over the 10-year period examined. Results of sensitivity analyses undertaken to examine uncertainty are presented in Table 6. For all the assumptions tested, the Rapid Review process results in a net reduction in NCPE appraisal days (see final column, Table 6).

Table 6 NCPE appraisal time in ‘world with’ and ‘world without’ a Rapid Review process: Rapid Reviews 2010–19 (N = 446)
Full size table

4 Discussion

This study describes the Rapid Review process, a system designed and implemented by the NCPE to facilitate prioritisation of resources within a national HTA agency. A total of 446 Rapid Review submissions were received by the NCPE over the 10-year period between 2010 and 2019, with a general year-on-year increase in the number of Rapid Review submissions received. This analysis highlights that a full HTA is deemed to be required in approximately half of drugs for which reimbursement is sought. The analysis of the factors associated with a Rapid Review recommendation is the first to use data uniquely available to the NCPE. As was reported in previous analyses, drugs for cancer indications and drugs designated ‘first-in-class’ were more likely to require a full HTA [10]. In contrast to previous analyses, which did not include factors relating to cost or clinical evidence, drugs for orphan indications were not significantly associated with the likelihood of requiring a full HTA [9, 10]. An exploratory analysis indicated that the Rapid Review process facilitates an appreciable reduction in NCPE appraisal time, with findings robust to a number of sensitivity analyses (Table 6).

Many HTA agencies have highlighted the challenges posed by both an increasing demand for HTA and increasing pressure for ‘rapid’ assessments [12]. In light of these pressures, a key challenge for HTA agencies is determining how a ‘formal, effective and acceptable’ prioritisation process can be selected [12]. Here, we share the experience of a national HTA agency in implementing one such prioritisation strategy. The Rapid Review process has been a core component of the NCPE’s assessment pathway for over 10 years. It facilitates the identification of drugs that require additional assessment in the form of a full HTA, while maintaining a robust appraisal of the relevant clinical and economic (in terms of cost and budget impact) evidence for drugs that do not require further assessment.

The NCPE’s Rapid Review process is unique to the Irish HTA pathway. A number of HTA agencies have developed accelerated HTA processes in order to improve workflow and efficiency, including the abbreviated HTA submission process at the Scottish Medicines Consortium and the ‘fast-track’ appraisal process at the National Institute for Health and Care Excellence [13, 14]. However, only drugs that meet specific criteria are eligible to be evaluated under these systems. In comparison, all drugs under assessment by the NCPE may be subject to a Rapid Review. It is worth recognising that the term ‘Rapid Review’ is used in other instances within HTAs. It is typically used to describe a review that describes the characteristics of the technology under assessment, as well as evaluating safety and effectiveness issues [15]. We do not propose that the ‘rapid’ or accelerated HTA is a novel concept, but highlight how the implementation within the overall HTA framework in Ireland represents a unique and adaptive way to manage resource prioritisation within the NCPE.

A criticism previously made elsewhere of the Rapid Review process was that it results in duplication, particularly for drugs where it is believed there is a high likelihood that a full HTA will be required [10]. The NCPE has developed strategies to mitigate these concerns. For example, the NCPE’s Budget Impact Model template may be used at the Rapid Review stage, and may later be re-submitted along with the electronic model at the full HTA stage if required [16]. Moreover, where drugs do require a full HTA, the Rapid Review offers an important opportunity for the NCPE Review Group to identify key uncertainties in advance of the full HTA. All drugs referred for a full HTA are discussed at a pre-submission meeting between the NCPE Review Group and the Applicant where these issues can be communicated and discussed in advance of the submission. It is not possible to quantify the downstream impact of this process on the full HTA submission. Anecdotally, it is considered a valuable exercise by both the NCPE and pharmaceutical industry representatives.

4.1 Limitations

This study is subject to a number of limitations. First, it should be noted that the Rapid Review process only became a part of the HTA process at the NCPE in mid-2009. Therefore, the number of Rapid Reviews recorded for 2010 may underestimate the total number of drugs that underwent assessment with the NCPE at this time.

Additionally, the analysis of the factors associated with the Rapid Review recommendations is limited by the scope of the variables included. It was not possible to include certain variables that were expected to be associated with the requirement for a full HTA. For example, the net drug budget impact is an important consideration in determining the requirement for a full HTA [5]. The Review Group frequently considered the point estimates presented by the Applicant as highly uncertain and it is beyond the scope of the Rapid Review process to revise the Applicant’s budget impact estimates. Therefore, accurate data on the expected net drug budget impact were not routinely available. In addition, when considering the variables selected to represent clinical evidence-related factors, it is important to highlight that the Rapid Review seeks to evaluate how likely it is that the intervention is equally or more effective than the relevant comparator(s), in the target population, with minimal uncertainty. It is unlikely that the clinical evidence-related variables included in the analysis have adequately captured this, as suggested by the unexpected finding that Rapid Reviews supported by phase II trials were less likely to require a full HTA than phase III trials.

Further, while it is likely that the Rapid Review process has resulted in substantial savings in NCPE appraisal time since implementation, estimating the additional NCPE appraisal time that would be required were the Rapid Review process not instituted required numerous assumptions, meaning the uncertainty associated with our findings is unavoidable. A number of one-way sensitivity analyses were performed to evaluate the robustness of our results. Two scenarios resulted in a reduction in NCPE appraisal time that was appreciably lower than our base-case analysis. The first (Base-case assumption B, Option 1) examined a scenario where NCPE completion time for a full HTA is shorter in the ‘world without’ the Rapid Review process. Given the Rapid Review is used to inform pre-submission meetings held between the NCPE and the Applicant prior to the submission of a full HTA, it is unlikely that any change in this parameter would occur in this direction. The second (Base-case assumption D, Option 2) examined a scenario where a proportion of Applicants elect to not submit a full HTA in the ‘world without’ the Rapid Review. Here, we found the Rapid Review process remains time saving at a reduction of 20% of submissions. It is unclear how realistic this scenario is (i.e. how much of a deterrent the full HTA process might be, in the absence of the Rapid Review process). A more general limitation of this analysis is the use of a very specific metric to measure the impact of the Rapid Review process. While NCPE appraisal time remains an informative and intuitive metric, there may be changes in other important outcomes arising from avoiding full HTAs not captured in this analysis.

Moreover, the primary aim of this analysis is to provide insight into the Rapid Review process from the perspective of the HTA agency. However, given the potential for a shorter time to a reimbursement decision depending on the outcome of the Rapid Review, we recognise that the process is of interest to a variety of stakeholders. Previous publications have suggested that reimbursement is likely in cases where a full HTA is not required; however, we highlight that the recommendation that a full HTA is not required does not necessarily indicate a more straightforward path to reimbursement [10, 19]. Reimbursement decisions in Ireland are made by the HSE on the basis of decision-making criteria set out in the Health (Pricing and Supply of Medical Goods) Act 2013, which includes additional criteria to those assessed by the NCPE [1]. Neither reimbursement outcomes, reimbursement timelines, nor any other factors outside the HTA appraisal process that impact on these outcomes have been evaluated in this analysis. As a result, it is not possible to evaluate if there is any association between the Rapid Review process and these outcomes. Further research in this area may provide valuable insight to a broader range of stakeholders.

5 Conclusions

As the demand for the reimbursement of drugs from finite healthcare budgets continues to grow in Europe and beyond, pressure on reimbursement systems persist. It is important that experiences of HTA agencies are shared as processes of evaluation continue to evolve. This work describes the NCPE’s Rapid Review process in Ireland over the last decade. The process is a pivotal and well-established part of the HTA process in Ireland that allows for appropriate resource prioritisation within a national HTA agency. It is considered to be an efficient way of determining the requirement for a full HTA and targeting resources for those drugs where there is most value in conducting a HTA.

The results of this study demonstrate that a full HTA was not required for approximately half of the Rapid Reviews submitted. Drugs for cancer indications and drugs designated first-in-class were more likely to require a full HTA. In contrast to previous analyses, drugs for orphan indications were not associated with an increased likelihood of requiring a full HTA. A number of variables relating to cost were found to be significantly associated with the likelihood of requiring a full HTA. Determining the relationship between the supporting clinical evidence and the Rapid Review recommendation is challenging because of the limited sensitivity of the variables considered. While it is likely the process has substantially reduced NCPE appraisal time, estimation of such savings is challenging and subject to a number of limitations. The importance of a robust appraisal process is highlighted given that for approximately half of submissions the Rapid Review is the only formal evidence-based assessment. Therefore, there is a need to ensure that the process is continuously evolving to meet the current requirements of the healthcare system.