FormalPara Key Points

Drawing on data from the World Health Organization International Clinical Trials Registry Platform and employing a difference-in-difference-in-differences estimation approach, our study builds on the existing literature and shows that the PRV does not show significant effect at generating trial activity.

The policy does not seem to have triggered an increase in R&D for the intended diseases following its implementation, not even in late-phase trials, which are generally common in the neglected diseases era when repurposing drug indications or developing new treatment regimens. Accordingly, the benefit of the voucher, whether used or sold, does not appear a sufficiently strong incentive for pharmaceutical companies to embark on risky R&D.

Three main policy recommendations arise from our study. First, the PRV should not be seen as a stand-alone solution but as a complementary solution alongside other incentivizing mechanisms when tackling diseases of the world’s poorest populations. Second, if the PRV program is to continue, the PRV should reward products that bring novelty and not reward products that are already licensed outside the US. Doing so would restrict the number of vouchers awarded and slow down their ongoing market depreciation. Lastly, the PRV should require product sponsors to submit an access plan for PRV-awarded products.

1 Introduction

Research and development (R&D) is lengthy and costly. It takes more than a decade and costs around $2.6 billion to bring a new drug to the market (2013 US dollars) [1].

Failure in R&D is common; for example, the overall probability that a drug entering clinical testing ends up being approved by the US FDA is estimated to be 11.8% [1]. Failure rates nonetheless vary across drug development phases and are estimated at around 45.9% for phase I, 43.5% for phase II, and 10.6% for phase III/regulatory review [1].

Given the costly and risky nature of R&D, pharmaceutical companies have an interest in investing in diseases with a large and stable market in high-income countries (HICs). Consequently, infectious diseases that are mostly prevalent in low-income countries (LICs), such as leishmaniasis, sleeping sickness, dengue fever and Chagas disease, have not historically attracted much interest.

These diseases mainly affect the world’s poorest populations with a purchasing power that is not high enough to generate a return on investment for the pharmaceutical industry. Moreover, for some of these diseases, the market is small and distributed in very concentrated and specific regions of the world, often with low ability-to-pay for therapeutic products. As a result, these diseases are labeled as ‘neglected diseases’.

Between September 1999 and December 2011, 118,634 trials were registered in the US National Institutes of Health (NIH) registry (ClinicalTrials.gov), but only 1541 (1%) were for neglected diseases [2]. While neglected diseases account for 12% of the global health burden, their share of R&D activity barely reaches 1%.

To address this unmet need, the US congress established the priority review voucher (PRV) program in September 2007. The PRV was initially designed for tropical diseases but was expanded to include rare pediatric conditions in 2012 and medical countermeasures in 2016.Footnote 1

The rationale behind the PRV was to reward the development of successful products for one of the eligible diseases, by awarding the products’ manufacturers/sponsors a voucher that reduces the duration of a product review by the FDA from the usual 10 months to 6 months [3]. The voucher thus grants faster review and can be used for any product of the PRV holder’s choice, either for an earlier market launch or during earlier R&D phases. Alternatively, the voucher can be sold to a third party. Thus far, prices of PRVs have ranged from $67.5 million in 2014 to $338 million in 2015 [4].

The PRV is said to lead to a ‘win-win’ situation, with the social welfare gains to patients both in LICs and HICs and the net gains to manufacturers being greater than the cost of FDA review incurred by the government [5,6,7]. This being said, the cost borne by the FDA is not to be neglected. The PRV program is said to hinder the ability of the FDA to predict its own workload and to divert staff attention from other important work to meet the required shortened timeframe for review [8].

The policy has specific eligibility requirements that touch upon diseases, registries, and product types. For tropical diseases, the PRV policy includes a specific list of eligible diseases (see Fig. 1). The policy was implemented in a staggered way; most tropical diseases became eligible as of September 2007, while a few became eligible in 2014, 2015, 2016, and 2018. The voucher can only be awarded to products (drugs, vaccines, and devices) for one of the eligible diseases and for which previous trials, other than phase I trials, were registered in the US registry (ClinicalTrials.gov) within 21 days of enrolling the first patient in the trial.

Fig. 1
figure 1

Source: Our elaboration on US FDA data [24]. PRV priority review voucher

Eligible diseases of the priority review voucher program. Most diseases became eligible for the PRV when the policy was implemented in 2007. Few additional diseases were added to the eligible list later on, in 2014, 2015, 2016 and 2018.

Full size image

Eligible products are products that demonstrate their non-inferiority to existing products and contain no active ingredient that has been approved in any other application (although combination products with at least one new active moiety are eligible).

Accordingly, translating these requirements to trial activity, eligible trials are trials other than phase I that (1) demonstrate the non-inferiority of the tested product; (2) are interventional; and (3) are registered on ClinicalTrials.gov within 21 days of enrolling the first patient but after the program start date (27 September 2007). Trials registered on or before that date are also considered eligible if they were still ongoing as of 26 December 2007. For diseases eligible in 2014, 2015, and 2016, trials should have been initiated after their date of eligibility.

Lastly, the program has faced structural changes since its implementation, in 2011 and 2014. Since 2011, a fee has to be paid to the FDA to compensate for the incurred added cost of conducting a priority review, which varies annually—from $2.32 million in 2014 up to $5.28 million in 2012 [9]. Before 2014, companies that were granted (or bought) the voucher and wanted to use it had to notify the FDA 365 days in advance, but as this was limiting the usefulness of the voucher, it was changed to a 90-day notice period. In that same year, it was decided that the voucher could be sold an unlimited number of times as opposed to once.

More than a decade after its implementation, it is unknown if the supposed benefits of the PRV have materialized. As of 2019, 31 PRVs have been awarded, of which 11 were awarded for tropical diseases (Table 1) [4].

Table 1 Priority review vouchers awarded for tropical diseases
Full size table

The PRV has been criticized for rewarding products already in use/licensed outside the US and/or manufacturers that purchased the license but who were not involved in the R&D process [10]. The PRV has also been criticized for rewarding products that would have been developed had the program not been implemented. This assertion is based on the idea that a potential 4-month early entry to the market is not sufficient to incentivize pharmaceutical companies to invest in risky projects for neglected diseases. The word ‘potential’ is important for two reasons: (1) R&D may not lead to successful product development and to access to the voucher; and (2) a voucher does not guarantee an earlier market launch since the FDA can decide to reject a product on which it conducted a priority review (e.g. the case of Novartis for its biological licensing application for canakinumab) [3]. Additionally, the voucher’s uncertainty is not only limited to its use but also extends to its sale; its market price has fluctuated significantly since the first voucher was sold in 2011, with a general depreciation since 2017 to around 100 million US dollars. Selling the voucher rather than using it may appear more appealing to smaller pharmaceutical companies that do not have a blockbuster candidate in their pipeline on which to use the voucher.

Until now, evidence of the PRV’s impact is limited. Only three studies—two for tropical diseases and one for rare pediatric conditions—have evaluated the PRV, but with research design and data limitations that limit the extent to which causal inferences can be made [11,12,13] (for more details refer to Online Resource 1 in the Electronic Supplementary Material [ESM]).

Therefore, the objective of this study was to assess whether the PRV has successfully incentivized R&D for the intended tropical diseases that would have not taken place in its absence. We built on previous studies by employing a difference-in-difference-in-differences approach. As the policy targets specific diseases (Fig. 1) from a specific regulatory body/trial registry (i.e. the FDA/ClinicalTrials.gov), we defined two control groups: the diseases that are not targeted by the PRV (non-eligible diseases) and the registries that are not affected by the PRV (any registries other than ClinicalTrials.gov that are members of the WHO International Clinical Trials Registry Platform [ICTPR]; (see Online Resource 2 in the ESM).

2 Methods

Data on clinical trials were retrieved from the WHO ICTRP, which gathers ongoing, completed, and terminated clinical trials from 18 registries (see Online Resource 1 in the ESM) and whose objective is to provide a comprehensive database on clinical trial activity to ultimately improve research transparency [14]. Since its creation in 2005, registries have progressively entered the platform, conditional on fulfilling specific requirements. To enter the WHO ICTRP, registries must fulfill the International Committee of Medical Journal Editors (ICMJE) requirements (i.e. prospective registration of trials) and meet the WHO Registry Criteria [15].

Each disease name was separately entered in the database to retrieve the data. In case a disease has more than one name (e.g. onchocerciasis/river blindness), all names were independently entered.

We estimated a DDD by differencing (1) trial activity for the eligible diseases with non-eligible diseases in the registry affected by the policy (i.e. ClinicalTrials.gov), (2) with trial activity for the eligible diseases with the non-eligible diseases in the control registries that were not affected by the policy (i.e. any registries other than ClinicalTrials.gov from the WHO ICTRP). Since the disease eligibility requirement of the PRV is to be an infectious disease with no market in HICs, we defined non-eligible diseases as non-communicable diseases (NCDs) with a significant market in HICs. Accordingly, we chose the NDCs that account for the highest number of disability-adjusted life-years (DALYs) in HICs according to the Global Burden of Disease project [16]: ischemic heart disease, diabetes, lung cancer, and stroke. In this way, we can assure that the diseases in the control group have no chance of becoming eligible for the policy, nor is there a reason to believe in any spillover effects of the policy on these diseases. Accordingly, the validity of the DDD estimator resides on the relative outcome (i.e. trial activity) of eligible diseases and non-eligible diseases in the treated registry (i.e. ClinicalTrials.gov) to trend the same way as the relative outcome of eligible diseases and non-eligible diseases in the control registries, in the absence of the policy [17, 18]. We test for this assumption in the analysis, as explained below in more detail.

Lastly, given the eligibility requirements on the products’ trials for the eligible diseases, only interventional and non-inferiority trials from phases II–III targeting either a drug or a vaccine were kept. We discarded devices from the analysis as transparency regarding trial registration tends to be lower than for drugs or vaccines [19]. Furthermore, incentives and regulations for devices differ from those of vaccines and therapeutic treatments.

We focused our analysis on trial activity from 2005 to 2019 inclusive. Data on trial registration before 2005 are incomplete because before then it was not compulsory to register a trial to be eligible to publish its result in a journal that follows ICMJE recommendations [20]. Indeed, in 2005 there was a peak in trial registration, both in ClinicalTrials.gov and in the control registries, with a percentage increase between the years 2004 and 2005 of around 310% and 170%, respectively.

Our analysis includes 13,803 trials: 7308 trials have been registered in ClinicalTrials.gov, of which 1029 trials were registered for the eligible diseases. About 6495 trials have been registered in the control registries, i.e. any registries other than ClinicalTrials.gov that are members of the WHO ICTRP, of which 704 were registered for the eligible diseases (refer to Fig. 2 for trial activity at the registry level).

Fig. 2
figure 2

Total number of trials for each registry of the WHO ICTRP across eligible and non-eligible diseases (2005–2019). ANZCTR Australian New Zealand Clinical Trials Registry, CRIS Clinical Research Information Service, CTRI Clinical Trials Registry – India, ChiCTR Chinese Clinical Trial Registry, ClinicalTrials.gov United States registry, ICTRP International Clinical Trials Registry Platform, IRCT Iranian Registry of Clinical Trials, ISRCTN International Standard Randomised Controlled Trial Number, JPRN Japan Primary Registries Network, LBCTR Lebanese Clinical Trials Registry, PACTR Pan African Clinical Trial Registry, REBEC Brazilian Clinical Trials Registry, REPEC Peruvian Clinical Trial Registry, RPCEC Cuban Public Registry of Clinical Trials, SLCTR Sri Lanka Clinical Trials Registry, TCTR Thai Clinical Trials Registry

Full size image

The database was then organized as follows: for each disease and each registry, we counted the number of clinical trials starting each year. Therefore, our dependent variable is the yearly number of starting clinical trials per registry and disease, resulting in a yearly panel of disease_registry (e.g. ClinicalTrials.gov_malaria).

2.1 Empirical Model

The outcome variable distribution is skewed to the left due to a significant number of zeros (Online Resources 3 and 4 in the ESM). Although the evidence of overdispersion would suggest the use of negative binomial, we employed a Poisson fixed-effects model with cluster-robust standard errors by disease registry as it generates more robust estimates [21]. Thanks to its inclusion criteria at the disease and registry level, the PRV provides a quasi-experimental design that allows us to isolate its impact using a DDD approach, estimated as follow:

$$\begin{aligned} E\left[ {y_{jit} | X_{jit} } \right]\; & = \;\exp \;\left( {\beta_{1} {\text{after}}_{it} + \beta_{2} {\text{after}}_{it} \times {\text{eligible}}_{i} + \beta_{3} {\text{after}}_{it} \times {\text{ClinicalTrials.gov}}} \right. \\ & + \left. {\beta_{4} {\text{after}}_{it} \times \;{\text{eligible}}_{i} \times {\text{ClinicalTrials.gov}} + \beta_{5} {\text{DALYs}}_{it} + \beta_{6} {\text{User}}\_{\text{fee}}_{jit - 1} + \alpha_{j} \times \delta_{i} + \gamma_{t} } \right). \\ \end{aligned}$$

The dependent variable \({y}_{jit}\), is the number of clinical trials per registry (j), disease (i), and year (t). Trials included are non-inferiority and interventional trials either in phase II or III (i.e. phase I/II; phase II; phase II/III; phase III; phase III/IV) and registered between 2005 and 2019. The variable after equals 1 from the year the disease became eligible for the PRV (i.e. 2007, 2014, 2015, 2016, and 2018 depending on the disease) and 0 otherwise. The variable eligible equals 1 if the disease is eligible for the PRV and 0 otherwise. The variable ClinicalTrials.gov equals 1 if the trial is registered in that registry and 0 otherwise. The coefficient \({\beta }_{4}\) is the DDD coefficient and compares the change in trial registration for the eligible diseases with the non-eligible diseases, in both the treated and control registries, before and after the policy. The coefficient \({\beta }_{2}\) is the DD coefficient and picks up any diseases and year-specific effects that are correlated with the policy. Similarly, the coefficient \({\beta }_{3}\) picks up any registry and year-specific effects that are correlated with the policy. The variable DALYs_share captures the yearly share of total DALYs per disease in upper middle- and high-income countries according to the world bank definition [16]. This measure is used as a proxy for the yearly ‘market size/potential’: the higher the DALYs’ share in those countries, the greater is the potential return on investment. Since DALY estimates are not yet available for the years 2018 and 2019, estimates for these 2 years were computed by multiplying the DALYs for the year 2017 by the average annual change since 2000. The variable User_fee is used to capture the imposed fee since 2011 by the FDA for performing a faster review. The fee may act as a turn-off and be against the PRV’s interest, particularly for smaller companies with lower profit margins. The variable \(\mathrm{User}\_\mathrm{fee}\) takes a non-zero value only for trials targeting eligible diseases and registered in ClinicalTrials.gov from 2011 onwards. The fee has been increasing over the years and as it is unlikely to have an immediate impact on trial registration, we considered potential lagged effects. More precisely, in the main specification we assumed a 1-year lag effect of the user fee value on trial activity (e.g. the user fee of the fiscal year 2014 is assumed to affect trial registration in 2015, etc.), but which we extended to further lags in the robustness section. We also tested for a longer delayed effect of the policy (up to 10-year lags) for diseases that were eligible since the policy started. As for the two policy modifications that took place in 2014, i.e. selling the voucher multiple times and the reduction to a 90-day notice to the FDA, these are captured as part of the overall policy effect for the diseases becoming eligible as of 2014. We also controlled for the fact the PRV was extended to rare pediatric conditions in 2012, which increased the number of vouchers awarded. The model includes registry fixed effects per disease fixed effects (\({\alpha }_{j} \times {\delta }_{i})\) to control for disease registry-specific time-invariant effects. One example is the different regulatory requirements across registries. The model also includes year fixed effects (\({\gamma }_{t})\) to control for year-specific cofounders that impact R&D activity across all registries and diseases.Footnote 2 Therefore, by estimating a two-way error component model, we were able to control for a broad range of factors, including exogenous yearly variations in R&D activity and unobserved time-invariant heterogeneity that are specific to trial registry and disease. To look into the dynamics effect of the PRV policy effect, we considered two further specifications of the model. Since R&D is a lengthy process, we believe it is relevant to look at how the policy effect, if any, varied over time, i.e. if different periods post PRV implementation show a different effect. In model 2 (M2), we bundle the lags together by blocks of 2 years. The first block includes the policy year and the year after, while the second block includes the second and third years post policy implementation, etc. To test for the parallel trend assumption, we follow a similar methodology as in the literature and include leads to M2 [22, 23]. The parallel trend assumption is a prerequisite for DD and DDD analyses and, in the case of a DDD, relies on the assumption that trials for eligible and non-eligible diseases would follow the same path, in the treated and control registries, had the policy not been implemented [17].

In the robustness checks section, we present the results of two robustness tests. The first looks at the policy effect when moving the policy introduction 1 year ahead. We do so to test for an anticipatory effect of the policy that could exist given the lag between announcement and implementation dates. Indeed, the policy was disclosed to the public in 2006 but implemented in 2007, therefore it could be that companies strategically anticipate its implementation and change their behavior. In the second robustness test, we look into additional lagged effect (up to a 4-year lagged effect) of a change in the user fee. R&D activity is a lengthy process and is thus unlikely to be capable of immediately responding (e.g. by switching to another product in the pipeline) to a changing user fee. Therefore, the latter is likely to have a lagged effect (of 1 or more years) on R&D activity. All analyses were undertaken in Stata 16 software (StataCorp LLC, College Station, TX, USA).

3 Results

The marginal effects from the Poisson regressions, along with their standard errors and p-values, are represented in Table 2 for both models 1 (M1) and 2 (M2). The overall marginal effect of the policy (M1) is 0.41 and is not statistically significant, indicating that PRV has had no impact on trial registration. When looking at the dynamics of the policy effect considering potential delays of the policy implementation (M2), regardless of the post policy period, the PRV has had no effect on stimulating trials registration for the intended diseases. Moreover, it is worth adding that even when including phase I trials in the main analysis, the results remain consistent and show no effect. The results of the parallel trend assumption is provided in Online Resource 3 in the ESM and proved to hold as indicated by the lack of statistical significance of the marginal effect for the year prior to the policy introduction (i.e. t-1).

Table 2 Triple-differences marginal effects on the impact of the priority review voucher on clinical trial registrations
Full size table

3.1 Robustness Checks

3.1.1 Changing the Year of Policy Introduction

Given that the policy was announced to the public a year before its implementation, we hereby test for a potential anticipatory effect of the policy. In addition to this, the policy starting date cannot be fully clear-cut from the data available. More specifically, we do not know whether trials registered before September 2007 were still ongoing as of 26 December 2007. As a result, we made the restrictive assumption in the main analysis that trials registered in 2006 or before were no longer ongoing at the end of 2007 and thus were not considered as part of the ‘after’ period. Likewise for the diseases that became eligible in 2014, 2015, 2016, and 2018. Therefore, we simulated the policy to have taken place 1 year earlier than its true year of introduction (i.e. 2006 instead of 2007, 2013 instead of 2014, etc.) but which did not qualitatively affect the results. The PRV remains ineffective at stimulating trial activity, thereby confirming the robustness of our findings.

3.1.2 Testing for Lagged Effect of the User Fee on Trials Registration

In the analysis, we have assumed a 1-year lagged effect of the covariate user fee on trial registration. To relax this assumption, we ran the model and tested for a potential longer delayed effect (up to 4 years) of the variable. A 4-year delayed effect would imply that the user fee’s value in 2011 would affect trial registration in 2015, but considering delayed effect of the user fee on trial activity did not affect the results. This would suggest that the value chosen and imposed by the FDA is not responsible for the PRV’s lack of effect.

3.1.3 Controlling for the Priority Review Voucher Extension to Rare Pediatric Conditions

In 2012, the PRV was extended to include rare pediatric conditions, which vastly increased the supply of PRVs and likely contributed to the downward trend in the price of the vouchers and investment incentives. We therefore included the pediatric PRV as an additional cofounding variable. This did not affect the results; the PRV decreased slightly but with a coefficient that remains not statistically significant.

4 Discussion

The PRV was implemented by the US congress in 2007 to encourage pharmaceutical investment in R&D for diseases of the poor. Given the specificities of the program, we were able to employ a DDD strategy to assess the PRV’s impact on trial activity. Our findings show that the program has not been associated with stimulating the number of trials for new products. Delayed effects of the policy could also not be found, with all the marginal effects lacking statistical significance. The user fee imposed by the FDA since 2011 to perform a priority review does not seem to be a reason for the policy’s lack of effect. Our findings suggest that the PRV reward, i.e. a 4-month anticipated review, whether used or sold, is not sufficient to generate R&D incentives for neglected diseases. Indeed, it seems reasonable to believe that large pharmaceutical companies, some with yearly revenues exceeding $50 billion, are unlikely to shift or expand their portfolio towards risky projects for tropical diseases based solely on a voucher that can be sold for as low as $68 million. Furthermore, even if sold at its highest price ($338 million), it would not be sufficient to cover the total cost of developing and launching a new product. While it is true that large pharmaceutical companies may have a greater interest in using the voucher rather than selling it (as they are more likely to have a blockbuster product in their pipeline), the benefit of a 4-month earlier entry on the market may not be a sufficiently strong incentive, especially against the high investment required in late-stage trials. Accordingly, if pharmaceutical companies are involved in such diseases projects, it is more likely to be within product-development partnerships (PDPs).

However, while we believe this study is the first to thoroughly evaluate the PRV for tropical diseases, we must highlight its various limitations, which mainly relate to the quality of the available data. First, we only had data on trial activity from 2005 onwards, which implies only 2 years preceding the policy enactment for diseases that became eligible in 2007. Unfortunately, data on trial activity is incomplete before 2005, as before that year it was not yet compulsory to register trials in order to be eligible to later publish the results. Nevertheless, given that we assess a staggered implementation of PRV, for diseases added later on, the before period is much longer and the overall parallel trend assumption proved to hold. Second, trials may be missing in the WHO platform. Having said this, if trials are equally missing across registries between the eligible and non-eligible diseases, then the issue is significantly minimized by using a DDD approach. Third, there is the issue of multicentered trials—trials may take place in different countries/regions simultaneously and end up being registered in more than one registry. In such cases, the ICTRP shows more than one record and bridges those into a single trial (and only the oldest trial will appear when downloading the data). Nevertheless, it seems that multiple records are few (e.g. 279 records for 273 trials were found for leishmaniasis, which implies 6 multicentered trials), and if this occurs equally for the eligible and non-eligible diseases, then again the issue no longer stands when using a DDD approach. Fourth, even though the PRV is only valid for products that “contain no active ingredient that has been approved in any other application”, we were not able, given the data available, to exclude those that failed to meet this requirement. This problem is nonetheless counterbalanced by the fact that drug combinations, usually very common for neglected diseases, with at least one new active moiety are eligible for the voucher. Lastly, while we believe it would have been relevant to look at the possible impact of the PRV across sponsor types (for-profit versus not-for-profit) and on outcome variables other than trial registration, such as the time to market launch, probability of market launch, or probability to move successfully across the clinical phases, we were not capable of doing so given the data available. Indeed, the data available do not permit to link trials into identifiable projects.

5 Conclusion

While we believe that the PRV is a great idea in theory, the reality is different and characterized by a lack of evidence in stimulating R&D, particularly for completing late-stage drug development, as shown by our study. Our findings suggest that the PRV alone may not be sufficient to promote R&D incentives for NTDs. The PRV may need to be supplemented with other types of government incentives, such as R&D grants and advance market commitment (AMC), particularly for diseases that may not generate sufficient sales. Indeed, it appears that these mechanisms, including the PRV, should not be considered as stand-alone solutions but as complementary solutions, together needed to stimulate R&D for diseases of the world’s poorest populations. Whether the PRV is to be maintained or not is potentially beyond the scope of this study. Nevertheless, if the program is to continue, several improvements must be made. First, the PRV should require a minimum level of novelty: a product cannot be granted a voucher if it was already licensed outside the US. Doing so would restrict the number of awarded vouchers (i.e. the supply) and slow down the ongoing depreciation of their market value. Second, the PRV should require sponsors to guarantee access to PRV-awarded health products with the submission of an access plan.