Improvement in clinical trial disclosures and analysts’ forecast accuracy: evidence from the pharmaceutical industry

Abstract

This paper examines whether financial analysts use the information contained in clinical trial disclosures to improve their forecast accuracy for pharmaceutical companies. Findings indicate that the improved clinical trial disclosures due to a quasi-regulation issued by the International Committee of Medical Journal Editors (ICMJE) significantly reduce analysts’ long-term forecast error. In addition, a propensity-score matching analysis provides additional strong evidence that issuance of the 2005 ICMJE’s regulation is accompanied by an average 45 % decrease in long-term forecast error, and a more than 50 % decrease in long-term forecast dispersion. This study contributes to the accounting literature regarding nonfinancial disclosures by providing the first insights into financial analysts’ use of clinical trial disclosures in their forecasts of future earnings. In addition, because the major event examined in this study is a quasi-regulation issued by the ICMJE, we provide additional insights on the effectiveness of industry-initiated regulations (or quasi-regulations) on nonfinancial disclosure practice.

Appendices

Appendix 1

Acronym	Term
AICPA	American Institute of Certified Public Accountants
CSR	Corporate social responsibility
FDA	Food and drug administration
FDAAA	Food and drug administration amendments act
GAAP	Generally accepted accounting principles
ICMJE	International Committee of Medical Journal Editors
IND	Investigational new drug
NDA	New drug application
NIH	National Institutes of Health
PhRMA	Pharmaceutical Research and Manufacturer of America
R&D	Research and development
SIC	Standard industry classification

Appendix 2: New drug development and approval process

The new drug development and approval process generally takes six steps, and two formal applications—the investigational new drug (IND) and new drug application (NDA)¹⁸—which need to be filed with and approved by the FDA before a new drug is approved for large-scale manufacture and marketing in the U.S. The six steps include, (1) pre-discovery, (2) new drug discovery, (3) pre-clinical trial and IND application, (4) clinical trials Phases I, II and III, (5) FDA review and NDA approval, and (6) post-marketing clinical trials Phase IV. The new drug development is an enormously time consuming process. According to the data of U.S. PhRMA, steps 2 through 5 generally take about 10 to 15 years, within which the first three phases of clinical trials (step 4) alone will take about six to seven years. Moreover, only one out of every 5000 compounds identified during the initial drug discovery process will eventually be approved for marketing by the FDA after this long and costly R&D process (PhRMA 2007).

The new drug development and approval process is summarized in Fig. 1, as discussed below.

Fig. 1

Source Drug discovery and development by the pharmaceutical research and manufacturers of America (PhRMA) (2007)

New drug development and approval process.

1.1 Pre-discovery and new drug discovery

The new drug development process starts with gaining an understanding of the diseases to be treated and the underlying causes of the health condition. During the pre-discovery phase, pharmaceutical researchers will select a target, likely a gene or protein involved in a particular disease, which can potentially interact with and be affected by a drug molecule (PhRMA 2007). Once the target is selected, the second step is to identify a promising drug molecule, or a lead compound, that may act on the target to alter the disease course. This stage is defined as the new drug discovery process, during which a series of lab tests are performed on several promising compounds, to test for safety and help researchers prioritize lead compounds and identify optimal ones.

1.2 Pre-clinical trials and IND application

Once a set of optimal compounds is identified, pre-clinical trials are performed to determine whether the optimal compounds identified from the drug discovery process are safe enough to be tested on human beings. During this phase, pharmaceutical researchers will perform thorough laboratory and animal tests, as requested by the FDA, to discover how the drug works, and whether it is likely to be safe and work well in humans. After pre-clinical testing on animals is completed, an IND is filed with the FDA to request approval for clinical trials on human beings. The new drug discovery and pre-clinical trials, together, take an average three to six years. After starting with approximately 5000 to 10,000 compounds, scientists will have narrowed the group down to between one and five “candidate drugs” which will be studied in clinical trials (PhRMA 2007). New drug discovery research and pre-clinical testing results are often voluntarily and selectively published in scientific journals, and companies only sporadically and selectively disclose pre-clinical research and patent approvals in their financial reports (Joos 2003).

1.3 Clinical trials (phase I, II, and III)

A clinical trial is a research study in human volunteers. The clinical trial process is both expensive and time-consuming, and ends more often in failure than success (PhRMA 2007). There are three phases of clinical trials the candidate drugs must pass through before an NDA can be filed with the FDA. In Phase I of the clinical trial, a candidate drug is tested for the first time on human beings with the disease or condition under study. The trial is conducted on a small group of patients, normally ranging from 20 to 100, to assess its safety, determine a safe dosage range, and identify any side effects. In Phase II, a candidate drug is provided to a larger group of patients with the disease or condition under study, to evaluate its effectiveness and further assess its safety. The usual sample size during Phase II clinical trials is between 100 and 500 patients. In a Phase III clinical trial, the sample size is increased to between 1000 and 5000 patients. Phase III trials are both the costliest and longest trials, during which hundreds of sites around the U.S. and the world participate. This enables the study to obtain a large and diverse group of patients, and generate statically significant data about the safety, effectiveness and overall benefit-risk relationship of the drug (PhRMA 2007).

The first three phases of clinical trials take an average of six to seven years to complete. During any of these three phases, either the FDA, or the pharmaceutical company who sponsors the study, can stop the testing, due to significant adverse effects or lack of effectiveness.

1.4 FDA final review and NDA approval

If the candidate drug successfully completes the first three phases of clinical trials, the pharmaceutical company will file an NDA with the FDA to request approval for marketing and large-scale manufacture. The FDA will review the application from the perspective of safety and effectiveness. Based on the review, the FDA can either: (1) approve the new drug, (2) request additional information or testing before final approval is granted, or (3) deny the application. The FDA review and approval normally takes between six months to two years. It is estimated that only one out of 5000 to 10,000 compounds identified during the initial drug discovery process will ultimately be approved by the FDA for large-scale manufacture and marketing (PhRMA 2007).

1.5 Post-marketing clinical trial (phase IV)

Even after the approval for marketing and large-scale manufacture, the testing of the new drug does not stop. The FDA requires pharmaceutical companies to continue monitoring the drug for any unexpected serious adverse effects after it is used in the larger population. These studies on approved drugs are Phase IV clinical trials. Failure to promptly conduct any mandatory Phase IV clinical trials or failure of the product during the post-market testing could result in withdrawal of approval or other legal sanctions (Optimer 2011). For example, one of Merck’s blockbuster drugs, Vioxx, was approved by the FDA for relief of the pain associated with arthritis in 1999. Before the FDA’s approval, Vioxx had been tested by 8076 patients during the Phase III clinical trials (Greener 2005). However, on September 30, 2004, Vioxx was withdrawn from the market due to the increased risk of heart disease. The withdraw of Vioxx is an example of failure on post-marketing clinical trials, and it also shows the significant consequence of negative results from clinical trials.

In summary, conducting clinical trials is a lengthy and expensive process, and it ends more often in failure than success. Therefore, a balanced, complete, and accurate disclosure of clinical trial status, especially the results from the latter phases of the drug development cycle, will enhance the market’s assessment of the pharmaceutical companies’ R&D pipeline and accordingly its revenue generating capabilities.

Appendix 3: Propensity score matching analysis and balance testing

Our purpose is to match pre-2005 and post-2005 observations with similar ex ante firm characteristics, to isolate the effects of the 2005 ICMJE’s editorial policy on analyst forecast accuracy. Rosenbaum and Rubin (1983, 1984) suggest that propensity score matching analysis offers a good control group design and reduces potential bias by largely eliminating pre-treatment characteristic differences when matching treatment and control groups.

Let $Y_{i}$ be the outcome of interest (i.e. forecast error and forecast dispersion), where i represents each firm-year observation. We want to estimate the causal effect of treatment T _i on various outcome variables. In our case, the treatment is the issuance of the 2005 ICMJE’s editorial policy (T _i  = 1). We define the treatment effect for an observation i as $D_{i}$ = $Y_{i}^{1} - Y_{i}^{0}$, where $Y_{i}^{1}$ represents the outcome of observation i with treatment ($T_{i}$ = 1, post-2005) and $Y_{i}^{0}$ represents the outcome without treatment ($T_{\text{i}}$ = 0, pre-2005). However, for any observation i we can only observe its outcome in one of the two possible regimes. We do not have the counter-factual evidence (i.e., the outcome in the unobserved regime). The most common evaluation parameter of interest is the average treatment effect on the treated, $= E\left( {Y_{i}^{1} - Y_{i}^{0} |T_{i} = 1} \right) = E[Y_{i}^{1} |T_{i} = 1] -$ $E[Y_{i}^{0} |T_{i} = 1]$, which measures the average treatment effect of the ICMJE’s editorial policy on analyst forecast accuracy. In this case, however, the counter-factual, $E[Y_{i}^{0} |T_{i} = 1]$, is unobserved. Under specific identifying assumptions (e.g., the conditional independence assumption and common support condition), we adopt the propensity-score matching techniques pioneered by Rosenbaum and Rubin (1983) to construct a valid counterfactual estimator for the average outcome of treated observations. To implement this, we define the propensity score as the probability of receiving the treatment given a set X = $x_{i}$ of firm characteristics: $p\left( X \right) = pr\left( {T_{\text{i}} = 1 |X = x_{i} } \right)$. We estimate the propensity score using a Logit model with various firm characteristics, as listed for model 4 and model 5 (control variables). The STATA report shows that the balancing property holds within all five blocks for all covariates across treatment and control groups, which indicates that the propensity score’s distribution is similar across groups within each block and that the propensity score is properly specified.¹⁹ Table 7 presents the Logit regression results.

Table 7 Propensity score estimation

Then, we match each treatment deal with a control deal on the basis of similar propensity scores. The average treatment effect of the ICMJE’s editorial policy on the treated deals (ATT) is estimated by computing the expected value of the difference in the outcome variable between each of the treated observations and the matched control observations. To ensure that our matching results are robust, we choose three of the most commonly used matching methods: nearest-neighbor matching, radius matching, and kernel matching. As to radius matching, we follow the general rule of thumb and use a caliper of 0.20 of the standard deviation of the propensity score (Austin 2011; Guo 2013). In the computation we also impose common support, to ensure that the matching estimation is taken in the region of common support. Finally, we evaluate the balance of our matched groups with a paired t-test between the treatment and control observations, which tests whether there are significant differences in the covariate means for both groups. Table 8 reports an example of the balancing test results after kernel matching. In unmatched samples, a few covariates (e.g., ANANO and FHORIZON) present significant differences between the treated and control groups. However, after matching, the t-statistics demonstrate that we fail to reject the hypothesis that the mean differences for all covariates between treated and control deals are equal to zero. Overall, we ensure the effectiveness of our chosen propensity score specification by accounting for selection bias in our sample.

Table 8 Balancing tests from kernel matching estimators