A Simple Approach to Incorporating Historical Control Data in Clinical Trial Design and Analysis

Abstract

With rising costs and prolonged timelines for drug development, more innovative trial designs are critical to improve efficiency. Use of available historical control data in a new trial can reduce the number of control patients and accordingly reduce costs and timelines. A major limitation of historical data borrowing is potential prior-data conflict. This difference can increase false decision rates and confound the outcome interpretation. The potential inflation in both type I error rate and type II error rate should be clearly characterized and controlled during the trial design stage. In this paper, we develop a simple approach to incorporating historical control data in clinical trial design and analysis. First we provide a simple statistical approach to evaluating design properties when using a Bayesian approach to incorporating historical data. We then propose a six-step process for trial design including selection and summarization of historical control data, sample size determination with and without borrowing, design property evaluation, and how much historical data should be borrowed to control false positive/negative rate inflation based on trial variability. A detailed procedure to select historical control data is also provided. Finally, we use an example to illustrate our approach. The simplicity of methodology (no simulation required), the streamlined processes for data selection, and explicit evaluation of the impact of prior-data conflict on type I error rate and power make the proposed approach statistically rigorous and easy to understand and implement.

Appendices

Appendix 1

$$\mathrm{Pr}\left(Z>C|{y}_{0,h}\right)=\mathrm{Pr}(\frac{{\widehat{\mu }}_{1, f}-{a}_{0}{\widehat{\mu }}_{0, h}-\left(1-{a}_{0}\right){\widehat{\mu }}_{0,c}}{\sqrt{var\left({\widehat{\mu }}_{1, f}\right)+{a}_{0}^{2}var\left({\widehat{\mu }}_{0,h}\right)+{\left(1-{a}_{0}\right)}^{2}var\left({\widehat{\mu }}_{0,c}\right)}}>C|{y}_{0,h})$$

$$=\mathrm{Pr}(\frac{{\widehat{\mu }}_{1, f}-\left(1-{a}_{0}\right){\widehat{\mu }}_{0,c}-\left({\mu }_{1,f}-\left(1-{a}_{0}\right){\mu }_{0,f}\right)}{\sqrt{{\left(1-{a}_{0}\right)}^{2}var\left({\widehat{\mu }}_{0,c}\right)+var\left({\widehat{\mu }}_{1}\right)}}>\frac{C\sqrt{{var\left({\widehat{\mu }}_{1}\right)+a}^{2}var\left({\widehat{\mu }}_{0,h}\right)+{\left(1-{a}_{0}\right)}^{2}var\left({\widehat{\mu }}_{0,c}\right)}+{a}_{0}{\widehat{\mu }}_{0,h}-\left({\mu }_{1,f}-\left(1-{a}_{0}\right){\mu }_{0,f}\right)}{\sqrt{{\left(1-{a}_{0}\right)}^{2}var\left({\widehat{\mu }}_{0,c}\right)+var\left({\widehat{\mu }}_{1}\right)}}|{y}_{0,h})$$

$$=\Phi \left(\frac{\left({\mu }_{1,f}-{\mu }_{0,f}\right)-{a}_{0}{(y}_{0,h}-{\mu }_{0,f})-C\sqrt{{a}^{2}var\left({\widehat{\mu }}_{0,h}\right)+{\left(1-{a}_{0}\right)}^{2}var\left({\widehat{\mu }}_{0.c}\right)+var\left({\widehat{\mu }}_{1}\right)}}{\sqrt{{\left(1-{a}_{0}\right)}^{2}var\left({\widehat{\mu }}_{0,c}\right)+var\left({\widehat{\mu }}_{1}\right)}}\right)$$

$$=\Phi \left(\frac{\left({\mu }_{1,f}-{\mu }_{0,f}\right)}{\sigma \sqrt{\frac{1}{{n}_{1}}+\frac{1-{a}_{0}}{{n}_{0,h}+{n}_{0,c}}}}-\frac{{a}_{0}\left({(y}_{0,h}-{\mu }_{0,f}\right)}{\sigma \sqrt{\frac{1}{{n}_{1}}+\frac{1-{a}_{0}}{{n}_{0,h}+{n}_{0,c}}}}-C\frac{\sqrt{\frac{1}{{n}_{1}}+\frac{1}{{n}_{0,h}+{n}_{0,c}}}}{\sqrt{\frac{1}{{n}_{1}}+\frac{1-{a}_{0}}{{n}_{0,h}+{n}_{0,c}}}}\right),$$

which is equal to (5) with C = ${{\Phi }^{-1}\left({p}_{0}\right)=\Phi }^{-1}\left(1-\alpha \right), var\left({\widehat{\mu }}_{0,h}\right)=\frac{{\sigma }^{2}}{{n}_{0,h}}, var\left({\widehat{\mu }}_{0,c}\right)=\frac{{\sigma }^{2}}{{n}_{0,c}}, var\left({\widehat{\mu }}_{1, f}\right) =\frac{{\sigma }^{2}}{{n}_{1}}$.

Appendix 2 Select relevant historical control data

Concerns abound about using historical control data to supplement or replace a control arm in a randomized trial (Neunschwander et al., 2010; [4]. Pocock [12] proposed six criteria to evaluate whether there is adequate similarity between historical controls and in-study controls. The criteria are comprehensive and still very relevant today, and therefore are included in the following discussion.

• i. Patient population To ensure patient populations are similar, enrollment criteria must be compared among all selected historical trials and the new trial. Baseline characteristics including demographics and disease status, if available, should also be compared among selected historical trials.

• ii. Prior and concomitant therapy In many trials control patients have received/failed or are on some background therapy that may affect the response level. For example, in rheumatoid arthritis trials, patients with inadequate response to biologics may have much lower response level compared to biologics naïve patients. All prior therapy and concomitant therapies must be compared among historical trials and to the proposed allowable therapies for the control patients in the new trial.

• iii. Control treatment If the control is not a placebo, then the control treatment (including dosage and regimen) must be the same for the historical trials and the new trial.

• iv. Endpoints The patients have been evaluated similarly and the endpoint to measure the patient response must be the same. This means that they are the same measurement defined in the same way, at the same follow-up time point, and evaluated in the same fashion. If the endpoint includes some lab component, central lab reading is recommended for all trials.

• v. Regions Ideally the historical trials and new trials should be conducted in the same regions. If it is known that in a particular region the control response rate is significantly higher, this region should be removed or accounted for to align the response rates.

• vi. Contemporaneity Because of advancement of general health care, there might be a drift over time in the response rate even for the same patients. Technology may also affect patient diagnosis and endpoint assessment. So it is cautioned to include very old historical trials.

• vii. Any other factors For example, as mentioned in Pocock [12], significantly different enrollment rates between trials may lead to subtle difference in patient response. Another example is that there is some plausible possibility that a smaller chance to receive control in multiple-arm trials may lead to higher placebo response rate.

It should be emphasized that the relevance of the historical data or similarity among historical control and in-study control patients is not determined by the similarity between historical control and current control data. If a historical trial had the same control response as the in-study control response, but the patient populations are very different, the historical control data should not be considered relevant and thus should not be used. On the other hand, if historical control patients are deemed similar but between-trial variability is large, there might be little advantage to borrowing historical control patients.

It is acknowledged that one can never find completely identical historical control patients to match the new in-study control patients. The differences should be carefully evaluated jointly by, at minimum, statisticians and medical doctors, to make sure they will not lead to difference in response rate larger than usual variation.

If significant differences in some of the above aspects are found between historical trials, borrowing using the approach in this paper may not be applicable. This does not mean that historical control cannot be used. If patient level data are available, one can borrow the data after accounting for these different factors with covariate adjustment (Ibrahim, 2018) or propensity score matching or regression [24]. Of course, in this situation, it is hard to incorporate borrowing in the design stage since new in-trial data are not yet available. Historical trials should be selected during trial design stage and documented appropriately in the protocol or statistical analysis plan.