Implementing Optimal Designs for Dose–Response Studies Through Adaptive Randomization for a Small Population Group
 390 Downloads
Abstract
In dose–response studies with censored timetoevent outcomes, Doptimal designs depend on the true model and the amount of censored data. In practice, such designs can be implemented adaptively, by performing dose assignments according to updated knowledge of the dose–response curve at interim analysis. It is also essential that treatment allocation involves randomization—to mitigate various experimental biases and enable valid statistical inference at the end of the trial. In this work, we perform a comparison of several adaptive randomization procedures that can be used for implementing Doptimal designs for dose–response studies with timetoevent outcomes with small to moderate sample sizes. We consider singlestage, twostage, and multistage adaptive designs. We also explore robustness of the designs to experimental (chronological and selection) biases. Simulation studies provide evidence that both the choice of an allocation design and a randomization procedure to implement the target allocation impact the quality of dose–response estimation, especially for small samples. For best performance, a multistage adaptive design with small cohort sizes should be implemented using a randomization procedure that closely attains the targeted Doptimal design at each stage. The results of the current work should help clinical investigators select an appropriate randomization procedure for their dose–response study.
KEY WORDS
Doptimal randomization design small population group timetoevent outcome unequal allocationINTRODUCTION
Multiarm clinical trials are increasingly used in modern clinical research. Some examples of multiarm trials include phase II dose–response studies (1), drug combination studies (2), multiarm multistage (MAMS) designs (3,4), and master protocols to study multiple therapies, multiple diseases, or both (5). A benefit of multiarm trials is the ability to test many new promising treatments and address multiple research objectives within a single protocol, thereby potentially speeding up research and development processes compared to a sequence of singlearm or twoarm trials (6).
When designing a multiarm trial, an important consideration is the choice of the allocation ratio, i.e., the target allocation proportions across the treatment arms. The choice of the allocation ratio usually stems from the study objectives. Many clinical trials are designed with an intent to have equal allocation to the treatment groups, which is consistent with a principle of “clinical equipoise” and frequently leads to maximum statistical power for treatment comparisons (e.g., if the primary outcome variance is constant across the groups) (7). On the other hand, unequal allocation designs have recently gained considerable attraction (8,9). For instance, unequal allocation designs may be preferred over equal allocation designs under the following circumstances: (i) in studies with nonlinear dose–response estimation objectives (10, 11, 12, 13); (ii) when there is heterogeneity of the outcome variance across the treatment arms (14, 15, 16); (iii) when there is an ethical imperative to allocate greater proportion of study patients to superior treatment arms (17, 18, 19); (iv) when there is an unequal interest in certain treatment comparisons (20); and (v) when there is a differential treatment cost and an investigator wants to get most power for the given budget (21). Importantly, unequal allocation designs can involve noninteger (irrational) numbers. For example, in a \( \left(K>2\right) \)arm trial comparing \( \left(K1\right) \) experimental treatments versus control (Dunnett’s procedure), the optimal allocation ratio minimizing the sum of variances of the \( \left(K1\right) \) pairwise comparisons is given by \( {\sigma}_1\sqrt{K1}:{\sigma}_2:\dots :{\sigma}_K \), where σ_{i} is the standard deviation of the outcome in the ith treatment group (7).
Once the target allocation ratio is chosen, a question is how to implement it in practice. It is well recognized that randomization is the hallmark of any wellconducted clinical trial (22). When properly implemented, randomization can promote selected study objectives while maintaining validity and integrity of the study results (23). There is a variety of randomization designs that can be applied in multiarm trials with equal or unequal integervalued allocation ratios. The most common one is the permuted block design (PBD) for which treatment assignments are made at random in blocks of a given size to achieve the desired allocation ratio C_{1} : C_{2} : … : C_{K}, where C_{i}’s are positive, not necessarily equal, integers with the greatest common divisor of 1. The PBD has been criticized by some authors as being too restrictive and susceptible to selection bias (24,25). Some alternatives to the PBD have been developed recently (26, 27, 28).
For multiarm trials with unequal allocation involving noninteger (irrational) proportions, the choice of a randomization design is less straightforward, as highlighted in (29). The simplest approach is to use complete randomization (CR) for which treatment assignments are generated independently, according to a multinomial distribution with cell probabilities equal to the target allocation proportions. A major drawback with CR is that it can result with nonnegligible probability in large departures from the desired allocation, especially in small trials. One useful alternative to CR is the mass weighted urn design (MWUD) which was shown to maintain a good tradeoff between treatment balance and allocation randomness (29). Other designs for irrational target allocations can be constructed by adopting the methodology of optimal responseadaptive randomization (30). Some promising designs for this purpose are the doubly adaptive biased coin design (31), the generalized droptheloser urn (32), and the optimal adaptive generalized Pólya urn (33), to name a few. However, all these designs rely on asymptotic results which may not hold in small to moderate sample sizes that are common in practice.
The present paper is motivated by our recent work (34) which investigated the structure of the Doptimal design for dosefinding experiments with timetoevent data. In particular, we found that for a quadratic dose–response model with Weibull outcomes that are subject to right censoring, the equal allocation (1:1:1) design can be highly inefficient when the amount of censoring is high. The Doptimal design is supported at 3 points, but the location of these points in the dose interval, as well as the optimal allocation proportions at these points, depend on the true model and the amount of censored data in the experiment. As such, the Doptimal allocation proportions are found through numerical optimization and they are generally quite different from the equal allocation. A twostage adaptive design was proposed and it was found to be nearly as efficient as the true Doptimal design. The authors of (34) also mentioned that practical implementation of the adaptive Doptimal design requires a judicious choice of a randomization procedure. Given that, in practice, dosefinding studies are relatively small (due to budgetary and ethical constraints), it is imperative that the chosen randomization procedure can closely attain the desired optimal allocation for small and moderate samples while maintaining the randomized nature of the experiment. Our main conjecture in the present paper is that the choice of randomization for the Doptimal design does matter as far as statistical properties such as quality of dose–response curve estimation are concerned.
The remainder of this paper is organized as follows. In the “MATERIALS AND METHODS” section, we give a statistical background and overview of randomization designs that can be used to target multiarm unequal allocation with possibly noninteger (irrational) proportions for trials with small and moderate sample sizes. In the “SIMULATION STUDY PLAN” section, we outline a strategy to investigate statistical properties of selected randomization procedures targeting the Doptimal design. The “RESULTS” section presents findings from our simulations, which includes a study of singlestage randomization procedures targeting locally Doptimal design, twostage adaptive optimal designs, and multistage adaptive designs with early stopping rules. We also explore robustness of our proposed designs to experimental (chronological and selection) biases. The “DISCUSSION” section concludes with a summary of our main findings and outlines some important future work.
MATERIALS AND METHODS
DOptimal Design
The study objective is to estimate the vector of model parameters θ = (β_{0}, β_{1}, β_{2}, b) as precisely as possible. For this purpose, we consider designs of the form ξ = {(x_{1}, ρ_{1}); (x_{2}, ρ_{2}); (x_{3}, ρ_{3})}, where x_{i}’s are distinct dose levels in \( \mathcal{X}=\left[0,1\right] \) and ρ_{k}’s are allocation proportions at these doses (0 < ρ_{k} < 1 and \( {\sum}_{k=1}^3{\rho}_k=1 \)). The design’s Fisher information matrix is a weighted sum \( \mathbf{M}\left(\xi, \boldsymbol{\theta} \right)={\sum}_{k=1}^3{\rho}_k{\mathbf{M}}_{x_k}\left(\xi, \boldsymbol{\theta} \right) \), where \( {\mathbf{M}}_{x_k}\left(\xi, \boldsymbol{\theta} \right) \) is the Fisher information matrix for a single observation at dose x_{k} (it is a 4 × 4 matrix whose expression is given in Eq. (6) in (34)). The locally Doptimal design ξ^{∗} minimizes − log ∣ M(ξ, θ)∣, which leads to the smallest volume of the confidence ellipsoid for θ. In (34), it was found that if there is no censored data in the experiment, then ξ^{∗} is the uniform (equal allocation) design, supported at dose levels 0, 1/2, and 1. However, in the presence of censoring, the structure of ξ^{∗} is more complex—both optimal dose levels and the allocation proportions depend on the true model and the amount of censoring, i.e., ξ^{∗} = {(x_{k}(θ), ρ_{k}(θ)), k = 1, 2, 3}, and ξ^{∗} must be found numerically, using, for example, a firstorder (exchange) algorithm (35). Since in practice θ is unknown, one can construct a twostage adaptive Doptimal design as follows. At stage 1, a cohort of n^{(1)} subjects is allocated to doses according to the uniform design ξ^{(1)} = {(0, 1/3); (0.5, 1/3); (1, 1/3)}. Based on observed data \( {\mathcal{F}}_{n^{(1)}}=\left\{\left({t}_i,{\delta}_i,{x}_i\right),i=1,\dots, {n}^{(1)}\right\} \), compute \( {\widehat{\boldsymbol{\theta}}}_{MLE}^{(1)} \), the maximum likelihood estimate (MLE) of θ, and approximate ξ^{∗} by \( {\overset{\sim }{\xi}}^{\ast }=\left\{\left({x}_k\left({\widehat{\boldsymbol{\theta}}}_{MLE}^{(1)}\right),{\rho}_k\left({\widehat{\boldsymbol{\theta}}}_{MLE}^{(1)}\right)\right),k=1,2,3\right\} \). At stage 2, additional n^{(2)} subjects are allocated to doses according to \( {\overset{\sim }{\xi}}^{\ast } \). The final analysis is based on data from the pooled sample of n = n^{(1)} + n^{(2)} subjects. In (34), it was shown that such a twostage design provides a very good approximation to, and it is nearly as efficient as, the true Doptimal design without the need for prior knowledge of the model parameters before the start of the trial.
An important open question is how to allocate subjects to doses for both stage 1 and stage 2 of these adaptive designs. The cohort sizes n^{(1)} and n^{(2)} can be small in practice, and an experimenter must ensure that actual allocation numbers are as close as possible (ideally are matching) the targeted ones. At the same time, the allocation must involve a random element to minimize the potential for selection bias (36). Thus, balance and randomization are two competing requirements. There are many randomization procedures that can be used implementing Doptimal allocation (22). In the next section, we describe a selection of procedures that are relevant to our study.
Randomization Procedures for Implementing DOptimal Design
To fix ideas, we start with an “idealized” setting when both the true model (θ) and the amount of censored data are known, and therefore the Doptimal design ξ^{∗} = {(x_{k}(θ), ρ_{k}(θ)), k = 1, 2, 3} is available to the experimenter. We shall also use notations d_{1}, d_{2}, and d_{3} to indicate the optimal dose levels x_{1}(θ), x_{2}(θ), and x_{3}(θ), respectively. For dose d_{k} (k = 1, 2, 3), the optimal proportion is \( {\rho}_k^{\ast }={\rho}_k\left(\boldsymbol{\theta} \right) \) (possibly an irrational number), with the obvious constraint of \( {\rho}_1^{\ast }+{\rho}_2^{\ast }+{\rho}_3^{\ast }=1 \).
In other words, for any restricted randomization procedure, the randomization probability for the next eligible subject depends on the current numbers of the dose assignments in the trial.

Completely randomized design (CRD): Every subject is randomized to the dose groups with probabilities equal to the Doptimal allocation, i.e., \( {P}_k(j)={\rho}_k^{\ast } \), j = 1, …, n, k = 1, 2, 3. The CRD is very simple to implement and it provides the highest degree of randomness, but for small samples, it can lead to deviations from the desired allocation with nonnegligible probability (22).

Permuted block design (PBD): To implement allocation (\( {\rho}_1^{\ast },{\rho}_2^{\ast },{\rho}_3^{\ast } \)) for a cohort of size n, the desired split of sample size n among the doses is \( n{\rho}_1^{\ast }:n{\rho}_2^{\ast }:n{\rho}_3^{\ast } \), which, after rounding to the integer values, is, say, C_{1} : C_{2} : C_{3}. Without loss of generality, we can assume that C_{k}’s are positive integers with the greatest common divisor of 1 and C_{1} + C_{2} + C_{3} = n. For the PBD, the conditional randomization probabilities are:

Doubly adaptive biased coin design (DBCD): Initial dose assignments (j = 1, …, m_{0}) are made using PBD with a block size that is a multiple of 3, e.g., m_{0} = 3, 6, or 9. Subsequently, the (j + 1)^{st} subject (j = m_{0}, …, n−1) is randomized to dose d_{k} with probability

Generalized droptheloser urn design (GDLUD): The GDLUD (32) utilizes an urn containing balls of four types: type 0 is the immigration ball, and types 1, 2, and 3 represent “dose” balls. Dose assignments for eligible subjects are made sequentially by drawing a ball at random from the urn. Let \( {Z}_0=\left(1,{\rho}_1^{\ast },{\rho}_2^{\ast },{\rho}_3^{\ast}\right) \) denote the initial urn composition (one immigration ball and \( {\rho}_1^{\ast }+{\rho}_2^{\ast }+{\rho}_3^{\ast } \) “dose” balls). The urn composition is changed adaptively during the course of the trial. Let Z_{j − 1} = (Z_{j − 1,0}, Z_{j − 1,1}, Z_{j − 1,2}, Z_{j − 1,3}) denote the urn composition after j − 1 steps (numbers Z_{j − 1, i},i = 1, 2, 3 can be negative and/or irrational). Let \( {Z}_{j1,k}^{+}=\max \left(0,{Z}_{j1,k}\right) \) and k = 0, 1, 2, 3. At the jth step, the probability of selecting a ball of type k is \( {Z}_{j1,k}^{+}/{\sum}_{i=0}^3{Z}_{j1,i}^{+} \), k = 0, 1, 2, 3. If selected ball is type 0 (immigration), no dose is assigned and the ball is replaced into the urn together with additional \( C{\rho}_1^{\ast }+C{\rho}_2^{\ast }+C{\rho}_3^{\ast } \) “dose” balls (C is some positive constant). Therefore, the urn composition becomes Z_{j, 0} = Z_{j − 1, 0} and \( {Z}_{j,i}={Z}_{j1,i}+C{\rho}_i^{\ast } \), i = 1, 2, 3. If selected ball is type ℓ (ℓ = 1, 2, 3), then it is not replaced, the eligible subject is assigned to the corresponding dose level, and the urn composition becomes Z_{j,ℓ} = Z_{j − 1,ℓ} − 1 and Z_{j, i} = Z_{j − 1,i} for i ≠ ℓ. The described procedure is repeated until a prespecified number of subjects (n) is randomized in the study. The GDLUD has established asymptotic properties (32): the allocation proportions are strongly consistent for the target proportions and follow an asymptotically normal distribution with known variance structure.

Mass Weighted Urn Design (MWUD): The MWUD (29) uses an urn containing three “dose” balls. Initially, each ball has mass proportional to the target allocation: \( {m}_{0,i}=\alpha {\rho}_i^{\ast } \), i = 1, 2, 3 (the parameter α is a positive integer controlling maximum tolerated imbalance). The mass of the balls is changing adaptively, according to the history of dose assignments. Among the balls with positive mass, a ball is drawn with probability proportional to its mass, and the corresponding dose is assigned to the next eligible subject. One unit mass is taken from the selected ball and redistributed among three balls in the ratio \( {\rho}_1^{\ast }:{\rho}_2^{\ast }:{\rho}_3^{\ast } \), after which the ball is returned into the urn. Therefore, after (j − 1) assignments, the probability mass for the ith dose group is \( {m}_{j1,i}=\alpha {\rho}_i^{\ast }{N}_i\left(j1\right)+\left(j1\right){\rho}_i^{\ast } \) and the total mass of the three balls in the urn at each step is \( {\sum}_{i=1}^3{m}_{j1,i}\equiv \alpha \). These steps are repeated until the prespecified number of subjects is enrolled in the study. The MWUD has a simple explicit formula for conditional randomization probability:

Maximum entropy constrained balance randomization (MaxEnt): The MaxEnt procedure is an extension of Efron’s biased coin design (38) to a multiarm setting with unequal allocation (39,40). Dose assignments for eligible subjects are made sequentially. Consider a point in the trial when j − 1 subjects have been randomized into the study, with N_{i}(j − 1) subjects assigned to dose d_{i}, i = 1, 2, 3. The randomization rule for the jth subject is as follows: Compute B_{1}, B_{2}, B_{3}, the hypothetical treatment imbalances which would result from assigning the jth subject to doses d_{1}, d_{2}, d_{3}:
In Eq. (6), \( {B}_{(1)}=\underset{i}{\min }{B}_i \) and η is a userdefined parameter (0 ≤ η ≤ 1) that controls amount of randomness of the procedure (η = 0 is most random and η = 1 is almost deterministic procedure). The explicit solution to problem in Eq. (6) can be found in (40).
Based on their observed data, the Doptimal design is estimated as \( {\widehat{\xi}}^{\ast }=\left\{\left({\widehat{d}}_k,{\widehat{\rho}}_k^{\ast}\right),k=1,2,3\right\} \), and in stage 2, an additional 30 patients are randomized using CRD, namely, the jth patient (j = 31, …, 60) is randomized among the doses \( {\widehat{d}}_1 \), \( {\widehat{d}}_2 \), and \( {\widehat{d}}_3 \) with probabilities \( {P}_k(j)={\widehat{\rho}}_k^{\ast } \), k = 1, 2, 3. We will denote such a twostage adaptive randomization design by PBD → CRD, emphasizing that PBD is used in stage 1 and CRD is used in stage 2.
Likewise, a multistage design PBD → CRD → CRD → ... means that the first cohort of patients is randomized into the study using PBD; the second cohort is randomized according to an updated Doptimal design using CRD; the third cohort is randomized according to an updated (using cumulative outcome data from first two cohorts) Doptimal design using CRD, etc.
Statistical Criteria for Comparison of Randomization Procedures
For given values of n and θ, Deff(n) is, in general, a random variable because ξ_{n} depends on N(n) = (N_{1}(n), N_{2}(n), N_{3}(n)) whose distribution is determined by a randomization procedure. We can take E(Deff(n)) as a measure of estimation precision of a randomization procedure targeting Doptimal allocation. High values of E(Deff(n)) are desirable.
In addition to statistical estimation, we consider several other useful metrics. A measure of allocation accuracy of a randomization procedure is the closeness of the realized allocation to the true Doptimal allocation. For a design with n subjects, an imbalance (using Euclidean distance) is \( Imb(n)=\sqrt{\sum_{k=1}^3{\left({N}_k(n)n{\rho}_k^{\ast}\right)}^2} \). Small values of Imb(n) are desirable; ideally Imb(n) = 0. Since N_{k}(n)’s are random variables, we take expected value, E(Imb(n)), and this is referred to as momentum of probability mass (MPM) (41).
The smaller FI(n) is, the more random (and therefore, potentially less predictable) a randomization procedure is. FI(n) ≡ 0 corresponds to CRD, which is most random and provides no potential for selection bias in the study.
Finally, we consider variability of randomization procedures by examining the average standard deviation of the allocation proportions: \( ASD(n)=\sqrt{n{\sum}_{i=1}^3{\left\{ SD\left({N}_i(n)/n\right)\right\}}^2} \). It is expected that randomization procedures with low values of ASD(n) should be more concentrated around the target Doptimal allocation, and therefore they should lead to more efficient dose–response estimation.
SIMULATION STUDY PLAN
Our simulation study consists of four major parts.
First, we evaluate various singlestage randomization procedures targeting the locally Doptimal design (assuming the true model is known) for small and moderate sample sizes. The design operating characteristics include measures of estimation precision, balance, and randomness, as described in the section “Statistical Criteria for Comparison of Randomization Procedures”.
Third, we implement a multistage adaptive design with early stopping criteria (34) using different combinations of randomization procedures. In this setting, all designs aim at achieving the same predefined level of estimation precision, and the key operating characteristic is the sample size at study termination. Our conjecture is that there are randomization procedures that require a smaller sample size, given the stopping rule.
Fourth, we evaluate the robustness of different adaptive randomization strategies to two types of experimental bias: chronological bias and selection bias. Chronological bias can arise if patient outcomes over time are affected by unobserved time trends (43,44). Selection bias can occur when an investigator knows or is able to guess with high probability which treatment is to be assigned to an upcoming patient (36). The advance knowledge of the treatment assignment can motivate an investigator to selectively enroll a particular type of patients who are thought to benefit most from the given treatment thereby confounding the true treatment effect. The importance of assessing robustness of randomization designs to chronological and selection biases has been recently documented by Hilgers and coauthors in the Evaluation of Randomization procedures for Design Optimization (ERDO) template (45).
Note that our simulation plan here is by no means exhaustive. However, we supply an R code (available upon request from the first author) that can be used to reproduce all results in this paper and generate additional findings under userdefined experimental scenarios and other combinations of adaptive randomization strategies.
RESULTS
Targeting Locally DOptimal Design
We consider seven randomization designs targeting Doptimal allocation (0.407, 0.336, 0.257). These designs are as follows: (I) CRD, (II) DBCD (γ = 2), (III) GDLUD (C = 10), (IV) MWUD (α = 10), (V) MaxEnt(η = 0.5), (VI) MaxEnt(η = 1), and (VII) PBD. We also consider a uniform allocation design which randomizes study subjects among the dose levels 0, 0.5, and 1 in equal proportions by means of PBD, i.e., (VIII) Uniform PBD.
Operating Characteristics of Eight Randomization Designs for a SingleStage Trial with Locally Doptimal Design
Randomization design^{a}  

I  II  III  IV  V  VI  VII  VIII  
n = 15  
Deff(n)  0.93  0.97  0.98  0.98  0.99  1.00  1.00  0.74 
RE(n)  0.99  0.99  0.98  0.99  0.98  1.00  1.00  1.03 
MPM(n)  1.97  1.40  1.35  1.38  0.90  0.50  1.14  0.54 
ASD(n)  0.81  0.46  0.48  0.46  0.30  0  0  0 
FI(n)  0  0.05  0.03  0.02  0.13  0.66  0.11  0.28 
n = 30  
Deff(n)  0.97  1.00  1.00  0.99  1.00  1.00  1.00  0.74 
RE(n)  0.99  1.00  1.00  0.99  1.01  1.00  0.99  0.82 
MPM(n)  2.70  1.51  1.53  1.50  0.94  0.50  1.14  0.54 
ASD(n)  0.81  0.37  0.37  0.33  0.22  0  0  0 
FI(n)  0  0.04  0.04  0.03  0.13  0.66  0.11  0.28 
n = 45  
Deff(n)  0.98  1.00  1.00  1.00  1.00  1.00  1.00  0.74 
RE(n)  0.97  0.98  0.97  0.98  0.97  1.00  0.99  0.67 
MPM(n)  3.25  1.67  1.61  1.53  0.96  0.50  1.14  0.54 
ASD(n)  0.80  0.37  0.32  0.27  0.18  0  0  0 
FI(n)  0  0.03  0.04  0.03  0.13  0.66  0.11  0.28 
n = 60  
Deff(n)  0.99  1.00  1.00  1.00  1.00  1.00  1.00  0.74 
RE(n)  0.91  1.00  0.98  1.00  0.96  1.00  0.99  0.38 
MPM(n)  3.75  1.84  1.67  1.56  0.97  0.50  1.14  0.54 
ASD(n)  0.81  0.36  0.27  0.23  0.16  0  0  0 
FI(n)  0  0.03  0.04  0.03  0.13  0.66  0.11  0.28 
From these results, we can make an important intermediate observation. In an “idealized” setting of a known nonlinear dose–response model with censored timetoevent data, the quality of estimation depends on both the choice of allocation design and the randomization procedure to implement the target allocation. The Doptimal allocation implemented by a randomization procedure with low variability (e.g., MaxEnt(η = 1) or PBD) results in most accurate estimation of the dose–response relationship, especially when sample size is small, e.g., n = 15 (cf. Fig. 2). Using a less restrictive (more random) randomization procedure can result in some deterioration of statistical estimation in small samples; however, the quality of estimation is improved with larger sample sizes. For instance, when the “most random” CRD procedure is applied to target the Doptimal allocation, the average (across 10,000 simulation runs) Defficiency values are 0.93, 0.97, 0.98, and 0.99, respectively, for sample sizes n = 15, 30, 45,and 60 (cf. Table I). Using a nonoptimal allocation (e.g., Uniform design), even with most restrictive and most balanced randomization procedure leads to inferior performance which does not improve with the increase in sample size. In our example, the average Defficiency of Uniform PBD was 0.74 for n = 15, 30, 45,and 60 (cf. Table I).
Of course, our observations here are obtained based on data generated from a selected model in Eq. (1), under one experimental scenario (visualized in Fig. 1) and four choices of the sample size, with n = 15 being the smallest one. Additional simulations under other experimental scenarios and using smaller values of n (e.g., n = 9 or 12) can be performed to investigate loss in efficiency due to imbalance induced by randomization in very small samples. We defer this task to the future work.
Overall, our findings from the considered example are in line with the template of Hu and Rosenberger (46) which suggests that for randomized comparative trials the performance of a randomization design is determined by an interplay between optimality (power) of a fixed allocation design, speed of convergence of a randomization procedure to the desired allocation, and variability of the allocation proportions. In our setting, we deal with estimation, not hypothesis testing; yet, we arrive at a similar conclusion: both Doptimality and variability of a randomization procedure (and, of course, the study size) determine the design performance.
TwoStage Adaptive Optimal Design
To appreciate the impact of randomization on performance of a twostage adaptive optimal design, we compare five adaptive design strategies using different combinations of randomization procedures (CRD, MaxEnt(η = 1), and PBD) at stages 1 and 2. We also include a nonadaptive Uniform PBD as a reference procedure. We use fixed total sample size n = 60 and investigate three choices of the firststage cohort size: n^{(1)} = 15, 30, and 45. For each design strategy, the main concern is dose–response estimation quality, as assessed by Deff(n) and RE(n).
Operating Characteristics of 5 TwoStage Adaptive Optimal Design Strategies and a Fixed Uniform Allocation Design for a Total Sample Size of n = 60
2stage adaptive design  Uniform design  

CRD → CRD  MaxEnt(1) → CRD  MaxEnt(1) → PBD  MaxEnt(1) → MaxEnt(1)  PBD → PBD  PBD  
n^{(1)} = 15; n^{(2)} = 45  
Deff(n)  0.79  0.80  0.81  0.81  0.81  0.74 
RE(n)  0.63  0.63  0.66  0.66  0.65  0.38 
n^{(1)} = 30; n^{(2)} = 30  
Deff(n)  0.84  0.85  0.85  0.85  0.85  0.74 
RE(n)  0.70  0.67  0.70  0.68  0.70  0.38 
n^{(1)} = 45; n^{(2)} = 15  
Deff(n)  0.82  0.83  0.83  0.83  0.83  0.74 
RE(n)  0.64  0.68  0.65  0.65  0.61  0.38 
Based on the results from Table II, one may argue that the improvements due to use of a “more balanced” randomization method such as PBD over a “less balanced” randomization procedure such as CRD are very modest (1–2% in our example). However, these results are obtained under only one experimental scenario, with a limited selection of sample sizes for stages 1 and 2, and a limited selection of the stage 1/stage 2 ratio of the sample sizes (15:45, 30:30, and 45:15 in our example). A more thorough study would be needed to carefully assess an impact of these parameters on the performance of various twostage randomization design strategies.
Percentage of Simulation Runs for a TwoStage Adaptive Design for Which the MLE of θ (and Therefore, an Estimate of the Doptimal Design) Could not Be Obtained Based on Data from Stage 1
Randomization procedure in stage 1  Size of stage 1 (n^{(1)})  

15  30  45  
CRD  23.00%  4.00%  1.70% 
MaxEnt(η = 1)  10.97%  1.20%  0.07% 
PBD  12.30%  1.40%  0.30% 
Adaptive Optimal Designs with Early Stopping
To evaluate an impact of randomization on adaptive designs with early stopping, we consider four competing adaptive design strategies. All designs randomize the first cohort of 15 subjects among the doses 0, 1/2, and 1 using target allocation (1/3, 1/3, 1/3). Thereafter, additional cohorts of 15 subjects are randomized into the study using different randomization procedures targeting an updated Doptimal design, until either the maximum sample size of the study n_{max} is reached, or the study stopping criterion is met. In our simulations, we set n_{max} = 1000. For the study stopping criterion, we use the rule based on the volume of the confidence ellipsoid, described in (34) as follows: the study should stop once \( \left{\mathbf{M}}_{obs}^{1}\left({\widehat{\boldsymbol{\theta}}}_{MLE},\xi \right)\right\le {\left({\eta}^4\left{\widehat{\beta}}_0\right\left{\widehat{\beta}}_1\right\left{\widehat{\beta}}_2\right\left\widehat{b}\right\right)}^2 \), where 0 < η < 1 is a userdefined constant. In our simulations, we explore four choices of η = 0.15; 0.20; 0.25; 0.35. We also include Uniform PBD with the same stopping rule as a reference procedure.
Robustness to Experimental Biases
A recently published ERDO template (45) emphasizes the importance of assessing the robustness of randomization procedures to chronological bias and selection bias. The chronological bias can arise, for example, in a longterm study with slow recruitment where patients enrolled later in the study may be healthier due to an overall improved standard of care, and if treatment assignments are not balanced over time, then treatment comparison may be biased. To mitigate the impact of chronological bias, it is recommended that a randomization design should balance treatment assignments over time, e.g., by means of some kind of restricted randomization (44,47). The potential negative impact of selection bias on statistical inference (test decisions) is acknowledged and well documented (48, 49, 50, 51). Strategies to reduce risk of selection bias exist (52,53); one recommendation is to use less restrictive randomization procedures, such as the maximal procedure (47,54).
The ERDO template provides a general framework for justifying the choice of a randomization procedure in practice. Here, we apply it in a setting of an adaptive randomized threearm trial with censored timetoevent outcomes and the Doptimal allocation.
Chronological Bias
We consider a twostage adaptive design with n = 60 and n^{(1)} = 30 in which data are generated according to model in Eq. (10) with three kinds of time trend described above. For the stepwise trend, we take c = 30, which means that the patients recruited in stage 2 (after interim analysis) are systematically different from the patients recruited in stage 1 of the study. We consider six choices for ν: ν = 0 (no time trend) and ν = 0.5; 1; 2; 5; 10 (time trend is present). We evaluate three different twostage adaptive designs and the uniform allocation design. The key interest is quality of estimation, as assessed by Deff(n) and RE(n).
Selection Bias
We adopt the approach described in (51) but accounting for a threearm randomization setting. We assume the outcome is survival time and longer times indicate better treatment efficacy. An investigator favors an experimental treatment, and if she anticipates that the next treatment assignment is either low or high dose, then she can select a terminally ill patient who meets all eligibility criteria patients to be allocated to this dose group.
We consider a twostage adaptive design with n = 60 and n^{(1)} = 30 in which data are generated according to model in Eq. (11) with ν = 0.5 (selection bias is present) and ν = 1 (no selection bias). We evaluate three different twostage design strategies: CRD → CRD, MaxEnt(η = 1) → MaxEnt(η = 1), and PBD → PBD. As before, we include Uniform PBD as a reference procedure.
DISCUSSION
In this paper, we evaluated impact of randomization on statistical properties of adaptive optimal designs in a timetoevent dose–response study with the Doptimal allocation that involves possibly noninteger (irrational) allocation proportions. To our knowledge, this is the first paper that systematically investigated the choice of a randomization procedure in such a setting.
Optimal designs for dose–response studies with nonlinear models depend on the true model parameters that are unknown in practice. A solution to this problem is to use adaptive optimal designs which attempt to achieve maximal incremental increase in information about the model at each step. Previous work has shown that such adaptive designs can successfully approximate true optimal designs in various dose–response settings (56, 57, 58, 59, 60). Many of these designs were developed for phase I dose escalation trials in which adaptations are applied sequentially, in a nonrandomized manner. On the other hand, phase II dose–response trials use randomized parallel group designs and attempt to gain maximum information about the dose–response over a given dose range for a given sample size. A practical solution for a phase II dose–response study is a twostage adaptive optimal design, for which data from a (pilot) first stage of the trial are used to ascertain an initial estimate of the dose–response curve and use this information to optimize the second stage of the trial. Twostage adaptive designs have been shown to be highly efficient in various settings (34,61, 62, 63, 64). With a twostage design, an important question is how to implement it in practice. Randomization is a powerful tool that can be used to achieve a predetermined treatment allocation ratio in each stage while protecting a study from bias and maintaining validity of the trial results. The choice of the “best” randomization procedure for use in practice can be elusive due to a variety of available methods (22). Many studies do not go into details on how randomization is implemented in practice (45). In the current paper, we provide an example of how different randomization options can be examined to select one for implementation in an adaptive dose–response trial.
We have shown that both the choice of an allocation design and a randomization procedure to implement the target allocation impact the quality of dose–response estimation, especially for small samples. The Doptimal allocation implemented by a randomization procedure with low variability leads to the most accurate estimation of the dose–response relationship. Our findings are consistent with the template of Hu and Rosenberger (46) which suggests that optimality of a fixed allocation design and variability of the randomization procedure are two major determinants of the performance of a randomization design in practice. From our simulation studies, we found that design optimality has a more profound impact on design performance than the randomization procedure. In other words, applying the “most balanced” randomization procedure such as PBD to target a nonoptimal design is an inferior strategy to applying the “most random” CRD procedure to target the Doptimal design. We found that while CRD (applied to the Doptimal target) can incur some loss in efficiency in small samples, it becomes more efficient as the sample size increases, e.g., the median Defficiency of CRD is ~ 99% for n = 60. Using more restrictive (and properly calibrated) randomization procedures (such as DBCD, GDLUD, etc.) can also be an attractive strategy.
For a twostage design with a predetermined total sample size, two important considerations involve the timing of an interim analysis and the choice of randomization procedures for stages 1 and 2. If stage 1 size is too small and a highly variable randomization procedure (such as CRD) is used to allocate patients to doses, then there is a substantial risk that the Doptimal design cannot be estimated after stage 1, thereby defying the purpose of design adaptation. If stage 1 size is too large, then an interim estimate of the Doptimal design can be readily available; yet the second stage may be too small to fully benefit from this interim knowledge. Our simulations show that an equal split of the total sample size between stages 1 and 2 and use of a “wellbalanced” randomization to implement target allocation in each stage (especially in stage 1) is an optimal strategy.
For a multistage adaptive design with early stopping, it is important that the first (pilot) cohort is randomized to doses according to a “wellbalanced” procedure such as PBD or MaxEnt(η = 1). Thereafter, additional cohorts can be randomized (according to an updated Doptimal design) using different methods, including CRD. Again, design optimality has a profound effect—using a suboptimal (uniform allocation) design requires much larger sample sizes to attain the same level of estimation accuracy as for the adaptive optimal designs.
In practice, the design performance may be affected by various experimental biases. It is increasingly common to evaluate the influence of potential selection bias and chronological bias on test decision (type I error rate) (44,49, 50, 51,65). In the current paper, we investigated the potential impact of experimental bias on dose–response estimation using a recently published ERDO template (45). In particular, our simulations provide evidence that selection bias can have a detrimental impact on quality of dose–response estimation. A striking finding is that a suboptimal (uniform allocation) design can lead to very misleading conclusions, in both scenarios with and without selection bias. Without selection bias, the Uniform PBD can overestimate the true dose–response curve, whereas when selection bias is present, the design can grossly underestimate the curve and even yield a false impression that the dose–response is flat. In our example, a twostage adaptive optimal design with PBD applied in both stages was more robust (while still being affected) to selection bias than other designs.
We would also like to highlight several important problems for future research. In the current paper, we only focused on estimation of dose–response. However, in many phase II clinical trials, the primary objective is to first test whether the dose–response is present and then estimate the dose–response curve. Testing the presence of dose–response in timetoevent settings may be a challenging task—due to small sample sizes, censored data, and model uncertainty. Which test (parametric or nonparametric) should be used? The impact of randomization on power of the test was studied in responseadaptive randomized comparative studies (46) but not in the context of dose–response studies, and this is one important open problem. Another problem is sample size justification for twostage adaptive designs. How large should a study be? Is equal split of the total sample size between stage 1 and stage 2 always optimal? Our simulations in the current paper suggest so, but the formal proof of this conjecture is yet to be provided.
In practice, historical data from previous studies may be available, in which case a Bayesian design may be a viable option, e.g., the first (pilot) stage may be implemented using Bayesian optimal design (which may be different from the uniform design), and subsequent adaptations can be implemented in a Bayesian manner (rather than using maximum likelihood updating). The impact of randomization on statistical properties of Bayesian adaptive dose–response designs certainly merits investigation.
The results in the current paper are based on the assumption that event times follow a quadratic Weibull regression model with four parameters. While such a model is quite flexible and can cover a broad variety of dose–response shapes (34), it may still be misspecified in a number of ways; e.g., a third or higherorder polynomial model may be a better choice, and/or a timetoevent distribution may be other than Weibull (say, loglogistic, Gamma, etc.). Finding locally Doptimal designs under different models and constructing responseadaptive designs that converge to the “true” ones can be done using similar arguments as in our previous work (34) and the current paper. More complex models may require larger amount of data to estimate the underlying dose–response and implement the corresponding Doptimal designs. However, the main findings of the current work are likely to be extended to such more complex models (provided that the functional form of the model and the event time distribution are chosen correctly). If the model form and/or the distribution of the event times are misspecified, then statistical properties of responseadaptive optimal designs (constructed under different assumptions) may be affected. The impact of such misspecifications is another important open problem which we hope to pursue in our future work.
In many timetoevent trials, there are important covariates (prognostic factors) that are correlated with the primary outcome. Rosenberger and Sverdlov (66) discuss strategies for handling covariates in the design of randomized comparative trials and advocate a class of covariateadjusted responseadaptive (CARA) randomization designs. CARA randomization can be particularly attractive in trials for personalized medicine (67). An application of CARA randomization in timetoevent dose–response trials is yet another open problem.
Finally, we think that further theoretical and simulation studies are warranted to better understand the impact of chronological bias and selection bias on estimation and statistical tests following adaptive optimal designs. The ERDO template (45) is an excellent starting point to facilitate such an investigation.
CONCLUSION
The current paper provides a systematic study of adaptive randomization procedures to target Doptimal designs for dose–response trials with timetoevent outcomes. Simulation studies provide evidence that the choice of randomization to implement the Doptimal design does matter as far as quality of dose–response curve estimation is concerned. For best performance, an adaptive design with small cohort sizes should be implemented with a randomization procedure that ensures a “wellbalanced” allocation according to the targeted Doptimal design at each stage. Using a suboptimal design can lead to very misleading results, in both scenarios with and without selection bias. The results of the current work should help clinical investigators select an appropriate randomization procedure for their dose–response study.
REFERENCES
 1.Bretz F, Hsu J, Pinheiro J, Liu Y. Dose finding—a challenge in statistics. Biom J. 2008;50(4):480–504.CrossRefPubMedGoogle Scholar
 2.Zhao W, Yang H. Statistical methods in drug combination studies. Boca Raton: Chapman & Hall/CRC Press; 2015. 240 p.Google Scholar
 3.Jaki T. Multiarm clinical trials with treatment selection: what can be gained and at what price? Clin Investig (Lond). 2015;5(4):393–9.CrossRefGoogle Scholar
 4.Wason J, Magirr D, Law M, Jaki T. Some recommendations for multiarm multistage trials. Stat Methods Med Res. 2012;25(2):716–27.CrossRefPubMedPubMedCentralGoogle Scholar
 5.Woodcock J, LaVange LM. Master protocols to study multiple therapies, multiple diseases, or both. N Engl J Med. 2017;377(1):62–70.CrossRefPubMedGoogle Scholar
 6.Saville BR, Berry SM. Efficiencies of platform clinical trials: a vision of the future. Clin Trials. 2015;13(3):358–66.CrossRefGoogle Scholar
 7.Sverdlov O, Rosenberger WF. On recent advances in optimal allocation designs in clinical trials. J Stat Theory Pract. 2013;74(7):753–73.CrossRefGoogle Scholar
 8.Dumville JC, Hahn S, Miles JNV, Torgerson DJ. The use of unequal randomisation ratios in clinical trials: a review. Contemp Clin Trials. 2006;27:1–12.Google Scholar
 9.Peckham E, Brabyn S, Cook L, Devlin T, Dumville J, Torgerson DJ. The use of unequal randomisation in clinical trials—an update. Contemp Clin Trials. 2015;45:113–22.CrossRefPubMedGoogle Scholar
 10.Biedermann S, Dette H, Zhu W. Optimal designs for doseresponse models with restricted design spaces. J Am Stat Assoc. 2006;101(474):747–59.CrossRefGoogle Scholar
 11.Miller F, Guilbaud O, Dette H. Optimal designs for estimating the interesting part of a doseeffect curve. J Biopharm Stat. 2007;17:1097–115.CrossRefPubMedGoogle Scholar
 12.Dette H, Bretz F, Pepelyshev A, Pinheiro J. Optimal designs for dosefinding studies. J Am Stat Assoc. 2008;103(483):1225–37.CrossRefGoogle Scholar
 13.Bretz F, Dette H, Pinheiro JC. Practical considerations for optimal designs in clinical dose finding studies. Stat Med. 2010;29(7–8):731–42.CrossRefPubMedGoogle Scholar
 14.Gwise TE, Zhou J, Hu F. An optimal response adaptive biased coin design with k heteroscedastic treatments. J Stat Plan Inference. 2011;141(1):235–42.CrossRefGoogle Scholar
 15.Wong WK, Zhu W. Optimum treatment allocation rules under a variance heterogeneity model. Stat Med. 2008;27(22):4581–95.CrossRefPubMedGoogle Scholar
 16.Zhu H, Hu F. Implementing optimal allocation for sequential continuous responses with multiple treatments. J Stat Plan Inference. 2009;139(7):2420–30.CrossRefGoogle Scholar
 17.Tymofyeyev Y, Rosenberger WF, Hu F. Implementing optimal allocation in sequential binary response experiments. J Am Stat Assoc. 2007;102(477):224–34.CrossRefGoogle Scholar
 18.Sverdlov O, Tymofyeyev Y, Wong WK. Optimal responseadaptive randomized designs for multiarmed survival trials. Stat Med. 2011;30(24):2890–910.CrossRefPubMedGoogle Scholar
 19.Sverdlov O, Ryeznik Y, Wong WK. Efficient and ethical responseadaptive randomization designs for multiarm clinical trials with Weibull timetoevent outcomes. J Biopharm Stat. 2014;24(4):732–54.CrossRefPubMedGoogle Scholar
 20.Zhu W, Wong WK. Optimal treatment allocation in comparative biomedical studies. Stat Med. 2000;19(5):639–48.CrossRefPubMedGoogle Scholar
 21.Feng C, Hu F. Optimal responsesadaptive designs based on efficiency, ethic, and cost. Stat Interface. 2018;11(1):99–107.CrossRefGoogle Scholar
 22.Rosenberger WF, Lachin JM. Randomization in clinical trials: theory and practice. 2nd ed. New York: Wiley; 2015. 284 p.Google Scholar
 23.Sverdlov O, Rosenberger WF. Randomization in clinical trials: can we eliminate bias? Clin Investig (Lond). 2013;3(1):37–47.CrossRefGoogle Scholar
 24.Zhao W. A better alternative to the inferior permuted block design is not necessarily complex. Stat Med. 2016;35:1736–8.CrossRefPubMedGoogle Scholar
 25.Berger VW, Bejleri K, Agnor R. Comparing MTI randomization procedures to blocked randomization. Stat Med. 2016;35(5):685–94.CrossRefPubMedGoogle Scholar
 26.Zhao W, Weng Y. Block urn design—a new randomization algorithm for sequential trials with two or more treatments and balanced or unbalanced allocation. Contemp Clin Trials. 2011;32(6):953–61.CrossRefPubMedPubMedCentralGoogle Scholar
 27.Kuznetsova OM, Tymofyeyev Y. Brick tunnel randomization for unequal allocation to two or more treatment groups. Stat Med. 2011;30(8):812–24.PubMedGoogle Scholar
 28.Kuznetsova OM, Tymofyeyev Y. Wide brick tunnel randomization—an unequal allocation procedure that limits the imbalance in treatment totals. Stat Med. 2014;33(9):1514–30.CrossRefPubMedGoogle Scholar
 29.Zhao W. Mass weighted urn design—a new randomization algorithm for unequal allocations. Contemp Clin Trials. 2015;43:209–16.CrossRefPubMedPubMedCentralGoogle Scholar
 30.Hu F, Rosenberger WF. The theory of responseadaptive randomization in clinical trials. New York: Wiley and Sons; 2006. 218 pp.Google Scholar
 31.Hu F, Zhang LX. Asymptotic properties of doubly adaptive biased coin designs for multitreatment clinical trials. Ann Stat. 2004;32(1):268–301.Google Scholar
 32.Sun R, Cheung SH, Zhang LX. A generalized droptheloser rule for multitreatment clinical trials. J Stat Plan Inference. 2007;137(6):2011–23.CrossRefGoogle Scholar
 33.Yuan A, Chai GX. Optimal adaptive generalized Pólya urn design for multiarm clinical trials. J Multivar Anal. 2008;99(1):1–24.CrossRefGoogle Scholar
 34.Ryeznik Y, Sverdlov O, Hooker AC. Adaptive optimal designs for dosefinding studies with timetoevent outcomes. AAPS J. 2018;20(1):24.CrossRefGoogle Scholar
 35.Fedorov VV, Hackl P. Modeloriented design of experiments. New York: Springer New York; 1997. 117 p.CrossRefGoogle Scholar
 36.Berger V. Selection bias and covariate imbalances in randomized clinical trials. Hoboken: Wiley; 2005. 205 p.Google Scholar
 37.Rosenberger WF, Hu F. Maximizing power and minimizing treatment failures in clinical trials. Clin Trials. 2004;1(2):141–7.CrossRefPubMedGoogle Scholar
 38.Efron B. Forcing a sequential experiment to be balanced. Biometrika. 1971;58(3):403–17.CrossRefGoogle Scholar
 39.Klotz JH. Maximum entropy constrained balance randomization for clinical trials. Biometrics. 1978;34(2):283–7.CrossRefPubMedGoogle Scholar
 40.Ryeznik Y, Sverdlov O. A comparative study of restricted randomization procedures for multiarm trials with equal or unequal treatment allocation ratios. Stat Med. 2018; https://doi.org/10.1002/sim.7817.
 41.Kuznetsova OM. Brick tunnel randomization and the momentum of the probability mass. Stat Med. 2015;34(30):4031–56.CrossRefPubMedGoogle Scholar
 42.Heritier S, Gebski V, Pillai A. Dynamic balancing randomization in controlled clinical trials. Stat Med. 2005;24(24):3729–41.CrossRefPubMedGoogle Scholar
 43.Altman DG, Royston JP. The hidden effect of time. Stat Med. 1988;7(6):629–37.CrossRefPubMedGoogle Scholar
 44.Tamm M, Hilgers RD. Chronological bias in randomized clinical trials arising from different types of unobserved time trends. Methods Inf Med. 2014;53(6):501–10.CrossRefPubMedGoogle Scholar
 45.Hilgers RD, Uschner D, Rosenberger WF, Heussen N. ERDO—a framework to select an appropriate randomization procedure for clinical trials. BMC Med Res Methodol. 2017;17(1):159.CrossRefPubMedPubMedCentralGoogle Scholar
 46.Hu F, Rosenberger WF. Optimality, variability, power: evaluating responseadaptive randomization procedures for treatment comparisons. J Am Stat Assoc. 2003;98(463):671–8.CrossRefGoogle Scholar
 47.Berger VW. Failure to look beyond blocks is a mistake. Methods Inf Med. 2015;54(3):290.CrossRefPubMedGoogle Scholar
 48.Proschan M. Influence of selection bias on type I error rate under random permuted block designs. Stat Sin. 1994;4(4):219–31.Google Scholar
 49.Kennes LN, Cramer E, Hilgers RD, Heussen N. The impact of selection bias on test decisions in randomized clinical trials. Stat Med. 2011;30(21):2573–81.PubMedGoogle Scholar
 50.Tamm M, Cramer E, Kennes LN, Heussen N. Influence of selection bias on the test decision: a simulation study. Methods Inf Med. 2012;51(2):138–43.CrossRefPubMedGoogle Scholar
 51.Rückbeil MV, Hilgers RD, Heussen N. Assessing the impact of selection bias on test decisions in trials with a timetoevent outcome. Stat Med. 2017;36(17):2656–68.CrossRefPubMedPubMedCentralGoogle Scholar
 52.Kahan BC, Rehal S, Cro S. Risk of selection bias in randomised trials. Trials. 2015;16(1):405.CrossRefPubMedPubMedCentralGoogle Scholar
 53.Berger VW. Risk of selection bias in randomized trials: further insight. Trials. 2016;17(1):485.CrossRefPubMedPubMedCentralGoogle Scholar
 54.Berger VW, Ivanova A, Knoll MD. Minimizing predictability while retaining balance through the use of less restrictive randomization procedures. Stat Med. 2003;22(19):3017–28.CrossRefPubMedGoogle Scholar
 55.Zhao W, Everett CC, Weng Y, Berger VW. Guessing strategies for treatment prediction under restricted randomization with unequal allocation. Contemp Clin Trials. 2017;59:118–20.Google Scholar
 56.Haines LM, Perevozskaya I, Rosenberger WF. Bayesian optimal designs for phase I clinical trials. Biometrics. 2003;59(3):591–600.CrossRefPubMedGoogle Scholar
 57.Liu G, Rosenberger WF, Haines LM. Sequential designs for logistic phase I clinical trials. J Biopharm Stat. 2006;16(5):605–21.CrossRefPubMedGoogle Scholar
 58.Liu G, Rosenberger WF, Haines LM. Sequential designs for ordinal phase I clinical trials. Biom J. 2009;51(2):335–47.CrossRefPubMedGoogle Scholar
 59.Roy A, Ghosal S, Rosenberger WF. Convergence properties of sequential Bayesian Doptimal designs. J Stat Plan Inference. 2009;139(2):425–40.CrossRefGoogle Scholar
 60.Roth K. Sequential designs for dose escalation studies in oncology. Commun Stat Simul Comput. 2012;417(41):1131–41.CrossRefGoogle Scholar
 61.Dragalin V, Fedorov VV, Wu Y. Twostage design for dosefinding that accounts for both efficacy and safety. Stat Med. 2008;27(25):5156–76.CrossRefPubMedGoogle Scholar
 62.Bornkamp B, Bretz F, Dette H, Pinheiro J. Responseadaptive dosefinding under model uncertainty. Ann Appl Stat. 2011;5(2 B):1611–31.CrossRefGoogle Scholar
 63.Ivanova A, Xiao C, Tymofyeyev Y. Twostage designs for phase 2 dosefinding trials. Stat Med. 2012;31(24):2872–81.CrossRefPubMedPubMedCentralGoogle Scholar
 64.Dette H, Bornkamp B, Bretz F. On the efficiency of twostage responseadaptive designs. Stat Med. 2013;32(10):1646–60.CrossRefPubMedGoogle Scholar
 65.Uschner D, Hilgers RD, Heussen N. The impact of selection bias in randomized multiarm parallel group clinical trials. PLoS One. 2018;13(1):e0192065.CrossRefPubMedPubMedCentralGoogle Scholar
 66.Rosenberger WF, Sverdlov O. Handling covariates in the design of clinical trials. Stat Sci. 2008;23(3):404–19.CrossRefGoogle Scholar
 67.Hu F. Statistical issues in trial design and personalized medicine. Clin Investig (Lond). 2012;2:121–4.CrossRefGoogle Scholar
Copyright information
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.