To the Editor,

Goldstein et al.1 express their “concerns regarding the methodological and ethical issues raised in trials such as the B-Free trial”. Correspondence from Spence et al.2 responds to the ethical issues raised; we would like to respond to the methodological issues. Specifically, Goldstein et al. mention the following concerns: (i) increases in Type I error, (ii) limited external validity, (iii) imbalance in baseline characteristics, and (iv) carry-over and period effects. In designing the Benzodiazepine-Free Cardiac Anesthesia for Reduction in Postoperative Delirium (B-Free) trial, we accounted for these methodological issues.3

Cluster-randomized trials could result in inflated Type I error because analyses are conducted at the individual level but randomization is at the cluster level and members of the same cluster tend to have more similar responses than members of different clusters. Therefore, it is crucial to account for the effect of clustering in sample size considerations and in statistical analysis. In addition, they may also suffer from low external validity if they include only a small number of clusters. In B-Free, we calculated the necessary sample size using the design effect approach, which uses a correction factor to account for the effect of clustering to control the Type I error and ensure the internal validity. Furthermore, the number of 16 clusters in B-Free, obtained based on a conservative intra-cluster correlation (ICC) coefficient of 0.02 and an inter-period correlation coefficient assumed to be half of the ICC, is higher than the minimum number of ten clusters recommended4 and the median number of nine clusters based on a systematic review of 91 cluster crossover randomized trials.5 External validity in all trials can be enhanced if the trial design incorporates a broader population to which the results would be applicable, or the interventions are studied in a wide range of settings and practitioner expertise, or the study is conducted under more “real-world” conditions; these aspects have been incorporated into the design of B-Free by including international hospitals of varying size, case volume, and complexity of cases offered.

Imbalance in baseline characteristics of subjects is a possibility in cluster-randomized trials since randomization is at the institution and not the subject level. Nevertheless, this potential imbalance may be overcome by incorporating crossovers into the trial design,6 thus estimating the treatment effect within a given cluster, comparing the treatment and comparator intervention periods, thus having each cluster act as its own control group. Subsequently, the analysis of cluster crossover studies provides a more statistically efficient comparison than analyses of clustered non-crossover studies under the assumption of no period and no carry-over effects. In addition, by including multiple crossovers of 12 four-week crossover periods, the B-Free study minimizes the unknown confounding due to clustering.

We recognize that incorporating crossovers into the trial design introduces the potential for period and carry-over effects. Period effects are the bias that may be introduced if patients managed during one period have a different baseline prognosis than those in another. By randomizing clusters to varied sequences of crossovers in B-Free for multiple periods, we can estimate and handle the impact of period effects. Carry-over effects are the bias that may result from practitioners’ failure to completely transition between intervention arms when moving from one crossover period to the next. In B-Free, each cluster is initially randomized to four four-week ‘run-in’ periods wherein we establish the ability of practitioners in that site to apply the intervention policy and crossover between arms three times without meaningful carry-over effects. If practitioners within that cluster are able to apply the policy in place in at least 80% of patients during each crossover period, they are included in the main trial and are randomized to the remaining eight trial crossover periods.

In summary, we maintain that the B-free trial is a rigorously designed and methodologically sound trial that will answer an important clinical question. In our opinion, the editorial by Goldstein et al.1 makes many theoretically correct statements about methodological concerns for cluster crossover trials in general, but applies them incorrectly to the B-Free trial, failing to appreciate the nuances of the trial design and how the methodological challenges have been addressed in our study.