To the Editor,
Goldstein et al.1 express their “concerns regarding the methodological and ethical issues raised in trials such as the B-Free trial”. Correspondence from Spence et al.2 responds to the ethical issues raised; we would like to respond to the methodological issues. Specifically, Goldstein et al. mention the following concerns: (i) increases in Type I error, (ii) limited external validity, (iii) imbalance in baseline characteristics, and (iv) carry-over and period effects. In designing the Benzodiazepine-Free Cardiac Anesthesia for Reduction in Postoperative Delirium (B-Free) trial, we accounted for these methodological issues.3
Cluster-randomized trials could result in inflated Type I error because analyses are conducted at the individual level but randomization is at the cluster level and members of the same cluster tend to have more similar responses than members of different clusters. Therefore, it is crucial to account for the effect of clustering in sample size considerations and in statistical analysis. In addition, they may also suffer from low external validity if they include only a small number of clusters. In B-Free, we calculated the necessary sample size using the design effect approach, which uses a correction factor to account for the effect of clustering to control the Type I error and ensure the internal validity. Furthermore, the number of 16 clusters in B-Free, obtained based on a conservative intra-cluster correlation (ICC) coefficient of 0.02 and an inter-period correlation coefficient assumed to be half of the ICC, is higher than the minimum number of ten clusters recommended4 and the median number of nine clusters based on a systematic review of 91 cluster crossover randomized trials.5 External validity in all trials can be enhanced if the trial design incorporates a broader population to which the results would be applicable, or the interventions are studied in a wide range of settings and practitioner expertise, or the study is conducted under more “real-world” conditions; these aspects have been incorporated into the design of B-Free by including international hospitals of varying size, case volume, and complexity of cases offered.
Imbalance in baseline characteristics of subjects is a possibility in cluster-randomized trials since randomization is at the institution and not the subject level. Nevertheless, this potential imbalance may be overcome by incorporating crossovers into the trial design,6 thus estimating the treatment effect within a given cluster, comparing the treatment and comparator intervention periods, thus having each cluster act as its own control group. Subsequently, the analysis of cluster crossover studies provides a more statistically efficient comparison than analyses of clustered non-crossover studies under the assumption of no period and no carry-over effects. In addition, by including multiple crossovers of 12 four-week crossover periods, the B-Free study minimizes the unknown confounding due to clustering.
We recognize that incorporating crossovers into the trial design introduces the potential for period and carry-over effects. Period effects are the bias that may be introduced if patients managed during one period have a different baseline prognosis than those in another. By randomizing clusters to varied sequences of crossovers in B-Free for multiple periods, we can estimate and handle the impact of period effects. Carry-over effects are the bias that may result from practitioners’ failure to completely transition between intervention arms when moving from one crossover period to the next. In B-Free, each cluster is initially randomized to four four-week ‘run-in’ periods wherein we establish the ability of practitioners in that site to apply the intervention policy and crossover between arms three times without meaningful carry-over effects. If practitioners within that cluster are able to apply the policy in place in at least 80% of patients during each crossover period, they are included in the main trial and are randomized to the remaining eight trial crossover periods.
In summary, we maintain that the B-free trial is a rigorously designed and methodologically sound trial that will answer an important clinical question. In our opinion, the editorial by Goldstein et al.1 makes many theoretically correct statements about methodological concerns for cluster crossover trials in general, but applies them incorrectly to the B-Free trial, failing to appreciate the nuances of the trial design and how the methodological challenges have been addressed in our study.
References
Goldstein CE, Giraudeau B, Weijer C, Taljaard M. When and how should we cluster and cross over: methodological and ethical issues. Can J Anesth 2018; 65: 760-5.
Spence J, Belley-Côté E, Ozchowski S, et al. When and how should we cluster and cross over: methodological and ethical issues (letter 1). Can J Anesth 2018; 65: DOI: https://doi.org/10.1007/s12630-018-1237-5.
Spence J, Belley-Côté E, Lee SF, et al. The role of randomized cluster crossover trials for comparative effectiveness testing in anesthesia: design of the Benzodiazepine-Free Cardiac Anesthesia for Reduction in Postoperative Delirium (B-Free) trial. Can J Anesth 2018; 65: 813-21.
Taljaard M, Teerenstra S, Ivers NM, Fergusson DA. Substantial risks associated with few clusters in cluster randomized and stepped wedge designs. Clin Trials 2016; 13: 459-63.
Arnup SJ, Forbes AB, Kahan BC, Morgan KE, McKenzie JE. Appropriate statistical methods were infrequently used in cluster-randomized crossover trials. J Clin Epidemiol 2016; 74: 40-50.
Reich NG, Milstone AM. Improving efficiency in cluster-randomized study design and implementation: taking advantage of a crossover. Open Access J Clin Trials 2014; 6: 11-5.
Financial disclosure and conflict of interest
None declared.
Editorial responsibility
This submission was handled by Dr. Gregory L. Bryson, Deputy Editor-in-Chief, Canadian Journal of Anesthesia.
Author information
Authors and Affiliations
Corresponding author
Additional information
This letter is accompanied by a reply. Please see Can J Anesth 2018; 65: this issue.
Rights and permissions
About this article
Cite this article
Lee, S.F., Bangdiwala, S.I., Spence, J. et al. When and how should we cluster and cross over: methodological and ethical issues (letter 2). Can J Anesth/J Can Anesth 66, 237–238 (2019). https://doi.org/10.1007/s12630-018-1238-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12630-018-1238-4