A Simulation Study of Statistical Approaches to Data Analysis in the Stepped Wedge Design

  • Yuqi Ren
  • James P. HughesEmail author
  • Patrick J. Heagerty


This paper studies model-based and design-based approaches for the analysis of data arising from a stepped wedge randomized design. Specifically, for different scenarios we compare robustness, efficiency, Type I error rate under the null hypothesis, and power under the alternative hypothesis for the leading analytical options including generalized estimating equations (GEE) and linear mixed model (LMM)-based approaches. We find that GEE models with exchangeable correlation structures are more efficient than GEE models with independent correlation structures under all scenarios considered. The model-based GEE Type I error rate can be inflated when applied with a small number of clusters, but this problem can be solved using a design-based approach. As expected, correct model specification is more important for LMM (compared to GEE) since the model is assumed correct when standard errors are calculated. However, in contrast to the model-based results, the design-based Type I error rates for LMM models under scenarios with a random treatment effect show Type I error inflation even though the fitted models perfectly match the corresponding data-generating scenarios. Therefore, greater robustness can be realized by combining GEE and permutation testing strategies.


Stepped wedge design GEE LMM Permutation test Simulation 



This research was supported by the National Institute of Allergy and Infectious Diseases Grant AI29168 and PCORI contract ME-1507-31750.


  1. 1.
    Brown CA, Lilford RJ (2006) The stepped wedge trial design: a systematic review. BMC Med Res Methodol 6:54CrossRefGoogle Scholar
  2. 2.
    Diggle P, Heagerty P, Liang K, Zeger S (2002) Analysis of longitudinal data, 2nd edn. Oxford University Press, OxfordzbMATHGoogle Scholar
  3. 3.
    Golden MR, Kerani RP, Stenger M, Hughes JP, Aubin M, Malinski C, Holmes KK (2015) Uptake and population-level impact of expedited partner therapy (EPT) on Chlamydia trachomatis and Neisseria gonorrhoeae: the Washington State Community-level Randomized Trial of EPT. PLoS Med 12(1):e1001777CrossRefGoogle Scholar
  4. 4.
    Good P (2005) Permutation, parametric and bootstrap tests of hypotheses, 3rd edn. Springer, New YorkzbMATHGoogle Scholar
  5. 5.
    Hooper R, Teerenstra S, de Hoop E, Eldridge S (2016) Sample size calculation for stepped wedge and other longitudinal cluster randomized trials. Stat Med 35:4718–4728MathSciNetCrossRefGoogle Scholar
  6. 6.
    Hussey MA, Hughes JP (2007) Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials 28:182–191CrossRefGoogle Scholar
  7. 7.
    Hughes JP, Granston TS, Heagerty PJ (2015) Current issues in the design and analysis of stepped wedge trials. Contemp Clin Trials 45:55–60CrossRefGoogle Scholar
  8. 8.
    Hughes JP, Heagerty PJ, Xia F, Ren Y (2019) Robust inference in the stepped wedge design. Biometrics. CrossRefGoogle Scholar
  9. 9.
    Ji X, Fink G, Robyn PJ, Small DS (2017) Randomization inference for stepped-wedge cluster-randomized trials: an application to community-based health insurance. Ann Appl Stat 11:1–20MathSciNetCrossRefGoogle Scholar
  10. 10.
    Killiam WP, Tambatamba BC, Chintu N, Rouose D, Stringer E, Bweupe M, Yu Y, Stringer JSA (2010) Antiretroviral therapy in antenatal care to increase treatment initiation in HIV-infected pregnant women: a stepped-wedge evaluation. AIDS 24:85–91CrossRefGoogle Scholar
  11. 11.
    Leyrat C, Morgan EK, Leurent B, Kahan CB (2018) Cluster randomized trials with a small number of clusters: which analyses should be used? Int J Epidemiol 47:321–331CrossRefGoogle Scholar
  12. 12.
    Li P, Redden DT (2015) Comparing denominator degrees of freedom approximations for the generalized linear mixed model in analyzing binary outcome in small sample cluster-randomized trials. BMC Med Res Methodol 15:38CrossRefGoogle Scholar
  13. 13.
    Mdege ND, Man MS, Taylor CA, Torgerson DJ (2011) Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol 64:936–948CrossRefGoogle Scholar
  14. 14.
    Rhoda DA, Murray DM, Andridge RR, Pennell ML, Hade EM (2011) Studies with staggered starts: multiple baseline designs and group-randomized trials. Am J Publ Health 101:2164–2169CrossRefGoogle Scholar
  15. 15.
    Sharples K, Breslow N (1992) Regression analysis of correlated binary data: some small sample results for the estimating equation approach. J Stat Comput Simul 42:1–20CrossRefGoogle Scholar
  16. 16.
    Taljaard M, Teerenstra S, Ivers NM, Fergusson DA (2016) Substantial risks associated with few clusters in cluster randomized and stepped wedge designs. Clin Trials 13:459–463CrossRefGoogle Scholar
  17. 17.
    Thompson JA, Davey C, Fielding K, Hargreaves JR, Hayes RJ (2018) Robust analysis of stepped wedge trials using cluster-level summaries within periods. Stat Med 37:2487–2500MathSciNetCrossRefGoogle Scholar
  18. 18.
    Westgate PM (2013) On small-sample inference in group randomized trials with binary outcomes and cluster-level covariates. Biom J 5:789–806MathSciNetCrossRefGoogle Scholar
  19. 19.
    Woertman W, de Hoop E, Moerbeek M, Zuidema SU, Gerritsen DL, Teerenstra S (2013) Stepped wedge designs could reduce the required sample size in cluster randomized trials. J Clin Epidemiol 66:752–758CrossRefGoogle Scholar

Copyright information

© International Chinese Statistical Association 2019

Authors and Affiliations

  • Yuqi Ren
    • 1
  • James P. Hughes
    • 2
    Email author
  • Patrick J. Heagerty
    • 3
  1. 1.University of WashingtonSeattleUSA
  2. 2.University of WashingtonSeattleUSA
  3. 3.University of WashingtonSeattleUSA

Personalised recommendations