Skip to main content

A Simulation Study of Statistical Approaches to Data Analysis in the Stepped Wedge Design

A Correction to this article was published on 17 August 2020

This article has been updated


This paper studies model-based and design-based approaches for the analysis of data arising from a stepped wedge randomized design. Specifically, for different scenarios we compare robustness, efficiency, Type I error rate under the null hypothesis, and power under the alternative hypothesis for the leading analytical options including generalized estimating equations (GEE) and linear mixed model (LMM)-based approaches. We find that GEE models with exchangeable correlation structures are more efficient than GEE models with independent correlation structures under all scenarios considered. The model-based GEE Type I error rate can be inflated when applied with a small number of clusters, but this problem can be solved using a design-based approach. As expected, correct model specification is more important for LMM (compared to GEE) since the model is assumed correct when standard errors are calculated. However, in contrast to the model-based results, the design-based Type I error rates for LMM models under scenarios with a random treatment effect show Type I error inflation even though the fitted models perfectly match the corresponding data-generating scenarios. Therefore, greater robustness can be realized by combining GEE and permutation testing strategies.

This is a preview of subscription content, access via your institution.

Fig. 1

Change history

  • 17 August 2020

    The original version of this article unfortunately contained an error in ‘R code’ under Appendix section.


  1. Brown CA, Lilford RJ (2006) The stepped wedge trial design: a systematic review. BMC Med Res Methodol 6:54

    Article  Google Scholar 

  2. Diggle P, Heagerty P, Liang K, Zeger S (2002) Analysis of longitudinal data, 2nd edn. Oxford University Press, Oxford

    MATH  Google Scholar 

  3. Golden MR, Kerani RP, Stenger M, Hughes JP, Aubin M, Malinski C, Holmes KK (2015) Uptake and population-level impact of expedited partner therapy (EPT) on Chlamydia trachomatis and Neisseria gonorrhoeae: the Washington State Community-level Randomized Trial of EPT. PLoS Med 12(1):e1001777

    Article  Google Scholar 

  4. Good P (2005) Permutation, parametric and bootstrap tests of hypotheses, 3rd edn. Springer, New York

    MATH  Google Scholar 

  5. Hooper R, Teerenstra S, de Hoop E, Eldridge S (2016) Sample size calculation for stepped wedge and other longitudinal cluster randomized trials. Stat Med 35:4718–4728

    Article  MathSciNet  Google Scholar 

  6. Hussey MA, Hughes JP (2007) Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials 28:182–191

    Article  Google Scholar 

  7. Hughes JP, Granston TS, Heagerty PJ (2015) Current issues in the design and analysis of stepped wedge trials. Contemp Clin Trials 45:55–60

    Article  Google Scholar 

  8. Hughes JP, Heagerty PJ, Xia F, Ren Y (2019) Robust inference in the stepped wedge design. Biometrics.

    Article  MATH  Google Scholar 

  9. Ji X, Fink G, Robyn PJ, Small DS (2017) Randomization inference for stepped-wedge cluster-randomized trials: an application to community-based health insurance. Ann Appl Stat 11:1–20

    Article  MathSciNet  Google Scholar 

  10. Killiam WP, Tambatamba BC, Chintu N, Rouose D, Stringer E, Bweupe M, Yu Y, Stringer JSA (2010) Antiretroviral therapy in antenatal care to increase treatment initiation in HIV-infected pregnant women: a stepped-wedge evaluation. AIDS 24:85–91

    Article  Google Scholar 

  11. Leyrat C, Morgan EK, Leurent B, Kahan CB (2018) Cluster randomized trials with a small number of clusters: which analyses should be used? Int J Epidemiol 47:321–331

    Article  Google Scholar 

  12. Li P, Redden DT (2015) Comparing denominator degrees of freedom approximations for the generalized linear mixed model in analyzing binary outcome in small sample cluster-randomized trials. BMC Med Res Methodol 15:38

    Article  Google Scholar 

  13. Mdege ND, Man MS, Taylor CA, Torgerson DJ (2011) Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol 64:936–948

    Article  Google Scholar 

  14. Rhoda DA, Murray DM, Andridge RR, Pennell ML, Hade EM (2011) Studies with staggered starts: multiple baseline designs and group-randomized trials. Am J Publ Health 101:2164–2169

    Article  Google Scholar 

  15. Sharples K, Breslow N (1992) Regression analysis of correlated binary data: some small sample results for the estimating equation approach. J Stat Comput Simul 42:1–20

    Article  Google Scholar 

  16. Taljaard M, Teerenstra S, Ivers NM, Fergusson DA (2016) Substantial risks associated with few clusters in cluster randomized and stepped wedge designs. Clin Trials 13:459–463

    Article  Google Scholar 

  17. Thompson JA, Davey C, Fielding K, Hargreaves JR, Hayes RJ (2018) Robust analysis of stepped wedge trials using cluster-level summaries within periods. Stat Med 37:2487–2500

    Article  MathSciNet  Google Scholar 

  18. Westgate PM (2013) On small-sample inference in group randomized trials with binary outcomes and cluster-level covariates. Biom J 5:789–806

    Article  MathSciNet  Google Scholar 

  19. Woertman W, de Hoop E, Moerbeek M, Zuidema SU, Gerritsen DL, Teerenstra S (2013) Stepped wedge designs could reduce the required sample size in cluster randomized trials. J Clin Epidemiol 66:752–758

    Article  Google Scholar 

Download references



This research was supported by the National Institute of Allergy and Infectious Diseases Grant AI29168 and PCORI contract ME-1507-31750.

Author information

Authors and Affiliations


Corresponding author

Correspondence to James P. Hughes.

Appendix: R, Stata, and SAS code

Appendix: R, Stata, and SAS code

Here we present basic R, Stata, and SAS code for fitting common models for stepped wedge designs with cross-sectional data collection at each time point. See [5,6,7].

  1. I

    Linear mixed models

    1. (1)

      \( {\text{Random cluster effect}}:Y_{ijt} = \mu + a_{i} + \beta_{t} + X_{it} \theta + e_{ijt} \)

    2. (2)

      \( {\text{Random cluster and cluster}} \times {\text{time effect}}:Y_{ijt} = \mu + a_{i} + \beta_{t} + b_{it} + X_{it} \theta + e_{ijt} \)

    3. (3)

      \( {\text{Random cluster}},{\text{ cluster}} \times {\text{time and treatment effect }}({\text{corr}}(a_{i} ,_{{}} c_{i} ) = 0):Y_{ijt} = \mu + a_{i} + \beta_{t} + b_{it} + X_{it} (\theta + c_{i} ) + e_{ijt} \)

    4. (4)

      \( {\text{Random cluster, cluster}} \times {\text{time and treatment effect }}({\text{corr}}(a_{i} ,_{{}} c_{i} ) = \rho ):Y_{ijt} = \mu + a_{i} + \beta_{t} + b_{it} + X_{it} (\theta + c_{i} ) + e_{ijt}, \)


      ai ~ N(0, τ2)

      bit ~ N(0, γ2)

      ci ~ N(0, ν2)

      eijt ~ N(0, σ2).

      figure a
  2. II

    Generalized estimating equation models

    1. (5)

      Independent working correlation

    2. (6)

      Exchangeable working correlation

      figure b

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, Y., Hughes, J.P. & Heagerty, P.J. A Simulation Study of Statistical Approaches to Data Analysis in the Stepped Wedge Design. Stat Biosci 12, 399–415 (2020).

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: