Sample Size Planning for Cluster-Randomized Interventions Probing Multilevel Mediation
- 256 Downloads
Multilevel mediation analyses play an essential role in helping researchers develop, probe, and refine theories of action underlying interventions and document how interventions impact outcomes. However, little is known about how to plan studies with sufficient power to detect such multilevel mediation effects. In this study, we describe how to prospectively estimate power and identify sufficient sample sizes for experiments intended to detect multilevel mediation effects. We outline a simple approach to estimate the power to detect mediation effects with individual- or cluster-level mediators using summary statistics easily obtained from empirical literature and the anticipated magnitude of the mediation effect. We draw on a running example to illustrate several different types of mediation and provide an accessible introduction to the design of multilevel mediation studies. The power formulas are implemented in the R package PowerUpR and the PowerUp software (causalevaluation.org).
KeywordsMediation Power Indirect effects Multilevel models Sample size determination
This study was supported by grants from the National Science Foundation (Award Nos. 1437679, 1552535, 1437745, and 1437692). The opinions expressed herein are those of the authors and not the funding agencies.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they have no conflicts of interest.
This article does not contain any studies with human participants or animals performed by any of the authors.
The manuscript does not report any studies with human participants or animals; no study was conducted requiring informed consent.
- Cox, K. & Kelcey, B. (2018). Optimal Sample Allocation in Group-randomized Studies of Multilevel Mediation with a Group-level Mediator. Journal of Experimental Education (in press).Google Scholar
- Dong, N., Kelcey, B., Spybrook, J., & Maynard, R. A. (2016). PowerUp!-Mediator: A tool for calculating statistical power for causally-defined mediation in cluster randomized trials. (Version 0.4) [Software]. Available from http://www.causalevaluation.org/. Accessed 1 May 2007.
- Kelcey, B., & Phelps, G. (2013). Considerations for Designing Group Randomized Trials of Professional Development with Teacher Knowledge Outcomes. Educational Evaluation and Policy Analysis, 35, 370–390.Google Scholar
- Kelcey, B., Dong, N., Spybrook, J., & Cox, K. (2017). Statistical power for causally-defined indirect effects in group-randomized trials. Journal of Educational and Behavioral Statistics. Google Scholar
- Kelcey, B., Dong, N., Spybrook, J., & Shen, Z. (2017a). Experimental Power for Indirect Effects in Group-randomized Studies with Group-level Mediators. Multivariate Behavioral Research, 52(6), 699–719.Google Scholar
- Kelcey, B., Dong, N., Spybrook, J. & Cox, K. (2017b). Statistical power for causally-defined indirect effects in group-randomized trials. Journal of Educational and Behavioral Statistics. Google Scholar
- Kelcey, B. & Shen, Z. (2018). Effective and Efficient Experimental Design 2-1-1 Mediation Studies. Journal of Experimental Education (in press).Google Scholar
- Texas Christian University [TCU] (2005) Institute of Behavioral Research. The Organizational Readiness for Change: Treatment Staff Version (TCU ORC-S). Available at: www.ibr.tcu.edu.