Monte Carlo based statistical power analysis for mediation models: methods and software

Behavior Research Methods volume 46, pages1184–1198(2014)Cite this article

Abstract

The existing literature on statistical power analysis for mediation models often assumes data normality and is based on a less powerful Sobel test instead of the more powerful bootstrap test. This study proposes to estimate statistical power to detect mediation effects on the basis of the bootstrap method through Monte Carlo simulation. Nonnormal data with excessive skewness and kurtosis are allowed in the proposed method. A free R package called bmem is developed to conduct the power analysis discussed in this study. Four examples, including a simple mediation model, a multiple-mediator model with a latent mediator, a multiple-group mediation model, and a longitudinal mediation model, are provided to illustrate the proposed method.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Beasley, T. M. (2012). Power of product tests of mediation as a function of mediator collinearity. Multiple Linear Regression Viewpoints, 38, 17–23.

    Google Scholar 

  2. Cheung, M. W. L. (2007). Comparison of approaches to constructing confidence intervals for mediating effects using structural equation models. Structural Equation Modeling, 14(3), 227–246.

    Article  Google Scholar 

  3. Daly, L. E. (1991). Confidence intervals and sample sizes: Don’t throw out all your old sample size tables. BMJ: British Medical Journal, 302, 333–336.

    PubMed Central  PubMed  Article  Google Scholar 

  4. Fritz, M. S., & MacKinnon, D. P. (2007). Required sample size to detect the mediated effect. Psychological Science, 18, 233–239.

    PubMed Central  PubMed  Article  Google Scholar 

  5. Fritz, M. S., Taylor, A. B., & MacKinnon, D. P. (2012). Explanation of two anomalous results in statistical mediation analysis. Multivariate Behavioral Research, 47, 61–87.

    PubMed Central  PubMed  Article  Google Scholar 

  6. Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY: Guilford Press.

    Google Scholar 

  7. Hayes, A. F., & Scharkow, M. (2013). The relative trustworthiness of inferential tests of the indirect effect in statistical mediation analysis: Does method really matter? Psychological Science, in press.

  8. Hoenig, J. M., & Heisey, D. M. (2001). The abuse of power. The American Statistician, 55, 19–24.

    Article  Google Scholar 

  9. Kenny, D. A., & Judd, C. M. (2013). Power anomalies in testing mediation. Psychological Science, in press. (R code at http://www.davidakenny.net/progs/PowMedR.txt)

  10. Knaus, J. (2013). snowfall: Easier cluster computing (based on snow). [Computer software manual]. Retrieved from http://CRAN.R-project.org/package=snowfall (R package version 1.84-4)

  11. MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York, NY: Taylor & Francis.

    Google Scholar 

  12. MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39(3), 99–128.

    PubMed Central  PubMed  Article  Google Scholar 

  13. Maxwell, S. E., & Cole, D. A. (2007). Bias in cross-sectional analyses of longitudinal mediation. Psychological Methods, 12, 23–44.

    PubMed  Article  Google Scholar 

  14. Meehl, P. E. (1997). The problem is epistemology, not statistics: Replace significance tests by confidence intervals and quantify accuracy of risky numerical predictions. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 393–425). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  15. Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105(3), 156–166.

    Article  Google Scholar 

  16. Muthén, L. K., & Muthén, B. O. (1998-2011). Mplus user’s guide (Sixth Ed.). Los Angeles, CA: Muthén and Muthén. (http://www.statmodel.com)

  17. Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9(3), 599–620.

    Article  Google Scholar 

  18. Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, & Computers, 36, 717–731.

    Article  Google Scholar 

  19. Qiu, W. (2013). powermediation: Power/sample size calculation for mediation analysis [Computer software manual]. Retrieved from http://CRAN.R-project.org/package=powerMediation (R package version 0.1.3)

  20. Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 1–36.

    Google Scholar 

  21. Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods, 7, 422–445.

    PubMed  Article  Google Scholar 

  22. Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equation models. In S. Leinhardt (Ed.), Sociological methodology (pp. 290–312). San Francisco: Jossey-Bass.

    Google Scholar 

  23. Thoemmes, F., MacKinnon, D. P., & Reiser, M. R. (2010). Power analysis for complex mediational designs using Monte Carlo methods. Structural Equation Modeling, 17, 510–534.

    PubMed Central  PubMed  Article  Google Scholar 

  24. Vale, C. D., & Maurelli, V. A. (1983). Simulating multivariate nonnormal disttributions. Psychometrika, 48, 465–471.

    Article  Google Scholar 

  25. Vittinghoff, E., Sen, S., & McCulloch, C. (2009). Sample size calculations for evaluating mediation. Statistics in Medicine, 28, 541–557.

    PubMed  Article  Google Scholar 

  26. von Oertzen, T., & Brick, T. (2013). Efficient hessian computation using sparse matrix derivatives in ram notation. Behavior Research Methods, in press.

  27. Wang, C., & Xue, X. (2012). Power and sample size calculations for evaluating mediation effects in longitudinal studies. Statistical Methods in Medical Research, in press.

  28. Wang, L., & Zhang, Z. (2011). Estimating and testing mediation effects with censored data. Structural Equation Modeling, 18(3), 18–34.

    Article  Google Scholar 

  29. Yuan, K. H., Bentler, P. M., & Zhang, W. (2005). The effect of skewness and kurtosis on mean and covariance structure analysis. Sociological Methods & Research, 34(3), 240–258. Retrieved from doi:10.1177/0049124105280200

    Article  Google Scholar 

  30. Zhang, Z., & Wang, L. (2009). Statistical power analysis for growth curve models using SAS. Behavior Research Methods, 41, 1083–1094.

    PubMed  Article  Google Scholar 

  31. Zhang, Z., & Wang, L. (2013a). bmem: Mediation analysis with missing data using bootstrap [Computer software manual]. Retrieved from http://CRAN.R-project.org/package=bmem (R package version 1.5)

  32. Zhang, Z., & Wang, L. (2013b). Methods for mediation analysis with missing data. Psychometrika, 78, 154–184.

    PubMed  Article  Google Scholar 

  33. Zu, J., & Yuan, K. H. (2010). Local influence and robust procedures for mediation analysis. Multivariate Behavioral Research, 45, 1–44.

    Article  Google Scholar 

Download references

Author Note

We thank Scott Maxwell and Ke-Hai Yuan for helpful discussions and David Kenny and one anonymous reviewer for constructive suggestions that have significantly improved this research. Path diagrams used in the article were generated using WebSEM (https://websem.psychstat.org).

Author information

Affiliations

  1. University of Notre Dame, Notre Dame, USA

    Zhiyong Zhang

Authors
  1. Zhiyong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhiyong Zhang.

Appendices

Appendix 1 R code for Example 1

figuref

Appendix 2 R code for Example 2

figureg

Appendix 3 R code for Example 3

figureh

Appendix 4 R code for Example 4

figurei

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Zhang, Z. Monte Carlo based statistical power analysis for mediation models: methods and software. Behav Res 46, 1184–1198 (2014). https://doi.org/10.3758/s13428-013-0424-0

Download citation

Keywords

  • Power analysis
  • Mediation models
  • Nonnormal data
  • Bootstrapping
  • R package bmem