Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R

Behavior Research Methods volume 46, pages29–40(2014)Cite this article

Abstract

Meta-analytic structural equation modeling (MASEM) combines the ideas of meta-analysis and structural equation modeling for the purpose of synthesizing correlation or covariance matrices and fitting structural equation models on the pooled correlation or covariance matrix. Cheung and Chan (Psychological Methods 10:40–64, 2005b, Structural Equation Modeling 16:28–53, 2009) proposed a two-stage structural equation modeling (TSSEM) approach to conducting MASEM that was based on a fixed-effects model by assuming that all studies have the same population correlation or covariance matrices. The main objective of this article is to extend the TSSEM approach to a random-effects model by the inclusion of study-specific random effects. Another objective is to demonstrate the procedures with two examples using the metaSEM package implemented in the R statistical environment. Issues related to and future directions for MASEM are discussed.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

References

  1. Allison, P. D. (1987). Estimation of linear models with incomplete data. Sociological Methodology, 17, 71–103.

    Article  Google Scholar 

  2. Barrett, P. (2007). Structural equation modelling: Adjudging model fit. Personality and Individual Differences, 42(5), 815–824.

    Google Scholar 

  3. Becker, B. J. (1992). Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 17, 341–362.

    Article  Google Scholar 

  4. Becker, B. J. (1995). Corrections to “Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 20, 100–102.

    Article  Google Scholar 

  5. Becker, G. (1996). The meta-analysis of factor analyses: An illustration based on the cumulation of correlation matrices. Psychological Methods, 1, 341–353.

    Article  Google Scholar 

  6. Becker, B. J. (2009). Model-based meta-analysis. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 377–395). New York, NY: Sage.

    Google Scholar 

  7. Becker, B. J., & Schram, C. M. (1994). Examining explanatory models through research synthesis. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 357–381). New York, NY: Sage.

    Google Scholar 

  8. Beretvas, S., & Furlow, C. (2006). Evaluation of an approximate method for synthesizing covariance matrices for use in meta-analytic SEM. Structural Equation Modeling, 13, 153–185.

    Article  Google Scholar 

  9. Boker, S., Neale, M., Maes, H., Wilde, M., Spiegel, M., Brick, T., & Fox, J. (2011). OpenMx: An open source extended structural equation modeling framework. Psychometrika, 76, 306–317.

    PubMed Central  PubMed  Article  Google Scholar 

  10. Bonett, D. (2009). Meta-analytic interval estimation for standardized and unstandardized mean differences. Psychological Methods, 14, 225–238.

    PubMed  Article  Google Scholar 

  11. Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2009). Introduction to meta-analysis. Hoboken, NJ: Wiley.

    Google Scholar 

  12. Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.

    PubMed  Article  Google Scholar 

  13. Cheung, M. W.-L. (2008). A model for integrating fixed-, random-, and mixed-effects meta-analyses into structural equation modeling. Psychological Methods, 13, 182–202.

    PubMed  Article  Google Scholar 

  14. Cheung, M. W.-L. (2009). Constructing approximate confidence intervals for parameters with structural equation models. Structural Equation Modeling, 16, 267–294.

    Article  Google Scholar 

  15. Cheung, M. W.-L. (2010). Fixed-effects meta-analyses as multiple-group structural equation models. Structural Equation Modeling, 17, 481–509.

    Article  Google Scholar 

  16. Cheung, M. W.-L. (2013a). Implementing restricted maximum likelihood estimation in structural equation models. Structural Equation Modeling, 20, 157–167.

    Article  Google Scholar 

  17. Cheung, M. W.-L. (2013b). metaSEM: Meta-analysis using structural equation modeling. Retrieved from http://courses.nus.edu.sg/course/psycwlm/Internet/metaSEM/

  18. Cheung, M. W.-L. (in press-a). Modeling dependent effect sizes with three-level meta-analysis: A structural equation modeling approach. Psychological Methods.

  19. Cheung, M. W.-L. (in press-b). Multivariate meta-analysis as structural equation models. Structural Equation Modeling.

  20. Cheung, M. W.-L., & Chan, W. (2004). Testing dependent correlation coefficients via structural equation modeling. Organizational Research Methods, 7, 206–223.

    Article  Google Scholar 

  21. Cheung, M. W.-L., & Chan, W. (2005a). Classifying correlation matrices into relatively homogeneous subgroups: A cluster analytic approach. Educational and Psychological Measurement, 65, 954–979.

    Article  Google Scholar 

  22. Cheung, M. W.-L., & Chan, W. (2005b). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10, 40–64.

    PubMed  Article  Google Scholar 

  23. Cheung, M. W.-L., & Chan, W. (2009). A two-stage approach to synthesizing covariance matrices in meta-analytic structural equation modeling. Structural Equation Modeling, 16, 28–53.

    Article  Google Scholar 

  24. Cheung, M. W.-L., Leung, K., & Au, K. (2006). Evaluating multilevel models in cross-cultural research: An illustration with Social Axioms. Journal of Cross-Cultural Psychology, 37, 522–541.

    Article  Google Scholar 

  25. Colquitt, J. A., LePine, J. A., & Noe, R. A. (2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85, 678–707.

    PubMed  Article  Google Scholar 

  26. Conway, J. M. (1999). Distinguishing contextual performance from task performance for managerial jobs. Journal of Applied Psychology, 84, 3–13.

    Article  Google Scholar 

  27. Cooper, H. M., Hedges, L. V., & Valentine, J. C. (2009). The handbook of research synthesis and meta-analysis. New York, NY: Sage.

    Google Scholar 

  28. Cudeck, R. (1989). Analysis of correlation-matrices using covariance structure models. Psychological Bulletin, 105, 317–327.

    Article  Google Scholar 

  29. Curran, P. J. (2009). The seemingly quixotic pursuit of a cumulative psychological science: Introduction to the special issue. Psychological Methods, 14, 77–80.

    PubMed Central  PubMed  Article  Google Scholar 

  30. Curran, P., West, S., & Finch, J. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16–29.

    Article  Google Scholar 

  31. Development Core Team, R. (2013). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from www.R-project.org

    Google Scholar 

  32. Digman, J. M. (1997). Higher-order factors of the Big Five. Journal of Personality and Social Psychology, 73, 1246–1256.

    PubMed  Article  Google Scholar 

  33. Eby, L. T., Freeman, D. M., Rush, M. C., & Lance, C. E. (1999). Motivational bases of affective organizational commitment: A partial test of an integrative theoretical model. Journal of Occupational and Organizational Psychology, 72, 463–483.

    Article  Google Scholar 

  34. Hardy, R. J., & Thompson, S. G. (1996). A likelihood approach to meta-analysis with random effects. Statistics in Medicine, 15, 619–629.

    PubMed  Article  Google Scholar 

  35. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.

    Google Scholar 

  36. Hedges, L., & Vevea, J. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486–504.

    Article  Google Scholar 

  37. Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539–1558.

    PubMed  Article  Google Scholar 

  38. Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327, 557–560.

    PubMed Central  PubMed  Article  Google Scholar 

  39. Hom, P. W., Caranikas-Walker, F., Prussia, G. E., & Griffeth, R. W. (1992). A meta-analytical structural equations analysis of a model of employee turnover. Journal of Applied Psychology, 77, 890–909.

    Article  Google Scholar 

  40. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  41. Jackson, D., White, I. R., & Riley, R. D. (2012). Quantifying the impact of between-study heterogeneity in multivariate meta-analyses. Statistics in Medicine, 31, 3805–3820.

    PubMed Central  PubMed  Article  Google Scholar 

  42. Landis, R. S. (2013). Successfully combining meta-analysis and structural equation modeling: Recommendations and strategies. Journal of Business and Psychology. doi:10.1007/s10869-013-9285-x. Advance online publication.

    Google Scholar 

  43. Lee, S.-Y. (2007). Structural equation modeling: A Bayesian approach. Chichester, UK: Wiley.

    Google Scholar 

  44. Litière, S., Alonso, A., & Molenberghs, G. (2008). The impact of a misspecified random-effects distribution on the estimation and the performance of inferential procedures in generalized linear mixed models. Statistics in Medicine, 27, 3125–3144.

    PubMed  Article  Google Scholar 

  45. MacCallum, R. C., & Austin, J. T. (2000). Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51, 201–226.

    PubMed  Article  Google Scholar 

  46. McArdle, J. J., & McDonald, R. P. (1984). Some algebraic properties of the Reticular Action Model for moment structures. British Journal of Mathematical and Statistical Psychology, 37, 234–251.

    PubMed  Article  Google Scholar 

  47. Michel, J. S., Viswesvaran, C., & Thomas, J. (2011). Conclusions from meta-analytic structural equation models generally do not change due to corrections for study artifacts. Research Synthesis Methods, 2, 174–187.

    Article  Google Scholar 

  48. Murayama, K., & Elliot, A. J. (2012). The competition–performance relation: A meta-analytic review and test of the opposing processes model of competition and performance. Psychological Bulletin, 138, 1035–1070.

    PubMed  Article  Google Scholar 

  49. Muthén, B., & Asparouhov, T. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17, 313–335.

    PubMed  Article  Google Scholar 

  50. Muthén, B., Kaplan, D., & Hollis, M. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 51, 431–462.

    Article  Google Scholar 

  51. Prevost, A., Mason, D., Griffin, S., Kinmonth, A., Sutton, S., & Spiegelhalter, D. (2007). Allowing for correlations between correlations in random-effects meta-analysis of correlation matrices. Psychological Methods, 12, 434–450.

    PubMed  Article  Google Scholar 

  52. Riley, R. D., Lambert, P. C., & Abo-Zaid, G. (2010). Meta-analysis of individual participant data: Rationale, conduct, and reporting. British Medical Journal, 340, c221.

    PubMed  Article  Google Scholar 

  53. Rosenthal, R. (1991). Meta-analytic procedures for social research (Revth ed.). Newbury Park, CA: Sage.

    Google Scholar 

  54. Saris, W. E., Satorra, A., & van der Veld, W. (2009). Testing structural equation models or detection of misspecifications? Structural Equation Modeling, 16, 561–582.

    Article  Google Scholar 

  55. Schmidt, F., Oh, I., & Hayes, T. (2009). Fixed-versus random-effects models in meta-analysis: Model properties and an empirical comparison of differences in results. British Journal of Mathematical and Statistical Psychology, 62, 97–128.

    PubMed  Article  Google Scholar 

  56. Shuster, J. J. (2010). Empirical vs. natural weighting in random effects meta-analysis. Statistics in Medicine, 29, 1259–1265.

    PubMed Central  PubMed  Article  Google Scholar 

  57. Sidik, K., & Jonkman, J. N. (2006). Robust variance estimation for random effects meta-analysis. Computational Statistics and Data Analysis, 50, 3681–3701.

    Article  Google Scholar 

  58. Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995). Bayesian approaches to random-effects meta-analysis: A comparative study. Statistics in Medicine, 14, 2685–2699.

    PubMed  Article  Google Scholar 

  59. Steinmetz, H., Baeuerle, N., & Isidor, R. (2012). Testing the circular structure of human values: A meta-analytical structural equation modeling approach. Survey Research Methods, 6, 61–75.

    Google Scholar 

  60. Stewart, G. B., Altman, D. G., Askie, L. M., Duley, L., Simmonds, M. C., & Stewart, L. A. (2012). Statistical analysis of individual participant data meta-analyses: A comparison of methods and recommendations for practice. PLoS ONE, 7, e46042.

    PubMed Central  PubMed  Article  Google Scholar 

  61. Sutton, A. J., & Abrams, K. R. (2001). Bayesian methods in meta-analysis and evidence synthesis. Statistical Methods in Medical Research, 10, 277–303.

    PubMed  Article  Google Scholar 

  62. Sutton, A. J., Kendrick, D., & Coupland, C. A. C. (2008). Meta-analysis of individual- and aggregate-level data. Statistics in Medicine, 27, 651–669.

    PubMed  Article  Google Scholar 

  63. Tett, R. P., & Meyer, J. P. (1993). Job satisfaction, organizational commitment, turnover intention, and turnover: Path analyses based on meta-analytic findings. Personnel Psychology, 46, 259–290.

    Article  Google Scholar 

  64. Verbeke, G., & Lesaffre, E. (1997). The effect of misspecifying the random-effects distribution in linear mixed models for longitudinal data. Computational Statistics and Data Analysis, 23, 541–556.

    Article  Google Scholar 

  65. Viswesvaran, C., & Ones, D. (1995). Theory testing: Combining psychometric meta-analysis and structural equations modeling. Personnel Psychology, 48, 865–885.

    Article  Google Scholar 

  66. Yuan, K.-H., & Bentler, P. M. (2007). Robust procedures in structural equation modeling. In S.-Y. Lee (Ed.), Handbook of latent variable and related models (pp. 367–397). Amsterdam, The Netherlands: Elsevier/North-Holland.

    Google Scholar 

Download references

Author Note

This research was supported by the Academic Research Fund Tier 1 (R581-000-111-112) from the Ministry of Education, Singapore.

Author information

Affiliations

  1. Department of Psychology, Faculty of Arts and Social Sciences, National University of Singapore, Block AS4, Level 2, 9 Arts Link, Singapore, 117570, Singapore

    Mike W.-L. Cheung

Authors
  1. Mike W.-L. Cheung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mike W.-L. Cheung.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Cheung, M.W. Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R. Behav Res 46, 29–40 (2014). https://doi.org/10.3758/s13428-013-0361-y

Download citation

Keywords

  • Structural equation modeling
  • Meta-analysis
  • Meta-analytic structural equation modeling
  • Random-effects model