Advertisement

Behavior Research Methods

, Volume 46, Issue 1, pp 29–40 | Cite as

Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R

Article

Abstract

Meta-analytic structural equation modeling (MASEM) combines the ideas of meta-analysis and structural equation modeling for the purpose of synthesizing correlation or covariance matrices and fitting structural equation models on the pooled correlation or covariance matrix. Cheung and Chan (Psychological Methods 10:40–64, 2005b, Structural Equation Modeling 16:28–53, 2009) proposed a two-stage structural equation modeling (TSSEM) approach to conducting MASEM that was based on a fixed-effects model by assuming that all studies have the same population correlation or covariance matrices. The main objective of this article is to extend the TSSEM approach to a random-effects model by the inclusion of study-specific random effects. Another objective is to demonstrate the procedures with two examples using the metaSEM package implemented in the R statistical environment. Issues related to and future directions for MASEM are discussed.

Keywords

Structural equation modeling Meta-analysis Meta-analytic structural equation modeling Random-effects model 

Notes

Author Note

This research was supported by the Academic Research Fund Tier 1 (R581-000-111-112) from the Ministry of Education, Singapore.

References

  1. Allison, P. D. (1987). Estimation of linear models with incomplete data. Sociological Methodology, 17, 71–103.CrossRefGoogle Scholar
  2. Barrett, P. (2007). Structural equation modelling: Adjudging model fit. Personality and Individual Differences, 42(5), 815–824.Google Scholar
  3. Becker, B. J. (1992). Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 17, 341–362.CrossRefGoogle Scholar
  4. Becker, B. J. (1995). Corrections to “Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 20, 100–102.CrossRefGoogle Scholar
  5. Becker, G. (1996). The meta-analysis of factor analyses: An illustration based on the cumulation of correlation matrices. Psychological Methods, 1, 341–353.CrossRefGoogle Scholar
  6. Becker, B. J. (2009). Model-based meta-analysis. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 377–395). New York, NY: Sage.Google Scholar
  7. Becker, B. J., & Schram, C. M. (1994). Examining explanatory models through research synthesis. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 357–381). New York, NY: Sage.Google Scholar
  8. Beretvas, S., & Furlow, C. (2006). Evaluation of an approximate method for synthesizing covariance matrices for use in meta-analytic SEM. Structural Equation Modeling, 13, 153–185.CrossRefGoogle Scholar
  9. Boker, S., Neale, M., Maes, H., Wilde, M., Spiegel, M., Brick, T., & Fox, J. (2011). OpenMx: An open source extended structural equation modeling framework. Psychometrika, 76, 306–317.PubMedCentralPubMedCrossRefGoogle Scholar
  10. Bonett, D. (2009). Meta-analytic interval estimation for standardized and unstandardized mean differences. Psychological Methods, 14, 225–238.PubMedCrossRefGoogle Scholar
  11. Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2009). Introduction to meta-analysis. Hoboken, NJ: Wiley.CrossRefGoogle Scholar
  12. Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.PubMedCrossRefGoogle Scholar
  13. Cheung, M. W.-L. (2008). A model for integrating fixed-, random-, and mixed-effects meta-analyses into structural equation modeling. Psychological Methods, 13, 182–202.PubMedCrossRefGoogle Scholar
  14. Cheung, M. W.-L. (2009). Constructing approximate confidence intervals for parameters with structural equation models. Structural Equation Modeling, 16, 267–294.CrossRefGoogle Scholar
  15. Cheung, M. W.-L. (2010). Fixed-effects meta-analyses as multiple-group structural equation models. Structural Equation Modeling, 17, 481–509.CrossRefGoogle Scholar
  16. Cheung, M. W.-L. (2013a). Implementing restricted maximum likelihood estimation in structural equation models. Structural Equation Modeling, 20, 157–167.CrossRefGoogle Scholar
  17. Cheung, M. W.-L. (2013b). metaSEM: Meta-analysis using structural equation modeling. Retrieved from http://courses.nus.edu.sg/course/psycwlm/Internet/metaSEM/
  18. Cheung, M. W.-L. (in press-a). Modeling dependent effect sizes with three-level meta-analysis: A structural equation modeling approach. Psychological Methods.Google Scholar
  19. Cheung, M. W.-L. (in press-b). Multivariate meta-analysis as structural equation models. Structural Equation Modeling.Google Scholar
  20. Cheung, M. W.-L., & Chan, W. (2004). Testing dependent correlation coefficients via structural equation modeling. Organizational Research Methods, 7, 206–223.CrossRefGoogle Scholar
  21. Cheung, M. W.-L., & Chan, W. (2005a). Classifying correlation matrices into relatively homogeneous subgroups: A cluster analytic approach. Educational and Psychological Measurement, 65, 954–979.CrossRefGoogle Scholar
  22. Cheung, M. W.-L., & Chan, W. (2005b). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10, 40–64.PubMedCrossRefGoogle Scholar
  23. Cheung, M. W.-L., & Chan, W. (2009). A two-stage approach to synthesizing covariance matrices in meta-analytic structural equation modeling. Structural Equation Modeling, 16, 28–53.CrossRefGoogle Scholar
  24. Cheung, M. W.-L., Leung, K., & Au, K. (2006). Evaluating multilevel models in cross-cultural research: An illustration with Social Axioms. Journal of Cross-Cultural Psychology, 37, 522–541.CrossRefGoogle Scholar
  25. Colquitt, J. A., LePine, J. A., & Noe, R. A. (2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85, 678–707.PubMedCrossRefGoogle Scholar
  26. Conway, J. M. (1999). Distinguishing contextual performance from task performance for managerial jobs. Journal of Applied Psychology, 84, 3–13.CrossRefGoogle Scholar
  27. Cooper, H. M., Hedges, L. V., & Valentine, J. C. (2009). The handbook of research synthesis and meta-analysis. New York, NY: Sage.Google Scholar
  28. Cudeck, R. (1989). Analysis of correlation-matrices using covariance structure models. Psychological Bulletin, 105, 317–327.CrossRefGoogle Scholar
  29. Curran, P. J. (2009). The seemingly quixotic pursuit of a cumulative psychological science: Introduction to the special issue. Psychological Methods, 14, 77–80.PubMedCentralPubMedCrossRefGoogle Scholar
  30. Curran, P., West, S., & Finch, J. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16–29.CrossRefGoogle Scholar
  31. Development Core Team, R. (2013). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from www.R-project.org Google Scholar
  32. Digman, J. M. (1997). Higher-order factors of the Big Five. Journal of Personality and Social Psychology, 73, 1246–1256.PubMedCrossRefGoogle Scholar
  33. Eby, L. T., Freeman, D. M., Rush, M. C., & Lance, C. E. (1999). Motivational bases of affective organizational commitment: A partial test of an integrative theoretical model. Journal of Occupational and Organizational Psychology, 72, 463–483.CrossRefGoogle Scholar
  34. Hardy, R. J., & Thompson, S. G. (1996). A likelihood approach to meta-analysis with random effects. Statistics in Medicine, 15, 619–629.PubMedCrossRefGoogle Scholar
  35. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.Google Scholar
  36. Hedges, L., & Vevea, J. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486–504.CrossRefGoogle Scholar
  37. Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539–1558.PubMedCrossRefGoogle Scholar
  38. Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327, 557–560.PubMedCentralPubMedCrossRefGoogle Scholar
  39. Hom, P. W., Caranikas-Walker, F., Prussia, G. E., & Griffeth, R. W. (1992). A meta-analytical structural equations analysis of a model of employee turnover. Journal of Applied Psychology, 77, 890–909.CrossRefGoogle Scholar
  40. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage.Google Scholar
  41. Jackson, D., White, I. R., & Riley, R. D. (2012). Quantifying the impact of between-study heterogeneity in multivariate meta-analyses. Statistics in Medicine, 31, 3805–3820.PubMedCentralPubMedCrossRefGoogle Scholar
  42. Landis, R. S. (2013). Successfully combining meta-analysis and structural equation modeling: Recommendations and strategies. Journal of Business and Psychology. doi: 10.1007/s10869-013-9285-x. Advance online publication.Google Scholar
  43. Lee, S.-Y. (2007). Structural equation modeling: A Bayesian approach. Chichester, UK: Wiley.CrossRefGoogle Scholar
  44. Litière, S., Alonso, A., & Molenberghs, G. (2008). The impact of a misspecified random-effects distribution on the estimation and the performance of inferential procedures in generalized linear mixed models. Statistics in Medicine, 27, 3125–3144.PubMedCrossRefGoogle Scholar
  45. MacCallum, R. C., & Austin, J. T. (2000). Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51, 201–226.PubMedCrossRefGoogle Scholar
  46. McArdle, J. J., & McDonald, R. P. (1984). Some algebraic properties of the Reticular Action Model for moment structures. British Journal of Mathematical and Statistical Psychology, 37, 234–251.PubMedCrossRefGoogle Scholar
  47. Michel, J. S., Viswesvaran, C., & Thomas, J. (2011). Conclusions from meta-analytic structural equation models generally do not change due to corrections for study artifacts. Research Synthesis Methods, 2, 174–187.CrossRefGoogle Scholar
  48. Murayama, K., & Elliot, A. J. (2012). The competition–performance relation: A meta-analytic review and test of the opposing processes model of competition and performance. Psychological Bulletin, 138, 1035–1070.PubMedCrossRefGoogle Scholar
  49. Muthén, B., & Asparouhov, T. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17, 313–335.PubMedCrossRefGoogle Scholar
  50. Muthén, B., Kaplan, D., & Hollis, M. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 51, 431–462.CrossRefGoogle Scholar
  51. Prevost, A., Mason, D., Griffin, S., Kinmonth, A., Sutton, S., & Spiegelhalter, D. (2007). Allowing for correlations between correlations in random-effects meta-analysis of correlation matrices. Psychological Methods, 12, 434–450.PubMedCrossRefGoogle Scholar
  52. Riley, R. D., Lambert, P. C., & Abo-Zaid, G. (2010). Meta-analysis of individual participant data: Rationale, conduct, and reporting. British Medical Journal, 340, c221.PubMedCrossRefGoogle Scholar
  53. Rosenthal, R. (1991). Meta-analytic procedures for social research (Revth ed.). Newbury Park, CA: Sage.Google Scholar
  54. Saris, W. E., Satorra, A., & van der Veld, W. (2009). Testing structural equation models or detection of misspecifications? Structural Equation Modeling, 16, 561–582.CrossRefGoogle Scholar
  55. Schmidt, F., Oh, I., & Hayes, T. (2009). Fixed-versus random-effects models in meta-analysis: Model properties and an empirical comparison of differences in results. British Journal of Mathematical and Statistical Psychology, 62, 97–128.PubMedCrossRefGoogle Scholar
  56. Shuster, J. J. (2010). Empirical vs. natural weighting in random effects meta-analysis. Statistics in Medicine, 29, 1259–1265.PubMedCentralPubMedCrossRefGoogle Scholar
  57. Sidik, K., & Jonkman, J. N. (2006). Robust variance estimation for random effects meta-analysis. Computational Statistics and Data Analysis, 50, 3681–3701.CrossRefGoogle Scholar
  58. Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995). Bayesian approaches to random-effects meta-analysis: A comparative study. Statistics in Medicine, 14, 2685–2699.PubMedCrossRefGoogle Scholar
  59. Steinmetz, H., Baeuerle, N., & Isidor, R. (2012). Testing the circular structure of human values: A meta-analytical structural equation modeling approach. Survey Research Methods, 6, 61–75.Google Scholar
  60. Stewart, G. B., Altman, D. G., Askie, L. M., Duley, L., Simmonds, M. C., & Stewart, L. A. (2012). Statistical analysis of individual participant data meta-analyses: A comparison of methods and recommendations for practice. PLoS ONE, 7, e46042.PubMedCentralPubMedCrossRefGoogle Scholar
  61. Sutton, A. J., & Abrams, K. R. (2001). Bayesian methods in meta-analysis and evidence synthesis. Statistical Methods in Medical Research, 10, 277–303.PubMedCrossRefGoogle Scholar
  62. Sutton, A. J., Kendrick, D., & Coupland, C. A. C. (2008). Meta-analysis of individual- and aggregate-level data. Statistics in Medicine, 27, 651–669.PubMedCrossRefGoogle Scholar
  63. Tett, R. P., & Meyer, J. P. (1993). Job satisfaction, organizational commitment, turnover intention, and turnover: Path analyses based on meta-analytic findings. Personnel Psychology, 46, 259–290.CrossRefGoogle Scholar
  64. Verbeke, G., & Lesaffre, E. (1997). The effect of misspecifying the random-effects distribution in linear mixed models for longitudinal data. Computational Statistics and Data Analysis, 23, 541–556.CrossRefGoogle Scholar
  65. Viswesvaran, C., & Ones, D. (1995). Theory testing: Combining psychometric meta-analysis and structural equations modeling. Personnel Psychology, 48, 865–885.CrossRefGoogle Scholar
  66. Yuan, K.-H., & Bentler, P. M. (2007). Robust procedures in structural equation modeling. In S.-Y. Lee (Ed.), Handbook of latent variable and related models (pp. 367–397). Amsterdam, The Netherlands: Elsevier/North-Holland.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2013

Authors and Affiliations

  1. 1.Department of Psychology, Faculty of Arts and Social SciencesNational University of SingaporeSingaporeSingapore

Personalised recommendations