# Meta-analyzing dependent correlations: An SPSS macro and an R script

- 849 Downloads
- 1 Citations

## Abstract

The presence of dependent correlation is a common problem in meta-analysis. Cheung and Chan (2004, 2008) have shown that samplewise-adjusted procedures perform better than the more commonly adopted simple within-sample mean procedures. However, samplewise-adjusted procedures have rarely been applied in meta-analytic reviews, probably due to the lack of suitable ready-to-use programs. In this article, we compare the samplewise-adjusted procedures with existing procedures to handle dependent effect sizes, and present the samplewise-adjusted procedures in a way that will make them more accessible to researchers conducting meta-analysis. We also introduce two tools, an SPSS macro and an R script, that researchers can apply to their meta-analyses; these tools are compatible with existing meta-analysis software packages.

## Keywords

Meta-analysis Dependent effect sizes## Notes

### Author note

This research was supported by Grant No. MYRG047(Y1-L1)-FSH11-CSF, a Multi-Year Research Grant from the University of Macau.

## Supplementary material

## References

- Baltes, B. B., Briggs, T. E., Huff, J. W., Wright, J. A., & Neuman, G. A. (1999). Flexible and compressed workweek schedules: A meta-analysis of their effects on work-related criteria.
*Journal of Applied Psychology, 84,*496–513.CrossRefGoogle Scholar - Becker, B. J. (1992). Using results from replicated studies to estimate linear models.
*Journal of Educational and Behavioral Statistics, 17,*341–362. doi: 10.3102/10769986017004341 Google Scholar - Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis.
*Research Synthesis Methods, 2,*97–111. doi: 10.1002/jrsm.12 CrossRefGoogle Scholar - Cheung, M.W.L. (2013)
*metaSEM: Meta-analysis using structural equation modeling*. Retrieved from http://courses.nus.edu.sg/course/psycwlm/Internet/metaSEM/metaSEM.pdf - Cheung, M. W.-L. (2013a). metaSEM: Meta-analysis: A structural equation modeling approach (R package version 0.8-4) [Software]. Retrieved from http://courses.nus.edu.sg/course/psycwlm/internet/metaSEM/index.html
- Cheung, M. W.-L. (2013b). Multivariate meta-analysis as structural equation models.
*Structural Equation Modeling, 20,*429–454. doi: 10.1080/10705511.2013.797827 CrossRefGoogle Scholar - Cheung, M. W.-L. (2013c). Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach.
*Psychological Methods*. doi: 10.1037/a0032968 - Cheung, S. F., & Chan, D. K.-S. (2004). Dependent effect sizes in meta-analysis: incorporating the degree of interdependence.
*Journal of Applied Psychology, 89,*780–791. doi: 10.1037/0021-9010.89.5.780 PubMedCrossRefGoogle Scholar - Cheung, S. F., & Chan, D. K.-S. (2008). Dependent correlations in meta-analysis: The case of heterogeneous dependence.
*Educational and Psychological Measurement, 68,*760–777. doi: 10.1177/0013164408315263 CrossRefGoogle Scholar - Cooper, H. M. (2009).
*Research synthesis and meta-analysis: A step-by-step approach*(4th ed.). Thousand Oaks, CA: Sage.Google Scholar - Gleser, L. J., & Olkin, I. (2009). Stochastically dependent effect sizes. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.),
*The handbook of research synthesis and meta-analysis*(2nd ed., pp. 357–376). New York, NY: Russell Sage.Google Scholar - Hayes, A. F. (2012). PROCESS: A versatile computational tool for observed variable mediation, moderation, and conditional process modeling [White paper]. Retrieved from www.afhayes.com/public/process2012.pdf
- Hedges, L. V., & Olkin, I. (1985).
*Statistical methods for meta-analysis*. Orlando FL: Academic.Google Scholar - Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates.
*Research Synthesis Methods, 1,*39–65. doi: 10.1002/jrsm.5 CrossRefGoogle Scholar - Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis.
*Psychological Methods, 3,*486–504.CrossRefGoogle Scholar - Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis.
*Statistics in Medicine, 21,*1539–1558. doi: 10.1002/sim.1186 PubMedCrossRefGoogle Scholar - Huffcutt, A. I., Conway, J. M., Roth, P. L., & Stone, N. J. (2001). Identification and meta-analytic assessment of psychological constructs measured in employment interviews.
*Journal of Applied Psychology, 86,*897–913.PubMedCrossRefGoogle Scholar - Hunter, J. E., & Schmidt, F. L. (2004).
*Methods of meta-analysis: Correcting error and bias in research findings*(2nd ed.). Thousand Oaks, CA: Sage.Google Scholar - Kalaian, H. A., & Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis.
*Psychological Methods, 1,*227–235. doi: 10.1037/1082-989X.1.3.227 CrossRefGoogle Scholar - Konstantopoulos, S. (2011). Fixed effects and variance components estimation in three-level meta-analysis.
*Research Synthesis Methods, 2,*61–76. doi: 10.1002/jrsm.35 CrossRefGoogle Scholar - Madden, T. J., Ellen, P. S., & Ajzen, I. (1992). A comparison of the theory of planned behavior and the theory of reasoned action.
*Personality and Social Psychology Bulletin, 18,*3–9. doi: 10.1177/0146167292181001 CrossRefGoogle Scholar - Marsh, H. W., Bornmann, L., Mutz, R., Daniel, H.-D., & O’Mara, A. (2009). Gender effects in the peer reviews of grant proposals: A comprehensive meta-analysis comparing traditional and multilevel approaches.
*Review of Educational Research, 79,*1290–1326. doi: 10.3102/0034654309334143 CrossRefGoogle Scholar - Martinussen, M., & Bjørnstad, J. F. (1999). Meta-analysis calculations based on independent and nonindependent cases.
*Educational and Psychological Measurement, 59,*928–950. doi: 10.1177/00131649921970260 CrossRefGoogle Scholar - Muenchen, R. A. (2012). The popularity of data analysis software [Webpage]. Retrieved from http://r4stats.com/articles/popularity/
- R Development Core Team. (2012).
*R: A language and environment for statistical computing*. Vienna, Austria: R Foundation for Statistical Computing. Downloaded from www.R-project.org/ Google Scholar - Raudenbush, S. W., Becker, B. J., & Kalaian, H. (1988). Modeling multivariate effect sizes.
*Psychological Bulletin, 103,*111–120. doi: 10.1037/0033-2909.103.1.111 CrossRefGoogle Scholar - Raudenbush, S. W., & Bryk, A. S. (2002).
*Hierarchical linear models: Applications and data analysis methods*(2nd ed.). Thousand Oaks, CA: Sage.Google Scholar - Schmidt, F. L., Law, K., Hunter, J. E., & Rothstein, H. R. (1993). Refinements in validity generalization methods: Implications for the situational specificity hypothesis.
*Journal of Applied Psychology, 78,*3–12.CrossRefGoogle Scholar - Schmidt, F. L., Le, H., & Oh, I.-S. (2009). Correcting for the distorting effects of study artifacts in meta-analysis. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.),
*The handbook of research synthesis and meta-analysis*(2nd ed., pp. 317–333). New York, NY: Russell Sage.Google Scholar - Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2013). Three-level meta-analysis of dependent effect sizes.
*Behavior Research Methods, 45,*576–594. doi: 10.3758/s13428-012-0261-6 PubMedCrossRefGoogle Scholar - Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package.
*Journal of Statistical Software*,*36*(3). Retrieved from www.jstatsoft.org/v36/i03/paper - Viswesvaran, C., Sanchez, J. L., & Fisher, J. (1999). The role of social support in the process of work stress: A meta-analysis.
*Journal of Vocational Behavior, 54,*314–334.CrossRefGoogle Scholar - Whitener, E. M. (1990). Confusion of confidence intervals and credibility intervals in meta-analysis.
*Journal of Applied Psychology, 75,*315–321. doi: 10.1037/0021-9010.75.3.315 CrossRefGoogle Scholar - Wilson, D. B. (2005). Macro for SPSS/Win Version 6.1 or Higher (Version 2005.05.23) [Computer Software]. Retrieved from http://mason.gmu.edu/~dwilsonb/ma.html