Behavior Research Methods

, Volume 41, Issue 2, pp 425–438 | Cite as

Comparison of methods for constructing confidence intervals of standardized indirect effects

Article

Abstract

Mediation models are often used as a means to explain the psychological mechanisms between an independent and a dependent variable in the behavioral and social sciences. A major limitation of the unstandardized indirect effect calculated from raw scores is that it cannot be interpreted as an effect-size measure. In contrast, the standardized indirect effect calculated from standardized scores can be a good candidate as a measure of effect size because it is scale invariant. In the present article, 11 methods for constructing the confidence intervals (CIs) of the standardized indirect effects were evaluated via a computer simulation. These included six Wald CIs, three bootstrap CIs, one likelihood-based CI, and the PRODCLIN CI. The results consistently showed that the percentile bootstrap, the bias-corrected bootstrap, and the likelihood-based approaches had the best coverage probability. Mplus, LISREL, and Mx syntax were included to facilitate the use of these preferred methods in applied settings. Future issues on the use of the standardized indirect effects are discussed.

References

  1. Alwin, D. F., & Hauser, R. M. (1975). The decomposition of effects in path analysis. American Sociological Review, 40, 37–47.CrossRefGoogle Scholar
  2. American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: American Psychological Association.Google Scholar
  3. Aroian, L. A. (1944). The probability function of the product of two normally distributed variables. Annals of Mathematical Statistics, 18, 265–271.CrossRefGoogle Scholar
  4. Baron, R. M., & Kenny, D. A. (1986). The moderator—mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality & Social Psychology, 51, 1173–1182.CrossRefGoogle Scholar
  5. Bauer, D. J., Preacher, K. J., & Gil, K. M. (2006). Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: New procedures and recommendations. Psychological Methods, 11, 142–163.CrossRefPubMedGoogle Scholar
  6. Bentler, P. M. (2007). Can scientifically useful hypotheses be tested with correlations? American Psychologist, 62, 772–782.CrossRefGoogle Scholar
  7. Bentler, P. M., & Lee, S. Y. (1983). Covariance structures under polynomial constraints: Applications to correlation and alpha-type structural models. Journal of Educational Statistics, 8, 207–222.CrossRefGoogle Scholar
  8. Bobko, P., & Rieck, A. (1980). Large sample estimators for standard errors of functions of correlation coefficients. Applied Psychological Measurement, 4, 385–398.CrossRefGoogle Scholar
  9. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.Google Scholar
  10. Bollen, K. A., & Stine, R. (1990). Direct and indirect effects: Classical and bootstrap estimates of variability. Sociological Methodology, 20, 115–140.CrossRefGoogle Scholar
  11. Cheung, G. W., & Lau, R. S. (2008). Testing mediation and suppression effects of latent variables: Bootstrapping with structural equation models. Organizational Research Methods, 11, 296–325.CrossRefGoogle Scholar
  12. Cheung, M. W.-L. (2007). Comparison of approaches to constructing confidence intervals for mediating effects using structural equation models. Structural Equation Modeling, 14, 227–246.Google Scholar
  13. Cheung, M. W.-L. (2009). Constructing approximate confidence intervals for parameters with structural equation models. Structural Equation Modeling, 16, 267–294.CrossRefGoogle Scholar
  14. Cheung, M. W.-L., & Chan, W. (2005). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10, 40–64.CrossRefPubMedGoogle Scholar
  15. Cheung, M. W.-L., & Chan, W. (2009). A two-stage approach to synthesizing covariance matrices in meta-analytic structural equation modeling. Structural Equation Modeling, 16, 28–53.CrossRefGoogle Scholar
  16. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Google Scholar
  17. Cohen, J. (1994). The world is round ( p <.05). American Psychologist, 49, 997–1003.CrossRefGoogle Scholar
  18. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Erlbaum.Google Scholar
  19. Cudeck, R. (1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105, 317–327.CrossRefGoogle Scholar
  20. Cumming, G., Fidler, F., Leonard, M., Kalinowski, P., Christiansen, A., Kleinig, A., et al. (2007). Statistical reform in psychology: Is anything changing? Psychological Science, 18, 230–232.CrossRefPubMedGoogle Scholar
  21. Davison, A. C., & Hinkley, D. V. (1997). Bootstrap methods and their application. New York: Cambridge University Press.Google Scholar
  22. Edwards, J. R., & Lambert, L. S. (2007). Methods for integrating moderation and mediation: A general analytical framework using moderated path analysis. Psychological Methods, 12, 1–22.CrossRefPubMedGoogle Scholar
  23. Fritz, M. S., & MacKinnon, D. P. (2007). Required sample size to detect the mediated effect. Psychological Science, 18, 233–239.CrossRefPubMedGoogle Scholar
  24. Goodman, L. A. (1960). On the exact variance of products. Journal of the American Statistical Association, 55, 708–713.CrossRefGoogle Scholar
  25. Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (1997). What if there were no significance tests? Mahwah, NJ: Erlbaum.Google Scholar
  26. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.Google Scholar
  27. Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486–504.CrossRefGoogle Scholar
  28. Hoyle, R. H., & Kenny, D. A. (1999). Sample size, reliability, and tests of statistical mediation. In R. H. Hoyle (Ed.), Statistical strategies for small sample research (pp. 195–222). Thousand Oakes, CA: Sage.Google Scholar
  29. Hunter, J. E., & Hamilton, M. A. (2002). The advantages of using standardized scores in causal analysis. Human Communication Research, 28, 552–561.CrossRefGoogle Scholar
  30. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage.Google Scholar
  31. James, L. R., & Brett, J. M. (1984). Mediators, moderators, and tests for mediation. Journal of Applied Psychology, 69, 307–321.CrossRefGoogle Scholar
  32. James, L. R., Mulaik, S. A., & Brett, J. M. (2006). A tale of two methods. Organizational Research Methods, 9, 233–244.CrossRefGoogle Scholar
  33. Jöreskog, K. G. (1999). How large can a standardized coefficient be? Lincolnwood, IL: Scientific Software International. Available at www.ssicentral.com.Google Scholar
  34. Jöreskog, K. G., & Sörbom, D. (1996). LISREL 8: A user’s reference guide. Chicago: Scientific Software International.Google Scholar
  35. Jöreskog, K. G., Sörbom, D., Du Toit, S., & Du Toit, M. (1999). LISREL 8: New Statistical Features. Chicago: Scientific Software International.Google Scholar
  36. Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating mediation in treatment evaluations. Evaluation Review, 5, 602–619.CrossRefGoogle Scholar
  37. Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational & Psychological Measurement, 56, 746–759.CrossRefGoogle Scholar
  38. Kline, R. B. (2004). Beyond significance testing: Reforming data analysis methods in behavioral research. Washington, DC: American Psychological Association.CrossRefGoogle Scholar
  39. MacKinnon, D. P. (2000). Contrasts in multiple mediator models. In J. S. Rose, L. Chassin, C. C. Presson, & S. J. Sherman (Eds.), Multivariate applications in substance use research: New methods for new questions (pp. 141–160). Mahwah, NJ: Erlbaum.Google Scholar
  40. MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. Mahwah, NJ: Erlbaum.Google Scholar
  41. MacKinnon, D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation analysis. Annual Review of Psychology, 58, 593–614.CrossRefPubMedGoogle Scholar
  42. MacKinnon, D. P., Fritz, M. S., Williams, J., & Lockwood, C. M. (2007). Distribution of the product confidence limits for the indirect effect: Program PRODLIN. Behavior Research Methods, 39, 384–389.CrossRefPubMedGoogle Scholar
  43. MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000). Equivalence of the mediation, confounding, and suppression effect. Prevention Science, 1, 173–181.CrossRefPubMedGoogle Scholar
  44. MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test the significance of the mediated effect. Psychological Methods, 7, 83–104.CrossRefPubMedGoogle Scholar
  45. MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99–128.CrossRefPubMedGoogle Scholar
  46. MacKinnon, D. P., Warsi, G., & Dwyer, J. H. (1995). A simulation study of mediated effect measures. Multivariate Behavioral Research, 30, 41–62.CrossRefPubMedGoogle Scholar
  47. Morgan-Lopez, A. A., & MacKinnon, D. P. (2006). Demonstration and evaluation of a method for assessing mediated moderation. Behavior Research Methods, 38, 77–87.CrossRefPubMedGoogle Scholar
  48. Muthén, L. K., & Muthén, B. O. (2007). Mplus user’s guide (5th ed.). Los Angeles: Muthén & Muthén.Google Scholar
  49. Neale, M. C., Boker, S. M., Xie, G., & Maes, H. H. (2005). Mx: Statistical modeling (6th ed.). Richmond, VA: Virginia Commonwealth University, Department of Psychiatry.Google Scholar
  50. Neale, M. C., & Miller, M. B. (1997). The use of likelihood-based confidence intervals in genetic models. Behavior Genetics, 27, 113–120.CrossRefPubMedGoogle Scholar
  51. Olkin, I., & Finn, J. D. (1995). Correlation redux. Psychological Bulletin, 118, 155–164.CrossRefGoogle Scholar
  52. Olkin, I., & Siotani, M. (1976). Asymptotic distribution of functions of a correlation matrix. In S. Ideka (Ed.), Essays in probability and statistics (pp. 235–251). Tokyo: Shinko Tsusho.Google Scholar
  53. Pituch, K. A., Whittaker, T. A., & Stapleton, L. M. (2005). A comparison of methods to test for mediation in multisite experiments. Multivariate Behavioral Research, 40, 1–23.CrossRefGoogle Scholar
  54. Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, & Computers, 36, 717–731.CrossRefGoogle Scholar
  55. Preacher, K. J., & Hayes, A. F. (2008a). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40, 879–891.CrossRefPubMedGoogle Scholar
  56. Preacher, K. J., & Hayes, A. F. (2008b). Contemporary approaches to assessing mediation in communication research. In A. F. Hayes, M. D. Slater, & L. B. Snyder (Eds.), The Sage sourcebook of advanced data analysis methods for communication research (pp. 13–54). Thousand Oaks, CA: Sage.Google Scholar
  57. Preacher, K. J., Rucker, D. D., & Hayes, A. F. (2007). Addressing moderated mediation hypothesis: Theory, methods, and prescriptions. Multivariate Behavioral Research, 42, 185–227.Google Scholar
  58. Quinones-Vidal, E., Lopez-Garcia, J. J., Penaranda-Ortega, M., & Tortosa-Gil, F. (2004). The nature of social and personality psychology as reflected in JPSP, 1965–2000. Journal of Personality & Social Psychology, 86, 435–452.CrossRefGoogle Scholar
  59. Raykov, T., Brennan, M., Reinhardt, J. P., & Horowitz, A. (2008). Comparison of mediated effects: A correlation structure modeling approach. Structural Equation Modeling, 15, 603–626.CrossRefGoogle Scholar
  60. Raykov, T., & Shrout, P. E. (2002). Reliability of scales with general structure: Point and interval estimation using a structural equation modeling approach. Structural Equation Modeling, 9, 195–212.CrossRefGoogle Scholar
  61. R Development Core Team (2008). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. Available at www.r-project.org.Google Scholar
  62. Rindskopf, D. (1984). Using phantom and imaginary latent variables to parameterize constraints in linear structural models. Psychometrika, 49, 37–47.CrossRefGoogle Scholar
  63. Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods, 7, 422–445.CrossRefPubMedGoogle Scholar
  64. Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equation models. Sociological Methodology, 13, 290–312.CrossRefGoogle Scholar
  65. Sobel, M. E. (1986). Some new results on indirect effects and their standard errors in covariance structural models. Sociological Methodology, 16, 159–186.CrossRefGoogle Scholar
  66. Stuart, A., & Ord, J. K. (1994). Kendall’s advanced theory of statistics: Vol. 1. Distribution theory (6th ed.). New York: Oxford University Press.Google Scholar
  67. Tamhane, A. C., & Dunlop, D. D. (2000). Statistics and data analysis: From elementary to intermediate. Upper Saddle River, NJ: Prentice Hall.Google Scholar
  68. Taylor, A. B., MacKinnon, D. P., & Tein, J. (2008). Tests of the threepath mediated effect. Organizational Research Methods, 11, 241–269.CrossRefGoogle Scholar
  69. Wansbeek, T., & Meijer, E. (2000). Measurement error and latent variables in econometrics. New York: Elsevier.Google Scholar
  70. Wilkinson, L., Task Force on Statistical Inference, American Psychological Association, Science Directorate, Washington DC (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologists, 54, 594–604.CrossRefGoogle Scholar
  71. Williams, J., & MacKinnon, D. P. (2008). Resampling and distribution of the product methods for testing indirect effects in complex models. Structural Equation Modeling, 15, 23–51.PubMedGoogle Scholar
  72. Yung, Y.-F. (2008, July). Testing and contrasting mediation or indirect effects in SEM: An analytic approach and its implementation. Paper presented at the 73rd Annual Meeting of the Psychometric Society, Durham, NH.Google Scholar

Copyright information

© Psychonomic Society, Inc. 2009

Authors and Affiliations

  1. 1.Department of Psychology, Faculty of Arts and Social SciencesNational University of SingaporeSingapore

Personalised recommendations