Comparing Latent Means Without Mean Structure Models: A Projection-Based Approach

Abstract

The conventional setup for multi-group structural equation modeling requires a stringent condition of cross-group equality of intercepts before mean comparison with latent variables can be conducted. This article proposes a new setup that allows mean comparison without the need to estimate any mean structural model. By projecting the observed sample means onto the space of the common scores and the space orthogonal to that of the common scores, the new setup allows identifying and estimating the means of the common and specific factors, although, without replicate measures, variances of specific factors cannot be distinguished from those of measurement errors. Under the new setup, testing cross-group mean differences of the common scores is done independently from that of the specific factors. Such independent testing eliminates the requirement for cross-group equality of intercepts by the conventional setup in order to test cross-group equality of means of latent variables using chi-square-difference statistics. The most appealing piece of the new setup is a validity index for mean differences, defined as the percentage of the sum of the squared observed mean differences that is due to that of the mean differences of the common scores. By analyzing real data with two groups, the new setup is shown to offer more information than what is obtained under the conventional setup.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    Using multiple starting values may increase the chance of convergence. However, such an effort may not work well in practice because non-convergences were common in simulation studies even when the true population values are used as starting values (Anderson & Gerbing, 1984; Hu, Bentler, & Kano, 1992; Jackson, 2001).

  2. 2.

    One might think that the new method is equivalent to mean comparison via the commonly used formulas of factor scores, we have an example in the next section showing that this is not the case.

  3. 3.

    Either specific factors or measurement errors do not correlate among themselves or with each other in theory. However, non-random measurement errors due to method effects or other hard-to-control factors do occur in practice (see e.g., Byrne, 1989). Thus, the covariance matrices \({\varvec{\Psi }}^{(j)}\) (\(j=1\), 2) do not have to be diagonal although they typically are.

  4. 4.

    The identification issue also occurs with the conventional setup and is closely related to latent variable modeling, where there is no unique way to identify the scale of each latent variable. Any rule of identification needs to make mean comparison with latent variables substantively meaningful.

  5. 5.

    The definition of \(\rho _{\tau }^2\) resembles that of the reliability of measurement. But both the denominator and numerator in (11) are measures of mean differences rather than variances. Also, we may use \(\rho _{\tau }\) as a validity index rather than \(\rho _{\tau }^2\). This simply makes the value greater (\(\rho _{\tau }>\rho _{\tau }^2\)), but \(\rho _{\tau }\) does not possess the desired meaning of “the percentage of the mean differences of the manifest variables that is due to the differences in means of the common scores.”

  6. 6.

    The statistic \(T_{\mu }\) is calculated the same way as how \(T_{\nu }\) is evaluated. But the matrix \({\hat{\mathbf{H}}}\) in (19), including the one in the formulation of \({\hat{{\varvec{\Pi }}}}_{\nu }\), is set as an identity matrix.

  7. 7.

    When \(H_{\mu }\): \({\varvec{\mu }}^{(1)}={\varvec{\mu }}^{(2)}\) holds or when \({\bar{\mathbf{y}}}^{(d)}\) is not statistically significant, the occurrence of statistically significant z-scores corresponding to \({\hat{\mu }}_{\tau }^{(d)}\) or \({\hat{\nu }}^{(d)}\) is simply due to type I error. When only a subset of \({\bar{\mathbf{y}}}^{(d)}\) is not statistically significant, the occurrence of statistically significant z-scores corresponding to \({\hat{\mu }}_{\tau }^{(d)}\) or \({\hat{\nu }}^{(d)}\) may not be due to sampling error.

References

  1. Anderson, J. C., & Gerbing, D. W. (1984). The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis. Psychometrika, 49, 155–173.

    Article  Google Scholar 

  2. Bentler, P. M. (2006). EQS 6 structural equations program manual. Encino, CA: Multivariate Software.

    Google Scholar 

  3. Bentler, P. M., Lee, S.-Y., & Weng, L.-J. (1987). Multiple population covariance structure analysis under arbitrary distribution theory. Communication in Statistics-Theory and Method, 16, 1951–1964.

    Article  Google Scholar 

  4. Bentler, P. M., & Yuan, K.-H. (2000). On adding a mean structure to a covariance structure model. Educational and Psychological Measurement, 60, 326–339.

    Article  Google Scholar 

  5. Beran, R., & Srivastava, M. S. (1985). Bootstrap tests and confidence regions for functions of a covariance matrix. Annals of Statistics, 13, 95–115.

    Article  Google Scholar 

  6. Bollen, K. A., & Stine, R. (1992). Bootstrapping goodness of fit measures in structural equation models. Sociological Methods & Research, 21, 205–229.

    Article  Google Scholar 

  7. Browne, M. W. (1984). Asymptotic distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.

    Article  PubMed  Google Scholar 

  8. Browne, M. W., & Shapiro, A. (2015). Comment on the asymptotics of a distribution-free goodness of fit test statistic. Psychometrika, 80, 196–199.

    Article  PubMed  Google Scholar 

  9. Byrne, B. M. (1989). Multigroup comparisons and the assumption of equivalent construct validity across groups: Methodological and substantive issues. Multivariate Behavioral Research, 24, 503–523.

    Article  PubMed  Google Scholar 

  10. Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456–466.

    Article  Google Scholar 

  11. Ferguson, T. S. (1996). A course in large sample theory. London: Chapman & Hall.

    Google Scholar 

  12. Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  13. Hall, P., & Wilson, S. R. (1991). Two guidelines for bootstrap hypothesis testing. Biometrics, 47, 757–762.

    Article  Google Scholar 

  14. Hancock, G. R., Stapleton, L. M., & Arnold-Berkovits, I. (2009). The tenuousness of invariance tests within multisample covariance and mean structure models. In T. Teo & M. S. Khine (Eds.), Structural equation modeling: Concepts and applications in educational research (pp. 137–174). Rotterdam, Netherlands: Sense Publishers.

    Google Scholar 

  15. Harman, H. H. (1976). Modern factor analysis (3rd ed.). Chicago: The University of Chicago Press.

    Google Scholar 

  16. Holzinger, K. J. & Swineford, F. (1939). A study in factoranalysis: The stability of a bi-factor solution. University of Chicago: Supplementary Educational Monographs, No. 48.

  17. Horn, J. L., & McArdle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18, 117–144.

    Article  PubMed  Google Scholar 

  18. Hu, L. T., Bentler, P. M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted? Psychological Bulletin, 112, 351–362.

    Article  PubMed  Google Scholar 

  19. Jackson, D. L. (2001). Sample size and number of parameter estimates in maximum likelihood confirmatory factor analysis: A Monte Carlo investigation. Structural Equation Modeling, 8, 205–223.

    Article  Google Scholar 

  20. Jennrich, R. I. (1995). An introduction to computational statistics: Regression analysis. Inglewood Cliffs, NJ: Prentice Hall.

    Google Scholar 

  21. Jennrich, R., & Satorra, A. (2013). Continuous orthogonal complement functions and distribution-free goodness of fit tests in moment structure analysis. Psychometrika, 78, 545–552.

    Article  PubMed  Google Scholar 

  22. Johnson, E. C., Meade, A. W., & DuVernet, A. M. (2009). The role of referent indicators in tests of measurement invariance. Structural Equation Modeling, 16, 642–657.

    Article  Google Scholar 

  23. Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183–202.

    Article  Google Scholar 

  24. Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409–426.

    Article  Google Scholar 

  25. Kano, Y. (2001). Structural equation modeling for experimental data. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Present and future (pp. 381–402). Lincolnwood, IL: Scientific Software International.

    Google Scholar 

  26. Kim, E. S., Kwok, O.-M., & Yoon, M. (2012). Testing factorial invariance in multilevel data: A Monte Carlo study. Structural Equation Modeling, 19, 250–267.

    Article  Google Scholar 

  27. Lawley, D. N., & Maxwell, A. E. (1971). Factor analysis as a statistical method (2nd ed.). New York, NY: American Elsevier.

    Google Scholar 

  28. Mardia, K. V. (1970). Measure of multivariate skewness and kurtosis with applications. Biometrika, 57, 519–530.

    Article  Google Scholar 

  29. Meredith, W. (1993). Measurement invariance, factor analysis, and factorial invariance. Psychometrika, 58, 525–542.

    Article  Google Scholar 

  30. Millsap, R. E. (2011). Statistical approaches to measurement invariance. New York: Routledge.

    Google Scholar 

  31. Millsap, R. E., & Kwok, O. M. (2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9, 93–115.

    Article  PubMed  Google Scholar 

  32. Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552–566.

    Article  PubMed  Google Scholar 

  33. Sörbom, D. (1974). A general method for studying differences in factor means and factor structures between groups. British Journal of Mathematical and Statistical Psychology, 27, 229–239.

    Article  Google Scholar 

  34. Sörbom, D. (1989). Model modification. Psychometrika, 54, 371–84.

    Article  Google Scholar 

  35. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70.

    Article  Google Scholar 

  36. Welch, B. L. (1938). The significance of the difference between two means when the population variances are unequal. Biometrika, 29, 350–362.

    Article  Google Scholar 

  37. Yanagihara, H., & Yuan, K.-H. (2005). Three approximate solutions to the multivariate Behrens–Fisher problem. Communications in Statistics: Simulation and Computation, 34, 975–988.

    Article  Google Scholar 

  38. Yanai, H., Takeuchi, K., & Takane, Y. (2011). Projection matrices, generalized inverse matrices, and singular value decomposition. New York: Springer.

    Google Scholar 

  39. Yuan, K.-H., & Bentler, P. M. (2004). On chi-square difference and \(z\) tests in mean and covariance structure analysis when the base model is misspecified. Educational and Psychological Measurement, 64, 737–757.

    Article  Google Scholar 

  40. Yuan, K.-H., Bentler, P. M., & Chan, W. (2004). Structural equation modeling with heavy tailed distributions. Psychometrika, 69, 421–436.

    Article  Google Scholar 

  41. Yuan, K.-H., & Hayashi, K. (2003). Bootstrap approach to inference and power analysis based on three statistics for covariance structure models. British Journal of Mathematical and Statistical Psychology, 56, 93–110.

    Article  PubMed  Google Scholar 

  42. Yuan, K.-H., Marshall, L. L., & Bentler, P. M. (2003). Assessing the effect of model misspecifications on parameter estimates in structural equation models. Sociological Methodology, 33, 241–265.

    Article  Google Scholar 

  43. Zhang, J., & Boos, D. D. (1993). Testing hypotheses about covariance matrices using bootstrap methods. Communications in Statistics, 22, 723–739.

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank Drs. Peter Bentler and Scott Maxwell for their discussion and suggestions in the process of writing this article. Thanks also go to Dr. Alberto Maydeu-Olivares, an associate editor and three reviewers for comments on the previous version. The research was supported in part by the National Science Foundation under Grant No. SES-1461355.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Ke-Hai Yuan.

Appendix

Appendix

In this appendix, we will first obtain the parameters under the conventional setup (i.e., \({\varvec{\kappa }}^{(1)}\), \({\varvec{\gamma }}^{(1)}\), \({\varvec{\kappa }}^{(2)}\), \({\varvec{\gamma }}^{(2)}\)) as functions of those under the new setup (i.e., \({\varvec{\tau }}^{(1)}\), \({\varvec{\nu }}^{(1)}\), \({\varvec{\tau }}^{(2)}\), \({\varvec{\nu }}^{(2)}\)), and then obtain the reversed functional relationship, as given in Equations (12) and (13).

It follows from Equations (2) and (3) under the conventional setup that

$$\begin{aligned} {\varvec{\kappa }}^{(1)}=\mathbf{0},\;\; {\varvec{\gamma }}^{(1)} = {\varvec{\mu }}^{(1)} = {\varvec{\Lambda }}{\varvec{\tau }}^{(1)}+{\varvec{\nu }}^{(1)}, \end{aligned}$$

and

$$\begin{aligned} {\varvec{\gamma }}^{(2)}={\varvec{\gamma }}^{(1)}={\varvec{\mu }}^{(1)}={\varvec{\Lambda }}{\varvec{\tau }}^{(1)}+{\varvec{\nu }}^{(1)}. \end{aligned}$$

By equating at \({\varvec{\mu }}^{(2)}\) and via Equation (3), we also have

$$\begin{aligned} {\varvec{\mu }}^{(2)}={\varvec{\gamma }}^{(2)}+{\varvec{\Lambda }}{\varvec{\kappa }}^{(2)} = {\varvec{\gamma }}^{(1)}+{\varvec{\Lambda }}{\varvec{\kappa }}^{(2)} ={\varvec{\Lambda }}{\varvec{\tau }}^{(2)}+{\varvec{\nu }}^{(2)}. \end{aligned}$$

Thus,

$$\begin{aligned} {\varvec{\Lambda }}({\varvec{\kappa }}^{(2)}-{\varvec{\tau }}^{(2)}) = {\varvec{\nu }}^{(2)}-{\varvec{\gamma }}^{(1)} = {\varvec{\nu }}^{(2)} - {\varvec{\mu }}^{(1)}. \end{aligned}$$
(23)

Notice that the vector \({\varvec{\nu }}^{(2)}\) is in the orthogonal space of that spanned by the columns of \({\varvec{\Lambda }}\), multiplying the left and right sides of (23) by \({\varvec{\Lambda }}'\) yields

$$\begin{aligned} {\varvec{\Lambda }}'{\varvec{\Lambda }}({\varvec{\kappa }}^{(2)}-{\varvec{\tau }}^{(2)}) = {\varvec{\Lambda }}'({\varvec{\nu }}^{(2)}-{\varvec{\mu }}^{(1)}) =-{\varvec{\Lambda }}'{\varvec{\mu }}^{(1)}. \end{aligned}$$
(24)

Since \({\varvec{\Lambda }}'{\varvec{\Lambda }}\) is non-singular, multiplying the left and right sides of (24) by \(({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1}\) and solving for \({\varvec{\kappa }}^{(2)}\) yield

$$\begin{aligned} {\varvec{\kappa }}^{(2)} = {\varvec{\tau }}^{(2)}-({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1} {\varvec{\Lambda }}'{\varvec{\mu }}^{(1)} = {\varvec{\tau }}^{(2)}-{\varvec{\tau }}^{(1)}. \end{aligned}$$

For the parameterization under the new setup, with \(\mathbf{Q}_{\lambda }\) being defined following Equation (13), we have

$$\begin{aligned} {\varvec{\tau }}^{(1)}= & {} ({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1}{\varvec{\Lambda }}'{\varvec{\mu }}^{(1)} = ({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1}{\varvec{\Lambda }}'{\varvec{\gamma }}^{(1)},\;\; {\varvec{\nu }}^{(1)}=\mathbf{Q}_{\lambda }{\varvec{\mu }}^{(1)}=\mathbf{Q}_{\lambda }{\varvec{\gamma }}^{(1)}, \\ {\varvec{\tau }}^{(2)}= & {} ({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1}{\varvec{\Lambda }}'{\varvec{\mu }}^{(2)} = ({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1}{\varvec{\Lambda }}' ({\varvec{\gamma }}^{(1)} + {\varvec{\Lambda }}{\varvec{\kappa }}^{(2)}) = {\varvec{\kappa }}^{(2)} + ({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1}{\varvec{\Lambda }}' {\varvec{\gamma }}^{(1)}, \end{aligned}$$

and

$$\begin{aligned} {\varvec{\nu }}^{(2)} = \mathbf{Q}_{\lambda }{\varvec{\mu }}^{(2)} = \mathbf{Q}_{\lambda } ({\varvec{\gamma }}^{(1)} + {\varvec{\Lambda }}{\varvec{\tau }}^{(2)})= \mathbf{Q}_{\lambda }{\varvec{\gamma }}^{(1)}. \end{aligned}$$

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Deng, L., Yuan, K. Comparing Latent Means Without Mean Structure Models: A Projection-Based Approach. Psychometrika 81, 802–829 (2016). https://doi.org/10.1007/s11336-015-9491-8

Download citation

Keywords

  • measurement invariance
  • common score
  • specific factor
  • bootstrap
  • chi-square-difference statistic
  • wald statistic