Skip to main content

Advertisement

Log in

Comparing Latent Means Without Mean Structure Models: A Projection-Based Approach

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

The conventional setup for multi-group structural equation modeling requires a stringent condition of cross-group equality of intercepts before mean comparison with latent variables can be conducted. This article proposes a new setup that allows mean comparison without the need to estimate any mean structural model. By projecting the observed sample means onto the space of the common scores and the space orthogonal to that of the common scores, the new setup allows identifying and estimating the means of the common and specific factors, although, without replicate measures, variances of specific factors cannot be distinguished from those of measurement errors. Under the new setup, testing cross-group mean differences of the common scores is done independently from that of the specific factors. Such independent testing eliminates the requirement for cross-group equality of intercepts by the conventional setup in order to test cross-group equality of means of latent variables using chi-square-difference statistics. The most appealing piece of the new setup is a validity index for mean differences, defined as the percentage of the sum of the squared observed mean differences that is due to that of the mean differences of the common scores. By analyzing real data with two groups, the new setup is shown to offer more information than what is obtained under the conventional setup.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Using multiple starting values may increase the chance of convergence. However, such an effort may not work well in practice because non-convergences were common in simulation studies even when the true population values are used as starting values (Anderson & Gerbing, 1984; Hu, Bentler, & Kano, 1992; Jackson, 2001).

  2. One might think that the new method is equivalent to mean comparison via the commonly used formulas of factor scores, we have an example in the next section showing that this is not the case.

  3. Either specific factors or measurement errors do not correlate among themselves or with each other in theory. However, non-random measurement errors due to method effects or other hard-to-control factors do occur in practice (see e.g., Byrne, 1989). Thus, the covariance matrices \({\varvec{\Psi }}^{(j)}\) (\(j=1\), 2) do not have to be diagonal although they typically are.

  4. The identification issue also occurs with the conventional setup and is closely related to latent variable modeling, where there is no unique way to identify the scale of each latent variable. Any rule of identification needs to make mean comparison with latent variables substantively meaningful.

  5. The definition of \(\rho _{\tau }^2\) resembles that of the reliability of measurement. But both the denominator and numerator in (11) are measures of mean differences rather than variances. Also, we may use \(\rho _{\tau }\) as a validity index rather than \(\rho _{\tau }^2\). This simply makes the value greater (\(\rho _{\tau }>\rho _{\tau }^2\)), but \(\rho _{\tau }\) does not possess the desired meaning of “the percentage of the mean differences of the manifest variables that is due to the differences in means of the common scores.”

  6. The statistic \(T_{\mu }\) is calculated the same way as how \(T_{\nu }\) is evaluated. But the matrix \({\hat{\mathbf{H}}}\) in (19), including the one in the formulation of \({\hat{{\varvec{\Pi }}}}_{\nu }\), is set as an identity matrix.

  7. When \(H_{\mu }\): \({\varvec{\mu }}^{(1)}={\varvec{\mu }}^{(2)}\) holds or when \({\bar{\mathbf{y}}}^{(d)}\) is not statistically significant, the occurrence of statistically significant z-scores corresponding to \({\hat{\mu }}_{\tau }^{(d)}\) or \({\hat{\nu }}^{(d)}\) is simply due to type I error. When only a subset of \({\bar{\mathbf{y}}}^{(d)}\) is not statistically significant, the occurrence of statistically significant z-scores corresponding to \({\hat{\mu }}_{\tau }^{(d)}\) or \({\hat{\nu }}^{(d)}\) may not be due to sampling error.

References

  • Anderson, J. C., & Gerbing, D. W. (1984). The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis. Psychometrika, 49, 155–173.

    Article  Google Scholar 

  • Bentler, P. M. (2006). EQS 6 structural equations program manual. Encino, CA: Multivariate Software.

    Google Scholar 

  • Bentler, P. M., Lee, S.-Y., & Weng, L.-J. (1987). Multiple population covariance structure analysis under arbitrary distribution theory. Communication in Statistics-Theory and Method, 16, 1951–1964.

    Article  Google Scholar 

  • Bentler, P. M., & Yuan, K.-H. (2000). On adding a mean structure to a covariance structure model. Educational and Psychological Measurement, 60, 326–339.

    Article  Google Scholar 

  • Beran, R., & Srivastava, M. S. (1985). Bootstrap tests and confidence regions for functions of a covariance matrix. Annals of Statistics, 13, 95–115.

    Article  Google Scholar 

  • Bollen, K. A., & Stine, R. (1992). Bootstrapping goodness of fit measures in structural equation models. Sociological Methods & Research, 21, 205–229.

    Article  Google Scholar 

  • Browne, M. W. (1984). Asymptotic distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.

    Article  PubMed  Google Scholar 

  • Browne, M. W., & Shapiro, A. (2015). Comment on the asymptotics of a distribution-free goodness of fit test statistic. Psychometrika, 80, 196–199.

    Article  PubMed  Google Scholar 

  • Byrne, B. M. (1989). Multigroup comparisons and the assumption of equivalent construct validity across groups: Methodological and substantive issues. Multivariate Behavioral Research, 24, 503–523.

    Article  PubMed  Google Scholar 

  • Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456–466.

    Article  Google Scholar 

  • Ferguson, T. S. (1996). A course in large sample theory. London: Chapman & Hall.

    Book  Google Scholar 

  • Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Hall, P., & Wilson, S. R. (1991). Two guidelines for bootstrap hypothesis testing. Biometrics, 47, 757–762.

    Article  Google Scholar 

  • Hancock, G. R., Stapleton, L. M., & Arnold-Berkovits, I. (2009). The tenuousness of invariance tests within multisample covariance and mean structure models. In T. Teo & M. S. Khine (Eds.), Structural equation modeling: Concepts and applications in educational research (pp. 137–174). Rotterdam, Netherlands: Sense Publishers.

    Google Scholar 

  • Harman, H. H. (1976). Modern factor analysis (3rd ed.). Chicago: The University of Chicago Press.

    Google Scholar 

  • Holzinger, K. J. & Swineford, F. (1939). A study in factoranalysis: The stability of a bi-factor solution. University of Chicago: Supplementary Educational Monographs, No. 48.

  • Horn, J. L., & McArdle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18, 117–144.

    Article  PubMed  Google Scholar 

  • Hu, L. T., Bentler, P. M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted? Psychological Bulletin, 112, 351–362.

    Article  PubMed  Google Scholar 

  • Jackson, D. L. (2001). Sample size and number of parameter estimates in maximum likelihood confirmatory factor analysis: A Monte Carlo investigation. Structural Equation Modeling, 8, 205–223.

    Article  Google Scholar 

  • Jennrich, R. I. (1995). An introduction to computational statistics: Regression analysis. Inglewood Cliffs, NJ: Prentice Hall.

    Google Scholar 

  • Jennrich, R., & Satorra, A. (2013). Continuous orthogonal complement functions and distribution-free goodness of fit tests in moment structure analysis. Psychometrika, 78, 545–552.

    Article  PubMed  Google Scholar 

  • Johnson, E. C., Meade, A. W., & DuVernet, A. M. (2009). The role of referent indicators in tests of measurement invariance. Structural Equation Modeling, 16, 642–657.

    Article  Google Scholar 

  • Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183–202.

    Article  Google Scholar 

  • Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409–426.

    Article  Google Scholar 

  • Kano, Y. (2001). Structural equation modeling for experimental data. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Present and future (pp. 381–402). Lincolnwood, IL: Scientific Software International.

    Google Scholar 

  • Kim, E. S., Kwok, O.-M., & Yoon, M. (2012). Testing factorial invariance in multilevel data: A Monte Carlo study. Structural Equation Modeling, 19, 250–267.

    Article  Google Scholar 

  • Lawley, D. N., & Maxwell, A. E. (1971). Factor analysis as a statistical method (2nd ed.). New York, NY: American Elsevier.

    Google Scholar 

  • Mardia, K. V. (1970). Measure of multivariate skewness and kurtosis with applications. Biometrika, 57, 519–530.

    Article  Google Scholar 

  • Meredith, W. (1993). Measurement invariance, factor analysis, and factorial invariance. Psychometrika, 58, 525–542.

    Article  Google Scholar 

  • Millsap, R. E. (2011). Statistical approaches to measurement invariance. New York: Routledge.

    Google Scholar 

  • Millsap, R. E., & Kwok, O. M. (2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9, 93–115.

    Article  PubMed  Google Scholar 

  • Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552–566.

    Article  PubMed  Google Scholar 

  • Sörbom, D. (1974). A general method for studying differences in factor means and factor structures between groups. British Journal of Mathematical and Statistical Psychology, 27, 229–239.

    Article  Google Scholar 

  • Sörbom, D. (1989). Model modification. Psychometrika, 54, 371–84.

    Article  Google Scholar 

  • Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70.

    Article  Google Scholar 

  • Welch, B. L. (1938). The significance of the difference between two means when the population variances are unequal. Biometrika, 29, 350–362.

    Article  Google Scholar 

  • Yanagihara, H., & Yuan, K.-H. (2005). Three approximate solutions to the multivariate Behrens–Fisher problem. Communications in Statistics: Simulation and Computation, 34, 975–988.

    Article  Google Scholar 

  • Yanai, H., Takeuchi, K., & Takane, Y. (2011). Projection matrices, generalized inverse matrices, and singular value decomposition. New York: Springer.

    Book  Google Scholar 

  • Yuan, K.-H., & Bentler, P. M. (2004). On chi-square difference and \(z\) tests in mean and covariance structure analysis when the base model is misspecified. Educational and Psychological Measurement, 64, 737–757.

    Article  Google Scholar 

  • Yuan, K.-H., Bentler, P. M., & Chan, W. (2004). Structural equation modeling with heavy tailed distributions. Psychometrika, 69, 421–436.

    Article  Google Scholar 

  • Yuan, K.-H., & Hayashi, K. (2003). Bootstrap approach to inference and power analysis based on three statistics for covariance structure models. British Journal of Mathematical and Statistical Psychology, 56, 93–110.

    Article  PubMed  Google Scholar 

  • Yuan, K.-H., Marshall, L. L., & Bentler, P. M. (2003). Assessing the effect of model misspecifications on parameter estimates in structural equation models. Sociological Methodology, 33, 241–265.

    Article  Google Scholar 

  • Zhang, J., & Boos, D. D. (1993). Testing hypotheses about covariance matrices using bootstrap methods. Communications in Statistics, 22, 723–739.

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank Drs. Peter Bentler and Scott Maxwell for their discussion and suggestions in the process of writing this article. Thanks also go to Dr. Alberto Maydeu-Olivares, an associate editor and three reviewers for comments on the previous version. The research was supported in part by the National Science Foundation under Grant No. SES-1461355.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ke-Hai Yuan.

Appendix

Appendix

In this appendix, we will first obtain the parameters under the conventional setup (i.e., \({\varvec{\kappa }}^{(1)}\), \({\varvec{\gamma }}^{(1)}\), \({\varvec{\kappa }}^{(2)}\), \({\varvec{\gamma }}^{(2)}\)) as functions of those under the new setup (i.e., \({\varvec{\tau }}^{(1)}\), \({\varvec{\nu }}^{(1)}\), \({\varvec{\tau }}^{(2)}\), \({\varvec{\nu }}^{(2)}\)), and then obtain the reversed functional relationship, as given in Equations (12) and (13).

It follows from Equations (2) and (3) under the conventional setup that

$$\begin{aligned} {\varvec{\kappa }}^{(1)}=\mathbf{0},\;\; {\varvec{\gamma }}^{(1)} = {\varvec{\mu }}^{(1)} = {\varvec{\Lambda }}{\varvec{\tau }}^{(1)}+{\varvec{\nu }}^{(1)}, \end{aligned}$$

and

$$\begin{aligned} {\varvec{\gamma }}^{(2)}={\varvec{\gamma }}^{(1)}={\varvec{\mu }}^{(1)}={\varvec{\Lambda }}{\varvec{\tau }}^{(1)}+{\varvec{\nu }}^{(1)}. \end{aligned}$$

By equating at \({\varvec{\mu }}^{(2)}\) and via Equation (3), we also have

$$\begin{aligned} {\varvec{\mu }}^{(2)}={\varvec{\gamma }}^{(2)}+{\varvec{\Lambda }}{\varvec{\kappa }}^{(2)} = {\varvec{\gamma }}^{(1)}+{\varvec{\Lambda }}{\varvec{\kappa }}^{(2)} ={\varvec{\Lambda }}{\varvec{\tau }}^{(2)}+{\varvec{\nu }}^{(2)}. \end{aligned}$$

Thus,

$$\begin{aligned} {\varvec{\Lambda }}({\varvec{\kappa }}^{(2)}-{\varvec{\tau }}^{(2)}) = {\varvec{\nu }}^{(2)}-{\varvec{\gamma }}^{(1)} = {\varvec{\nu }}^{(2)} - {\varvec{\mu }}^{(1)}. \end{aligned}$$
(23)

Notice that the vector \({\varvec{\nu }}^{(2)}\) is in the orthogonal space of that spanned by the columns of \({\varvec{\Lambda }}\), multiplying the left and right sides of (23) by \({\varvec{\Lambda }}'\) yields

$$\begin{aligned} {\varvec{\Lambda }}'{\varvec{\Lambda }}({\varvec{\kappa }}^{(2)}-{\varvec{\tau }}^{(2)}) = {\varvec{\Lambda }}'({\varvec{\nu }}^{(2)}-{\varvec{\mu }}^{(1)}) =-{\varvec{\Lambda }}'{\varvec{\mu }}^{(1)}. \end{aligned}$$
(24)

Since \({\varvec{\Lambda }}'{\varvec{\Lambda }}\) is non-singular, multiplying the left and right sides of (24) by \(({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1}\) and solving for \({\varvec{\kappa }}^{(2)}\) yield

$$\begin{aligned} {\varvec{\kappa }}^{(2)} = {\varvec{\tau }}^{(2)}-({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1} {\varvec{\Lambda }}'{\varvec{\mu }}^{(1)} = {\varvec{\tau }}^{(2)}-{\varvec{\tau }}^{(1)}. \end{aligned}$$

For the parameterization under the new setup, with \(\mathbf{Q}_{\lambda }\) being defined following Equation (13), we have

$$\begin{aligned} {\varvec{\tau }}^{(1)}= & {} ({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1}{\varvec{\Lambda }}'{\varvec{\mu }}^{(1)} = ({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1}{\varvec{\Lambda }}'{\varvec{\gamma }}^{(1)},\;\; {\varvec{\nu }}^{(1)}=\mathbf{Q}_{\lambda }{\varvec{\mu }}^{(1)}=\mathbf{Q}_{\lambda }{\varvec{\gamma }}^{(1)}, \\ {\varvec{\tau }}^{(2)}= & {} ({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1}{\varvec{\Lambda }}'{\varvec{\mu }}^{(2)} = ({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1}{\varvec{\Lambda }}' ({\varvec{\gamma }}^{(1)} + {\varvec{\Lambda }}{\varvec{\kappa }}^{(2)}) = {\varvec{\kappa }}^{(2)} + ({\varvec{\Lambda }}'{\varvec{\Lambda }})^{-1}{\varvec{\Lambda }}' {\varvec{\gamma }}^{(1)}, \end{aligned}$$

and

$$\begin{aligned} {\varvec{\nu }}^{(2)} = \mathbf{Q}_{\lambda }{\varvec{\mu }}^{(2)} = \mathbf{Q}_{\lambda } ({\varvec{\gamma }}^{(1)} + {\varvec{\Lambda }}{\varvec{\tau }}^{(2)})= \mathbf{Q}_{\lambda }{\varvec{\gamma }}^{(1)}. \end{aligned}$$

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deng, L., Yuan, KH. Comparing Latent Means Without Mean Structure Models: A Projection-Based Approach. Psychometrika 81, 802–829 (2016). https://doi.org/10.1007/s11336-015-9491-8

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-015-9491-8

Keywords

Navigation