Psychometrika

, Volume 78, Issue 2, pp 380–394

# Factor Analysis with EM Algorithm Never Gives Improper Solutions when Sample Covariance and Initial Parameter Matrices Are Proper

Article

## Abstract

Rubin and Thayer (Psychometrika, 47:69–76, 1982) proposed the EM algorithm for exploratory and confirmatory maximum likelihood factor analysis. In this paper, we prove the following fact: the EM algorithm always gives a proper solution with positive unique variances and factor correlations with absolute values that do not exceed one, when the covariance matrix to be analyzed and the initial matrices including unique variances and inter-factor correlations are positive definite. We further numerically demonstrate that the EM algorithm yields proper solutions for the data which lead the prevailing gradient algorithms for factor analysis to produce improper solutions. The numerical studies also show that, in real computations with limited numerical precision, Rubin and Thayer’s (Psychometrika, 47:69–76, 1982) original formulas for confirmatory factor analysis can make factor correlation matrices asymmetric, so that the EM algorithm fails to converge. However, this problem can be overcome by using an EM algorithm in which the original formulas are replaced by those guaranteeing the symmetry of factor correlation matrices, or by formulas used to prove the above fact.

## Key words

factor analysis EM algorithm improper solutions maximum likelihood method

## References

1. Anderson, J.C., & Gerbing, D.W. (1984). The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis. Psychometrika, 49, 155–173.
2. Anderson, T.W., & Rubin, H. (1956). Statistical inference in factor analysis. In J. Neyman (Ed.), Proceedings of the third Berkeley symposium on mathematical statistics and probability (pp. 111–150). Berkeley: University of California Press. Google Scholar
3. Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B, 39, 1–38. Google Scholar
4. Gerbing, D.W., & Anderson, J.C. (1987). Improper solutions in the analysis of covariance structures: their interpretability and a comparison of alternative respecifications. Psychometrika, 52, 99–111.
5. Jennrich, R.I., & Robinson, S.M. (1969). A Newton–Raphson algorithm for maximum likelihood factor analysis. Psychometrika, 34, 111–123.
6. Jöreskog, K.G. (1967). Some contributions to maximum likelihood factor analysis. Psychometrika, 32, 443–482.
7. Jöreskog, K.G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183–202.
8. Kano, Y. (1998). Causes and treatment of improper solutions: exploratory factor analysis. Bulletin of the Department of Human Sciences, Osaka University, 24, 303–327 (in Japanese). Google Scholar
9. Kuroda, M., & Sakakihara, M. (2006). Accelerating the convergence of the EM algorithm using the vector epsilon algorithm. Computational Statistics & Data Analysis, 51, 1549–1561.
10. Lütkepohl, H. (1996). Handbook of matrices. Chichester: Wiley. Google Scholar
11. Magnus, J.R., & Neudecker, H. (1991). Matrix differential calculus with applications in statistics and econometrics (2nd ed.). Chichester: Wiley. Google Scholar
12. Maxwell, A.E. (1961). Recent trends in factor analysis. Journal of the Royal Statistical Society. Series A, 124, 49–59.
13. McDonald, R.P., & Hartmann, W.M. (1992). A procedure for obtaining initial values of parameters in the RAM model. Multivariate Behavioral Research, 27, 57–76.
14. Minami, M. (2004). Convergence speed and acceleration of the EM algorithm. In M. Watanabe & K. Yamaguchi (Eds.), The EM algorithm and related statistical models (pp. 85–94). New York: Marcel Dekker Google Scholar
15. Mulaik, S.A. (2010). Foundations of factor analysis (2nd ed.). Boca Raton: CRC Press. Google Scholar
16. Rubin, D.B., & Thayer, D.T. (1982). EM algorithms for ML factor analysis. Psychometrika, 47, 69–76.
17. SAS Institute Inc. (2009). SAS/STAT ® 9.2 users guide, version (2nd ed.). Cary: SAS Institute Inc. Google Scholar
18. Sato, M. (1987). Pragmatic treatment of improper solutions in factor analysis. Annals of the Institute of Statistical Mathematics, 39, 443–455.
19. Savalei, V., & Kolenikov, S. (2008). Constrained versus unconstrained estimation in structural equation modeling. Psychological Methods, 13, 150–170.
20. Seber, G.A.F. (2008). A matrix handbook for statisticians. Hoboken: Wiley. Google Scholar
21. SPSS Inc. (1997). SPSS 7.5 statistical algorithm. Chicago: SPSS Inc. Google Scholar
22. Van Driel, O.P. (1978). On various causes of improper solutions in maximum likelihood factor analysis. Psychometrika, 43, 225–243.
23. Yanai, H., & Ichikawa, M. (2007). Factor analysis. In C.R. Rao & S. Sinharay (Eds.), Handbook of statistics: Vol. 26. Psychometrics (pp. 257–296). Amsterdam: Elsevier.