# Factor Analysis with EM Algorithm Never Gives Improper Solutions when Sample Covariance and Initial Parameter Matrices Are Proper

- 570 Downloads
- 6 Citations

## Abstract

Rubin and Thayer (Psychometrika, 47:69–76, 1982) proposed the EM algorithm for exploratory and confirmatory maximum likelihood factor analysis. In this paper, we prove the following fact: the EM algorithm always gives a proper solution with positive unique variances and factor correlations with absolute values that do not exceed one, when the covariance matrix to be analyzed and the initial matrices including unique variances and inter-factor correlations are positive definite. We further numerically demonstrate that the EM algorithm yields proper solutions for the data which lead the prevailing gradient algorithms for factor analysis to produce improper solutions. The numerical studies also show that, in real computations with limited numerical precision, Rubin and Thayer’s (Psychometrika, 47:69–76, 1982) original formulas for confirmatory factor analysis can make factor correlation matrices asymmetric, so that the EM algorithm fails to converge. However, this problem can be overcome by using an EM algorithm in which the original formulas are replaced by those guaranteeing the symmetry of factor correlation matrices, or by formulas used to prove the above fact.

## Key words

factor analysis EM algorithm improper solutions maximum likelihood method## References

- Anderson, J.C., & Gerbing, D.W. (1984). The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis.
*Psychometrika*,*49*, 155–173. CrossRefGoogle Scholar - Anderson, T.W., & Rubin, H. (1956). Statistical inference in factor analysis. In J. Neyman (Ed.),
*Proceedings of the third Berkeley symposium on mathematical statistics and probability*(pp. 111–150). Berkeley: University of California Press. Google Scholar - Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm.
*Journal of the Royal Statistical Society. Series B*,*39*, 1–38. Google Scholar - Gerbing, D.W., & Anderson, J.C. (1987). Improper solutions in the analysis of covariance structures: their interpretability and a comparison of alternative respecifications.
*Psychometrika*,*52*, 99–111. CrossRefGoogle Scholar - Jennrich, R.I., & Robinson, S.M. (1969). A Newton–Raphson algorithm for maximum likelihood factor analysis.
*Psychometrika*,*34*, 111–123. CrossRefGoogle Scholar - Jöreskog, K.G. (1967). Some contributions to maximum likelihood factor analysis.
*Psychometrika*,*32*, 443–482. CrossRefGoogle Scholar - Jöreskog, K.G. (1969). A general approach to confirmatory maximum likelihood factor analysis.
*Psychometrika*,*34*, 183–202. CrossRefGoogle Scholar - Kano, Y. (1998). Causes and treatment of improper solutions: exploratory factor analysis.
*Bulletin of the Department of Human Sciences, Osaka University*,*24*, 303–327 (in Japanese). Google Scholar - Kuroda, M., & Sakakihara, M. (2006). Accelerating the convergence of the EM algorithm using the vector epsilon algorithm.
*Computational Statistics & Data Analysis*,*51*, 1549–1561. CrossRefGoogle Scholar - Lütkepohl, H. (1996).
*Handbook of matrices*. Chichester: Wiley. Google Scholar - Magnus, J.R., & Neudecker, H. (1991).
*Matrix differential calculus with applications in statistics and econometrics*(2nd ed.). Chichester: Wiley. Google Scholar - Maxwell, A.E. (1961). Recent trends in factor analysis.
*Journal of the Royal Statistical Society. Series A*,*124*, 49–59. CrossRefGoogle Scholar - McDonald, R.P., & Hartmann, W.M. (1992). A procedure for obtaining initial values of parameters in the RAM model.
*Multivariate Behavioral Research*,*27*, 57–76. CrossRefGoogle Scholar - Minami, M. (2004). Convergence speed and acceleration of the EM algorithm. In M. Watanabe & K. Yamaguchi (Eds.),
*The EM algorithm and related statistical models*(pp. 85–94). New York: Marcel Dekker Google Scholar - Mulaik, S.A. (2010).
*Foundations of factor analysis*(2nd ed.). Boca Raton: CRC Press. Google Scholar - Rubin, D.B., & Thayer, D.T. (1982). EM algorithms for ML factor analysis.
*Psychometrika*,*47*, 69–76. CrossRefGoogle Scholar - SAS Institute Inc. (2009).
*SAS/STAT*^{®}*9.2 users guide*, version (2nd ed.). Cary: SAS Institute Inc. Google Scholar - Sato, M. (1987). Pragmatic treatment of improper solutions in factor analysis.
*Annals of the Institute of Statistical Mathematics*,*39*, 443–455. CrossRefGoogle Scholar - Savalei, V., & Kolenikov, S. (2008). Constrained versus unconstrained estimation in structural equation modeling.
*Psychological Methods*,*13*, 150–170. PubMedCrossRefGoogle Scholar - Seber, G.A.F. (2008).
*A matrix handbook for statisticians*. Hoboken: Wiley. Google Scholar - SPSS Inc. (1997).
*SPSS 7.5 statistical algorithm*. Chicago: SPSS Inc. Google Scholar - Van Driel, O.P. (1978). On various causes of improper solutions in maximum likelihood factor analysis.
*Psychometrika*,*43*, 225–243. CrossRefGoogle Scholar - Yanai, H., & Ichikawa, M. (2007). Factor analysis. In C.R. Rao & S. Sinharay (Eds.),
*Handbook of statistics: Vol.**26*.*Psychometrics*(pp. 257–296). Amsterdam: Elsevier. CrossRefGoogle Scholar