, Volume 78, Issue 2, pp 380–394 | Cite as

Factor Analysis with EM Algorithm Never Gives Improper Solutions when Sample Covariance and Initial Parameter Matrices Are Proper

  • Kohei AdachiEmail author


Rubin and Thayer (Psychometrika, 47:69–76, 1982) proposed the EM algorithm for exploratory and confirmatory maximum likelihood factor analysis. In this paper, we prove the following fact: the EM algorithm always gives a proper solution with positive unique variances and factor correlations with absolute values that do not exceed one, when the covariance matrix to be analyzed and the initial matrices including unique variances and inter-factor correlations are positive definite. We further numerically demonstrate that the EM algorithm yields proper solutions for the data which lead the prevailing gradient algorithms for factor analysis to produce improper solutions. The numerical studies also show that, in real computations with limited numerical precision, Rubin and Thayer’s (Psychometrika, 47:69–76, 1982) original formulas for confirmatory factor analysis can make factor correlation matrices asymmetric, so that the EM algorithm fails to converge. However, this problem can be overcome by using an EM algorithm in which the original formulas are replaced by those guaranteeing the symmetry of factor correlation matrices, or by formulas used to prove the above fact.

Key words

factor analysis EM algorithm improper solutions maximum likelihood method 


  1. Anderson, J.C., & Gerbing, D.W. (1984). The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis. Psychometrika, 49, 155–173. CrossRefGoogle Scholar
  2. Anderson, T.W., & Rubin, H. (1956). Statistical inference in factor analysis. In J. Neyman (Ed.), Proceedings of the third Berkeley symposium on mathematical statistics and probability (pp. 111–150). Berkeley: University of California Press. Google Scholar
  3. Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B, 39, 1–38. Google Scholar
  4. Gerbing, D.W., & Anderson, J.C. (1987). Improper solutions in the analysis of covariance structures: their interpretability and a comparison of alternative respecifications. Psychometrika, 52, 99–111. CrossRefGoogle Scholar
  5. Jennrich, R.I., & Robinson, S.M. (1969). A Newton–Raphson algorithm for maximum likelihood factor analysis. Psychometrika, 34, 111–123. CrossRefGoogle Scholar
  6. Jöreskog, K.G. (1967). Some contributions to maximum likelihood factor analysis. Psychometrika, 32, 443–482. CrossRefGoogle Scholar
  7. Jöreskog, K.G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183–202. CrossRefGoogle Scholar
  8. Kano, Y. (1998). Causes and treatment of improper solutions: exploratory factor analysis. Bulletin of the Department of Human Sciences, Osaka University, 24, 303–327 (in Japanese). Google Scholar
  9. Kuroda, M., & Sakakihara, M. (2006). Accelerating the convergence of the EM algorithm using the vector epsilon algorithm. Computational Statistics & Data Analysis, 51, 1549–1561. CrossRefGoogle Scholar
  10. Lütkepohl, H. (1996). Handbook of matrices. Chichester: Wiley. Google Scholar
  11. Magnus, J.R., & Neudecker, H. (1991). Matrix differential calculus with applications in statistics and econometrics (2nd ed.). Chichester: Wiley. Google Scholar
  12. Maxwell, A.E. (1961). Recent trends in factor analysis. Journal of the Royal Statistical Society. Series A, 124, 49–59. CrossRefGoogle Scholar
  13. McDonald, R.P., & Hartmann, W.M. (1992). A procedure for obtaining initial values of parameters in the RAM model. Multivariate Behavioral Research, 27, 57–76. CrossRefGoogle Scholar
  14. Minami, M. (2004). Convergence speed and acceleration of the EM algorithm. In M. Watanabe & K. Yamaguchi (Eds.), The EM algorithm and related statistical models (pp. 85–94). New York: Marcel Dekker Google Scholar
  15. Mulaik, S.A. (2010). Foundations of factor analysis (2nd ed.). Boca Raton: CRC Press. Google Scholar
  16. Rubin, D.B., & Thayer, D.T. (1982). EM algorithms for ML factor analysis. Psychometrika, 47, 69–76. CrossRefGoogle Scholar
  17. SAS Institute Inc. (2009). SAS/STAT ® 9.2 users guide, version (2nd ed.). Cary: SAS Institute Inc. Google Scholar
  18. Sato, M. (1987). Pragmatic treatment of improper solutions in factor analysis. Annals of the Institute of Statistical Mathematics, 39, 443–455. CrossRefGoogle Scholar
  19. Savalei, V., & Kolenikov, S. (2008). Constrained versus unconstrained estimation in structural equation modeling. Psychological Methods, 13, 150–170. PubMedCrossRefGoogle Scholar
  20. Seber, G.A.F. (2008). A matrix handbook for statisticians. Hoboken: Wiley. Google Scholar
  21. SPSS Inc. (1997). SPSS 7.5 statistical algorithm. Chicago: SPSS Inc. Google Scholar
  22. Van Driel, O.P. (1978). On various causes of improper solutions in maximum likelihood factor analysis. Psychometrika, 43, 225–243. CrossRefGoogle Scholar
  23. Yanai, H., & Ichikawa, M. (2007). Factor analysis. In C.R. Rao & S. Sinharay (Eds.), Handbook of statistics: Vol. 26. Psychometrics (pp. 257–296). Amsterdam: Elsevier. CrossRefGoogle Scholar

Copyright information

© The Psychometric Society 2012

Authors and Affiliations

  1. 1.Graduate School of Human SciencesOsaka UniversityOsakaJapan

Personalised recommendations