Abstract
Exploratory Factor Analysis (EFA) and Principal Component Analysis (PCA) are popular techniques for simplifying the presentation of, and investigating the structure of, an (n×p) data matrix. However, these fundamentally different techniques are frequently confused, and the differences between them are obscured, because they give similar results in some practical cases. We therefore investigate conditions under which they are expected to be close to each other, by considering EFA as a matrix decomposition so that it can be directly compared with the data matrix decomposition underlying PCA. Correspondingly, we propose an extended version of PCA, called the EFA-like PCA, which mimics the EFA matrix decomposition in the sense that they contain the same unknowns. We provide iterative algorithms for estimating the EFA-like PCA parameters, and derive conditions that have to be satisfied for the two techniques to give similar results. Throughout, we consider separately the cases n>p and p≥n. All derived algorithms and matrix conditions are illustrated on two data sets, one for each of these two cases.
Similar content being viewed by others
References
Berry, M.W., Pulatova, S.A., Stewart, G.W.: Algorithm 844: computing sparse reduced-rank approximations to sparse matrices. ACM Trans. Math. Softw. 31, 252–269 (2005)
De Leeuw, J.: Least squares optimal scaling of partially observed linear systems. In: van Montfort, K., Oud, J., Satorra, A. (eds.) Recent Developments on Structural Equation Models: Theory and Applications, pp. 121–134. Kluwer Academic, Dordrecht (2004)
Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. The John Hopkins University Press, Baltimore (1996)
Harman, H.H.: Modern Factor Analysis, 3rd edn. University of Chicago Press, Chicago (1976)
Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24(6), 417–441 (1933). Continued on 24(7), 498–520
Jolliffe, I.T.: Principal Component Analysis, 2nd edn. Springer, New York (2002)
Magnus, I.R., Neudecker, H.: Matrix Differential Calculus with Applications in Statistics and Econometrics, 3rd edn. Wiley, Chichester (2007)
Mulaik, S.A.: Looking back on the indeterminacy controversies in factor analysis. In: Maydeu-Olivares, A., McArdle, J.J. (eds.) Contemporary Psychometrics: A Festschift for Roderick P. McDonald, pp. 173–206. Lawrence Erlbaum, Mahwah (2005)
Mulaik, S.A.: Foundations of Factor Analysis, 2nd edn. Chapman and Hall/CRC, Boca Raton (2010)
Pearson, K.: On lines and planes of closest fit to systems of points in space. Philos. Mag. 2, 559–572 (1901)
Rao, C.R.: Principal component and factor analyses. In: Maddala, G.S., Rao, C.R. (eds.) Handbook of Statistics, 14, pp. 489–505. Elsevier, Amsterdam (1996)
Schneeweiss, H., Mathes, H.: Factor analysis and principal components. J. Multivar. Anal. 55, 105–124 (1995)
Spearman, C.: “General intelligence,” objectively determined and measured. Am. J. Psychol. 15, 201–292 (1904)
Stewart, G.W.: Matrix Algorithms, Vol I: Basic Decompositions. Philadelphia, SIAM (1998)
Thurstone, L.L.: Multiple Factor Analysis. University of Chicago Press, Chicago (1947)
Trendafilov, N.T.: The dynamical system approach to multivariate data analysis, a review. J. Comput. Graph. Stat. 15, 628–650 (2006)
Trendafilov, N.T., Unkel, S.: Exploratory factor analysis of data matrices with more variables than observations. J. Comput. Graph. Stat. (2011). doi:10.1198/jcgs.2011.0921
Unkel, S., Trendafilov, N.T.: Simultaneous parameter estimation in exploratory factor analysis: an expository review. Int. Stat. Rev. 78, 363–382 (2010)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Trendafilov, N.T., Unkel, S. & Krzanowski, W. Exploratory factor and principal component analyses: some new aspects. Stat Comput 23, 209–220 (2013). https://doi.org/10.1007/s11222-011-9303-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-011-9303-7