Skip to main content
Log in

Exploratory factor and principal component analyses: some new aspects

Statistics and Computing Aims and scope Submit manuscript

Abstract

Exploratory Factor Analysis (EFA) and Principal Component Analysis (PCA) are popular techniques for simplifying the presentation of, and investigating the structure of, an (n×p) data matrix. However, these fundamentally different techniques are frequently confused, and the differences between them are obscured, because they give similar results in some practical cases. We therefore investigate conditions under which they are expected to be close to each other, by considering EFA as a matrix decomposition so that it can be directly compared with the data matrix decomposition underlying PCA. Correspondingly, we propose an extended version of PCA, called the EFA-like PCA, which mimics the EFA matrix decomposition in the sense that they contain the same unknowns. We provide iterative algorithms for estimating the EFA-like PCA parameters, and derive conditions that have to be satisfied for the two techniques to give similar results. Throughout, we consider separately the cases n>p and pn. All derived algorithms and matrix conditions are illustrated on two data sets, one for each of these two cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  • Berry, M.W., Pulatova, S.A., Stewart, G.W.: Algorithm 844: computing sparse reduced-rank approximations to sparse matrices. ACM Trans. Math. Softw. 31, 252–269 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  • De Leeuw, J.: Least squares optimal scaling of partially observed linear systems. In: van Montfort, K., Oud, J., Satorra, A. (eds.) Recent Developments on Structural Equation Models: Theory and Applications, pp. 121–134. Kluwer Academic, Dordrecht (2004)

    Google Scholar 

  • Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. The John Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  • Harman, H.H.: Modern Factor Analysis, 3rd edn. University of Chicago Press, Chicago (1976)

    Google Scholar 

  • Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24(6), 417–441 (1933). Continued on 24(7), 498–520

    Article  Google Scholar 

  • Jolliffe, I.T.: Principal Component Analysis, 2nd edn. Springer, New York (2002)

    MATH  Google Scholar 

  • Magnus, I.R., Neudecker, H.: Matrix Differential Calculus with Applications in Statistics and Econometrics, 3rd edn. Wiley, Chichester (2007)

    Google Scholar 

  • Mulaik, S.A.: Looking back on the indeterminacy controversies in factor analysis. In: Maydeu-Olivares, A., McArdle, J.J. (eds.) Contemporary Psychometrics: A Festschift for Roderick P. McDonald, pp. 173–206. Lawrence Erlbaum, Mahwah (2005)

    Google Scholar 

  • Mulaik, S.A.: Foundations of Factor Analysis, 2nd edn. Chapman and Hall/CRC, Boca Raton (2010)

    MATH  Google Scholar 

  • Pearson, K.: On lines and planes of closest fit to systems of points in space. Philos. Mag. 2, 559–572 (1901)

    Article  Google Scholar 

  • Rao, C.R.: Principal component and factor analyses. In: Maddala, G.S., Rao, C.R. (eds.) Handbook of Statistics, 14, pp. 489–505. Elsevier, Amsterdam (1996)

    Google Scholar 

  • Schneeweiss, H., Mathes, H.: Factor analysis and principal components. J. Multivar. Anal. 55, 105–124 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  • Spearman, C.: “General intelligence,” objectively determined and measured. Am. J. Psychol. 15, 201–292 (1904)

    Article  Google Scholar 

  • Stewart, G.W.: Matrix Algorithms, Vol I: Basic Decompositions. Philadelphia, SIAM (1998)

    Book  MATH  Google Scholar 

  • Thurstone, L.L.: Multiple Factor Analysis. University of Chicago Press, Chicago (1947)

    MATH  Google Scholar 

  • Trendafilov, N.T.: The dynamical system approach to multivariate data analysis, a review. J. Comput. Graph. Stat. 15, 628–650 (2006)

    Article  MathSciNet  Google Scholar 

  • Trendafilov, N.T., Unkel, S.: Exploratory factor analysis of data matrices with more variables than observations. J. Comput. Graph. Stat. (2011). doi:10.1198/jcgs.2011.0921

  • Unkel, S., Trendafilov, N.T.: Simultaneous parameter estimation in exploratory factor analysis: an expository review. Int. Stat. Rev. 78, 363–382 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nickolay T. Trendafilov.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Trendafilov, N.T., Unkel, S. & Krzanowski, W. Exploratory factor and principal component analyses: some new aspects. Stat Comput 23, 209–220 (2013). https://doi.org/10.1007/s11222-011-9303-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-011-9303-7

Keywords

Navigation