The Goodness of Sample Loadings of Principal Component Analysis in Approximating to Factor Loadings with High Dimensional Data

Abstract

Guttman (Psychometrika 21 273–286:1956) showed that the loadings of factor analysis (FA) and those of principal component analysis (PCA) approach each other as the number of variables p goes to infinity. Because the computation for PCA is simpler than FA, PCA can be used as an approximation for FA when p is large. However, another side of the coin is that as p increases, non-consistency might become an issue. Therefore, it is necessary to simultaneously consider the closeness between the estimated FA and the estimated PCA loadings as well as the closeness between the estimated and the population FA loadings. Using Monte Carlo simulation, this article studies the behavior of three kinds of closeness under high-dimensional conditions: (1) between the estimated FA and the estimated PCA loadings, (2) between the estimated FA and the population FA loadings, and (3) between the estimated PCA and the population FA loadings. To deal with high-dimensionality, a ridge method proposed by Yuan and Chan (Computational Statistics and Data Analysis 52:4842–4828, 2008) is employed. As a measure for closeness, the average canonical correlation (CC) between two loading matrices and its Fisher-z transformation are employed. Results indicate that the Fisher-z transformed average CC between the estimated FA and the estimated PCA loadings is larger than that between the estimated FA and the population FA loadings as well as that between the estimated PCA and the population FA loadings. It is concluded that, under high-dimensional conditions, the closeness between the estimated FA and PCA loadings is easier to achieve than that between the estimated and the population FA loadings and also that between the estimated PCA and the population FA loadings.