, Volume 84, Issue 4, pp 1048–1067 | Cite as

Clustered Common Factor Exploration in Factor Analysis

  • Kohei UnoEmail author
  • Kohei Adachi
  • Nickolay T. Trendafilov


The factor analysis (FA) model does not permit unique estimation of the common and unique factor scores. This weakness is notorious as the factor indeterminacy in FA. Luckily, some part of the factor scores can be uniquely determined. Thus, as a whole, they can be viewed as a sum of determined and undetermined parts. The paper proposes to select the undetermined part, such that the resulting common factor scores have the following feature: the rows (i.e., individuals) of the common factor score matrix are as well classified as possible into few clusters. The clear benefit is that we can easily interpret the factor scores simply by focusing on the clusters. The procedure is called clustered common factor exploration (CCFE). An alternating least squares algorithm is developed for CCFE. It is illustrated with real data examples. The proposed approach can be viewed as a parallel to the rotation techniques in FA. They exploit another FA indeterminacy, the rotation indeterminacy, which is resolved by choosing the rotation that transforms the loading matrix into the ‘most’ interpretable one according to a pre-specified criterion. In contrast to the rotational indeterminacy, the factor indeterminacy is utilized to achieve well-clustered factor scores by CCFE. To the best of our knowledge, such an approach to the FA interpretation has not been studied yet.


exploratory factor analysis factor indeterminacy clustered common factor scores factor identification matrix decomposition solution 



This study was funded by Japan Society for the Promotion of Science (Grant No. (C)- 26330039).


  1. Adachi, K. (2012). Some contributions to data-fitting factor analysis with empirical comparisons to covariance-fitting factor analysis. Journal of the Japanese Society of Computational Statistics, 25, 25–38.CrossRefGoogle Scholar
  2. Adachi, K. (2016). Matrix-based introduction to multivariate data analysis. Singapore: Springer.CrossRefGoogle Scholar
  3. Adachi, K., & Trendafilov, N. T. (2018). Some mathematical properties of the matrix decomposition solution in factor analysis. Psychometrika, 83, 407–424.CrossRefGoogle Scholar
  4. Anderberg, M. (1973). Cluster analysis for applications. New York: Academic Press.Google Scholar
  5. Anderson, T. W., & Rubin, H. (1956). Statistical inference in factor analysis. In J. Neyman (Ed.), Proceedings of the third Berkeley Symposium on Mathematical Statistics and Probability., vol. 5. (pp. 111-150). Berkeley, CA: University of California Press.Google Scholar
  6. Bartlett, M. S. (1937). The statistical conception of mental factors. British Journal of Psychology, 28, 97–104.Google Scholar
  7. Bezdek, J. C. (1980). A convergence theorem for the fuzzy ISODATA clustering algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2, 1–8.CrossRefGoogle Scholar
  8. Browne, M. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Researches, 36, 111–150.CrossRefGoogle Scholar
  9. de Leeuw, J. (2004). Least squares optimal scaling of partially observed linear systems. In K. van Montfort, J. Oud, & A. Satorra (Eds.), Recent Developments of Structural Equation Models: Theory and Applications (pp. 121–134). Dordrecht: Kluwer Academic.CrossRefGoogle Scholar
  10. Calinski, R. B., & Harabasz, J. (1974). A dendrite method for cluster analysis. Communications in Statistics, 3, 1–27.Google Scholar
  11. Committee for Guiding Psychological Experiments. (1985). Experiments and tests: The foundations of psychology (Explanation version). Tokyo: Bifukan. (in Japanese).Google Scholar
  12. Gan, G., Ma, C., & Wu, J. (2007). Data clustering: Theory, algorithms, and applications. Philadelphia, PA: Society of Industrial and Applied Mathematics (SIAM).CrossRefGoogle Scholar
  13. Guttman, L. (1955). The determinacy of factor score matrices with implications for five other basic problems of common factor theory. British Journal of Statistical Psychology, 8, 65–81.CrossRefGoogle Scholar
  14. Guttman, L. (1988). ETA, DISCO, ODISCO, and $F$. Psychometrika, 53, 393–405.CrossRefGoogle Scholar
  15. Harman, H. H. (1976). Modern factor analysis (3rd ed.). Chicago: The University of Chicago Press.Google Scholar
  16. Hastie, T., Tibshirani, R., & Wainwright, M. (2015). Statistical learning with sparsity: The lasso and generalizations. Boca Raton: CRC Press.CrossRefGoogle Scholar
  17. Hubert, L., & Arabie, P. (1985). Comparing Partitions. Journal of Classification, 2, 193–218.CrossRefGoogle Scholar
  18. Izenman, A. J. (2008). Modern multivariate statistical techniques. New York, NY: Springer.CrossRefGoogle Scholar
  19. Krijnen, W. P., Wansbeek, T. J., & ten Berge, J. M. F. (1996). Best linear predictors for factor scores. Communications in Statistics: Theory and Methods, 25, 3013–3025.CrossRefGoogle Scholar
  20. Lütkepohl, H. (1996). Handbook of matrices. Chichester: Wiley.Google Scholar
  21. MacQueen, J. B. (1967). Some methods for classification and analysis of multivariate observations. Proceedings of the 5th Berkeley Symposium, 1, 281–297.Google Scholar
  22. Milligan, G. W., & Cooper, M. C. (1985). An examination of procedures for determining the number of clusters in a data set. Psychometrika, 50, 159–179.CrossRefGoogle Scholar
  23. Mulaik, S. A. (1976). Comments on “The measurement of factorial indeterminacy”. Psychometrika, 41, 249–262.CrossRefGoogle Scholar
  24. Mulaik, S. A. (2010). Foundations of factor analysis (2nd ed.). Boca Raton: CRC Press.Google Scholar
  25. Sočan, G. (2003). The incremental value of minimum rank factor analysis. Ph.D Thesis, University of Groningen: Groningen.Google Scholar
  26. Stegeman, A. (2016). A new method for simultaneous estimation of the factor model parameters, factor scores, and unique parts. Computational Statistics and Data Analysis, 99, 189–203.CrossRefGoogle Scholar
  27. ten Berge, J. M. F. (1983). A generalization of Kristof’s theorem on the trace of certain matrix products. Psychometrika, 48, 519–523.CrossRefGoogle Scholar
  28. ten Berge, J. M. F. (1993). Least squares optimization in multivariate analysis. Leiden: DSWO Press.Google Scholar
  29. Thurstone, L. L. (1935). Vector of mind. Chicago: The University of Chicago Press.Google Scholar
  30. Trendafilov, N. T., & Unkel, S. (2011). Exploratory factor analysis of data matrices with more variables than observations. Journal of Computational and Graphical Statistics, 20, 874–891.CrossRefGoogle Scholar
  31. Unkel, S., & Trendafilov, N. T. (2010). Simultaneous parameter estimation in exploratory factor analysis: An expository review. International Statistical Review, 78, 363–382.CrossRefGoogle Scholar

Copyright information

© The Psychometric Society 2019

Authors and Affiliations

  1. 1.Graduate School of Human SciencesOsaka UniversityOsakaJapan
  2. 2.Open UniversityMilton KeynesUK

Personalised recommendations