, Volume 79, Issue 3, pp 426–443 | Cite as

Three-Mode Factor Analysis by Means of Candecomp/Parafac

  • Alwin Stegeman
  • Tam T. T. Lam


A three-mode covariance matrix contains covariances of N observations (e.g., subject scores) on J variables for K different occasions or conditions. We model such an JK×JK covariance matrix as the sum of a (common) covariance matrix having Candecomp/Parafac form, and a diagonal matrix of unique variances. The Candecomp/Parafac form is a generalization of the two-mode case under the assumption of parallel factors. We estimate the unique variances by Minimum Rank Factor Analysis. The factors can be chosen oblique or orthogonal. Our approach yields a model that is easy to estimate and easy to interpret. Moreover, the unique variances, the factor covariance matrix, and the communalities are guaranteed to be proper, a percentage of explained common variance can be obtained for each variable-condition combination, and the estimated model is rotationally unique under mild conditions. We apply our model to several datasets in the literature, and demonstrate our estimation procedure in a simulation study.

Key words

three-mode factor analysis multitrait-multimethod Candecomp Parafac minimum rank factor analysis 



Research is supported by the Dutch Organisation for Scientific Research (NWO), VIDI grant 452-08-001.

The authors would like to thank the anonymous reviewers for their helpful comments, and Katherine Stroebe (Department of Social Psychology, University of Groningen) for providing the belief in a just world dataset.


  1. Acar, E., & Yener, B. (2009). Unsupervised multiway data analysis: a literature survey. IEEE Transactions on Knowledge and Data Engineering, 21, 1–15. CrossRefGoogle Scholar
  2. Bentler, P.M., & Lee, S.-Y. (1978). Statistical aspects of a three-mode factor analysis model. Psychometrika, 43, 343–352. CrossRefGoogle Scholar
  3. Bentler, P.M., & Lee, S.-Y. (1979). A statistical development of three-mode factor analysis. British Journal of Mathematical and Statistical Psychology, 32, 87–104. CrossRefGoogle Scholar
  4. Bloxom, B. (1968). A note on invariance in three-mode factor analysis. Psychometrika, 33, 347–350. PubMedCrossRefGoogle Scholar
  5. Bro, R., Harshman, R.A., Sidiropoulos, N.D., & Lundy, M.E. (2009). Modeling multi-way data with linearly dependent loadings. Journal of Chemometrics, 23, 324–340. CrossRefGoogle Scholar
  6. Browne, M.W. (1984). The decomposition of multitrait-multimethod matrices. British Journal of Mathematical and Statistical Psychology, 37, 1–21. PubMedCrossRefGoogle Scholar
  7. Campbell, D.T., & Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105. PubMedCrossRefGoogle Scholar
  8. Carroll, J.D., & Chang, J.J. (1970). Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart–Young decomposition. Psychometrika, 35, 283–319. CrossRefGoogle Scholar
  9. Ceulemans, E., & Kiers, H.A.L. (2006). Selecting among three-mode principal component models of different types and complexities: a numerical convex hull based method. British Journal of Mathematical and Statistical Psychology, 59, 133–150. PubMedCrossRefGoogle Scholar
  10. Comon, P., & De Lathauwer, L. (2010). Algebraic identification of under-determined mixtures. In P. Comon & C. Jutten (Eds.), Handbook of blind source separation: independent component analysis and applications (pp. 325–366). San Diego: Academic Press. CrossRefGoogle Scholar
  11. De Almeida, A.L.F., Favier, G., & Mota, J.C.M. (2008a). Constrained tensor modeling approach to blind multiple-antenna CDMA schemes. IEEE Transactions on Signal Processing, 56, 2417–2428. CrossRefGoogle Scholar
  12. De Almeida, A.L.F., Favier, G., & Mota, J.C.M. (2008b). A constrained factor decomposition with application to MIMO antenna systems. IEEE Transactions on Signal Processing, 56, 2429–2442. CrossRefGoogle Scholar
  13. De Lathauwer, L. (2010). Algebraic methods after prewhitening. In P. Comon & C. Jutten (Eds.), Handbook of blind source separation: independent component analysis and applications (pp. 155–178). San Diego: Academic Press. CrossRefGoogle Scholar
  14. De Silva, V., & Lim, L.-H. (2008). Tensor rank and the ill-posedness of the best low-rank approximation problem. SIAM Journal on Matrix Analysis and Applications, 30, 1084–1127. CrossRefGoogle Scholar
  15. Eckart, C., & Young, G. (1936). The approximation of one matrix by another of lower rank. Psychometrika, 1, 211–218. CrossRefGoogle Scholar
  16. Eid, M. (2000). A multitrait-multimethod model with minimal assumptions. Psychometrika, 65, 241–261. CrossRefGoogle Scholar
  17. Eid, M., Nussbeck, F.W., Geiser, C., Cole, D.A., Gollwitzer, M., & Lischetzke, T. (2008). Structural equation modeling of multitrait-multimethod data: different models for different types of methods. Psychological Methods, 13, 230–253. PubMedCrossRefGoogle Scholar
  18. Harshman, R.A. (1970). Foundations of the Parafac procedure: models and conditions for an “explanatory” multimodal factor analysis. UCLA Working Papers in Phonetics, 16, 1–84. Google Scholar
  19. Harshman, R.A., & Lundy, M.E. (1984). Data preprocessing and the extended Parafac model. In H.G. Law, C.W. Snyder Jr., J.A. Hattie, & R.P. McDonald (Eds.), Research methods for multimode data analysis (pp. 216–284). New York: Praeger. Google Scholar
  20. Harshman, R.A. (2004). The problem and nature of degenerate solutions or decompositions of 3-way arrays. In Talk at the tensor decompositions workshop, Palo Alto, USA. Google Scholar
  21. Hitchcock, F.L. (1927a). The expression of a tensor or a polyadic as a sum of products. Journal of Mathematics and Physics, 6, 164–189. Google Scholar
  22. Hitchcock, F.L. (1927b). Multiple invariants and generalized rank of a p-way matrix or tensor. Journal of Mathematics and Physics, 7, 39–70. Google Scholar
  23. Jöreskog, K.G. (1970). A general method for analysis of covariance structures. Biometrika, 57, 239–251. CrossRefGoogle Scholar
  24. Jöreskog, K.G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109–133. CrossRefGoogle Scholar
  25. Kiers, H.A.L. (1998a). Three-way SIMPLIMAX for oblique rotation of the three-mode factor analysis core to simple structure. Computational Statistics and Data Analysis, 28, 307–324. CrossRefGoogle Scholar
  26. Kiers, H.A.L. (1998b). Joint orthomax rotation of the core and component matrices resulting from three-mode principal components analysis. Journal of Classification, 15, 245–263. CrossRefGoogle Scholar
  27. Kiers, H.A.L., Kroonenberg, P.M., & Ten Berge, J.M.F. (1992). An efficient algorithm for TUCKALS3 on data with large number of observation units. Psychometrika, 57, 415–422. CrossRefGoogle Scholar
  28. Kiers, H.A.L., Takane, Y., & Ten Berge, J.M.F. (1996). The analysis of multitrait-multimethod matrices via constrained component analysis. Psychometrika, 61, 601–628. CrossRefGoogle Scholar
  29. Kiers, H.A.L., & Van Mechelen, I. (2001). Three-way component analysis: principles and illustrative application. Psychological Methods, 6, 84–110. PubMedCrossRefGoogle Scholar
  30. Kolda, T.G., & Bader, B.W. (2009). Tensor decompositions and applications. SIAM Review, 51, 455–500. CrossRefGoogle Scholar
  31. Krijnen, W.P., Dijkstra, T.K., & Stegeman, A. (2008). On the non-existence of optimal solutions and the occurrence of “degeneracy” in the Candecomp/Parafac model. Psychometrika, 73, 431–439. PubMedCrossRefPubMedCentralGoogle Scholar
  32. Kroonenberg, P.M., & De Leeuw, J. (1980). Principal component analysis of three-mode data by means of alternating least squares algorithms. Psychometrika, 45, 69–97. CrossRefGoogle Scholar
  33. Kroonenberg, P.M., & Oort, F.J. (2003). Three-mode analysis of multimode covariance matrices. British Journal of Mathematical and Statistical Psychology, 56, 305–335. PubMedCrossRefGoogle Scholar
  34. Kroonenberg, P.M. (2008). Wiley series in probability and statistics. Applied multiway data analysis. Hoboken: Wiley. Google Scholar
  35. Kruskal, J.B. (1977). Three-way arrays: rank and uniqueness of trilinear decompositions, with applications to arithmetic complexity and statistics. Linear Algebra and Its Applications, 18, 95–138. CrossRefGoogle Scholar
  36. Kruskal, J.B., Harshman, R.A., & Lundy, M.E. (1989). How 3-MFA data can cause degenerate Parafac solutions, among other relationships. In R. Coppi & S. Bolasco (Eds.), Multiway data analysis (pp. 115–121). Amsterdam: North-Holland. Google Scholar
  37. Lipkus, I.M., Dalbert, C., & Siegler, I.C. (1996). The importance of distinguishing the belief in a just world for self versus for others: implications for psychological well-being. Personality and Social Psychology Bulletin, 22, 666–677. CrossRefGoogle Scholar
  38. Liu, X., & Sidiropoulos, N.D. (2001). Cramér-Rao lower bounds for low-rank decomposition of multidimensional arrays. IEEE Transactions on Signal Processing, 49, 2074–2086. CrossRefGoogle Scholar
  39. Lorenzo-Seva, U., & Ten Berge, J.M.F. (2006). Tucker’s congruence coefficient as a meaningful index of factor similarity. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 2, 57–64. Google Scholar
  40. Millsap, R.E. (1992). Sufficient conditions for rotational uniqueness in the additive MTMM model. British Journal of Mathematical and Statistical Psychology, 45, 125–138. CrossRefGoogle Scholar
  41. Paatero, P. (2000). Construction and analysis of degenerate Parafac models. Journal of Chemometrics, 14, 285–299. CrossRefGoogle Scholar
  42. Smilde, A., Bro, R., & Geladi, P. (2004). Multi-way analysis: applications in the chemical sciences. Chichester: Wiley. Google Scholar
  43. Sočan, G. (2003). The incremental value of minimum rank factor analysis. Ph.D. Thesis, Department of Psychometrics & Statistics, University of Groningen, Groningen, The Netherlands. Google Scholar
  44. Stegeman, A. (2006). Degeneracy in Candecomp/Parafac explained for p×p×2 arrays of rank p+1 or higher. Psychometrika, 71, 483–501. CrossRefGoogle Scholar
  45. Stegeman, A. (2007). Degeneracy in Candecomp/Parafac explained for several three-sliced arrays with a two-valued typical rank. Psychometrika, 72, 601–619. PubMedCrossRefPubMedCentralGoogle Scholar
  46. Stegeman, A. (2008). Low-rank approximation of generic p×q×2 arrays and diverging components in the Candecomp/Parafac model. SIAM Journal on Matrix Analysis and Applications, 30, 988–1007. CrossRefGoogle Scholar
  47. Stegeman, A. (2009). Using the simultaneous generalized Schur decomposition as a Candecomp/Parafac algorithm for ill-conditioned data. Journal of Chemometrics, 23, 385–392. CrossRefGoogle Scholar
  48. Stegeman, A. (2012). Candecomp/Parafac—from diverging components to a decomposition in block terms. SIAM Journal on Matrix Analysis and Applications, 33, 291–316. CrossRefGoogle Scholar
  49. Stegeman, A., & Sidiropoulos, N.D. (2007). On Kruskal’s uniqueness condition for the Candecomp/Parafac decomposition. Linear Algebra and Its Applications, 420, 540–552. CrossRefGoogle Scholar
  50. Stegeman, A., & de Almeida, A.L.F. (2009). Uniqueness conditions for constrained three-way factor decompositions with linearly dependent loadings. SIAM Journal on Matrix Analysis and Applications, 31, 1469–1490. CrossRefGoogle Scholar
  51. Stegeman, A., & Lam, T.T.T. (2012). Improved uniqueness conditions for canonical tensor decompositions with linearly dependent loadings. SIAM Journal on Matrix Analysis and Applications, 33, 1250–1271. CrossRefGoogle Scholar
  52. Ten Berge, J.M.F., & Kiers, H.A.L. (1991). A numerical approach to the approximate and the exact minimum rank of a covariance matrix. Psychometrika, 56, 309–315. CrossRefGoogle Scholar
  53. Tendeiro, J.N., Ten Berge, J.M.F., & Kiers, H.A.L. (2009). Simplicity transformations for three-way arrays with symmetric slices, and applications to Tucker-3 models with sparse core arrays. Linear Algebra and Its Applications, 430, 924–940. CrossRefGoogle Scholar
  54. Tucker, L.R. (1966). Some mathematical notes on three-mode factor analysis. Psychometrika, 31, 279–311. PubMedCrossRefGoogle Scholar
  55. Widaman, K.F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9, 1–26. CrossRefGoogle Scholar
  56. Wothke, W. (1996). Models for multitrait-multimethod matrix analysis. In G.A. Marcoulides & R.E. Schumacker (Eds.), Advanced structural equation modeling: issues and techniques (pp. 7–56). Mahwah: Erlbaum. Google Scholar

Copyright information

© The Psychometric Society 2013

Authors and Affiliations

  1. 1.Heymans Institute for Psychological ResearchUniversity of GroningenGroningenThe Netherlands

Personalised recommendations