Mathematical Programming

, Volume 151, Issue 1, pp 225–248 | Cite as

Numerical optimization for symmetric tensor decomposition

  • Tamara G. Kolda
Full Length Paper Series B


We consider the problem of decomposing a real-valued symmetric tensor as the sum of outer products of real-valued vectors. Algebraic methods exist for computing complex-valued decompositions of symmetric tensors, but here we focus on real-valued decompositions, both unconstrained and nonnegative, for problems with low-rank structure. We discuss when solutions exist and how to formulate the mathematical program. Numerical results show the properties of the proposed formulations (including one that ignores symmetry) on a set of test problems and illustrate that these straightforward formulations can be effective even though the problem is nonconvex.


Symmetric Outer product Canonical polyadic Tensor decomposition Completely positive Nonnegative 

Mathematics Subject Classification

90C30 15A69 



The anonymous referees provided extremely useful feedback that has greatly improved the manuscript. This material is based upon work supported by the US Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Applied Mathematics program. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the US Department of Energy’s National Nuclear Security Administration under contract DE–AC04–94AL85000.


  1. 1.
    Acar, E., Dunlavy, D.M., Kolda, T.G.: A scalable optimization approach for fitting canonical tensor decompositions. J. Chemom. 25(2), 67–86 (2011). doi: 10.1002/cem.1335 CrossRefGoogle Scholar
  2. 2.
    Acar, E., Dunlavy, D.M., Kolda, T.G., Mørup, M.: Scalable tensor factorizations with missing data. In: SDM10 Proceedings of the 2010 SIAM International Conference on Data Mining, pp. 701–712 (2010). doi: 10.1137/1.9781611972801.61
  3. 3.
    Acar, E., Dunlavy, D.M., Kolda, T.G., Mørup, M.: Scalable tensor factorizations for incomplete data. Chemom. Intell. Lab. Syst. 106(1), 41–56 (2011). doi: 10.1016/j.chemolab.2010.08.004 CrossRefGoogle Scholar
  4. 4.
    Alexander, J., Hirschowitz, A.: Polynomial interpolation in several variables. J. Algebra. Geom. 4, 201–222 (1995)zbMATHMathSciNetGoogle Scholar
  5. 5.
    Austin, W., Kolda, T.G., Plantenga, T.: Tensor rank prediction via cross validation. In preparation (2015)Google Scholar
  6. 6.
    Bader, B.W., Kolda, T.G.: Algorithm 862: MATLAB tensor classes for fast algorithm prototyping. ACM Trans. Math. Soft. 32(4), 635–653 (2006). doi: 10.1145/1186785.1186794 CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Bader, B.W., Kolda, T.G.: Efficient MATLAB computations with sparse and factored tensors. SIAM J. Sci. Comput. 30(1), 205–231 (2007). doi: 10.1137/060676489 CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Bader, B.W., Kolda, T.G., et al.: Matlab tensor toolbox version 2.5. (2012)
  9. 9.
    Ballard, G., Kolda, T.G., Plantenga, T.: Efficiently computing tensor eigenvalues on a GPU. In: IPDPSW’11: Proceedings of the 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Ph.D. Forum, pp. 1340–1348. IEEE Computer Society (2011). doi: 10.1109/IPDPS.2011.287
  10. 10.
    Ballico, E., Bernardi, A.: Decomposition of homogeneous polynomials with low rank. Math. Z. 271(3–4), 1141–1149 (2011). doi: 10.1007/s00209-011-0907-6 MathSciNetGoogle Scholar
  11. 11.
    Bernardi, A., Gimigliano, A., Id, M.: Computing symmetric rank for symmetric tensors. J. Symb. Comput. 46(1), 34–53 (2011). doi: 10.1016/j.jsc.2010.08.001.
  12. 12.
    Brachat, J., Comon, P., Mourrain, B., Tsigaridas, E.: Symmetric tensor decomposition. Linear Algebra Appl. 433(11–12), 1851–1872 (2010). doi: 10.1016/j.laa.2010.06.046.
  13. 13.
    Cambre, J., De Lathauwer, L., De Moor, B.: Best rank (R, R, R) super-symmetric tensor approximation-a continuous-time approach. In: Proceedings of the IEEE 1999 Signal Processing Workshop on Higher-Order Statistics, pp. 242–246 (1999). doi: 10.1109/HOST.1999.778734.
  14. 14.
    Carroll, J.D., Chang, J.J.: Analysis of individual differences in multidimensional scaling via an N-way generalization of “Eckart-Young” decomposition. Psychometrika 35, 283–319 (1970). doi: 10.1007/BF02310791 CrossRefzbMATHGoogle Scholar
  15. 15.
    Comon, P., Golub, G., Lim, L.H., Mourrain, B.: Symmetric tensors and symmetric tensor rank. SIAM J. Matrix Anal. Appl. 30(3), 1254–1279 (2008). doi: 10.1137/060661569 CrossRefMathSciNetGoogle Scholar
  16. 16.
    Cui, C.F., Dai, Y.H., Nie, J.: All real eigenvalues of symmetric tensors, arXiv:1403.3720 (2014)
  17. 17.
    De Lathauwer, L., De Moor, B., Vandewalle, J.: On the best rank-1 and rank-\((R_1, R_2, \cdots, R_N)\) approximation of higher-order tensors. SIAM J. Matrix Anal. Appl. 21(4), 1324–1342 (2000). doi: 10.1137/S0895479898346995 CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Dosse, M.B., Ten Berge, J.M.F.: The assumption of proportional components when CANDECOMP is applied to symmetric matrices in the context of INDSCAL. Psychometrika 73(2), 303–307 (2008). doi: 10.1007/s11336-007-9044-x.
  19. 19.
    Faber, N.K.M., Bro, R., Hopke, P.K.: Recent developments in CANDECOMP/PARAFAC algorithms: a critical review. Chemom. Intell. Lab. Syst. 65(1), 119–137 (2003). doi: 10.1016/S0169-7439(02)00089-8 CrossRefGoogle Scholar
  20. 20.
    Fan, J., Zhou, A.: Completely positive tensor decomposition, arXiv:1411.5149 (2014)
  21. 21.
    Gill, P.E., Murray, W., Saunders, M.A.: SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM Rev. 47(1), 99–131 (2005). doi: 10.1137/S0036144504446096 CrossRefzbMATHMathSciNetGoogle Scholar
  22. 22.
    Gill, P.E., Murray, W., Saunders, M.A.: User’s guide for SNOPT version 7: software for large-scale nonlinear programming (2008)Google Scholar
  23. 23.
    Han, L.: An unconstrained optimization approach for finding real eigenvalues of even order symmetric tensors. Numer. Algebra Control Optim. (NACO) 3(3), 583–599 (2012). doi: 10.3934/naco.2013.3.583 CrossRefGoogle Scholar
  24. 24.
    Harshman, R.A.: Foundations of the PARAFAC procedure: Models and conditions for an “explanatory” multi-modal factor analysis. UCLA working papers in phonetics 16, 1–84 (1970).
  25. 25.
    Hitchcock, F.L.: The expression of a tensor or a polyadic as a sum of products. J. Math. Phys. 6(1), 164–189 (1927)zbMATHMathSciNetGoogle Scholar
  26. 26.
    Hitchcock, F.L.: Multilple invariants and generalized rank of a p-way matrix or tensor. J. Math. Phys. 7(1), 39–79 (1927)zbMATHMathSciNetGoogle Scholar
  27. 27.
    Ishteva, M., Absil, P.A., Van Dooren, P.: Jacobi algorithm for the best low multilinear rank approximation of symmetric tensors. SIAM J. Matrix Anal. Appl. 34(2), 651–672 (2013). doi: 10.1137/11085743X CrossRefzbMATHMathSciNetGoogle Scholar
  28. 28.
    Kiers, H.A.L.: A three-step algorithm for CANDECOMP/PARAFAC analysis of large data sets with multicollinearity. J. Chemom. 12(3), 255–171 (1998). doi: 10.1002/(SICI)1099-128X(199805/06)12:3<155::AID-CEM502>3.0.CO;2-5
  29. 29.
    Kiers, H.A.L.: Towards a standardized notation and terminology in multiway analysis. J. Chemom. 14(3), 105–122 (2000). doi: 10.1002/1099-128X(200005/06)14:3<105::AID-CEM582>3.0.CO;2-I
  30. 30.
    Kofidis, E., Regalia, P.A.: On the best rank-1 approximation of higher-order supersymmetric tensors. SIAM J. Matrix Anal. Appl. 23(3), 863–884 (2002). doi: 10.1137/S0895479801387413.
  31. 31.
    Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009). doi: 10.1137/07070111X CrossRefzbMATHMathSciNetGoogle Scholar
  32. 32.
    Kolda, T.G., Mayo, J.R.: Shifted power method for computing tensor eigenpairs. SIAM J. Matrix Anal. Appl. 32(4), 1095–1124 (2011). doi: 10.1137/100801482 CrossRefzbMATHMathSciNetGoogle Scholar
  33. 33.
    Kolda, T.G., Mayo, J.R.: An adaptive shifted power method for computing generalized tensor eigenpairs, arXiv:1401.1183 (2014)
  34. 34.
    Lim, L.H.: Singular values and eigenvalues of tensors: A variational approach. In: CAMSAP’05: Proceedings of the IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, pp. 129–132 (2005). doi: 10.1109/CAMAP.2005.1574201
  35. 35.
    Nie, J.: Generating polynomials and symmetric tensor decompositions, arXiv:1408.5664 (2014)
  36. 36.
    Nie, J., Wang, L.: Semidefinite relaxations for best rank-1 tensor approximations. SIAM J. Matrix Anal. Appl. 35(3), 1155–1179 (2014). doi: 10.1137/130935112 CrossRefzbMATHMathSciNetGoogle Scholar
  37. 37.
    Oeding, L., Ottaviani, G.: Eigenvectors of tensors and algorithms for waring decomposition. J. Symb. Comput. 54, 9–35 (2013). doi: 10.1016/j.jsc.2012.11.005 CrossRefzbMATHMathSciNetGoogle Scholar
  38. 38.
    Qi, L.: Eigenvalues of a real supersymmetric tensor. J. Symb. Comput. 40, 1302–1324 (2005). doi: 10.1016/j.jsc.2005.05.007 CrossRefzbMATHGoogle Scholar
  39. 39.
    Qi, L., Xu, C., Xu, Y.: Nonnegative tensor factorization, completely positive tensors and an hierarchical elimination algorithm, arXiv:1305.5344 (2013)
  40. 40.
    Regalia, P.A.: Monotonically convergent algorithms for symmetric tensor approximation. Linear Algebra Appl. 438(2), 875–890 (2013). doi: 10.1016/j.laa.2011.10.033.
  41. 41.
    Regalia, P.A., Kofidis, E.: Monotonic convergence of fixed-point algorithms for ICA. IEEE Trans. Neural Netw. 14(4), 943–949 (2003). doi: 10.1109/TNN.2003.813843 CrossRefGoogle Scholar
  42. 42.
    Schmidt, M., Fung, G., Rosales, R.: Fast optimization methods for L1 regularization: A comparative study and two new approaches. Lecture notes in computer science pp. 286–297 (2007). doi: 10.1007/978-3-540-74958-5_28
  43. 43.
    Sidiropoulos, N.D., Bro, R.: On the uniqueness of multilinear decomposition of N-way arrays. J. Chemom. 14(3), 229–239 (2000). doi: 10.1002/1099-128X(200005/06)14:3<229::AID-CEM587>3.0.CO;2-N
  44. 44.
    Tomasi, G., Bro, R.: A comparison of algorithms for fitting the PARAFAC model. Comput. Stat. Data Anal. 50(7), 1700–1734 (2006). doi: 10.1016/j.csda.2004.11.013 CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2015

Authors and Affiliations

  1. 1.Sandia National LaboratoriesLivermoreUSA

Personalised recommendations