Skip to main content
SpringerLink
Log in

Numerical optimization for symmetric tensor decomposition

  • Full Length Paper
  • Series B
  • Published:
Mathematical Programming Submit manuscript

Abstract

We consider the problem of decomposing a real-valued symmetric tensor as the sum of outer products of real-valued vectors. Algebraic methods exist for computing complex-valued decompositions of symmetric tensors, but here we focus on real-valued decompositions, both unconstrained and nonnegative, for problems with low-rank structure. We discuss when solutions exist and how to formulate the mathematical program. Numerical results show the properties of the proposed formulations (including one that ignores symmetry) on a set of test problems and illustrate that these straightforward formulations can be effective even though the problem is nonconvex.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Acar, E., Dunlavy, D.M., Kolda, T.G.: A scalable optimization approach for fitting canonical tensor decompositions. J. Chemom. 25(2), 67–86 (2011). doi:10.1002/cem.1335

    Article  Google Scholar 

  2. Acar, E., Dunlavy, D.M., Kolda, T.G., Mørup, M.: Scalable tensor factorizations with missing data. In: SDM10 Proceedings of the 2010 SIAM International Conference on Data Mining, pp. 701–712 (2010). doi:10.1137/1.9781611972801.61

  3. Acar, E., Dunlavy, D.M., Kolda, T.G., Mørup, M.: Scalable tensor factorizations for incomplete data. Chemom. Intell. Lab. Syst. 106(1), 41–56 (2011). doi:10.1016/j.chemolab.2010.08.004

    Article  Google Scholar 

  4. Alexander, J., Hirschowitz, A.: Polynomial interpolation in several variables. J. Algebra. Geom. 4, 201–222 (1995)

    MATH  MathSciNet  Google Scholar 

  5. Austin, W., Kolda, T.G., Plantenga, T.: Tensor rank prediction via cross validation. In preparation (2015)

  6. Bader, B.W., Kolda, T.G.: Algorithm 862: MATLAB tensor classes for fast algorithm prototyping. ACM Trans. Math. Soft. 32(4), 635–653 (2006). doi:10.1145/1186785.1186794

    Article  MATH  MathSciNet  Google Scholar 

  7. Bader, B.W., Kolda, T.G.: Efficient MATLAB computations with sparse and factored tensors. SIAM J. Sci. Comput. 30(1), 205–231 (2007). doi:10.1137/060676489

    Article  MATH  MathSciNet  Google Scholar 

  8. Bader, B.W., Kolda, T.G., et al.: Matlab tensor toolbox version 2.5. http://www.sandia.gov/~tgkolda/TensorToolbox/ (2012)

  9. Ballard, G., Kolda, T.G., Plantenga, T.: Efficiently computing tensor eigenvalues on a GPU. In: IPDPSW’11: Proceedings of the 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Ph.D. Forum, pp. 1340–1348. IEEE Computer Society (2011). doi:10.1109/IPDPS.2011.287

  10. Ballico, E., Bernardi, A.: Decomposition of homogeneous polynomials with low rank. Math. Z. 271(3–4), 1141–1149 (2011). doi:10.1007/s00209-011-0907-6

    MathSciNet  Google Scholar 

  11. Bernardi, A., Gimigliano, A., Id, M.: Computing symmetric rank for symmetric tensors. J. Symb. Comput. 46(1), 34–53 (2011). doi:10.1016/j.jsc.2010.08.001. http://www.sciencedirect.com/science/article/pii/S0747717110001240

  12. Brachat, J., Comon, P., Mourrain, B., Tsigaridas, E.: Symmetric tensor decomposition. Linear Algebra Appl. 433(11–12), 1851–1872 (2010). doi:10.1016/j.laa.2010.06.046. http://www.sciencedirect.com/science/article/B6V0R-50M0TVJ-2/2/060f24da5301d406f2c2504cce6fff9e

  13. Cambre, J., De Lathauwer, L., De Moor, B.: Best rank (R, R, R) super-symmetric tensor approximation-a continuous-time approach. In: Proceedings of the IEEE 1999 Signal Processing Workshop on Higher-Order Statistics, pp. 242–246 (1999). doi:10.1109/HOST.1999.778734. http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=778734

  14. Carroll, J.D., Chang, J.J.: Analysis of individual differences in multidimensional scaling via an N-way generalization of “Eckart-Young” decomposition. Psychometrika 35, 283–319 (1970). doi:10.1007/BF02310791

    Article  MATH  Google Scholar 

  15. Comon, P., Golub, G., Lim, L.H., Mourrain, B.: Symmetric tensors and symmetric tensor rank. SIAM J. Matrix Anal. Appl. 30(3), 1254–1279 (2008). doi:10.1137/060661569

    Article  MathSciNet  Google Scholar 

  16. Cui, C.F., Dai, Y.H., Nie, J.: All real eigenvalues of symmetric tensors, arXiv:1403.3720 (2014)

  17. De Lathauwer, L., De Moor, B., Vandewalle, J.: On the best rank-1 and rank-\((R_1, R_2, \cdots, R_N)\) approximation of higher-order tensors. SIAM J. Matrix Anal. Appl. 21(4), 1324–1342 (2000). doi:10.1137/S0895479898346995

    Article  MATH  MathSciNet  Google Scholar 

  18. Dosse, M.B., Ten Berge, J.M.F.: The assumption of proportional components when CANDECOMP is applied to symmetric matrices in the context of INDSCAL. Psychometrika 73(2), 303–307 (2008). doi:10.1007/s11336-007-9044-x. http://www.springerlink.com/content/l683x3734320025h/fulltext.pdf

  19. Faber, N.K.M., Bro, R., Hopke, P.K.: Recent developments in CANDECOMP/PARAFAC algorithms: a critical review. Chemom. Intell. Lab. Syst. 65(1), 119–137 (2003). doi:10.1016/S0169-7439(02)00089-8

    Article  Google Scholar 

  20. Fan, J., Zhou, A.: Completely positive tensor decomposition, arXiv:1411.5149 (2014)

  21. Gill, P.E., Murray, W., Saunders, M.A.: SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM Rev. 47(1), 99–131 (2005). doi:10.1137/S0036144504446096

    Article  MATH  MathSciNet  Google Scholar 

  22. Gill, P.E., Murray, W., Saunders, M.A.: User’s guide for SNOPT version 7: software for large-scale nonlinear programming (2008)

  23. Han, L.: An unconstrained optimization approach for finding real eigenvalues of even order symmetric tensors. Numer. Algebra Control Optim. (NACO) 3(3), 583–599 (2012). doi:10.3934/naco.2013.3.583

    Article  Google Scholar 

  24. Harshman, R.A.: Foundations of the PARAFAC procedure: Models and conditions for an “explanatory” multi-modal factor analysis. UCLA working papers in phonetics 16, 1–84 (1970). http://www.psychology.uwo.ca/faculty/harshman/wpppfac0.pdf

  25. Hitchcock, F.L.: The expression of a tensor or a polyadic as a sum of products. J. Math. Phys. 6(1), 164–189 (1927)

    MATH  MathSciNet  Google Scholar 

  26. Hitchcock, F.L.: Multilple invariants and generalized rank of a p-way matrix or tensor. J. Math. Phys. 7(1), 39–79 (1927)

    MATH  MathSciNet  Google Scholar 

  27. Ishteva, M., Absil, P.A., Van Dooren, P.: Jacobi algorithm for the best low multilinear rank approximation of symmetric tensors. SIAM J. Matrix Anal. Appl. 34(2), 651–672 (2013). doi:10.1137/11085743X

    Article  MATH  MathSciNet  Google Scholar 

  28. Kiers, H.A.L.: A three-step algorithm for CANDECOMP/PARAFAC analysis of large data sets with multicollinearity. J. Chemom. 12(3), 255–171 (1998). doi:10.1002/(SICI)1099-128X(199805/06)12:3<155::AID-CEM502>3.0.CO;2-5

  29. Kiers, H.A.L.: Towards a standardized notation and terminology in multiway analysis. J. Chemom. 14(3), 105–122 (2000). doi:10.1002/1099-128X(200005/06)14:3<105::AID-CEM582>3.0.CO;2-I

  30. Kofidis, E., Regalia, P.A.: On the best rank-1 approximation of higher-order supersymmetric tensors. SIAM J. Matrix Anal. Appl. 23(3), 863–884 (2002). doi:10.1137/S0895479801387413. http://link.aip.org/link/?SML/23/863/1

  31. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009). doi:10.1137/07070111X

    Article  MATH  MathSciNet  Google Scholar 

  32. Kolda, T.G., Mayo, J.R.: Shifted power method for computing tensor eigenpairs. SIAM J. Matrix Anal. Appl. 32(4), 1095–1124 (2011). doi:10.1137/100801482

    Article  MATH  MathSciNet  Google Scholar 

  33. Kolda, T.G., Mayo, J.R.: An adaptive shifted power method for computing generalized tensor eigenpairs, arXiv:1401.1183 (2014)

  34. Lim, L.H.: Singular values and eigenvalues of tensors: A variational approach. In: CAMSAP’05: Proceedings of the IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, pp. 129–132 (2005). doi:10.1109/CAMAP.2005.1574201

  35. Nie, J.: Generating polynomials and symmetric tensor decompositions, arXiv:1408.5664 (2014)

  36. Nie, J., Wang, L.: Semidefinite relaxations for best rank-1 tensor approximations. SIAM J. Matrix Anal. Appl. 35(3), 1155–1179 (2014). doi:10.1137/130935112

    Article  MATH  MathSciNet  Google Scholar 

  37. Oeding, L., Ottaviani, G.: Eigenvectors of tensors and algorithms for waring decomposition. J. Symb. Comput. 54, 9–35 (2013). doi:10.1016/j.jsc.2012.11.005

    Article  MATH  MathSciNet  Google Scholar 

  38. Qi, L.: Eigenvalues of a real supersymmetric tensor. J. Symb. Comput. 40, 1302–1324 (2005). doi:10.1016/j.jsc.2005.05.007

    Article  MATH  Google Scholar 

  39. Qi, L., Xu, C., Xu, Y.: Nonnegative tensor factorization, completely positive tensors and an hierarchical elimination algorithm, arXiv:1305.5344 (2013)

  40. Regalia, P.A.: Monotonically convergent algorithms for symmetric tensor approximation. Linear Algebra Appl. 438(2), 875–890 (2013). doi:10.1016/j.laa.2011.10.033. http://www.sciencedirect.com/science/article/pii/S0024379511007300

  41. Regalia, P.A., Kofidis, E.: Monotonic convergence of fixed-point algorithms for ICA. IEEE Trans. Neural Netw. 14(4), 943–949 (2003). doi:10.1109/TNN.2003.813843

    Article  Google Scholar 

  42. Schmidt, M., Fung, G., Rosales, R.: Fast optimization methods for L1 regularization: A comparative study and two new approaches. Lecture notes in computer science pp. 286–297 (2007). doi:10.1007/978-3-540-74958-5_28

  43. Sidiropoulos, N.D., Bro, R.: On the uniqueness of multilinear decomposition of N-way arrays. J. Chemom. 14(3), 229–239 (2000). doi:10.1002/1099-128X(200005/06)14:3<229::AID-CEM587>3.0.CO;2-N

  44. Tomasi, G., Bro, R.: A comparison of algorithms for fitting the PARAFAC model. Comput. Stat. Data Anal. 50(7), 1700–1734 (2006). doi:10.1016/j.csda.2004.11.013

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

The anonymous referees provided extremely useful feedback that has greatly improved the manuscript. This material is based upon work supported by the US Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Applied Mathematics program. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the US Department of Energy’s National Nuclear Security Administration under contract DE–AC04–94AL85000.

Author information

Authors and Affiliations

  1. Sandia National Laboratories, Livermore, CA, USA

    Tamara G. Kolda

Authors
  1. Tamara G. Kolda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tamara G. Kolda.

Additional information

This material is based upon work supported by the US Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Applied Mathematics program. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the US Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kolda, T.G. Numerical optimization for symmetric tensor decomposition. Math. Program. 151, 225–248 (2015). https://doi.org/10.1007/s10107-015-0895-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-015-0895-0

Keywords

Mathematics Subject Classification

Search

Navigation