Structured Matrix Problems from Tensors

  • Charles F. Van LoanEmail author
Part of the Lecture Notes in Mathematics book series (LNM, volume 2173)


This chapter looks at the structured matrix computations that arise in the context of various “svd-like” tensor decompositions. Kronecker products and low-rank manipulations are central to the theme. Algorithmic details include the exploitation of partial symmetries, componentwise optimization, and how we might beat the “curse of dimensionality.” Order-4 tensors figure heavily in the discussion.


Symmetric Tensor Block Matrix Kronecker Product Tensor Decomposition Generalize Singular Value Decomposition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    G. Baumgartner, A. Auer, D. Bernholdt, A. Bibireata, V. Choppella, D. Cociorva, X. Gao, R. Harrison, S. Hirata, S. Krishnamoorthy, S. Krishnan, C. Lam, Q. Lu, M. Nooijen, R. Pitzer, J. Ramanujam, P. Sadayappan, A. Sibiryakov, Synthesis of high-performance parallel programs for a class of ab initio quantum chemistry models. Proc. IEEE 93 (2), 276–292 (2005)CrossRefGoogle Scholar
  2. 2.
    G. Beylkin, M.J. Mohlenkamp, Numerical operator calculus in higher dimensions. Proc. Natl. Acad. Sci. 99 (16), 10246–10251 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    G. Beylkin, M.J. Mohlenkamp, Algorithms for numerical analysis in high dimensions. SIAM J. Sci. Comp. 26, 2133–2159 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    P. Comon, G. Golub, L.-H. Lim, B. Mourrain, Genericity and rank deficiency of high order symmetric tensors. Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP ’06) 31 (3), 125–128 (2006)Google Scholar
  5. 5.
    P. Comon, G. Golub, L.-H. Lim, B. Mourrain, Symmetric tensors and symmetric tensor rank. SIAM J. Matrix Anal. Appl. 30, 1254–1279 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    L. de Lathauwer, Signal Processing Based on Multilinear Algebra. Ph.D. thesis, K.U. Leuven, 1997Google Scholar
  7. 7.
    L. De Lathauwer, P. Comon, B. De Moor, J. Vandewalle, Higher-order power method–application in independent component analysis, in Proceedings of the International Symposium on Nonlinear Theory and Its Applications (NOLTA ’95), Las Vegas, NV (1995), pp. 91–96Google Scholar
  8. 8.
    L. De Lathauwer, B. De Moor, J. Vandewalle, A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21, 1253–1278 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    V. De Silva, L.-H. Lim, Tensor rank and the ill-posedness of the best low-rank approximation problem. SIAM J. Matrix Anal. Appl. 30, 1084–1127 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    G.H. Golub, C.F. Van Loan, Matrix Computations, 4th edn. (Johns Hopkins University Press, Baltimore, MD, 2013)zbMATHGoogle Scholar
  11. 11.
    W. Hackbusch, B.N. Khoromskij, Tensor-product approximation to operators and functions in high dimensions. J. Complexity 23, 697–714 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    H.V. Henderson, S.R. Searle, The vec-permutation matrix, the vec operator and Kronecker products: a review. Linear Multilinear Algebra 9, 271–288 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    C.J. Hillar, L.-H. Lim, Most tensor problems are NP-hard. J. ACM 60 (6), Art. 33–47 (2013)Google Scholar
  14. 14.
    E. Kofidis, P.A. Regalia, On the best rank-1 approximation of higher-order supersymmetric tensors. SIAM J. Matrix Anal. Appl. 23, 863–884 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    T.G. Kolda, B.W. Bader, Algorithm 862: MATLAB tensor classes for fast algorithm prototyping. ACM Trans. Math. Softw. 32, 635–653 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    T.G. Kolda, B.W. Bader, Tensor decompositions and applications. SIAM Rev. 51, 455–500 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    L.-H. Lim, Singular values and eigenvalues of tensors: a variational approach, in Proceedings of the IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP ’05), vol. 1 (2005), pp. 129–132Google Scholar
  18. 18.
    L.-H. Lim, Tensors and hypermatrices, in Handbook of Linear Algebra, Chap. 15, 2nd edn., ed. by L. Hogben (CRC Press, Boca Raton, FL, 2013), 30 pp.Google Scholar
  19. 19.
    C. Martin, C. Van Loan, A Jacobi-type method for computing orthogonal tensor decompositions. SIAM J. Matrix Anal. Appl. 30, 1219–1232 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    I.V. Oseledets, E.E. Tyrtyshnikov, Breaking the curse of dimensionality, or how to use SVD in many dimensions. SIAM J. Sci. Comput. 31, 3744–3759 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    I. Oseledets, E. Tyrtyshnikov, TT-cross approximation for multidimensional arrays, Linear Algebra Appl. 432, 70–88 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    C.C. Paige, C.F. Van Loan, A Schur decomposition for Hamiltonian matrices. Linear Algebra Appl. 41,11–32 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    N. Pitsianis, C. Van Loan, Approximations with Kronecker products, in Linear Algebra for Large Scale and Real-Time Applications, ed. by M.S. Moonen, G.H. Golub (Kluwer, Dordrecht, 1993), pp. 293–314Google Scholar
  24. 24.
    J. Poulson, B. Marker, R.A. van de Geijn, J.R. Hammond, N.A. Romero, Elemental: A new framework for distributed memory dense matrix computations. ACM Trans. Math. Softw. 39 (2), 13:1–13:24, February 2013 (2014). arXiv:1301.7744Google Scholar
  25. 25.
    S. Ragnarsson, C.F. Van Loan, Block tensor unfoldings. SIAM J. Matrix Anal. Appl. 33 (1), 149–169 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    S. Ragnarsson, C.F. Van Loan, Block tensors and symmetric embeddings. Linear Algebra Appl. 438, 853–874 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    A. Smilde, R. Bro, P. Geladi, Multi-way Analysis with Applications in the Chemical Sciences (Wiley, Chichester, 2004)CrossRefGoogle Scholar
  28. 28.
    E. Solomonik, D. Matthews, J. Hammond, J. Demmel, Cyclops Tensor Framework: reducing communication and eliminating load imbalance in massively parallel contractions, Berkeley Technical Report No. UCB/EECS-2013-1 (2013)Google Scholar
  29. 29.
    C.F. Van Loan, Computational Frameworks for the Fast Fourier Transform (SIAM, Philadelphia, PA, 1992)CrossRefzbMATHGoogle Scholar
  30. 30.
    C. Van Loan, The ubiquitous Kronecker product. J. Comput. Appl. Math. 123, 85–100 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    C.F. Van Loan, J.P. Vokt, Approximating matrices with multiple symmetries. SIAM J. Matrix Anal. Appl. 36 (3), 974–993 (2015)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceCornell UniversityIthacaUSA

Personalised recommendations