Skip to main content

O(dlog N)-Quantics Approximation of N-d Tensors in High-Dimensional Numerical Modeling

Abstract

In the present paper, we discuss the novel concept of super-compressed tensor-structured data formats in high-dimensional applications. We describe the multifolding or quantics-based tensor approximation method of O(dlog N)-complexity (logarithmic scaling in the volume size), applied to the discrete functions over the product index set {1,…,N}d, or briefly N-d tensors of size N d, and to the respective discretized differential-integral operators in ℝd. As the basic approximation result, we prove that a complex exponential sampled on an equispaced grid has quantics rank 1. Moreover, a Chebyshev polynomial, sampled over a Chebyshev Gauss–Lobatto grid, has separation rank 2 in the quantics tensor format, while for the polynomial of degree m over a Chebyshev grid the respective quantics rank is at most 2m+1. For N-d tensors generated by certain analytic functions, we give a constructive proof of the O(dlog Nlog ε −1)-complexity bound for their approximation by low-rank 2-(dlog N) quantics tensors up to the accuracy ε>0. In the case ε=O(N α), α>0, our approach leads to the quantics tensor numerical method in dimension d, with the nearly optimal asymptotic complexity O(d/αlog 2 ε −1). From numerical examples presented here, we observe that the quantics tensor method has proved its value in application to various function related tensors/matrices arising in computational quantum chemistry and in the traditional finite element method/boundary element method (FEM/BEM). The tool apparently works.

This is a preview of subscription content, access via your institution.

References

  1. Beck, M.H., Jäckle, A., Worth, G.A., Meyer, H.-D.: The multiconfiguration time-dependent Hartree (MCTDH) method: A highly efficient algorithm for propagating wavepackets. Phys. Rep. 324, 1–105 (2000)

    Article  Google Scholar 

  2. Beylkin, G., Mohlenkamp, M.J.: Algorithms for numerical analysis in high dimension. SIAM J. Sci. Comput. 26(6), 2133–2159 (2005)

    MathSciNet  MATH  Article  Google Scholar 

  3. Beylkin, G., Garcke, J., Mohlenkamp, M.J.: Multivariate regression and machine learning with sums of separable functions. SIAM J. Sci. Comput. 31(3), 1840–1857 (2009)

    MathSciNet  MATH  Article  Google Scholar 

  4. Binev, P., Dahmen, W., Lamby, P.: Fast high-dimensional approximation with sparse occupancy trees. J. Comput. Appl. Math. 235(8), 2063–2076 (2011)

    MathSciNet  MATH  Article  Google Scholar 

  5. Bungartz, H.J., Griebel, M.: Sparse grids. Acta Numer. 1–123 (2004)

  6. Comon, P., Mourrain, B.: Decomposition of quantics in sums of powers of linear forms. Signal Process. 53(2), 96–107 (1996)

    Article  Google Scholar 

  7. Dahmen, W.: Wavelet and multiscale methods for operator equations. Acta Numer. 6, 55–228 (1997)

    MathSciNet  Article  Google Scholar 

  8. Dahmen, W., Proessdorf, S., Schneider, R.: Multiscale methods for pseudo-differential equations on manifolds. In: Wavelet Analysis and Its Applications, vol. 5. Academic Press, New York (1995)

    Google Scholar 

  9. De Lathauwer, L., De Moor, B., Vandewalle, J.: On the best rank-1 and rank-(R 1,…,R N ) approximation of higher-order tensors. SIAM J. Matrix Anal. Appl. 21, 1324–1342 (2000)

    MathSciNet  MATH  Article  Google Scholar 

  10. De Lathauwer, L., De Moor, B., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21, 1253–1278 (2000)

    MathSciNet  MATH  Article  Google Scholar 

  11. De Silva, V., Lim, L.-H.: Tensor rank and the ill-posedness of the best low-rank approximation problem. SIAM J. Matrix Anal. Appl. 30(3), 1084–1127 (2008)

    MathSciNet  Article  Google Scholar 

  12. Gavrilyuk, I.P., Hackbusch, W., Khoromskij, B.N.: Tensor-product approximation to elliptic and parabolic solution operators in higher dimensions. Computing 74, 131–157 (2005)

    MathSciNet  MATH  Article  Google Scholar 

  13. Grasedyck, L.: Hierarchical singular value decomposition of tensors. Preprint 27/2009, MPI MIS Leipzig (2009)

  14. Grasedyck, L.: Polynomial approximation in hierarchical Tucker format by vector tensorization. Preprint 43, DFG/SPP1324, RWTH Aachen (2010)

  15. Hackbusch, W., Khoromskij, B.N.: Tensor-product approximation to operators and functions in high dimension. J. Complex. 23, 697–714 (2007)

    MathSciNet  MATH  Article  Google Scholar 

  16. Hackbusch, W., Kühn, S.: A new scheme for the tensor representation. J. Fourier Anal. Appl. 15(5), 706–722 (2009)

    MathSciNet  MATH  Article  Google Scholar 

  17. Hackbusch, W., Khoromskij, B.N., Sauter, S., Tyrtyshnikov, E.: Use of tensor formats in elliptic eigenvalue problems. Preprint 78/2008, MPI MIS Leipzig (2008, submitted)

  18. Hitchcock, F.L.: The expression of a tensor or a polyadic as a sum of products. J. Math. Phys. 6, 164–189 (1927)

    Google Scholar 

  19. Holtz, S., Rohwedder, T., Schneider, R.: On manifold of tensors of fixed TT-rank. Tech. Rep. 61, TU Berlin (2010)

  20. Jaravine, A., Ibraghimov, I., Orekhov, V.: Removal of time barrier for high-resolution multidimensional NMR spectroscopy. Nat. Methods 3(8), 605–607 (2006)

    Article  Google Scholar 

  21. Khoromskaia, V.: Computation of the Hartree–Fock exchange in the tensor-structured format. Comput. Methods Appl. Math. 10(2), 204–218 (2010)

    MathSciNet  Google Scholar 

  22. Khoromskij, B.N.: Structured rank-(r 1)(…,r d ) decomposition of function-related tensors in ℝd. Comput. Methods Appl. Math. 6(2), 194–220 (2006)

    MathSciNet  MATH  Google Scholar 

  23. Khoromskij, B.N.: On tensor approximation of Green iterations for Kohn–Sham equations. Comput. Vis. Sci. 11, 259–271 (2008)

    MathSciNet  Article  Google Scholar 

  24. Khoromskij, B.N.: Tensor-structured preconditioners and approximate inverse of elliptic operators in ℝd. J. Constr. Approx. 30, 599–620 (2009)

    MathSciNet  MATH  Article  Google Scholar 

  25. Khoromskij, B.N.: Fast and accurate tensor approximation of a multivariate convolution with linear scaling in dimension. J. Comput. Appl. Math. 234, 3122–3139 (2010)

    MathSciNet  MATH  Article  Google Scholar 

  26. Khoromskij, B.N.: Tensors-structured numerical methods in scientific computing: survey on recent advances. Preprint 21/2010, MPI MiS Leipzig (2010, submitted)

  27. Khoromskij, B.N., Khoromskaia, V.: Multigrid tensor approximation of function related arrays. SIAM J. Sci. Comput. 31(4), 3002–3026 (2009)

    MathSciNet  MATH  Article  Google Scholar 

  28. Khoromskij, B.N., Oseledets, I.: Quantics-TT approximation of elliptic solution operators in higher dimensions. Preprint MPI MiS 79/2009, Leipzig (2009) (J. Numer. Math. 2011, to appear)

  29. Khoromskij, B.N., Oseledets, I.: Quantics-TT collocation approximation of parameter-dependent and stochastic elliptic PDEs. Comput. Methods Appl. Math. 10(4), 345–365 (2010)

    MathSciNet  Google Scholar 

  30. Khoromskij, B.N., Schwab, C.: Tensor-structured Galerkin approximation of parametric and stochastic elliptic PDEs. SIAM J. Sci. Comput. 33(1), 364–385 (2011)

    MathSciNet  Article  Google Scholar 

  31. Khoromskij, B.N., Khoromskaia, V., Flad, H.-J.: Numerical solution of the Hartree–Fock equation in multilevel tensor-structured format. SIAM J. Sci. Comput. 33(1), 45–65 (2011)

    MathSciNet  Article  Google Scholar 

  32. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)

    MathSciNet  MATH  Article  Google Scholar 

  33. Lubich, C.: From Quantum to Classical Molecular Dynamics: Reduced Models and Numerical Analysis. Zurich Lectures in Advanced Mathematics. EMS, Zurich (2008)

    MATH  Book  Google Scholar 

  34. Oseledets, I.V.: Compact matrix form of the d-dimensional tensor decomposition. Preprint 09-01, INM RAS, Moscow (2009)

  35. Oseledets, I.V.: Approximation of 2d×2d matrices using tensor decomposition. SIAM J. Matrix Anal. Appl. 31(4), 2130–2145 (2010)

    MathSciNet  MATH  Article  Google Scholar 

  36. Oseledets, I.V., Tyrtyshnikov, E.E.: Breaking the curse of dimensionality, or how to use SVD in many dimensions. SIAM J. Sci. Comput. 31, 3744–3759 (2009)

    MathSciNet  MATH  Article  Google Scholar 

  37. Oseledets, I.V., Tyrtyshnikov, E.E.: Tensor tree decomposition does not need a tree. Preprint 8/2009, INM RUS, Moscow (2009, submitted)

  38. Papy, J.-M., De Lathauwer, L., Van Huffel, S.: Exponential data fitting using multilinear algebra: the decimative case. Chemometrics. doi:10.1002/cem.1212

  39. Schneider, R., Rohwedder, T., Blauert, J., Neelov, A.: Direct minimization for calculating invariant subspaces in density functional computations of the electronic structure. J. Comput. Math. 27(2–3), 360–387 (2009)

    MathSciNet  MATH  Google Scholar 

  40. Schwab, C., Todor, R.: Sparse finite elements for elliptic problems with stochastic loading. Numer. Math. 95(4), 707–713 (2003)

    MathSciNet  MATH  Article  Google Scholar 

  41. Sidiropoulos, N.D.: Generalized Carathéodory’s uniqueness of harmonic parametrization to N dimensions. IEEE Trans. Inf. Theory 47, 1687–1690 (2001)

    MathSciNet  MATH  Article  Google Scholar 

  42. Temlyakov, V.N.: Nonlinear Methods of Approximation. Foundation of Comp. Math. Springer, Berlin (2008)

    Google Scholar 

  43. Tucker, L.R.: Some mathematical notes on three-mode factor analysis. Psychometrika 31, 279–311 (1966)

    MathSciNet  Article  Google Scholar 

  44. Vidal, G.: Efficient classical simulation of slightly entangled quantum computations. Phys. Rev. Lett. 91(14), 147902 (2003)

    Article  Google Scholar 

  45. White, S.R.: Density-matrix algorithms for quantum renormalization groups. Phys. Rev. B 48(14), 10345–10356 (1993)

    Article  Google Scholar 

  46. Yserentant, H.: The hyperbolic cross space approximation of electronic wavefunctions. Numer. Math. 105, 659–690 (2007)

    MathSciNet  MATH  Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Boris N. Khoromskij.

Additional information

Communicated by Wolfgang Dahmen.

Dedicated to Prof. W. Dahmen on the occasion of his 60th birthday.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Khoromskij, B.N. O(dlog N)-Quantics Approximation of N-d Tensors in High-Dimensional Numerical Modeling. Constr Approx 34, 257–280 (2011). https://doi.org/10.1007/s00365-011-9131-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00365-011-9131-1

Keywords

  • High-dimensional problems
  • Quantics folding of vectors
  • Rank-structured tensor approximation
  • Matrix-valued functions
  • FEM
  • Material sciences
  • Stochastic modeling

Mathematics Subject Classification (2000)

  • 65F30
  • 65F50
  • 65N35
  • 65F10