Foundations of Computational Mathematics

, Volume 15, Issue 4, pp 839–898 | Cite as

Adaptive Near-Optimal Rank Tensor Approximation for High-Dimensional Operator Equations

  • Markus Bachmayr
  • Wolfgang DahmenEmail author


We consider a framework for the construction of iterative schemes for operator equations that combine low-rank approximation in tensor formats and adaptive approximation in a basis. Under fairly general assumptions, we conduct a rigorous convergence analysis, where all parameters required for the execution of the methods depend only on the underlying infinite-dimensional problem, but not on a concrete discretization. Under certain assumptions on the rates for the involved low-rank approximations and basis expansions, we can also give bounds on the computational complexity of the iteration as a function of the prescribed target error. Our theoretical findings are illustrated and supported by computational experiments. These demonstrate that problems in very high dimensions can be treated with controlled solution accuracy.


Low-rank tensor approximation Adaptive methods High-dimensional operator equations Computational complexity 

Mathematics Subject Classification

41A46 41A63 65D99 65J10 65N12 65N15 



This work was funded in part by the Excellence Initiative of the German Federal and State Governments, DFG Grant GSC 111 (Graduate School AICES), the DFG Special Priority Program 1324, and NSF Grant #1222390.


  1. 1.
    Alpert, B.: A class of bases in \(L^2\) for the sparse representation of integral operators. SIAM J. Math. Anal. 24(1), 246–262 (1991)Google Scholar
  2. 2.
    Bachmayr, M.: Adaptive low-rank wavelet methods and applications to two-electron Schrödinger equations. Ph.D. thesis, RWTH Aachen (2012)Google Scholar
  3. 3.
    Ballani, J., Grasedyck, L.: A projection method to solve linear systems in tensor format. Numer. Linear Algebra Appl. 20(1), 27–43 (2013)Google Scholar
  4. 4.
    Barinka, A.: Fast evaluation tools for adaptive wavelet schemes. Ph.D. thesis, RWTH Aachen (2005)Google Scholar
  5. 5.
    Beylkin, G., Mohlenkamp, M.J.: Numerical operator calculus in higher dimensions. PNAS 99(16), 10246–10251 (2002)Google Scholar
  6. 6.
    Beylkin, G., Mohlenkamp, M.J.: Algorithms for numerical analysis in high dimensions. SIAM J. Sci. Comput. 26(6), 2133–2159 (2005)Google Scholar
  7. 7.
    Cances, E., Ehrlacher, V., Lelievre, T.: Convergence of a greedy algorithm for high-dimensional convex nonlinear problems. Math. Models Methods Appl. Sci. 21(12), 2433–2467 (2011)Google Scholar
  8. 8.
    Cohen, A.: Numerical Analysis of Wavelet Methods, Studies in Mathematics and Its Applications, vol. 32. Elsevier, Amsterdam (2003)Google Scholar
  9. 9.
    Cohen, A., Dahmen, W., DeVore, R.: Adaptive wavelet methods for elliptic operator equations: Convergence rates. Math. Comput. 70(233), 27–75 (2001)Google Scholar
  10. 10.
    Cohen, A., Dahmen, W., DeVore, R.: Adaptive wavelet methods II—beyond the elliptic case. Found. Comput. Math. 2(3), 203–245 (2002)Google Scholar
  11. 11.
    Dahmen, W.: Wavelet and multiscale methods for operator equations. Acta Numer. 6, 55–228 (1997)Google Scholar
  12. 12.
    DeVore, R., Petrova, G., Wojtaszczyk, P.: Approximation of functions of few variables in high dimensions. Constr. Approx. 33, 125–143 (2011)Google Scholar
  13. 13.
    Dijkema, T.J., Schwab, C., Stevenson, R.: An adaptive wavelet method for solving high-dimensional elliptic PDEs. Constr. Approx. 30(3), 423–455 (2009)Google Scholar
  14. 14.
    Falcó, A., Hackbusch, W.: On minimal subspaces in tensor representations. Found. Comput. Math. 12, 765–803 (2012)Google Scholar
  15. 15.
    Falcó, A., Hackbusch, W., Nouy, A.: Geometric structures in tensor representations. Preprint 9/2013, Max Planck Institute of Mathematics in the Sciences, Leipzig (2013)Google Scholar
  16. 16.
    Falcó, A., Nouy, A.: Proper generalized decomposition for nonlinear convex problems in tensor banach spaces. Numer. Math. 121, 503–530 (2012)Google Scholar
  17. 17.
    Grasedyck, L.: Hierarchical singular value decomposition of tensors. SIAM J. Matrix Anal. Appl. 31(4), 2029–2054 (2010)Google Scholar
  18. 18.
    Grasedyck, L., Kressner, D., Tobler, C.: A literature survey of low-rank tensor approximation techniques. GAMM-Mitt. 36, 53–78 (2013)Google Scholar
  19. 19.
    Griebel, M., Harbrecht, H.: Approximation of two-variate functions: Singular value decomposition versus regular sparse grids. INS Preprint No. 1109, Universität Bonn (2011)Google Scholar
  20. 20.
    Hackbusch, W.: Tensor Spaces and Numerical Tensor Calculus, Springer Series in Computational Mathematics, vol. 42. Springer, Berlin (2012)Google Scholar
  21. 21.
    Hackbusch, W., Khoromskij, B., Tyrtyshnikov, E.: Approximate iterations for structured matrices. Numer. Math. 109, 119–156 (2008)Google Scholar
  22. 22.
    Hackbusch, W., Kühn, S.: A new scheme for the tensor representation. J. Fourier Anal. Appl. 15(5), 706–722 (2009)Google Scholar
  23. 23.
    Hitchcock, F.L.: Multiple invariants and generalized rank of a \(p\)-way matrix or tensor. J. Math. Phys. 7, 39–79 (1927)Google Scholar
  24. 24.
    Khoromskij, B.N., Schwab, C.: Tensor-structured Galerkin approximation of parametric and stochastic elliptic PDEs. SIAM J. Sci. Comput. 33(1), 364–385 (2011)Google Scholar
  25. 25.
    Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)Google Scholar
  26. 26.
    Kressner, D., Tobler, C.: Preconditioned low-rank methods for high-dimensional elliptic PDE eigenvalue problems. Comput. Methods Appl. Math. 11(3), 363–381 (2011)Google Scholar
  27. 27.
    Lathauwer, L.D., Moor, B.D., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21(4), 1253–1278 (2000)Google Scholar
  28. 28.
    Matthies, H.G., Zander, E.: Solving stochastic systems with low-rank tensor compression. Linear Algebra Appl. 436(10), 3819–3838 (2012)Google Scholar
  29. 29.
    Metselaar, A.: Handling wavelet expansions in numerical methods. Ph.D. thesis, University of Twente (2002)Google Scholar
  30. 30.
    Novak, E., Wozniakowski, H.: Approximation of infinitely differentiable multivariate functions is intractable. J. Complex. 25, 398–404 (2009)Google Scholar
  31. 31.
    Oseledets, I., Tyrtyshnikov, E.: Breaking the curse of dimensionality, or how to use SVD in many dimensions. SIAM J. Sci. Comput. 31(5), 3744–3759 (2009)Google Scholar
  32. 32.
    Oseledets, I., Tyrtyshnikov, E.: Tensor tree decomposition does not need a tree. Tech. Rep., RAS, Moscow 2009–08 (2009)Google Scholar
  33. 33.
    Oseledets, I.V.: Tensor-train decomposition. SIAM J. Sci. Comput. 33(5), 2295–2317 (2011)Google Scholar
  34. 34.
    Schneider, R., Uschmajew, A.: Approximation rates for the hierarchical tensor format in periodic Sobolev spaces. J. Complexity 30, 56–71 (2014)Google Scholar
  35. 35.
    de Silva, V., Lim, L.H.: Tensor rank and the ill-posedness of the best low-rank approximation problem. SIAM J. Matrix Anal. Appl. 30(3), 1084–1127 (2008)Google Scholar
  36. 36.
    Stevenson, R.: On the compressibility of operators in wavelet coordinates. SIAM J. Math. Anal. 35(5), 1110–1132 (2004)Google Scholar
  37. 37.
    Tucker, L.R.: The extension of factor analysis to three-dimensional matrices. Contributions to Mathematical Psychology, pp. 109–127. Holt, Rinehart & Winston, New York (1964)Google Scholar
  38. 38.
    Tucker, L.R.: Some mathematical notes on three-mode factor analysis. Psychometrika 31, 279–311 (1966)Google Scholar
  39. 39.
    Uschmajew, A.: Well-posedness of convex maximization problems on Stiefel manifolds and orthogonal tensor product approximations. Numer. Math. 115, 309–331 (2010)Google Scholar
  40. 40.
    Uschmajew, A.: Regularity of tensor product approximations to square integrable functions. Constr. Approx. 34, 371–391 (2011)Google Scholar

Copyright information

© SFoCM 2014

Authors and Affiliations

  1. 1.Institut für Geometrie und Praktische MathematikRWTH AachenAachenGermany

Personalised recommendations