Matrices with Hierarchical Low-Rank Structures

  • Jonas Ballani
  • Daniel KressnerEmail author
Part of the Lecture Notes in Mathematics book series (LNM, volume 2173)


Matrices with low-rank off-diagonal blocks are a versatile tool to perform matrix compression and to speed up various matrix operations, such as the solution of linear systems. Often, the underlying block partitioning is described by a hierarchical partitioning of the row and column indices, thus giving rise to hierarchical low-rank structures. The goal of this chapter is to provide a brief introduction to these techniques, with an emphasis on linear algebra aspects.


Singular Value Decomposition Singular Vector Column Index Unitarily Invariant Norm Hierarchical Matrice 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



The first author has been supported by an EPFL fellowship through the European Union’s Seventh Framework Programme under grant agreement no. 291771.


  1. 1.
    S. Ambikasaran, D. Foreman-Mackey, L. Greengard, D.W. Hogg, M. O’Neil, Fast direct methods for Gaussian processes. IEEE Trans. Pattern Anal. Mach. Intell. 38 (2), 252–265 (2016)CrossRefGoogle Scholar
  2. 2.
    A. Aminfar, E. Darve, A fast sparse solver for finite-element matrices. Preprint arXiv:1410.2697 (2014)Google Scholar
  3. 3.
    J. Ballani, D. Kressner, Sparse inverse covariance estimation with hierarchical matrices. Technical Report, EPFL-MATHICSE-ANCHP, Oct (2014)Google Scholar
  4. 4.
    U. Baur, P. Benner, Factorized solution of Lyapunov equations based on hierarchical matrix arithmetic. Computing 78 (3), 211–234 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    M. Bebendorf, Approximation of boundary element matrices. Numer. Math. 86 (4), 565–589 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    M. Bebendorf, Hierarchical Matrices. Lecture Notes in Computational Science and Engineering, vol. 63 (Springer, Berlin, 2008)Google Scholar
  7. 7.
    M. Bebendorf, T. Fischer, On the purely algebraic data-sparse approximation of the inverse and the triangular factors of sparse matrices. Numer. Linear Algebra Appl. 18 (1), 105–122 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    M. Bebendorf, W. Hackbusch, Existence of \(\mathcal{H}\)-matrix approximants to the inverse FE-matrix of elliptic operators with L -coefficients. Numer. Math. 95 (1), 1–28 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    M. Bebendorf, S. Rjasanow, Adaptive low-rank approximation of collocation matrices. Computing 70 (1), 1–24 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    M. Bebendorf, Y. Maday, B. Stamm, Comparison of some reduced representation approximations, in Reduced Order Methods for Modeling and Computational Reduction. MS & A Modelling, Simulation and Applications, vol. 9 (Springer, Cham, 2014), pp. 67–100Google Scholar
  11. 11.
    B. Beckermann, The condition number of real Vandermonde, Krylov and positive definite Hankel matrices. Numer. Math. 85 (4), 553–577 (2000)MathSciNetzbMATHGoogle Scholar
  12. 12.
    P. Benner, T. Mach, Computing all or some eigenvalues of symmetric \(\mathcal{H}_{\ell}\)-matrices. SIAM J. Sci. Comput. 34 (1), A485–A496 (2012)CrossRefzbMATHGoogle Scholar
  13. 13.
    P. Benner, T. Mach, The preconditioned inverse iteration for hierarchical matrices. Numer. Linear Algebra Appl. 20, 150–166 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    P. Benner, S. Börm, T. Mach, K. Reimer, Computing the eigenvalues of symmetric \(\mathcal{H}^{2}\)-matrices by slicing the spectrum. Comput. Visual. Sci. 16, 271–282 (2013)MathSciNetCrossRefGoogle Scholar
  15. 15.
    M. Benzi, N. Razouk, Decay bounds and O(n) algorithms for approximating functions of sparse matrices. Electron. Trans. Numer. Anal. 28, 16–39 (2007/2008)Google Scholar
  16. 16.
    M.W. Berry. Large scale singular value decomposition. Internat. J. Supercomput. Appl. 6, 13–49 (1992)Google Scholar
  17. 17.
    S. Börm, Efficient numerical methods for non-local operators, in EMS Tracts in Mathematics, vol. 14 (European Mathematical Society (EMS), Zürich, 2010)CrossRefGoogle Scholar
  18. 18.
    S. Börm, L. Grasedyck, Hybrid cross approximation of integral operators. Numer. Math. 101 (2), 221–249 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    S. Börm, L. Grasedyck, W. Hackbusch, Hierarchical Matrices. Lecture Notes 21 (MPI for Mathematics in the Sciences, Leipzig, 2003)Google Scholar
  20. 20.
    A. Çivril, M. Magdon-Ismail. Exponential inapproximability of selecting a maximum volume sub-matrix. Algorithmica 65 (1), 159–176 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    S. Chandrasekaran, P. Dewilde, M. Gu, T. Pals, X. Sun, A.-J. van der Veen, D. White. Some fast algorithms for sequentially semiseparable representations. SIAM J. Matrix Anal. Appl. 27 (2), 341–364 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    J. Chiu, L. Demanet, Sublinear randomized algorithms for skeleton decompositions. SIAM J. Matrix Anal. Appl. 34 (3), 1361–1383 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    A. Çivril, M. Magdon-Ismail, On selecting a maximum volume sub-matrix of a matrix and related problems. Theor. Comput. Sci. 410 (47–49), 4801–4811 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    E. Corona, P.-G. Martinsson, D. Zorin, An O(N) direct solver for integral equations on the plane. Appl. Comput. Harmon. Anal. 38 (2), 284–317 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    T.A. Davis, Direct Methods for Sparse Linear Systems. Fundamentals of Algorithms, vol. 2 (Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2006)Google Scholar
  26. 26.
    S. Demko, W.F. Moss, P.W. Smith, Decay rates for inverses of band matrices. Math. Comput. 43 (168), 491–499 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    J.W. Demmel, M. Gu, S. Eisenstat, I. Slapničar, K. Veselić, Z. Drmač. Computing the singular value decomposition with high relative accuracy. Linear Algebra Appl. 299 (1–3), 21–80 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Z. Drmač, K. Veselić, New fast and accurate Jacobi SVD algorithm. I. SIAM J. Matrix Anal. Appl. 29 (4), 1322–1342 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    P. Gatto, J.S. Hesthaven, A preconditioner based on low-rank approximation of Schur complements. Technical Report, EPFL-MATHICSE-MCSS (2015)Google Scholar
  30. 30.
    I.P. Gavrilyuk, W. Hackbusch, B.N. Khoromskij, \(\mathcal{H}\)-matrix approximation for the operator exponential with applications. Numer. Math. 92 (1), 83–111 (2002)Google Scholar
  31. 31.
    P. Ghysels, X.S. Li, F.-H. Rouet, S. Williams, A. Napov, An efficient multi-core implementation of a novel HSS-structured multifrontal solver using randomized sampling. SIAM J. Sci. Comput. 38 (5), S358–S384 (2016)CrossRefzbMATHGoogle Scholar
  32. 32.
    A. Gillman, P.M. Young, P.G. Martinsson, A direct solver with O(N) complexity for integral equations on one-dimensional domains. Front. Math. China 7 (2), 217–247 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    G.H. Golub, C.F. Van Loan, Matrix Computations, 4th edn. (Johns Hopkins University Press, Baltimore, MD, 2013)zbMATHGoogle Scholar
  34. 34.
    G.H. Golub, F.T. Luk, M.L. Overton, A block Lánczos method for computing the singular values of corresponding singular vectors of a matrix. ACM Trans. Math. Softw. 7 (2), 149–169 (1981)CrossRefzbMATHGoogle Scholar
  35. 35.
    S.A. Goreinov, E.E. Tyrtyshnikov, The maximal-volume concept in approximation by low-rank matrices, in Structured Matrices in Mathematics, Computer Science, and Engineering, I (Boulder, CO, 1999). Contemporary Mathematics, vol. 280 (American Mathematical Society, Providence, RI, 2001), pp. 47–51Google Scholar
  36. 36.
    S.A. Goreinov, E.E. Tyrtyshnikov, N.L. Zamarashkin, A theory of pseudoskeleton approximations. Linear Algebra Appl. 261, 1–21 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    L. Grasedyck, W. Hackbusch, B.N. Khoromskij, Solution of large scale algebraic matrix Riccati equations by use of hierarchical matrices. Computing 70 (2), 121–165 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    L. Greengard, V. Rokhlin, A fast algorithm for particle simulations. J. Comput. Phys. 73 (2), 325–348 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    L. Greengard, D. Gueyffier, P.G. Martinsson, V. Rokhlin. Fast direct solvers for integral equations in complex three-dimensional domains. Acta Numer. 18, 243–275 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    W. Hackbusch, A sparse matrix arithmetic based on \(\mathcal{H}\)-matrices. Part I: Introduction to \(\mathcal{H}\)-matrices. Computing 62 (2), 89–108 (1999)Google Scholar
  41. 41.
    W. Hackbusch, Entwicklungen nach Exponentialsummen. Technical Report 4/2005, MPI MIS Leipzig (2010) Revised version Sept (2010)Google Scholar
  42. 42.
    W. Hackbusch, Hierarchical Matrices: Algorithms and Analysis (Springer, Berlin, 2015)CrossRefzbMATHGoogle Scholar
  43. 43.
    W. Hackbusch, New estimates for the recursive low-rank truncation of block-structured matrices. Numer. Math. 132 (2), 303–328 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    W. Hackbusch, B.N. Khoromskij, R. Kriemann, Hierarchical matrices based on a weak admissibility criterion. Computing 73 (3), 207–243 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    N. Halko, P.G. Martinsson, J.A. Tropp, Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev. 53 (2), 217–288 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    H. Harbrecht, M. Peters, R. Schneider, On the low-rank approximation by the pivoted Cholesky decomposition. Appl. Numer. Math. 62 (4), 428–440 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    N.J. Higham, Accuracy and Stability of Numerical Algorithms (SIAM, Philadelphia, PA, 1996)zbMATHGoogle Scholar
  48. 48.
    R.A. Horn, C.R. Johnson, Matrix Analysis, 2nd edn. (Cambridge University Press, Cambridge, 2013)zbMATHGoogle Scholar
  49. 49.
    R.B. Lehoucq, D.C. Sorensen, C. Yang, ARPACK Users’ Guide (SIAM, Philadelphia, PA, 1998)CrossRefzbMATHGoogle Scholar
  50. 50.
    J.Y. Li, S. Ambikasaran, E.F. Darve, P.K. Kitanidis, A Kalman filter powered by H2-matrices for quasi-continuous data assimilation problems. Water Resour. Res. 50, 3734–3749 (2014)CrossRefGoogle Scholar
  51. 51.
    D. Liu, H.G. Matthies, Pivoted Cholesky decomposition by cross approximation for efficient solution of kernel systems. Preprint arXiv:1505.06195 (2015)Google Scholar
  52. 52.
    Y. Maday, O. Mula, A.T. Patera, M. Yano, The generalized empirical interpolation method: stability theory on Hilbert spaces with an application to the Stokes equation. Comput. Methods Appl. Mech. Eng. 287, 310–334 (2015)MathSciNetCrossRefGoogle Scholar
  53. 53.
    P.G. Martinsson, V. Rokhlin, A fast direct solver for boundary integral equations in two dimensions. J. Comput. Phys. 205 (1), 1–23 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  54. 54.
    A. Napov, X.S. Li, An algebraic multifrontal preconditioner that exploits the low-rank property. Numer. Linear Algebra Appl. 23 (1), 61–82 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  55. 55.
    J. Ostrowski, M. Bebendorf, R. Hiptmair, and F. Krämer. H-matrix based operator preconditioning for full Maxwell at low frequencies. IEEE Trans. Magn. 46 (8), 3193–3196 (2010)CrossRefGoogle Scholar
  56. 56.
    H. Pouransari, P. Coulier, E. Darve, Fast hierarchical solvers for sparse matrices. Preprint arXiv:1510.07363 (2015)Google Scholar
  57. 57.
    S.A. Sauter, C. Schwab, Boundary Element Methods. Springer Series in Computational Mathematics (Springer, Heidelberg, 2010)Google Scholar
  58. 58.
    B. Savas, I.S. Dhillon, Clustered low rank approximation of graphs in information science applications, in SDM, pp. 164–175 (2011)Google Scholar
  59. 59.
    R. Schneider and A. Uschmajew. Approximation rates for the hierarchical tensor format in periodic Sobolev spaces. J. Complex. 30 (2), 56–71 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  60. 60.
    Z. Sheng, P. Dewilde, S. Chandrasekaran, Algorithms to solve hierarchically semi-separable systems, in System Theory, the Schur Algorithm and Multidimensional Analysis, ed. by D. Alpay, V. Vinnikov. Operator Theory: Advances and Applications, vol. 176 (Birkhäuser Basel, 2007), pp. 255–294Google Scholar
  61. 61.
    S. Si, C.-J. Hsieh, I.S. Dhillon, Memory efficient kernel approximation, in Proceedings of the 31st International Conference on Machine Learning (ICML), pp. 701–709 (2014)Google Scholar
  62. 62.
    H.D. Simon, H. Zha, Low-rank matrix approximation using the Lanczos bidiagonalization process with applications. SIAM J. Sci. Comput. 21 (6), 2257–2274 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  63. 63.
    G.W. Stewart, Perturbation theory for the singular value decomposition, in SVD and Signal Processing II: Algorithms, Analysis and Applications, ed. by R.J. Vaccaro (Elsevier, Amsterdam, 1991)Google Scholar
  64. 64.
    E.E. Tyrtyshnikov, Incomplete cross approximation in the mosaic-skeleton method. Computing 64 (4), 367–380 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  65. 65.
    R. Vandebril, M. Van Barel, N. Mastronardi, A note on the representation and definition of semiseparable matrices. Numer. Linear Algebra Appl. 12 (8), 839–858 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  66. 66.
    R. Vandebril, M. Van Barel, N. Mastronardi, Matrix Computations and Semiseparable Matrices, vol. 1 (Johns Hopkins University Press, Baltimore, MD, 2008)zbMATHGoogle Scholar
  67. 67.
    R. Vandebril, M. Van Barel, N. Mastronardi, Matrix Computations and Semiseparable Matrices, vol. II (Johns Hopkins University Press, Baltimore, MD, 2008)zbMATHGoogle Scholar
  68. 68.
    J. Vogel, J. Xia, S. Cauley, B. Venkataramanan, Superfast divide-and-conquer method and perturbation analysis for structured eigenvalue solutions. SIAM J. Sci. Comput. 38 (3), A1358–A1382 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  69. 69.
    S. Wang, X.S. Li, F. Rouet, J. Xia, M.V. de Hoop, A parallel geometric multifrontal solver using hierarchically semiseparable structure. ACM Trans. Math. Softw. 42 (3), 21:1–21:21 (2016)Google Scholar
  70. 70.
    R. Wang, R. Li, M.W. Mahoney, E. Darve, Structured block basis factorization for scalable kernel matrix evaluation. Preprint arXiv:1505.00398 (2015)Google Scholar
  71. 71.
    P.-Å. Wedin, Perturbation bounds in connection with singular value decomposition. Nordisk Tidskr. Informationsbehandling (BIT) 12, 99–111 (1972)MathSciNetzbMATHGoogle Scholar
  72. 72.
    J. Xia, S. Chandrasekaran, M. Gu, X.S. Li, Fast algorithms for hierarchically semiseparable matrices. Numer. Linear Algebra Appl. 17 (6), 953–976 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  73. 73.
    R. Yokota, G. Turkiyyah, D. Keyes, Communication complexity of the fast multipole method and its algebraic variants. Preprint arXiv:1406.1974 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.ANCHP, EPF LausanneLausanneSwitzerland

Personalised recommendations