An \(\mathcal O (N \log N)\)  Fast Direct Solver for Partial Hierarchically Semi-Separable Matrices

With Application to Radial Basis Function Interpolation

Abstract

This article describes a fast direct solver (i.e., not iterative) for partial hierarchically semi-separable systems. This solver requires a storage of \(\mathcal O (N \log N)\) and has a computational complexity of \(\mathcal O (N \log N)\) arithmetic operations. The numerical benchmarks presented illustrate the method in the context of interpolation using radial basis functions. The key ingredients behind this fast solver are recursion, efficient low rank factorization using Chebyshev interpolation, and the Sherman–Morrison–Woodbury formula. The algorithm and the analysis are worked out in detail. The performance of the algorithm is illustrated for a variety of radial basis functions and target accuracies.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

References

  1. 1.

    Andrianakis, I., Challenor, P.: The effect of the nugget on gaussian process emulators of computer models. Comput. Stat. Data Anal. 56(12), 4215–4228 (2012)

    Google Scholar 

  2. 2.

    Arnoldi, W.: The principle of minimized iterations in the solution of the matrix eigenvalue problem. Quart. Appl. Math. 9(1), 17–29 (1951)

    MathSciNet  MATH  Google Scholar 

  3. 3.

    Barnes, J., Hut, P.: A hierarchical \({\cal {O}}({N} \log {N})\) force-calculation algorithm. Nature 324(4), 446–449 (1986)

    Google Scholar 

  4. 4.

    Baxter, B.: The interpolation theory of radial basis functions. ArXiv, preprint arXiv:1006.2443 (2010)

  5. 5.

    Beatson, R., Cherrie, J., Mouat, C.: Fast fitting of radial basis functions: methods based on preconditioned GMRES iteration. Adv. Comput. Math. 11(2), 253–270 (1999)

    MathSciNet  Article  MATH  Google Scholar 

  6. 6.

    Beatson, R., Greengard, L.: A short course on fast multipole methods. Wavelets, Multilevel Methods Elliptic PDEs pp. 1–37 (1997)

  7. 7.

    Beatson, R., Newsam, G.: Fast evaluation of radial basis functions: I. Comput. Math. Appl. 24(12), 7–19 (1992)

    MathSciNet  Article  MATH  Google Scholar 

  8. 8.

    Billings, S., Beatson, R., Newsam, G.: Interpolation of geophysical data using continuous global surfaces. Geophysics 67(6), 1810–1822 (2002)

    Google Scholar 

  9. 9.

    Börm, S., Grasedyck, L., Hackbusch, W.: Hierarchical matrices. In: Lecture notes 21 (2005)

  10. 10.

    Buhmann, M.: Radial Basis Functions: Theory and Implementations, vol. 12. Cambridge University Press, Cambridge (2003)

    Google Scholar 

  11. 11.

    Chandrasekaran, S., Dewilde, P., Gu, M., Pals, T., Sun, X., van der Veen, A., White, D.: Some fast algorithms for sequentially semiseparable representations. SIAM J. Matrix Anal. Appl. 27(2), 341–364 (2006)

    Google Scholar 

  12. 12.

    Chandrasekaran, S., Gu, M., Pals, T.: A fast ULV decomposition solver for hierarchically semiseparable representations. SIAM J. Matrix Anal. Appl. 28(3), 603–622 (2006)

    Google Scholar 

  13. 13.

    Chen, K.: An analysis of sparse approximate inverse preconditioners for boundary integral equations. SIAM J. Matrix Anal. Appl. 22, 1058 (2001)

    MathSciNet  Article  MATH  Google Scholar 

  14. 14.

    Cheng, H., Gimbutas, Z., Martinsson, P., Rokhlin, V.: On the compression of low rank matrices. SIAM J. Sci. Comput. 26(4), 1389–1404 (2005)

    MathSciNet  Article  MATH  Google Scholar 

  15. 15.

    Cheng, H., Greengard, L., Rokhlin, V.: A fast adaptive multipole algorithm in three dimensions. J. Comput. Phys. 155(2), 468–498 (1999)

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    Coifman, R., Rokhlin, V., Wandzura, S.: The fast multipole method for the wave equation: a pedestrian prescription. IEEE Antennas Propag. Mag. 35(3), 7–12 (1993)

    Article  Google Scholar 

  17. 17.

    Darve, E.: The fast multipole method: numerical implementation. J. Comput. Phys. 160(1), 195–240 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  18. 18.

    Darve, E.: The fast multipole method. I: error analysis and asymptotic complexity. SIAM J. Numer. Anal. 38(1), 98–128 (2000)

    Google Scholar 

  19. 19.

    Davis, G., Morris, M.: Six factors which affect the condition number of matrices associated with kriging. Math. Geol. 29(5), 669–683 (1997)

    Article  Google Scholar 

  20. 20.

    De Boer, A., Van der Schoot, M., Bijl, H.: Mesh deformation based on radial basis function interpolation. Comput. Struct. 85(11–14), 784–795 (2007)

    Article  Google Scholar 

  21. 21.

    Dietrich, C., Newsam, G.: Efficient generation of conditional simulations by Chebyshev matrix polynomial approximations to the symmetric square root of the covariance matrix. Math. Geol. 27(2), 207–228 (1995)

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    Fong, W., Darve, E.: The black-box fast multipole method. J. Comput. Phys. 228(23), 8712–8725 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  23. 23.

    Freund, R.: A transpose-free quasi-minimal residual algorithm for non-Hermitian linear systems. SIAM J. Sci. Comput. 14, 470–482 (1993)

    Google Scholar 

  24. 24.

    Freund, R., Nachtigal, N.: QMR: a quasi-minimal residual method for non-Hermitian linear systems. Numerische Mathematik 60(1), 315–339 (1991)

    MathSciNet  Article  MATH  Google Scholar 

  25. 25.

    Frieze, A., Kannan, R., Vempala, S.: Fast Monte–Carlo algorithms for finding low-rank approximations. J. ACM (JACM) 51(6), 1025–1041 (2004)

    MathSciNet  Article  MATH  Google Scholar 

  26. 26.

    Gillman, A., Young, P., Martinsson, P.: A direct solver with \({\cal {O}}(N)\) complexity for integral equations on one-dimensional domains. ArXiv, preprint arXiv:1105.5372 (2011)

  27. 27.

    Golub, G., Van Loan, C.: Matrix Computations, vol. 3. Johns Hopkins University Press, Baltimore (1996)

    Google Scholar 

  28. 28.

    Goreinov, S., Tyrtyshnikov, E., Zamarashkin, N.: A theory of pseudoskeleton approximations. Linear Algebra Appl. 261(1–3), 1–21 (1997)

    MathSciNet  Article  MATH  Google Scholar 

  29. 29.

    Grasedyck, L., Hackbusch, W.: Construction and arithmetics of \({\cal {H}}\)-matrices. Computing 70(4), 295–334 (2003)

    MathSciNet  Article  MATH  Google Scholar 

  30. 30.

    Greengard, L., Gueyffier, D., Martinsson, P., Rokhlin, V.: Fast direct solvers for integral equations in complex three-dimensional domains. Acta Numerica 18(1), 243–275 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  31. 31.

    Greengard, L., Rokhlin, V.: A fast algorithm for particle simulations. J. Comput. Phys. 73(2), 325–348 (1987)

    MathSciNet  Article  MATH  Google Scholar 

  32. 32.

    Greengard, L., Rokhlin, V.: A new version of the fast multipole method for the Laplace equation in three dimensions. Acta Numerica 6(1), 229–269 (1997)

    MathSciNet  Article  Google Scholar 

  33. 33.

    Gu, M., Eisenstat, S.: Efficient algorithms for computing a strong rank-revealing QR factorization. SIAM J. Sci. Comput. 17(4), 848–869 (1996)

    MathSciNet  Article  MATH  Google Scholar 

  34. 34.

    Guennebaud, G., Jacob, B., et al.: Eigen v3. http://eigen.tuxfamily.org (2010)

  35. 35.

    Gumerov, N., Duraiswami, R.: Fast radial basis function interpolation via preconditioned Krylov iteration. SIAM J. Sci. Comput. 29(5), 1876–1899 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  36. 36.

    Hackbusch, W.: A sparse matrix arithmetic based on \({\cal {H}}\)-matrices. Part I: introduction to \({\cal {H}}\)-matrices. Computing 62(2), 89–108 (1999)

    MathSciNet  Article  MATH  Google Scholar 

  37. 37.

    Hackbusch, W., Börm, S.: Data-sparse approximation by adaptive \({\cal {H}}^2\)-matrices. Computing 69(1), 1–35 (2002)

    MathSciNet  Article  MATH  Google Scholar 

  38. 38.

    Hackbusch, W., Khoromskij, B.: A sparse \(\cal {H}\)-matrix arithmetic. Computing 64(1), 21–47 (2000)

    MathSciNet  MATH  Google Scholar 

  39. 39.

    Hackbusch, W., Nowak, Z.: On the fast matrix multiplication in the boundary element method by panel clustering. Numerische Mathematik 54(4), 463–491 (1989)

    MathSciNet  Article  MATH  Google Scholar 

  40. 40.

    Hager, W.: Updating the inverse of a matrix. SIAM Rev. 31(2), 221–239 (1989)

    Google Scholar 

  41. 41.

    Hestenes, M., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)

    Google Scholar 

  42. 42.

    Kong, W., Bremer, J., Rokhlin, V.: An adaptive fast direct solver for boundary integral equations in two dimensions. Appl. Comput. Harm. Anal. 31(3), 346–369 (2011)

    Google Scholar 

  43. 43.

    Liberty, E., Woolfe, F., Martinsson, P., Rokhlin, V., Tygert, M.: Randomized algorithms for the low-rank approximation of matrices. Proc. Natl. Acad. Sci. 104, 20167–20172 (2007)

    Google Scholar 

  44. 44.

    Martinsson, P.: A fast direct solver for a class of elliptic partial differential equations. J. Sci. Comput. 38(3), 316–330 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  45. 45.

    Martinsson, P., Rokhlin, V.: A fast direct solver for boundary integral equations in two dimensions. J. Comput. Phys. 205(1), 1–23 (2005)

    MathSciNet  Article  MATH  Google Scholar 

  46. 46.

    Messner, M., Schanz, M., Darve, E.: Fast directional multilevel summation for oscillatory kernels based on Chebyshev interpolation. J. Comp. Phys. 231(4), 1175–1196 (2012)

    Google Scholar 

  47. 47.

    Miranian, L., Gu, M.: Strong rank revealing LU factorizations. Linear Algebra Appl. 367, 1–16 (2003)

    MathSciNet  Article  MATH  Google Scholar 

  48. 48.

    Nishimura, N.: Fast multipole accelerated boundary integral equation methods. Appl. Mech. Rev. 55(4), 299–324 (2002)

    Google Scholar 

  49. 49.

    O’Dowd, R.: Conditioning of coefficient matrices of ordinary kriging. Math. Geol. 23(5), 721–739 (1991)

    MathSciNet  Article  MATH  Google Scholar 

  50. 50.

    Paige, C., Saunders, M.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12(4), 617–629 (1975)

    Google Scholar 

  51. 51.

    Pan, C.T.: On the existence and computation of rank-revealing LU factorizations. Linear Algebra Appl. 316(1), 199–222 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  52. 52.

    Rjasanow, S.: Adaptive cross approximation of dense matrices. In: IABEM 2002, International Association for Boundary Element Methods (2002)

  53. 53.

    Saad, Y., Schultz, M.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7(3), 856–869 (1986)

    MathSciNet  Article  MATH  Google Scholar 

  54. 54.

    Schaback, R.: Creating surfaces from scattered data using radial basis functions. Math. Methods Curves Surf. 477–496 (1995)

  55. 55.

    Schmitz, P., Ying, L.: A fast direct solver for elliptic problems on general meshes in 2D. J. Comput. Phys. 231(4), 1314–1338 (2012)

    Google Scholar 

  56. 56.

    Schmitz, P., Ying, L.: A fast direct solver for elliptic problems on Cartesian meshes in 3D (in review) (2012)

  57. 57.

    Vandebril, R., Barel, M., Golub, G., Mastronardi, N.: A bibliography on semiseparable matrices. Calcolo 42(3), 249–270 (2005)

    MathSciNet  Article  Google Scholar 

  58. 58.

    Vavasis, S.: Preconditioning for boundary integral equations. SIAM J. Matrix Anal. Appl. 13, 905–925 (1992)

    Google Scholar 

  59. 59.

    Van der Vorst, H.: Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of non-symmetric linear systems. SIAM J. Sci. Stat. Comput. 13(2), 631–644 (1992)

  60. 60.

    Wang, J., Liu, G.: A point interpolation meshless method based on radial basis functions. Int. J. Numer. Methods Eng. 54(11), 1623–1648 (2002)

    Article  MATH  Google Scholar 

  61. 61.

    Woodbury, M.A.: Inverting modified matrices. Statistical Research Group, Memo. Rep. no. 42, Princeton University, Princeton (1950)

  62. 62.

    Woolfe, F., Liberty, E., Rokhlin, V., Tygert, M.: A fast randomized algorithm for the approximation of matrices. Appl. Comput. Harmon. Anal. 25(3), 335–366 (2008)

    MathSciNet  Article  MATH  Google Scholar 

  63. 63.

    Wu, Z., Schaback, R.: Local error estimates for radial basis function interpolation of scattered data. IMA J. Numer. Anal. 13(1), 13–27 (1993)

    MathSciNet  Article  MATH  Google Scholar 

  64. 64.

    Xia, J., Chandrasekaran, S., Gu, M., Li, X.: Fast algorithms for hierarchically semiseparable matrices. Numer. Linear Algebra Appl. 17(6), 953–976 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  65. 65.

    Ying, L., Biros, G., Zorin, D.: A kernel-independent adaptive fast multipole algorithm in two and three dimensions. J. Comput. Phys. 196(2), 591–626 (2004)

    MathSciNet  Article  MATH  Google Scholar 

Download references

Acknowledgments

Sivaram Ambikasaran would like to thank Krithika Narayanaswamy for proof reading the paper and helping in generating the figures.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Sivaram Ambikasaran.

Additional information

The authors would like to thank the “Army High Performance Computing Research Center” (AHPCRC) and “The Global Climate and Energy Project” (GCEP) at Stanford for supporting the project.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ambikasaran, S., Darve, E. An \(\mathcal O (N \log N)\)  Fast Direct Solver for Partial Hierarchically Semi-Separable Matrices. J Sci Comput 57, 477–501 (2013). https://doi.org/10.1007/s10915-013-9714-z

Download citation

Keywords

  • Fast direct solver
  • Numerical linear algebra
  • Partial hierarchically semi-separable representation
  • Hierarchical matrix
  • Radial basis function

Mathematics Subject Classification (2000)

  • 65F05