Journal of Scientific Computing

, Volume 57, Issue 3, pp 477–501 | Cite as

An \(\mathcal O (N \log N)\)  Fast Direct Solver for Partial Hierarchically Semi-Separable Matrices

With Application to Radial Basis Function Interpolation
Article
First Online:
Received:
Revised:
Accepted:

Abstract

This article describes a fast direct solver (i.e., not iterative) for partial hierarchically semi-separable systems. This solver requires a storage of \(\mathcal O (N \log N)\) and has a computational complexity of \(\mathcal O (N \log N)\) arithmetic operations. The numerical benchmarks presented illustrate the method in the context of interpolation using radial basis functions. The key ingredients behind this fast solver are recursion, efficient low rank factorization using Chebyshev interpolation, and the Sherman–Morrison–Woodbury formula. The algorithm and the analysis are worked out in detail. The performance of the algorithm is illustrated for a variety of radial basis functions and target accuracies.

Keywords

Fast direct solver Numerical linear algebra Partial hierarchically semi-separable representation Hierarchical matrix Radial basis function 

Mathematics Subject Classification (2000)

65F05 
This is a preview of subscription content, log in to check access

Notes

Acknowledgments

Sivaram Ambikasaran would like to thank Krithika Narayanaswamy for proof reading the paper and helping in generating the figures.

References

  1. 1.
    Andrianakis, I., Challenor, P.: The effect of the nugget on gaussian process emulators of computer models. Comput. Stat. Data Anal. 56(12), 4215–4228 (2012)Google Scholar
  2. 2.
    Arnoldi, W.: The principle of minimized iterations in the solution of the matrix eigenvalue problem. Quart. Appl. Math. 9(1), 17–29 (1951)MathSciNetMATHGoogle Scholar
  3. 3.
    Barnes, J., Hut, P.: A hierarchical \({\cal {O}}({N} \log {N})\) force-calculation algorithm. Nature 324(4), 446–449 (1986)Google Scholar
  4. 4.
    Baxter, B.: The interpolation theory of radial basis functions. ArXiv, preprint arXiv:1006.2443 (2010)Google Scholar
  5. 5.
    Beatson, R., Cherrie, J., Mouat, C.: Fast fitting of radial basis functions: methods based on preconditioned GMRES iteration. Adv. Comput. Math. 11(2), 253–270 (1999)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Beatson, R., Greengard, L.: A short course on fast multipole methods. Wavelets, Multilevel Methods Elliptic PDEs pp. 1–37 (1997)Google Scholar
  7. 7.
    Beatson, R., Newsam, G.: Fast evaluation of radial basis functions: I. Comput. Math. Appl. 24(12), 7–19 (1992)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Billings, S., Beatson, R., Newsam, G.: Interpolation of geophysical data using continuous global surfaces. Geophysics 67(6), 1810–1822 (2002)Google Scholar
  9. 9.
    Börm, S., Grasedyck, L., Hackbusch, W.: Hierarchical matrices. In: Lecture notes 21 (2005)Google Scholar
  10. 10.
    Buhmann, M.: Radial Basis Functions: Theory and Implementations, vol. 12. Cambridge University Press, Cambridge (2003)CrossRefGoogle Scholar
  11. 11.
    Chandrasekaran, S., Dewilde, P., Gu, M., Pals, T., Sun, X., van der Veen, A., White, D.: Some fast algorithms for sequentially semiseparable representations. SIAM J. Matrix Anal. Appl. 27(2), 341–364 (2006)Google Scholar
  12. 12.
    Chandrasekaran, S., Gu, M., Pals, T.: A fast ULV decomposition solver for hierarchically semiseparable representations. SIAM J. Matrix Anal. Appl. 28(3), 603–622 (2006)Google Scholar
  13. 13.
    Chen, K.: An analysis of sparse approximate inverse preconditioners for boundary integral equations. SIAM J. Matrix Anal. Appl. 22, 1058 (2001)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Cheng, H., Gimbutas, Z., Martinsson, P., Rokhlin, V.: On the compression of low rank matrices. SIAM J. Sci. Comput. 26(4), 1389–1404 (2005)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Cheng, H., Greengard, L., Rokhlin, V.: A fast adaptive multipole algorithm in three dimensions. J. Comput. Phys. 155(2), 468–498 (1999)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Coifman, R., Rokhlin, V., Wandzura, S.: The fast multipole method for the wave equation: a pedestrian prescription. IEEE Antennas Propag. Mag. 35(3), 7–12 (1993)CrossRefGoogle Scholar
  17. 17.
    Darve, E.: The fast multipole method: numerical implementation. J. Comput. Phys. 160(1), 195–240 (2000)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Darve, E.: The fast multipole method. I: error analysis and asymptotic complexity. SIAM J. Numer. Anal. 38(1), 98–128 (2000)Google Scholar
  19. 19.
    Davis, G., Morris, M.: Six factors which affect the condition number of matrices associated with kriging. Math. Geol. 29(5), 669–683 (1997)CrossRefGoogle Scholar
  20. 20.
    De Boer, A., Van der Schoot, M., Bijl, H.: Mesh deformation based on radial basis function interpolation. Comput. Struct. 85(11–14), 784–795 (2007)CrossRefGoogle Scholar
  21. 21.
    Dietrich, C., Newsam, G.: Efficient generation of conditional simulations by Chebyshev matrix polynomial approximations to the symmetric square root of the covariance matrix. Math. Geol. 27(2), 207–228 (1995)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Fong, W., Darve, E.: The black-box fast multipole method. J. Comput. Phys. 228(23), 8712–8725 (2009)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Freund, R.: A transpose-free quasi-minimal residual algorithm for non-Hermitian linear systems. SIAM J. Sci. Comput. 14, 470–482 (1993)Google Scholar
  24. 24.
    Freund, R., Nachtigal, N.: QMR: a quasi-minimal residual method for non-Hermitian linear systems. Numerische Mathematik 60(1), 315–339 (1991)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Frieze, A., Kannan, R., Vempala, S.: Fast Monte–Carlo algorithms for finding low-rank approximations. J. ACM (JACM) 51(6), 1025–1041 (2004)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Gillman, A., Young, P., Martinsson, P.: A direct solver with \({\cal {O}}(N)\) complexity for integral equations on one-dimensional domains. ArXiv, preprint arXiv:1105.5372 (2011)Google Scholar
  27. 27.
    Golub, G., Van Loan, C.: Matrix Computations, vol. 3. Johns Hopkins University Press, Baltimore (1996)MATHGoogle Scholar
  28. 28.
    Goreinov, S., Tyrtyshnikov, E., Zamarashkin, N.: A theory of pseudoskeleton approximations. Linear Algebra Appl. 261(1–3), 1–21 (1997)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Grasedyck, L., Hackbusch, W.: Construction and arithmetics of \({\cal {H}}\)-matrices. Computing 70(4), 295–334 (2003)MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Greengard, L., Gueyffier, D., Martinsson, P., Rokhlin, V.: Fast direct solvers for integral equations in complex three-dimensional domains. Acta Numerica 18(1), 243–275 (2009)MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Greengard, L., Rokhlin, V.: A fast algorithm for particle simulations. J. Comput. Phys. 73(2), 325–348 (1987)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Greengard, L., Rokhlin, V.: A new version of the fast multipole method for the Laplace equation in three dimensions. Acta Numerica 6(1), 229–269 (1997)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Gu, M., Eisenstat, S.: Efficient algorithms for computing a strong rank-revealing QR factorization. SIAM J. Sci. Comput. 17(4), 848–869 (1996)MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Guennebaud, G., Jacob, B., et al.: Eigen v3. http://eigen.tuxfamily.org (2010)
  35. 35.
    Gumerov, N., Duraiswami, R.: Fast radial basis function interpolation via preconditioned Krylov iteration. SIAM J. Sci. Comput. 29(5), 1876–1899 (2007)MathSciNetCrossRefMATHGoogle Scholar
  36. 36.
    Hackbusch, W.: A sparse matrix arithmetic based on \({\cal {H}}\)-matrices. Part I: introduction to \({\cal {H}}\)-matrices. Computing 62(2), 89–108 (1999)MathSciNetCrossRefMATHGoogle Scholar
  37. 37.
    Hackbusch, W., Börm, S.: Data-sparse approximation by adaptive \({\cal {H}}^2\)-matrices. Computing 69(1), 1–35 (2002)MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    Hackbusch, W., Khoromskij, B.: A sparse \(\cal {H}\)-matrix arithmetic. Computing 64(1), 21–47 (2000)MathSciNetMATHGoogle Scholar
  39. 39.
    Hackbusch, W., Nowak, Z.: On the fast matrix multiplication in the boundary element method by panel clustering. Numerische Mathematik 54(4), 463–491 (1989)MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Hager, W.: Updating the inverse of a matrix. SIAM Rev. 31(2), 221–239 (1989)Google Scholar
  41. 41.
    Hestenes, M., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)Google Scholar
  42. 42.
    Kong, W., Bremer, J., Rokhlin, V.: An adaptive fast direct solver for boundary integral equations in two dimensions. Appl. Comput. Harm. Anal. 31(3), 346–369 (2011)Google Scholar
  43. 43.
    Liberty, E., Woolfe, F., Martinsson, P., Rokhlin, V., Tygert, M.: Randomized algorithms for the low-rank approximation of matrices. Proc. Natl. Acad. Sci. 104, 20167–20172 (2007)Google Scholar
  44. 44.
    Martinsson, P.: A fast direct solver for a class of elliptic partial differential equations. J. Sci. Comput. 38(3), 316–330 (2009)MathSciNetCrossRefMATHGoogle Scholar
  45. 45.
    Martinsson, P., Rokhlin, V.: A fast direct solver for boundary integral equations in two dimensions. J. Comput. Phys. 205(1), 1–23 (2005)MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
    Messner, M., Schanz, M., Darve, E.: Fast directional multilevel summation for oscillatory kernels based on Chebyshev interpolation. J. Comp. Phys. 231(4), 1175–1196 (2012)Google Scholar
  47. 47.
    Miranian, L., Gu, M.: Strong rank revealing LU factorizations. Linear Algebra Appl. 367, 1–16 (2003)MathSciNetCrossRefMATHGoogle Scholar
  48. 48.
    Nishimura, N.: Fast multipole accelerated boundary integral equation methods. Appl. Mech. Rev. 55(4), 299–324 (2002)Google Scholar
  49. 49.
    O’Dowd, R.: Conditioning of coefficient matrices of ordinary kriging. Math. Geol. 23(5), 721–739 (1991)MathSciNetCrossRefMATHGoogle Scholar
  50. 50.
    Paige, C., Saunders, M.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12(4), 617–629 (1975)Google Scholar
  51. 51.
    Pan, C.T.: On the existence and computation of rank-revealing LU factorizations. Linear Algebra Appl. 316(1), 199–222 (2000)MathSciNetCrossRefMATHGoogle Scholar
  52. 52.
    Rjasanow, S.: Adaptive cross approximation of dense matrices. In: IABEM 2002, International Association for Boundary Element Methods (2002)Google Scholar
  53. 53.
    Saad, Y., Schultz, M.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7(3), 856–869 (1986)MathSciNetCrossRefMATHGoogle Scholar
  54. 54.
    Schaback, R.: Creating surfaces from scattered data using radial basis functions. Math. Methods Curves Surf. 477–496 (1995)Google Scholar
  55. 55.
    Schmitz, P., Ying, L.: A fast direct solver for elliptic problems on general meshes in 2D. J. Comput. Phys. 231(4), 1314–1338 (2012)Google Scholar
  56. 56.
    Schmitz, P., Ying, L.: A fast direct solver for elliptic problems on Cartesian meshes in 3D (in review) (2012)Google Scholar
  57. 57.
    Vandebril, R., Barel, M., Golub, G., Mastronardi, N.: A bibliography on semiseparable matrices. Calcolo 42(3), 249–270 (2005)MathSciNetCrossRefGoogle Scholar
  58. 58.
    Vavasis, S.: Preconditioning for boundary integral equations. SIAM J. Matrix Anal. Appl. 13, 905–925 (1992)Google Scholar
  59. 59.
    Van der Vorst, H.: Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of non-symmetric linear systems. SIAM J. Sci. Stat. Comput. 13(2), 631–644 (1992)Google Scholar
  60. 60.
    Wang, J., Liu, G.: A point interpolation meshless method based on radial basis functions. Int. J. Numer. Methods Eng. 54(11), 1623–1648 (2002)CrossRefMATHGoogle Scholar
  61. 61.
    Woodbury, M.A.: Inverting modified matrices. Statistical Research Group, Memo. Rep. no. 42, Princeton University, Princeton (1950)Google Scholar
  62. 62.
    Woolfe, F., Liberty, E., Rokhlin, V., Tygert, M.: A fast randomized algorithm for the approximation of matrices. Appl. Comput. Harmon. Anal. 25(3), 335–366 (2008)MathSciNetCrossRefMATHGoogle Scholar
  63. 63.
    Wu, Z., Schaback, R.: Local error estimates for radial basis function interpolation of scattered data. IMA J. Numer. Anal. 13(1), 13–27 (1993)MathSciNetCrossRefMATHGoogle Scholar
  64. 64.
    Xia, J., Chandrasekaran, S., Gu, M., Li, X.: Fast algorithms for hierarchically semiseparable matrices. Numer. Linear Algebra Appl. 17(6), 953–976 (2010)MathSciNetCrossRefMATHGoogle Scholar
  65. 65.
    Ying, L., Biros, G., Zorin, D.: A kernel-independent adaptive fast multipole algorithm in two and three dimensions. J. Comput. Phys. 196(2), 591–626 (2004)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Institute for Computational and Mathematical Engineering, Huang Engineering Center 053BStanford UniversityStanfordUSA
  2. 2.Mechanical EngineeringStanford UniversityStanfordUSA

Personalised recommendations