An \(\mathcal O (N \log N)\) Fast Direct Solver for Partial Hierarchically Semi-Separable Matrices
- 981 Downloads
- 46 Citations
Abstract
This article describes a fast direct solver (i.e., not iterative) for partial hierarchically semi-separable systems. This solver requires a storage of \(\mathcal O (N \log N)\) and has a computational complexity of \(\mathcal O (N \log N)\) arithmetic operations. The numerical benchmarks presented illustrate the method in the context of interpolation using radial basis functions. The key ingredients behind this fast solver are recursion, efficient low rank factorization using Chebyshev interpolation, and the Sherman–Morrison–Woodbury formula. The algorithm and the analysis are worked out in detail. The performance of the algorithm is illustrated for a variety of radial basis functions and target accuracies.
Keywords
Fast direct solver Numerical linear algebra Partial hierarchically semi-separable representation Hierarchical matrix Radial basis functionMathematics Subject Classification (2000)
65F05Notes
Acknowledgments
Sivaram Ambikasaran would like to thank Krithika Narayanaswamy for proof reading the paper and helping in generating the figures.
References
- 1.Andrianakis, I., Challenor, P.: The effect of the nugget on gaussian process emulators of computer models. Comput. Stat. Data Anal. 56(12), 4215–4228 (2012)Google Scholar
- 2.Arnoldi, W.: The principle of minimized iterations in the solution of the matrix eigenvalue problem. Quart. Appl. Math. 9(1), 17–29 (1951)MathSciNetMATHGoogle Scholar
- 3.Barnes, J., Hut, P.: A hierarchical \({\cal {O}}({N} \log {N})\) force-calculation algorithm. Nature 324(4), 446–449 (1986)Google Scholar
- 4.Baxter, B.: The interpolation theory of radial basis functions. ArXiv, preprint arXiv:1006.2443 (2010)Google Scholar
- 5.Beatson, R., Cherrie, J., Mouat, C.: Fast fitting of radial basis functions: methods based on preconditioned GMRES iteration. Adv. Comput. Math. 11(2), 253–270 (1999)MathSciNetCrossRefMATHGoogle Scholar
- 6.Beatson, R., Greengard, L.: A short course on fast multipole methods. Wavelets, Multilevel Methods Elliptic PDEs pp. 1–37 (1997)Google Scholar
- 7.Beatson, R., Newsam, G.: Fast evaluation of radial basis functions: I. Comput. Math. Appl. 24(12), 7–19 (1992)MathSciNetCrossRefMATHGoogle Scholar
- 8.Billings, S., Beatson, R., Newsam, G.: Interpolation of geophysical data using continuous global surfaces. Geophysics 67(6), 1810–1822 (2002)Google Scholar
- 9.Börm, S., Grasedyck, L., Hackbusch, W.: Hierarchical matrices. In: Lecture notes 21 (2005)Google Scholar
- 10.Buhmann, M.: Radial Basis Functions: Theory and Implementations, vol. 12. Cambridge University Press, Cambridge (2003)CrossRefGoogle Scholar
- 11.Chandrasekaran, S., Dewilde, P., Gu, M., Pals, T., Sun, X., van der Veen, A., White, D.: Some fast algorithms for sequentially semiseparable representations. SIAM J. Matrix Anal. Appl. 27(2), 341–364 (2006)Google Scholar
- 12.Chandrasekaran, S., Gu, M., Pals, T.: A fast ULV decomposition solver for hierarchically semiseparable representations. SIAM J. Matrix Anal. Appl. 28(3), 603–622 (2006)Google Scholar
- 13.Chen, K.: An analysis of sparse approximate inverse preconditioners for boundary integral equations. SIAM J. Matrix Anal. Appl. 22, 1058 (2001)MathSciNetCrossRefMATHGoogle Scholar
- 14.Cheng, H., Gimbutas, Z., Martinsson, P., Rokhlin, V.: On the compression of low rank matrices. SIAM J. Sci. Comput. 26(4), 1389–1404 (2005)MathSciNetCrossRefMATHGoogle Scholar
- 15.Cheng, H., Greengard, L., Rokhlin, V.: A fast adaptive multipole algorithm in three dimensions. J. Comput. Phys. 155(2), 468–498 (1999)MathSciNetCrossRefMATHGoogle Scholar
- 16.Coifman, R., Rokhlin, V., Wandzura, S.: The fast multipole method for the wave equation: a pedestrian prescription. IEEE Antennas Propag. Mag. 35(3), 7–12 (1993)CrossRefGoogle Scholar
- 17.Darve, E.: The fast multipole method: numerical implementation. J. Comput. Phys. 160(1), 195–240 (2000)MathSciNetCrossRefMATHGoogle Scholar
- 18.Darve, E.: The fast multipole method. I: error analysis and asymptotic complexity. SIAM J. Numer. Anal. 38(1), 98–128 (2000)Google Scholar
- 19.Davis, G., Morris, M.: Six factors which affect the condition number of matrices associated with kriging. Math. Geol. 29(5), 669–683 (1997)CrossRefGoogle Scholar
- 20.De Boer, A., Van der Schoot, M., Bijl, H.: Mesh deformation based on radial basis function interpolation. Comput. Struct. 85(11–14), 784–795 (2007)CrossRefGoogle Scholar
- 21.Dietrich, C., Newsam, G.: Efficient generation of conditional simulations by Chebyshev matrix polynomial approximations to the symmetric square root of the covariance matrix. Math. Geol. 27(2), 207–228 (1995)MathSciNetCrossRefMATHGoogle Scholar
- 22.Fong, W., Darve, E.: The black-box fast multipole method. J. Comput. Phys. 228(23), 8712–8725 (2009)MathSciNetCrossRefMATHGoogle Scholar
- 23.Freund, R.: A transpose-free quasi-minimal residual algorithm for non-Hermitian linear systems. SIAM J. Sci. Comput. 14, 470–482 (1993)Google Scholar
- 24.Freund, R., Nachtigal, N.: QMR: a quasi-minimal residual method for non-Hermitian linear systems. Numerische Mathematik 60(1), 315–339 (1991)MathSciNetCrossRefMATHGoogle Scholar
- 25.Frieze, A., Kannan, R., Vempala, S.: Fast Monte–Carlo algorithms for finding low-rank approximations. J. ACM (JACM) 51(6), 1025–1041 (2004)MathSciNetCrossRefMATHGoogle Scholar
- 26.Gillman, A., Young, P., Martinsson, P.: A direct solver with \({\cal {O}}(N)\) complexity for integral equations on one-dimensional domains. ArXiv, preprint arXiv:1105.5372 (2011)Google Scholar
- 27.Golub, G., Van Loan, C.: Matrix Computations, vol. 3. Johns Hopkins University Press, Baltimore (1996)MATHGoogle Scholar
- 28.Goreinov, S., Tyrtyshnikov, E., Zamarashkin, N.: A theory of pseudoskeleton approximations. Linear Algebra Appl. 261(1–3), 1–21 (1997)MathSciNetCrossRefMATHGoogle Scholar
- 29.Grasedyck, L., Hackbusch, W.: Construction and arithmetics of \({\cal {H}}\)-matrices. Computing 70(4), 295–334 (2003)MathSciNetCrossRefMATHGoogle Scholar
- 30.Greengard, L., Gueyffier, D., Martinsson, P., Rokhlin, V.: Fast direct solvers for integral equations in complex three-dimensional domains. Acta Numerica 18(1), 243–275 (2009)MathSciNetCrossRefMATHGoogle Scholar
- 31.Greengard, L., Rokhlin, V.: A fast algorithm for particle simulations. J. Comput. Phys. 73(2), 325–348 (1987)MathSciNetCrossRefMATHGoogle Scholar
- 32.Greengard, L., Rokhlin, V.: A new version of the fast multipole method for the Laplace equation in three dimensions. Acta Numerica 6(1), 229–269 (1997)MathSciNetCrossRefGoogle Scholar
- 33.Gu, M., Eisenstat, S.: Efficient algorithms for computing a strong rank-revealing QR factorization. SIAM J. Sci. Comput. 17(4), 848–869 (1996)MathSciNetCrossRefMATHGoogle Scholar
- 34.Guennebaud, G., Jacob, B., et al.: Eigen v3. http://eigen.tuxfamily.org (2010)
- 35.Gumerov, N., Duraiswami, R.: Fast radial basis function interpolation via preconditioned Krylov iteration. SIAM J. Sci. Comput. 29(5), 1876–1899 (2007)MathSciNetCrossRefMATHGoogle Scholar
- 36.Hackbusch, W.: A sparse matrix arithmetic based on \({\cal {H}}\)-matrices. Part I: introduction to \({\cal {H}}\)-matrices. Computing 62(2), 89–108 (1999)MathSciNetCrossRefMATHGoogle Scholar
- 37.Hackbusch, W., Börm, S.: Data-sparse approximation by adaptive \({\cal {H}}^2\)-matrices. Computing 69(1), 1–35 (2002)MathSciNetCrossRefMATHGoogle Scholar
- 38.Hackbusch, W., Khoromskij, B.: A sparse \(\cal {H}\)-matrix arithmetic. Computing 64(1), 21–47 (2000)MathSciNetMATHGoogle Scholar
- 39.Hackbusch, W., Nowak, Z.: On the fast matrix multiplication in the boundary element method by panel clustering. Numerische Mathematik 54(4), 463–491 (1989)MathSciNetCrossRefMATHGoogle Scholar
- 40.Hager, W.: Updating the inverse of a matrix. SIAM Rev. 31(2), 221–239 (1989)Google Scholar
- 41.Hestenes, M., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)Google Scholar
- 42.Kong, W., Bremer, J., Rokhlin, V.: An adaptive fast direct solver for boundary integral equations in two dimensions. Appl. Comput. Harm. Anal. 31(3), 346–369 (2011)Google Scholar
- 43.Liberty, E., Woolfe, F., Martinsson, P., Rokhlin, V., Tygert, M.: Randomized algorithms for the low-rank approximation of matrices. Proc. Natl. Acad. Sci. 104, 20167–20172 (2007)Google Scholar
- 44.Martinsson, P.: A fast direct solver for a class of elliptic partial differential equations. J. Sci. Comput. 38(3), 316–330 (2009)MathSciNetCrossRefMATHGoogle Scholar
- 45.Martinsson, P., Rokhlin, V.: A fast direct solver for boundary integral equations in two dimensions. J. Comput. Phys. 205(1), 1–23 (2005)MathSciNetCrossRefMATHGoogle Scholar
- 46.Messner, M., Schanz, M., Darve, E.: Fast directional multilevel summation for oscillatory kernels based on Chebyshev interpolation. J. Comp. Phys. 231(4), 1175–1196 (2012)Google Scholar
- 47.Miranian, L., Gu, M.: Strong rank revealing LU factorizations. Linear Algebra Appl. 367, 1–16 (2003)MathSciNetCrossRefMATHGoogle Scholar
- 48.Nishimura, N.: Fast multipole accelerated boundary integral equation methods. Appl. Mech. Rev. 55(4), 299–324 (2002)Google Scholar
- 49.O’Dowd, R.: Conditioning of coefficient matrices of ordinary kriging. Math. Geol. 23(5), 721–739 (1991)MathSciNetCrossRefMATHGoogle Scholar
- 50.Paige, C., Saunders, M.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12(4), 617–629 (1975)Google Scholar
- 51.Pan, C.T.: On the existence and computation of rank-revealing LU factorizations. Linear Algebra Appl. 316(1), 199–222 (2000)MathSciNetCrossRefMATHGoogle Scholar
- 52.Rjasanow, S.: Adaptive cross approximation of dense matrices. In: IABEM 2002, International Association for Boundary Element Methods (2002)Google Scholar
- 53.Saad, Y., Schultz, M.: GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7(3), 856–869 (1986)MathSciNetCrossRefMATHGoogle Scholar
- 54.Schaback, R.: Creating surfaces from scattered data using radial basis functions. Math. Methods Curves Surf. 477–496 (1995)Google Scholar
- 55.Schmitz, P., Ying, L.: A fast direct solver for elliptic problems on general meshes in 2D. J. Comput. Phys. 231(4), 1314–1338 (2012)Google Scholar
- 56.Schmitz, P., Ying, L.: A fast direct solver for elliptic problems on Cartesian meshes in 3D (in review) (2012)Google Scholar
- 57.Vandebril, R., Barel, M., Golub, G., Mastronardi, N.: A bibliography on semiseparable matrices. Calcolo 42(3), 249–270 (2005)MathSciNetCrossRefGoogle Scholar
- 58.Vavasis, S.: Preconditioning for boundary integral equations. SIAM J. Matrix Anal. Appl. 13, 905–925 (1992)Google Scholar
- 59.Van der Vorst, H.: Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of non-symmetric linear systems. SIAM J. Sci. Stat. Comput. 13(2), 631–644 (1992)Google Scholar
- 60.Wang, J., Liu, G.: A point interpolation meshless method based on radial basis functions. Int. J. Numer. Methods Eng. 54(11), 1623–1648 (2002)CrossRefMATHGoogle Scholar
- 61.Woodbury, M.A.: Inverting modified matrices. Statistical Research Group, Memo. Rep. no. 42, Princeton University, Princeton (1950)Google Scholar
- 62.Woolfe, F., Liberty, E., Rokhlin, V., Tygert, M.: A fast randomized algorithm for the approximation of matrices. Appl. Comput. Harmon. Anal. 25(3), 335–366 (2008)MathSciNetCrossRefMATHGoogle Scholar
- 63.Wu, Z., Schaback, R.: Local error estimates for radial basis function interpolation of scattered data. IMA J. Numer. Anal. 13(1), 13–27 (1993)MathSciNetCrossRefMATHGoogle Scholar
- 64.Xia, J., Chandrasekaran, S., Gu, M., Li, X.: Fast algorithms for hierarchically semiseparable matrices. Numer. Linear Algebra Appl. 17(6), 953–976 (2010)MathSciNetCrossRefMATHGoogle Scholar
- 65.Ying, L., Biros, G., Zorin, D.: A kernel-independent adaptive fast multipole algorithm in two and three dimensions. J. Comput. Phys. 196(2), 591–626 (2004)MathSciNetCrossRefMATHGoogle Scholar