Abstract
In this article, we review a few fast algorithms for solving large-scale stochastic inverse problems using Bayesian methods. After a brief introduction to the Bayesian stochastic inverse methodology, we review the following computational techniques, to solve large scale problems: the fast Fourier transform, the fast multipole method (classical and a black-box version), and finallym the hierarchical matrix approach. We emphasize that this is mainly a survey paper presenting a few fast algorithms applicable to large-scale Bayesian inversion techniques, applicable to applications arising from geostatistics. The article is presented at a level accessible to graduate students and computational engineers. Hence, we mainly present the algorithmic ideas and theoretical results.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ambikasaran, S., Li, J., Darve, E., Kitanidis, P.K.: Large-scale stochastic linear inversion using hierarchical matrices (2012). In review
Beatson, R., Greengard, L.: A short course on fast multipole methods. Wavelets, multilevel methods and elliptic PDEs pp. 1–37 (1997)
Beatson, R., Newsam, G.: Fast evaluation of radial basis functions: I. Computers & Mathematics with Applications 24(12), 7–19 (1992)
Bebendorf, M.: Approximation of boundary element matrices. Numerische Mathematik 86(4), 565–589 (2000)
Bebendorf, M.: Hierarchical Matrices: A Means to Efficiently Solve Elliptic Boundary Value Problems, Lecture Notes in Computational Science and Engineering (LNCSE), vol. 63. Springer-Verlag (2008). ISBN 978-3-540-77146-3
Bebendorf, M., Rjasanow, S.: Adaptive low-rank approximation of collocation matrices. Computing 70(1), 1–24 (2003)
Beylkin, G., Coifman, R., Rokhlin, V.: Fast wavelet transforms and numerical algorithms I. Communications on Pure and Applied Mathematics 44(2), 141–183 (1991)
Bjørnstad, O., Falck, W.: Nonparametric spatial covariance functions: estimation and testing. Environmental and Ecological Statistics 8(1), 53–70 (2001)
Börm, S., Grasedyck, L., Hackbusch, W.: Hierarchical matrices. Lecture notes 21 (2003)
Burrus, C., Gopinath, R., Guo, H.: Introduction to wavelets and wavelet transforms: A primer. Recherche 67, 02 (1998)
Chandrasekaran, S., Dewilde, P., Gu, M., Pals, T., Sun, X., van der Veen, A., White, D.: Some fast algorithms for sequentially semiseparable representations. SIAM Journal on Matrix Analysis and Applications 27(2), 341 (2006)
Chandrasekaran, S., Gu, M., Pals, T.: A fast ULV decomposition solver for hierarchically semiseparable representations. SIAM Journal on Matrix Analysis and Applications 28(3), 603 (2006)
Cheng, H., Greengard, L., Rokhlin, V.: A fast adaptive multipole algorithm in three dimensions. Journal of Computational Physics 155(2), 468–498 (1999)
Coifman, R., Rokhlin, V., Wandzura, S.: The fast multipole method for the wave equation: A pedestrian prescription. Antennas and Propagation Magazine, IEEE 35(3), 7–12 (1993)
Cooley, J., Tukey, J.: An algorithm for the machine calculation of complex fourier series. Math. Comput 19(90), 297–301 (1965)
Cornford, D., Csató, L., Opper, M.: Sequential, Bayesian geostatistics: a principled method for large data sets. Geographical Analysis 37(2), 183–199 (2005)
Darve, E.: The fast multipole method: numerical implementation. Journal of Computational Physics 160(1), 195–240 (2000)
Darve, E.: The fast multipole method - I: Error analysis and asymptotic complexity. SIAM Journal on Numerical Analysis pp. 98–128 (2001)
Darve, E., Havé, P.: Efficient fast multipole method for low-frequency scattering. Journal of Computational Physics 197(1), 341–363 (2004)
Darve, E., Havé, P.: A fast multipole method for Maxwell equations stable at all frequencies. Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences 362(1816), 603–628 (2004)
Fong, W., Darve, E.: The black-box fast multipole method. Journal of Computational Physics 228(23), 8712–8725 (2009)
Fritz, J., Neuweiler, I., Nowak, W.: Application of FFT-based algorithms for large-scale universal kriging problems. Mathematical Geosciences 41(5), 509–533 (2009)
Golub, G., Van Loan, C.: Matrix computations, vol. 3. Johns Hopkins Univ Press (1996)
Goreinov, S., Tyrtyshnikov, E., Zamarashkin, N.: A theory of pseudoskeleton approximations. Linear Algebra and Its Applications 261(1–3), 1–21 (1997)
Grasedyck, L., Hackbusch, W.: Construction and arithmetics of \(\mathcal{H}\)-matrices. Computing 70, 2003 (2003)
Greengard, L., Rokhlin, V.: A fast algorithm for particle simulations. Journal of Computational Physics 73(2), 325–348 (1987)
Greengard, L., Rokhlin, V.: A new version of the fast multipole method for the Laplace equation in three dimensions. Acta Numerica 6(1), 229–269 (1997)
Greengard, L., Rokhlin, V., SCIENCE., Y.U.N.H.C.D.O.C.: On the efficient implementation of the fast multipole algorithm. Defense Technical Information Center (1988)
Hackbusch, W.: A sparse matrix arithmetic based on \(\mathcal{H}\)-matrices. Part I: Introduction to \(\mathcal{H}\)-matrices. Computing 62(2), 89–108 (1999)
Hackbusch, W., Börm, S.: Data-sparse approximation by adaptive \({\mathcal{H}}^{2}\)-matrices. Computing 69(1), 1–35 (2002)
Hackbusch, W., Grasedyck, L., Börm, S.: An introduction to hierarchical matrices. Max-Planck-Inst. für Mathematik in den Naturwiss. (2001)
Hackbusch, W., Khoromskij, B.: A sparse \(\mathcal{H}\)-matrix arithmetic. Computing 64(1), 21–47 (2000)
Hackbusch, W., Nowak, Z.: On the fast matrix multiplication in the boundary element method by panel clustering. Numerische Mathematik 54(4), 463–491 (1989)
Hrycak, T., Rokhlin, V.: An improved fast multipole algorithm for potential fields. Tech. rep., DTIC Document (1995)
Kitanidis, P.K.: Statistical estimation of polynomial generalized covariance functions and hydrologic applications. Water Resources Research 19(4), 909–921 (1983)
Kitanidis, P.K.: Generalized covariance functions in estimation. Mathematical Geology 25(5), 525–540 (1993)
Kitanidis, P.K.: Quasi-linear geostatistical theory for inversing. Water Resources Research 31(10), 2411–2419 (1995)
Kitanidis, P.K.: On the geostatistical approach to the inverse problem. Advances in Water Resources 19(6), 333–342 (1996)
Kitanidis, P.K.: Introduction to geostatistics: applications to hydrogeology. Cambridge Univ Pr (1997)
Kitanidis, P.K.: Generalized covariance functions associated with the Laplace equation and their use in interpolation and inverse problems. Water Resources Research 35(5), 1361–1367 (1999)
Kitanidis, P.K.: On stochastic inverse modeling. Geophysical Monograph-American Geophysical Union 171, 19 (2007)
Kitanidis, P.K., Vomvoris, E.G.: A geostatistical approach to the inverse problem in groundwater modeling (steady state) and one-dimensional simulations. Water Resources Research 19(3), 677–690 (1983)
Liu, X., Illman, W., Craig, A., Zhu, J., Yeh, T.: Laboratory sandbox validation of transient hydraulic tomography. Water Resources Research 43(5), W05,404 (2007)
Mallat, S.: A theory for multiresolution signal decomposition: The wavelet representation. Pattern Analysis and Machine Intelligence, IEEE Transactions on 11(7), 674–693 (1989)
Mallat, S.: A wavelet tour of signal processing. Academic Press (1999)
Nishimura, N.: Fast multipole accelerated boundary integral equation methods. Applied Mechanics Reviews 55, 299 (2002)
Nowak, W., Cirpka, O.A.: Geostatistical inference of hydraulic conductivity and dispersivities from hydraulic heads and tracer data. Water Resources Research 42(8), 8416 (2006)
Nowak, W., Tenkleve, S., Cirpka, O.: Efficient computation of linearized cross-covariance and auto-covariance matrices of interdependent quantities. Mathematical geology 35(1), 53–66 (2003)
Pollock, D., Cirpka, O.: Fully coupled hydrogeophysical inversion of synthetic salt tracer experiments. Water Resources Research 46(7), W07,501 (2010)
Rjasanow, S.: Adaptive cross approximation of dense matrices. IABEM 2002, International Association for Boundary Element Methods (2002)
Rjasanow, S., Steinbach, O.: The fast solution of boundary integral equations. Mathematical and Analytical Techniques with Applications to Engineering. Springer, New York (2007)
Rokhlin, V.: Rapid solution of integral equations of classical potential theory. Journal of Computational Physics 60(2), 187–207 (1985)
Saibaba, A., Ambikasaran, S., Li, J., Darve, E., Kitanidis, P.: Application of hierarchical matrices to linear inverse problems in geostatistics. Oil & Gas Science and Technology–Revue d’IFP Energies nouvelles 67(5), 857–875 (2012)
Saibaba, A., Kitanidis, P.: Efficient methods for large-scale linear inversion using a geostatistical approach. Water Resources Research 48(5), W05,522 (2012)
Starks, T., Fang, J.: On the estimation of the generalized covariance function. Mathematical Geology 14(1), 57–64 (1982)
Vandebril, R., Barel, M., Golub, G., Mastronardi, N.: A bibliography on semiseparable matrices. Calcolo 42(3), 249–270 (2005)
Xia, J., Chandrasekaran, S., Gu, M., Li, X.: Fast algorithms for hierarchically semiseparable matrices. Numerical Linear Algebra with Applications 17(6), 953–976 (2010)
Ying, L., Biros, G., Zorin, D.: A kernel-independent adaptive fast multipole algorithm in two and three dimensions. Journal of Computational Physics 196(2), 591–626 (2004)
Acknowledgements
The authors were supported by “NSF Award 0934596, Subsurface Imaging and Uncertainty Quantification,” “Army High Performance Computing Research Center” (AHPCRC, sponsored by the U.S. Army Research Laboratory under contract No. W911NF-07-2-0027) and “The Global Climate and Energy Project” (GCEP) at Stanford.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media New York
About this paper
Cite this paper
Ambikasaran, S., Saibaba, A.K., Darve, E.F., Kitanidis, P.K. (2013). Fast Algorithms for Bayesian Inversion. In: Dawson, C., Gerritsen, M. (eds) Computational Challenges in the Geosciences. The IMA Volumes in Mathematics and its Applications, vol 156. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7434-0_5
Download citation
DOI: https://doi.org/10.1007/978-1-4614-7434-0_5
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-7433-3
Online ISBN: 978-1-4614-7434-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)