Large-scale stochastic linear inversion using hierarchical matrices

Illustrated with an application to crosswell tomography in seismic imaging


Stochastic inverse modeling deals with the estimation of functions from sparse data, which is a problem with a nonunique solution, with the objective to evaluate best estimates, measures of uncertainty, and sets of solutions that are consistent with the data. As finer resolutions become desirable, the computational requirements increase dramatically when using conventional solvers. A method is developed in this paper to solve large-scale stochastic linear inverse problems, based on the hierarchical matrix (or 2 matrix) approach. The proposed approach can also exploit the sparsity of the underlying measurement operator, which relates observations to unknowns. Conventional direct algorithms for solving large-scale linear inverse problems, using stochastic linear inversion techniques, typically scale as 𝒪(n 2 m+nm 2), where n is the number of measurements and m is the number of unknowns. We typically have nm. In contrast, the algorithm presented here scales as 𝒪(n 2 m), i.e., it scales linearly with the larger problem dimension m. The algorithm also allows quantification of uncertainty in the solution at a computational cost that also grows only linearly in the number of unknowns. The speedup gained is significant since the number of unknowns m is often large. The effectiveness of the algorithm is demonstrated by solving a realistic crosswell tomography problem by formulating it as a stochastic linear inverse problem. In the case of the crosswell tomography problem, the sparsity of the measurement operator allows us to further reduce the cost of our proposed algorithm from 𝒪(n 2 m) to \(\mathcal {O}(n^{2} \sqrt {m} + nm)\). The computational speedup gained by using the new algorithm makes it easier, among other things, to optimize the location of sources and receivers, by minimizing the mean square error of the estimation. Without this fast algorithm, this optimization would be computationally impractical using conventional methods.

This is a preview of subscription content, log in to check access.


  1. 1.

    Ajo-Franklin, J.: Optimal experiment design for time-lapse travel time tomography. Geophysics 74(4), Q27–Q40 (2009)

    Article  Google Scholar 

  2. 2.

    Barnes, J., Hut, P.: A hierarchical 𝒪(N log N) force-calculation algorithm. Nature 324(4), 446–449 (1986)

    Article  Google Scholar 

  3. 3.

    Beatson, R., Greengard, L.: A short course on fast multipole methods. Wavelets, Multilevel Methods and Elliptic PDEs, pp 1–37 (1997)

  4. 4.

    Beatson, R., Newsam, G.: Fast evaluation of radial basis functions: I. Comput. Math. Appl. 24(12), 7–19 (1992)

    Article  Google Scholar 

  5. 5.

    Bjørnstad, O., Falck, W.: Nonparametric spatial covariance functions: estimation and testing. Environ. Ecol. Stat. 8(1), 53–70 (2001)

    Article  Google Scholar 

  6. 6.

    Börm, S., Grasedyck, L., Hackbusch, W.: Hierarchical matrices. Lect. Notes, 21 (2005)

  7. 7.

    Burrus, C., Gopinath, R., Guo, H.: Introduction to wavelets and wavelet transforms: a primer. Recherche 67, 02 (1998)

    Google Scholar 

  8. 8.

    Cheng, H., Greengard, L., Rokhlin, V.: A fast adaptive multipole algorithm in three dimensions. J. Comput. Phys. 155(2), 468–498 (1999)

    Article  Google Scholar 

  9. 9.

    Cheng, H., Gimbutas, Z., Martinsson, P., Rokhlin, V.: On the compression of low-rank matrices. SIAM J. Sci. Comput. 26(4), 1389–1404 (2005)

    Article  Google Scholar 

  10. 10.

    Coifman, R., Rokhlin, V., Wandzura, S.: The fast multipole method for the wave equation: a pedestrian prescription. IEEE Trans. Antennas Propag. Mag. 35(3), 7–12 (1993)

    Article  Google Scholar 

  11. 11.

    Cooley, J., Tukey, J.: An algorithm for the machine calculation of complex fourier series. Math. Comput. 19(90), 297–301 (1965)

    Article  Google Scholar 

  12. 12.

    Cornford, D., Csató, L., Opper, M.: Sequential, bayesian geostatistics: a principled method for large data sets. Geogr. Anal. 37(2), 183–199 (2005)

    Article  Google Scholar 

  13. 13.

    Daley, T., Ajo-Franklin, J., Doughty, C.: Integration of crosswell CASSM (continuous active source seismic monitoring) and flow modeling for imaging of a CO2 plume in a brine aquiferIn: 2008 SEG Annual Meeting (2008)

  14. 14.

    Daley, T.M., Solbau, R.D., Ajo-Franklin, J.B., Benson, S.M.: Continuous active-source seismic monitoring of CO2 injection in a brine aquifer. Geophysics 72(5), A57–A61 (2007)

    Article  Google Scholar 

  15. 15.

    Darve, E.: The fast multipole method I: error analysis and asymptotic complexity. SIAM J. Numer. Anal. 38(1), 98–128 (2000a)

    Article  Google Scholar 

  16. 16.

    Darve, E.: The fast multipole method: numerical implementation. J. Comput. Phys. 160(1), 195–240 (2000b)

    Article  Google Scholar 

  17. 17.

    Davis, T.: Direct methods for sparse linear systems. Soc. Ind. Math. 2 (2006)

  18. 18.

    Doughty, C., Freifeld, B., Trautz, R.: Site characterization for CO2 geologic storage and vice versa: the Frio brine pilot, Texas, USA as a case study. Environ. Geol. 54(8), 1635–1656 (2008)

    Article  Google Scholar 

  19. 19.

    Fong, W., Darve, E.: The black-box fast multipole method. J. Comput. Phys. 228(23), 8712–8725 (2009)

    Article  Google Scholar 

  20. 20.

    Frieze, A., Kannan, R., Vempala, S.: Fast Monte-Carlo algorithms for finding low-rank approximations. J. ACM (JACM) 51(6), 1025–1041 (2004)

    Article  Google Scholar 

  21. 21.

    Fritz, J., Neuweiler, I., Nowak, W.: Application of FFT-based algorithms for large-scale universal kriging problems. Math. Geosci. 41(5), 509–533 (2009)

    Article  Google Scholar 

  22. 22.

    Golub, G., Van Loan, C.: Matrix Computations, vol. 3. Johns Hopkins Univ Press (1996)

  23. 23.

    Goreinov, S., Tyrtyshnikov, E., Zamarashkin, N.: A theory of pseudoskeleton approximations. Linear Algebra Appl. 261(1-3), 1–21 (1997)

    Article  Google Scholar 

  24. 24.

    Grasedyck, L., Hackbusch, W.: Construction and arithmetics of -matrices. Computing 70(4), 295–334 (2003)

    Article  Google Scholar 

  25. 25.

    Greengard, L., Rokhlin, V.: A fast algorithm for particle simulations. J. Comput. Phys. 73(2), 325–348 (1987)

    Article  Google Scholar 

  26. 26.

    Greengard, L., Rokhlin, V.: A new version of the fast multipole method for the Laplace equation in three dimensions. Acta Numer. 6(1), 229–269 (1997)

    Article  Google Scholar 

  27. 27.

    Gu, M., Eisenstat, S.: Efficient algorithms for computing a strong rank-revealing QR factorization. Society 17(4), 848–869 (1996)

    Google Scholar 

  28. 28.

    Hackbusch, W.: A sparse matrix arithmetic based on -matrices. Part I: introduction to -matrices. Computing 62(2), 89–108 (1999)

    Article  Google Scholar 

  29. 29.

    Hackbusch, W., Börm, S.: Data-sparse approximation by adaptive 2-matrices. Computing 69(1), 1–35 (2002)

    Article  Google Scholar 

  30. 30.

    Hackbusch, W., Khoromskij, B.: A sparse -matrix arithmetic. Computing 64(1), 21–47 (2000)

    Google Scholar 

  31. 31.

    Hackbusch, W., Nowak, Z.: On the fast matrix multiplication in the boundary element method by panel clustering. Numer. Math. 54(4), 463–491 (1989)

    Article  Google Scholar 

  32. 32.

    Kitanidis, P.: Statistical estimation of polynomial generalized covariance functions and hydrologic applications. Water Resour. Res. 19(4), 909–921 (1983)

    Article  Google Scholar 

  33. 33.

    Kitanidis, P.: Generalized covariance functions in estimation. Math. Geol. 25(5), 525–540 (1993)

    Article  Google Scholar 

  34. 34.

    Kitanidis, P.: Quasi-linear geostatistical theory for inversing. Water Resour. Res. 31(10), 2411–2419 (1995)

    Article  Google Scholar 

  35. 35.

    Kitanidis, P.: On the geostatistical approach to the inverse problem. Adv. Water Resour. 19(6), 333–342 (1996)

    Article  Google Scholar 

  36. 36.

    Kitanidis, P.: Generalized covariance functions associated with the Laplace equation and their use in interpolation and inverse problems. Water Resour. Res. 35(5), 1361–1367 (1999)

    Article  Google Scholar 

  37. 37.

    Kitanidis, P., Vomvoris, E.: A geostatistical approach to the inverse problem in groundwater modeling (steady state) and one-dimensional simulations. Water Resour. Res. 19(3), 677–690 (1983)

    Article  Google Scholar 

  38. 38.

    Kitanidis, P.K.: Introduction to Geostatistics: Applications to Hydrogeology. Cambridge Univ Pr (1997)

  39. 39.

    Kitanidis, P.K.: On stochastic inverse modeling. Geophys. Monogr. Ser. 171, 19–30 (2007)

    Google Scholar 

  40. 40.

    Li, W., Cirpka, O.: Efficient geostatistical inverse methods for structured and unstructured grids. Water Resour. Res. 42(6), W06,402 (2006)

    Google Scholar 

  41. 41.

    Liberty, E., Woolfe, F., Martinsson, P., Rokhlin, V., Tygert, M.: Randomized algorithms for the low-rank approximation of matrices. Proc. Natl. Acad. Sci. 104(51), 20,167 (2007)

    Article  Google Scholar 

  42. 42.

    Liu, X., Kitanidis, P.: Large-scale inverse modeling with an application in hydraulic tomography. Water Resour. Res. 47(2), W02,501 (2011)

    Google Scholar 

  43. 43.

    Liu, X., Illman, W., Craig, A., Zhu, J., Yeh, T.: Laboratory sandbox validation of transient hydraulic tomography. Water Resour. Res. 43(5), W05,404 (2007)

    Google Scholar 

  44. 44.

    Mallat, S.: A theory for multiresolution signal decomposition: the wavelet representation.IEEE Trans. Pattern Anal. Mach. Intell. 11(7), 674–693 (1989)

    Article  Google Scholar 

  45. 45.

    Mallat, S.: A Wavelet Tour of Signal Processing. Academic (1999)

  46. 46.

    Messner, M., Schanz, M., Darve, E.: Fast directional multilevel summation for oscillatory kernels based on Chebyshev interpolation. J. Comput. Phys. 231(4), 1175–1196 (2012)

    Article  Google Scholar 

  47. 47.

    Miranian, L., Gu, M.: Strong rank revealing LU factorizations. Linear Algebra Appl. 367, 1–16 (2003)

    Article  Google Scholar 

  48. 48.

    Nishimura, N.: Fast multipole accelerated boundary integral equation methods. Appl. Mech. Rev. 55(4), 299–324 (2002)

    Article  Google Scholar 

  49. 49.

    Nowak, W., Cirpka, O.: Geostatistical inference of hydraulic conductivity and dispersivities from hydraulic heads and tracer data. Water Resour. Res. 42(8), 8416 (2006)

    Google Scholar 

  50. 50.

    Nowak, W., Tenkleve, S., Cirpka, O.: Efficient computation of linearized cross-covariance and auto-covariance matrices of interdependent quantities. Math. Geol. 35(1), 53–66 (2003)

    Article  Google Scholar 

  51. 51.

    Pollock, D., Cirpka, O.: Fully coupled hydrogeophysical inversion of synthetic salt tracer experiments. Water Resour. Res. 46(7), W07,501 (2010)

    Google Scholar 

  52. 52.

    Pruess, K.: ECO2N: A TOUGH2 Fluid Property Module For Mixtures Of Water, NaCl, and CO2. Lawrence Berkeley National Laboratory (2005)

  53. 53.

    Pruess, K., Oldenburg, C., Moridis, G.: TOUGH2 User’s Guide, Version 2.0 (1999)

  54. 54.

    Rjasanow, S.: Adaptive cross approximation of dense matrices. In: IABEM 2002, International Association for Boundary Element Methods (2002)

  55. 55.

    Saad, Y.: Iterative Methods for Sparse Linear Systems, vol. 20. PWS, Boston (1996)

  56. 56.

    Starks, T., Fang, J.: On the estimation of the generalized covariance function. Math. Geol. 14(1), 57–64 (1982)

    Article  Google Scholar 

  57. 57.

    White, J.: Computed seismic speeds and attenuation in rocks with partial gas saturation. Geophysics 40(2), 224–232 (1975)

    Article  Google Scholar 

  58. 58.

    Woolfe, F., Liberty, E., Rokhlin, V., Tygert, M.: A fast randomized algorithm for the approximation of matrices. Appl. Comput. Harmon. Anal. 25(3), 335–366 (2008)

    Article  Google Scholar 

  59. 59.

    Xia, J., Chandrasekaran, S., Gu, M., Li, X.: Fast algorithms for hierarchically semiseparable matrices. Numer. Linear Algebra Appl. 17(6), 953–976 (2010) z

    Article  Google Scholar 

  60. 60.

    Ying, L., Biros, G., Zorin, D.: A kernel-independent adaptive fast multipole algorithm in two and three dimensions. J. Comput. Phys. 196(2), 591–626 (2004)

    Article  Google Scholar 

  61. 61.

    Zimmerman, D.: Computationally efficient restricted maximum likelihood estimation of generalized covariance functions. Math. Geol. 21(7), 655–672 (1989a)

    Article  Google Scholar 

  62. 62.

    Zimmerman, D.: Computationally exploitable structure of covariance matrices and generalized covariance matrices in spatial models. J. Stat. Comp. Simul. 32(1-2), 1–15 (1989b)

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Sivaram Ambikasaran.

Additional information

The authors were supported by “US Department of Energy, National Energy Technology Laboratory” (DOE, NETL) under the award number: DE-FE0009260, and also by the “National Science Foundation” —Division of Mathematical Sciences —under the award number: 1228275. The authors would also like to thank the “The Global Climate and Energy Project” (GCEP) at Stanford University, and the “Army High Performance Computing Research Center” (AHPCRC, sponsored by the U.S. Army Research Laboratory under contract No. W911NF-07-2-0027).

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ambikasaran, S., Li, J.Y., Kitanidis, P.K. et al. Large-scale stochastic linear inversion using hierarchical matrices. Comput Geosci 17, 913–927 (2013).

Download citation


  • Stochastic inverse modeling
  • Numerical linear algebra
  • Hierarchical matrices
  • Large-scale problems
  • Subsurface imaging
  • Tomography
  • Geostatistical estimation

Mathematics Subject Classification (2010)

  • 86-08
  • 65F30