Skip to main content

Ensemble sampler for infinite-dimensional inverse problems

Statistics and Computing volume 31, Article number: 28 (2021) Cite this article

Abstract

We introduce a new Markov chain Monte Carlo (MCMC) sampler for infinite-dimensional inverse problems. Our new sampler is based on the affine invariant ensemble sampler, which uses interacting walkers to adapt to the covariance structure of the target distribution. We extend this ensemble sampler for the first time to infinite-dimensional function spaces, yielding a highly efficient gradient-free MCMC algorithm. Because our new ensemble sampler does not require gradients or posterior covariance estimates, it is simple to implement and broadly applicable.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Notes

  1. https://github.com/jeremiecoullon/functional_ensemble_sampler

References

  • Akeret, J., Seehars, S., Amara, A., Refregier, A., Csillaghy, A.: Cosmohammer: cosmological parameter estimation with the MCMC hammer. Astron. Comput. 2, 27–39 (2013)

    Article  Google Scholar 

  • Beskos, A., Girolami, M., Lan, S., Farrell, P.E., Stuart, A.M.: Geometric MCMC for infinite-dimensional inverse problems. J. Comput. Phys. 335, 327–351 (2017)

    MathSciNet  Article  Google Scholar 

  • Beskos, A., Jasra, A., Law, K., Marzouk, Y., Zhou, Y.: Multilevel sequential Monte Carlo with dimension-independent likelihood-informed proposals. SIAM/ASA J. Uncertain. Quantif. 6(2), 762–786 (2018)

    MathSciNet  Article  Google Scholar 

  • Beskos, A., Roberts, G., Stuart, A., Voss, J.: MCMC methods for diffusion bridges. Stoch. Dyn. 8(03), 319–350 (2008)

    MathSciNet  Article  Google Scholar 

  • Chen, Y., Keyes, D., Law, K.J., Ltaief, H.: Accelerated dimension-independent adaptive Metropolis. SIAM J. Sci. Comput. 38(5), S539–S565 (2016)

    MathSciNet  Article  Google Scholar 

  • Cotter, S.L., Roberts, G.O., Stuart, A.M., White, D.: MCMC methods for functions: modifying old algorithms to make them faster. Stat. Sci. **424–446 (2013)

  • Coullon, J., Pokern, Y.: MCMC for a hyperbolic Bayesian inverse problem in traffic flow modelling (2020). arXiv:2001.02013

  • Cui, T., Law, K.J., Marzouk, Y.M.: Dimension-independent likelihood-informed MCMC. J. Comput. Phys. 304, 109–137 (2016)

    MathSciNet  Article  Google Scholar 

  • Cui, T., Martin, J., Marzouk, Y.M., Solonen, A., Spantini, A.: Likelihood-informed dimension reduction for nonlinear inverse problems. Inverse Probl. 30(11), 114015 (2014)

    MathSciNet  Article  Google Scholar 

  • Dunlop, M.M., Stuart, A.M.: The Bayesian formulation of EIT: analysis and algorithms. Inverse Probl. Imaging 10(4), 1007–1036 (2016)

    MathSciNet  Article  Google Scholar 

  • Foreman-Mackey, D., Hogg, D.W., Lang, D., Goodman, J.: emcee: the MCMC hammer. Publ. Astron. Soc. Pac. 125(925), 306 (2013)

    Article  Google Scholar 

  • Goodman, J., Weare, J.: Ensemble samplers with affine invariance. Commun. Appl. Math. Comput. Sci. 5(1), 65–80 (2010)

    MathSciNet  Article  Google Scholar 

  • Hairer, M., Stuart, A.M., Vollmer, S.J., et al.: Spectral gaps for a metropolis-hastings algorithm in infinite dimensions. Ann. Appl. Probab. 24(6), 2455–2490 (2014)

    MathSciNet  Article  Google Scholar 

  • Hu, Z., Yao, Z., Li, J.: On an adaptive preconditioned Crank–Nicolson MCMC algorithm for infinite dimensional Bayesian inference. J. Comput. Phys. 332, 492–503 (2017)

  • Iglesias, M.A., Lin, K., Stuart, A.M.: Well-posed Bayesian geometric inverse problems arising in subsurface flow. Inverse Probl. 30(11), 114001 (2014). https://doi.org/10.1088/0266-5611/30/11/114001

  • Kantas, N., Beskos, A., Jasra, A.: Sequential Monte Carlo methods for high-dimensional inverse problems: a case study for the Navier–Stokes equations. SIAM/ASA J. Uncertain. Quantif. 2(1), 464–489 (2014)

    MathSciNet  Article  Google Scholar 

  • Law, K.J.: Proposals which speed up function-space MCMC. J. Comput. Appl. Math. 262, 127–138 (2014)

    MathSciNet  Article  Google Scholar 

  • Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)

    Article  Google Scholar 

  • Roberts, G.O., Rosenthal, J.S.: Examples of adaptive MCMC. J. Comput. Graph. Stat. 18(2), 349–367 (2009)

    MathSciNet  Article  Google Scholar 

  • Rudolf, D., Sprungk, B.: On a generalization of the preconditioned Crank–Nicolson Metropolis algorithm. Found. Comput. Math. 18(2), 309–343 (2018)

    MathSciNet  Article  Google Scholar 

  • Sokal, A.: Monte Carlo methods in statistical mechanics: Foundations and new algorithms. In: Functional Integration, pp. 131–192. Springer (1997)

  • Stuart, A.M.: Inverse problems: a Bayesian perspective. Acta Numer. 19, 451 (2010)

    MathSciNet  Article  Google Scholar 

  • Zhou, Q., Hu, Z., Yao, Z., Li, J.: A hybrid adaptive MCMC algorithm in function spaces. SIAM/ASA J. Uncertain. Quantifi. 5(1), 621–639 (2017)

    MathSciNet  Article  Google Scholar 

Download references

Acknowledgements

We thank Christopher Nemeth, Gideon Simpson, and Jonathan Weare for offering useful critiques. JC was supported by EPSRC grant EP/S00159X/1. RJW was supported by the National Science Foundation award DMS-1646339 and by New York University’s Dean’s Dissertation Fellowship. The High End Computing facility at Lancaster University provided computing resources.

Author information

Authors and Affiliations

  1. Lancaster University, LA1 4YF, United Kingdom, United Kingdom

    Jeremie Coullon

  2. New York University, 10012, New York, United States

    Robert J. Webber

Authors
  1. Jeremie Coullon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert J. Webber
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeremie Coullon.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Coullon, J., Webber, R.J. Ensemble sampler for infinite-dimensional inverse problems . Stat Comput 31, 28 (2021). https://doi.org/10.1007/s11222-021-10004-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11222-021-10004-y

Keywords

  • Bayesian inverse problems
  • Markov chain Monte Carlo
  • Infinite-dimensional inverse problems
  • Dimensionality reduction

Mathematics Subject Classification

  • 65C05
  • 35R30
  • 62F15