Skip to main content
SpringerLink
Log in

Convergence analysis of multifidelity Monte Carlo estimation

  • Published:
Numerische Mathematik Aims and scope Submit manuscript

Abstract

The multifidelity Monte Carlo method provides a general framework for combining cheap low-fidelity approximations of an expensive high-fidelity model to accelerate the Monte Carlo estimation of statistics of the high-fidelity model output. In this work, we investigate the properties of multifidelity Monte Carlo estimation in the setting where a hierarchy of approximations can be constructed with known error and cost bounds. Our main result is a convergence analysis of multifidelity Monte Carlo estimation, for which we prove a bound on the costs of the multifidelity Monte Carlo estimator under assumptions on the error and cost bounds of the low-fidelity approximations. The assumptions that we make are typical in the setting of similar Monte Carlo techniques. Numerical experiments illustrate the derived bounds.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Babuška, I., Nobile, F.: A stochastic collocation method for elliptic partial dierential equations with random input data. SIAM J. Numer. Anal. 45(3), 1005–1034 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  2. Barth, A., Schwab, C., Zollinger, N.: Multi-level Monte Carlo finite element method for elliptic PDEs with stochastic coefficients. Numer. Math. 119(1), 123–161 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  3. Benner, P., Gugercin, S., Willcox, K.: A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Rev. 57(4), 483–531 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bierig, C., Chernov, A.: Convergence analysis of multilevel Monte Carlo variance estimators and application for random obstacle problems. Numer. Math. 130(4), 579–613 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bierig, C., Chernov, A.: Approximation of probability density functions by the multilevel Monte Carlo maximum entropy method. J. Comput. Phys. 314, 661–681 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  6. Bierig, C., Chernov, A.: Estimation of arbitrary order central statistical moments by the multilevel Monte Carlo method. Stoch. Partial Differ. Equ. Anal. Comput. 4(1), 3–40 (2016)

    MathSciNet  MATH  Google Scholar 

  7. Bungartz, H.-J., Griebel, M.: Sparse grids. Acta Numer. 13, 1–123 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  8. Charrier, J., Scheichl, R., Teckentrup, A.L.: Finite element error analysis of elliptic PDEs with random coefficients and its application to multilevel Monte Carlo methods. SIAM J. Numer. Anal. 51(1), 322–352 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  9. Cliffe, K.A., Giles, M., Scheichl, R., Teckentrup, A.L.: Multilevel Monte Carlo methods and applications to elliptic PDEs with random coefficients. Comput. Vis. Sci. 14(1), 3–15 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  10. Collier, N., Haji-Ali, A.-L., Nobile, F., von Schwerin, E., Tempone, R.: A continuation multilevel Monte Carlo algorithm. BIT Numer. Math. 55(2), 399–432 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  11. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  12. Forrester, A.I.J., Keane, A.J.: Recent advances in surrogate-based optimization. Prog. Aerosp. Sci. 45(1–3), 50–79 (2009)

    Article  Google Scholar 

  13. Forrester, K.A., Sóbester, A.: Engineering Design Via Surrogate Modelling: A Practical Guide. Wiley, Hoboken (2008)

    Book  Google Scholar 

  14. Franzelin, F., Diehl, P., Pflüger, D.: Non-intrusive uncertainty quantification with sparse grids for multivariate peridynamic simulations. In: Griebel, M., Schweitzer, A.M. (eds.) Meshfree Methods for Partial Differential Equations VII, pp. 115–143. Springer, Cham (2015)

    Google Scholar 

  15. Franzelin, F., Pflüger, D.: From data to uncertainty: an efficient integrated data-driven sparse grid approach to propagate uncertainty. In: Garcke, J., Pflüger, D. (eds.) Sparse Grids and applications–Stuttgart 2014, pp. 29–49. Springer, Cham (2016)

    Chapter  Google Scholar 

  16. Gerstner, T., Griebel, M.: Numerical integration using sparse grids. Numer. Algorithms 18(3), 209–232 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  17. Gerstner, T., Griebel, M.: Dimension-adaptive tensor-product quadrature. Computing 71(1), 65–87 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  18. Giles, M.: Multi-level Monte Carlo path simulation. Oper. Res. 56(3), 607–617 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  19. Griebel, M., Harbrecht,H., Peters, M.: Multilevel quadrature for elliptic parametric partial differential equations on non-nested meshes. Stoch. Partial Differ. Equ. Anal. Comput. (2015). arXiv:1509.09058

  20. Gugercin, A., Antoulas, A., Beattie, C.: \(\cal{H}_2\) model reduction for large-scale linear dynamical systems. SIAM J. Matrix Anal. Appl. 30(2), 609–638 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  21. Gunzburger, M.D., Webster, C.G., Zhang, G.: Stochastic finite element methods for partial differential equations with random input data. Acta Numer. 23, 521–650 (2014)

    Article  MathSciNet  Google Scholar 

  22. Haji-Ali, A.-L., Nobile, F., Tempone, R.: Multi-index Monte Carlo: when sparsity meets sampling. Numer. Math. 132(4), 767–806 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hammersley, J.M., Handscomb, D.C.: Monte Carlo Methods. Methuen, London (1964)

    Book  MATH  Google Scholar 

  24. Harbrecht, H., Peters, M., Siebenmorgen, M.: On multilevel quadrature for elliptic stochastic partial differential equations. In: Garcke, J., Griebel, M. (eds.) Sparse Grids and Applications, pp. 161–179. Springer, Berlin (2013)

    Google Scholar 

  25. Harbrecht, H., Peters, M., Siebenmorgen, M.: Multilevel accelerated quadrature for PDEs with log-normally distributed diffusion coefficient. SIAM/ASA J. Uncertain. Quantif. 4(1), 520–551 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  26. Heinrich, S.: Multilevel Monte Carlo methods. In: Margenov, S., Waniewški, J., Yalamov, P. (eds.) Large-Scale Scientific Computing, number 2179 in Lecture Notes in Computer Science, pp. 58–67. Springer, Berlin (2001)

  27. Li, J., Xiu, D.: Evaluation of failure probability via surrogate models. J. Comput. Phys. 229(23), 8966–8980 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  28. Liu, J.S.: Monte Carlo Strategies in Scientific Computing. Springer, Berlin (2008)

    MATH  Google Scholar 

  29. Majda, J., Gershgorin, B.: Quantifying uncertainty in climate change science through empirical information theory. Proc. Natl. Acad. Sci. USA 107(34), 14958–14963 (2010)

    Article  Google Scholar 

  30. Nelson, B.L.: On control variate estimators. Comput. Oper. Res. 14(3), 219–225 (1987)

    Article  MATH  Google Scholar 

  31. Ng, L., Willcox, K.: Multifidelity approaches for optimization under uncertainty. Int. J. Numer. Methods Eng. 100(10), 746–772 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  32. Ng, L., Willcox, K.: Monte-Carlo information-reuse approach to aircraft conceptual design optimization under uncertainty. J. Aircr. 53(2), 427–438 (2016)

    Article  Google Scholar 

  33. Nobile, F., Tempone, R., Webster, C.G.: A sparse grid stochastic collocation method for partial differential equations with random input data. SIAM J. Numer. Anal. 46(5), 2309–2345 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  34. Pauli, S., Arbenz, P.: Determining optimal multilevel Monte Carlo parameters with application to fault tolerance. Comput. Math. Appl. 70(11), 2638–2651 (2015)

    Article  MathSciNet  Google Scholar 

  35. Pauli, P., Arbenz, P., Schwab, C.: Intrinsic fault tolerance of multilevel Monte Carlo methods. J. Parallel Distrib. Comput. 84, 24–36 (2015)

    Article  Google Scholar 

  36. Peherstorfer, B., Cui, T., Marzouk, Y., Willcox, K.: Multifidelity importance sampling. Comput. Methods Appl. Mech. Eng. 300, 490–509 (2016)

    Article  MathSciNet  Google Scholar 

  37. Peherstorfer, B., Willcox, K.: Online adaptive model reduction for nonlinear systems via low-rank updates. SIAM J. Sci. Comput. 37(4), A2123–A2150 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  38. Peherstorfer, B., Willcox, K., Gunzburger, M.: Optimal model management for multifidelity Monte Carlo estimation. SIAM J. Sci. Comput. 38(5), A3163–A3194 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  39. Robert, C., Casella, G.: Monte Carlo Statistical Methods. Springer, Berlin (2004)

    Book  MATH  Google Scholar 

  40. Rozza, G., Huynh, D., Patera, A.: Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations. Arch. Comput. Methods Eng. 15(3), 1–47 (2007)

    Article  MATH  Google Scholar 

  41. Sirovich, L.: Turbulence and the dynamics of coherent structures. Q. Appl. Math. 45, 561–571 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  42. Teckentrup, L., Scheichl, R., Giles, M., Ullmann, E.: Further analysis of multilevel Monte Carlo methods for elliptic PDEs with random coefficients. Numer. Math. 125(3), 569–600 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  43. Ullmann, E., Elman, H.C., Ernst, O.G.: Efficient iterative solvers for stochastic Galerkin discretizations of log-transformed random diffusion problems. SIAM J. Sci. Comput. 34(2), A659–A682 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  44. Ullmann, E., Papaioannou, I.: Multilevel estimation of rare events. SIAM/ASA J. Uncertain. Quantif. 3(1), 922–953 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  45. Ullmann, E., Powell, C.E.: Solving log-transformed random diffusion problems by stochastic Galerkin mixed finite element methods. SIAM/ASA J. Uncertain. Quantif. 3(1), 509–534 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  46. Vapnik, V.: Statistical Learning Theory. Wiley, Hoboken (1998)

    MATH  Google Scholar 

  47. Xiu, D.: Fast numerical methods for stochastic computations: a review. Commun. Comput. Phys. 5, 242–272 (2009)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The first and the third author were supported in part by the AFOSR MURI on multi-information sources of multi-physics systems under Award Number FA9550-15-1-0038, program manager Jean-Luc Cambier, and by the United States Department of Energy Applied Mathematics Program, Awards DE-FG02-08ER2585 and DE-SC0009297, as part of the DiaMonD Multifaceted Mathematics Integrated Capability Center. The second author was supported by the US Department of Energy Office of Science grant DE-SC0009324 and the Air Force Office of Scientific Grant FA9550-15-1-0001. Some of the numerical examples were computed on the computer cluster of the Munich Centre of Advanced Computing.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering and Wisconsin Institute for Discovery, University of Wisconsin-Madison, Madison, WI, 53706, USA

    Benjamin Peherstorfer

  2. Department of Scientific Computing, Florida State University, 400 Dirac Science Library, Tallahassee, FL, 32306-412, USA

    Max Gunzburger

  3. Department of Aeronautics and Astronautics, MIT, Cambridge, MA, 02139, USA

    Karen Willcox

Authors
  1. Benjamin Peherstorfer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Max Gunzburger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Karen Willcox
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin Peherstorfer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peherstorfer, B., Gunzburger, M. & Willcox, K. Convergence analysis of multifidelity Monte Carlo estimation. Numer. Math. 139, 683–707 (2018). https://doi.org/10.1007/s00211-018-0945-7

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00211-018-0945-7

Mathematics Subject Classification

Search

Navigation