Numerische Mathematik

, Volume 139, Issue 3, pp 683–707 | Cite as

Convergence analysis of multifidelity Monte Carlo estimation

  • Benjamin PeherstorferEmail author
  • Max Gunzburger
  • Karen Willcox


The multifidelity Monte Carlo method provides a general framework for combining cheap low-fidelity approximations of an expensive high-fidelity model to accelerate the Monte Carlo estimation of statistics of the high-fidelity model output. In this work, we investigate the properties of multifidelity Monte Carlo estimation in the setting where a hierarchy of approximations can be constructed with known error and cost bounds. Our main result is a convergence analysis of multifidelity Monte Carlo estimation, for which we prove a bound on the costs of the multifidelity Monte Carlo estimator under assumptions on the error and cost bounds of the low-fidelity approximations. The assumptions that we make are typical in the setting of similar Monte Carlo techniques. Numerical experiments illustrate the derived bounds.

Mathematics Subject Classification

35Q62 65C05 60H35 35R60 65N15 



The first and the third author were supported in part by the AFOSR MURI on multi-information sources of multi-physics systems under Award Number FA9550-15-1-0038, program manager Jean-Luc Cambier, and by the United States Department of Energy Applied Mathematics Program, Awards DE-FG02-08ER2585 and DE-SC0009297, as part of the DiaMonD Multifaceted Mathematics Integrated Capability Center. The second author was supported by the US Department of Energy Office of Science grant DE-SC0009324 and the Air Force Office of Scientific Grant FA9550-15-1-0001. Some of the numerical examples were computed on the computer cluster of the Munich Centre of Advanced Computing.


  1. 1.
    Babuška, I., Nobile, F.: A stochastic collocation method for elliptic partial dierential equations with random input data. SIAM J. Numer. Anal. 45(3), 1005–1034 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Barth, A., Schwab, C., Zollinger, N.: Multi-level Monte Carlo finite element method for elliptic PDEs with stochastic coefficients. Numer. Math. 119(1), 123–161 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Benner, P., Gugercin, S., Willcox, K.: A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Rev. 57(4), 483–531 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bierig, C., Chernov, A.: Convergence analysis of multilevel Monte Carlo variance estimators and application for random obstacle problems. Numer. Math. 130(4), 579–613 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Bierig, C., Chernov, A.: Approximation of probability density functions by the multilevel Monte Carlo maximum entropy method. J. Comput. Phys. 314, 661–681 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Bierig, C., Chernov, A.: Estimation of arbitrary order central statistical moments by the multilevel Monte Carlo method. Stoch. Partial Differ. Equ. Anal. Comput. 4(1), 3–40 (2016)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Bungartz, H.-J., Griebel, M.: Sparse grids. Acta Numer. 13, 1–123 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Charrier, J., Scheichl, R., Teckentrup, A.L.: Finite element error analysis of elliptic PDEs with random coefficients and its application to multilevel Monte Carlo methods. SIAM J. Numer. Anal. 51(1), 322–352 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Cliffe, K.A., Giles, M., Scheichl, R., Teckentrup, A.L.: Multilevel Monte Carlo methods and applications to elliptic PDEs with random coefficients. Comput. Vis. Sci. 14(1), 3–15 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Collier, N., Haji-Ali, A.-L., Nobile, F., von Schwerin, E., Tempone, R.: A continuation multilevel Monte Carlo algorithm. BIT Numer. Math. 55(2), 399–432 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  12. 12.
    Forrester, A.I.J., Keane, A.J.: Recent advances in surrogate-based optimization. Prog. Aerosp. Sci. 45(1–3), 50–79 (2009)CrossRefGoogle Scholar
  13. 13.
    Forrester, K.A., Sóbester, A.: Engineering Design Via Surrogate Modelling: A Practical Guide. Wiley, Hoboken (2008)CrossRefGoogle Scholar
  14. 14.
    Franzelin, F., Diehl, P., Pflüger, D.: Non-intrusive uncertainty quantification with sparse grids for multivariate peridynamic simulations. In: Griebel, M., Schweitzer, A.M. (eds.) Meshfree Methods for Partial Differential Equations VII, pp. 115–143. Springer, Cham (2015)Google Scholar
  15. 15.
    Franzelin, F., Pflüger, D.: From data to uncertainty: an efficient integrated data-driven sparse grid approach to propagate uncertainty. In: Garcke, J., Pflüger, D. (eds.) Sparse Grids and applications–Stuttgart 2014, pp. 29–49. Springer, Cham (2016)CrossRefGoogle Scholar
  16. 16.
    Gerstner, T., Griebel, M.: Numerical integration using sparse grids. Numer. Algorithms 18(3), 209–232 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Gerstner, T., Griebel, M.: Dimension-adaptive tensor-product quadrature. Computing 71(1), 65–87 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Giles, M.: Multi-level Monte Carlo path simulation. Oper. Res. 56(3), 607–617 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Griebel, M., Harbrecht,H., Peters, M.: Multilevel quadrature for elliptic parametric partial differential equations on non-nested meshes. Stoch. Partial Differ. Equ. Anal. Comput. (2015). arXiv:1509.09058
  20. 20.
    Gugercin, A., Antoulas, A., Beattie, C.: \(\cal{H}_2\) model reduction for large-scale linear dynamical systems. SIAM J. Matrix Anal. Appl. 30(2), 609–638 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Gunzburger, M.D., Webster, C.G., Zhang, G.: Stochastic finite element methods for partial differential equations with random input data. Acta Numer. 23, 521–650 (2014)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Haji-Ali, A.-L., Nobile, F., Tempone, R.: Multi-index Monte Carlo: when sparsity meets sampling. Numer. Math. 132(4), 767–806 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Hammersley, J.M., Handscomb, D.C.: Monte Carlo Methods. Methuen, London (1964)CrossRefzbMATHGoogle Scholar
  24. 24.
    Harbrecht, H., Peters, M., Siebenmorgen, M.: On multilevel quadrature for elliptic stochastic partial differential equations. In: Garcke, J., Griebel, M. (eds.) Sparse Grids and Applications, pp. 161–179. Springer, Berlin (2013)Google Scholar
  25. 25.
    Harbrecht, H., Peters, M., Siebenmorgen, M.: Multilevel accelerated quadrature for PDEs with log-normally distributed diffusion coefficient. SIAM/ASA J. Uncertain. Quantif. 4(1), 520–551 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Heinrich, S.: Multilevel Monte Carlo methods. In: Margenov, S., Waniewški, J., Yalamov, P. (eds.) Large-Scale Scientific Computing, number 2179 in Lecture Notes in Computer Science, pp. 58–67. Springer, Berlin (2001)Google Scholar
  27. 27.
    Li, J., Xiu, D.: Evaluation of failure probability via surrogate models. J. Comput. Phys. 229(23), 8966–8980 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Liu, J.S.: Monte Carlo Strategies in Scientific Computing. Springer, Berlin (2008)zbMATHGoogle Scholar
  29. 29.
    Majda, J., Gershgorin, B.: Quantifying uncertainty in climate change science through empirical information theory. Proc. Natl. Acad. Sci. USA 107(34), 14958–14963 (2010)CrossRefGoogle Scholar
  30. 30.
    Nelson, B.L.: On control variate estimators. Comput. Oper. Res. 14(3), 219–225 (1987)CrossRefzbMATHGoogle Scholar
  31. 31.
    Ng, L., Willcox, K.: Multifidelity approaches for optimization under uncertainty. Int. J. Numer. Methods Eng. 100(10), 746–772 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Ng, L., Willcox, K.: Monte-Carlo information-reuse approach to aircraft conceptual design optimization under uncertainty. J. Aircr. 53(2), 427–438 (2016)CrossRefGoogle Scholar
  33. 33.
    Nobile, F., Tempone, R., Webster, C.G.: A sparse grid stochastic collocation method for partial differential equations with random input data. SIAM J. Numer. Anal. 46(5), 2309–2345 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Pauli, S., Arbenz, P.: Determining optimal multilevel Monte Carlo parameters with application to fault tolerance. Comput. Math. Appl. 70(11), 2638–2651 (2015)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Pauli, P., Arbenz, P., Schwab, C.: Intrinsic fault tolerance of multilevel Monte Carlo methods. J. Parallel Distrib. Comput. 84, 24–36 (2015)CrossRefGoogle Scholar
  36. 36.
    Peherstorfer, B., Cui, T., Marzouk, Y., Willcox, K.: Multifidelity importance sampling. Comput. Methods Appl. Mech. Eng. 300, 490–509 (2016)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Peherstorfer, B., Willcox, K.: Online adaptive model reduction for nonlinear systems via low-rank updates. SIAM J. Sci. Comput. 37(4), A2123–A2150 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Peherstorfer, B., Willcox, K., Gunzburger, M.: Optimal model management for multifidelity Monte Carlo estimation. SIAM J. Sci. Comput. 38(5), A3163–A3194 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Robert, C., Casella, G.: Monte Carlo Statistical Methods. Springer, Berlin (2004)CrossRefzbMATHGoogle Scholar
  40. 40.
    Rozza, G., Huynh, D., Patera, A.: Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations. Arch. Comput. Methods Eng. 15(3), 1–47 (2007)CrossRefzbMATHGoogle Scholar
  41. 41.
    Sirovich, L.: Turbulence and the dynamics of coherent structures. Q. Appl. Math. 45, 561–571 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Teckentrup, L., Scheichl, R., Giles, M., Ullmann, E.: Further analysis of multilevel Monte Carlo methods for elliptic PDEs with random coefficients. Numer. Math. 125(3), 569–600 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Ullmann, E., Elman, H.C., Ernst, O.G.: Efficient iterative solvers for stochastic Galerkin discretizations of log-transformed random diffusion problems. SIAM J. Sci. Comput. 34(2), A659–A682 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Ullmann, E., Papaioannou, I.: Multilevel estimation of rare events. SIAM/ASA J. Uncertain. Quantif. 3(1), 922–953 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Ullmann, E., Powell, C.E.: Solving log-transformed random diffusion problems by stochastic Galerkin mixed finite element methods. SIAM/ASA J. Uncertain. Quantif. 3(1), 509–534 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Vapnik, V.: Statistical Learning Theory. Wiley, Hoboken (1998)zbMATHGoogle Scholar
  47. 47.
    Xiu, D.: Fast numerical methods for stochastic computations: a review. Commun. Comput. Phys. 5, 242–272 (2009)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Benjamin Peherstorfer
    • 1
    Email author
  • Max Gunzburger
    • 2
  • Karen Willcox
    • 3
  1. 1.Department of Mechanical Engineering and Wisconsin Institute for DiscoveryUniversity of Wisconsin-MadisonMadisonUSA
  2. 2.Department of Scientific ComputingFlorida State UniversityTallahasseeUSA
  3. 3.Department of Aeronautics and AstronauticsMITCambridgeUSA

Personalised recommendations