Multifidelity Uncertainty Quantification Using Spectral Stochastic Discrepancy Models

  • Michael S. EldredEmail author
  • Leo W. T. Ng
  • Matthew F. Barone
  • Stefan P. Domino
Reference work entry


When faced with a restrictive evaluation budget that is typical of today’s high-fidelity simulation models, the effective exploitation of lower-fidelity alternatives within the uncertainty quantification (UQ) process becomes critically important. Herein, we explore the use of multifidelity modeling within UQ, for which we rigorously combine information from multiple simulation-based models within a hierarchy of fidelity, in seeking accurate high-fidelity statistics at lower computational cost. Motivated by correction functions that enable the provable convergence of a multifidelity optimization approach to an optimal high-fidelity point solution, we extend these ideas to discrepancy modeling within a stochastic domain and seek convergence of a multifidelity uncertainty quantification process to globally integrated high-fidelity statistics. For constructing stochastic models of both the low-fidelity model and the model discrepancy, we employ stochastic expansion methods (non-intrusive polynomial chaos and stochastic collocation) computed by integration/interpolation on structured sparse grids or regularized regression on unstructured grids. We seek to employ a coarsely resolved grid for the discrepancy in combination with a more finely resolved grid for the low-fidelity model. The resolutions of these grids may be defined statically or determined through uniform and adaptive refinement processes. Adaptive refinement is particularly attractive, as it has the ability to preferentially target stochastic regions where the model discrepancy becomes more complex, i.e., where the predictive capabilities of the low-fidelity model start to break down and greater reliance on the high-fidelity model (via the discrepancy) is necessary. These adaptive refinement processes can either be performed separately for the different grids or within a coordinated multifidelity algorithm. In particular, we present an adaptive greedy multifidelity approach in which we extend the generalized sparse grid concept to consider candidate index set refinements drawn from multiple sparse grids, as governed by induced changes in the statistical quantities of interest and normalized by relative computational cost. Through a series of numerical experiments using statically defined sparse grids, adaptive multifidelity sparse grids, and multifidelity compressed sensing, we demonstrate that the multifidelity UQ process converges more rapidly than a single-fidelity UQ in cases where the variance of the discrepancy is reduced relative to the variance of the high-fidelity model (resulting in reductions in initial stochastic error), where the spectrum of the expansion coefficients of the model discrepancy decays more rapidly than that of the high-fidelity model (resulting in accelerated convergence rates), and/or where the discrepancy is more sparse than the high-fidelity model (requiring the recovery of fewer significant terms).


Multifidelity Uncertainty quantification Discrepancy model Polynomial chaos Stochastic collocation Sparse grid Compressed sensing Wind turbine 


  1. 1.
    Adams, B.M., Bauman, L.E., Bohnhoff, W.J., Dalbey, K.R., Ebeida, M.S., Eddy, J.P., Eldred, M.S., Hough, P.D., Hu, K.T., Jakeman, J.D., Swiler, L.P., Stephens, J.A., Vigil, D.M., Wildey, T.M.: Dakota, a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis: version 6.2 theory manual. Tech. Rep. SAND2014-4253, Sandia National Laboratories, Albuquerque (Updated May 2015). Available online from
  2. 2.
    Agarwal, N., Aluru, N.: A domain adaptive stochastic collocation approach for analysis of MEMS under uncertainties. J. Comput. Phys. 228, 7662–7688 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Alexandrov, N.M., Lewis, R.M., Gumbert, C.R., Green, L.L., Newman, P.A.: Approximation and model management in aerodynamic optimization with variable fidelity models. AIAA J. Aircr. 38(6), 1093–1101 (2001)CrossRefGoogle Scholar
  4. 4.
    Askey, R., Wilson, J.: Some Basic Hypergeometric Orthogonal Polynomials that Generalize Jacobi Polynomials. No. 319 in Memoirs of the American Mathematical Society. AMS, Providence (1985)Google Scholar
  5. 5.
    Babuška, I., Nobile, F., Tempone, R.: A stochastic collocation method for elliptic partial differential equations with random input data. SIAM J. Numer. Anal. 45(3), 1005–1034 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Barth, A., Schwab, C., Zollinger, N.: Multi-level monte carlo finite element method for elliptic PDEs with stochastic coefficients. Numer. Math. 119, 123–161 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Barthelmann, V., Novak, E., Ritter, K.: High dimensional polynomial interpolation on sparse grids. Adv. Comput. Math. 12(4), 273–288 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Bichon, B.J., Eldred, M.S., Swiler, L.P., Mahadevan, S., McFarland, J.M.: Efficient global reliability analysis for nonlinear implicit performance functions. AIAA J. 46(10), 2459–2468 (2008)CrossRefGoogle Scholar
  9. 9.
    Bungartz, H.J., Griebel, M.: Sparse grids. Acta Numer. 13, 147–269 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Cheung, S.H., Oliver, T.A., Prudencio, E.E., Prudhomme, S., Moser, R.D.: Bayesian uncertainty analysis with aplications to turbulence modeling. Reliab. Eng. Syst. Saf. 96, 1137–1149 (2011)CrossRefGoogle Scholar
  11. 11.
    Constantine, P.G., Eldred, M.S., Phipps, E.T.: Sparse pseudospectral approximation method. Comput. Methods Appl. Mech. Eng. Volumes 229–232, pp. 1–12 (2004)zbMATHGoogle Scholar
  12. 12.
    Der Kiureghian, A., Liu, P.L.: Structural reliability under incomplete information. J. Eng. Mech. ASCE 112(EM-1), 85–104 (1986)Google Scholar
  13. 13.
    Domino, S.P.: Towards verification of sliding mesh algorithms for complex applications using MMS. In: Proceedings of 2010 Center for Turbulence Research Summer Program, Stanford University (2010)Google Scholar
  14. 14.
    Eftang, J.L., Huynh, D.B.P., Knezevic, D.J., Patera, A.T.: A two-step certified reduced basis method. J. Sci. Comput. 51(1), pp 28–58 (2012)Google Scholar
  15. 15.
    Eldred, M., Wildey, T.: Propagation of model form uncertainty for thermal hydraulics using rans turbulence models in drekar. Tech. Rep. SAND2012-5845, Sandia National Laboratories, Albuquerque (2012)Google Scholar
  16. 16.
    Eldred, M.S., Giunta, A.A., Collis, S.S.: Second-order corrections for surrogate-based optimization with model hierarchies. In: 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Albany, AIAA 2004-4457 (2004)Google Scholar
  17. 17.
    Eldred, M.S., Phipps, E.T., Dalbey, K.R.: Adjoint enhancement within global stochastic methods. In: Proceedings of the SIAM Conference on Uncertainty Quantification, Raleigh (2012)Google Scholar
  18. 18.
    Gano, S.E., Renaud, J.E., Sanders, B.: Hybrid variable fidelity optimization by using a Kriging-based scaling function. AIAA J. 43(11), 2422–2430 (2005)CrossRefGoogle Scholar
  19. 19.
    Gautschi, W.: Orthogonal Polynomials: Computation and Approximation. Oxford University Press, New York (2004)zbMATHGoogle Scholar
  20. 20.
    Gerstner, T., Griebel, M.: Numerical integration using sparse grids. Numer. Algorithms 18(3), 209–232 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Gerstner, T., Griebel, M.: Dimension-adaptive tensor-product quadrature. Computing 71(1), 65–87 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Ghanem, R.G., Spanos, P.D.: Stochastic Finite Elements: A Spectral Approach. Springer, New York (1991)CrossRefzbMATHGoogle Scholar
  23. 23.
    Giles, M.B.: Multilevel Monte Carlo path simulation. Oper. Res. 56(3), 607–617 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Goh, J., Bingham, D., Holloway, J.P., Grosskopf, M.J., Kuranz, C.C., Rutter, E.: Prediction and computer model calibration using outputs from multi-fidelity simulators. Technometrics in review (2012)Google Scholar
  25. 25.
    Golub, G.H., Welsch, J.H.: Calculation of gauss quadrature rules. Math. Comput. 23(106), 221–230 (1969)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Huang, D., Allen, T.T., Notz, W.I., Miller, R.A.: Sequential Kriging optimization using multiple-fidelity evaluations. Struct. Multidisciplinary Optim. 32(5), 369–382 (2006)CrossRefGoogle Scholar
  27. 27.
    Jakeman, J., Eldred, M.S., Sargsyan, K.: Enhancing 1-minimization estimates of polynomial chaos expansions using basis selection. J. Comput. Phys. 289, 18–34 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Jakeman, J.D., Roberts, S.G.: Local and dimension adaptive stochastic collocation for uncertainty quantification. In: Proceedings of the Workshop on Sparse Grids and Applications, Bonn (2011)Google Scholar
  29. 29.
    Kennedy, M.C., O’Hagan, A.: Predicting the output from a complex computer code when fast approximations are available. Biometrika 87(1), 1–13 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Kennedy, M.C., O’Hagan, A.: Bayesian calibration of computer models. J. R. Stat. Soc. 63, 425–464 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Klimke, A., Wohlmuth, B.: Algorithm 847: spinterp: Piecewise multilinear hierarchical sparse grid interpolation in matlab. ACM Trans. Math. Softw. 31(4), 561–579 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Kuschel, N., Rackwitz, R.: Two basic problems in reliability-based structural optimization. Math. Method Oper. Res. 46, 309–333 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    March, A., Willcox, K.: Convergent multifidelity optimization using Bayesian model calibration. In: 13th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Fort Worth, AIAA 2010-9198 (2010)Google Scholar
  34. 34.
    Murray, J., Barone, M.: The development of CACTUS, a wind and marine turbine performance simulation code. In: 49th AIAA Aerospace Sciences Meeting, Orlando, AIAA 2011-147 (2011)Google Scholar
  35. 35.
    Narayan, A., Gittelson, C., Xiu, D.: A stochastic collocation algorithm with multifidelity models. SIAM J. Sci. Comput. 36(2), A495–A521 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Ng, L.W.T., Eldred, M.S.: Multifidelity uncertainty quantification using nonintrusive polynomial chaos and stochastic collocation. In Proceedings of the 53rd SDM Conference, Honolulu, Hawaii, AIAA-2012-1852 (2012)Google Scholar
  37. 37.
    Picard, R.R., Williams, B.J., Swiler, L.P., Urbina, A., Warr, R.L.: Multiple model inference with application to uncertainty quantification for complex codes. Tech. Rep. LA-UR-10-06382, Los Alamos National Laboratory, Los Alamos (2010)Google Scholar
  38. 38.
    Rajnarayan, D., Haas, A., Kroo, I.: A multifidelity gradient-free optimization method and application to aerodynamic design. In: 12th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Victoria, AIAA 2008-6020 (2008)Google Scholar
  39. 39.
    Rosenblatt, M.: Remarks on a multivariate transformation. Ann. Math. Stat. 23(3), 470–472 (1952)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Rozza, G., Huynh, D.B.P., Patera, A.T.: Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations. Arch. Comput. Methods Eng. 15(3), 229–275 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Witteveen, J.A.S., Bijl, H.: Modeling arbitrary uncertainties using gram-schmidt polynomial chaos. In: 44th AIAA Aerospace Sciences Meeting and Exhibit, Reno, AIAA 2006-896 (2006)Google Scholar
  42. 42.
    Xiu, D., Hesthaven, J.S.: High-order collocation methods for differential equations with random inputs. SIAM J. Sci. Comput. 27(3), 1118–1139 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Xiu, D., Karniadakis, G.E.: The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM J. Sci. Comput. 24(2), 619–644 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Zhu, X., Narayan, A., Xiu, D.: Computational aspects of stochastic collocation with multi-fidelity models. SIAM/ASA J. Uncertain. Quantif. 2, 444–463 (2014)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  • Michael S. Eldred
    • 1
    Email author
  • Leo W. T. Ng
    • 2
  • Matthew F. Barone
    • 3
  • Stefan P. Domino
    • 3
  1. 1.Optimization and Uncertainty Quantification DepartmentSandia National LaboratoriesAlbuquerqueUSA
  2. 2.Department of Aeronautics and AstronauticsMassachusetts Institute of TechnologyCambridgeUSA
  3. 3.Sandia National LaboratoriesAlbuquerqueUSA

Personalised recommendations