Surrogate Models for Uncertainty Propagation and Sensitivity Analysis

  • Khachik SargsyanEmail author
Reference work entry


For computationally intensive tasks such as design optimization, global sensitivity analysis, or parameter estimation, a model of interest needs to be evaluated multiple times exploring potential parameter ranges or design conditions. If a single simulation of the computational model is expensive, it is common to employ a precomputed surrogate approximation instead. The construction of an appropriate surrogate does still require a number of training evaluations of the original model. Typically, more function evaluations lead to more accurate surrogates, and therefore a careful accuracy-vs-efficiency tradeoff needs to take place for a given computational task. This chapter specifically focuses on polynomial chaos surrogates that are well suited for forward uncertainty propagation tasks, discusses a few construction mechanisms for such surrogates, and demonstrates the computational gain on select test functions.


Bayesian inference Global sensitivity analysis Polynomial chaos Regression Surrogate modeling 


  1. 1.
    Acquah, H.: Comparison of Akaike information criterion (AIC) and Bayesian information criterion (BIC) in selection of an asymmetric price relationship. J. Dev. Agric. Econ. 2(1), 001–006 (2010)Google Scholar
  2. 2.
    Arlot, S., Celisse, A., et al.: A survey of cross-validation procedures for model selection. Stat. Surv. 4, 40–79 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Babacan, S., Molina, R., Katsaggelos, A.: Bayesian compressive sensing using Laplace priors. IEEE Trans. Image Process. 19(1), 53–63 (2010)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Barthelmann, V., Novak, E., Ritter, K.: High-dimensional polynomial interpolation on sparse grids. Adv. Comput. Math. 12, 273–288 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Bastos, L., O’Hagan, A.: Diagnostics for Gaussian process emulators. Technometrics 51(4), 425–438 (2009)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Bernardo, J., Smith, A.: Bayesian Theory. Wiley Series in Probability and Statistics. Wiley, Chichester (2000)zbMATHGoogle Scholar
  7. 7.
    Blatman, G., Sudret, B.: Adaptive sparse polynomial chaos expansion based on least angle regression. J. Comput. Phys. 230(6), 2345–2367 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Borgonovo, E., Castaings, W., Tarantola, S.: Model emulation and moment-independent sensitivity analysis: an application to environmental modelling. Environ. Model. Softw. 34, 105–115 (2012)CrossRefGoogle Scholar
  9. 9.
    Candès, E., Romberg, J.: Sparsity and incoherence in compressive sampling. Inverse Probl. 23(3), 969–985 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Candès, E., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Carlin, B.P., Louis, T.A.: Bayesian Methods for Data Analysis. Chapman and Hall/CRC, Boca Raton (2011)zbMATHGoogle Scholar
  12. 12.
    Chantrasmi, T., Doostan, A., Iaccarino, G.: Padé-Legendre approximants for uncertainty analysis with discontinuous response surfaces. J. Comput. Phys. 228(19), 7159–7180 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Cox, D.A., Little, J., O’Shea, D.: Ideals, Varieties and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra. Springer,New York (1997)Google Scholar
  14. 14.
    Crestaux, T., Le Maître, O., Martinez, J.: Polynomial chaos expansion for sensitivity analysis. Reliab. Eng. Syst. Saf. 94(7), 1161–1172 (2009)CrossRefGoogle Scholar
  15. 15.
    Donoho, D.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Ernst, O., Mugler, A., Starkloff, H.J., Ullmann, E.: On the convergence of generalized polynomial chaos expansions. ESAIM: Math. Model. Numer. Anal. 46, 317–339 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Gamerman, D., Lopes, H.F.: Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference. Chapman and Hall/CRC, Boca Raton (2006)zbMATHGoogle Scholar
  18. 18.
    Genz, A.: Testing multidimensional integration routines. In: Proceedings of International Conference on Tools, Methods and Languages for Scientific and Engineering Computation. Elsevier North-Holland, Inc., pp 81–94 (1984)Google Scholar
  19. 19.
    Gerstner, T., Griebel, M.: Numerical integration using sparse grids. Numer. Algorithms 18, 209–232 (1998). doi:10.1023/A:1019129717644, (also as SFB 256 preprint 553, Univ. Bonn, 1998)Google Scholar
  20. 20.
    Ghanem, R., Spanos, P.: Stochastic Finite Elements: A Spectral Approach. Springer, New York (1991)CrossRefzbMATHGoogle Scholar
  21. 21.
    Ghosh, D., Ghanem, R.: Stochastic convergence acceleration through basis enrichment of polynomial chaos expansions. Int. J. Numer. Method Eng. 73, 162–174 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Gilks, W.R.: Markov Chain Monte Carlo. Wiley Online Library (2005)Google Scholar
  23. 23.
    Griebel, M.: Sparse grids and related approximation schemes for high dimensional problems. In: Proceedings of the Conference on Foundations of Computational Mathematics. Santander, Spain (2005)zbMATHGoogle Scholar
  24. 24.
    Huan, X., Marzouk, Y.: Simulation-based optimal Bayesian experimental design for nonlinear systems. J. Comput. Phys. 232, 288–317 (2013)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Isukapalli, S., Roy, A., Georgopoulos, P.: Stochastic response surface methods (SRSMs) for uncertainty propagation: application to environmental and biological systems. Risk Anal. 18(3), 351–363 (1998)CrossRefGoogle Scholar
  26. 26.
    Jakeman, J.D., Eldred, M.S., Sargsyan, K.: Enhancing 1-minimization estimates of polynomial chaos expansions using basis selection. J. Comput. Phys. 289, 18–34 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Jansen, M.J.: Analysis of variance designs for model output. Comput. Phys. Commun. 117(1), 35–43 (1999)CrossRefzbMATHGoogle Scholar
  28. 28.
    Jefferys, W.H., Berger, J.O.: Ockham’s razor and Bayesian analysis. Am. Sci. 80, 64–72 (1992)Google Scholar
  29. 29.
    Kapur, J.N.: Maximum-Entropy Models in Science and Engineering. Wiley, New Delhi (1989)zbMATHGoogle Scholar
  30. 30.
    Kass, R., Raftery, A.: Bayes factors. J. Am. Stat. Assoc. 90(430), 773–795 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Kersaudy, P., Sudret, B., Varsier, N., Picon, O., Wiart, J.: A new surrogate modeling technique combining Kriging and polynomial chaos expansions–application to uncertainty analysis in computational dosimetry. J. Comput. Phys. 286, 103–117 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Kullback, S., Leibler, R.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Le Maître, O., Knio, O.: Spectral Methods for Uncertainty Quantification. Springer, New York (2010)CrossRefzbMATHGoogle Scholar
  34. 34.
    Le Maître, O., Knio, O., Debusschere, B., Najm, H., Ghanem, R.: A multigrid solver for two-dimensional stochastic diffusion equations. Comput. Methods Appl. Mech. Eng. 192, 4723–4744 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Le Maître, O., Ghanem, R., Knio, O., Najm, H.: Uncertainty propagation using Wiener-Haar expansions. J. Comput. Phys. 197(1), 28–57 (2004a)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Le Maître, O., Najm, H., Ghanem, R., Knio, O.: Multi-resolution analysis of Wiener-type uncertainty propagation schemes. J. Comput. Phys. 197, 502–531 (2004b)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Le Maître, O., Najm, H., Pébay, P., Ghanem, R., Knio, O.: Multi-resolution analysis scheme for uncertainty quantification in chemical systems. SIAM J. Sci. Comput. 29(2), 864–889 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Li, G., Rosenthal, C., Rabitz, H.: High dimensional model representations. J. Phys. Chem. A 105, 7765–7777 (2001)CrossRefGoogle Scholar
  39. 39.
    Marrel, A., Iooss, B., Laurent, B., Roustant, O.: Calculations of Sobol indices for the Gaussian process metamodel. Reliab. Eng. Syst. Saf. 94(3), 742–751 (2009)CrossRefGoogle Scholar
  40. 40.
    Marzouk, Y.M., Najm, H.N.: Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems. J. Comput. Phys. 228(6), 1862–1902 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Marzouk, Y.M., Najm, H.N., Rahn, L.A.: Stochastic spectral methods for efficient Bayesian solution of inverse problems. J. Comput. Phys. 224(2), 560–586 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Mathelin, L., Gallivan, K.: A compressed sensing approach for partial differential equations with random input data. Commun. Comput. Phys. 12(4), 919–954 (2012)MathSciNetCrossRefGoogle Scholar
  43. 43.
    Moore, B., Natarajan, B.: A general framework for robust compressive sensing based nonlinear regression. In: 2012 IEEE 7th Sensor Array and Multichannel Signal Processing Workshop (SAM), Hoboken. IEEE, pp 225–228 (2012)Google Scholar
  44. 44.
    Najm, H.: Uncertainty quantification and polynomial chaos techniques in computational fluid dynamics. Ann. Rev. Fluid Mech. 41(1), 35–52 (2009). doi:10.1146/annurev.fluid.010908.165248MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Orr, M.: Introduction to radial basis function networks. Technical Report, Center for Cognitive Science, University of Edinburgh (1996)Google Scholar
  46. 46.
    Park, T., Casella, G.: The Bayesian Lasso. J. Am. Stat. Assoc. 103(482), 681–686 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Peng, J., Hampton, J., Doostan, A.: A weighted 1-minimization approach for sparse polynomial chaos expansions. J. Comput. Phys. 267, 92–111 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    Queipo, N.V., Haftka, R.T., Shyy, W., Goel, T., Vaidyanathan, R., Tucker, P.K.: Surrogate-based analysis and optimization. Prog. Aerosp. Sci. 41(1), 1–28 (2005)CrossRefGoogle Scholar
  49. 49.
    Rabitz, H., Alis, O.F.: General foundations of high-dimensional model representations. J. Math. Chem. 25, 197–233 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  50. 50.
    Rabitz, H., Alis, O.F., Shorter, J., Shim, K.: Efficient input-output model representations. Comput. Phys. Commun. 117, 11–20 (1999)CrossRefzbMATHGoogle Scholar
  51. 51.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT, Cambridge (2006)zbMATHGoogle Scholar
  52. 52.
    Rauhut, H., Ward, R.: Sparse Legendre expansions via 1-minimization. J. Approx. Theory 164(5), 517–533 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  53. 53.
    Reagan, M., Najm, H., Ghanem, R., Knio, O.: Uncertainty quantification in reacting flow simulations through non-intrusive spectral projection. Combust. Flame 132, 545–555 (2003)CrossRefGoogle Scholar
  54. 54.
    Reagan, M., Najm, H., Debusschere, B., Le Maître, O., Knio, O., Ghanem, R.: Spectral stochastic uncertainty quantification in chemical systems. Combust. Theory Model. 8, 607–632 (2004)CrossRefzbMATHGoogle Scholar
  55. 55.
    Rutherford, B., Swiler, L., Paez, T., Urbina, A.: Response surface (meta-model) methods and applications. In: Proceedings of 24th International Modal Analysis Conference, St. Louis, pp 184–197 (2006)Google Scholar
  56. 56.
    Saltelli, A.: Making best use of model evaluations to compute sensitivity indices. Comput. Phys. Commun. 145, 280–297 (2002). doi:10.1016/S0010-4655(02)00280-1CrossRefzbMATHGoogle Scholar
  57. 57.
    Saltelli, A., Tarantola, S., Campolongo, F., Ratto, M.: Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. Wiley, Chichester/Hoboken (2004)zbMATHGoogle Scholar
  58. 58.
    Saltelli, A., Annoni, P., Azzini, I., Campolongo, F., Ratto, M., Tarantola, S.: Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Comput. Phys. Commun. 181(2), 259–270 (2010)zbMATHGoogle Scholar
  59. 59.
    Sargsyan, K., Debusschere, B., Najm, H., Le Maître, O.: Spectral representation and reduced order modeling of the dynamics of stochastic reaction networks via adaptive data partitioning. SIAM J. Sci. Comput. 31(6), 4395–4421 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  60. 60.
    Sargsyan, K., Safta, C., Debusschere, B., Najm, H.: Multiparameter spectral representation of noise-induced competence in Bacillus subtilis. IEEE/ACM Trans. Comput. Biol. Bioinf. 9(6), 1709–1723 (2012a). doi:10.1109/TCBB.2012.107CrossRefGoogle Scholar
  61. 61.
    Sargsyan, K., Safta, C., Debusschere, B., Najm, H.: Uncertainty quantification given discontinuous model response and a limited number of model runs. SIAM J. Sci. Comput. 34(1), B44–B64 (2012b)MathSciNetCrossRefzbMATHGoogle Scholar
  62. 62.
    Sargsyan, K., Safta, C., Najm, H., Debusschere, B., Ricciuto, D., Thornton, P.: Dimensionality reduction for complex models via Bayesian compressive sensing. Int. J. Uncertain. Quantif. 4(1), 63–93 (2014). doi:10.1615/Int.J.UncertaintyQuantification.2013006821MathSciNetCrossRefGoogle Scholar
  63. 63.
    Sargsyan, K., Rizzi, F., Mycek, P., Safta, C., Morris, K., Najm, H., Le Maître, O., Knio, O., Debusschere, B.: Fault resilient domain decomposition preconditioner for PDEs. SIAM J. Sci. Comput. 37(5), A2317–A2345 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  64. 64.
    Schumaker, L.: Spline Functions: Basic Theory. Cambridge University Press, New York (2007)CrossRefzbMATHGoogle Scholar
  65. 65.
    Sivia, D.S., Skilling, J.: Data Analysis: A Bayesian Tutorial, 2nd edn. Oxford University Press, Oxford (2006)zbMATHGoogle Scholar
  66. 66.
    Sobol, I.M.: Sensitivity estimates for nonlinear mathematical models. Math. Model. Comput. Exp. 1, 407–414 (1993)MathSciNetzbMATHGoogle Scholar
  67. 67.
    Sobol, I.M.: Theorems and examples on high dimensional model representation. Reliab. Eng. Syst. Saf. 79, 187–193 (2003)CrossRefGoogle Scholar
  68. 68.
    Stein, M.L.: Interpolation of Spatial Data: Some Theory for Kriging. Springer Science & Business Media, New York (2012)Google Scholar
  69. 69.
    Sudret, B.: Global sensitivity analysis using polynomial Chaos expansions. Reliab. Eng. Syst. Saf. (2007). doi:10.1016/j.ress.2007.04.002Google Scholar
  70. 70.
    Sudret, B.: Meta-models for structural reliability and uncertainty quantification. In: Asian-Pacific Symposium on Structural Reliability and its Applications, Singapore, pp 1–24 (2012)Google Scholar
  71. 71.
    Wan, X., Karniadakis, G.E.: An adaptive multi-element generalized polynomial chaos method for stochastic differential equations. J. Comput. Phys. 209, 617–642 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  72. 72.
    Webster, M., Tatang, M., McRae, G.: Application of the probabilistic collocation method for an uncertainty analysis of a simple ocean model. Technical report, MIT Joint Program on the Science and Policy of Global Change Reports Series 4, MIT (1996)Google Scholar
  73. 73.
    Wiener, N.: The homogeneous chaos. Am. J. Math. 60, 897–936 (1938). doi:10.2307/2371268MathSciNetCrossRefzbMATHGoogle Scholar
  74. 74.
    Xiu, D.: Efficient collocational approach for parametric uncertainty analysis. Commun. Comput. Phys. 2(2), 293–309 (2007)MathSciNetzbMATHGoogle Scholar
  75. 75.
    Xiu, D.: Fast numerical methods for stochastic computations: a review. J. Comput. Phys. 5(2–4), 242–272 (2009)MathSciNetzbMATHGoogle Scholar
  76. 76.
    Xiu, D., Hesthaven, J.S.: High-order collocation methods for differential equations with random inputs. SIAM J. Sci. Comput. 27(3), 1118–1139 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  77. 77.
    Xiu, D., Karniadakis, G.: The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM J. Sci. Comput. 24(2), 619–644 (2002). doi:10.1137/S1064827501387826MathSciNetCrossRefzbMATHGoogle Scholar
  78. 78.
    Zuniga, M.M., Kucherenko, S., Shah, N.: Metamodelling with independent and dependent inputs. Comput. Phys. Commun. 184(6), 1570–1580 (2013)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland (outside the USA) 2017

Authors and Affiliations

  1. 1.Reacting Flow Research DepartmentSandia National LaboratoriesLivermoreUSA

Personalised recommendations