Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Global sensitivity analysis of stochastic computer models with joint metamodels


The global sensitivity analysis method used to quantify the influence of uncertain input variables on the variability in numerical model responses has already been applied to deterministic computer codes; deterministic means here that the same set of input variables always gives the same output value. This paper proposes a global sensitivity analysis methodology for stochastic computer codes, for which the result of each code run is itself random. The framework of the joint modeling of the mean and dispersion of heteroscedastic data is used. To deal with the complexity of computer experiment outputs, nonparametric joint models are discussed and a new Gaussian process-based joint model is proposed. The relevance of these models is analyzed based upon two case studies. Results show that the joint modeling approach yields accurate sensitivity index estimators even when heteroscedasticity is strong.

This is a preview of subscription content, log in to check access.


  1. Ankenman, B., Nelson, B., Staum, J.: Stochastic kriging for simulation metamodeling. Oper. Res. 58, 371–382 (2010)

  2. Bayarii, M.J., Berger, J., Cafeo, J., Garcia-Donato, G., Liu, F., Palomo, J., Parthasarathy, R.J., Paulo, R., Sacks, J., Walsh, D.: Computer model validation with functional output. Ann. Stat. 35, 1874–1906 (2007a)

  3. Bayarii, M.J., Berger, J., Paulo, R., Sacks, J., Cafeo, J.A., Cavendish, J., Lin, C., Tu, J.: A framework for validation of computer models. Technometrics 49, 138–154 (2007b)

  4. Boukouvalas, A., Cornford, D.: Learning heteroscedastic Gaussian processes for complex datasets. Technical Report, Neural Computing Research Group, Aston University, Birmingham, UK (2009)

  5. Bursztyn, D., Steinberg, D.: Screening experiments for dispersion effects. In: Dean, A., Lewis, S. (eds.) Screening—Methods for Experimentation in Industry, Drug Discovery and Genetics. Springer, Berlin (2006)

  6. Chen, V., Tsui, K.-L., Barton, R., Meckesheimer, M.: A review on design, modeling and applications of computer experiments. IIE Trans. 38, 273–291 (2006)

  7. Chilès, J.-P., Delfiner, P.: Geostatistics: Modeling Spatial Uncertainty. Wiley, New York (1999)

  8. Dellino, G., Kleijnen, J.P.C., Meloni, C.: Robust optimization in simulation: Taguchi and response surface methodology. Int. J. Prod. Econ. 125, 52–59 (2010)

  9. De Rocquigny, E., Devictor, N., Tarantola, S. (eds.): Uncertainty in Industrial Practice. Wiley, New York (2008)

  10. Fang, K.-T., Li, R., Sudjianto, A.: Design and Modeling for Computer Experiments. Chapman & Hall/CRC Press, London/Boca Raton (2006)

  11. Forrester, A.I.J., Keane, A.J., Bressloff, N.W.: Design and analysis of “Noisy” computer experiments. AIAA J. 44, 2331–2339 (2006)

  12. Gijbels, I., Prosdocimi, I., Claeskens, G.: Nonparametric estimation of mean and dispersion functions in extended generalized linear models. Test 19, 580–608 (2010)

  13. Ginsbourger, D., Roustant, O., Richet, Y.: Kriging with heterogeneous nugget effect for the approximation of noisy simulators with tunable fidelity. In: Proceedings of Joint Meeting of the Statistical Society of Canada and the Société Française de Statistique, Ottawa, Canada (2008)

  14. Gramacy, R.B., Lee, H.K.H.: Bayesian treed Gaussian process models with an application to computer modeling. J. Am. Stat. Assoc. 103, 1119–1130 (2008)

  15. Hastie, T., Tibshirani, R.: Generalized Additive Models. Chapman and Hall, London (1990)

  16. Helton, J.C.: Conceptual and computational basis for the quantification of margins and uncertainty. Sandia National Laboratories, Report SAND2009-3055 (2009)

  17. Helton, J.C., Johnson, J., Salaberry, C., Storlie, C.: Survey of sampling-based methods for uncertainty and sensitivity analysis. Reliab. Eng. Syst. Saf. 91, 1175–1209 (2006)

  18. Homma, T., Saltelli, A.: Importance measures in global sensitivity analysis of non linear models. Reliab. Eng. Syst. Saf. 52, 1–17 (1996)

  19. Iooss, B., Ribatet, M.: Global sensitivity analysis of computer models with functional inputs. Reliab. Eng. Syst. Saf. 94, 1194–1204 (2009)

  20. Iooss, B., Lhuillier, C., Jeanneau, H.: Numerical simulation of transit-time ultrasonic flowmeters due to flow profile and fluid turbulence. Ultrasonics 40, 1009–1015 (2002)

  21. Kelton, W.D., Sadowski, R.P., Sturrock, D.T.: Simulation with Arena, 4th edn. McGraw-Hill, Boston (2007)

  22. Kennedy, M., O’Hagan, A.: Bayesian calibration of computer models. J. R. Stat. Soc. 63(3), 425–464 (2001)

  23. Kersting, K., Plagemann, C., Pfaff, P., Burgard, W.: Most likely heteroscedastic Gaussian process regression. In: Proceedings of the 24th International Conference on Machine Learning, Corvallis, Oregon, USA (2007)

  24. Kleijnen, J.P.C.: Sensitivity analysis and related analyses: a review of some statistical techniques. J. Stat. Comput. Simul. 57, 111–142 (1997)

  25. Kleijnen, J.P.C.: Design and Analysis of Simulation Experiments. Springer, Berlin (2008)

  26. Kleijnen, J.P.C., van Beers, W.: Robustness of kriging when interpolating in random simulation with heterogeneous variances: some experiments. Eur. J. Oper. Res. 165, 826–834 (2005)

  27. Lee, Y., Nelder, J.: Robust design via generalized linear models. J. Qual. Technol. 35(1), 2–12 (2003)

  28. Manceau, E., Mezghani, M., Zabalza-Mezghani, I., Roggero, F.: Combination of experimental design and joint modeling methods for quantifying the risk associated with deterministic and stochastic uncertainties—an integrated test study. In: 2001 SPE Annual Technical Conference and Exhibition, New Orleans, 30 September–3 October (2001). paper SPE 71620

  29. Marrel, A., Iooss, B., Van Dorpe, F., Volkova, E.: An efficient methodology for modeling complex computer codes with Gaussian processes. Comput. Stat. Data Anal. 52, 4731–4744 (2008)

  30. Martin, J., Simpson, T.: Use of kriging models to approximate deterministic computer models. AIAA J. 43, 853–863 (2005)

  31. McCullagh, P., Nelder, J.: Generalized Linear Models. Chapman & Hall, London (1989)

  32. Myers, R.H., Montgomery, D.C., Anderson-Cook, C.M.: Response Surface Methodology: Process and Product Optimization Using Designed Experiments, 3rd edn. Wiley, New York (2009)

  33. Nelder, J., Pregibon, D.: An extended quasi-likelihood function. Biometrika 74, 221–232 (1987)

  34. Nelder, J., Wedderburn, R.: Generalized linear models. J. R. Stat. Soc. A 135, 370–384 (1972)

  35. Phadke, M.: Quality Engineering Using Robust Design. Prentice-Hall, New York (1989)

  36. Picheny, V., Ginsbourger, D., Richet, Y., Caplin, G.: Optimization of noisy computer experiments with tunable precision. Technometrics (2011, accepted, in revision). http://hal.archives-ouvertes.fr/hal-00578550_v1/

  37. Pope, B.: Lagrangian pdf methods for turbulent reactive flows. Annu. Rev. Fluid Mech. 26, 23–63 (1994)

  38. Reich, B.J., Kalendra, E., Storlie, C.B., Bondell, H.D., Fuentes, M.: Variable selection for high dimensional Bayesian density estimation: application to human exposure simulation. J. R. Stat. Soc., Ser. C (2011). doi:10.1111/j.1467-9876.2011.01020.x

  39. Robinson, T.J., Birch, J., Alden Starnes, B.: A semi-parametric approach to dual modeling when no replication exists. J. Stat. Plan. Inference 140, 2860–2869 (2010)

  40. Ruffo, P., Bazzana, L., Consonni, A., Corradi, A., Saltelli, A., Tarantola, S.: Hydrocarbon exploration risk evaluation through uncertainty and sensitivity analysis techniques. Reliab. Eng. Syst. Saf. 91, 1155–1162 (2006)

  41. Sacks, J., Welch, W., Mitchell, T., Wynn, H.: Design and analysis of computer experiments. Stat. Sci. 4, 409–435 (1989)

  42. Saltelli, A., Chan, K., Scott, E. (eds.): Sensitivity Analysis. Wiley Series in Probability and Statistics. Wiley, New York (2000)

  43. Saltelli, A., Annoni, P., Azzini, I., Campolongo, F., Ratto, M., Tarantola, S.: Variance based sensitivity analysis of model output. Design and estimator for total sensitivity index. Comput. Phys. Commun. 181, 259–270 (2010)

  44. Siebers, P.O., Macal, C.M., Garnett, J., Buxton, D., Pidd, M.: Discrete-event simulation is dead, long live agent-based simulation! J. Simul. 4, 204–210 (2010)

  45. Smyth, G.: Generalized linear models with varying dispersion. J. R. Stat. Soc. B 51, 47–60 (1989)

  46. Sobol, I.: Sensitivity estimates for non linear mathematical models. In: Mathematical Modelling and Computational Experiments, vol. 1, pp. 407–414 (1993)

  47. Storlie, C.B., Swiler, L.P., Helton, J.C., Salaberry, C.J.: Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models. Reliab. Eng. Syst. Saf. 94, 1735–1763 (2009)

  48. Vining, G., Myers, R.: Combining Taguchi and response-surface philosophies—a dual response approach. J. Qual. Technol. 22, 38–45 (1990)

  49. Volkova, E., Iooss, B., Van Dorpe, F.: Global sensitivity analysis for a numerical model of radionuclide migration from the RRC “Kurchatov Institute” radwaste disposal site. Stoch. Environ. Res. Risk Assess. 22, 17–31 (2008)

  50. Wood, S., Augustin, N.: GAMs with integrated model selection using penalized regression splines and applications to environmental modelling. Ecol. Model. 157, 157–177 (2002)

  51. Yeşilyurt, S., Ghaddar, C.K., Cruz, M.E., Patera, A.T.: Bayesian-validated surrogates for noisy computer simulations; application to random media. SIAM J. Sci. Comput. 17, 973–992 (1996)

  52. Zabalza, I., Dejean, J., Collombier, D.: Prediction and density estimation of a horizontal well productivity index using generalized linear models. In: Proceedings of ECMOR VI, Peebles, Scotland, 8–11 September 1998

  53. Zabalza, I., Manceau, E., Roggero, F.: A new approach for quantifying the impact of geostatistical uncertainty on production forecasts: The joint modeling method. In: Proceedings of IAMG Conference, Cancun, Mexico, 6–12 September 2001

  54. Zabalza-Mezghani, I., Manceau, E., Feraille, M., Jourdan, A.: Uncertainty management: from geological scenarios to production scheme optimization. J. Pet. Sci. Eng. 44, 11–25 (2004)

Download references

Author information

Correspondence to Amandine Marrel.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Marrel, A., Iooss, B., Da Veiga, S. et al. Global sensitivity analysis of stochastic computer models with joint metamodels. Stat Comput 22, 833–847 (2012). https://doi.org/10.1007/s11222-011-9274-8

Download citation


  • Computer experiment
  • Generalized additive model
  • Gaussian process
  • Joint modeling
  • Sobol indices
  • Uncertainty