Statistics and Computing

, Volume 22, Issue 3, pp 833–847 | Cite as

Global sensitivity analysis of stochastic computer models with joint metamodels

  • Amandine MarrelEmail author
  • Bertrand Iooss
  • Sébastien Da Veiga
  • Mathieu Ribatet


The global sensitivity analysis method used to quantify the influence of uncertain input variables on the variability in numerical model responses has already been applied to deterministic computer codes; deterministic means here that the same set of input variables always gives the same output value. This paper proposes a global sensitivity analysis methodology for stochastic computer codes, for which the result of each code run is itself random. The framework of the joint modeling of the mean and dispersion of heteroscedastic data is used. To deal with the complexity of computer experiment outputs, nonparametric joint models are discussed and a new Gaussian process-based joint model is proposed. The relevance of these models is analyzed based upon two case studies. Results show that the joint modeling approach yields accurate sensitivity index estimators even when heteroscedasticity is strong.


Computer experiment Generalized additive model Gaussian process Joint modeling Sobol indices Uncertainty 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Ankenman, B., Nelson, B., Staum, J.: Stochastic kriging for simulation metamodeling. Oper. Res. 58, 371–382 (2010) MathSciNetzbMATHCrossRefGoogle Scholar
  2. Bayarii, M.J., Berger, J., Cafeo, J., Garcia-Donato, G., Liu, F., Palomo, J., Parthasarathy, R.J., Paulo, R., Sacks, J., Walsh, D.: Computer model validation with functional output. Ann. Stat. 35, 1874–1906 (2007a) CrossRefGoogle Scholar
  3. Bayarii, M.J., Berger, J., Paulo, R., Sacks, J., Cafeo, J.A., Cavendish, J., Lin, C., Tu, J.: A framework for validation of computer models. Technometrics 49, 138–154 (2007b) MathSciNetCrossRefGoogle Scholar
  4. Boukouvalas, A., Cornford, D.: Learning heteroscedastic Gaussian processes for complex datasets. Technical Report, Neural Computing Research Group, Aston University, Birmingham, UK (2009) Google Scholar
  5. Bursztyn, D., Steinberg, D.: Screening experiments for dispersion effects. In: Dean, A., Lewis, S. (eds.) Screening—Methods for Experimentation in Industry, Drug Discovery and Genetics. Springer, Berlin (2006) Google Scholar
  6. Chen, V., Tsui, K.-L., Barton, R., Meckesheimer, M.: A review on design, modeling and applications of computer experiments. IIE Trans. 38, 273–291 (2006) CrossRefGoogle Scholar
  7. Chilès, J.-P., Delfiner, P.: Geostatistics: Modeling Spatial Uncertainty. Wiley, New York (1999) zbMATHCrossRefGoogle Scholar
  8. Dellino, G., Kleijnen, J.P.C., Meloni, C.: Robust optimization in simulation: Taguchi and response surface methodology. Int. J. Prod. Econ. 125, 52–59 (2010) CrossRefGoogle Scholar
  9. De Rocquigny, E., Devictor, N., Tarantola, S. (eds.): Uncertainty in Industrial Practice. Wiley, New York (2008) Google Scholar
  10. Fang, K.-T., Li, R., Sudjianto, A.: Design and Modeling for Computer Experiments. Chapman & Hall/CRC Press, London/Boca Raton (2006) zbMATHGoogle Scholar
  11. Forrester, A.I.J., Keane, A.J., Bressloff, N.W.: Design and analysis of “Noisy” computer experiments. AIAA J. 44, 2331–2339 (2006) CrossRefGoogle Scholar
  12. Gijbels, I., Prosdocimi, I., Claeskens, G.: Nonparametric estimation of mean and dispersion functions in extended generalized linear models. Test 19, 580–608 (2010) MathSciNetzbMATHCrossRefGoogle Scholar
  13. Ginsbourger, D., Roustant, O., Richet, Y.: Kriging with heterogeneous nugget effect for the approximation of noisy simulators with tunable fidelity. In: Proceedings of Joint Meeting of the Statistical Society of Canada and the Société Française de Statistique, Ottawa, Canada (2008) Google Scholar
  14. Gramacy, R.B., Lee, H.K.H.: Bayesian treed Gaussian process models with an application to computer modeling. J. Am. Stat. Assoc. 103, 1119–1130 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  15. Hastie, T., Tibshirani, R.: Generalized Additive Models. Chapman and Hall, London (1990) zbMATHGoogle Scholar
  16. Helton, J.C.: Conceptual and computational basis for the quantification of margins and uncertainty. Sandia National Laboratories, Report SAND2009-3055 (2009) Google Scholar
  17. Helton, J.C., Johnson, J., Salaberry, C., Storlie, C.: Survey of sampling-based methods for uncertainty and sensitivity analysis. Reliab. Eng. Syst. Saf. 91, 1175–1209 (2006) CrossRefGoogle Scholar
  18. Homma, T., Saltelli, A.: Importance measures in global sensitivity analysis of non linear models. Reliab. Eng. Syst. Saf. 52, 1–17 (1996) CrossRefGoogle Scholar
  19. Iooss, B., Ribatet, M.: Global sensitivity analysis of computer models with functional inputs. Reliab. Eng. Syst. Saf. 94, 1194–1204 (2009) CrossRefGoogle Scholar
  20. Iooss, B., Lhuillier, C., Jeanneau, H.: Numerical simulation of transit-time ultrasonic flowmeters due to flow profile and fluid turbulence. Ultrasonics 40, 1009–1015 (2002) CrossRefGoogle Scholar
  21. Kelton, W.D., Sadowski, R.P., Sturrock, D.T.: Simulation with Arena, 4th edn. McGraw-Hill, Boston (2007) Google Scholar
  22. Kennedy, M., O’Hagan, A.: Bayesian calibration of computer models. J. R. Stat. Soc. 63(3), 425–464 (2001) MathSciNetzbMATHCrossRefGoogle Scholar
  23. Kersting, K., Plagemann, C., Pfaff, P., Burgard, W.: Most likely heteroscedastic Gaussian process regression. In: Proceedings of the 24th International Conference on Machine Learning, Corvallis, Oregon, USA (2007) Google Scholar
  24. Kleijnen, J.P.C.: Sensitivity analysis and related analyses: a review of some statistical techniques. J. Stat. Comput. Simul. 57, 111–142 (1997) zbMATHCrossRefGoogle Scholar
  25. Kleijnen, J.P.C.: Design and Analysis of Simulation Experiments. Springer, Berlin (2008) zbMATHGoogle Scholar
  26. Kleijnen, J.P.C., van Beers, W.: Robustness of kriging when interpolating in random simulation with heterogeneous variances: some experiments. Eur. J. Oper. Res. 165, 826–834 (2005) zbMATHCrossRefGoogle Scholar
  27. Lee, Y., Nelder, J.: Robust design via generalized linear models. J. Qual. Technol. 35(1), 2–12 (2003) Google Scholar
  28. Manceau, E., Mezghani, M., Zabalza-Mezghani, I., Roggero, F.: Combination of experimental design and joint modeling methods for quantifying the risk associated with deterministic and stochastic uncertainties—an integrated test study. In: 2001 SPE Annual Technical Conference and Exhibition, New Orleans, 30 September–3 October (2001). paper SPE 71620 Google Scholar
  29. Marrel, A., Iooss, B., Van Dorpe, F., Volkova, E.: An efficient methodology for modeling complex computer codes with Gaussian processes. Comput. Stat. Data Anal. 52, 4731–4744 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  30. Martin, J., Simpson, T.: Use of kriging models to approximate deterministic computer models. AIAA J. 43, 853–863 (2005) CrossRefGoogle Scholar
  31. McCullagh, P., Nelder, J.: Generalized Linear Models. Chapman & Hall, London (1989) zbMATHGoogle Scholar
  32. Myers, R.H., Montgomery, D.C., Anderson-Cook, C.M.: Response Surface Methodology: Process and Product Optimization Using Designed Experiments, 3rd edn. Wiley, New York (2009) zbMATHGoogle Scholar
  33. Nelder, J., Pregibon, D.: An extended quasi-likelihood function. Biometrika 74, 221–232 (1987) MathSciNetzbMATHCrossRefGoogle Scholar
  34. Nelder, J., Wedderburn, R.: Generalized linear models. J. R. Stat. Soc. A 135, 370–384 (1972) CrossRefGoogle Scholar
  35. Phadke, M.: Quality Engineering Using Robust Design. Prentice-Hall, New York (1989) Google Scholar
  36. Picheny, V., Ginsbourger, D., Richet, Y., Caplin, G.: Optimization of noisy computer experiments with tunable precision. Technometrics (2011, accepted, in revision).
  37. Pope, B.: Lagrangian pdf methods for turbulent reactive flows. Annu. Rev. Fluid Mech. 26, 23–63 (1994) MathSciNetCrossRefGoogle Scholar
  38. Reich, B.J., Kalendra, E., Storlie, C.B., Bondell, H.D., Fuentes, M.: Variable selection for high dimensional Bayesian density estimation: application to human exposure simulation. J. R. Stat. Soc., Ser. C (2011). doi: 10.1111/j.1467-9876.2011.01020.x Google Scholar
  39. Robinson, T.J., Birch, J., Alden Starnes, B.: A semi-parametric approach to dual modeling when no replication exists. J. Stat. Plan. Inference 140, 2860–2869 (2010) zbMATHCrossRefGoogle Scholar
  40. Ruffo, P., Bazzana, L., Consonni, A., Corradi, A., Saltelli, A., Tarantola, S.: Hydrocarbon exploration risk evaluation through uncertainty and sensitivity analysis techniques. Reliab. Eng. Syst. Saf. 91, 1155–1162 (2006) CrossRefGoogle Scholar
  41. Sacks, J., Welch, W., Mitchell, T., Wynn, H.: Design and analysis of computer experiments. Stat. Sci. 4, 409–435 (1989) MathSciNetzbMATHCrossRefGoogle Scholar
  42. Saltelli, A., Chan, K., Scott, E. (eds.): Sensitivity Analysis. Wiley Series in Probability and Statistics. Wiley, New York (2000) zbMATHGoogle Scholar
  43. Saltelli, A., Annoni, P., Azzini, I., Campolongo, F., Ratto, M., Tarantola, S.: Variance based sensitivity analysis of model output. Design and estimator for total sensitivity index. Comput. Phys. Commun. 181, 259–270 (2010) MathSciNetzbMATHCrossRefGoogle Scholar
  44. Siebers, P.O., Macal, C.M., Garnett, J., Buxton, D., Pidd, M.: Discrete-event simulation is dead, long live agent-based simulation! J. Simul. 4, 204–210 (2010) CrossRefGoogle Scholar
  45. Smyth, G.: Generalized linear models with varying dispersion. J. R. Stat. Soc. B 51, 47–60 (1989) MathSciNetGoogle Scholar
  46. Sobol, I.: Sensitivity estimates for non linear mathematical models. In: Mathematical Modelling and Computational Experiments, vol. 1, pp. 407–414 (1993) Google Scholar
  47. Storlie, C.B., Swiler, L.P., Helton, J.C., Salaberry, C.J.: Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models. Reliab. Eng. Syst. Saf. 94, 1735–1763 (2009) CrossRefGoogle Scholar
  48. Vining, G., Myers, R.: Combining Taguchi and response-surface philosophies—a dual response approach. J. Qual. Technol. 22, 38–45 (1990) Google Scholar
  49. Volkova, E., Iooss, B., Van Dorpe, F.: Global sensitivity analysis for a numerical model of radionuclide migration from the RRC “Kurchatov Institute” radwaste disposal site. Stoch. Environ. Res. Risk Assess. 22, 17–31 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  50. Wood, S., Augustin, N.: GAMs with integrated model selection using penalized regression splines and applications to environmental modelling. Ecol. Model. 157, 157–177 (2002) CrossRefGoogle Scholar
  51. Yeşilyurt, S., Ghaddar, C.K., Cruz, M.E., Patera, A.T.: Bayesian-validated surrogates for noisy computer simulations; application to random media. SIAM J. Sci. Comput. 17, 973–992 (1996) MathSciNetCrossRefGoogle Scholar
  52. Zabalza, I., Dejean, J., Collombier, D.: Prediction and density estimation of a horizontal well productivity index using generalized linear models. In: Proceedings of ECMOR VI, Peebles, Scotland, 8–11 September 1998 Google Scholar
  53. Zabalza, I., Manceau, E., Roggero, F.: A new approach for quantifying the impact of geostatistical uncertainty on production forecasts: The joint modeling method. In: Proceedings of IAMG Conference, Cancun, Mexico, 6–12 September 2001 Google Scholar
  54. Zabalza-Mezghani, I., Manceau, E., Feraille, M., Jourdan, A.: Uncertainty management: from geological scenarios to production scheme optimization. J. Pet. Sci. Eng. 44, 11–25 (2004) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Amandine Marrel
    • 1
    Email author
  • Bertrand Iooss
    • 2
  • Sébastien Da Veiga
    • 1
  • Mathieu Ribatet
    • 3
  1. 1.IFP Energies NouvellesRueil-MalmaisonFrance
  2. 2.EDF, R&DChatouFrance
  3. 3.Université Montpellier IIMontpellierFrance

Personalised recommendations