An Industrial Viewpoint on Uncertainty Quantification in Simulation: Stakes, Methods, Tools, Examples

  • Alberto Pasanisi
  • Anne Dutfoy
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 377)


Simulation is nowadays a major tool in R&D and engineering studies. In industrial practice, in both design and operating stages, the behavior of a complex system is described and forecast by a computer model, which is, most of time, deterministic. Yet, engineers coping with quantitative predictions using deterministic models deal actually with several sources of uncertainties affecting the inputs (and occasionally the model itself) which are transferred to the outputs. Therefore, uncertainty quantification in simulation has garnered increased importance in recent years. In this paper we present an industrial viewpoint of this practice. After a reminder of the main stakes related to uncertainty quantification and probabilistic computing, we will focus on the specific methodology and software tools which have been developed for treating this problem at EDF R&D. We conclude with examples illustrating applied studies recently performed by EDF R&D engineers arising from different physical domains.


Simulation Computer Experiments Risk Uncertainty Reliability Sensitivity Analysis 


  1. 1.
    Schweber, S., Wachter, M.: Complex Systems, Modelling and Simulation. Stud. Hist. Phil. Mod. Phys. 31, 583–609 (2000)MathSciNetMATHCrossRefGoogle Scholar
  2. 2.
    Heymann, M.: Understanding and misunderstanding computer simulation: The case of atmospheric and climate science - An introduction. Stud. Hist. Phil. Mod. Phys. 41, 193–200 (2010)CrossRefGoogle Scholar
  3. 3.
    Sundberg, M.: Cultures of simulations vs. cultures of calculations? The development of simulation practices in meteorology and astrophysics. Stud. Hist. Phil. Mod. Phys. 41, 273–281 (2010)CrossRefGoogle Scholar
  4. 4.
    Pilkey, O.H., Pilkey-Jarvis, L.: Useless Arithmetic: Why Environmental Scientists Can’t Predict the Future. Columbia University Press, New York (2007)Google Scholar
  5. 5.
    Helton, J.C., Sallaberry, C.J.: Conceptual basis for the definition and calculation of expected dose in performance assessments for the proposed high-level radioactive waste repository at Yucca Mountain, Nevada. Reliab. Eng. Syst. Safe. 94, 677–698 (2009)CrossRefGoogle Scholar
  6. 6.
    Helton, J.C., Hansen, C.W., Sallaberry, C.J.: Uncertainty and sensitivity analysis in performance assessment for the proposed repository for high-level radioactive waste at Yucca Mountain, Nevada. Reliability Engineering & System Safety (2011) (in press), doi:10.1016/j.ress.2011.07.002Google Scholar
  7. 7.
    Zheng, Y., Wang, W., Han, F., Ping, J.: Uncertainty assessment for watershed water quality modeling: A Probabilistic Collocation Method based approach. Adv. Water Resourc. 34, 887–898 (2011)CrossRefGoogle Scholar
  8. 8.
    Eckert, N., Naaim, M., Parent, E.: Long-term avalanche hazard assessment with a Bayesian depth averaged propagation model. J. Glaciol. 56, 563–586 (2010)CrossRefGoogle Scholar
  9. 9.
    Asserin, O., Loredo, A., Petelet, M., Iooss, B.: Global sensitivity analysis in welding simulations - What are the material data you really need? Fin. El. Analys. Des. 47, 1004–1016 (2011)CrossRefGoogle Scholar
  10. 10.
    Hopfe, C.J., Hensen, J.L.M.: Uncertainty analysis in building performance simulation for design support. Energ. Buildings 43, 2798–2805 (2011)CrossRefGoogle Scholar
  11. 11.
    Vernon, I., Goldstein, M., Bower, R.G.: Galaxy Formation: a Bayesian Uncertainty Analysis. Bayes. Anal. 5, 619–670 (2010)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Crucifix, M., Rougier, J.: A Bayesian prediction of the next glacial inception. Eur. Phys. J. Special Topics 174, 11–31 (2009)CrossRefGoogle Scholar
  13. 13.
    Antoniadis, A., Helbert, C., Prieur, C., Viry, L.: Spatio-temporal prediction for West African monsoon. Environmetrics 23, 24–36 (2012)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Allard, A., Fischer, N., Didieux, F., Guillaume, E., Iooss, B.: Evaluation of the most influent input variables on quantities of interest in a fire simulation. J. Soc. Franc. Stat. 152, 103–117 (2011)Google Scholar
  15. 15.
    Baraldi, P., Pedroni, N., Zio, E., Ferrario, E., Pasanisi, A., Couplet, M.: Monte Carlo and fuzzy interval propagation of hybrid uncertainties on a risk model for the design of a flood protection dike. In: Berenguer, C., Grall, A., Guedes Soares, C. (eds.) Advances in Safety, Reliability and Risk Management: ESREL 2011. CRC Press, Leiden (2011)Google Scholar
  16. 16.
    O’Hagan, A., Buck, C.E., Daneshkhah, A., Eiser, J.R., Garthwaite, P.H., Jenkinson, D.J., Oakley, J.E., Rakow, T.: Uncertain judgements: eliciting expert probabilities. John Wiley & Sons, Chichester (2006)MATHCrossRefGoogle Scholar
  17. 17.
    Shannon, C.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423, 623–656 (1948)Google Scholar
  18. 18.
    Embrechts, P., Lindskog, F., McNeil, A.: Modelling dependence with copulas and applications to risk management. In: Rachev, S.T. (ed.) Handbook of Heavy Tailed Distributions in Finance, pp. 329–384. Elsevier, Amsterdam (2003)CrossRefGoogle Scholar
  19. 19.
    Dutfoy, A., Lebrun, R.: A practical approach to dependence modeling using copulas. Proc. Inst. Mech. Eng. O J. Risk Reliab. 223, 347–361 (2009)Google Scholar
  20. 20.
    Lebrun, R., Dutfoy, A.: An innovating analysis of the Nataf transformation from the viewpoint of copula. Probabilist. Eng. Mech. 24, 312–320 (2009)CrossRefGoogle Scholar
  21. 21.
    Del Moral, P.: Feynman-Kac Formulae - Genealogical and interacting particle systems with applications. Springer, New York (2004)MATHGoogle Scholar
  22. 22.
    Lebrun, R., Dutfoy, A.: A generalization of the Nataf transformation to distributions with elliptical copula. Probabilist. Eng. Mech. 24, 172–178 (2009)CrossRefGoogle Scholar
  23. 23.
    Lebrun, R., Dutfoy, A.: Do Rosenblatt and Nataf isoprobabilistic transformations really differ? Probabilist. Eng. Mech. 24, 577–584 (2009)CrossRefGoogle Scholar
  24. 24.
    Rosenblatt, M.: Remarks on a multivariate transformation. Ann. Math. Statist. 23, 470–472 (1952)MathSciNetMATHCrossRefGoogle Scholar
  25. 25.
    Dolinski, K.: First-order second-moment approximation in reliability of structural systems: critical review and alternative approach. Struct. Saf. 1, 211–231 (1983)CrossRefGoogle Scholar
  26. 26.
    Hasofer, A.M., Lind, N.C.: Exact and invariant second moment code format. J. Eng. Mech. 100, 111–121 (1974)Google Scholar
  27. 27.
    Tvedt, L.: Second order probability by an exact integral. In: Thoft-Christensen, P. (ed.) 2nd IFIP Working Conference on Reliability and Optimization on Structural Systems, pp. 377–384. Springer, Berlin (1988)Google Scholar
  28. 28.
    Ghanem, R.G., Spanos, P.D.: Stochastic Finite Elements: A Spectral Approach, Revised Edition. Dover, Mineola (2003)Google Scholar
  29. 29.
    Sacks, J., Welch, W.J., Mitchell, T.J., Wynn, H.P.: Design and Analysis of Computer Experiments. Stat. Sci. 4, 409–435 (1989)MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    Saltelli, A., Tarantola, S., Campolongo, F., Ratto, M.: Sensitivity analysis in practice: a guide to assessing scientific models. John Wiley & Sons, Chichester (2004)MATHGoogle Scholar
  31. 31.
    Open TURNS, Open Treatment of Uncertainties, Risk’s aNd Statistics, an open source source platform,
  32. 32.
    Saito, M., Matsumoto, M.: SIMD-oriented Fast Mersenne Twister: a 128-bit Pseudorandom Number Generator. In: Keller, A., Heinrich, S., Niederreiter, H. (eds.) Monte Carlo and Quasi-Monte Carlo Methods 2006, pp. 607–622. Springer, Berlin (2006)Google Scholar
  33. 33.
    Doornik, J.A.: An Improved Ziggurat Method to Generate Normal Random Samples. Working paper, Department of Economics, University of Oxford (2005)Google Scholar
  34. 34.
    Marsaglia, G., Tsang, W.W.: The Ziggurat method for generating random variables. J. Stat. Softw. 5, 1–7 (2000)Google Scholar
  35. 35.
    Benton, D., Krishnamoorthy, K.: Computing discrete mixtures of continuous distributions: noncentral chisquare, noncentral t and the distribution of the square of the sample multiple correlation coefficient. Comput. Stat. Data An. 43, 249–267 (2003)MathSciNetMATHCrossRefGoogle Scholar
  36. 36.
    Blatman, G., Sudret, B.: Efficient computation of global sensitivity indices using sparse polynomial chaos expansions. Reliab. Eng. Syst. Safe. 95, 1216–1229 (2010)CrossRefGoogle Scholar
  37. 37.
    Munoz Zuniga, M., Garnier, J., Remy, E., De Rocquigny, E.: Analysis of adaptive directional stratification for the controlled estimation of rare event probabilities. Stat. Comput. 22, 809–821 (2012)CrossRefGoogle Scholar
  38. 38.
    Arnaud, A., Goutal, N., De Rocquigny, E.: Influence des incertitudes sur les hydrogrammes de vidange de retenue en cas de rupture progressive d’un barrage en enrochements sur les zones inondées en aval. In: SimHydro 2010 Conference, Sophia Antipolis (2010) (in French)Google Scholar
  39. 39.
    Goutal, N., Maurel, F.: A finite volume solver for 1D shallow-water equations applied to an actual river. Int. J. Numer. Meth. Fl. 38, 1–19 (2002)MATHCrossRefGoogle Scholar
  40. 40.
    Berveiller, M., Blatman, G.: Sensitivity and reliability analysis of a globe valve using an adaptive sparse polynomial chaos expansion. In: 11th International Conference on Applications of Statistics and Probability in Civil Engineering, Zurich (2011)Google Scholar
  41. 41.
    OPUS Contributors: Final Report of the ANR OPUS project (2011),
  42. 42.
    EDF R&D: Code_Aster, Analysis of Structures and Thermomechanics for Studies & Research,
  43. 43.
    Sudret, B.: Global sensitivity analysis using polynomial chaos expansions. Reliab. Eng. Syst. Safe. 93, 964–979 (2008)CrossRefGoogle Scholar
  44. 44.
    Crestaux, T., Martinez, J.M., Le Maitre, O.: Polynomial chaos expansions for sensitivitiy analysis. Reliab. Eng. Syst. Safe. 94, 1161–1172 (2009)CrossRefGoogle Scholar
  45. 45.
    MUCM: Managing Uncertainty in Complex Models,
  46. 46.
    O’Hagan, A.: Bayesian analysis of computer code outputs: a tutorial. Reliab. Eng. Syst. Safe. 91, 1290–1300 (2006)CrossRefGoogle Scholar
  47. 47.
    Goldstein, M.: External Bayesian analysis for computer simulators. In: Bernardo, J.M., Bayarri, M.J., Berger, J.O., Dawid, A.P., Heckerman, D., Smith, A.F.M., West, M. (eds.) Bayesian Statistics 9, pp. 201–228. Oxford University Press (2011)Google Scholar
  48. 48.
    Celeux, G., Grimaud, A., Lefebvre, Y., De Rocquigny, E.: Identifying intrinsic variability in multivariate systems through linearised inverse methods. INRIA Research Report RR-6400 (2007)Google Scholar
  49. 49.
    Aven, T.: Some reflections on uncertainty analysis and management. Reliab. Eng. Syst. Safe. 95, 195–201 (2010)CrossRefGoogle Scholar
  50. 50.
    Limbourg, P., De Rocquigny, E.: Uncertainty analysis using evidence theory – confronting level-1 and level-2 approaches with data availability and computational constraints. Reliab. Eng. Syst. Safe. 95, 550–564 (2010)CrossRefGoogle Scholar
  51. 51.
    Le Duy, T.D., Vasseur, D., Couplet, M., Dieulle, L., Bérenguer, C.: A study on updating belief functions for parameter uncertainty representation in Nuclear Probabilistic Risk Assessment. In: 7th International Symposium on Imprecise Probability: Theories and Applications, Innsbruck (2011)Google Scholar
  52. 52.
    Baraldi, P., Pedroni, N., Zio, E., Ferrario, E., Pasanisi, A., Couplet, M.: Monte Carlo and fuzzy interval propagation of hybrid uncertainties on a risk model for the design of a flood protection dike. In: Berenguer, C., Grall, A., Guedes Soares, C. (eds.) Advances in Safety, Reliability and Risk Management: ESREL 2011, CRC Press, Leiden (2011)Google Scholar
  53. 53.
    Campolongo, F., Tarantola, S., Saltelli, A.: Tackling quantitatively large dimensionality problems. Comput. Phys. Commun. 117, 75–85 (1999)CrossRefGoogle Scholar
  54. 54.
    Morris, M.: Factorial sampling plans for preliminary computational experiments. Technometrics 33, 161–174 (1991)CrossRefGoogle Scholar
  55. 55.
    De Rocquigny, E., Devictor, N., Tarantola, S. (eds.): Uncertainty in Industrial Practice. Wiley, Chichester (2008)Google Scholar
  56. 56.
    Kurowicka, D., Cooke, R.: Uncertainty analysis with high dimensional dependence modelling. Wiley, Chichester (2006)MATHCrossRefGoogle Scholar
  57. 57.
    Xu, C., Gertner, G.Z.: Uncertainty and sensitivity analysis for models with correlated parameters. Reliab. Eng. Syst. Safe. 93, 1563–1573 (2008)CrossRefGoogle Scholar
  58. 58.
    Li, G., Rabitz, H., Yelvington, P.E., Oluwole, O.O., Bacon, F., Kolb, C.E., Schoendorf, J.: Global Sensitivity Analysis for Systems with Independent and/or Correlated Inputs. J. Phys. Chem. 114, 6022–6032 (2010)CrossRefGoogle Scholar
  59. 59.
    Borgonovo, E.: A new uncertainty importance measure. Reliab. Eng. Sys. Safe. 92, 771–784 (2007)CrossRefGoogle Scholar
  60. 60.
    Kucherenko, S., Munoz Zuniga, M., Tarantola, S., Annoni, P.: Metamodelling and Global Sensitivity Analysis of Models with Dependent Variables. In: AIP Conf. Proc., vol. 1389, pp. 1913–1916 (2011)Google Scholar
  61. 61.
    Group, E.D.F.: Exploiter les centrales nucléaires dans la durée (in French). Information Note (2011),
  62. 62.
    EDF Group: 2010 Activity and Sustainable Development Report (2011),

Copyright information

© IFIP International Federation for Information Processing 2012

Authors and Affiliations

  • Alberto Pasanisi
    • 1
  • Anne Dutfoy
    • 2
  1. 1.R&D. Industrial Risk Management Dept.EDFChatouFrance
  2. 2.R&D. Industrial Risk Management Dept.EDFClamartFrance

Personalised recommendations