Advertisement

A Review on Global Sensitivity Analysis Methods

  • Bertrand Iooss
  • Paul Lemaître
Part of the Operations Research/Computer Science Interfaces Series book series (ORCS, volume 59)

Abstract

This chapter makes a review, in a complete methodological framework, of various global sensitivity analysis methods of model output. Numerous statistical and probabilistic tools (regression, smoothing, tests, statistical learning, Monte Carlo, …) aim at determining the model input variables which mostly contribute to an interest quantity depending on model output. This quantity can be for instance the variance of an output variable. Three kinds of methods are distinguished: the screening (coarse sorting of the most influential inputs among a large number), the measures of importance (quantitative sensitivity indices) and the deep exploration of the model behaviour (measuring the effects of inputs on their all variation range). A progressive application methodology is illustrated on a scholar application. A synthesis is given to place every method according to several axes, mainly the cost in number of model evaluations, the model complexity and the nature of brought information.

Keywords

Computer code Numerical experiment Uncertainty Metamodel Design of experiment 

Notes

Acknowledgements

Part of this work has been backed by French National Research Agency (ANR) through COSINUS program (project COSTA BRAVA no. ANR-09-COSI-015). We thank Anne-Laure Popelin and Merlin Keller for providing the cobweb plot and the flood model figure.

References

  1. 1.
    Archer, G., Saltelli, A., Sobol, I.: Sensitivity measures, ANOVA-like techniques and the use of bootstrap. J. Stat. Comput. Simul. 58, 99–120 (1997)CrossRefGoogle Scholar
  2. 2.
    Auder, B., de Crécy, A., Iooss, B., Marquès, M.: Screening and metamodeling of computer experiments with functional outputs. Application to thermal-hydraulic computations. Reliab. Eng. Syst. Saf. 107, 122–131 (2012)CrossRefGoogle Scholar
  3. 3.
    Auder, B., Iooss, B.: Global sensitivity analysis based on entropy. In: Martorell, S., Guedes Soares, C., Barnett, J. (eds.), Safety, Reliability and Risk Analysis - Proceedings of the ESREL 2008 Conference, pp. 2107–2115. CRC Press, Valencia (2008)Google Scholar
  4. 4.
    Badea, A., Bolado, R.: Review of sensitivity analysis methods and experience. PAMINA 6th FPEC Project, European Commission (2008). http://www.ip-pamina.eu/downloads/pamina.m2.1.d.4.pdf
  5. 5.
    Bettonvil, B., Kleijnen, J.P.C.: Searching for important factors in simulation models with many factors: Sequential bifurcation. Eur. J. Oper. Res. 96, 180–194 (1996)CrossRefGoogle Scholar
  6. 6.
    Borgonovo, E.: A new uncertainty importance measure. Reliab. Eng. Syst. Saf. 92, 771–784 (2007)CrossRefGoogle Scholar
  7. 7.
    Borgonovo, E., Castaings, W., Tarantola, S.: Moment independent importance measures: new results and analytical test cases. Risk Anal. 31, 404–428 (2011)CrossRefGoogle Scholar
  8. 8.
    Box, G.E., Draper, N.R.: Empirical Model Building and Response Surfaces. Wiley Series in Probability and Mathematical Statistics. Wiley, New York (1987)Google Scholar
  9. 9.
    Cacuci, D.G.: Sensitivity theory for nonlinear systems. I. Nonlinear functional analysis approach. J. Math. Phys. 22, 2794 (1981)Google Scholar
  10. 10.
    Cacuci, D.G.: Sensitivity and Uncertainty Analysis - Theory. Chapman & Hall/CRC, Boca Raton (2003)CrossRefGoogle Scholar
  11. 11.
    Campbell, K., McKay, M.D., Williams, B.J.: Sensitivity analysis when model ouputs are functions. Reliab. Eng. Syst. Saf. 91, 1468–1472 (2006)CrossRefGoogle Scholar
  12. 12.
    Cannamela, C., Garnier, J., Iooss, B.: Controlled stratification for quantile estimation. Ann. Appl. Stat. 2, 1554–1580 (2008)CrossRefGoogle Scholar
  13. 13.
    Castaings, W., Dartus, D., Le Dimet, F.-X., Saulnier, G.-M.: Sensitivity analysis and parameter estimation for distributed hydrological modeling: potential of variational methods. Hydrol. Earth Syst. Sci. Discuss. 13, 503–517 (2009)Google Scholar
  14. 14.
    Chastaing, G., Gamboa, F., Prieur, C.: Generalized Hoeffding-sobol decomposition for dependent variables - Application to sensitivity analysis. Electron. J. Stat. 6, 2420–2448 (2012)CrossRefGoogle Scholar
  15. 15.
    Christensen, R.: Linear Models for Multivariate, Time Series and Spatial Data. Springer, New York (1990)Google Scholar
  16. 16.
    Cukier, H., Levine, R.I., Shuler, K.: Nonlinear sensitivity analysis of multiparameter model systems. J. Comput. Phys. 26, 1–42 (1978)CrossRefGoogle Scholar
  17. 17.
    Da Veiga, S., Wahl, F., Gamboa, F.: Local polynomial estimation for sensitivity analysis on models with correlated inputs. Technometrics 51(4), 452–463 (2009)CrossRefGoogle Scholar
  18. 18.
    de Rocquigny, E.: La maîtrise des incertitudes dans un contexte industriel - 1ère partie : une approche méthodologique globale basée sur des exemples. Journal de la Société Française de Statistique 147(3), 33–71 (2006)Google Scholar
  19. 19.
    de Rocquigny, E., Devictor, N., Tarantola, S. (eds.): Uncertainty in Industrial Practice. Wiley, Chichester (2008)Google Scholar
  20. 20.
    Dean, A., Lewis, S., (eds.): Screening - Methods for Experimentation in Industry, Drug Discovery and Genetics. Springer, New York (2006)Google Scholar
  21. 21.
    Downing, D.J., Gardner, R.H., Hoffman, F.O.: An examination of response surface methodologies for uncertainty analysis in assessment models. Technometrics 27(2), 151–163 (1985)CrossRefGoogle Scholar
  22. 22.
    Efron, B., Stein, C.: The jacknife estimate of variance. Ann. Stat. 9, 586–596 (1981)CrossRefGoogle Scholar
  23. 23.
    Faivre, R., Iooss, B., Mahévas, S., Makowski, D., Monod, H., (eds.): Analyse de sensibilité et exploration de modèles. Éditions Quaé, Versailles (2013)Google Scholar
  24. 24.
    Fang, K.-T., Li, R., Sudjianto, A.: Design and Modeling for Computer Experiments. Chapman & Hall/CRC, Boston (2006)Google Scholar
  25. 25.
    Frey, H.C., Patil, S.R.: Identification and review of sensitivity analysis methods. Risk Anal. 22, 553–578 (2002)CrossRefGoogle Scholar
  26. 26.
    Gamboa, F., Janon, A., Klein, T., Lagnoux, A.: Sensitivity indices for multivariate outputs. C. R. Acad. Sci. 351, 307–310 (2013)CrossRefGoogle Scholar
  27. 27.
    Gauchi, J.P., Lehuta, S., Mahévas, S.: Optimal sensitivity analysis under constraints. In: Procedia Social and Behavioral Sciences, vol. 2, pp. 7658–7659, Milan (2010)Google Scholar
  28. 28.
    Hastie, T., Tibshirani, R.: Generalized Additive Models. Chapman & Hall, London (1990)Google Scholar
  29. 29.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2002)Google Scholar
  30. 30.
    Helton, J.C.: Uncertainty and sensitivity analysis techniques for use in performance assesment for radioactive waste disposal. Reliab. Eng. Syst. Saf. 42, 327–367 (1993)CrossRefGoogle Scholar
  31. 31.
    Helton, J.C., Davis, F.J.: Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems. Reliab. Eng. Syst. Saf. 81, 23–69 (2003)CrossRefGoogle Scholar
  32. 32.
    Helton, J.C., Johnson, J.D., Salaberry, C.J., Storlie, C.B.: Survey of sampling-based methods for uncertainty and sensitivity analysis. Reliab. Eng. Syst. Saf. 91, 1175–1209 (2006)CrossRefGoogle Scholar
  33. 33.
    Hoeffding, W.: A class of statistics with asymptotically normal distributions. Ann. Math. Stat. 19, 293–325 (1948)CrossRefGoogle Scholar
  34. 34.
    Homma, T., Saltelli, A.: Importance measures in global sensitivity analysis of non linear models. Reliab. Eng. Syst. Saf. 52, 1–17 (1996)CrossRefGoogle Scholar
  35. 35.
    Iooss, B.: Revue sur l’analyse de sensibilité globale de modèles numériques. Journal de la Société Française de Statistique 152, 1–23 (2011)Google Scholar
  36. 36.
    Iooss, B., Ribatet, M.: Global sensitivity analysis of computer models with functional inputs. Reliab. Eng. Syst. Saf. 94, 1194–1204 (2009)CrossRefGoogle Scholar
  37. 37.
    Iooss, B., Van Dorpe, F., Devictor, N.: Response surfaces and sensitivity analyses for an environmental model of dose calculations. Reliab. Eng. Syst. Saf. 91, 1241–1251 (2006)CrossRefGoogle Scholar
  38. 38.
    Iooss, B., Boussouf, L., Feuillard, V., Marrel, A.: Numerical studies of the metamodel fitting and validation processes. Int. J. Adv. Syst. Meas. 3, 11–21 (2010)Google Scholar
  39. 39.
    Jacques, J., Lavergne, C., Devictor, N.: Sensitivity analysis in presence of model uncertainty and correlated inputs. Reliab. Eng. Syst. Saf. 91, 1126–1134 (2006)CrossRefGoogle Scholar
  40. 40.
    Janon, A., Klein, T., Lagnoux, A., Nodet, M., Prieur, C.: Asymptotic normality and efficiency of two sobol index estimators. ESAIM Probab. Stat. 18, 342–364 (2013)CrossRefGoogle Scholar
  41. 41.
    Jansen, M.J.W.: Analysis of variance designs for model output. Comput. Phys. Commun. 117, 25–43 (1999)CrossRefGoogle Scholar
  42. 42.
    Jourdan, A., Zabalza-Mezghani, I.: Response surface designs for scenario mangement and uncertainty quantification in reservoir production. Math. Geol. 36(8), 965–985 (2004)CrossRefGoogle Scholar
  43. 43.
    Kleijnen, J.P.C.: Sensitivity analysis and related analyses: a review of some statistical techniques. J. Stat. Comput. Simul. 57, 111–142 (1997)CrossRefGoogle Scholar
  44. 44.
    Kleijnen, J.P.C.: An overview of the design and analysis of simulation experiments for sensitivity analysis. Eur. J.Oper. Res. 164, 287–300 (2005)Google Scholar
  45. 45.
    Kleijnen, J.P.C., Helton, J.C.: Statistical analyses of scatterplots to identify important factors in large-scale simulations, 1: review and comparison of techniques. Reliab. Eng. Syst. Saf. 65, 147–185 (1999)CrossRefGoogle Scholar
  46. 46.
    Kleijnen, J.P.C., Sargent, R.G.: A methodology for fitting and validating metamodels in simulation. Eur. J.Oper. Res. 120, 14–29 (2000)Google Scholar
  47. 47.
    Krzykacz-Hausmann, B.: Epistemic sensitivity analysis based on the concept of entropy. In: Proceedings of SAMO 2001, pp. 31–35. CIEMAT, Madrid (2001)Google Scholar
  48. 48.
    Kurowicka, D., Cooke, R.: Uncertainty Analysis with High Dimensional Dependence Modelling. Wiley, Chichester (2006)CrossRefGoogle Scholar
  49. 49.
    Lamboni, M., Iooss, B., Popelin, A.-L., Gamboa, F.: Derivative-based global sensitivity measures: general links with sobol’ indices and numerical tests. Math. Comput. Simul. 87, 45–54 (2013)CrossRefGoogle Scholar
  50. 50.
    Lamboni, M., Monod, H., Makowski, D.: Multivariate sensitivity analysis to measure global contribution of input factors in dynamic models. Reliab. Eng. Syst. Saf. 96, 450–459 (2011)CrossRefGoogle Scholar
  51. 51.
    Le Gratiet, L., Cannamela, C., Iooss, B.: A Bayesian approach for global sensitivity analysis of (multifidelity) computer codes. SIAM/ASA J. Uncertain. Quantif. 2, 336–363 (2014)CrossRefGoogle Scholar
  52. 52.
    Lefebvre, S., Roblin, A., Varet, S., Durand, G.: A methodological approach for statistical evaluation of aircraft infrared signature. Reliab. Eng. Syst. Saf. 95, 484–493 (2010)CrossRefGoogle Scholar
  53. 53.
    Lemaître, P., Sergienko, E., Arnaud, A., Bousquet, N., Gamboa, F., Iooss, B.: Density modification based reliability sensitivity analysis. J. Stat. Comput. Simul. 85, 1200–1223 (2015)CrossRefGoogle Scholar
  54. 54.
    Li, G., Rabitz, H., Yelvington, P.E., Oluwole, O.O., Bacon, F., Kolb, C.E., Schoendorf, J.: Global sensitivity analysis for systems with independent and/or correlated inputs. J. Phys. Chem. 114, 6022–6032 (2010)CrossRefGoogle Scholar
  55. 55.
    Lilburne, L., Tarantola, S.: Sensitivity analysis of spatial models. Int. J. Geogr. Inf. Sci. 23, 151–168 (2009)CrossRefGoogle Scholar
  56. 56.
    Lin, D.K.J.: A new class of supersaturated design. Technometrics 35, 28–31 (1993)CrossRefGoogle Scholar
  57. 57.
    Liu, H., Chen, W., Sudjianto, A.: Relative entropy based method for probabilistic sensitivity analysis in engineering design. ASME J. Mech. Des. 128, 326–336 (2006)CrossRefGoogle Scholar
  58. 58.
    Makowski, D., Naud, C., Jeuffroy, M.H., Barbottin, A., Monod, H.: Global sensitivity analysis for calculating the contribution of genetic parameters to the variance of crop model prediction. Reliab. Eng. Syst. Saf. 91, 1142–1147 (2006)CrossRefGoogle Scholar
  59. 59.
    Marrel, A., Iooss, B., Van Dorpe, F., Volkova, E.: An efficient methodology for modeling complex computer codes with Gaussian processes. Comput. Stat. Data Anal. 52, 4731–4744 (2008)CrossRefGoogle Scholar
  60. 60.
    Marrel, A., Iooss, B., Laurent, B., Roustant, O.: Calculations of the Sobol indices for the Gaussian process metamodel. Reliab. Eng. Syst. Saf. 94, 742–751 (2009)CrossRefGoogle Scholar
  61. 61.
    Marrel, A., Iooss, B., Jullien, M., Laurent, B., Volkova, E.: Global sensitivity analysis for models with spatially dependent outputs. Environmetrics 22, 383–397 (2011)CrossRefGoogle Scholar
  62. 62.
    Marrel, A., Iooss, B., Da Veiga, S., Ribatet, M.: Global sensitivity analysis of stochastic computer models with joint metamodels. Stat. Comput. 22, 833–847 (2012)CrossRefGoogle Scholar
  63. 63.
    McKay, M.D., Beckman, R.J., Conover, W.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21, 239–245 (1979)Google Scholar
  64. 64.
    Montgomery, D.C.: Design and Analysis of Experiments, 6th edn. Wiley, Hoboken (2004)Google Scholar
  65. 65.
    Morris, M.D.: Factorial sampling plans for preliminary computational experiments. Technometrics 33, 161–174 (1991)CrossRefGoogle Scholar
  66. 66.
    Oakley, J.E., O’Hagan, A.: Probabilistic sensitivity analysis of complex models: a Bayesian approach. J. R. Stat. Soc. Ser. B 66, 751–769 (2004)CrossRefGoogle Scholar
  67. 67.
    Pappenberger, F., Ratto, M., Vandenberghe, V.: Review of sensitivity analysis methods. In: Vanrolleghem, P.A., (ed.) Modelling Aspects of Water Framework Directive Implementation, pp. 191–265. IWA Publishing, London (2010)Google Scholar
  68. 68.
    Pujol, G. (2009). Simplex-based screening designs for estimating metamodels. Reliab. Eng. Syst. Saf. 94, 1156–1160 (2009)Google Scholar
  69. 69.
    Roustant, O., Ginsbourger, D., Deville, Y.: DiceKriging, DiceOptim: Two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization. J. Stat. Softw., 21 1–55 (2012)Google Scholar
  70. 70.
    Roustant, O., Fruth, J., Iooss, B., Kuhnt, S.: Crossed-derivative-based sensitivity measures for interaction screening. 105, 105–118 (2014)Google Scholar
  71. 71.
    Sacks, J., Welch, W.J., Mitchell, T.J., Wynn, H.P.: Design and analysis of computer experiments. Stat. Sci. 4, 409–435 (1989)CrossRefGoogle Scholar
  72. 72.
    Saint-Geours, N., Lavergne, C., Bailly, J.-S., Grelot, F.: Analyse de sensibilité de sobol d’un modèle spatialisé pour l’evaluation économique du risque d’inondation. Journal de la Société Française de Statistique 152, 24–46 (2011)Google Scholar
  73. 73.
    Saltelli, A.: Making best use of model evaluations to compute sensitivity indices. Comput. Phys. Commun. 145, 280–297 (2002)CrossRefGoogle Scholar
  74. 74.
    Saltelli, A., Annoni, P.: How to avoid a perfunctory sensitivity analysis. Environ. Model. Softw. 25, 1508–1517 (2010)CrossRefGoogle Scholar
  75. 75.
    Saltelli, A., Tarantola, S.: On the relative importance of input factors in mathematical models: Safety assessment for nuclear waste disposal. J. Am. Stat. Assoc. 97, 702–709 (2002)CrossRefGoogle Scholar
  76. 76.
    Saltelli, A., Tarantola, S., Chan, K.: A quantitative, model-independent method for global sensitivity analysis of model output. Technometrics 41, 39–56 (1999)CrossRefGoogle Scholar
  77. 77.
    Saltelli, A., Chan, K., Scott, E.M. (eds.): Sensitivity Analysis. Wiley Series in Probability and Statistics. Wiley, New York (2000)Google Scholar
  78. 78.
    Saltelli, A., Tarantola, S., Campolongo, F., Ratto, M.: Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. Wiley, New York (2004)Google Scholar
  79. 79.
    Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Salsana, M., Tarantola, S.: Global Sensitivity Analysis - The Primer. Wiley, New York (2008)Google Scholar
  80. 80.
    Santner, T., Williams, B., Notz, W.: The Design and Analysis of Computer Experiments. Springer, New York (2003)CrossRefGoogle Scholar
  81. 81.
    Schonlau, M., Welch, W.J.: Screening the input variables to a computer model. In: Dean, A., Lewis, S., (eds.) Screening - Methods for Experimentation in Industry, Drug Discovery and Genetics. Springer, New York (2006)Google Scholar
  82. 82.
    Simpson, T.W., Peplinski, J.D., Kock, P.N., Allen, J.K.: Metamodel for computer-based engineering designs: Survey and recommandations. Eng. Comput. 17, 129–150 (2001)CrossRefGoogle Scholar
  83. 83.
    Sobol, I.M.: Sensitivity estimates for non linear mathematical models. Math. Model. Comput. Exp. 1, 407–414 (1993)Google Scholar
  84. 84.
    Sobol, I.M.: Theorems and examples on high dimensional model representation. Reliab. Eng. Syst. Saf. 79, 187–193 (2003)CrossRefGoogle Scholar
  85. 85.
    Sobol, I.M., Kucherenko, S.: Derivative based global sensitivity measures and their links with global sensitivity indices. Math. Comput. Simul. 79, 3009–3017 (2009)CrossRefGoogle Scholar
  86. 86.
    Sobol, I.M., Tarantola, S., Gatelli, D., Kucherenko, S.S., Mauntz, W.: Estimating the approximation errors when fixing unessential factors in global sensitivity analysis. Reliab. Eng. Syst. Saf. 92, 957–960 (2007)CrossRefGoogle Scholar
  87. 87.
    Storlie, C.B., Helton, J.C.: Multiple predictor smoothing methods for sensitivity analysis: Description of techniques. Reliab. Eng. Syst. Saf. 93, 28–54 (2008)CrossRefGoogle Scholar
  88. 88.
    Storlie, C.B., Swiler, L.P., Helton, J.C., Salaberry, C.J.: Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models. Reliab. Eng. Syst. Saf. 94, 1735–1763 (2009)CrossRefGoogle Scholar
  89. 89.
    Sudret, B.: Uncertainty propagation and sensitivity analysis in mechanical models - Contributions to structural reliability and stochastics spectral methods. Mémoire d’Habilitation à Diriger des Recherches de l’Université Blaise Pascal - Clermont II (2008)Google Scholar
  90. 90.
    Tarantola, S., Gatelli, D., Mara, T.: Random balance designs for the estimation of first order global sensitivity indices. Reliab. Eng. Syst. Saf. 91, 717–727 (2006)CrossRefGoogle Scholar
  91. 91.
    Tissot, J.-Y., Prieur, C.: A bias correction method for the estimation of sensitivity indices based on random balance designs. Reliab. Eng. Syst. Saf. 107, 205–213 (2012)CrossRefGoogle Scholar
  92. 92.
    Volkova, E., Iooss, B., Van Dorpe, F.: Global sensitivity analysis for a numerical model of radionuclide migration from the RRC “Kurchatov Institute” radwaste disposal site. Stoch. Env. Res. Risk A. 22, 17–31 (2008)CrossRefGoogle Scholar
  93. 93.
    Xu, C., Gertner, G.: Extending a global sensitivity analysis technique to models with correlated parameters. Comput. Stat. Data Anal. 51, 5579–5590 (2007)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Bertrand Iooss
    • 1
    • 2
  • Paul Lemaître
    • 1
    • 3
  1. 1.EDF R&DChatouFrance
  2. 2.Institut de Mathématiques de ToulouseToulouseFrance
  3. 3.INRIA Sud-OuestTalenceFrance

Personalised recommendations