Advertisement

Introduction to Sensitivity Analysis

  • Bertrand IoossEmail author
  • Andrea Saltelli
Reference work entry

Abstract

Sensitivity analysis provides users of mathematical and simulation models with tools to appreciate the dependency of the model output from model input and to investigate how important is each model input in determining its output. All application areas are concerned, from theoretical physics to engineering and socio-economics. This introductory paper provides the sensitivity analysis aims and objectives in order to explain the composition of the overall “Sensitivity Analysis” chapter of the Springer Handbook. It also describes the basic principles of sensitivity analysis, some classification grids to understand the application ranges of each method, a useful software package, and the notations used in the chapter papers. This section also offers a succinct description of sensitivity auditing, a new discipline that tests the entire inferential chain including model development, implicit assumptions, and normative issues and which is recommended when the inference provided by the model needs to feed into a regulatory or policy process. For the “Sensitivity Analysis” chapter, in addition to this introduction, eight papers have been written by around twenty practitioners from different fields of application. They cover the most widely used methods for this subject: the deterministic methods as the local sensitivity analysis, the experimental design strategies, the sampling-based and variance-based methods developed from the 1980s, and the new importance measures and metamodel-based techniques established and studied since the 2000s. In each paper, toy examples or industrial applications illustrate their relevance and usefulness.

Keywords

Computer experiments Uncertainty analysis Sensitivity analysis Sensitivity auditing Risk assessment Impact assessment 

References

  1. 1.
    Berger, J.: An overview of robust Bayesian analysis (with discussion). Test 3, 5–124 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Borgonovo, E., Plischke, E.: Sensitivity analysis: a review of recent advances. Eur. J. Oper. Res. 248, 869–887 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Cacuci, D.: Sensitivity and Uncertainty Analysis – Theory. Chapman & Hall/CRC, Boca Raton (2003)CrossRefzbMATHGoogle Scholar
  4. 4.
    Castaings, W., Dartus, D., Le Dimet, F.X., Saulnier, G.M.: Sensitivity analysis and parameter estimation for distributed hydrological modeling: potential of variational methods. Hydrol. Earth Syst. Sci. Discuss. 13, 503–517 (2009)Google Scholar
  5. 5.
    Chastaing, G., Gamboa, F., Prieur, C.: Generalized Hoeffding-Sobol decomposition for dependent variables – application to sensitivity analysis. Electron. J. Stat. 6, 2420–2448 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Da Veiga, S.: Global sensitivity analysis with dependence measures. J. Stat. Comput. Simul. 85, 1283–1305 (2015)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Da Veiga, S., Wahl, F., Gamboa, F.: Local polynomial estimation for sensitivity analysis on models with correlated inputs. Technometrics 51(4), 452–463 (2009)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Dean, A., Lewis, S. (eds.): Screening – Methods for Experimentation in Industry, Drug Discovery and Genetics. Springer, New York (2006)zbMATHGoogle Scholar
  9. 9.
    De Castro, Y., Janon, A.: Randomized pick-freeze for sparse Sobol indices estimation in high dimension. ESAIM Probab. Stat. 19, 725–745 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    de Rocquigny, E., Devictor, N., Tarantola, S. (eds.): Uncertainty in Industrial Practice. Wiley, Chichester/Hoboken (2008)Google Scholar
  11. 11.
    Faivre, R., Iooss, B., Mahévas, S., Makowski, D., Monod, H. (eds.): Analyse de sensibilité et exploration de modèles. Éditions Quaé (2013)Google Scholar
  12. 12.
    Fang, K.T., Li, R., Sudjianto, A.: Design and Modeling for Computer Experiments. Chapman & Hall/CRC, Boca Raton (2006)zbMATHGoogle Scholar
  13. 13.
    Fisher, R.W.: Remembering Carol Reed, Aesop’s Fable, Kenneth Arrow and Thomas Dewey. In: Speech: An Economic Overview: What’s Next, Federal Reserve Bank of Dallas. http://www.dallasfed.org/news/speeches/fisher/2011/fs110713.cfm (2011)
  14. 14.
    Fort, J., Klein, T., Rachdi, N.: New sensitivity analysis subordinated to a contrast. Commun. Stat. Theory Methods (2014, in press). http://www.tandfonline.com/doi/full/10.1080/03610926.2014.901369#abstract
  15. 15.
    Frey, H., Patil, S.: Identification and review of sensitivity analysis methods. Risk Anal. 22, 553–578 (2002)CrossRefGoogle Scholar
  16. 16.
    Fruth, J., Roustant, O., Kuhnt, S.: Total interaction index: a variance-based sensitivity index for second-order interaction screening. J. Stat. Plan. Inference 147, 212–223 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Funtowicz, S., Ravetz, J.: Uncertainty and Quality in Science for Policy. Kluwer Academic, Dordrecht (1990)CrossRefGoogle Scholar
  18. 18.
    Geraci, G., Congedo, P., Iaccarino, G.: Decomposing high-order statistics for sensitivity analysis. In: Thermal & Fluid Sciences Industrial Affiliates and Sponsors Conference, Stanford University, Stanford (2015)Google Scholar
  19. 19.
    Grundmann, R.: The role of expertise in governance processes. For. Policy Econ. 11, 398–403 (2009)CrossRefGoogle Scholar
  20. 20.
    Helton, J.: Uncertainty and sensitivity analysis techniques for use in performance assesment for radioactive waste disposal. Reliab. Eng. Syst. Saf. 42, 327–367 (1993)CrossRefGoogle Scholar
  21. 21.
    Helton, J.: Uncertainty and sensitivity analysis for models of complex systems. In: Graziani, F. (ed.) Computational Methods in Transport: Verification and Validation, pp. 207–228. Springer, New-York (2008)CrossRefGoogle Scholar
  22. 22.
    Helton, J., Johnson, J., Obekampf, W., Salaberry, C.: Sensitivity analysis in conjunction with evidence theory representations of epistemic uncertainty. Reliab. Eng. Syst. Saf. 91, 1414–1434 (2006a)CrossRefGoogle Scholar
  23. 23.
    Helton, J., Johnson, J., Salaberry, C., Storlie, C.: Survey of sampling-based methods for uncertainty and sensitivity analysis. Reliab. Eng. Syst. Saf. 91, 1175–1209 (2006b)CrossRefGoogle Scholar
  24. 24.
    Insua, D., Ruggeri, F. (eds.): Robust Bayesian Analysis. Springer, New York (2000)zbMATHGoogle Scholar
  25. 25.
    Ioannidis, J.P.A.: Why most published research findings are false. PLoS Med. 2(8), 696–701 (2005)CrossRefGoogle Scholar
  26. 26.
    Iooss, B., Lemaître, P.: A review on global sensitivity analysis methods. In: Meloni, C., Dellino, G. (eds.) Uncertainty Management in Simulation-Optimization of Complex Systems: Algorithms and Applications. Springer, New York (2015)Google Scholar
  27. 27.
    Jacques, J., Lavergne, C., Devictor, N.: Sensitivity analysis in presence of model uncertainty and correlated inputs. Reliab. Eng. Syst. Saf. 91, 1126–1134 (2006)CrossRefGoogle Scholar
  28. 28.
    Kay, J.: A wise man knows one thing – the limits of his knowledge. Financial Times 29 Nov 2011Google Scholar
  29. 29.
    Kennedy, P.: A Guide to Econometrics, 5th edn. Blackwell Publishing, Oxford (2007)Google Scholar
  30. 30.
    Kleijnen, J.: Sensitivity analysis and related analyses: a review of some statistical techniques. J. Stat. Comput. Simul. 57, 111–142 (1997)CrossRefzbMATHGoogle Scholar
  31. 31.
    Kucherenko, S., Tarantola, S., Annoni, P.: Estimation of global sensitivity indices for models with dependent variables. Comput. Phys. Commun. 183, 937–946 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Kurowicka, D., Cooke, R.: Uncertainty Analysis with High Dimensional Dependence Modelling. Wiley, Chichester/Hoboken (2006)CrossRefzbMATHGoogle Scholar
  33. 33.
    Latour, B.: We Have Never Been Modern. Harvard University Press, Cambridge (1993)Google Scholar
  34. 34.
    Leamer, E.E.: Tantalus on the road to asymptopia. J. Econ. Perspect. 4(2), 31–46 (2010)CrossRefGoogle Scholar
  35. 35.
    Lemaître, P., Sergienko, E., Arnaud, A., Bousquet, N., Gamboa, F., Iooss, B.: Density modification based reliability sensitivity analysis. J. Stat. Comput. Simul. 85, 1200–1223 (2015)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Li, G., Rabitz, H., Yelvington, P., Oluwole, O., Bacon, F., Kolb, C., Schoendorf, J.: Global sensitivity analysis for systems with independent and/or correlated inputs. J. Phys. Chem. 114, 6022–6032 (2010)CrossRefGoogle Scholar
  37. 37.
    Mara, T.: Extension of the RBD-FAST method to the computation of global sensitivity indices. Reliab. Eng. Syst. Saf. 94, 1274–1281 (2009)CrossRefGoogle Scholar
  38. 38.
    Mara, T., Joseph, O.: Comparison of some efficient methods to evaluate the main effect of computer model factors. J. Stat. Comput. Simul. 78, 167–178 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Mara, T., Tarantola, S.: Variance-based sensitivity indices for models with dependent inputs. Reliability Engineering and System Safety 107, 115–121 (2012)CrossRefGoogle Scholar
  40. 40.
    Marrel, A., Iooss, B., Da Veiga, S., Ribatet, M.: Global sensitivity analysis of stochastic computer models with joint metamodels. Stat. Comput. 22, 833–847 (2012)Google Scholar
  41. 41.
    Marris, C., Wynne, B., Simmons, P., Weldon, S.: Final report of the PABE research project funded by the Commission of European Communities. Technical report contract number: FAIR CT98-3844 (DG12 – SSMI), Commission of European Communities (2001)Google Scholar
  42. 42.
    Monbiot, G.: Beware the rise of the government scientists turned lobbyists. The Guardian 29 Apr 2013Google Scholar
  43. 43.
    Moutoussamy, V., Nanty, S., Pauwels, B.: Emulators for stochastic simulation codes. ESAIM: Proc. Surv. 48, 116–155 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Oreskes, N., Conway, E.M.: Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Global Warming. Bloomsbury Press, New York (2010)Google Scholar
  45. 45.
    Owen, A.: Better estimation of small Sobol’ sensitivity indices. ACM Trans. Model. Comput. Simul. 23, 11 (2013a)MathSciNetCrossRefGoogle Scholar
  46. 46.
    Owen, A.: Variance components and generalized Sobol’ indices. J. Uncert. Quantif. 1, 19–41 (2013b)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Owen, A., Dick, J., Chen, S.: Higher order Sobol’ indices. Inf. Inference: J. IMA 3, 59–81 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    Park, K., Xu, L.: Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications. Springer, Dordrecht (2008)Google Scholar
  49. 49.
    Pujol, G., Iooss, B., Janon, A.: Sensitivity Package, Version 1.11. The Comprenhensive R Archive Network. http://www.cran.r-project.org/web/packages/sensitivity/ (2015)
  50. 50.
    Rakovec, O., Hill, M.C., Clark, M.P., Weerts, A.H., Teuling, A.J., Uijlenhoet, R.: Distributed evaluation of local sensitivity analysis (DELSA), with application to hydrologic models. Water Resour. Res. 50, 1–18 (2014)CrossRefGoogle Scholar
  51. 51.
    Saltelli, A.: Making best use of model evaluations to compute sensitivity indices. Comput. Phys. Commun. 145, 280–297 (2002)CrossRefzbMATHGoogle Scholar
  52. 52.
    Saltelli, A., d’Hombres, B.: Sensitivity analysis didn’t help. A practitioners critique of the Stern review. Glob. Environ. Change 20(2), 298–302 (2010)CrossRefGoogle Scholar
  53. 53.
    Saltelli, A., Funtowicz, S.: When all models are wrong: more stringent quality criteria are needed for models used at the science-policy interface. Issues Sci. Technol. XXX(2), 79–85 (2014, Winter)Google Scholar
  54. 54.
    Saltelli, A., Funtowicz, S.: Evidence-based policy at the end of the Cartesian dream: the case of mathematical modelling. In: Pereira, G., Funtowicz, S. (eds.) The End of the Cartesian Dream. Beyond the Techno–Scientific Worldview. Routledge’s Series: Explorations in Sustainability and Governance, pp. 147–162. Routledge, London (2015)Google Scholar
  55. 55.
    Saltelli, A., Tarantola, S.: On the relative importance of input factors in mathematical models: safety assessment for nuclear waste disposal. J. Am. Stat. Assoc. 97, 702–709 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  56. 56.
    Saltelli, A., Chan, K., Scott, E. (eds.): Sensitivity Analysis. Wiley Series in Probability and Statistics. Wiley, Chichester/New York (2000a)zbMATHGoogle Scholar
  57. 57.
    Saltelli, A., Tarantola, S., Campolongo, F.: Sensitivity analysis as an ingredient of modelling. Stat. Sci. 15, 377–395 (2000b)CrossRefGoogle Scholar
  58. 58.
    Saltelli, A., Tarantola, S., Campolongo, F., Ratto, M.: Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. Wiley, Chichester/Hoboken (2004)zbMATHGoogle Scholar
  59. 59.
    Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Salsana, M., Tarantola, S.: Global Sensitivity Analysis – The Primer. Wiley, Chichester (2008)zbMATHGoogle Scholar
  60. 60.
    Saltelli, A., Annoni, P., Azzini, I., Campolongo, F., Ratto, M., Tarantola, S.: Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Comput. Phys. Commun. 181, 259–270 (2010)zbMATHGoogle Scholar
  61. 61.
    Saltelli, A., Pereira, G., Van der Sluijs, J.P., Funtowicz, S.: What do I make of your latinorum? Sensitivity auditing of mathematical modelling. Int. J. Foresight Innov. Policy 9(2/3/4), 213–234 (2013)Google Scholar
  62. 62.
    Saltelli, A., Stark, P., Becker, W., Stano, P.: Climate models as economic guides. Scientific challenge or quixotic quest? Issues Sci. Technol. XXXI(3), 79–84 (2015)Google Scholar
  63. 63.
    Savage, S.L.: The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty. Wiley, Hoboken (2009)Google Scholar
  64. 64.
    Stiglitz, J.: Freefall, Free Markets and the Sinking of the Global Economy. Penguin, London (2010)Google Scholar
  65. 65.
    Szenberg, M.: Eminent Economists: Their Life Philosophies. Cambridge University Press, Cambridge (1992)Google Scholar
  66. 66.
    The Economist: How science goes wrong. The Economist 19 Oct 2013Google Scholar
  67. 67.
    Tissot, J.Y., Prieur, C.: A randomized orthogonal array-based procedure for the estimation of first- and second-order Sobol’ indices. J. Stat. Comput. Simul. 85, 1358–1381 (2015)MathSciNetCrossRefGoogle Scholar
  68. 68.
    Turanyi, T.: Sensitivity analysis for complex kinetic system, tools and applications. J. Math. Chem. 5, 203–248 (1990)MathSciNetCrossRefGoogle Scholar
  69. 69.
    Van der Sluijs, J.P., Craye, M., Funtowicz, S., Kloprogge, P., Ravetz, J., Risbey, J.: Combining quantitative and qualitative measures of uncertainty in model based environmental assessment: the NUSAP system. Risk Anal. 25(2), 481–492 (2005)CrossRefGoogle Scholar
  70. 70.
    Wang, J., Faivre, R., Richard, H., Monod, H.: mtk: a general-purpose and extensible R environment for uncertainty and sensitivity analyses of numerical experiments. R J. 7/2, 206–226 (2016)Google Scholar
  71. 71.
    Winner, L.: The Whale and the Reactor: A Search for Limits in an Age of High Technology. The University of Chicago Press, Chicago (1989)Google Scholar
  72. 72.
    Xu, C., Gertner, G.: Extending a global sensitivity analysis technique to models with correlated parameters. Comput. Stat. Data Anal. 51, 5579–5590 (2007)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  1. 1.Industrial Risk Management DepartmentEDF R&DChatouFrance
  2. 2.Institut de Mathématiques de ToulouseUniversité Paul SabatierToulouseFrance
  3. 3.Centre for the Study of the Sciences and the Humanities (SVT)University of Bergen (UIB)BergenNorway
  4. 4.Institut de Ciència i Tecnologia Ambientals (ICTA)Universitat Autonoma de Barcelona (UAB)BarcelonaSpain

Personalised recommendations