Advertisement

Sensitivity Analysis Methods

  • Yanjun GanEmail author
  • Qingyun Duan
Reference work entry

Abstract

Sensitivity analysis (SA) is an important tool for assessing and reducing uncertainties in computer-based models. This chapter presents a comprehensive review of some commonly used SA methods, including gradient-based, variance-based, and regression-based methods. Features and applicability of those methods are described and illustrated with some examples. Merits and limitations of different methods are explained, and the criteria of choosing appropriate SA methods for different applications are suggested.

Keywords

Uncertainty quantification Sensitivity analysis Uncertainty analysis Parameter estimation Design of experiment Sampling Parameter screening Variance decomposition 

Notes

Acknowledgment

This study was supported by the National Natural Science Foundation of China (41505092) and National Key Research and Development Program of China (2017YFC1404000).

References

  1. E.A. Anderson, National Weather Service River Forecast System – Snow Accumulation and Ablation Model (NOAA, Silver Spring, 1973)Google Scholar
  2. J.G. Arnold, R. Srinivasan, R.S. Muttiah, J.R. Williams, Large area hydrologic modeling and assessment. Part I: model development. J. Am. Water Resour. Assoc. 34, 73–89 (1998).  https://doi.org/10.1111/j.1752-1688.1998.tb05961.xCrossRefGoogle Scholar
  3. D.L. Beres, D.M. Hawkins, Plackett – Burman technique for sensitivity analysis of many-parametered models. Ecol. Model. 141, 171–183 (2001).  https://doi.org/10.1016/S0304-3800(01)00271-XCrossRefGoogle Scholar
  4. K. Beven, TOPMODEL: a critique. Hydrol. Process. 11, 1069–1085 (1997).  https://doi.org/10.1002/(SICI)1099-1085(199707)11:9<1069::AID-HYP545>3.0.CO;2-OCrossRefGoogle Scholar
  5. E. Borgonovo, W. Castaings, S. Tarantola, Model emulation and moment-independent sensitivity analysis: an application to environmental modelling. Environ. Model. Softw. 34, 105–115 (2012).  https://doi.org/10.1016/j.envsoft.2011.06.006CrossRefGoogle Scholar
  6. G.E.P. Box, J.S. Hunter, The 2k-p fractional factorial designs: part I. Technometrics 3, 311–351 (1961a).  https://doi.org/10.1080/00401706.1961.10489951CrossRefGoogle Scholar
  7. G.E.P. Box, J.S. Hunter, The 2k-p fractional factorial designs: part II. Technometrics 3, 449–458 (1961b).  https://doi.org/10.1080/00401706.1961.10489967CrossRefGoogle Scholar
  8. R.J.C. Burnash, R.L. Ferral, R.A. McGuire, A Generalized Streamflow Simulation System: Conceptual Modeling for Digital Computers (US Department of Commerce, National Weather Service, Sacramento, 1973)Google Scholar
  9. F. Campolongo, A. Saltelli, Sensitivity analysis of an environmental model: an application of different analysis methods. Reliab. Eng. Syst. Saf. 57, 49–69 (1997).  https://doi.org/10.1016/S0951-8320(97)00021-5CrossRefGoogle Scholar
  10. F. Campolongo, J. Cariboni, A. Saltelli, An effective screening design for sensitivity analysis of large models. Environ. Model. Softw. 22, 1509–1518 (2007).  https://doi.org/10.1016/j.envsoft.2006.10.004CrossRefGoogle Scholar
  11. F. Campolongo, A. Saltelli, J. Cariboni, From screening to quantitative sensitivity analysis. A unified approach. Comput. Phys. Commun. 182, 978–988 (2011).  https://doi.org/10.1016/j.cpc.2010.12.039CrossRefGoogle Scholar
  12. J. Cariboni, D. Gatelli, R. Liska, A. Saltelli, The role of sensitivity analysis in ecological modelling. Ecol. Model. 203, 167–182 (2007).  https://doi.org/10.1016/j.ecolmodel.2005.10.045CrossRefGoogle Scholar
  13. H.A. Chipman, E.I. George, R.E. McCulloch, BART: Bayesian additive regression trees. Ann. Appl. Stat. 4, 266–298 (2010).  https://doi.org/10.1214/09-AOAS285CrossRefGoogle Scholar
  14. D.C. Collins, R. Avissar, An evaluation with the Fourier amplitude sensitivity test (FAST) of which land-surface parameters are of greatest importance in atmospheric modeling. J. Clim. 7, 681–703 (1994).  https://doi.org/10.1175/1520-0442(1994)007<0681:AEWTFA>2.0.CO;2CrossRefGoogle Scholar
  15. R. Confalonieri, G. Bellocchi, S. Bregaglio, M. Donatelli, M. Acutis, Comparison of sensitivity analysis techniques: a case study with the rice model WARM. Ecol. Model. 221, 1897–1906 (2010).  https://doi.org/10.1016/j.ecolmodel.2010.04.021CrossRefGoogle Scholar
  16. S.A. Cryer, P.L. Havens, Regional sensitivity analysis using a fractional factorial method for the USDA model GLEAMS. Environ. Model. Softw. 14, 613–624 (1999).  https://doi.org/10.1016/S1364-8152(99)00003-1CrossRefGoogle Scholar
  17. R.I. Cukier, C.M. Fortuin, K.E. Shuler, A.G. Petschek, J.H. Schaibly, Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I theory. J. Chem. Phys. 59, 3873–3878 (1973).  https://doi.org/10.1063/1.1680571CrossRefGoogle Scholar
  18. C. Daniel, On varying one factor at a time. Biometrics 14, 430–431 (1958)CrossRefGoogle Scholar
  19. R.E. Dickinson, A. Henderson-Sellers, P.J. Kennedy, M.F. Wilson, Biosphere-Atmosphere Transfer Scheme (BATS) for the NCAR Community Climate Model (NCAR, Boulder, 1986).  https://doi.org/10.5065/D6668B58CrossRefGoogle Scholar
  20. E. Dion, L. VanSchalkwyk, E.F. Lambin, The landscape epidemiology of foot-and-mouth disease in South Africa: a spatially explicit multi-agent simulation. Ecol. Model. 222, 2059–2072 (2011).  https://doi.org/10.1016/j.ecolmodel.2011.03.026CrossRefGoogle Scholar
  21. Q. Duan, J. Schaake, V. Andreassian, S. Franks, G. Goteti, H.V. Gupta, Y.M. Gusev, F. Habets, A. Hall, L. Hay, T. Hogue, M. Huang, G. Leavesley, X. Liang, O.N. Nasonova, J. Noilhan, L. Oudin, S. Sorooshian, T. Wagener, E.F. Wood, Model parameter estimation experiment (MOPEX): an overview of science strategy and major results from the second and third workshops. J. Hydrol. 320, 3–17 (2006).  https://doi.org/10.1016/j.jhydrol.2005.07.031CrossRefGoogle Scholar
  22. E. Eirola, E. Liitiäinen, A. Lendasse, F. Corona, M. Verleysen, Using the Delta test for variable selection, in ESANN 2008 Proceedings, European Symposium on Artificial Neural Networks, Bruges, Belgium (2008)Google Scholar
  23. L. Foglia, M.C. Hill, S.W. Mehl, P. Burlando, Sensitivity analysis, calibration, and testing of a distributed hydrological model using error-based weighting and one objective function. Water Resour. Res. 45, W6427 (2009).  https://doi.org/10.1029/2008WR007255CrossRefGoogle Scholar
  24. A. Francos, F.J. Elorza, F. Bouraoui, G. Bidoglio, L. Galbiati, Sensitivity analysis of distributed environmental simulation models: understanding the model behaviour in hydrological studies at the catchment scale. Reliab. Eng. Syst. Saf. 79, 205–218 (2003).  https://doi.org/10.1016/S0951-8320(02)00231-4CrossRefGoogle Scholar
  25. H.C. Frey, S.R. Patil, Identification and review of sensitivity analysis methods. Risk Anal. 22, 553–578 (2002).  https://doi.org/10.1111/0272-4332.00039CrossRefGoogle Scholar
  26. H.C. Frey, A. Mokhtari, T. Danish, Evaluation of selected sensitivity analysis methods based upon applications to two food safety process risk models. Department of Civil, Construction, and Environmental Engineering, North Carolina State University, Raleigh, NC (2003)Google Scholar
  27. J.H. Friedman, Multivariate adaptive regression splines. Ann. Stat. 19, 1–67 (1991).  https://doi.org/10.1214/aos/1176347963CrossRefGoogle Scholar
  28. F. Galton, Regression towards mediocrity in hereditary stature. J. Anthropol. Inst. G. B. Irel. 15, 246–263 (1886)Google Scholar
  29. Y. Gan, Q. Duan, W. Gong, C. Tong, Y. Sun, W. Chu, A. Ye, C. Miao, Z. Di, A comprehensive evaluation of various sensitivity analysis methods: a case study with a hydrological model. Environ. Model. Softw. 51, 269–285 (2014).  https://doi.org/10.1016/j.envsoft.2013.09.031CrossRefGoogle Scholar
  30. Y. Gan, X.-Z. Liang, Q. Duan, H.I. Choi, Y. Dai, H. Wu, Stepwise sensitivity analysis from qualitative to quantitative: application to the terrestrial hydrological modeling of a conjunctive surface-subsurface process (CSSP) land surface model. J. Adv. Model. Earth Syst. 7, 648–669 (2015).  https://doi.org/10.1002/2014MS000406CrossRefGoogle Scholar
  31. M. Gibbs, D.J.C. MacKay, Efficient implementation of Gaussian processes. Unpublished manuscript (1997)Google Scholar
  32. J. Grant, K.J. Curran, T.L. Guyondet, G. Tita, C. Bacher, V. Koutitonsky, M. Dowd, A box model of carrying capacity for suspended mussel aquaculture in Lagune de la Grande-Entrée, Iles-de-la-Madeleine, Québec. Ecol. Model. 200, 193–206 (2007).  https://doi.org/10.1016/j.ecolmodel.2006.07.026CrossRefGoogle Scholar
  33. A. van Griensven, T. Meixner, S. Grunwald, T. Bishop, M. Diluzio, R. Srinivasan, A global sensitivity analysis tool for the parameters of multi-variable catchment models. J. Hydrol. 324, 10–23 (2006).  https://doi.org/10.1016/j.jhydrol.2005.09.008CrossRefGoogle Scholar
  34. D.M. Hamby, A review of techniques for parameter sensitivity analysis of environmental models. Environ. Monit. Assess. 32, 135–154 (1994).  https://doi.org/10.1007/BF00547132CrossRefGoogle Scholar
  35. J.C. Helton, Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal. Reliab. Eng. Syst. Saf. 42, 327–367 (1993).  https://doi.org/10.1016/0951-8320(93)90097-ICrossRefGoogle Scholar
  36. A. Henderson-Sellers, A factorial assessment of the sensitivity of the BATS land-surface parameterization scheme. J. Clim. 6, 227–247 (1993).  https://doi.org/10.1175/1520-0442(1993)006<0227:AFAOTS>2.0.CO;2CrossRefGoogle Scholar
  37. B. Henderson-Sellers, A. Henderson-Sellers, Sensitivity evaluation of environmental models using fractional factorial experimentation. Ecol. Model. 86, 291–295 (1996).  https://doi.org/10.1016/0304-3800(95)00066-6CrossRefGoogle Scholar
  38. T. Homma, A. Saltelli, Importance measures in global sensitivity analysis of nonlinear models. Reliab. Eng. Syst. Saf. 52, 1–17 (1996).  https://doi.org/10.1016/0951-8320(96)00002-6CrossRefGoogle Scholar
  39. R.L. Iman, W.J. Conover, The use of the rank transform in regression. Technometrics 21, 499–509 (1979).  https://doi.org/10.1080/00401706.1979.10489820CrossRefGoogle Scholar
  40. A.J. Jakeman, R.A. Letcher, J.P. Norton, Ten iterative steps in development and evaluation of environmental models. Environ. Model. Softw. 21, 602–614 (2006).  https://doi.org/10.1016/j.envsoft.2006.01.004CrossRefGoogle Scholar
  41. J.P.C. Kleijnen, Sensitivity analysis and related analyses: a review of some statistical techniques. J. Stat. Comput. Simul. 57, 111–142 (1997).  https://doi.org/10.1080/00949659708811805CrossRefGoogle Scholar
  42. T. Lenhart, K. Eckhardt, N. Fohrer, H.G. Frede, Comparison of two different approaches of sensitivity analysis. Phys. Chem. Earth 27, 645–654 (2002).  https://doi.org/10.1016/S1474-7065(02)00049-9CrossRefGoogle Scholar
  43. D.J.C. MacKay, Introduction to Gaussian processes, in Neural Networks and Machine Learning, ed. by C.M. Bishop, (Springer, Berlin, 1998). pp 133–165Google Scholar
  44. S. Marino, I.B. Hogue, C.J. Ray, D.E. Kirschner, A methodology for performing global uncertainty and sensitivity analysis in systems biology. J. Theor. Biol. 254, 178–196 (2008).  https://doi.org/10.1016/j.jtbi.2008.04.011CrossRefGoogle Scholar
  45. L.S. Matott, J.E. Babendreier, S.T. Purucker, Evaluating uncertainty in integrated environmental models: a review of concepts and tools. Water Resour. Res. 45, W6421 (2009).  https://doi.org/10.1029/2008WR007301CrossRefGoogle Scholar
  46. M.D. McKay, Evaluating Prediction Uncertainty (Los Alamos National Laboratory (LANL), Los Alamos, 1995)CrossRefGoogle Scholar
  47. M.D. McKay, R.J. Beckman, W.J. Conover, A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21, 239–245 (1979).  https://doi.org/10.1080/00401706.2000.10485979CrossRefGoogle Scholar
  48. A. Mokhtari, H.C. Frey, Sensitivity analysis of a two-dimensional probabilistic risk assessment model using analysis of variance. Risk Anal. 25, 1511–1529 (2005).  https://doi.org/10.1111/j.1539-6924.2005.00679.xCrossRefGoogle Scholar
  49. M.D. Morris, Factorial sampling plans for preliminary computational experiments. Technometrics 33, 161–174 (1991).  https://doi.org/10.1080/00401706.1991.10484804CrossRefGoogle Scholar
  50. J. Nossent, P. Elsen, W. Bauwens, Sobol’ sensitivity analysis of a complex environmental model. Environ. Model. Softw. 26, 1515–1525 (2011).  https://doi.org/10.1016/j.envsoft.2011.08.010CrossRefGoogle Scholar
  51. A.B.. Owen, Orthogonal arrays for computer experiments, integration and visualization. Stat. Sin. 2, 439–452 (1992)Google Scholar
  52. H. Pi, C. Peterson, Finding the embedding dimension and variable dependencies in time series. Neural Comput. 6, 509–520 (1994).  https://doi.org/10.1162/neco.1994.6.3.509CrossRefGoogle Scholar
  53. R.L. Plackett, J.P. Burman, The design of optimum multifactorial experiments. Biometrika 33, 305–325 (1946).  https://doi.org/10.1093/biomet/33.4.305CrossRefGoogle Scholar
  54. O. Rakovec, M.C. Hill, M.P. Clark, A.H. Weerts, A.J. Teuling, R. Uijlenhoet, Distributed evaluation of local sensitivity analysis (DELSA), with application to hydrologic models. Water Resour. Res. 50, 409–426 (2014).  https://doi.org/10.1002/2013WR014063CrossRefGoogle Scholar
  55. S. Razavi, B.A. Tolson, D.H. Burn, Review of surrogate modeling in water resources. Water Resour. Res. 48, W7401 (2012).  https://doi.org/10.1029/2011WR011527CrossRefGoogle Scholar
  56. B. Renard, D. Kavetski, G. Kuczera, M. Thyer, S.W. Franks, Understanding predictive uncertainty in hydrologic modeling: the challenge of identifying input and structural errors. Water Resour. Res. 46, W5521 (2010).  https://doi.org/10.1029/2009WR008328CrossRefGoogle Scholar
  57. D.E. Reusser, W. Buytaert, E. Zehe, Temporal dynamics of model parameter sensitivity for computationally expensive models with the Fourier amplitude sensitivity test. Water Resour. Res. 47, W7551 (2011).  https://doi.org/10.1029/2010WR009947
  58. E. Rodríguez-Camino, R. Avissar, Comparison of three land-surface schemes with the Fourier amplitude sensitivity test (FAST). Tellus A 50, 313–332 (1998).  https://doi.org/10.3402/tellusa.v50i3.14529CrossRefGoogle Scholar
  59. A. Saltelli, Sensitivity analysis: could better methods be used? J. Geophys. Res. 104, 3789–3793 (1999).  https://doi.org/10.1029/1998JD100042CrossRefGoogle Scholar
  60. A. Saltelli, R. Bolado, An alternative way to compute Fourier amplitude sensitivity test (FAST). Comput. Stat. Data Anal. 26, 445–460 (1998).  https://doi.org/10.1016/S0167-9473(97)00043-1CrossRefGoogle Scholar
  61. A. Saltelli, S. Tarantola, K.P.S. Chan, A quantitative model-independent method for global sensitivity analysis of model output. Technometrics 41, 39–56 (1999).  https://doi.org/10.2307/1270993CrossRefGoogle Scholar
  62. A. Saltelli, S. Tarantola, F. Campolongo, Sensitivity analysis as an ingredient of modeling. Stat. Sci. 15, 377–395 (2000).  https://doi.org/10.1214/ss/1009213004CrossRefGoogle Scholar
  63. A. Saltelli, M. Ratto, S. Tarantola, F. Campolongo, Sensitivity analysis for chemical models. Chem. Rev. 105, 2811–2827 (2005).  https://doi.org/10.1021/cr040659dCrossRefGoogle Scholar
  64. A. Saltelli, M. Ratto, S. Tarantola, F. Campolongo, Sensitivity analysis practices: strategies for model-based inference. Reliab. Eng. Syst. Saf. 91, 1109–1125 (2006).  https://doi.org/10.1016/j.ress.2005.11.014CrossRefGoogle Scholar
  65. A. Saltelli, M. Ratto, T. Andres, F. Campolongo, Global Sensitivity Analysis: The Primer (Wiley, Chichester, 2008)Google Scholar
  66. J.H. Schaibly, K.E. Shuler, Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. II applications. J. Chem. Phys. 59, 3879–3888 (1973).  https://doi.org/10.1063/1.1680572CrossRefGoogle Scholar
  67. D. Shahsavani, A. Grimvall, Variance-based sensitivity analysis of model outputs using surrogate models. Environ. Model. Softw. 26, 723–730 (2011).  https://doi.org/10.1016/j.envsoft.2011.01.002CrossRefGoogle Scholar
  68. A. Sieber, S. Uhlenbrook, Sensitivity analyses of a distributed catchment model to verify the model structure. J. Hydrol. 310, 216–235 (2005).  https://doi.org/10.1016/j.jhydrol.2005.01.004CrossRefGoogle Scholar
  69. I.M. Sobol’, Quasi-Monte Carlo methods. Prog. Nucl. Energy 24, 55–61 (1990).  https://doi.org/10.1016/0149-1970(90)90022-WCrossRefGoogle Scholar
  70. I.M. Sobol’, Sensitivity analysis for nonlinear mathematical models. Math. Model. Comput. Exp. 1, 407–414 (1993)Google Scholar
  71. I.M. Sobol’, Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math. Comput. Simul. 55, 271–280 (2001).  https://doi.org/10.1016/S0378-4754(00)00270-6CrossRefGoogle Scholar
  72. D. Steinberg, P.L. Colla, K. Martin, MARS User Guide (Salford Systems, San Diego, 1999)Google Scholar
  73. C.B. Storlie, L.P. Swiler, J.C. Helton, C.J. Sallaberry, Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models. Reliab. Eng. Syst. Saf. 94, 1735–1763 (2009).  https://doi.org/10.1016/j.ress.2009.05.007CrossRefGoogle Scholar
  74. X.Y. Sun, L. Newham, B. Croke, J.P. Norton, Three complementary methods for sensitivity analysis of a water quality model. Environ. Model. Softw. 37, 19–29 (2012).  https://doi.org/10.1016/j.envsoft.2012.04.010CrossRefGoogle Scholar
  75. B. Tang, Orthogonal array-based Latin hypercubes. J. Am. Stat. Assoc. 88, 1392–1397 (1993).  https://doi.org/10.2307/2291282CrossRefGoogle Scholar
  76. Y. Tang, P. Reed, T. Wagener, K. van Werkhoven, Comparing sensitivity analysis methods to advance lumped watershed model identification and evaluation. Hydrol. Earth Syst. Sci. 11, 793–817 (2007).  https://doi.org/10.5194/hess-11-793-2007CrossRefGoogle Scholar
  77. C. Tong, PSUADE User’s Manual (Lawrence Livermore National Laboratory (LLNL), Livermore, 2005)Google Scholar
  78. T. Turányi, Sensitivity analysis of complex kinetic systems. Tools and applications. J. Math. Chem. 5, 203–248 (1990).  https://doi.org/10.1007/BF01166355CrossRefGoogle Scholar
  79. L. Uusitalo, A. Lehikoinen, I. Helle, K. Myrberg, An overview of methods to evaluate uncertainty of deterministic models in decision support. Environ. Model. Softw. 63, 24–31 (2015).  https://doi.org/10.1016/j.envsoft.2014.09.017CrossRefGoogle Scholar
  80. T. Wagener, K. van Werkhoven, P. Reed, Y. Tang, Multiobjective sensitivity analysis to understand the information content in streamflow observations for distributed watershed modeling. Water Resour. Res. 45, W2501 (2009).  https://doi.org/10.1029/2008WR007347CrossRefGoogle Scholar
  81. H.M. Wainwright, S. Finsterle, Y. Jung, Q. Zhou, J.T. Birkholzer, Making sense of global sensitivity analyses. Comput. Geosci. UK 65, 84–94 (2014).  https://doi.org/10.1016/j.cageo.2013.06.006CrossRefGoogle Scholar
  82. W.E. Walker, P. Harremoës, J. Rotmans, J.P. van der Sluijs, M.B. van Asselt, P. Janssen, M.P. Krayer Von Krauss, Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support. Integr. Assess. 4, 5–17 (2003).  https://doi.org/10.1076/iaij.4.1.5.16466CrossRefGoogle Scholar
  83. G.G. Wang, S. Shan, Review of metamodeling techniques in support of engineering design optimization. J. Mech. Des. 129, 370–380 (2007).  https://doi.org/10.1115/1.2429697CrossRefGoogle Scholar
  84. J. Wang, X. Li, L. Lu, F. Fang, Parameter sensitivity analysis of crop growth models based on the extended Fourier amplitude sensitivity test method. Environ. Model. Softw. 48, 171–182 (2013).  https://doi.org/10.1016/j.envsoft.2013.06.007CrossRefGoogle Scholar
  85. C. Wang, Q. Duan, C. Tong, W. Gong, A GUI platform for uncertainty quantification of complex dynamical models. Environ. Model. Softw. 76, 1–12 (2016).  https://doi.org/10.1016/j.envsoft.2015.11.004CrossRefGoogle Scholar
  86. J.E. Ward, R.E. Wendell, Approaches to sensitivity analysis in linear programming. Ann. Oper. Res. 27, 3–38 (1990).  https://doi.org/10.1007/BF02055188CrossRefGoogle Scholar
  87. H. Weyl, Mean motion. Am. J. Math. 60, 889–896 (1938)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.State Key Laboratory of Severe WeatherChinese Academy of Meteorological SciencesBeijingChina
  2. 2.Faculty of Geographical ScienceBeijing Normal UniversityBeijingChina

Section editors and affiliations

  • Dmitri Kavetski
    • 1
  • Kuolin Hsu
    • 2
  • Yuqiong Liu
    • 3
  1. 1.School of Civil, Environmental and Mining Engineering, University of AdelaideAdelaideAustralia
  2. 2.Civil & Environmental Engineering, The Henry Samueli School of Engineering, University of CaliforniaIrvineUSA
  3. 3.NASA Goddard Space Flight CenterWashington D.C.USA

Personalised recommendations