Advertisement

Validation and Equifinality

  • Keith BevenEmail author
Chapter
Part of the Simulation Foundations, Methods and Applications book series (SFMA)

Abstract

In this chapter, the concept of equifinality of model representations is discussed, from a background of model applications in the environmental sciences. Equifinality in this context is used to indicate that there may be many different model structures, parameter sets and auxiliary conditions that might appear to give equivalent output predictions or acceptable fits to any observation data available for use in model calibration. This does not imply that the resulting ensemble of models will give similar predictions when used to predict the future under some changed conditions. As new information becomes available to allow model validation, this can be used to constrain the ensemble of models within a Bayesian updating framework, although epistemic sources of uncertainty can make it difficult to define appropriate likelihood measures. It seems likely that the equifinality concept will persist into the future in the form of ensembles of (stochastic) model runs being used to estimate prediction uncertainties. However, more research is needed into the limitations of model structures, information content of data sets subject to epistemic uncertainties and means of evaluating and validating models in the inexact sciences.

Keywords

Inexact sciences Model ensembles Epistemic uncertainties Environmental models Equifinality thesis Audit trail 

References

  1. Bertalanffy, L. von. (1951). An outline of general systems theory. British Journal for the Philosophy of Science, 1, 134–165.Google Scholar
  2. Bertalanffy, L. von. (1968). General systems theory. New York: Braziller.Google Scholar
  3. Beven, K. J. (1975). A deterministic spatially distributed model of catchment hydrology. Unpublished Ph.D. thesis, University of East Anglia: Norwich, UK.Google Scholar
  4. Beven, K. J. (1993). Prophecy, reality and uncertainty in distributed hydrological modelling. Advances in Water Resources, 16, 41–51.CrossRefGoogle Scholar
  5. Beven, K. J. (2002). Towards a coherent philosophy for environmental modelling. Proceedings of the Royal Society of London A, 458, 2465–2484.MathSciNetCrossRefGoogle Scholar
  6. Beven, K. J. (2006). A manifesto for the equifinality thesis. J. Hydrology, 320, 18–36.CrossRefGoogle Scholar
  7. Beven, K. J. (2009). Environmental modelling: An uncertain future?. London: Routledge.Google Scholar
  8. Beven, K. J. (2012a). Rainfall-runoff modelling: The primer (2nd ed.). Chichester: Wiley-Blackwell.CrossRefGoogle Scholar
  9. Beven, K. J. (2012b). Causal models as multiple working hypotheses about environmental processes. Comptes Rendus Geoscience, Académie de Sciences, Paris, 344, 77–88.  https://doi.org/10.1016/j.crte.2012.01.005.CrossRefGoogle Scholar
  10. Beven, K. J. (2016). EGU Leonardo lecture: Facets of Hydrology-epistemic error, non-stationarity, likelihood, hypothesis testing, and communication. Hydrological Sciences Journal, 61(9), 1652–1665.  https://doi.org/10.1080/02626667.2015.1031761.CrossRefGoogle Scholar
  11. Beven, K. J., & Kirkby, M. J. (1979). A physically-based variable contributing area model of basin hydrology. Hydrological Sciences Bulletin, 24(1), 43–69.CrossRefGoogle Scholar
  12. Beven, K. J., & Binley, A. M. (1992). The future of distributed models: model calibration and uncertainty prediction. Hydrological Processes, 6, 279–298.CrossRefGoogle Scholar
  13. Beven, K. J., & Westerberg, I. (2011). On red herrings and real herrings: disinformation and information in hydrological inference. Hydrological Processes, 25, 1676–1680.  https://doi.org/10.1002/hyp.7963.CrossRefGoogle Scholar
  14. Beven, K. J., & Alcock, R. (2012). Modelling everything everywhere: A new approach to decision making for water management under uncertainty. Freshwater Biology, 56, 124–132.  https://doi.org/10.1111/j.1365-2427.2011.02592.x.CrossRefGoogle Scholar
  15. Beven, K., & Binley, A. (2014). GLUE: 20 years on. Hydrological Processes, 28(24), 5897–5918.CrossRefGoogle Scholar
  16. Beven, K. J., & Smith, P. J. (2015). Concepts of information content and likelihood in parameter calibration for hydrological simulation models. ASCE Jornal of Hydrologic. Engineering.  https://doi.org/10.1061/(asce)he.1943-5584.0000991.
  17. Beven, K. J., Smith, P. J., & Freer, J. (2008). So just why would a modeller choose to be incoherent? Journal of Hydrology, 354, 15–32.CrossRefGoogle Scholar
  18. Beven, K. J., Leedal, D. T., McCarthy, S. (2011a). Framework for assessing uncertainty in fluvial flood risk mapping, CIRIA report C721, 2014, at http://www.ciria.org/Resources/Free_publications/fluvial_flood_risk_mapping.aspx.
  19. Beven, K., Smith, P. J., & Wood, A. (2011b). On the colour and spin of epistemic error (and what we might do about it). Hydrology and Earth System Sciences, 15, 3123–3133.  https://doi.org/10.5194/hess-15-3123-2011.
  20. Beven, K. J., Leedal, D. T., & McCarthy, S. (2014). Framework for assessing uncertainty in fluvial flood risk mapping, CIRIA report C721. Available at http://www.ciria.org/Resources/Free_publications/fluvial_flood_risk_mapping.aspx.
  21. Chorley, R. J. (1962). Geomorphology and general systems theory, U.S. Geological Survey, Prof. Paper 500-1B, Washington, DC.Google Scholar
  22. Coxon, G., Freer, J., Westerberg, I. K., Wagener, T., Woods, R., & Smith, P. J. (2015). A novel framework for discharge uncertainty quantification applied to 500 UK gauging stations. Water Resources Research, 51(7), 5531–5546.CrossRefGoogle Scholar
  23. Culling, W. E. H. (1957). Mulitcycle streams and the equilibrium theory of grade. The Journal of Geology, 65, 259–274.Google Scholar
  24. Culling, W. E. H. (1987). Equifinality: Modern approaches to dynamical systems and their potential for geomorphological thought. Transactions of the Institute of British Geographers, 13, 345–360.Google Scholar
  25. Dean, S., Freer, J. E., Beven, K. J., Wade, A. J., & Butterfield, D. (2009). Uncertainty assessment of a process-based integrated catchment model of phosphorus (INCA-P). Stochastic Environmental Research and Risk Assessment, 2009(23), 991–1010.  https://doi.org/10.1007/s00477-008-0273-z.CrossRefGoogle Scholar
  26. Evangelinos, C., & Hill, C. (2008). Cloud computing for parallel scientific HPC applications: Feasibility of running coupled atmosphere-ocean climate models on Amazon’s EC2. Ratio2(2.40), 2–34.Google Scholar
  27. Fowler, H. J., Cooley, D., Sain, S. R., & Thurston, M. (2010). Detecting change in UK extreme precipitation using results from the climateprediction. net BBC climate change experiment. Extremes13(2), 241–267.Google Scholar
  28. Frame, D. J., Aina, T., Christensen, C. M., Faull, N. E., Knight, S. H. E., Piani, C., et al. (2009). The climateprediction. net BBC climate change experiment: Design of the coupled model ensemble. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences367(1890), 855–870.Google Scholar
  29. Franks, S. W., & Beven, K. J. (1999). Conditioning a multiple patch SVAT model using uncertain time-space estimates of latent heat fluxes as inferred from remotely-sensed data. Water Resources Research, 35(9), 2751–2761.CrossRefGoogle Scholar
  30. Gahegan, M., & Ehlers, M. (2000). A framework for the modelling of uncertainty between remote sensing and geographic information systems. ISPRS Journal of Photogrammetry and Remote Sensing, 55(3), 176–188.CrossRefGoogle Scholar
  31. Gupta, H. V. & Kling, H. (2011). On typical range, sensitivity, and normalization of Mean Squared Error and Nash‐Sutcliffe Efficiency type metrics. Water Resources Research47(10).Google Scholar
  32. Gupta, H. V., Kling, H., Yilmaz, K. K., & Martinez, G. F. (2009). Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling. Journal of Hydrology, 377(1), 80–91.CrossRefGoogle Scholar
  33. Halpern, J. Y. (2005). Reasoning about uncertainty. Cambridge, MA: MIT Press.zbMATHGoogle Scholar
  34. Helmer, O., & Rescher, N. (1959). On an epistemology of the inexact sciences. Management Science, 6(1), 25–52.CrossRefGoogle Scholar
  35. Hollaway, M. J., Beven, K. J., Benskin, C. M. W. H., Collins, A. L., Evans, R., Falloon, P. D. et al. (2018). The challenges of modelling phosphorus in a headwater catchment: Applying a ‘limits of acceptability’ uncertainty framework to a water quality model, Journal of Hydrology (in press).Google Scholar
  36. Hornberger, G. M., & Spear, R. C. (1981). An approach to the preliminary analysis of environmental systems. Journal of Environmental Management, 12, 7–18.Google Scholar
  37. Klemes, V. (1986). Delettantism in hydrology: Transition or destiny? Water Resources Research, 22, S177–S188.CrossRefGoogle Scholar
  38. Lobell, D. B., Asner, G. P., Ortiz-Monasterio, J. I., & Benning, T. L. (2003). Remote sensing of regional crop production in the Yaqui Valley, Mexico: Estimates and uncertainties. Agriculture, Ecosystems & Environment, 94(2), 205–220.CrossRefGoogle Scholar
  39. Madsen, H. (2003). Parameter estimation in distributed hydrological catchment modelling using automatic calibration with multiple objectives. Advances in Water Resources, 26(2), 205–216.CrossRefGoogle Scholar
  40. Montanari, A. (2005). Large sample behaviors of the generalized likelihood uncertainty estimation (GLUE) in assessing the uncertainty of rainfall-runoff simulations. Water Resources Research, 41(8), W08406.MathSciNetCrossRefGoogle Scholar
  41. Mantovan, P., & Todini, E. (2006). Hydrological forecasting uncertainty assessment: Incoherence of the GLUE methodology. Journal of Hydrology, 330(1), 368–381.CrossRefGoogle Scholar
  42. Mitchell, S, Beven, K. J., Freer, J., & Law, B. (2011). Processes influencing model-data mismatch in drought-stressed, fire-disturbed, eddy flux sites. JGR-Biosciences, 116.  https://doi.org/10.1029/2009jg001146.
  43. McMillan, H. K., & Westerberg, I. K. (2015). Rating curve estimation under epistemic uncertainty. Hydrological Processes, 29(7), 1873–1882.CrossRefGoogle Scholar
  44. Mizukami, N., Rakovec, O., Newman, A., Clark, M., Wood, A., Gupta, H., et al. (2018). On the choice of calibration metrics for “high flow” estimation using hydrologic models. Hydrology and Earth system Science Discussions. https://www.hydrol-earth-syst-sci-discuss.net/hess-2018-391/.
  45. Nash, J. E., & Sutcliffe, J. S. (1970). River-flow forecasting through conceptual models. 1. A discussion of principles. Journal of Hydrology, 10, 282–290.CrossRefGoogle Scholar
  46. Nearing, G. S., Tian, Y., Gupta, H. V., Clark, M. P., Harrison, K. W., & Weijs, S. V. (2016). A philosophical basis for hydrological uncertainty. Hydrological Sciences Journal, 61(9), 1666–1678.CrossRefGoogle Scholar
  47. O’Hagan, A., & Oakley, A. E. (2004). Probability is perfect but we can’t elicit it perfectly. Reliability Engineering and System Safety, 85, 239–248.CrossRefGoogle Scholar
  48. Page, T., Beven, K. J., & Freer, J. (2007). Modelling the chloride signal at the Plynlimon catchments, wales using a modified dynamic TOPMODEL. Hydrological Processes, 21, 292–307.CrossRefGoogle Scholar
  49. Pappenberger, F., Frodsham, K., Beven, K. J., Romanovicz, R., & Matgen, P. (2007). Fuzzy set approach to calibrating distributed flood inundation models using remote sensing observations. Hydrology and Earth System Sciences, 11(2), 739–752.CrossRefGoogle Scholar
  50. Pokhrel, P., Yilmaz, K. K., & Gupta, H. V. (2012). Multiple-criteria calibration of a distributed watershed model using spatial regularization and response signatures. Journal of Hydrology, 418, 49–60.CrossRefGoogle Scholar
  51. Refsgaard, J. C., & Knudsen, J. (1996). Operational validation and intercomparison of different types of hydrological models. Water Resources Research, 32(7), 2189–2202.Google Scholar
  52. Rose, K. A., Smith, E. P., Gardner, R. H., Brenkert, A. L., & Bartell, S. M. (1991). Parameter sensitivities, Monte Carlo filtering, and model forecasting under uncertainty. Journal of Forecasting, 10(1–2), 117–133.CrossRefGoogle Scholar
  53. Romanowicz, R., Beven, K. J., & Tawn, J. (1994). Evaluation of predictive uncertainty in non-linear hydrological models using a Bayesian approach. In V. Barnett & K. F. Turkman (Eds.), Statistics for the environment II. Water related issues (pp. 297–317). Wiley.Google Scholar
  54. Romanowicz, R., Beven, K. J., & Tawn, J. (1996). Bayesian calibration of flood inundation models. In M. G. Anderson, D. E. Walling, & P. D. Bates, (Eds.) Floodplain Processes (pp. 333–360).Google Scholar
  55. Reusser, D. E., Blume, T., Schaefli, B., & Zehe, E. (2009). Analysing the temporal dynamics of model performance for hydrological models. Hydrology and earth system sciences13(EPFL-ARTICLE-162488), 999–1018.Google Scholar
  56. Renard, B., Kavetski, D., Kuczera, G., Thyer, M., & Franks, S. W. (2010). Understanding predictive uncertainty in hydrologic modeling: The challenge of identifying input and structural errors. Water Resources Research46(5).Google Scholar
  57. Rowlands, D. J., Frame, D. J., Ackerley, D., Aina, T., Booth, B. B., Christensen, C., et al. (2012). Broad range of 2050 warming from an observationally constrained large climate model ensemble. Nature Geoscience, 5(4), 256–260.CrossRefGoogle Scholar
  58. Schaefli, B., & Gupta, H. V. (2007). Do Nash values have value? Hydrological Processes, 21(15), 2075–2080.CrossRefGoogle Scholar
  59. Schoups, G., & Vrugt, J. A. (2010). A formal likelihood function for parameter and predictive inference of hydrologic models with correlated, heteroscedastic, and non‐Gaussian errors. Water Resources Research46(10).Google Scholar
  60. Smith, P., Beven, K. J., & Tawn, J. A. (2008). Informal likelihood measures in model assessment: Theoretic development and investigation. Advances in Water Resources, 31(8), 1087–1100.CrossRefGoogle Scholar
  61. Sorooshian, S., & Gupta, H. V. (1995). Model calibration. In V. P. Singh (Ed.), Computer models of watershed hydrology. Highlands Ranch CO: Water Resource Publications.Google Scholar
  62. Spear, R. C., & Hornberger, G. M. (1980). Eutrophication in peel inlet—II. Identification of critical uncertainties via generalized sensitivity analysis. Water Research, 14(1), 43–49.Google Scholar
  63. Stedinger, J. R., Vogel, R. M., Lee, S. U., & Batchelder, R. (2008). Appraisal of the generalized likelihood uncertainty estimation (GLUE) method. Water Resources Research, 44(12), W00806.CrossRefGoogle Scholar
  64. Thompson, T. D. (1961). Numerical weather analysis and prediction. New York: Macmillan.Google Scholar
  65. Van Straten, G. T., & Keesman, K. J. (1991). Uncertainty propagation and speculation in projective forecasts of environmental change: A lake-eutrophication example. Journal of Forecasting, 10(1–2), 163–190.CrossRefGoogle Scholar
  66. Vrugt, J. A., Gupta, H. V., Bastidas, L. A., Bouten, W., & Sorooshian, S. (2003). Effective and efficient algorithm for multiobjective optimization of hydrologic models. Water Resources Research, 39(8), W01214.Google Scholar
  67. Westerberg, I., Guerrero, J. L., Seibert, J., Beven, K. J., & Halldin, S. (2011). Stage-discharge uncertainty derived with a non-stationary rating curve in the Choluteca River, Honduras. Hydrological Processes, 25(4), 603–613.CrossRefGoogle Scholar
  68. Yapo, P. O., Gupta, H. V., & Sorooshian, S. (1998). Multi-objective global optimization for hydrologic models. Journal of Hydrology, 204(1–4), 83–97.CrossRefGoogle Scholar
  69. Zhang, D., Beven, K. J., & Mermoud, A. (2006). A comparison of nonlinear least square and GLUE for model calibration and uncertainty estimation for pesticide transport in soils. Advances in Water Resources, 29, 1924–1933.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Lancaster Environment Centre, Lancaster UniversityLancasterUK

Personalised recommendations