Validation and Equifinality
In this chapter, the concept of equifinality of model representations is discussed, from a background of model applications in the environmental sciences. Equifinality in this context is used to indicate that there may be many different model structures, parameter sets and auxiliary conditions that might appear to give equivalent output predictions or acceptable fits to any observation data available for use in model calibration. This does not imply that the resulting ensemble of models will give similar predictions when used to predict the future under some changed conditions. As new information becomes available to allow model validation, this can be used to constrain the ensemble of models within a Bayesian updating framework, although epistemic sources of uncertainty can make it difficult to define appropriate likelihood measures. It seems likely that the equifinality concept will persist into the future in the form of ensembles of (stochastic) model runs being used to estimate prediction uncertainties. However, more research is needed into the limitations of model structures, information content of data sets subject to epistemic uncertainties and means of evaluating and validating models in the inexact sciences.
KeywordsInexact sciences Model ensembles Epistemic uncertainties Environmental models Equifinality thesis Audit trail
- Bertalanffy, L. von. (1951). An outline of general systems theory. British Journal for the Philosophy of Science, 1, 134–165.Google Scholar
- Bertalanffy, L. von. (1968). General systems theory. New York: Braziller.Google Scholar
- Beven, K. J. (1975). A deterministic spatially distributed model of catchment hydrology. Unpublished Ph.D. thesis, University of East Anglia: Norwich, UK.Google Scholar
- Beven, K. J. (2009). Environmental modelling: An uncertain future?. London: Routledge.Google Scholar
- Beven, K. J., & Smith, P. J. (2015). Concepts of information content and likelihood in parameter calibration for hydrological simulation models. ASCE Jornal of Hydrologic. Engineering. https://doi.org/10.1061/(asce)he.1943-5584.0000991.
- Beven, K. J., Leedal, D. T., McCarthy, S. (2011a). Framework for assessing uncertainty in fluvial flood risk mapping, CIRIA report C721, 2014, at http://www.ciria.org/Resources/Free_publications/fluvial_flood_risk_mapping.aspx.
- Beven, K., Smith, P. J., & Wood, A. (2011b). On the colour and spin of epistemic error (and what we might do about it). Hydrology and Earth System Sciences, 15, 3123–3133. https://doi.org/10.5194/hess-15-3123-2011.
- Beven, K. J., Leedal, D. T., & McCarthy, S. (2014). Framework for assessing uncertainty in fluvial flood risk mapping, CIRIA report C721. Available at http://www.ciria.org/Resources/Free_publications/fluvial_flood_risk_mapping.aspx.
- Chorley, R. J. (1962). Geomorphology and general systems theory, U.S. Geological Survey, Prof. Paper 500-1B, Washington, DC.Google Scholar
- Culling, W. E. H. (1957). Mulitcycle streams and the equilibrium theory of grade. The Journal of Geology, 65, 259–274.Google Scholar
- Culling, W. E. H. (1987). Equifinality: Modern approaches to dynamical systems and their potential for geomorphological thought. Transactions of the Institute of British Geographers, 13, 345–360.Google Scholar
- Dean, S., Freer, J. E., Beven, K. J., Wade, A. J., & Butterfield, D. (2009). Uncertainty assessment of a process-based integrated catchment model of phosphorus (INCA-P). Stochastic Environmental Research and Risk Assessment, 2009(23), 991–1010. https://doi.org/10.1007/s00477-008-0273-z.CrossRefGoogle Scholar
- Evangelinos, C., & Hill, C. (2008). Cloud computing for parallel scientific HPC applications: Feasibility of running coupled atmosphere-ocean climate models on Amazon’s EC2. Ratio, 2(2.40), 2–34.Google Scholar
- Fowler, H. J., Cooley, D., Sain, S. R., & Thurston, M. (2010). Detecting change in UK extreme precipitation using results from the climateprediction. net BBC climate change experiment. Extremes, 13(2), 241–267.Google Scholar
- Frame, D. J., Aina, T., Christensen, C. M., Faull, N. E., Knight, S. H. E., Piani, C., et al. (2009). The climateprediction. net BBC climate change experiment: Design of the coupled model ensemble. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 367(1890), 855–870.Google Scholar
- Gupta, H. V. & Kling, H. (2011). On typical range, sensitivity, and normalization of Mean Squared Error and Nash‐Sutcliffe Efficiency type metrics. Water Resources Research, 47(10).Google Scholar
- Hollaway, M. J., Beven, K. J., Benskin, C. M. W. H., Collins, A. L., Evans, R., Falloon, P. D. et al. (2018). The challenges of modelling phosphorus in a headwater catchment: Applying a ‘limits of acceptability’ uncertainty framework to a water quality model, Journal of Hydrology (in press).Google Scholar
- Hornberger, G. M., & Spear, R. C. (1981). An approach to the preliminary analysis of environmental systems. Journal of Environmental Management, 12, 7–18.Google Scholar
- Mitchell, S, Beven, K. J., Freer, J., & Law, B. (2011). Processes influencing model-data mismatch in drought-stressed, fire-disturbed, eddy flux sites. JGR-Biosciences, 116. https://doi.org/10.1029/2009jg001146.
- Mizukami, N., Rakovec, O., Newman, A., Clark, M., Wood, A., Gupta, H., et al. (2018). On the choice of calibration metrics for “high flow” estimation using hydrologic models. Hydrology and Earth system Science Discussions. https://www.hydrol-earth-syst-sci-discuss.net/hess-2018-391/.
- Refsgaard, J. C., & Knudsen, J. (1996). Operational validation and intercomparison of different types of hydrological models. Water Resources Research, 32(7), 2189–2202.Google Scholar
- Romanowicz, R., Beven, K. J., & Tawn, J. (1994). Evaluation of predictive uncertainty in non-linear hydrological models using a Bayesian approach. In V. Barnett & K. F. Turkman (Eds.), Statistics for the environment II. Water related issues (pp. 297–317). Wiley.Google Scholar
- Romanowicz, R., Beven, K. J., & Tawn, J. (1996). Bayesian calibration of flood inundation models. In M. G. Anderson, D. E. Walling, & P. D. Bates, (Eds.) Floodplain Processes (pp. 333–360).Google Scholar
- Reusser, D. E., Blume, T., Schaefli, B., & Zehe, E. (2009). Analysing the temporal dynamics of model performance for hydrological models. Hydrology and earth system sciences, 13(EPFL-ARTICLE-162488), 999–1018.Google Scholar
- Renard, B., Kavetski, D., Kuczera, G., Thyer, M., & Franks, S. W. (2010). Understanding predictive uncertainty in hydrologic modeling: The challenge of identifying input and structural errors. Water Resources Research, 46(5).Google Scholar
- Schoups, G., & Vrugt, J. A. (2010). A formal likelihood function for parameter and predictive inference of hydrologic models with correlated, heteroscedastic, and non‐Gaussian errors. Water Resources Research, 46(10).Google Scholar
- Sorooshian, S., & Gupta, H. V. (1995). Model calibration. In V. P. Singh (Ed.), Computer models of watershed hydrology. Highlands Ranch CO: Water Resource Publications.Google Scholar
- Spear, R. C., & Hornberger, G. M. (1980). Eutrophication in peel inlet—II. Identification of critical uncertainties via generalized sensitivity analysis. Water Research, 14(1), 43–49.Google Scholar
- Thompson, T. D. (1961). Numerical weather analysis and prediction. New York: Macmillan.Google Scholar
- Vrugt, J. A., Gupta, H. V., Bastidas, L. A., Bouten, W., & Sorooshian, S. (2003). Effective and efficient algorithm for multiobjective optimization of hydrologic models. Water Resources Research, 39(8), W01214.Google Scholar