Ensembles vs. information theory: supporting science under uncertainty
- 143 Downloads
Multi-model ensembles are one of the most common ways to deal with epistemic uncertainty in hydrology. This is a problem because there is no known way to sample models such that the resulting ensemble admits a measure that has any systematic (i.e., asymptotic, bounded, or consistent) relationship with uncertainty. Multi-model ensembles are effectively sensitivity analyses and cannot – even partially – quantify uncertainty. One consequence of this is that multi-model approaches cannot support a consistent scientific method – in particular, multi-model approaches yield unbounded errors in inference. In contrast, information theory supports a coherent hypothesis test that is robust to (i.e., bounded under) arbitrary epistemic uncertainty. This paper may be understood as advocating a procedure for hypothesis testing that does not require quantifying uncertainty, but is coherent and reliable (i.e., bounded) in the presence of arbitrary (unknown and unknowable) uncertainty. We conclude by offering some suggestions about how this proposed philosophy of science suggests new ways to conceptualize and construct simulation models of complex, dynamical systems.
Keywordsinformation theory multi-model ensembles Bayesian methods uncertainty quantification hypothesis testing
Unable to display preview. Download preview PDF.
We thank Professor Steven Fassnacht for his invitation and encouragement to submit this manuscript to the special issue on “Uncertainty in Water Resources”, and are grateful to Anneli Guthke, Uwe Ehret, and one other anonymous reviewer for their helpful and constructive comments that helped to clarify points raised herein.
- Clark M P, Kavetski D, Fenicia F (2011). Pursuing the method of multiple working hypotheses for hydrological modeling. Water Resour Res, 47(9): https://doi.org/10.1029/2010WR009827
- Metropolis N (1987). The beginning of the Monte Carlo method. Los Alamos Sci, 15(584): 125–130Google Scholar
- Popper K R (1959). The Logic of Scientific Discovery. London: Hutchinson & Co.Google Scholar
- Rasmussen C, Williams C (2006). Gaussian Processes for Machine Learning. Gaussian Processes for Machine Learning. Cambridge, MA: MIT PressGoogle Scholar
- Renard B, Kavetski D, Kuczera G, Thyer M, Franks S W (2010). Understanding predictive uncertainty in hydrologic modeling: the challenge of identifying input and structural errors. Water Resour Res, 46(5): https://doi.org/10.1029/2009WR008328
- Stanford K (2016). Underdetermination of Scientific Theory. In: Zalta N, ed. The Stanford Encyclopedia of PhilosophyGoogle Scholar
- Taleb N N (2010). The Black Swan: the Impact of the Highly Improbable Fragility. New York: Random House GroupGoogle Scholar