Advertisement

Frontiers of Earth Science

, Volume 12, Issue 4, pp 653–660 | Cite as

Ensembles vs. information theory: supporting science under uncertainty

  • Grey S. Nearing
  • Hoshin V. GuptaEmail author
Review

Abstract

Multi-model ensembles are one of the most common ways to deal with epistemic uncertainty in hydrology. This is a problem because there is no known way to sample models such that the resulting ensemble admits a measure that has any systematic (i.e., asymptotic, bounded, or consistent) relationship with uncertainty. Multi-model ensembles are effectively sensitivity analyses and cannot – even partially – quantify uncertainty. One consequence of this is that multi-model approaches cannot support a consistent scientific method – in particular, multi-model approaches yield unbounded errors in inference. In contrast, information theory supports a coherent hypothesis test that is robust to (i.e., bounded under) arbitrary epistemic uncertainty. This paper may be understood as advocating a procedure for hypothesis testing that does not require quantifying uncertainty, but is coherent and reliable (i.e., bounded) in the presence of arbitrary (unknown and unknowable) uncertainty. We conclude by offering some suggestions about how this proposed philosophy of science suggests new ways to conceptualize and construct simulation models of complex, dynamical systems.

Keywords

information theory multi-model ensembles Bayesian methods uncertainty quantification hypothesis testing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

We thank Professor Steven Fassnacht for his invitation and encouragement to submit this manuscript to the special issue on “Uncertainty in Water Resources”, and are grateful to Anneli Guthke, Uwe Ehret, and one other anonymous reviewer for their helpful and constructive comments that helped to clarify points raised herein.

References

  1. Albrecht A, Phillips D (2014). Origin of probabilities and their application to the multiverse. Phys Rev D Part Fields Gravit Cosmol, 90(12): 123514CrossRefGoogle Scholar
  2. Beven K, Freer J (2001). Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology. J Hydrol (Amst), 249(1–4): 11–29CrossRefGoogle Scholar
  3. Beven K J (2006). Searching for the Holy Grail of scientific hydrology: Q t = (SR)A as closure. Hydrol Earth Syst Sci, 10(5): 609–618CrossRefGoogle Scholar
  4. Beven K J (2016). Facets of uncertainty: epistemic uncertainty, nonstationarity, likelihood, hypothesis testing, and communication. Hydrol Sci J, 61(9): 1652–1665CrossRefGoogle Scholar
  5. Beven K J, Smith P J, Freer J E (2008). So just why would a modeller choose to be incoherent? J Hydrol (Amst), 354(1): 15–32CrossRefGoogle Scholar
  6. Clark M P, Kavetski D, Fenicia F (2011). Pursuing the method of multiple working hypotheses for hydrological modeling. Water Resour Res, 47(9): https://doi.org/10.1029/2010WR009827
  7. Clark M P, Nijssen B, Lundquist J D, Kavetski D, Rupp D E, Woods R A, Freer J E, Gutmann E D, Wood A W, Brekke L D, Arnold J R, Gochis D J, Rasmussen R M (2015). A unified approach for processbased hydrologic modeling: 1. Modeling concept. Water Resour Res, 51(4): 2498–2514CrossRefGoogle Scholar
  8. Eyring V, Bony S, Meehl G A, Senior C A, Stevens B, Stouffer R J, Taylor K E (2016). Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci Model Dev, 9(5): 1937–1958CrossRefGoogle Scholar
  9. Gelman A, Shalizi C R (2013). Philosophy and the practice of Bayesian statistics. Br J Math Stat Psychol, 66(1): 8–38CrossRefGoogle Scholar
  10. Gong W, Gupta H V, Yang D, Sricharan K, Hero A O III (2013). Estimating epistemic and aleatory uncertainties during hydrologic modeling: an information theoretic approach. Water Resour Res, 49 (4): 2253–2273CrossRefGoogle Scholar
  11. Grünwald P, Langford J (2007). Suboptimal behavior of Bayes and MDL in classification under misspecification. Mach Learn, 66(2–3): 119–149CrossRefGoogle Scholar
  12. Hornik K (1991). Approximation capabilities of multilayer feedforward networks. Neural Netw, 4(2): 251–257CrossRefGoogle Scholar
  13. Kinney J B, Atwal G S (2014). Equitability, mutual information, and the maximal information coefficient. Proc Natl Acad Sci USA, 111(9): 3354–3359CrossRefGoogle Scholar
  14. Metropolis N (1987). The beginning of the Monte Carlo method. Los Alamos Sci, 15(584): 125–130Google Scholar
  15. Montanari A (2007). What do we mean by ‘uncertainty’? The need for a consistent wording about uncertainty assessment in hydrology. Hydrol Processes, 21(6): 841–845CrossRefGoogle Scholar
  16. Nearing G S, Gupta H V (2015). The quantity and quality of information in hydrologic models. Water Resour Res, 51(1): 524–538CrossRefGoogle Scholar
  17. Nearing G S, Mocko D M, Peters-Lidard C D, Kumar S V, Xia Y (2016a). Benchmarking NLDAS-2 soil moisture and evapotranspiration to separate uncertainty contributions. J Hydrometeorol, 17(3): 745–759CrossRefGoogle Scholar
  18. Nearing G S, Tian Y, Gupta H V, Clark M P, Harrison K W, Weijs S V (2016b). A philosophical basis for hydrologic uncertainty. Hydrol Sci J, 61(9): 1666–1678CrossRefGoogle Scholar
  19. Popper K R (1959). The Logic of Scientific Discovery. London: Hutchinson & Co.Google Scholar
  20. Rasmussen C, Williams C (2006). Gaussian Processes for Machine Learning. Gaussian Processes for Machine Learning. Cambridge, MA: MIT PressGoogle Scholar
  21. Renard B, Kavetski D, Kuczera G, Thyer M, Franks S W (2010). Understanding predictive uncertainty in hydrologic modeling: the challenge of identifying input and structural errors. Water Resour Res, 46(5): https://doi.org/10.1029/2009WR008328
  22. Shannon C E (1948). A mathematical theory of communication. Bell Syst Tech J, 27(3): 379–423CrossRefGoogle Scholar
  23. Stanford K (2016). Underdetermination of Scientific Theory. In: Zalta N, ed. The Stanford Encyclopedia of PhilosophyGoogle Scholar
  24. Taleb N N (2010). The Black Swan: the Impact of the Highly Improbable Fragility. New York: Random House GroupGoogle Scholar
  25. Van Horn K S (2003). Constructing a logic of plausible inference: a guide to Cox’s theorem. Int J Approx Reason, 34(1): 3–24CrossRefGoogle Scholar
  26. Weijs S V, Schoups G, van de Giesen N (2010). Why hydrological predictions should be evaluated using information theory. Hydrol Earth Syst Sci, 14(12): 2545–2558CrossRefGoogle Scholar

Copyright information

© Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of Alabama, Department of Geological SciencesTuscaloosaUSA
  2. 2.University of Arizona, Department of Hydrology and Atmospheric SciencesTucsonUSA

Personalised recommendations