Skip to main content
SpringerLink
Log in

Methods used for quantifying the prediction uncertainty of artificial neural network based hydrologic models

  • Review
  • Published:
Stochastic Environmental Research and Risk Assessment Aims and scope Submit manuscript

Abstract

Application of artificial neural network (ANN) models has been reported to solve variety of water resources and environmental related problems including prediction, forecasting and classification, over the last two decades. Though numerous research studies have witnessed the improved estimate of ANN models, the practical applications are sometimes limited. The black box nature of ANN models and their parameters hardly convey the physical meaning of catchment characteristics, which result in lack of transparency. In addition, it is perceived that the point prediction provided by ANN models does not explain any information about the prediction uncertainty, which reduce the reliability. Thus, there is an increasing consensus among researchers for developing methods to quantify the uncertainty of ANN models, and a comprehensive evaluation of uncertainty methods applied in ANN models is an emerging field that calls for further improvements. In this paper, methods used for quantifying the prediction uncertainty of ANN based hydrologic models are reviewed based on the research articles published from the year 2002 to 2015, which focused on modeling streamflow forecast/prediction. While the flood forecasting along with uncertainty quantification has been frequently reported in applications other than ANN in the literature, the uncertainty quantification in ANN model is a recent progress in the field, emerged from the year 2002. Based on the review, it is found that methods for best way of incorporating various aspects of uncertainty in ANN modeling require further investigation. Though model inputs, parameters and structure uncertainty are mainly considered as the source of uncertainty, information of their mutual interaction is still lacking while estimating the total prediction uncertainty. The network topology including number of layers, nodes, activation function and training algorithm has often been optimized for the model accuracy, however not in terms of model uncertainty. Finally, the effective use of various uncertainty evaluation indices should be encouraged for the meaningful quantification of uncertainty. This review article also discusses the effectiveness and drawbacks of each method and suggests recommendations for further improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Abrahart RJ, See LM, Dawson CW, Shamseldin AY, Wilby RL (2010) Nearly two decades of neural network hydrologic modeling. In: Sivakumar B, Berndtsson R (eds) Advances in data-based approaches for hydrologic modeling and forecasting. World Scientific Publishing, Hackensack, NJ, pp 267–346

    Chapter  Google Scholar 

  • Alvisi S, Franchini M (2011) Fuzzy neural networks for water level and discharge forecasting with uncertainty. Environ Model Softw 26:523–537. doi:10.1016/j.envsoft.2010.10.016

    Article  Google Scholar 

  • Alvisi S, Franchini M (2012) Grey neural networks for river stage forecasting with uncertainty. Phys Chem Earth Parts A/B/C 42–44:108–118. doi:10.1016/j.pce.2011.04.002

    Article  Google Scholar 

  • Araghinejad S, Azmi M, Kholghi M (2011) Application of artificial neural network ensembles in probabilistic hydrological forecasting. J Hydrol 407:94–104. doi:10.1016/j.jhydrol.2011.07.011

    Google Scholar 

  • Artigue G, Johannet A, Borrell V, Pistre S (2012) Flash flood forecasting in poorly gauged basins using neural networks: case study of the Gardon de Mialet basin (southern France). Nat Hazards Earth Syst Sci 12:3307–3324. doi:10.5194/nhess-12-3307-2012

    Article  Google Scholar 

  • ASCE Task Committee on Application of Artificial Neural Networks in Hydrology (2000a) Artificial neural networks in hydrology. I: preliminary concepts. J Hydrol Eng 5(2):115–123

    Article  Google Scholar 

  • ASCE Task Committee on Application of Artificial Neural Networks in Hydrology (2000b) Artificial neural networks in hydrology. II: hydrologic applications. J Hydrol Eng 5(2):124–137

    Article  Google Scholar 

  • Asefa T (2009) Ensemble streamflow forecast: a glue-based neural network approach. J Am Water Resour Assoc 45:1155–1163. doi:10.1111/j.1752-1688.2009.00351.x

    Article  Google Scholar 

  • Bishop CM (2004) Neural networks for pattern recognition. Oxford University Press, Oxford

    Google Scholar 

  • Boucher MA, Laliberté J-P, Anctil F (2009a) An experiment on the evolution of an ensemble of neural networks for streamflow forecasting. Hydrol Earth Syst Sci Discuss 6:6265–6291

    Article  Google Scholar 

  • Boucher M-A, Perreault L, Anctil F (2009b) Tools for the assessment of hydrological ensemble forecasts obtained by neural networks. J Hydroinform 11:297. doi:10.2166/hydro.2009.037

    Article  Google Scholar 

  • Cannon AJ, Whitfield PH (2002) Downscaling recent streamflow conditions in British Columbia, Canada using ensemble neural network models. J Hydrol 259:136–151. doi:10.1016/S0022-1694(01)00581-9

    Article  Google Scholar 

  • Chang FJ, Chen YC (2001) A counterpropagation fuzzy-neural network modelling approach to real time streamflow prediction. J Hydrol 245:153–164

    Article  Google Scholar 

  • Citakoglu H (2015) Comparison of artificial intelligence techniques via empirical equations for prediction of solar radiation. Comput Electron Agric 118:28–37

    Article  Google Scholar 

  • Cobaner M (2011) Evapotranspiration estimation by two different neuro-fuzzy inference systems. J Hydrol 398(3–4):292–302

    Article  Google Scholar 

  • Cullmann J, Krausse T, Philipp A (2009) Communicating flood forecast uncertainty under operational circumstances. J Flood Risk Manag 2:306–314. doi:10.1111/j.1753-318X.2009.01048.x

    Article  Google Scholar 

  • Elshorbagy A, Corzo G, Srinivasulu S, Solomatine DP (2010a) Experimental investigation of the predictive capabilities of data driven modeling techniques in hydrology—part 1: concepts and methodology. Hydrol Earth Syst Sci 14:1931–1941

    Article  Google Scholar 

  • Elshorbagy A, Corzo G, Srinivasulu S, Solomatine DP (2010b) Experimental investigation of the predictive capabilities of data driven modeling techniques in hydrology—part 2: application. Hydrol Earth Syst Sci 14:1943–1961

    Article  Google Scholar 

  • Fleming SW, Bourdin DR, Campbell D et al (2015) Development and operational testing of a super-ensemble artificial intelligence flood-forecast model for a Pacific northwest river. JAWRA J Am Water Resour Assoc 51:502–512. doi:10.1111/jawr.12259

    Article  Google Scholar 

  • Flood I, Kartam N (1994) Neural networks in civil engineering. I: principles and understanding. J Comput Civil Eng 8(2):131–148

    Article  Google Scholar 

  • Guo J, Zhou J, Song L et al (2013) Uncertainty assessment and optimization of hydrological model with the shuffled complex evolution metropolis algorithm: an application to artificial neural network rainfall-runoff model. Stoch Environ Res Risk Assess 27:985–1004. doi:10.1007/s00477-012-0639-0

    Article  Google Scholar 

  • Han DT, Kwong Li S (2007) Uncertainties in real-time flood forecasting with neural networks. Hydrol Process 21(2):223–228. doi:10.1002/hyp.6184

    Article  Google Scholar 

  • Hersbach H (2000) Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather Forecast 15:559–570

    Article  Google Scholar 

  • Jayakrishnan R, Srinivasan R, Santhi C, Arnold JG (2005) Advances in the application of the SWAT model for water resources management. Hydrol Process 19:749–762

    Article  Google Scholar 

  • Jeong D, Kim Y (2005) Rainfall-runoff models using artificial neural networks for ensemble streamflow prediction. Hydrol Process 19:3819–3835. doi:10.1002/hyp.5983

    Article  Google Scholar 

  • Jin X, Xu CY, Zhang Q et al (2010) Parameter and modeling uncertainty simulated by GLUE and a formal Bayesian method for a conceptual hydrological model. J Hydrol 383(3–4):147–155

    Article  Google Scholar 

  • Kan G, Yao C, Li Q et al (2015) Improving event-based rainfall-runoff simulation using an ensemble artificial neural network based hybrid data-driven model. Stoch Environ Res Risk Assess 29:1345–1370. doi:10.1007/s00477-015-1040-6

    Article  Google Scholar 

  • Kant A, Suman PK, Giri BK et al (2013) Comparison of multi-objective evolutionary neural network, adaptive neuro-fuzzy inference system and bootstrap-based neural network for flood forecasting. Neural Comput Appl 23:231–246. doi:10.1007/s00521-013-1344-8

    Article  Google Scholar 

  • Kasiviswanathan KS, Sudheer KP (2013) Quantification of the predictive uncertainty of artificial neural network based river flow forecast models. Stoch Environ Res Risk Assess 27:137–146. doi:10.1007/s00477-012-0600-2

    Article  Google Scholar 

  • Kasiviswanathan KS, Cibin R, Sudheer KP, Chaubey I (2013) Constructing prediction interval for artificial neural network rainfall runoff models based on ensemble simulations. J Hydrol 499:275–288. doi:10.1016/j.jhydrol.2013.06.043

    Article  Google Scholar 

  • Khan MS, Coulibaly P (2006) Bayesian neural network for rainfall-runoff modeling. Water Resour Res 42:1–18. doi:10.1029/2005WR003971

    Article  Google Scholar 

  • Khan MS, Coulibaly P (2010) Assessing hydrologic impact of climate change with uncertainty estimates: Bayesian neural network approach. J Hydrometeorol 11:482–495. doi:10.1175/2009JHM1160.1

    Article  Google Scholar 

  • Kim S, Kim HS (2008) Uncertainty reduction of the flood stage forecasting using neural networks model 1. J Am Water Resour Assoc 44:148–165

    Article  CAS  Google Scholar 

  • Kim SE, Seo IW (2015) Artificial neural network ensemble modeling with conjunctive data clustering for water quality prediction in rivers. J Hydro-Environ Res. doi:10.1016/j.jher.2014.09.006

    Google Scholar 

  • Kitanidis PK, Bras RL (1980) Real-time forecasting with a conceptual hydrologic model 2. Applications and results. Water Resour Res 16(6):1034–1044

    Article  Google Scholar 

  • Krupnick AR, Morgenstern M, Batz P et al (2006) Not a sure thing: making regulatory choices under uncertainty. Resources for the Future, Washington, DC

    Google Scholar 

  • Kumar S, Tiwari MK, Chatterjee C, Mishra A (2015) Reservoir inflow forecasting using ensemble models based on neural networks, wavelet analysis and bootstrap method. Water Resour Manag 29:4863–4883. doi:10.1007/s11269-015-1095-7

    Article  Google Scholar 

  • MacKay DJC (1992) Bayesian interpolation. Neural Comput 4:720–736

    Article  Google Scholar 

  • Maier HR, Dandy GC (2000) Neural networks for the prediction and forecasting of water resources variables: a review of modelling issues and applications. Environ Model Softw 15(1):101–124

    Article  Google Scholar 

  • Maier HR, Jain A, Dandy GC, Sudheer KP (2010) Methods used for the development of neural networks for the prediction of water resource variables in river systems: current status and future directions. Environ Model Softw 25:891–909. doi:10.1016/j.envsoft.2010.02.003

    Article  Google Scholar 

  • Matheson JE, Winkler RL (1976) Scoring rules for continuous probability distributions. Manag Sci 22:1087–1095

    Article  Google Scholar 

  • McKay M, Beckman R, Conover W (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245

    Google Scholar 

  • Morgan M, Henrion GM, Small M (1990) Uncertainty: a guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge University Press, New York

    Book  Google Scholar 

  • Nayak PC, Sudheer KP, Rangan DM, Ramasastri KS (2005) Short-term flood forecasting with a neurofuzzy model. Water Resour Res 41:1–16. doi:10.1029/2004WR003562

    Article  Google Scholar 

  • Neal RM (1996) Bayesian learning for neural networks. Springer, New York

    Book  Google Scholar 

  • Oliveira GG, Pedrollo OC, Castro NMR (2015) Stochastic approach to analyzing the uncertainties and possible changes in the availability of water in the future based on scenarios of climate change. Hydrol Earth Syst Sci 19:3585–3604. doi:10.5194/hess-19-3585-2015

    Article  Google Scholar 

  • Roulston MS, Smith LA (2002) Evaluating probabilistic fore- casts using information theory. Mon Weather Rev 130:1653–1660

    Article  Google Scholar 

  • Schreiber HA, Kincaid DR (1967) Regression models for predicting on-site runoff from short-duration convective storms. Water Resour Res 3(2):389–395

    Article  Google Scholar 

  • Sharma SK, Tiwari KN (2009) Bootstrap based artificial neural network (BANN) analysis for hierarchical prediction of monthly runoff in Upper Damodar Valley Catchment. J Hydrol 374:209–222. doi:10.1016/j.jhydrol.2009.06.003

    Article  Google Scholar 

  • Shrestha RR, Nestmann F (2009) Physically based and data-driven models and propagation of input uncertainties in river flood prediction. J Hydrol Eng 14:1309–1319. doi:10.1061/(ASCE)HE.1943-5584.0000123

    Article  Google Scholar 

  • Shrestha DL, Solomatine DP (2008) Data-driven approaches for estimating uncertainty in rainfall-runoff modelling. Int J River Basin Manag 6:109–122. doi:10.1080/15715124.2008.9635341

    Article  Google Scholar 

  • Shrestha DL, Kayastha N, Solomatine DP (2009) A novel approach to parameter uncertainty analysis of hydrological models using neural networks. Hydrol Earth Syst Sci 13:1235–1248. doi:10.5194/hess-13-1235-2009

    Article  Google Scholar 

  • Srivastav RK, Sudheer KP, Chaubey I (2007) A simplified approach to quantifying predictive and parametric uncertainty in artificial neural network hydrologic models. Water Resour Res 43. doi:10.1029/2006WR005352

  • Taormina R, Chau K-W (2015) ANN-based interval forecasting of streamflow discharges using the LUBE method and MOFIPS. Eng Appl Artif Intell 45:429–440. doi:10.1016/j.engappai.2015.07.019

    Article  Google Scholar 

  • Tiwari MK, Chatterjee C (2010a) Development of an accurate and reliable hourly flood forecasting model using wavelet-bootstrap-ANN (WBANN) hybrid approach. J Hydrol 394:458–470. doi:10.1016/j.jhydrol.2010.10.001

    Article  Google Scholar 

  • Tiwari MK, Chatterjee C (2010b) Uncertainty assessment and ensemble flood forecasting using bootstrap based artificial neural networks (BANNs). J Hydrol 382:20–33. doi:10.1016/j.jhydrol.2009.12.013

    Article  Google Scholar 

  • Tiwari MK, Chatterjee C (2011) A new wavelet–bootstrap–ANN hybrid model for daily discharge forecasting. J Hydroinf 13:500. doi:10.2166/hydro.2010.142

    Article  Google Scholar 

  • Tung YK, Yen BC (2005) Hydrosystems engineering uncertainty analysis. McGraw Hill, New York

    Google Scholar 

  • Xiong L, Wan M, Wei X, O’Connor KM (2009) Indices for assessing the prediction bounds of hydrological models and application by generalised likelihood uncertainty estimation/Indices pour évaluer les bornes de prévision de modèleshydrologiques et miseenœuvre pour une estimation d’incertitude. Hydrol Sci J 54:852–871. doi:10.1623/hysj.54.5.852

    Article  Google Scholar 

  • Yang CC, Chen CS (2009) Application of integrated back-propagation network and self organizing map for flood forecasting. Hydrol Process 23:1313–1323. doi:10.1002/hyp.7248

    Article  Google Scholar 

  • Yu JJ, Qin XS, Larsen O (2015) Uncertainty analysis of flood inundation modelling using GLUE with surrogate models in stochastic sampling. Hydrol Process 29:1267–1279. doi:10.1002/hyp.10249

    Article  Google Scholar 

  • Zhang X, Zhao K (2012) Bayesian neural networks for uncertainty analysis of hydrologic modeling: a comparison of two schemes. Water Resour Manag 26:2365–2382. doi:10.1007/s11269-012-0021-5

    Article  Google Scholar 

  • Zhang X, Liang F, Srinivasan R, Van Liew M (2009) Estimating uncertainty of streamflow simulation using Bayesian neural networks. Water Resour Res 45:1–16. doi:10.1029/2008WR007030

    Google Scholar 

  • Zhang X, Liang F, Yu B, Zong Z (2011) Explicitly integrating parameter, input, and structure uncertainties into Bayesian neural networks for probabilistic hydrologic forecasting. J Hydrol 409:696–709. doi:10.1016/j.jhydrol.2011.09.002

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank two anonymous reviewers for their insightful comments, which helped improving the quality of the paper.

Author information

Authors and Affiliations

  1. Department of Civil Engineering, Schulich School of Engineering, University of Calgary, 2500 University Drive NW, Calgary, T2N 1N4, Canada

    K. S. Kasiviswanathan

  2. Department of Civil Engineering, Indian Institute of Technology Madras, Chennai, 600 036, India

    K. P. Sudheer

  3. Department of Agricultural and Biological Engineering, Purdue University, West Lafayette, IN, 47906, USA

    K. P. Sudheer

Authors
  1. K. S. Kasiviswanathan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. K. P. Sudheer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. S. Kasiviswanathan.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This paper does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

For this type of study, formal consent is not required.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kasiviswanathan, K.S., Sudheer, K.P. Methods used for quantifying the prediction uncertainty of artificial neural network based hydrologic models. Stoch Environ Res Risk Assess 31, 1659–1670 (2017). https://doi.org/10.1007/s00477-016-1369-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00477-016-1369-5

Keywords

Search

Navigation