Skip to main content
SpringerLink
Log in

Quantification of parametric uncertainty of ANN models with GLUE method for different streamflow dynamics

  • Original Paper
  • Published:
Stochastic Environmental Research and Risk Assessment Aims and scope Submit manuscript

Abstract

This study describes the parametric uncertainty of artificial neural networks (ANNs) by employing the generalized likelihood uncertainty estimation (GLUE) method. The ANNs are used to forecast daily streamflow for three sub-basins of the Rhine Basin (East Alpine, Main, and Mosel) having different hydrological and climatological characteristics. We have obtained prior parameter distributions from 5000 ANNs in the training period to capture the parametric uncertainty and subsequently 125,000 correlated parameter sets were generated. These parameter sets were used to quantify the uncertainty in the forecasted streamflow in the testing period using three uncertainty measures: percentage of coverage, average relative length, and average asymmetry degree. The results indicated that the highest uncertainty was obtained for the Mosel sub-basin and the lowest for the East Alpine sub-basin mainly due to hydro-climatic differences between these basins. The prediction results and uncertainty estimates of the proposed methodology were compared to the direct ensemble and bootstrap methods. The GLUE method successfully captured the observed discharges with the generated prediction intervals, especially the peak flows. It was also illustrated that uncertainty bands are sensitive to the selection of the threshold value for the Nash–Sutcliffe efficiency measure used in the GLUE method by employing the Wilcoxon–Mann–Whitney test.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Aqil M, Kita I, Yano A, Nishiyama S (2007) Neural networks for real time catchment flow modeling and prediction. Water Resour Manag 21(10):1781–1796. doi:10.1007/s11269-006-9127-y

    Article  Google Scholar 

  • Badrzadeh H, Sarukkalige R, Jayawardena AW (2013) Impact of multi-resolution analysis of artificial intelligence models inputs on multi-step ahead river flow forecasting. J Hydrol 507:75–85. doi:10.1016/j.jhydrol.2013.10.017

    Article  Google Scholar 

  • Badrzadeh H, Sarukkalige R, Jayawardena A (2016) Improving ann-based short-term and long-term seasonal river flow forecasting with signal processing techniques. River Res Appl 32(3):245–256

    Article  Google Scholar 

  • Beven K, Binley A (1992) The future of distributed models: model calibration and uncertainty prediction. Hydrol Process 6(3):279–298. doi:10.1002/hyp.3360060305

    Article  Google Scholar 

  • Beven K, Freer J (2001) Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology. J Hydrol 249(1):11–29

    Article  Google Scholar 

  • Boucher M-A, Laliberté J-P, Anctil F (2010) An experiment on the evolution of an ensemble of neural networks for streamflow forecasting. Hydrol Earth Syst Sci 14(3):603–612

    Article  Google Scholar 

  • Breinholt A, Grum M, Madsen H, Örn Thordarson F, Mikkelsen PS (2013) Informal uncertainty analysis (GLUE) of continuous flow simulation in a hybrid sewer system with infiltration inflow–consistency of containment ratios in calibration and validation? Hydrol Earth Syst Sci 17(10):4159–4176. doi:10.5194/hess-17-4159-2013

    Article  Google Scholar 

  • Chang F-J, Chiang Y-M, Chang L-C (2007) Multi-step-ahead neural networks for flood forecasting. Hydrol Sci J 52(1):114–130

    Article  Google Scholar 

  • Chen X, Yang T, Wang X, Xu C-Y, Yu Z (2013) Uncertainty intercomparison of different hydrological models in simulating extreme flows. Water Resour Manag 27(5):1393–1409

    Article  Google Scholar 

  • Chiang Y-M, Chang L-C, Chang F-J (2004) Comparison of static-feedforward and dynamic-feedback neural networks for rainfall–runoff modeling. J Hydrol 290(3):297–311

    Article  Google Scholar 

  • De Villiers J, Barnard E (1993) Backpropagation neural nets with one and two hidden layers. IEEE Trans Neural Netw 4(1):136–141

    Article  Google Scholar 

  • Demirel MC, Booij MJ, Hoekstra AY (2013a) Effect of different uncertainty sources on the skill of 10 day ensemble low flow forecasts for two hydrological models. Water Resour Res 49(7):4035–4053

    Article  Google Scholar 

  • Demirel MC, Booij MJ, Hoekstra AY (2013b) Impacts of climate change on the seasonality of low flows in 134 catchments in the River Rhine basin using an ensemble of bias-corrected regional climate simulations. Hydrol Earth Syst Sci 17(10):4241–4257. doi:10.5194/hess-17-4241-2013

    Article  Google Scholar 

  • Disse M, Engel H (2001) Flood events in the Rhine basin: genesis, influences and mitigation. Nat Hazards 23(2–3):271–290

    Article  Google Scholar 

  • Elipot S, Lumpkin R, Perez RC, Lilly JM, Early JJ, Sykulski AM (2016) A global surface drifter data set at hourly resolution. J Geophys Res Oceans 121(5):2937–2966. doi:10.1002/2016JC011716

    Article  Google Scholar 

  • Ghavidel SZZ, Montaseri M (2014) Application of different data-driven methods for the prediction of total dissolved solids in the Zarinehroud basin. Stoch Env Res Risk Assess 28(8):2101–2118

    Article  Google Scholar 

  • Gong Y, Shen Z, Hong Q, Liu R, Liao Q (2011) Parameter uncertainty analysis in watershed total phosphorus modeling using the GLUE methodology. Agric Ecosyst Environ 142(3):246–255

    Article  Google Scholar 

  • Hagan MT, Menhaj MB (1994) Training feedforward networks with the Marquardt algorithm. IEEE Trans Neural Netw 5(6):989–993

    Article  CAS  Google Scholar 

  • Hornberger GM, Spear R (1981) Approach to the preliminary analysis of environmental systems. J Environ Manag 12(1):7–18

    Google Scholar 

  • Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366

    Article  Google Scholar 

  • Hurkmans R, Troch P, Uijlenhoet R, Moors E (2007) Simulating Rhine River discharges using a land surface model. In: CAIWA conference vol. 1215

  • Jain A, Sudheer K, Srinivasulu S (2004) Identification of physical processes inherent in artificial neural network rainfall runoff models. Hydrol Process 18(3):571–581

    Article  Google Scholar 

  • Jin X, Xu C-Y, Zhang Q, Singh V (2010) Parameter and modeling uncertainty simulated by GLUE and a formal Bayesian method for a conceptual hydrological model. J Hydrol 383(3):147–155

    Article  Google Scholar 

  • Jothiprakash V, Magar RB (2012) Multi-time-step ahead daily and hourly intermittent reservoir inflow prediction by artificial intelligent techniques using lumped and distributed data. J Hydrol 450–451:293–307. doi:10.1016/j.jhydrol.2012.04.045

    Article  Google Scholar 

  • Jung I-W, Moradkhani H, Chang H (2012) Uncertainty assessment of climate change impacts for hydrologically distinct river basins. J Hydrol 466–467:73–87

    Article  Google Scholar 

  • Kasiviswanathan KS, Sudheer KP (2013) Quantification of the predictive uncertainty of artificial neural network based river flow forecast models. Stoch Env Res Risk Assess 27(1):137–146. doi:10.1007/s00477-012-0600-2

    Article  Google Scholar 

  • Kasiviswanathan K, Sudheer K (2016) Comparison of methods used for quantifying prediction interval in artificial neural network hydrologic models. Model Earth Syst Environ 2(1):1–11

    Article  Google Scholar 

  • Kasiviswanathan KS, Sudheer KP, He J (2016) Quantification of prediction uncertainty in artificial neural network models. In: Shanmuganathan S, Samarasinghe S (eds) Artificial neural network modelling. Springer, Cham, pp 145–159

    Chapter  Google Scholar 

  • Khan MS, Coulibaly P (2006) Bayesian neural network for rainfall-runoff modeling. Water Resour Res. doi:10.1029/2005WR003971

    Google Scholar 

  • Kingston GB, Lambert MF, Maier HR (2005) Bayesian training of artificial neural networks used for water resources modeling. Water Resour Res. doi:10.1029/2005WR004152

    Google Scholar 

  • Klein B, Meissner D, Kobialka H-U, Reggiani P (2016) Predictive uncertainty estimation of hydrological multi-model ensembles using pair-copula construction. Water 8(4):125

    Article  Google Scholar 

  • Kumar S, Tiwari MK, Chatterjee C, Mishra A (2015) Reservoir inflow forecasting using ensemble models based on neural networks, wavelet analysis and bootstrap method. Water Resour Manag 29(13):4863–4883

    Article  Google Scholar 

  • Kunkel ML, Pierce JL (2010) Reconstructing snowmelt in Idaho’s watershed using historic streamflow records. Clim Change 98(1–2):155–176

    Article  Google Scholar 

  • Lee D-H, Kang D-S (2016) The application of the artificial neural network ensemble model for simulating streamflow. Procedia Eng 154:1217–1224. doi:10.1016/j.proeng.2016.07.434

    Article  Google Scholar 

  • Liang F (2005) Bayesian neural networks for nonlinear time series forecasting. Stat Comput 15(1):13–29

    Article  Google Scholar 

  • Lohani AK, Goel N, Bhatia K (2011) Comparative study of neural network, fuzzy logic and linear transfer function techniques in daily rainfall-runoff modelling under different input domains. Hydrol Process 25(2):175–193

    Article  Google Scholar 

  • Maier HR, Dandy GC (2000) Neural networks for the prediction and forecasting of water resources variables: a review of modelling issues and applications. Environ Model Softw 15(1):101–124. doi:10.1016/S1364-8152(99)00007-9

    Article  Google Scholar 

  • May RJ, Maier HR, Dandy GC, Fernando TG (2008) Non-linear variable selection for artificial neural networks using partial mutual information. Environ Model Softw 23(10):1312–1326

    Article  Google Scholar 

  • Melesse AM, Ahmad S, McClain ME, Wang X, Lim YH (2011) Suspended sediment load prediction of river systems: an artificial neural network approach. Agric Water Manag 98(5):855–866. doi:10.1016/j.agwat.2010.12.012

    Article  Google Scholar 

  • Middelkoop H, Daamen K, Gellens D, Grabs W, Kwadijk JCJ, Lang H, Parmet BWAH, Schädler B, Schulla J, Wilke K (2001) Impact of climate change on hydrological regimes and water resources management in the Rhine basin. Clim Change 49(1):105–128. doi:10.1023/a:1010784727448

    Article  CAS  Google Scholar 

  • Mirzaei M, Huang YF, El-Shafie A, Shatirah A (2015) Application of the generalized likelihood uncertainty estimation (GLUE) approach for assessing uncertainty in hydrological models: a review. Stoch Env Res Risk Assess 29(5):1265–1273

    Article  Google Scholar 

  • Ng WW, Panu US, Lennox WC (2007) Chaos based analytical techniques for daily extreme hydrological observations. J Hydrol 342:17–41

    Article  Google Scholar 

  • Noori R, Hoshyaripour G, Ashrafi K, Araabi BN (2010) Uncertainty analysis of developed ANN and ANFIS models in prediction of carbon monoxide daily concentration. Atmos Environ 44(4):476–482

    Article  CAS  Google Scholar 

  • Papadopoulos G, Edwards PJ, Murray AF (2001) Confidence estimation methods for neural networks: a practical comparison. IEEE Trans Neural Netw 12(6):1278–1287

    Article  CAS  Google Scholar 

  • Pramanik N, Panda RK (2009) Application of neural network and adaptive neuro-fuzzy inference systems for river flow prediction. Hydrol Sci J 54(2):247–260. doi:10.1623/hysj.54.2.247

    Article  Google Scholar 

  • Ranjithan S, Eheart J, Garrett J (1993) Neural network-based screening for groundwater reclamation under uncertainty. Water Resour Res 29(3):563–574

    Article  Google Scholar 

  • Rogiers B, Mallants D, Batelaan O, Gedeon M, Huysmans M, Dassargues A (2012) Estimation of hydraulic conductivity and its uncertainty from grain-size data using GLUE and artificial neural networks. Math Geosci 44(6):739–763

    Article  Google Scholar 

  • Selle B, Hannah M (2010) A bootstrap approach to assess parameter uncertainty in simple catchment models. Environ Model Softw 25(8):919–926

    Article  Google Scholar 

  • Singh KP, Basant A, Malik A, Jain G (2009) Artificial neural network modeling of the river water quality—a case study. Ecol Model 220(6):888–895

    Article  CAS  Google Scholar 

  • Spear R, Hornberger G (1980) Eutrophication in Peel Inlet—II. Identification of critical uncertainties via generalized sensitivity analysis. Water Res 14(1):43–49

    Article  CAS  Google Scholar 

  • Srivastav RK, Sudheer KP, Chaubey I (2007) A simplified approach to quantifying predictive and parametric uncertainty in artificial neural network hydrologic models. Water Resour Res 43(10):W10407. doi:10.1029/2006WR005352

    Article  Google Scholar 

  • Stefánsson A, Končar N, Jones AJ (1997) A note on the gamma test. Neural Comput Appl 5(3):131–133

    Article  Google Scholar 

  • Talebizadeh M, Morid S, Ayyoubzadeh SA, Ghasemzadeh M (2010) Uncertainty analysis in sediment load modeling using ANN and SWAT model. Water Resour Manag 24(9):1747–1761

    Article  Google Scholar 

  • Tian Y, Booij M, Xu Y-P (2014) Uncertainty in high and low flows due to model structure and parameter errors. Stoch Env Res Risk Assess 28(2):319–332. doi:10.1007/s00477-013-0751-9

    Article  Google Scholar 

  • Tiwari MK, Chatterjee C (2010) Uncertainty assessment and ensemble flood forecasting using bootstrap based artificial neural networks (BANNs). J Hydrol 382(1):20–33

    Article  Google Scholar 

  • Tiwari MK, Chatterjee C (2011) A new wavelet–bootstrap–ANN hybrid model for daily discharge forecasting. J Hydroinform 13(3):500–519

    Article  Google Scholar 

  • Tongal H, Berndtsson R (2016) Impact of complexity on daily and multi-step forecasting of streamflow with chaotic, stochastic, and black-box models. Stoch Env Res Risk Assess. doi:10.1007/s00477-016-1236-4

    Google Scholar 

  • Tongal H, Demirel MC, Booij MJ (2013) Seasonality of low flows and dominant processes in the Rhine river. Stoch Env Res Risk Assess 27(2):489–503. doi:10.1007/s00477-012-0594-9

    Article  Google Scholar 

  • Toth E, Brath A (2007) Multistep ahead streamflow forecasting: role of calibration data in conceptual and neural network modeling. Water Resour Res 43(11):W11405. doi:10.1029/2006WR005383

    Article  Google Scholar 

  • Uehlinger U, Arndt H, Wantzen KM, Leuven RSEW (2009) The Rhine river basin. Rivers of Europe, chap 6. Academic Press, London, pp 199–245

  • Uniyal B, Jha MK, Verma AK (2015) Parameter identification and uncertainty analysis for simulating streamflow in a river basin of Eastern India. Hydrol Process 29(17):3744–3766

    Article  Google Scholar 

  • Walker WE, Harremoës P, Rotmans J, van der Sluijs JP, van Asselt MBA, Janssen P, Krayer von Krauss MP (2003) Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support. Integr Assess 4(1):5–17. doi:10.1076/iaij.4.1.5.16466

    Article  Google Scholar 

  • Wang W-C, Chau K-W, Cheng C-T, Qiu L (2009a) A comparison of performance of several artificial intelligence methods for forecasting monthly discharge time series. J Hydrol 374(3–4):294–306. doi:10.1016/j.jhydrol.2009.06.019

    Article  Google Scholar 

  • Wang W, Jin J, Li Y (2009b) Prediction of inflow at three gorges dam in yangtze river with wavelet network model. Water Resour Manag 23(13):2791–2803. doi:10.1007/s11269-009-9409-2

    Article  Google Scholar 

  • Wang Y, Zheng T, Zhao Y, Jiang J, Wang Y, Guo L, Wang P (2013) Monthly water quality forecasting and uncertainty assessment via bootstrapped wavelet neural networks under missing data for Harbin, China. Environ Sci Pollut Res 20(12):8909–8923. doi:10.1007/s11356-013-1874-8

    Article  Google Scholar 

  • Warmink JJ, Booij MJ (2015) Uncertainty analysis in river modelling. In: Rowiński P, Radecki-Pawlik A (eds) Rivers—physical, fluvial and environmental processes. Springer, Cham, pp 255–277

    Chapter  Google Scholar 

  • Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83

    Article  Google Scholar 

  • Xiong L, Wan M, Wei X, O’connor KM (2009) Indices for assessing the prediction bounds of hydrological models and application by generalised likelihood uncertainty estimation/Indices pour évaluer les bornes de prévision de modèles hydrologiques et mise en œuvre pour une estimation d’incertitude par vraisemblance généralisée. Hydrol Sci J 54(5):852–871

    Article  Google Scholar 

  • Yu J, Qin X, Larsen O (2015) Uncertainty analysis of flood inundation modelling using GLUE with surrogate models in stochastic sampling. Hydrol Process 29(6):1267–1279

    Article  Google Scholar 

  • Zeroual A, Meddi M, Assani AA (2016) Artificial neural network rainfall-discharge model assessment under rating curve uncertainty and monthly discharge volume predictions. Water Resour Manag. doi:10.1007/s11269-016-1340-8

    Google Scholar 

  • Zhang W, Li T (2015) The influence of objective function and acceptability threshold on uncertainty assessment of an urban drainage hydraulic model with generalized likelihood uncertainty estimation methodology. Water Resour Manag 29(6):2059–2072

    Article  Google Scholar 

  • Zhang X, Liang F, Srinivasan R, Van Liew M (2009) Estimating uncertainty of streamflow simulation using Bayesian neural networks. Water Resour Res 45(2):1–16

    Article  Google Scholar 

  • Zhang H, Zhou J, Ye L, Zeng X, Chen Y (2015) Lower upper bound estimation method considering symmetry for construction of prediction intervals in flood forecasting. Water Resour Manag 29(15):5505–5519

    Article  Google Scholar 

  • Zhang J, Lin X, Guo B (2016) Multivariate copula-based joint probability distribution of water supply and demand in irrigation district. Water Resour Manag 30(7):2361–2375. doi:10.1007/s11269-016-1293-y

    Article  Google Scholar 

Download references

Acknowledgement

The authors thank two anonymous reviewers for their constructive comments that improved the paper considerably.

Author information

Authors and Affiliations

  1. Department of Civil Engineering, Engineering Faculty, Süleyman Demirel University, Isparta, Turkey

    Hakan Tongal

  2. Department of Water Engineering and Management, Faculty of Engineering Technology, University of Twente, Enschede, The Netherlands

    Martijn J. Booij

Authors
  1. Hakan Tongal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martijn J. Booij
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hakan Tongal.

Appendices

Appendix 1

Obtained distributions and parameters of the ANN models for the East Alpine, Main, and Mosel sub-basins from the 5000 ANNs during the training period can be found below Tables 5, 6, 7.

Table 5 Characteristics of probability distribution functions of ANN parameters for the East Alpine sub-basin
Full size table
Table 6 Characteristics of probability distribution functions of ANN parameters for the Main sub-basin
Full size table
Table 7 Characteristics of probability distribution functions of ANN parameters for the Mosel sub-basin
Full size table

Appendix 2

The formulas of the indices for assessing the prediction bounds can be found below Table 8:

Table 8 Indices for assessing the prediction bounds
Full size table

where \(n\) is the number of time steps used for constructing the prediction bands and \(c_{i}\) denotes whether the corresponding observation is covered by the prediction band being 1 or 0 if the observed value is contained in the prediction band or not, \(L_{t}^{upper}\) and \(L_{t}^{lower}\) are the upper and lower prediction boundary values of the 95% confidence interval, and \(Q_{t}\) is the observed value at time t.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tongal, H., Booij, M.J. Quantification of parametric uncertainty of ANN models with GLUE method for different streamflow dynamics. Stoch Environ Res Risk Assess 31, 993–1010 (2017). https://doi.org/10.1007/s00477-017-1408-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00477-017-1408-x

Keywords

Search

Navigation