Abstract
Characterized by essential complexity, dynamism, and dynamics, streamflow forecasting presents a great challenge to hydrologists. Long short-term memory (LSTM) streamflow forecast model has received a lot of attention in recent years due to its powerful non-linear modeling ability. But probabilistic streamflow forecasting has rarely been addressed by the LSTM approach. In this study, a probabilistic Long Short-Term Memory network coupled with the Gaussian process (GP) is proposed to deal with the probabilistic daily streamflow forecasting. Moreover, considering that changing mean and variance over time exist in the daily streamflow time series, the heteroscedastic Gaussian process regression is adopted to produce a varying prediction interval. The proposed method encapsulates the inductive biases of the LSTM recurrent network and retains the non-parametric, probabilistic property of Gaussian processes. The performance of the proposed model is investigated by predicting the daily streamflow time series collected from the upper Yangtze River and its tributaries. Artificial neuron network, generalized linear model, heteroscedastic GP, and regular LSTM models are also developed for comparison. Results indicated that the performance of the proposed model is satisfying. It improves prediction accuracy as well as provides an adaptive prediction interval, which is of great significance for water resources management and planning.
Similar content being viewed by others
References
Akram M, El C (2016) Sequence to sequence weather forecasting with long short-term memory recurrent neural networks. Int J Comput Appl 143(11):7–11
Al-Shedivat M, Wilson AG, Saatchi Y, Hu Z, Xing EP (2016) Learning scalable deep kernels with recurrent structure. arXiv preprint arXiv:161008936
Binois M, Gramacy RB, Ludkovski M (2018) Practical heteroscedastic gaussian process modeling for large simulation experiments. J Comput Graph Stat 27(4):808–821
Chai T, Draxler RR (2014) Root mean square error (RMSE) or mean absolute error (MAE)? Arguments against avoiding rmse in the literature. Geosci Model Dev 7(3):1247–1250
Charles SP, Wang QJ, Ahmad MUD, Hashmi D, Schepen A, Podger G, Robertson DE (2018) Seasonal streamflow forecasting in the upper indus basin of pakistan: an assessment of methods. Hydrol Earth Syst Sci 22(6):3533–3549
Chen L, Singh VP, Guo S, Zhou J, Zhang J (2015) Copula-based method for multisite monthly and daily streamflow simulation. J Hydrol 528:369–384
Chollet F et al (2015) Keras
Clark MP, Kavetski D, Fenicia F (2012) Reply to comment by K. J. Beven et al. on pursuing the method of multiple working hypotheses for hydrological modeling. Water Resour Res 48(11):802–808
De Vos NJ, Rientjes THM (2008) Multiobjective training of artificial neural networks for rainfall runoff modeling. Water Resour Res 44(8):134–143
El-Shafie A, Alsulami HM, Jahanbani H, Najah A (2013) Multi-lead ahead prediction model of reference evapotranspiration utilizing ann with ensemble procedure. Stoch Environ Res Risk Assess 27(6):1423–1440
Elman JL (1990) Finding structure in time. Cogn Sci 14(2):179–211
Fu W, Wang K, Li C, Tan J (2019a) Multi-step short-term wind speed forecasting approach based on multi-scale dominant ingredient chaotic analysis, improved hybrid GWO-SCA optimization and ELM. Energy Convers Manag 187:356–377
Fu W, Wang K, Zhang C, Tan J (2019b) A hybrid approach for measuring the vibrational trend of hydroelectric unit with enhanced multi-scale chaotic series analysis and optimized least squares support vector machine. Trans Inst Meas Control 41(15):4436–4449
Gers FA, Schmidhuber JA, Cummins FA (2002) Learning to forget: continual prediction with LSTM. In: Artificial neural networks, 1999. ICANN 99. Ninth international conference on, p 2451
Gibbs MN (1998) Bayesian gaussian processes for regression and classification. Ph.D. thesis, Citeseer
Gruet MA, Chandorkar M, Sicard A, Camporeale E (2018) Multiple-hour-ahead forecast of the dst index using a combination of long short-term memory neural network and gaussian process. Space Weather 16(11):1882–1896
Gupta HV, Kling H, Yilmaz KK, Martinez GF (2009) Decomposition of the mean squared error and nse performance criteria: implications for improving hydrological modelling. J Hydrol 377(1):80–91
Guttag J (2016) Introduction to computation and programming using python: with application to understanding data. MIT Press, Cambridge
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Hogue TS, Sorooshian S, Gupta H, Holz A, Braatz D (2000) A multistep automatic calibration scheme for river forecasting models. J Hydrometeorol 1(6):524–542
Ishak S, Kotha P, Alecsandru C, Student G (2003) Optimization of dynamic neural network performance for short-term traffic prediction. Transp Res Rec 1836(1):27–31
Karimi S, Shiri J, Kisi O, Xu T (2018) Forecasting daily streamflow values: assessing heuristic models. Hydrol Res 49(3):658–669
Kersting K, Plagemann C, Pfaff P, Burgard W (2007) Most likely heteroscedastic gaussian process regression. In: Proceedings of the 24th international conference on Machine learning, ACM, pp 393–400
Kisi O, Cimen M (2011) A wavelet-support vector machine conjunction model for monthly streamflow forecasting. J Hydrol 399(1):132–140
Kratzert F, Klotz D, Brenner C, Schulz K, Herrnegger M (2018) Rainfall-runoff modelling using long short-term memory (LSTM) networks. Hydrol Earth Syst Sci 22(11):6005–6022
Lázaro-Gredilla M, Titsias MK (2011) Variational heteroscedastic gaussian process regression. In: ICML, pp 841–848
Lei Y, Zhou J, Zeng X, Guo J, Zhang X (2014) Multi-objective optimization for construction of prediction interval of hydrological models based on ensemble simulations. J Hydrol 519:925–933
Lu X, Wang X, Zhang L, Zhang T, Yang C, Song X, Yang Q (2018) Improving forecasting accuracy of river flow using gene expression programming based on wavelet decomposition and de-noising. Hydrol Res 49(3):711–723
Moradkhani H (2015) Statistical-dynamical drought forecast within bayesian networks and data assimilation: how to quantify drought recovery. In: EGU general assembly conference abstracts, vol 17
Papacharalampous G, Tyralis H, Koutsoyiannis D (2018a) One-step ahead forecasting of geophysical processes within a purely statistical framework. Geosci Lett 5(1):12
Papacharalampous G, Tyralis H, Koutsoyiannis D (2018b) Predictability of monthly temperature and precipitation using automatic time series forecasting methods. Acta Geophys 66(4):807–831
Papacharalampous G, Tyralis H, Koutsoyiannis D (2018c) Univariate time series forecasting of temperature and precipitation with a focus on machine learning algorithms: a multiple-case study from greece. Water Resour Manag 32(15):5207–5239
Papacharalampous G, Tyralis H, Koutsoyiannis D (2019) Comparison of stochastic and machine learning methods for multi-step ahead forecasting of hydrological processes. Stoch Environ Res Risk Assess 33(2):1–34
Rasmussen CE (2003) Gaussian processes in machine learning. In: Summer school on machine learning, Springer, Berlin, pp 63–71
Rasmussen CE (2004) Gaussian processes in machine learning. In: Advanced lectures on machine learning, Springer, Berlin, pp 63–71
Shen C (2018) A transdisciplinary review of deep learning research and its relevance for water resources scientists. Water Resour Res 54(11):8558–8593
Shen C, Laloy E, Elshorbagy A, Albert A, Bales J, Chang FJ, Ganguly S, Hsu KL, Kifer D, Fang Z, Fang K, Li D, Li X, Tsai WP (2018) Hess opinions: incubating deep-learning-powered hydrologic science advancesas a community. Hydrol Earth Syst Sci 22(11):5639–5656
Siqueira H, Boccato L, Luna I, Attux R, Lyra C (2018) Performance analysis of unorganized machines in streamflow forecasting of brazilian plants. Appl Soft Comput 68:494–506
Sun AY, Wang D, Xu X (2014) Monthly streamflow forecasting using gaussian process regression. J Hydrol 511:72–81
Tolvanen V, Jylänki P, Vehtari A (2014) Expectation propagation for nonstationary heteroscedastic gaussian process regression. In: Machine learning for signal processing (MLSP), 2014 IEEE international workshop on, IEEE, pp 1–6
Tyralis H, Koutsoyiannis D (2011) Simultaneous estimation of the parameters of the Hurst–Kolmogorov stochastic process. Stoch Environ Res Risk Assess 25(1):21–33
Tyralis H, Koutsoyiannis D (2014) A bayesian statistical model for deriving the predictive distribution of hydroclimatic variables. Clim Dyn 42(11–12):2867–2883
Wang W, Jin J, Li Y (2009) Prediction of inflow at three gorges dam in Yangtze River with wavelet network model. Water Resour Manag 23(13):2791–2803
Wilson AG, Hu Z, Salakhutdinov R, Xing EP (2016) Deep kernel learning. In: Artificial intelligence and statistics, pp 370–378
Wu W, Chen K, Qiao Y, Lu Z (2016) Probabilistic short-term wind power forecasting based on deep neural networks. In: 2016 International conference on probabilistic methods applied to power systems (PMAPS), IEEE, pp 1–8
Yaseen ZM, Sulaiman SO, Deo RC, Chau KW (2019) An enhanced extreme learning machine model for river flow forecasting: state-of-the-art, practical applications in water resource engineering area and future research direction. J Hydrol 569:387–408
Yuan X, Abouelenien M (2015) A multi-class boosting method for learning from imbalanced data. Int J Granul Comput Rough Sets Intell Syst 4(1):13
Yuan X, Sarma V (2010) Automatic urban water-body detection and segmentation from sparse alsm data via spatially constrained model-driven clustering. IEEE Geosci Remote Sens Lett 8(1):73–77
Yuan X, Chen C, Lei X, Yuan Y, Adnan RM (2018a) Monthly runoff forecasting based on lstm-alo model. Stoch Environ Res Risk Assess 32(8):2199–2212
Yuan X, Xie L, Abouelenien M (2018b) A regularized ensemble framework of deep learning for cancer detection from multi-class, imbalanced training data. Pattern Recognit 77:160–172
Zhang J, Zhu Y, Zhang X, Ye M, Yang J (2018) Developing a long short-term memory (LSTM) based model for predicting water table depth in agricultural areas. J Hydrol 561:918–929
Zhang J, Yan J, Infield D, Liu Y, Lien F (2019) Short-term forecasting and uncertainty analysis of wind turbine power based on long short-term memory network and Gaussian mixture model. Appl Energy 241:229–244
Zhu S, Luo X, Xu Z, Ye L (2018) Seasonal streamflow forecasts using mixture-kernel GPR and advanced methods of input variable selection. Hydrol Res 50(1):200–214
Acknowledgements
This work is supported by the National Natural Science Foundation of China (51809242), the Fundamental Research Funds for the Central Universities, China University of Geosciences (Wuhan) (G1323541875, G1323519436)
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhu, S., Luo, X., Yuan, X. et al. An improved long short-term memory network for streamflow forecasting in the upper Yangtze River. Stoch Environ Res Risk Assess 34, 1313–1329 (2020). https://doi.org/10.1007/s00477-020-01766-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00477-020-01766-4