Abstract
Runoff prediction with high accuracy is vital to protect people’s properties and lives. In this study, a Short-Term Streamflow Forecasting Framework combining Long Short-Term Memory (LSTM) with Self-Organizing Map (SOM) is proposed, which clusters hydro-meteorological factors through SOM network to cluster the streamflow into several flow modes with different physical characteristics, converting complex global modeling into local modeling. The framework also considers the temporal characteristics of the streamflow, so the data of different flow modes are fitted with multiple LSTM separately. Besides, Random Forest is used in the framework for feature selection. The model is used for streamflow forecasting 1 to 3 day(s) ahead in the Qingxi River Basin in China, and the performance is compared with those of SVR, ANN, LSTM, SOM-SVR, SOM-ANN, FCM-LSTM, and K-means-LSTM. In addition, SOM-LSTM with three and four clusters were compared. The results show that (1) the sub-processes obtained by SOM clustering show some physical characteristics consistent with the actual process. (2) SOM-LSTM significantly improves the prediction accuracy of short-term streamflow forecasting. (3) All the integrated models using different clustering methods can improve the prediction accuracy of a single LSTM, and SOM-LSTM has the most reliable prediction when the lead time is 1–2 day(s). (4) The number of clusters being set to 4 is more suitable for the proposed prediction framework. The overall results show that the SOM-LSTM provides an efficient method for short-term streamflow forecasting.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Afan HA, Allawi MF, El-Shafie A, Yaseen ZM et al (2020) Input attributes optimization using the feasibility of genetic nature inspired algorithm: application of river flow forecasting. Sci Rep. https://doi.org/10.1038/s41598-020-61355-x
Ahmed AAM, Deo RC, Ghahramani A, Raj N et al (2021) LSTM integrated with boruta-random forest optimiser for soil moisture estimation under RCP4.5 and RCP8.5 global warming scenarios. Stochastic Environ Res Risk Assess 35(9):1851–1881. https://doi.org/10.1007/s00477-021-01969-3
Avand M, Janizadeh S, Naghibi SA, Pourghasemi HR, Blaschke TJW (2019) A comparative assessment of random forest and k-nearest neighbor classifiers for gully erosion susceptibility mapping. Water 11(10):2076
Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166. https://doi.org/10.1109/72.279181
Bharti B, Pandey A, Tripathi SK, Kumar D (2017) Modelling of runoff and sediment yield using ANN, LS-SVR, REPTree and M5 models. Hydrol Res 48(6):1489–1507. https://doi.org/10.2166/nh.2017.153
Chang MJ, Lin GF, Lee FZ, Wang YC et al (2020) Outflow sediment concentration forecasting by integrating machine learning approaches and time series analysis in reservoir desilting operation. Stochastic Environ Res Risk Assess 34(6):849–866. https://doi.org/10.1007/s00477-020-01802-3
Chu H, Wei J, Wu W, Jiang Y et al (2021) A classification-based deep belief networks model framework for daily streamflow forecasting. J Hydrol. https://doi.org/10.1016/j.jhydrol.2021.125967
Du K, Zhao Y, Lei J (2017) The incorrect usage of singular spectral analysis and discrete wavelet transform in hybrid models to predict hydrological time series. J Hydrol 552:44–51. https://doi.org/10.1016/j.jhydrol.2017.06.019
Fang H-T, Jhong B-C, Tan Y-C, Ke K-Y, Chuang M-H (2019) A two-stage approach integrating SOM- and MOGA-SVM-based algorithms to forecast spatial-temporal groundwater level with meteorological factors. Water Resour Manag 33(2):797–818. https://doi.org/10.1007/s11269-018-2143-x
Frame JM, Kratzert F, Klotz D, Gauch M et al (2022) Deep learning rainfall–runoff predictions of extreme events. Hydrol Earth Syst Sci 26(13):3377–3392. https://doi.org/10.5194/hess-26-3377-2022
Giaccone E, Oriani F, Tonini M, Lambiel C, Mariethoz G (2022) Using data-driven algorithms for semi-automated geomorphological mapping. Stochastic Environ Res Risk Assess. https://doi.org/10.1007/s00477-021-02062-5
Gu P, Wu Y, Liu G, Xia C et al (2021) Application of meteorological element combination-driven SWAT model based on meteorological datasets in alpine basin. Water Supply. https://doi.org/10.2166/ws.2021.397
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Huang SS, Dong QJ, Zhang X, Deng WS (2021) Catchment natural driving factors and prediction of baseflow index for continental United States based on random forest technique. Stochastic Environ Res Risk Assess 35(12):2567–2581. https://doi.org/10.1007/s00477-021-02057-2
Huang X, Li Y, Tian Z, Ye Q et al (2021) Evaluation of short-term streamflow prediction methods in Urban river basins. Phys Chem Earth. https://doi.org/10.1016/j.pce.2021b.103027
Hunt KMR, Matthews GR, Pappenberger F, Prudhomme C (2022) Using a long short-term memory (LSTM) neural network to boost river streamflow forecasts over the western United States. Hydrol Earth Syst Sci Discuss 2022:1–30. https://doi.org/10.5194/hess-2022-53
Jambak MI, Jambak AII (2019) Comparison of dimensional reduction using the Singular value decomposition algorithm and the self organizing map algorithm in clustering result of text documents. IOP Conf Ser Mater Sci Eng 551:012046–012046. https://doi.org/10.1088/1757-899x/551/1/012046
Kim SE, Seo IW (2015) Artificial neural network ensemble modeling with exploratory factor analysis for streamflow forecasting. J Hydroinf 17(4):614–639. https://doi.org/10.2166/hydro.2015.033
Lees T, Reece S, Kratzert F, Klotz D et al (2022) Hydrological concept formation inside long short-term memory (LSTM) networks. Hydrol Earth Syst Sci 26(12):3079–3101. https://doi.org/10.5194/hess-26-3079-2022
Li Y, Liang Z, Hu Y, Li B et al (2020) A multi-model integration method for monthly streamflow prediction: modified stacking ensemble strategy. J Hydroinf 22(2):310–326. https://doi.org/10.2166/hydro.2019.066
Lin G-F, Wang T-C, Chen L-H (2016) A forecasting approach combining self-organizing map with support vector regression for reservoir inflow during typhoon periods. Adv Meteorol. https://doi.org/10.1155/2016/7575126
Liu X, Zhang L, She D, Chen J et al (2022) Postprocessing of hydrometeorological ensemble forecasts based on multisource precipitation in Ganjiang River basin. China J Hydrol. https://doi.org/10.1016/j.jhydrol.2021.127323
Luo X, Yuan X, Zhu S, Xu Z et al (2019) A hybrid support vector regression framework for streamflow forecast. J Hydrol 568:184–193. https://doi.org/10.1016/j.jhydrol.2018.10.064
Meng E, Huang S, Huang Q, Fang W et al (2021) A Hybrid VMD-SVM model for practical streamflow prediction using an innovative input selection framework. Water Resour Manag 35(4):1321–1337. https://doi.org/10.1007/s11269-021-02786-7
Mwale FD, Adeloye AJ, Rustum R (2014) Application of self-organising maps and multi-layer perceptron-artificial neural networks for streamflow and water level forecasting in data-poor catchments: the case of the lower shire floodplain. Malawi Hydrol Res 45(6):838–854. https://doi.org/10.2166/nh.2014.168
Nevo S, Morin E, Gerzi Rosenthal A, Metzger A et al (2022) Flood forecasting with machine learning models in an operational framework. Hydrol Earth Syst Sci 26(15):4013–4032. https://doi.org/10.5194/hess-26-4013-2022
Ni L, Wang D, Wu J, Wang Y et al (2020) Streamflow forecasting using extreme gradient boosting model coupled with Gaussian mixture model. J Hydrol. https://doi.org/10.1016/j.jhydrol.2020.124901
Nourani V (2017) An Emotional ANN (EANN) approach to modeling rainfall-runoff process. J Hydrol 544:267–277. https://doi.org/10.1016/j.jhydrol.2016.11.033
Osman A, Afan HA, Allawi MF, Jaafar O et al (2020) Adaptive fast orthogonal search (FOS) algorithm for forecasting streamflow. J Hydrol. https://doi.org/10.1016/j.jhydrol.2020.124896
Peng T, Zhou J, Zhang C, Fu W (2017) Streamflow forecasting using empirical wavelet Transform and artificial neural networks. Water. https://doi.org/10.3390/w9060406
Rezaei F, Ahmadzadeh MR, Safavi HR (2017) SOM-DRASTIC: using self-organizing map for evaluating groundwater potential to pollution. Stochastic Environ Res Risk Assess 31(8):1941–1956. https://doi.org/10.1007/s00477-016-1334-3
Song C, Yao L, Hua C, Ni Q (2021) A novel hybrid model for water quality prediction based on synchrosqueezed wavelet transform technique and improved long short-term memory. J Hydrol. https://doi.org/10.1016/j.jhydrol.2021.126879
Toth E (2009) Classification of hydro-meteorological conditions and multiple artificial neural networks for streamflow forecasting. Hydrol Earth Syst Sci 13(9):1555–1566. https://doi.org/10.5194/hess-13-1555-2009
Tyralis H, Papacharalampous G, Langousis A (2021) Chapter 2-streamflow forecasting at large time scales using statistical models. In: Sharma P, Machiwal D (eds) Advances in streamflow forecasting. Elsevier, Amsterdam, pp 51–86. https://doi.org/10.1016/B978-0-12-820673-7.00004-4
Wang H, Sun J, Sun J, Wang J (2017) Using random forests to select optimal input variables for short-term wind speed forecasting models. Energies. https://doi.org/10.3390/en10101522
Wen X, Feng Q, Deo RC, Wu M et al (2019) Two-phase extreme learning machines integrated with the complete ensemble empirical mode decomposition with adaptive noise algorithm for multi-scale runoff prediction problems. J Hydrol 570:167–184. https://doi.org/10.1016/j.jhydrol.2018.12.060
Williams RN, de Souza PA, Jones EM (2014) Analysing coastal ocean model outputs using competitive-learning pattern recognition techniques. Environ Modell Softw 57:165–176. https://doi.org/10.1016/j.envsoft.2014.03.001
Wu J, Miao C, Yang T, Duan Q, Zhang X (2018) Modeling streamflow and sediment responses to climate change and human activities in the Yanhe River. China Hydrol Res 49(1):150–162. https://doi.org/10.2166/nh.2017.168
Xie T, Zhang G, Hou J, Xie J et al (2019) Hybrid forecasting model for non-stationary daily runoff series: a case study in the han river basin. China J Hydrol. https://doi.org/10.1016/j.jhydrol.2019.123915
Yajima H, Derot J (2018) Application of the random forest model for chlorophyll-a forecasts in fresh and brackish water bodies in Japan, using multivariate long-term databases. J Hydroinf 20(1):206–220. https://doi.org/10.2166/hydro.2017.010
Yin W, Fan Z, Tangdamrongsub N, Hu L, Zhang M (2021) Comparison of physical and data-driven models to forecast groundwater level changes with the inclusion of grace-a case study over the state of victoria, Australia. J Hydrol. https://doi.org/10.1016/j.jhydrol.2021.126735
Yu X, Wang Y, Wu L, Chen G et al (2020) Comparison of support vector regression and extreme gradient boosting for decomposition-based data-driven 10-day streamflow forecasting. J Hydrol. https://doi.org/10.1016/j.jhydrol.2019.124293
Zhang L, Zhuang Y, Du Y (2015a) Application of the self-organizing map for aqueous phosphorus modeling and monitoring in a natural freshwater wetland: a case study. Fresenius Environ Bull 24(11A):3865–3870
Zhang X, Peng Y, Zhang C, Wang B (2015) Are hybrid models integrated with data preprocessing techniques suitable for monthly streamflow forecasting? Some experiment evidences. J Hydrol 530:137–152. https://doi.org/10.1016/j.jhydrol.2015b.09.047
Zhou J, Wang Y, Xiao F, Wang Y, Sun L (2018) Water quality prediction method based on IGRA and LSTM. Water. https://doi.org/10.3390/w10091148
Zhu S, Luo X, Xu Z, Ye L (2019) Seasonal streamflow forecasts using mixture-kernel GPR and advanced methods of input variable selection. Hydrol Res 50(1):200–214. https://doi.org/10.2166/nh.2018.023
Funding
This work was supported by Sichuan Science and Technology Program (NO: 2021YFG0121) and Sichuan Science and Technology Program (NO: 2022ZHCG0042).
Author information
Authors and Affiliations
Contributions
Liu Shuqi conducted the main experiments and wrote the main manuscript. Xinzhi Zhou participated in the design of the proposed framework and checked the manuscript. Bo Li is responsible for data acquisition and proofreading. Xin He is responsible for the acquisition of the financial support for the project. Yuexin Zhang and Yi Fu prepared figures 2-4. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Conflict of interests
The authors declare that they don’t have any conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liu, S., Zhou, X., Li, B. et al. Improving short-term streamflow forecasting by flow mode clustering. Stoch Environ Res Risk Assess 37, 1799–1819 (2023). https://doi.org/10.1007/s00477-022-02367-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00477-022-02367-z