Abstract
Deep learning models have a high application value in runoff forecasting, but their prediction mechanism is difficult to interpret and their computational cost is high when dealing with complex hydrological relationships, limiting their feasibility in hydrological process mechanism analysis. To address these concerns, the paper first introduces an attention mechanism (AM) for building a long short-term memory network (LSTM) model with AM in the hidden layer (AM-LSTM). The AM-LSTM model employs attention layers to improve information extraction from hidden layers, resulting in a more accurate representation of the relationships between runoff-related elements. Furthermore, in the hidden layers of the AM-LSTM model, interpretable spatiotemporal attention units are established, which not only improves the model's prediction accuracy but also provides interpretability to the forecasting process. Furthermore, parallelization techniques are used in the paper to address the issue of model runtime cost. Simultaneously, to address the accuracy degradation caused by parallelization, the paper employs wavelet denoising (WD) techniques and builds the WD-AM-LSTM model. This accomplishment enables the runoff forecasting model to predict runoff in real time and with high accuracy. Based on validation using ten-day runoff data from the Huanren Reservoir in the Hun River's middle reaches, the results show that, with two layers and an eight-batch size, the AM-LSTM model outperforms the LSTM model in capturing spatiotemporal runoff features. During the model testing phase, the AM-LSTM model improves the MAE, RMSE, and NSE performance metrics by 8.46%, 13%, and 3.82%, respectively. The WD-AM-LSTM model effectively mitigates the noise impact caused by data parallelization under the conditions of two layers and a batch size of 512, achieving the same level of prediction performance while reducing computational cost by 92.01%. By incorporating attention mechanisms and wavelet denoising techniques, this study obtains high-speed and accurate predictions with interpretable results. It expands the deep learning models' applicability in ten-day runoff forecasting work.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
Bahdanau D, Cho K, Bengio Y (2014) Neural Machine Translation by Jointly Learning to Align and Translate.arXiv:1409.0473. https://arxiv.org/abs/1409.0473
Ćalasan M, Abdel Aleem SHE, Zobaa AF (2020) On the root mean square error (RMSE) calculation for parameter estimation of photovoltaic models: a novel exact analytical solution based on Lambert W function. Energy Convers Manag 210:112716. https://doi.org/10.1016/j.enconman.2020.112716
Chadalawada J, Babovic V (2017) Review and comparison of performance indices for automatic model induction. J Hydroinf 21:13–31. https://doi.org/10.2166/hydro.2017.078
Chen X et al (2020) The importance of short lag-time in the runoff forecasting model based on long short-term memory. J Hydrol 589:125359. https://doi.org/10.1016/j.jhydrol.2020.125359
Dean J et al (2012) Large Scale Distributed Deep Networks. Advances in neural information processing systems. https://doi.org/10.5555/2999134.2999271
Ditthakit P, Pinthong S, Salaeh N, Weekaew J, Thanh Tran T, Bao Pham Q (2023) Comparative study of machine learning methods and GR2M model for monthly runoff prediction. Ain Shams Eng J 14:101941. https://doi.org/10.1016/j.asej.2022.101941
Elman JL (1990) Finding structure in time. Cogn Sci 14:179–211. https://doi.org/10.1207/s15516709cog1402_1
Frame JM et al (2022) Deep learning rainfall–runoff predictions of extreme events. Hydrol Earth Syst Sci 26:3377–3392. https://doi.org/10.5194/hess-26-3377-2022
Gao S, Zhang S, Huang Y, Han J, Luo H, Zhang Y, Wang G (2022) A new seq2seq architecture for hourly runoff prediction using historical rainfall and runoff as input. J Hydrol 612:128099. https://doi.org/10.1016/j.jhydrol.2022.128099
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Hu C, Wu Q, Li H, Jian S, Li N, Lou Z (2018) Deep learning with a long short-term memory networks approach for rainfall-runoff simulation. Water 10:1543. https://doi.org/10.3390/w10111543
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. Paper presented at the Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, Lille, France
Jiang S, Zheng Y, Wang C, Babovic V (2022) Uncovering flooding mechanisms across the contiguous United States through interpretive deep learning on representative catchments. Water Resour Res 58:e2021WR030185. https://doi.org/10.1029/2021WR030185
Jin W, Ma J, Chen X, Yan G (2022) Changes in the surface water nitrogen content in the upper Hun River Basin, Northeast China. Comput Intel Neurosci 2022:4175218. https://doi.org/10.1155/2022/4175218
Kingma DP, Ba J (2014) Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980. https://arxiv.org/abs/1412.6980
Kumar PS, Praveen TV, Prasad MA (2016) Artificial neural network model for rainfall-runoff -a case study. Int J Hybrid Inf Technol 9:263–272. https://doi.org/10.14257/ijhit.2016.9.3.24
Li B-J, Sun G-L, Liu Y, Wang W-C, Huang X-D (2022a) Monthly runoff forecasting using variational mode decomposition coupled with gray wolf optimizer-based long short-term memory neural networks. Water Resour Manage 36:2095–2115. https://doi.org/10.1007/s11269-022-03133-0
Li G, Li F, Xu C, Fang X (2022b) A spatial-temporal layer-wise relevance propagation method for improving interpretability and prediction accuracy of LSTM building energy prediction. Energy Build 271:112317. https://doi.org/10.1016/j.enbuild.2022.112317
Li H, Yang Z, Hong X, Zhao Z, Chen J, Shi Y, Pan J (2022c) DnSwin: Toward real-world denoising via a continuous Wavelet Sliding Transformer. Knowl-Based Syst 255:109815. https://doi.org/10.1016/j.knosys.2022.109815
Liu G, Tang Z, Qin H, Liu S, Shen Q, Qu Y, Zhou J (2022) Short-term runoff prediction using deep learning multi-dimensional ensemble method. J Hydrol 609:127762. https://doi.org/10.1016/j.jhydrol.2022.127762
Mallat SG (1989) A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans Pattern Anal Mach Intell 11:674–693. https://doi.org/10.1109/34.192463
Min X, Hao B, Sheng Y, Huang Y, Qin J (2023) Transfer performance of gated recurrent unit model for runoff prediction based on the comprehensive spatiotemporal similarity of catchments. J Environ Manag 330:117182. https://doi.org/10.1016/j.jenvman.2022.117182
Noh S-H (2021) Analysis of gradient vanishing of RNNs and performance comparison. Information 12. https://doi.org/10.3390/info12110442
Paudel D, de Wit A, Boogaard H, Marcos D, Osinga S, Athanasiadis IN (2023) Interpretability of deep learning models for crop yield forecasting. Comput Electron Agric 206:107663. https://doi.org/10.1016/j.compag.2023.107663
Ruiming F (2018) Wavelet based relevance vector machine model for monthly runoff prediction. Water Qual Res J 54:134–141. https://doi.org/10.2166/wcc.2018.196
Shi W, Wang M, Li D, Li X, Sun M (2023) An improved method that incorporates the estimated runoff for peak discharge prediction on the Chinese Loess Plateau. Int Soil Water Conserv Res 11:290–300. https://doi.org/10.1016/j.iswcr.2022.09.001
Tian C, Zheng M, Zuo W, Zhang B, Zhang Y, Zhang D (2023) Multi-stage image denoising with the wavelet transform. Pattern Recogn 134:109050. https://doi.org/10.1016/j.patcog.2022.109050
Visintainer MRM, Alminhana GW, Braun AL (2023) Numerical simulation of wind-structure-soil interaction effects on the CAARC tall building model using hybrid CUDA-OpenMP parallelization. J Build Eng 73:106669. https://doi.org/10.1016/j.jobe.2023.106669
Wang W-C, Du Y-J, Chau K-W, Xu D-M, Liu C-J, Ma Q (2021) An ensemble hybrid forecasting model for annual runoff based on sample entropy, secondary decomposition, and long short-term memory neural network. Water Resour Manag 35:4695–4726. https://doi.org/10.1007/s11269-021-02920-5
Wang W-C, Cheng Q, Chau K-W, Hu H, Zang H-F, Xu D-M (2023) An enhanced monthly runoff time series prediction using extreme learning machine optimized by salp swarm algorithm based on time varying filtering based empirical mode decomposition. J Hydrol 620:129460. https://doi.org/10.1016/j.jhydrol.2023.129460
Wi S, Steinschneider S (2022) Assessing the physical realism of deep learning hydrologic model projections under climate change. Water Resour Res 58:e2022WR032123. https://doi.org/10.1029/2022WR032123
Xie Y, Sun W, Ren M, Chen S, Huang Z, Pan X (2023) Stacking ensemble learning models for daily runoff prediction using 1D and 2D CNNs. Expert Syst Appl 217:119469. https://doi.org/10.1016/j.eswa.2022.119469
Yang X, Maihemuti B, Simayi Z, Saydi M, Na L (2022) Prediction of glacially derived runoff in the Muzati River watershed based on the PSO-LSTM model. Water 14. https://doi.org/10.3390/w14132018
Yin HL, Guo ZL, Zhang XW, Chen JJ, Zhang YN (2021) Runoff predictions in ungauged basins using sequence-to-sequence models. J Hydrol 603:126975. https://doi.org/10.1016/j.jhydrol.2021.126975
Yue Z, Ai P, Yuan D, Xiong C (2022) Ensemble approach for mid-long term runoff forecasting using hybrid algorithms. J Ambient Intell Humaniz Comput 13:5103–5122. https://doi.org/10.1007/s12652-020-02345-9
Zhang F, Kang Y, Cheng X, Chen P, Song S (2022) A hybrid model integrating Elman neural network with variational mode decomposition and box-cox transformation for monthly runoff time series prediction. Water Resour Manag 36:3673–3697. https://doi.org/10.1007/s11269-022-03220-2
Funding
This work was supported by Special project for collaborative innovation of science and technology in 2021 (No: 202121206) and Henan Province University Scientific and Technological Innovation Team (No: 18IRTSTHN009).
Author information
Authors and Affiliations
Contributions
All authors contributed to the study Conceptualization and Methodology. Writing—original draft preparation, data collection and analysis were performed by Yi-yang Wang, Wen-chuan Wang, Dong-mei Xu, Yan-wei Zhao, Hong-fei Zang. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Consent for publication
All authors approved the final manuscript and the submission to this journal.
Competing interests
The authors declare no competing interests.
Additional information
Communicated by: H. Babaie
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, Yy., Wang, Wc., Xu, Dm. et al. A compound approach for ten-day runoff prediction by coupling wavelet denoising, attention mechanism, and LSTM based on GPU parallel acceleration technology. Earth Sci Inform 17, 1281–1299 (2024). https://doi.org/10.1007/s12145-023-01212-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12145-023-01212-3