Abstract
Runoff forecasting is crucial for planning and managing water resources. As hydrological data becomes more available, more data-driven models are being employed to enhance the effectiveness of runoff forecasting. To accurately and quantitatively assess the whole runoff forecast process, from model construction to application, a whole life cycle forecasting evaluation index system was established in this study to evaluate the data, factors, sample integrity, model construction, and forecast result. The analysis included quantitative evaluation criteria to comprehensively consider data quality, forecasting factor characterization, sample representativeness, model generalization, and result quality. An evaluation of forecast results from 7 river basins and their 85 hydrological stations in China showed that the proposed index system could accurately reflect the performance of the forecasting process. The overall performance of the forecasting model and process can be evaluated quantitatively based on the Euclidean distance, and the pathways to improve the forecasting effectiveness of the model can be identified based on the evaluation results. The validity of the proposed index is also experimentally demonstrated. The proposed index system can be applied to the evaluation of data-driven forecasting processes in various fields.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The authors have restrictions on sharing data publicly.
References
Albenis A, Daniel GC, Jos CFA, Yoel MG (2022) Improving monthly rainfall forecast in a watershed by combining neural networks and autoregressive models. Environ Process 9:53. https://doi.org/10.1007/s40710-022-00602-x
Bournas A, Baltas E (2021) Increasing the efficiency of the Sacramento model on event basis in a mountainous river basin. Environ Process 8:943–958. https://doi.org/10.1007/s40710-021-00504-4
Clark M, Bierkens M, Samaniego L et al (2017) The evolution of process-based hydrologic models: historical challenges and the collective quest for physical realism. Hydrol Earth Syst Sci 21:3427–3440. https://doi.org/10.5194/hess-21-3427-2017
Cui JS, Wu YX, Gang H et al (2018) Evaluation on light sources for electric power emergency recovery system based on grille method and maximum information coefficient method. Energy Procedia 145:181–186. https://doi.org/10.1016/j.egypro.2018.04.031
Davletgaliev SK, Alimkulov SK, Talipova EK (2020) The possibility to applying simulated series for compile scenario forecasting river runoff. Environ Earth Sci 79:397. https://doi.org/10.1007/s12665-020-09138-5
Fan YR, Huang G, Li YP et al (2016) Probabilistic Prediction for monthly streamflow through coupling stepwise cluster analysis and quantile regression methods. Water Resour Manag 30:5313–5331. https://doi.org/10.1007/s11269-016-1489-1
Gupta H, Kling H, Yilmaz KK, Martinez GF (2009) Decomposition of the mean squared error and NSE performance criteria: implications for improving hydrological modelling. J Hydrol 377:80–91. https://doi.org/10.1016/j.jhydrol.2009.08.003
Hu Z, Chen X, Zhou Q et al (2019) DISO: A rethink of Taylor diagram. Int J Climatol 39:2825–2832. https://doi.org/10.1002/joc.5972
Ioniță I, Niacșu L, Poesen J, Fullen MA (2022) Medium-term development of discontinuous gullies. Geomorphology 398:108024. https://doi.org/10.1016/j.geomorph.2021.108024
Kasiviswanathan KS, He J, Tay J-H, Sudheer KP (2019) Enhancement of model reliability by integrating prediction interval optimization into hydrogeological modeling. Water Resour Manag 33:229–243. https://doi.org/10.1007/s11269-018-2099-x
Lian Y, Luo J, Wang J et al (2022) Climate-driven model based on long short-term memory and Bayesian optimization for multi-day-ahead daily streamflow forecasting. Water Resour Manag 36:21–37. https://doi.org/10.1007/s11269-021-03002-2
Nearing GS, Kratzert F, Sampson AK et al (2021) What role does hydrological science play in the age of machine learning? Water Resour Res 57:e2020WR028091. https://doi.org/10.1029/2020WR028091
Nourani V, Baghanam AH, Adamowski J, Gebremichael M (2013) Using self-organizing maps and wavelet transforms for space–time pre-processing of satellite precipitation and runoff data in neural network based rainfall–runoff modeling. J Hydrol 476:228–243. https://doi.org/10.1016/j.jhydrol.2012.10.054
Razavi S (2021) Deep learning, explained: Fundamentals, explainability, and bridgeability to process-based modelling. Environ Model Softw 144:105159. https://doi.org/10.1016/j.envsoft.2021.105159
Reonaldo LMD, Herwindiati DE (2020) The simple boxplot method for an effective prediction. IOP Conf Ser Mater Sci Eng 1007:012141. https://doi.org/10.1088/1757-899X/1007/1/012141
Sevgen S, Şahin S, Samli R (2022) Modeling of sunflower oil treated with lemon balm (Melissa officinalis): artificial neural networks versus multiple linear regression. J Food Process Preserv 46:e16650. https://doi.org/10.1111/jfpp.16650
Uncuoglu E, Citakoglu H, Latifoglu L et al (2022) Comparison of neural network, Gaussian regression, support vector machine, long short-term memory, multi-gene genetic programming, and M5 Trees methods for solving civil engineering problems. Appl Soft Comput 129:109623. https://doi.org/10.1016/j.asoc.2022.109623
Xu W, Chen J, Zhang XJ et al (2022) A framework of integrating heterogeneous data sources for monthly streamflow prediction using a state-of-the-art deep learning model. J Hydrol 614:128599. https://doi.org/10.1016/j.jhydrol.2022.128599
Young CC, Liu WC, Chung CE (2015) Genetic algorithm and fuzzy neural networks combined with the hydrological modeling system for forecasting watershed runoff discharge. Neural Comput Appl 26:1631–1643. https://doi.org/10.1007/s00521-015-1832-0
Zhao W, Li J, Zhao J et al (2020) Research on evaporation duct height prediction based on back propagation neural network. IET Microw Antennas Propag 14:1547–1554. https://doi.org/10.1049/iet-map.2019.1136
Zhou L, Liu B, Zhao Y, Jiang Z (2019) Application research of grey fuzzy evaluation method in enterprise product reputation evaluation. Procedia CIRP 83:759–766. https://doi.org/10.1016/j.procir.2019.05.014
Zhou Q, Chen D, Hu Z, Chen X (2021) Decompositions of Taylor diagram and DISO performance criteria. Int J Climatol 41:5726–5732. https://doi.org/10.1002/joc.7149
Funding
This work was supported by the national natural science foundation of China (No. U2340211), the national key research and development program of China (No. 2021YFC3200405), and China Yangtze Power Co.,Ltd. (No. 2423020043, No. Z242302026).
Author information
Authors and Affiliations
Contributions
All authors have equal contribution.
Corresponding author
Ethics declarations
Ethics Approval
Not required as no animal/human was involved in this study.
Consent to Participate
The authors agree to participate in any survey or feedback tasks.
Consent to Publish
The authors agree to publish this manuscript to the journal publisher.
Competing Interests
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yuan, X., Hu, W., Wang, C. et al. A New Whole Life Cycle Index System for Evaluation of Runoff Forecasting. Water Resour Manage 38, 1419–1435 (2024). https://doi.org/10.1007/s11269-023-03728-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11269-023-03728-1