Abstract
The joint optimal operation of cascade reservoir system can greatly improve the utilization of water resources. However, the complex high-dimensional and non-linear features and calculated costs often hinder the refined operation and management of reservoirs. Recently, the local parallel computing has become an effective way to alleviate the "curse of dimensionality". Current local parallel computing has hardware limitations, which is difficult to adapt to large-scale computing. This study proposes a novel parallel dynamic programming algorithm based on Spark (PDPoS) via cloud computing. The simulation experiments are carried out for a comparative analysis of the solution efficiency, influence factors and stability of cloud computing. The results are as follows: (1) The efficiency of the cloud-based PDPoS is related to some factors; the number of CPU cores is the main influencing factor, followed by the operator, and the architecture has the least influence. (2) The runtime variance of cloud computing is 2.03, indicating cloud computing has high stability. (3) Under the same configuration (i.e., CPU and memory), the runtime of cloud computing is 41.5% ~ 110.3% longer than that of physical machines. However, cloud computing has rich resources, good scalability, and good portability of online operations, which is an attractive alternative for optimal operation of large-scale reservoir system.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
All data used during in this study are available from the corresponding author by request.
References
Avila L, Miriam RM, Kaviski E (2020) Probabilistic long-term reservoir operation employing copulas and implicit stochastic optimization. Stoch Env Res Risk A 34(7):931–947. https://doi.org/10.1007/s00477-020-01826-9
Chen D, Leon AS, Gibson NL et al (2016) Dimension reduction of decision variables for multireservoir operation: a spectral optimization model. Water Resour Res 52(1):36–51. https://doi.org/10.1002/2015WR017756
Cheng CT, Wang S, Chau KW et al (2014) Parallel discrete differential dynamic programming for multireservoir operation. Environ Modell Softw 57:152–164. https://doi.org/10.1016/j.envsoft.2014.02.018
Dean J, Ghemawat S (2010) MapReduce: a flexible data processing tool. Commun ACM 53(1):72–77. https://doi.org/10.1145/1629175.1629198
Emami M, Nazif S, Mousavi SF et al (2021) A hybrid constrained coral reefs optimization algorithm with machine learning for optimizing multi-reservoir systems operation. J Environ Manage 286:112250. https://doi.org/10.1016/j.jenvman.2021.112250
Jiang ZQ, Qin H, Ji CM et al (2017) Two dimension reduction methods for multi-dimensional dynamic programming and its application in cascade reservoirs operation optimization. Water 9(9):634. https://doi.org/10.3390/w9090634
Khadr M, Schlenkhoff A (2021) GA-based implicit stochastic optimization and RNN-based simulation for deriving multi-objective reservoir hedging rules. Environ Sci Pollut R 28(15):19107–19120. https://doi.org/10.1007/s11356-020-12291-w
Li JY, Zhong PA, Yang MZ et al (2019) Dynamic and intelligent modeling methods for joint operation of a flood control system. J Water Resour Plan Manag 145(10):04019044. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001110
Ma YF, Zhong PA, Xu B et al (2020) Multidimensional Parallel Dynamic Programming Algorithm Based on Spark for Large-scale Hydropower Systems. Water Resour Manag. https://doi.org/10.1007/s11269-020-02566-9
Mao J, Tian M, Hu T et al (2019) Shuffled complex evolution coupled with stochastic ranking for reservoir scheduling problems. Water Sci Eng 12(4):307–318. https://doi.org/10.1016/j.wse.2019.12.003
Ming B, Liu P, Bai T et al (2017) Improving optimization efficiency for reservoir operation using a search space reduction method. Water Resour Manag 31(4):1173–1190. https://doi.org/10.1007/s11269-017-1569-x
Mariño MA, Mohammadi B (1983) Reservoir operation by linear and dynamic programming. J Water Resour Plan Manag 109(4):303–319. https://doi.org/10.1061/(ASCE)0733-9496(1983)109:4(303)
Opan M (2010) Irrigation-energy management using a DPSA-based optimization model in the Ceyhan Basin of Turkey. J Hydrol 385(1–4):353–360. https://doi.org/10.1016/j.jhydrol.2010.03.003
Tang W, Li Z, Qiang M et al (2013) Risk management of hydropower development in China. Energy 60:316–324. https://doi.org/10.1016/j.energy.2013.08.034
Xia Y, Feng Z, Niu W et al (2019) Simplex quantum-behaved particle swarm optimization algorithm with application to ecological operation of cascade hydropower reservoirs. Appl Soft Comput 84:105715. https://doi.org/10.1016/j.asoc.2019.105715
Xu B, Zhong PA, Zambon RC et al (2015) Scenario tree reduction in stochastic programming with recourse for hydropower operations. Water Resour Res 51(8):6359–6380. https://doi.org/10.1002/2014WR016828
Xu B, Zhu FL, Zhong PA et al (2019) Identifying long-term effects of using hydropower to complement wind power uncertainty through stochastic programming. Appl Energ 253:113535. https://doi.org/10.1016/j.apenergy.2019.113535
Yang CC, Chang LC, Yeh CH et al (2007) Multiobjective planning of surface water resources by multiobjective genetic algorithm with constrained differential dynamic programming. J Water Resous Plan Manag 133(6):499–508. https://doi.org/10.1061/(ASCE)0733-9496(2007)133:6(499)
Zhang Q, Cheng L, Boutaba R (2010) Cloud computing: state-of-the-art and research challenges. J Internet Serv Appl 1(1):7–18. https://doi.org/10.1007/s13174-010-0007-6
Zhang X, Beeson P, Link R et al (2013) Efficient multi-objective calibration of a computationally intensive hydrologic model with parallel computing software in python. Environ Modell Softw 46:208–218. https://doi.org/10.1016/j.envsoft.2013.03.013
Zhang YK, Jiang ZQ, Ji CM et al (2015) Contrastive analysis of three parallel modes in multi-dimensional dynamic programming and its application in cascade reservoirs operation. J Hydrol 529:22–34. https://doi.org/10.1016/j.jhydrol.2015.07.017
Zhang W, Liu P, Chen XZ et al (2016) Optimal operation of multi-reservoir systems considering time-lags of flood routing. Water Resour Manag 30(2):523–540. https://doi.org/10.1007/s11269-015-1175-8
Zhang W, Huang Y (2019) Using big data computing framework and parallelized PSO algorithm to construct the reservoir dispatching rule optimization. Soft Comput 1–12. https://doi.org/10.1007/s00500-019-04188-9
Zhang Y, Tang W, Duffield C et al (2021) Environment Management of Hydropower Development: A Case Study. Energies 14(7):2029. https://doi.org/10.3390/en14072029
Zhao M, Huang S, Huang Q et al (2019) Copula-Based Research on the Multi-Objective Competition Mechanism in Cascade Reservoirs Optimal Operation. Water 11(5):995. https://doi.org/10.3390/w11050995
Acknowledgements
This study is supported by the National Key R&D Program of China (Grant No. 2017YFC0405606); the National Natural Science Foundation of China (Grant No. 52079037, 52009029); the Fundamental Research Funds for the Central Universities (Grant No. B210203012, B200202032); the China Postdoctoral Science Foundation (Grant No. 2020T130169, 2019M661715).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ma, Y., Zhong, Pa., Xu, B. et al. Cloud-Based Multidimensional Parallel Dynamic Programming Algorithm for a Cascade Hydropower System. Water Resour Manage 35, 2705–2721 (2021). https://doi.org/10.1007/s11269-021-02859-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11269-021-02859-7