Skip to main content
SpringerLink
Log in

Fuzzy Reinforcement Learning for Canal Control

  • Chapter
  • First Online:
Computational Intelligence for Water and Environmental Sciences

Part of the book series: Studies in Computational Intelligence ((SCI,volume 1043))

  • 547 Accesses

Abstract

The poor performance of surface irrigation water distribution systems in terms of reliability, sufficiency, and timely delivery makes researchers develop and employ new methods to reduce its consequent challenges, including environmental, energy, and groundwater withdrawal issues. In this regard, many model-based control systems have been considered to automate canal structures. Artificial intelligence, as model-free systems, has recently gained researchers’ attraction to be employed for canal control purposes. In this research, the Reinforcement Learning (RL) methods with critic-only architecture, Fuzzy SARSA Learning (FSL) and Fuzzy Q Learning (FQL) that use a scalar reward/penalty to adapt system parameters online were developed and introduced to control irrigation canals. The main difference between the mentioned methods lies in the mathematical guarantees regarding FSL convergence and FQL divergence observation. Applications of these two methods to a case study canal allows assessing their performance and convergence in this context using standard performance indicators.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Alamiyan-Harandi, F., Derhami, V., & Jamshidi, F. (2018). A new framework for mobile robot trajectory tracking using depth data and learning algorithms. Journal of Intelligent & Fuzzy Systems, 34(6), 3969–3982.

    Article  Google Scholar 

  • Amein, M. (1968). An implicit method for numerical flood routing. Water Resources Research, 4(4), 719–726.

    Article  Google Scholar 

  • Amein, M., and Fang, C. S. (1970). Implicit flood routing in natural channels. Journal of the Hydraulics Division.

    Google Scholar 

  • Arauz, T., Maestre, J. M., Tian, X., & Guan, G. (2020). Design of PI controllers for irrigation canals based on linear matrix inequalities. Water, 12(3), 855.

    Article  Google Scholar 

  • Barkhordari, S., & Shahdany, S. M. H. (2021). Developing a smart operating system for fairly distribution of irrigation water, based on social, economic, and environmental considerations. Agricultural Water Management, 250, 106833.

    Article  Google Scholar 

  • Brittain, M., & Wei, P. (2019). Autonomous air traffic controller: A deep multi-agent reinforcement learning approach. arXiv preprint arXiv:1905.01303.

  • Carlucho, I., De Paula, M., Villar, S. A., & Acosta, G. G. (2017). Incremental Q-learning strategy for adaptive PID control of mobile robots. Expert Systems with Applications, 80, 183–199.

    Article  Google Scholar 

  • Chu, T., Chinchali, S., & Katti, S. (2020). Multi-agent reinforcement learning for networked system control. In Proceedings of International Conference on Learning Representations.

    Google Scholar 

  • Clemmens, A. J., Kacerek, T. F., Grawitz, B., & Schuurmans, W. (1998). Test cases for canal control algorithms. Journal of Irrigation and Drainage Engineering, 124(1), 23–30.

    Article  Google Scholar 

  • Derhami, V., Majd, V. J., & Ahmadabadi, M. N. (2010). Exploration and exploitation balance management in fuzzy reinforcement learning. Fuzzy Sets and Systems, 161(4), 578–595.

    Article  MathSciNet  Google Scholar 

  • Fatemeh, O., Hesam, G., & Shahverdi, K. (2020). Comparing fuzzy SARSA learning (FSL) and ant colony optimization (ACO) algorithms in water delivery scheduling under water shortage conditions. Irrigation and Drainage Engineering, 146(9), 04020028.

    Google Scholar 

  • Fread, D., & Harbaugh, T. (1971). Open-channel profiles by Newton’s iteration technique. Journal of Hydrology, 13, 70–80.

    Article  Google Scholar 

  • Glorennec, P. Y., & Jouffe, L. (1997) Fuzzy Q-learning. In Proceedings of 6th International Fuzzy Systems Conference (pp. 659–662). IEEE.

    Google Scholar 

  • Harandi, F. A., & Derhami, V. (2016). A reinforcement learning algorithm for adjusting antecedent parameters and weights of fuzzy rules in a fuzzy classifier. Journal of Intelligent & Fuzzy Systems, 30(4), 2339–2347.

    Article  Google Scholar 

  • Henderson, F. M. (1966). Open channel flow.

    Google Scholar 

  • Hernández, J., and Merkley, G. (2011a). Canal structure automation rules using an accuracy-based learning classifier system, a genetic algorithm, and a hydraulic simulation model. I: Design. Journal of irrigation and drainage engineering, 137(1).

    Google Scholar 

  • Hernández, J., & Merkley, G. (2011b). Canal structure automation rules using an accuracy-based learning classifier system, a genetic algorithm, and a hydraulic simulation model. I: Result. Journal of Irrigation and Drainage Engineering, 137(5).

    Google Scholar 

  • Kempka, M., Wydmuch, M., Runc, G., Toczek, J., & JaĹ›kowski, W. (2016). Vizdoom: A doom-based ai research platform for visual reinforcement learning. In 2016 IEEE Conference on Computational Intelligence and Games (CIG) (pp. 1–8). IEEE.

    Google Scholar 

  • Khiabani, M. Y., Shahdan, S. M. H., Hassani, Y., & Maestre, J. M. (2021). Introducing an economic agricultural water distribution in a hyper-arid region: A case study in Iran. Journal of Hydroinformatics, 23(3), 548–566.

    Google Scholar 

  • Liu, Y., Yang, T., Zhao, R.-H., Li, Y.-B., Zhao, W.-J., & Ma, X.-Y. (2018). Irrigation canal system delivery scheduling based on a particle swarm optimization algorithm. Water, 10(9), 1281.

    Article  Google Scholar 

  • Manz , D. H. (1990). Use of the ICSS model for prediction of conveyance system operational characteristics. In Transactions of the Fourteenth International Congress on Irrigation and Drainage (ICID) (1–18). Rio de Janerio, Brazil.

    Google Scholar 

  • Mao, H., Alizadeh, M., Menache, I., & Kandula, S. (2016) Resource management with deep reinforcement learning. In Proceedings of the 15th ACM Workshop on Hot Topics in Networks (pp. 50–56).

    Google Scholar 

  • Molden, D. J., & Gates, T. K. (1990). Performance measures for evaluation of irrigation-water-delivery systems. Journal of Irrigation and Drainage Engineering, 116(6), 804–823.

    Article  Google Scholar 

  • Pretorius, A., Cameron, S., Van Biljon, E., Makkink, T., Mawjee, S., Plessis, J. d., Shock, J., Laterre, A., & Beguir, K. (2020). A game-theoretic analysis of networked system control for common-pool resource management using multi-agent reinforcement learning. arXiv preprint arXiv:2010.07777.

  • Ren, T., Niu, J., Cui, J., Ouyang, Z., & Liu, X. (2021). An application of multi-objective reinforcement learning for efficient model-free control of canals deployed with IoT networks. Journal of Network and Computer Applications, 182, 103049.

    Article  Google Scholar 

  • Savari, H., Monem, M., & Shahverdi, K. (2016). Comparing the performance of FSL and traditional operation methods for on-request water delivery in the Aghili Network, Iran. Journal of Irrigation and Drainage Engineering, 142(11), 04016055.

    Article  Google Scholar 

  • Shahverdi, K., Maestre, J., Alamiyan-Harandi, F., & Tian, X. (2020). Generalizing fuzzy SARSA learning for real-time operation of irrigation canals. Water, 12(9), 2407.

    Article  Google Scholar 

  • Shahverdi, K., amp; Maestre, J. M. (2022). Gray wolf optimization for scheduling irrigation water. Journal of Irrigation and Drainage Engineering, 148(7), 04022020.

    Google Scholar 

  • Shahverdi, K., & Monem, M. J. (2012). Construction and evaluation of the bival automatic control system for irrigation canals in a laboratory flume. Irrigation and Drainage, 61(2), 201–207.

    Article  Google Scholar 

  • Shahverdi, K., & Monem, M. J. (2015). Application of reinforcement learning algorithm for automation of canal structures. Irrigation and Drainage, 64(1), 77–84.

    Article  Google Scholar 

  • Shahverdi, K., Monem, M. J., & Nili, M. (2016). Fuzzy SARSA learning of operational instructions to schedule water distribution and delivery. Irrigation and Drainage, 65(3), 276–284.

    Article  Google Scholar 

  • Strelkoff, T. (1969). One-dimensional equations of open-channel flow. Journal of the Hydraulics Division.

    Google Scholar 

  • Sugeno, M., & Takagi, T. (1983). Multi-dimensional fuzzy reasoning. Fuzzy Sets and Systems, 9(1–3), 313–325.

    Article  MathSciNet  Google Scholar 

  • Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction, MIT Press Cambridge.

    Google Scholar 

  • Wang, L.-X. (1999). A course in fuzzy systems. Prentice-Hall Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Water Science Engineering, Bu-Ali Sina University, Hamedan, Iran

    Kazem Shahverdi

  2. Computer Engineering Department, Faculty of Engineering, Yazd University, Yazd, Iran

    Farinaz Alamiyan-Harandi

  3. Systems and Automation Engineering Department, University of Seville, Seville, Spain

    J. M. Maestre

Authors
  1. Kazem Shahverdi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Farinaz Alamiyan-Harandi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. M. Maestre
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kazem Shahverdi .

Editor information

Editors and Affiliations

  1. College of Agriculture & Natural Resources, University of Tehran, Karaj, Alborz, Iran

    Omid Bozorg-Haddad

  2. QUEX Joint-researcher, Centre for Water in the Minerals Industry, Sustainable Minerals Institute, The University of Queensland, Brisbane, QLD, Australia

    Babak Zolghadr-Asli

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Shahverdi, K., Alamiyan-Harandi, F., Maestre, J.M. (2022). Fuzzy Reinforcement Learning for Canal Control. In: Bozorg-Haddad, O., Zolghadr-Asli, B. (eds) Computational Intelligence for Water and Environmental Sciences. Studies in Computational Intelligence, vol 1043. Springer, Singapore. https://doi.org/10.1007/978-981-19-2519-1_15

Download citation

Publish with us

Policies and ethics

Search

Navigation