Abstract
Since water pollution is one of the most serious environmental problems today, control of wastewater treatment plants (WWTPs) is a crucial issue nowadays and stricter standards for the operation of WWTPs have been imposed by authorities. One of the main problems in the automation of the control of Wastewater Treatment Plants (WWTPs) appears when the control system does not respond as it should because of changes on influent load or flow. Thus, it is desirable the development of autonomous systems that learn from interaction with a WWTP and that can operate taking into account changing environmental circumstances. In this paper we present an intelligent agent using reinforcement learning for the oxygen control in the N-Ammonia removal process in the well known Benchmark Simulation Model no.1 (BSM1). The aim of the approach presented in this paper is to minimize the operation cost changing the set-points of the control system autonomously.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Alex, J., Benedetti, L., Copp, J., Gernaey, K.V., Jeppsson, U., Nopens, I., Pons, M.N., Rosen, C., Steyer, J.P., Vanrolleghem, P.: Benchmark Simulation Model no. 1 (BSM1). In: IWA Taskgroup on Benchmarking of Control Strategies for WWTP, p. 61 (April 2008)
Belanche, L., Valdes, J.J., Comas, J., Roda, I.R., Poch, M.: Prediction of the bulking phenomenon in wastewater treatment plants. Artificial Intelligence in Engineering 14(4), 307–317 (2000)
Bertsekas, D.P.: Dynamic Programming and Optimal Control, 3rd edn., vol. 2. Athenas Scientific (2007)
Carrasco, E.F., Rodriguez, J., Punyal, A., Roca, E., Lema, J.M.: Rule-based diagnosis and supervision of a pilot-scale wastewater treatment plant using fuzzy logic techniques. Expert Systems with Applications 22(1), 11–20 (2002)
Carrasco, E.F., Rodriguez, J., Punyal, A., Roca, E., Lema, J.M.: Diagnosis of acidification states in an anaerobic wastewater treatment plant using a fuzzy-based expert system. Control Engineering Practice 12(1), 59–64 (2004)
Chen, J.C., Chang, N.B., Shieh, W.K.: Assessing wastewater reclamation potential by neural network model. Engineering Applications of Artificial Intelligence 16(2), 149–157 (2003)
Copp, J.: The COST Simulation Benchmark: Description and Simulator Manual, p. 154, Office for Official Publications of the European Community, Luxembourg (2002); ISBN: 92-894-1658-0
Cortes, U., Sanchez-Marre, M., Ceccaroni, L., R-Roda, I., Poch, M.: Artificial intelligence and environmental decision support systems. Applied Intelligence 13(1), 77–91 (2000)
Huang, M., Wan, J., Ma, Y., Li, W., Sun, X., Wan, Y.: A fast predicting neural fuzzy model for on-line estimation of nutrient dynamics in an anoxic/oxic process. Bioresource Technology 101(6), 1642–1651 (2010)
Huang, M., Wan, J., Yongwen, M., Yan, W., Weijiang, L., Xiaofei, S.: Control rules of aeration in a submerged biofilm wastewater treatment process using fuzzy neural networks. Expert Systems with Applications 36(7), 10428–10437 (2009)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Liao, S.: Expert system methodologies and applications: a decade review from 1995 to 2004. Expert Systems with Appications 28, 93–103 (2005)
Olsson, G., Nielsen, M., Yuan, Z., Lynggaard-Jensen, A., Steyer, J.P.: Instrumentation, Control and Automation in Wastewater Systems. IWA Publishing (2005)
Samuelsson, P., Halvarsson, B., Carlsson, B.: Cost-efficient operation of a denitrifying activated sludge process. Water Research 41, 2325–2332 (2007)
Stare, A., Vrecko, D., Hvala, N., Strmcnik, S.: Comparison of control strategies for nitrogen removal in activated sludge process in terms of operating costs: A simulation study. Water Research 41, 2004–2014 (2007)
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Syafiie, S., Tadeo, F., Martinez, E.: Model-free learning control of neutralization processes using reinforcement learning. Engineering Applications of Artificial Intelligence 20(6), 767–782 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hernandez-del-Olmo, F., Gaudioso, E. (2011). Reinforcement Learning Techniques for the Control of WasteWater Treatment Plants. In: Ferrández, J.M., Álvarez Sánchez, J.R., de la Paz, F., Toledo, F.J. (eds) New Challenges on Bioinspired Applications. IWINAC 2011. Lecture Notes in Computer Science, vol 6687. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21326-7_24
Download citation
DOI: https://doi.org/10.1007/978-3-642-21326-7_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21325-0
Online ISBN: 978-3-642-21326-7
eBook Packages: Computer ScienceComputer Science (R0)