Abstract
Air conditioning water systems account for a large proportion of building energy consumption. In a pressure-controlled water system, one of the key measures to save energy is to adjust the differential pressure setpoints during operation. Typically, such adjustments are based either on certain rules, which rely on operator experience, or on complicated models that are not easy to calibrate. In this paper, a data-driven control method based on reinforcement learning is proposed. The main idea is to construct an agent model that adapts to the researched problem. Instead of directly being told how to react, the agent must rely on its own experiences to learn. Compared with traditional control strategies, reinforcement learning control (RLC) exhibits more accurate and steady performances while maintaining indoor air temperature within a limited range. A case study shows that the RLC strategy is able to save substantial amounts of energy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barrett E, Linder S (2015). Autonomous HVAC control, a reinforcement learning approach. In: Bifet A, et al. (eds), Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2015. Lecture Notes in Computer Science, vol 9286. Cham, Switzerland.
Chen Y, Norford LK, Samuelson HW, et al. (2018). Optimal control of HVAC and window systems for natural ventilation through reinforcement learning. Energy and Buildings, 169: 195–205.
Dayan P, Niv Y (2008). Reinforcement learning: The good, the bad and the ugly. Current Opinion in Neurobiology, 18: 185–196.
Gao D, Wang S, Shan K (2016). In-situ implementation and evaluation of an online robust pump speed control strategy for avoiding low delta-T syndrome in complex chilled water systems of high-rise buildings. Applied Energy, 171: 541–554.
Han M, May R, Zhang X, et al. (2019). A review of reinforcement learning methodologies for controlling occupant comfort in buildings. Sustainable Cities and Society, 51: 101748.
Hou J, Xu P, Lu X, et al. (2018). Implementation of expansion planning in existing district energy system: A case study in China. Applied Energy, 211: 269–281.
Ji Y, Peng S, Geng L, et al. (2009). Pressure loop control of pump and valve combined EHA based on FFIM. In: Proceedings of the 9th International Conference on Electronic Measurement and Instruments, Beijing, China.
Jin X, Du Z, Xiao X (2007). Energy evaluation of optimal control strategies for central VWV chiller systems. Applied Thermal Engineering, 27: 934–941.
Lewis FL, Vrabie D, Vamvoudakis KG (2012). Reinforcement learning and feedback control: Using natural decision methods to design optimal adaptive controllers. IEEE Control Systems Magazine, 32(6): 76–105.
Li W, Xu P, Lu X, et al. (2016). Electricity demand response in China: Status, feasible market schemes and pilots. Energy, 114: 981–994.
Liu S, Henze GP (2006a). Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage inventory: Part 1. Theoretical foundation. Energy and Buildings, 38: 142–147.
Liu S, Henze GP (2006b). Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage inventory: Part 2: Results and analysis. Energy and Buildings, 38: 148–161.
Liu X, Liu J, Lu J, et al. (2012). Research on operating characteristics of direct-return chilled water system controlled by variable temperature difference. Energy, 40: 236–249.
Ma Z, Wang S (2009). Energy efficient control of variable speed pumps in complex building central air-conditioning systems. Energy and Buildings, 41: 197–205.
Mehlhase A (2012). A python package for simulating variable-structure models with dymola. IFAC Proceedings Volumes, 45: 1081–1086.
Pérez-Lombard L, Ortiz J, Pout C (2008). A review on buildings energy consumption information. Energy and Buildings, 40: 394–398.
Recht B (2019). A tour of reinforcement learning: The view from continuous control. Annual Review of Control, Robotics, and Autonomous Systems, 2: 253–279.
Sutton R, Barto A (2018). Reinforcement Learning:An Introduction. Cambridge, MA, USA: MIT Press.
Wang Z, Hong T (2020). Reinforcement learning for building controls: The opportunities and challenges. Applied Energy, 269: 115036.
Wei T, Wang Y, Zhu Q (2017). Deep Reinforcement Learning for Building HVAC Control. In: Proceedings of the 54th Annual Design Automation Conference, Austin, TX, USA.
Yang L, Nagy Z, Goffin P, Schlueter A (2015). Reinforcement learning for optimal control of low exergy buildings. Applied Energy, 156: 577–586.
Zhao T, Ma L, Zhang J (2016). An optimal differential pressure reset strategy based on the most unfavorable thermodynamic loop on-line identification for a variable water flow air conditioning system. Energy and Buildings, 110: 257–268.
Zhou Y, Chen J, Yu ZJ, et al. (2020). A novel model based on multi-grained cascade forests with wavelet denoising for indoor occupancy estimation. Building and Environment, 167: 106461.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, X., Li, Z., Li, Z. et al. Differential pressure reset strategy based on reinforcement learning for chilled water systems. Build. Simul. 15, 233–248 (2022). https://doi.org/10.1007/s12273-021-0808-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12273-021-0808-5