Abstract
This paper explores the feasibility and strategies of using model-free reinforcement learning-based control (RLC) for the slow response radiant floor heating (RFH) systems with a setback setting. First, a detailed physics-based virtual testbed is developed and validated. Then based on the virtual testbed, four different strategies of RLC to handle the slow response are studied, along with a conventional rule-based control (RBC) without setback as a baseline and an MPC with a setback for the upper bound on the performance. The results show that the DQN_TD(λ) with forecasted weather data as states provides the best performance, showing potential for applications. Compared to the baseline, the heating demand is reduced by 19.1% with RLC and 18.5% with MPC. The unmet hours of RLC with our settings are higher than that of MPC, which suggests that more research is needed for RLC to better meet the constraints.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Arroyo J, Manna C, Spiessens F, Helsen L (2022) Reinforced model predictive control (RL-MPC) for building energy management. Appl Energy 309:118346
Arteconi A, Costola D, Hoes P, Hensen J (2014) Analysis of control strategies for thermally activated building systems under demand side management mechanisms. Energy Build 80:384–393
Blad C, Koch S, Ganeswarathas S, Kallesøe C, Bøgh S (2019) Control of hvac-systems with slow thermodynamic using reinforcement learning. Procedia Manuf 38:1308–1315
Daley B, Amato C (2019) Reconciling λ-returns with experience replay. Adv Neural Inf Process Syst:32
Gayeski N, Armstrong P, Norford L (2012) Predictive pre-cooling of thermo-active building systems with low-lift chillers. HVAC&R Res 18(5):858–873
Hoogmartens J, Sourbron M (2011) Review report of existing control strategies for GEO-HP-TABS
Kalz DE (2010) Heating and cooling concepts employing environmental energy and thermo-active building systems for low-energy buildings: system analysis and optimization, Ph.D. Fakultät für Architektur der Universität Karlsruhe
Privara S, Široký J, Ferkl L, Cigler J (2011) Model predictive control of a building heating system: the first experience. Energy Build 43(2–3):564–572
Wetter M, Zuo W, Nouidui TS, Pang X (2014) Modelica buildings library. J Build Perform Simul 7(4):253–270
Yan B et al (2022) Comprehensive assessment of operational performance of coupled natural ventilation and thermally active building system via an extensive sensor network. Energy Build:111921
Zhang Z, Lam KP (2018) Practical implementation and evaluation of deep reinforcement learning control for a radiant heating system. In: Proceedings of the 5th conference on systems for built environments, pp 148–157
Acknowledgements
The authors are grateful for the help from Runyu Zhang and Na Li from SEAS Harvard.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Han, X., Malkawi, A. (2023). Model-Free Reinforcement Learning-Based Control for Radiant Floor Heating Systems. In: Wang, L.L., et al. Proceedings of the 5th International Conference on Building Energy and Environment. COBEE 2022. Environmental Science and Engineering. Springer, Singapore. https://doi.org/10.1007/978-981-19-9822-5_150
Download citation
DOI: https://doi.org/10.1007/978-981-19-9822-5_150
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-9821-8
Online ISBN: 978-981-19-9822-5
eBook Packages: EngineeringEngineering (R0)