Abstract
The management of thermal comfort in a building is a challenging and multi-faced problem because it requires considering subjective parameters, such as human preferences and behaviors, and also objective parameters, which can be related to other environmental aspects like the reduction of energy consumption. This chapter exploits cognitive technologies, based on deep reinforcement learning (DRL), for automatically learning how to control the HVAC system in an office. The goal is to develop a cyber-controller able to minimize both the perceived thermal discomfort and the needed energy. The learning process is driven through the definition of a cumulative reward, which includes and combines two reward components that consider, respectively, user comfort and energy consumption. Moreover, a human reward, inferred by the frequency of user interactions with the HVAC system, helps the DRL controller learn the requirements of users and readily adapt to them. Simulation experiments are performed to assess the impact that the two components of the reward have on the behavior of the DRL controller and on the learning process.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Human-in-the-loop is an approach that places people’s knowledge and experience at the center of ML processes, which requires the mediation of users, who give computers access to knowledge of the real world and make sure that they are effectively capable of responding to real needs.
- 2.
Emmeti Silence fan coil: Technical data sheet. https://www.schede-tecniche.it/schede-tecniche-ventilconvettori/EMMETI-scheda-tecnica-ventilconvettori-SILENCE.pdf. Last seen: 2021-12-27.
References
Alanne, K., Sierla, S.: An overview of machine learning applications for smart buildings. Sustain. Cities Soc. 76, 103445 (2022)
Avendano, D.N., Ruyssinck, J., Vandekerckhove, S., Van Hoecke, S., Deschrijver, D.: Data-driven optimization of energy efficiency and comfort in an apartment. In: 2018 International Conference on Intelligent Systems (IS), pp. 174–182. IEEE (2018)
Cicirelli, F., Guerrieri, A., Mastroianni, C., Spezzano, G., Vinci, A.: The Internet of Things for Smart Urban Ecosystems. Internet of Things. Springer International Publishing (2018)
Cicirelli, F., Guerrieri, A., Mastroianni, C., Spezzano, G., Vinci, A.: Thermal comfort management leveraging deep reinforcement learning and human-in-the-loop. In: 2020 IEEE International Conference on Human-Machine Systems (ICHMS), pp. 1–6 (2020). https://doi.org/10.1109/ICHMS49158.2020.9209555
Ding, X., Du, W., Cerpa, A.: Octopus: Deep reinforcement learning for holistic smart building control. In: Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, pp. 326–335 (2019)
Fuller, A., Fan, Z., Day, C., Barlow, C.: Digital twin: Enabling technologies, challenges and open research. IEEE Access 8, 108952–108971 (2020)
Gao, G., Li, J., Wen, Y.: Energy-efficient thermal comfort control in smart buildings via deep reinforcement learning. Preprint (2019). arXiv:1901.04693
Gupta, A., Badr, Y., Negahban, A., Qiu, R.G.: Energy-efficient heating control for smart buildings with deep reinforcement learning. J. Build. Eng. 34, 101739 (2021)
Jia, R., Jin, M., Sun, K., Hong, T., Spanos, C.: Advanced building control via deep reinforcement learning. Energy Procedia 158, 6158–6163 (2019)
Lissa, P., Deane, C., Schukat, M., Seri, F., Keane, M., Barrett, E.: Deep reinforcement learning for home energy management system control. Energy AI 3, 100043 (2021)
MartÃnez-Molina, A., Tort-Ausina, I., Cho, S., Vivancos, J.L.: Energy efficiency and thermal comfort in historic buildings: A review. Renew. Sustain. Energy Rev. 61, 70–85 (2016)
Mason, K., Grijalva, S.: A review of reinforcement learning for autonomous building energy management. Preprint (2019). arXiv:1903.05196
Panchalingam, R., Chan, K.C.: A state-of-the-art review on artificial intelligence for smart buildings. Intell. Build. Int. 13(4), 203–226 (2021)
Qi, Q., Tao, F., Hu, T., Anwer, N., Liu, A., Wei, Y., Wang, L., Nee, A.: Enabling technologies and tools for digital twin. J. Manuf. Syst. 58, 3–21 (2021)
Sewak, M.: Deep q network (dqn), double dqn, and dueling dqn. In: Deep Reinforcement Learning, pp. 95–108. Springer (2019)
Shen, Z., Yang, K., Du, W., Zhao, X., Zou, J.: Deepapp: a deep reinforcement learning framework for mobile application usage prediction. In: Proceedings of the 17th Conference on Embedded Networked Sensor Systems, pp. 153–165 (2019)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, USA (2011)
Wei, T., Wang, Y., Zhu, Q.: Deep reinforcement learning for building hvac control. In: Proceedings of the 54th Annual Design Automation Conference 2017, p. 22. ACM (2017)
Yu, L., Xie, W., Xie, D., Zou, Y., Zhang, D., Sun, Z., Zhang, L., Zhang, Y., Jiang, T.: Deep reinforcement learning for smart home energy management. IEEE Internet Things J. (2019)
Yu, L., Qin, S., Zhang, M., Shen, C., Jiang, T., Guan, X.: A review of deep reinforcement learning for smart building energy management. IEEE Internet Things J. (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Scarcello, L., Mastroianni, C. (2023). Cognitive Systems for Energy Efficiency and Thermal Comfort in Smart Buildings. In: Cicirelli, F., Guerrieri, A., Vinci, A., Spezzano, G. (eds) IoT Edge Solutions for Cognitive Buildings. Internet of Things. Springer, Cham. https://doi.org/10.1007/978-3-031-15160-6_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-15160-6_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15159-0
Online ISBN: 978-3-031-15160-6
eBook Packages: Computer ScienceComputer Science (R0)