Abstract
This paper proposes a reinforcement learning model for intelligent energy management in buildings, using a UCB1 based approach. Energy management in buildings has become a critical task in recent years, due to the incentives to the increase of energy efficiency and renewable energy sources penetration. Managing the energy consumption, generation and storage in this domain, becomes, however, an arduous task, due to the large uncertainty of the different resources, adjacent to the dynamic characteristics of this environment. In this scope, reinforcement learning is a promising solution to provide adaptiveness to the energy management methods, by learning with the on-going changes in the environment. The model proposed in this paper aims at supporting decisions on the best actions to take in each moment, regarding buildings energy management. A UCB1 based algorithm is applied, and the results are compared to those of an EXP3 approach and a simple reinforcement learning algorithm. Results show that the proposed approach is able to achieve a higher quality of results, by reaching a higher rate of successful actions identification, when compared to the other considered reference approaches.
This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 641794 (project DREAM-GO) and from Project SIMOCE (ANI|P2020 17690).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Borlase, S.: Smart Grids: Infrastructure, Technology, and Solutions (2012)
Rosen, C., Madlener, R.: Regulatory options for local reserve energy markets: implications for prosumers, utilities, and other stakeholders. Energy J. 37, 39–50 (2016)
Fernandes, F., Morais, H., Vale, Z., Ramos, C.: Dynamic load management in a smart home to participate in demand response events. Energy Build. 82, 592–606 (2014)
Acone, M., Romano, R., Piccolo, A., Siano, P., Loia, F., Ippolito, M.G., Zizzo, G.: Designing an energy management system for smart houses. In: 2015 IEEE 15th International Conference on Environment and Electrical Engineering (EEEIC), pp. 1677–1682 (2015)
Zhou, B., Li, W., Chan, K.W., Cao, Y., Kuang, Y., Liu, X., Wang, X.: Smart home energy management systems: Concept, configurations, and scheduling strategies. Renew. Sustain. Energy Rev. 61, 30–40 (2016)
Afrakhte, H., Bayat, P., Bayat, P.: Energy management system for smart house with multi-sources using PI-CA controller. In: 2016 Iranian Conference on Renewable Energy Distributed Generation (ICREDG), pp. 24–31 (2016)
Pinto, T., Vale, Z., Sousa, T.M., Praça, I., Santos, G., Morais, H.: Adaptive learning in agents behaviour: a framework for electricity markets simulation. Integr. Comput. Aided. Eng. 21, 399–415 (2014)
Li, D., Jayaweera, S.K.: Reinforcement learning aided smart-home decision-making in an interactive smart grid. In: 2014 IEEE Green Energy and Systems Conference (IGESC), pp. 1–6 (2014)
Burtini, G., Loeppky, J., Lawrence, R.: A Survey of Online Experiment Design with the Stochastic Multi-Armed Bandit. arXiv1510.00757 [cs, stat] (2015)
Bouneffouf, D., Féraud, R.: Multi-armed bandit problem with known trend. Neurocomputing 205, 16–21 (2016)
Shimkin, N.: Reinforcement Learning – Basic Algorithms (2011)
Xu, X., Zuo, L., Huang, Z.: Reinforcement learning algorithms with function approximation: recent advances and applications. Inf. Sci. (Ny) 261, 1–31 (2014)
Faia, R., Pinto, T., Abrishambaf, O., Fernandes, F., Vale, Z., Corchado, J.M.: Case based reasoning with expert system and swarm intelligence to determine energy reduction in buildings energy management. Energy Build. 155, 269–281 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Andrade, R., Pinto, T., Praça, I., Vale, Z. (2019). UCB1 Based Reinforcement Learning Model for Adaptive Energy Management in Buildings. In: Rodríguez, S., et al. Distributed Computing and Artificial Intelligence, Special Sessions, 15th International Conference. DCAI 2018. Advances in Intelligent Systems and Computing, vol 801. Springer, Cham. https://doi.org/10.1007/978-3-319-99608-0_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-99608-0_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-99607-3
Online ISBN: 978-3-319-99608-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)