Minimizing Grid Interaction of Solar Generation and DHW Loads in nZEBs Using Model-Free Reinforcement Learning

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10691)


This study applies model-free reinforcement learning (RL) on a case study based in Utrecht province in the Netherlands to optimize for on-site renewable energy. This aims at reducing the interaction of net zero-energy buildings with the grid as a result of an increase of heat pump installations and renewable energy systems (RES) integration. It is believed that this will become increasingly more important since the regulations regarding 2020 and beyond ascribe significant increase in energy efficiency of the built environment. On-site RES self-consumption is therefore a central lead in this research. The project data comprise air source heat pump and solar energy data of 6 different households for the months June to November 2016. The RL algorithm is applied to the different data sets to derive an optimized individual and generalized control strategy. Simulations were carried on, to acquire the resulting energy consumption, self-consumption, and self-sufficiency. The results show an increase of individual self-consumption between 17% and 348% and self-sufficiency between 18% and 72%. This results in an additional monetary benefit for the occupants based on the transition proposals of 2020 for the renewable energy generation net-metering abolishment in the Netherlands. Furthermore, reducing the grid interaction implies benefits for the grid operators in terms of investments required for grid reinforcement.


nZEB Demand response Smart grid Reinforcement learning Solar power Heat pumps Self-consumption Self-sufficiency 


  1. 1.
    Rijksoverheid website, Onderzoeken Energie Gebouwde Omgeving, Accessed 07 Apr 2017
  2. 2.
    Van den Oosterkamp, P., Koutstaal, O., van der Welle, A., de Joode, J., Lenstra, J., van Hussen, K., Haffner, R.: The role of DSOs in a Smart Grid environment. ECORYS, Amsterdam/Rotterdam (2014)Google Scholar
  3. 3.
    Lawrence, T.M., Boudreau, M., Helsen, L., Henze, G., Mohammadpour, J., Noonan, D., Pateeuw, D., Pless, S., Watson, R.T.: Ten questions concerning integration smart buildings into the smart grid. In: Building and Environment, vol. 108, pp. 273–283. Elsevier (2016)Google Scholar
  4. 4.
    Yang, R., Wang, L.: Multi-objective optimization for decision-making of energy management in building automation and control. In: Sustainable Cities and Society, vol. 2, pp. 1–7. Elsevier (2012)Google Scholar
  5. 5.
    Klein, L., Kwak, J., Kavulya, G., Jazizadeh, F., Becerik-Gerber, B., Varakantham, P., Tambe, M.: Coordinating occupant behavior for building energy and comfort management using multi-agent systems. In: Automation in Constructions, vol. 22, pp. 525–526. Elsevier (2012)Google Scholar
  6. 6.
    Lund, P., Lindgren, J., Mikkola, J., Salpakari, J.: Review of energy system flexibility measures to enable high levels of variable renewable electricity. In: Renewable and Sustainable Energy Reviews, vol. 45, pp. 785–807. Elsevier (2015)Google Scholar
  7. 7.
    Sossan, F., Kosek, A.M., Martinenas, S., Marinelli, M., Bindner, H.W.: Scheduling of domestic water heater power demand for maximizing PV self-consumption using model predictive control. Technical University of Denmark, Lyngby (2013)CrossRefGoogle Scholar
  8. 8.
    Arteconi, A., Hewitt, N.J., Polonara, F.: Domestic demand-side management (DSM): Role of heat pumps and thermal energy storage (TES) systems. In: Applied Thermal Engineering, vol. 51(1–2), pp. 155–165. Elsevier (2013)Google Scholar
  9. 9.
    Kazmi, H., D’Oca, S., Delmastro C., Lodeweyckx, S., Corgnati, S.P. Generalizable occupant-driven optimization model for domestic hot water production in NZEB. In: Applied Energy, vol. 175, pp. 1–15. Elsevier (2016)Google Scholar
  10. 10.
    Ijaz Dar, U., Sartori, I., Georges, L., Novakovic, V.: Advanced control of heat pumps for improved flexibility of Net-ZEB towards the grid. In: Energy and Buildings, vol. 69, pp. 74–84. Elsevier (2014)Google Scholar
  11. 11.
    De Coninck, R., Baetens, D., Saelens, D., Woyte, A., Helsen, L.: Rule-based demand side management of domestic hot water production with heat pumps in zero energy neighborhoods. J. Build. Perform. Simul. 7, 271–288 (2014)CrossRefGoogle Scholar
  12. 12.
    Ruelens, F.: Residential Demand Response Using Reinforcement Learning. Arenberg Doctoral School, Faculty of Engineering Science, KU Leuven, Leuven (2016)Google Scholar
  13. 13.
    Portaal homepage, Accessed 02 July 2016
  14. 14.
    Pruissen, O.P., Kamphuis, I.G.: Grote concentraties warmtepompen in een woonwijk en gevolgen elektriciteitsnetwerk. Energy research Centre of the Netherlands (ECN), Petten (2010)Google Scholar
  15. 15.
    Rijksdienst voor Ondernemend Nederland, Saldering, zelflevering en tariefkorting, Accessed 26 June 2016
  16. 16.
    Merosch, de effecten van en oplossingen voor aanpassing van salderingsregeling op NOM-woningen in 2020,

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Technical University of DelftDelftThe Netherlands
  2. 2.EnervalisHouthalen-HelchterenBelgium

Personalised recommendations