Skip to main content
SpringerLink
Log in

Learning that Grid-Convenience Does Not Hurt Resilience in the Presence of Uncertainty

  • Conference paper
  • First Online:
Formal Modeling and Analysis of Timed Systems (FORMATS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13465))

Abstract

Grid-convenience is often seen as conflicting with other measures of interest, such as self-use or resilience. We adopt a Hybrid Petri net model (HPnG) of a smart home with local power generation, local storage and different battery management strategies in the presence of power outages. Applying Q-learning allows us to derive schedulers with an optimal loading percentages for the battery. We show that (near-)optimal schedulers can be synthesized for accurate predictions, which achieve grid-convenience without decreasing resilience and self-use. Introducing uncertainty to the predictions of solar production shows that a good balance can be maintained between such measures, when allowing the system to adapt the loading percentage during the day.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cai, M., Peng, H., Li, Z., Kan, Z.: Learning-based probabilistic LTL motion planning with environment and motion uncertainties. IEEE Trans. Autom. Control 66(5), 2386–2392 (2021)

    Article  MathSciNet  Google Scholar 

  2. del Real, A.J., Dorado, F., Durán, J.: Energy demand forecasting using deep learning: applications for the French grid. Energies 13(9), 2242 (2020)

    Article  Google Scholar 

  3. EDSN: EDSN demand profiles. http://www.edsn.nl/verbruiksprofielen/

  4. Fulton, N., Hunt, N., Hoang, N., Das, S.: Formal verification of end-to-end learning in cyber-physical systems: progress and challenges. arXiv:2006.09181 (2020)

  5. Gasser, P., et al.: A review on resilience assessment of energy systems. Sustain. Resilient Infrastruct. 6(5), 1–27 (2019)

    Google Scholar 

  6. Ghasemieh, H., Haverkort, B.R., Jongerden, M.R., Remke, A.: Energy resilience modelling for smart houses. In: Proceedings of the 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2015), pp. 275–286 (2015)

    Google Scholar 

  7. Gieseke, F., Igel, C.: Training big random forests with little resources. In: 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, pp. 1445–1454. ACM (2018)

    Google Scholar 

  8. Hahn, E.M., Perez, M., Schewe, S., Somenzi, F., Trivedi, A., Wojtczak, D.: Faithful and effective reward schemes for model-free reinforcement learning of omega-regular objectives. In: Hung, D.V., Sokolsky, O. (eds.) ATVA 2020. LNCS, vol. 12302, pp. 108–124. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59152-6_6

    Chapter  Google Scholar 

  9. Hasanbeig, M., Kantaros, Y., Abate, A., Kroening, D., Pappas, G.J., Lee, I.: Reinforcement learning for temporal logic control synthesis with probabilistic satisfaction guarantees. In: IEEE Conference on Decision and Control (CDC), pp. 5338–5343. IEEE, Nice, France (2019)

    Google Scholar 

  10. Hasanbeig, M., Abate, A., Kroening, D.: Cautious reinforcement learning with logical constraints. In: Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2020, Auckland, New Zealand, 9–13 May 2020, pp. 483–491. International Foundation for Autonomous Agents and Multiagent Systems (2020)

    Google Scholar 

  11. Huels, J., Remke, A.: Energy storage in smart homes: grid-convenience versus self-use and survivability. In: 24th IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, MASCOTS 2016, London, United Kingdom, 19–21 September 2016, pp. 385–390 (2016)

    Google Scholar 

  12. Jongerden, M., Hüls, J., Remke, A., Haverkort, B.R.: Does your domestic photovoltaic energy system survive grid outages? Energies 9(9), 736:1–736:17 (2016)

    Google Scholar 

  13. Junges, S., Jansen, N., Katoen, J.-P., Topcu, U., Zhang, R., Hayhoe, M.: Model checking for safe navigation among humans. In: McIver, A., Horvath, A. (eds.) QEST 2018. LNCS, vol. 11024, pp. 207–222. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99154-2_13

    Chapter  Google Scholar 

  14. Lavaei, A., Somenzi, F., Soudjani, S., Trivedi, A., Zamani, M.: Formal controller synthesis for continuous-space MDPs via model-free reinforcement learning. In: 2020 ACM/IEEE 11th International Conference on Cyber-Physical Systems (ICCPS), pp. 98–107. IEEE, Sydney, Australia (2020)

    Google Scholar 

  15. Mola, M., Feofilovs, M., Romagnoli, F.: Energy resilience: research trends at urban, municipal and country levels. Energy Procedia 147, 104–113 (2018)

    Article  Google Scholar 

  16. Niehage, M., Hartmanns, A., Remke, A.: Learning optimal decisions for stochastic hybrid systems. In: 19th ACM-IEEE International Conference on Formal Methods and Models for System Design, MEMOCODE 2021. ACM (2021)

    Google Scholar 

  17. Niehage, M., Remke, A.: Learning that grid-convenience does not hurt resilience in the presence of uncertainty (artifact) (2022). https://doi.org/10.5281/zenodo.6840881

  18. NREL: PVWatts. http://pvwatts.nrel.gov/index.php

  19. Pilch, C., Edenfeld, F., Remke, A.: HYPEG: statistical model checking for hybrid petri nets: tool paper. In: EAI International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS), pp. 186–191. ACM Press (2017)

    Google Scholar 

  20. Pilch, C., Remke, A.: Statistical model checking for hybrid petri nets with multiple general transitions. In: 2017 47th IEEE/IFIP International Conference on Dependable Systems & Networks, DSN, pp. 475–486. IEEE (2017)

    Google Scholar 

  21. Pilch, C., Remke, A.: Statistical model checking for hybrid petri nets with multiple general transitions. In: Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 475–486. IEEE (2017)

    Google Scholar 

  22. Sadigh, D., Kim, E.S., Coogan, S., Sastry, S.S., Seshia, S.A.: A learning based approach to control synthesis of Markov decision processes for linear temporal logic specifications. In: IEEE Conference on Decision and Control, pp. 1091–1096. IEEE (2014)

    Google Scholar 

  23. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning Series, 2nd edn. The MIT Press, Cambridge (2018)

    MATH  Google Scholar 

  24. Tagawa, Y., et al.: Day-ahead scheduling for supply-demand-storage balancing - model predictive generation with interval prediction of photovoltaics. In: 2015 European Control Conference (ECC), pp. 247–252 (2015)

    Google Scholar 

  25. Yu, L., et al.: Deep reinforcement learning for smart home energy management. IEEE Internet Things J. 7(4), 2751–2762 (2020)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Münster, Einsteinstr. 62, 48149, Münster, Germany

    Mathis Niehage & Anne Remke

Authors
  1. Mathis Niehage
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anne Remke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mathis Niehage .

Editor information

Editors and Affiliations

  1. Newcastle University, Newcastle upon Tyne, UK

    Sergiy Bogomolov

  2. University of Birmingham, Birmingham, UK

    David Parker

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Niehage, M., Remke, A. (2022). Learning that Grid-Convenience Does Not Hurt Resilience in the Presence of Uncertainty. In: Bogomolov, S., Parker, D. (eds) Formal Modeling and Analysis of Timed Systems. FORMATS 2022. Lecture Notes in Computer Science, vol 13465. Springer, Cham. https://doi.org/10.1007/978-3-031-15839-1_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-15839-1_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-15838-4

  • Online ISBN: 978-3-031-15839-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation