Skip to main content
SpringerLink
Log in

Shielded Learning for Resilience and Performance Based on Statistical Model Checking in Simulink

  • Conference paper
  • First Online:
Bridging the Gap Between AI and Reality (AISoLA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14380))

Included in the following conference series:

  • 638 Accesses

Abstract

Safety, resilience and performance are crucial properties in intelligent hybrid systems, in particular if they are used in critical infrastructures or safety-critical systems. In this paper, we present a case study that illustrates how to construct provably safe and resilient systems that still achieve certain performance levels with a statistical guarantee in the industrially widely used modeling language Simulink. The key ideas of our paper are threefold: First, we show how to model failures and repairs in Simulink. Second, we use hybrid contracts to non-deterministically overapproximate the failure and repair model and to deductively verify safety properties in the presence of worst-case behavior. Third, we show how to learn optimal decisions using statistical model checking (SMC-based learning), which uses the results from deductive verification as a shield to ensure that only safe actions are chosen. We take component failures into account and learn a schedule that is optimized for performance and ensures resilience in a given Simulink model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.uni-muenster.de/EmbSys/research/Simulink2dL.html.

  2. 2.

    https://zivgitlab.uni-muenster.de/ag-sks/tools/hypeg.

References

  1. Adelt, J., Brettschneider, D., Herber, P.: Reusable contracts for safe integration of reinforcement learning in hybrid systems. In: Automated Technology for Verification and Analysis: 20th International Symposium, ATVA 2022, Virtual Event, 25–28 October 2022, Proceedings, pp. 58–74. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-19992-9_4

  2. Adelt, J., Herber, P., Niehage, M., Remke, A.: Towards safe and resilient hybrid systems in the presence of learning and uncertainty. In: Leveraging Applications of Formal Methods, Verification and Validation. Verification Principles: 11th International Symposium, ISoLA 2022, Rhodes, Greece, 22–30 October 2022, Proceedings, Part I, pp. 299–319. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-19849-6_18

  3. Adelt, J., Liebrenz, T., Herber, P.: Formal verification of intelligent hybrid systems that are modeled with simulink and the reinforcement learning toolbox. In: Huisman, M., Păsăreanu, C., Zhan, N. (eds.) FM 2021. LNCS, vol. 13047, pp. 349–366. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-90870-6_19

    Chapter  Google Scholar 

  4. Agresti, A., Coull, B.: Approximate is better than “exact’’ for interval estimation of binomial proportions. Am. Stat. 52, 119–126 (1998)

    MathSciNet  Google Scholar 

  5. Alshiekh, M., Bloem, R., Ehlers, R., Könighofer, B., Niekum, S., Topcu, U.: Safe reinforcement learning via shielding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  6. Alur, R.: Formal verification of hybrid systems. In: ACM International Conference on Embedded Software (EMSOFT), pp. 273–278 (2011)

    Google Scholar 

  7. Araiza-Illan, D., Eder, K., Richards, A.: Formal verification of control systems’ properties with theorem proving. In: UKACC International Conference on Control (CONTROL), pp. 244–249. IEEE (2014)

    Google Scholar 

  8. Boyer, B., Corre, K., Legay, A., Sedwards, S.: PLASMA-lab: a flexible, distributable statistical model checking library. In: Joshi, K., Siegle, M., Stoelinga, M., D’Argenio, P.R. (eds.) QEST 2013. LNCS, vol. 8054, pp. 160–164. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40196-1_12

    Chapter  Google Scholar 

  9. Budde, C.E., D’Argenio, P.R., Hartmanns, A., Sedwards, S.: An efficient statistical model checker for nondeterminism and rare events. Int. J. Softw. Tools Technol. Transf. 22(6), 759–780 (2020)

    Article  Google Scholar 

  10. Cai, M., Peng, H., Li, Z., Kan, Z.: Learning-based probabilistic LTL motion planning with environment and motion uncertainties. IEEE Trans. Autom. Control 66(5), 2386–2392 (2021)

    Article  MathSciNet  Google Scholar 

  11. Carr, S., Jansen, N., Junges, S., Topcu, U.: Safe reinforcement learning via shielding under partial observability. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 12, pp. 14748–14756 (2023)

    Google Scholar 

  12. Chen, M., et al.: MARS: a toolchain for modelling, analysis and verification of hybrid systems. In: Hinchey, M.G., Bowen, J.P., Olderog, E.-R. (eds.) Provably Correct Systems. NMSSE, pp. 39–58. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-48628-4_3

    Chapter  Google Scholar 

  13. Chutinan, A., Krogh, B.H.: Computational techniques for hybrid system verification. IEEE Trans. Autom. Control 48(1), 64–75 (2003)

    Article  MathSciNet  Google Scholar 

  14. D’Argenio, P., Legay, A., Sedwards, S., Traonouez, L.M.: Smart sampling for lightweight verification of Markov decision processes. Int. J. Softw. Tools Technol. Transfer 17(4), 469–484 (2015)

    Article  Google Scholar 

  15. D’Argenio, P.R., Hartmanns, A., Sedwards, S.: Lightweight statistical model checking in nondeterministic continuous time. In: Margaria, T., Steffen, B. (eds.) ISoLA 2018. LNCS, vol. 11245, pp. 336–353. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03421-4_22

    Chapter  Google Scholar 

  16. Ellen, C., Gerwinn, S., Fränzle, M.: Statistical model checking for stochastic hybrid systems involving nondeterminism over continuous domains. Int. J. Softw. Tools Technol. Transfer 17(4), 485–504 (2015)

    Article  Google Scholar 

  17. Filipovikj, P., et al.: Analyzing industrial simulink models by statistical model checking (2017)

    Google Scholar 

  18. Fulton, N., Mitsch, S., Quesel, J.-D., Völp, M., Platzer, A.: KeYmaera X: an axiomatic tactical theorem prover for hybrid systems. In: Felty, A.P., Middeldorp, A. (eds.) CADE 2015. LNCS (LNAI), vol. 9195, pp. 527–538. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21401-6_36

    Chapter  Google Scholar 

  19. Fulton, N., Platzer, A.: Safe reinforcement learning via formal methods: toward safe control through proof and learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  20. Gomes, A., Mota, A., Sampaio, A., Ferri, F., Buzzi, J.: Systematic model-based safety assessment via probabilistic model checking. In: Margaria, T., Steffen, B. (eds.) ISoLA 2010. LNCS, vol. 6415, pp. 625–639. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16558-0_50

    Chapter  Google Scholar 

  21. Gomes, A., Mota, A., Sampaio, A., Ferri, F., Watanabe, E.: Constructive model-based analysis for safety assessment. Int. J. Softw. Tools Technol. Transfer 14, 673–702 (2012)

    Article  Google Scholar 

  22. Gudemann, M., Ortmeier, F.: A framework for qualitative and quantitative formal model-based safety analysis. In: IEEE International Symposium on High Assurance Systems Engineering, pp. 132–141. IEEE (2010)

    Google Scholar 

  23. Hahn, E.M., Perez, M., Schewe, S., Somenzi, F., Trivedi, A., Wojtczak, D.: Faithful and effective reward schemes for model-free reinforcement learning of omega-regular objectives. In: Hung, D.V., Sokolsky, O. (eds.) ATVA 2020. LNCS, vol. 12302, pp. 108–124. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59152-6_6

    Chapter  Google Scholar 

  24. Hartmanns, A., Hermanns, H.: The modest toolset: an integrated environment for quantitative modelling and verification. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014. LNCS, vol. 8413, pp. 593–598. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-54862-8_51

    Chapter  Google Scholar 

  25. Hasanbeig, M., Kantaros, Y., Abate, A., Kroening, D., Pappas, G.J., Lee, I.: Reinforcement learning for temporal logic control synthesis with probabilistic satisfaction guarantees. In: IEEE Conference on Decision and Control (CDC), pp. 5338–5343. IEEE, Nice (2019)

    Google Scholar 

  26. Hasanbeig, M., Abate, A., Kroening, D.: Cautious reinforcement learning with logical constraints. In: AAMAS 2020, International Foundation for Autonomous Agents and Multiagent Systems, pp. 483–491 (2020)

    Google Scholar 

  27. Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., Meger, D.: Deep reinforcement learning that matters. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th Innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, 2–7 February 2018, vol. 32, pp. 3207–3214. AAAI Press (2018)

    Google Scholar 

  28. Herber, P., Reicherdt, R., Bittner, P.: Bit-precise formal verification of discrete-time MATLAB/Simulink models using SMT solving. In: International Conference on Embedded Software (EMSOFT), pp. 1–10. IEEE (2013)

    Google Scholar 

  29. Kanwar, K., Vajpai, D.J.: Performance evaluation of different models of PV panel in matlab/simulink environment. Appl. Solar Energy 58(1), 86–94 (2022)

    Article  Google Scholar 

  30. Knüppel, A., Thüm, T., Schaefer, I.: GUIDO: automated guidance for the configuration of deductive program verifiers. In: IEEE/ACM International Conference on Formal Methods in Software Engineering (FormaliSE), pp. 124–129. IEEE (2021)

    Google Scholar 

  31. Könighofer, B., Lorber, F., Jansen, N., Bloem, R.: Shield synthesis for reinforcement learning. In: Margaria, T., Steffen, B. (eds.) ISoLA 2020. LNCS, vol. 12476, pp. 290–306. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61362-4_16

    Chapter  Google Scholar 

  32. Kwiatkowska, M., Norman, G., Parker, D.: PRISM: probabilistic symbolic model checker. In: Field, T., Harrison, P.G., Bradley, J., Harder, U. (eds.) TOOLS 2002. LNCS, vol. 2324, pp. 200–204. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46029-2_13

    Chapter  Google Scholar 

  33. Legay, A., Sedwards, S., Traonouez, L.-M.: Scalable verification of Markov decision processes. In: Canal, C., Idani, A. (eds.) SEFM 2014. LNCS, vol. 8938, pp. 350–362. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-15201-1_23

    Chapter  Google Scholar 

  34. Legay, A., Traonouez, L.-M.: Statistical model checking of simulink models with plasma lab. In: Artho, C., Ölveczky, P.C. (eds.) FTSCS 2015. CCIS, vol. 596, pp. 259–264. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-29510-7_15

    Chapter  Google Scholar 

  35. Liebrenz, T., Herber, P., Glesner, S.: Deductive verification of hybrid control systems modeled in simulink with KeYmaera X. In: Sun, J., Sun, M. (eds.) ICFEM 2018. LNCS, vol. 11232, pp. 89–105. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02450-5_6

    Chapter  Google Scholar 

  36. Liebrenz, T., Herber, P., Glesner, S.: A service-oriented approach for decomposing and verifying hybrid system models. In: Arbab, F., Jongmans, S.-S. (eds.) FACS 2019. LNCS, vol. 12018, pp. 127–146. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-40914-2_7

    Chapter  Google Scholar 

  37. Liebrenz, T., Herber, P., Glesner, S.: Service-oriented decomposition and verification of hybrid system models using feature models and contracts. Sci. Comput. Program. 211, 102694 (2021)

    Article  Google Scholar 

  38. Lygeros, J., Prandini, M.: Stochastic hybrid systems: a powerful framework for complex, large scale applications. Eur. J. Control. 16(6), 583–594 (2010)

    Article  MathSciNet  Google Scholar 

  39. Mahto, R.K., Kaur, J., Jain, P.: Performance analysis of robotic arm using simulink. In: 2022 IEEE World Conference on Applied Intelligence and Computing (AIC), pp. 508–512. IEEE (2022)

    Google Scholar 

  40. Maler, O., Nickovic, D.: Monitoring temporal properties of continuous signals. In: Lakhnech, Y., Yovine, S. (eds.) FORMATS/FTRTFT -2004. LNCS, vol. 3253, pp. 152–166. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30206-3_12

    Chapter  Google Scholar 

  41. Manno, G., Chiacchio, F., Compagno, L., D’Urso, D., Trapani, N.: Matcarlore: an integrated FT and monte carlo simulink tool for the reliability assessment of dynamic fault tree. Expert Syst. Appl. 39(12), 10334–10342 (2012)

    Article  Google Scholar 

  42. Minopoli, S., Frehse, G.: SL2SX translator: from Simulink to SpaceEx models. In: International Conference on Hybrid Systems: Computation and Control, pp. 93–98. ACM (2016)

    Google Scholar 

  43. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)

    Article  Google Scholar 

  44. Niehage, M., Hartmanns, A., Remke, A.: Learning optimal decisions for stochastic hybrid systems. In: ACM-IEEE International Conference on Formal Methods and Models for System Design (MEMOCODE), pp. 44–55. ACM (2021)

    Google Scholar 

  45. Niehage, M., Pilch, C., Remke, A.: Simulating hybrid petri nets with general transitions and non-linear differential equations. In: VALUETOOLS 2020: 13th EAI International Conference on Performance Evaluation Methodologies and Tools, Tsukuba, Japan, 18–20 May 2020, pp. 88–95. ACM (2020)

    Google Scholar 

  46. Niehage, M., Remke, A.: Learning that grid-convenience does not hurt resilience in the presence of uncertainty. In: Formal Modeling and Analysis of Timed Systems, vol. 13465, pp. 298–306. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-15839-1_17

  47. Pilch, C., Edenfeld, F., Remke, A.: HYPEG: statistical model checking for hybrid petri nets: tool paper. In: EAI International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS), pp. 186–191. ACM Press (2017)

    Google Scholar 

  48. Pilch, C., Niehage, M., Remke, A.: HPnGs go Non-linear: statistical dependability evaluation of battery-powered systems. In: IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), pp. 157–169. IEEE (2018)

    Google Scholar 

  49. Pilch, C., Remke, A.: Statistical model checking for hybrid petri nets with multiple general transitions. In: Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 475–486. IEEE (2017)

    Google Scholar 

  50. Platzer, A.: Differential dynamic logic for hybrid systems. J. Autom. Reason. 41(2), 143–189 (2008)

    Article  MathSciNet  Google Scholar 

  51. Reicherdt, R., Glesner, S.: Formal verification of discrete-time MATLAB/simulink models using boogie. In: Giannakopoulou, D., Salaün, G. (eds.) SEFM 2014. LNCS, vol. 8702, pp. 190–204. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10431-7_14

    Chapter  Google Scholar 

  52. Sadigh, D., Kim, E.S., Coogan, S., Sastry, S.S., Seshia, S.A.: A learning based approach to control synthesis of Markov decision processes for linear temporal logic specifications. In: IEEE Conference on Decision and Control, pp. 1091–1096. IEEE (2014)

    Google Scholar 

  53. Saraoğlu, M., Morozov, A., Söylemez, M.T., Janschek, K.: ErrorSim: a tool for error propagation analysis of simulink models. In: Tonetta, S., Schoitsch, E., Bitsch, F. (eds.) SAFECOMP 2017. LNCS, vol. 10488, pp. 245–254. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66266-4_16

    Chapter  Google Scholar 

  54. Shmarov, F., Zuliani, P.: Probabilistic hybrid systems verification via SMT and monte carlo techniques. In: Bloem, R., Arbel, E. (eds.) HVC 2016. LNCS, vol. 10028, pp. 152–168. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49052-6_10

    Chapter  Google Scholar 

  55. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. The MIT Press, Cambridge (2018)

    Google Scholar 

  56. The MathWorks: Simulink. https://de.mathworks.com/products/simulink.html

  57. The MathWorks: Reinforcement Learning Toolbox. https://www.mathworks.com/products/reinforcement-learning.html

  58. The MathWorks: Simulink Design Verifier. https://de.mathworks.com/products/simulink-design-verifier.html

  59. The MathWorks: Simulink Example: Water Distribution System Scheduling Using Reinforcement Learning. https://de.mathworks.com/help/reinforcement-learning/ug/water-distribution-scheduling-system.html

  60. Tsoutsanis, E., Meskin, N., Benammar, M., Khorasani, K.: Dynamic performance simulation of an aeroderivative gas turbine using the matlab simulink environment. In: ASME International Mechanical Engineering Congress and Exposition, vol. 56246, p. V04AT04A050. American Society of Mechanical Engineers (2013)

    Google Scholar 

  61. Wilson, E.: Probable inference, the law of succession, and statistical inference. J. Am. Stat. Assoc. 22(158), 209–212 (1927)

    Article  Google Scholar 

  62. Zou, L., Zhan, N., Wang, S., Fränzle, M.: Formal verification of simulink/stateflow diagrams. In: Finkbeiner, B., Pu, G., Zhang, L. (eds.) ATVA 2015. LNCS, vol. 9364, pp. 464–481. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24953-7_33

    Chapter  Google Scholar 

  63. Zuliani, P., Platzer, A., Clarke, E.M.: Bayesian statistical model checking with application to stateflow/simulink verification. Formal Methods Syst. Des. 43, 338–367 (2013)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Münster, Einsteinstr. 62, 48149, Münster, Germany

    Julius Adelt, Sebastian Bruch, Paula Herber, Mathis Niehage & Anne Remke

Authors
  1. Julius Adelt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sebastian Bruch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paula Herber
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mathis Niehage
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Anne Remke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Julius Adelt , Paula Herber , Mathis Niehage or Anne Remke .

Editor information

Editors and Affiliations

  1. TU Dortmund University, Dortmund, Germany

    Bernhard Steffen

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Adelt, J., Bruch, S., Herber, P., Niehage, M., Remke, A. (2024). Shielded Learning for Resilience and Performance Based on Statistical Model Checking in Simulink. In: Steffen, B. (eds) Bridging the Gap Between AI and Reality. AISoLA 2023. Lecture Notes in Computer Science, vol 14380. Springer, Cham. https://doi.org/10.1007/978-3-031-46002-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-46002-9_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-46001-2

  • Online ISBN: 978-3-031-46002-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation