Skip to main content
SpringerLink
Log in

Active Model Learning of Stochastic Reactive Systems

  • Conference paper
  • First Online:
Software Engineering and Formal Methods (SEFM 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 13085))

Included in the following conference series:

Abstract

Black-box systems are inherently hard to verify. Many verification techniques, like model checking, require formal models as a basis. However, such models often do not exist, or they might be outdated. Active automata learning helps to address this issue by offering to automatically infer formal models from system interactions. Hence, automata learning has been receiving much attention in the verification community in recent years. This led to various efficiency improvements, paving the way towards industrial applications. Most research, however, has been focusing on deterministic systems. Here, we present an approach to efficiently learn models of stochastic reactive systems. Our approach adapts \(L^*\)-based learning for Markov decision processes, which we improve and extend to stochastic Mealy machines. Our evaluation demonstrates that we can reduce learning costs by a factor of up to 8.7 in comparison to previous work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    An interactive example illustrating learning is available at https://github.com/DES-Lab/AALpy/blob/master/notebooks/MDP_and_SMM_Example.ipynb.

References

  1. Aichernig, B.K., Pferscher, A., Tappler, M.: From passive to active: learning timed automata efficiently. In: Lee, R., Jha, S., Mavridou, A., Giannakopoulou, D. (eds.) NFM 2020. LNCS, vol. 12229, pp. 1–19. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-55754-6_1

    Chapter  Google Scholar 

  2. Aichernig, B.K., Tappler, M.: Probabilistic black-box reachability checking (extended version). Formal Methods Syst. Des. 54(3), 416–448 (2019). https://doi.org/10.1007/s10703-019-00333-0

    Article  MATH  Google Scholar 

  3. Aichernig, B.K., Tappler, M., Wallner, F.: Benchmarking combinations of learning and testing algorithms for active automata learning. In: Ahrendt, W., Wehrheim, H. (eds.) TAP 2020. LNCS, vol. 12165, pp. 3–22. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50995-8_1

    Chapter  Google Scholar 

  4. Angluin, D.: Learning regular sets from queries and counterexamples. Inf. Comput. 75(2), 87–106 (1987)

    Article  MathSciNet  Google Scholar 

  5. Carrasco, R.C., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: Carrasco, R.C., Oncina, J. (eds.) ICGI 1994. LNCS, vol. 862, pp. 139–152. Springer, Heidelberg (1994). https://doi.org/10.1007/3-540-58473-0_144

    Chapter  MATH  Google Scholar 

  6. Carrasco, R.C., Oncina, J.: Learning deterministic regular grammars from stochastic samples in polynomial time. RAIRO: Theor. Inform. Appl. (RAIRO: ITA) 33(1), 1–20 (1999)

    Article  MathSciNet  Google Scholar 

  7. Casacuberta, F., Vidal, E.: Machine translation with inferred stochastic finite-state transducers. Comput. Linguist. 30(2), 205–225 (2004)

    Article  MathSciNet  Google Scholar 

  8. Chen, Y., Nielsen, T.D.: Active learning of Markov decision processes for system verification. In: 11th International Conference on Machine Learning and Applications, ICMLA, Boca Raton, FL, USA, 12–15 December 2012, vol. 2, pp. 289–294. IEEE (2012)

    Google Scholar 

  9. Dehnert, C., Junges, S., Katoen, J.-P., Volk, M.: A Storm is coming: a modern probabilistic model checker. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10427, pp. 592–600. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63390-9_31

    Chapter  Google Scholar 

  10. El-Fakih, K., Groz, R., Irfan, M.N., Shahbaz, M.: Learning finite state models of observable nondeterministic systems in a testing context. In: ICTSS 2010, pp. 97–102 (2010)

    Google Scholar 

  11. Fiterău-Broştean, P., Janssen, R., Vaandrager, F.: Combining model learning and model checking to analyze TCP implementations. In: Chaudhuri, S., Farzan, A. (eds.) CAV 2016. LNCS, vol. 9780, pp. 454–471. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41540-6_25

    Chapter  Google Scholar 

  12. Grinchtein, O., Jonsson, B., Leucker, M.: Learning of event-recording automata. Theor. Comput. Sci. 411(47), 4029–4054 (2010)

    Article  MathSciNet  Google Scholar 

  13. de la Higuera, C.: Grammatical Inference: Learning Automata and Grammars. Cambridge University Press, New York (2010)

    Book  Google Scholar 

  14. Hoeffding, W.: Probability inequalities for sums of bounded random variables. J. Am. Stat. Assoc. 58(301), 13–30 (1963)

    Article  MathSciNet  Google Scholar 

  15. Hungar, H., Niese, O., Steffen, B.: Domain-specific optimization in automata learning. In: Hunt, W.A., Somenzi, F. (eds.) CAV 2003. LNCS, vol. 2725, pp. 315–327. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-45069-6_31

    Chapter  Google Scholar 

  16. Isberner, M., Howar, F., Steffen, B.: The TTT algorithm: a redundancy-free approach to active automata learning. In: Bonakdarpour, B., Smolka, S.A. (eds.) RV 2014. LNCS, vol. 8734, pp. 307–322. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11164-3_26

    Chapter  Google Scholar 

  17. Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22110-1_47

    Chapter  Google Scholar 

  18. Mao, H., Chen, Y., Jaeger, M., Nielsen, T.D., Larsen, K.G., Nielsen, B.: Learning Markov decision processes for model checking. In: Proceedings of the Quantities in Formal Methods, QFM 2012. EPTCS, Paris, France, 28 August 2012, vol. 103, pp. 49–63 (2012)

    Google Scholar 

  19. Mao, H., Chen, Y., Jaeger, M., Nielsen, T.D., Larsen, K.G., Nielsen, B.: Learning deterministic probabilistic automata from a model checking perspective. Mach. Learn. 105(2), 255–299 (2016). https://doi.org/10.1007/s10994-016-5565-9

    Article  MathSciNet  MATH  Google Scholar 

  20. Margaria, T., Niese, O., Raffelt, H., Steffen, B.: Efficient test-based model generation for legacy reactive systems. In: 2004 Ninth IEEE International High-Level Design Validation and Test Workshop, Sonoma Valley, CA, USA, 10–12 November 2004, pp. 95–100. IEEE Computer Society (2004)

    Google Scholar 

  21. Muškardin, E., Aichernig, B.K., Pill, I., Pferscher, A., Tappler, M.: AALpy: an active automata learning library. In: Hou, Z., Ganesh, V. (eds.) ATVA 2021. LNCS, vol. 12971, pp. 67–73. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88885-5_5https://github.com/DES-Lab/AALpy

    Chapter  Google Scholar 

  22. Nerode, A.: Linear automaton transformations. Proc. Am. Math. Soc. 9(4), 541–544 (1958)

    Article  MathSciNet  Google Scholar 

  23. Pferscher, A., Aichernig, B.K.: Learning abstracted non-deterministic finite state machines. In: Casola, V., De Benedictis, A., Rak, M. (eds.) ICTSS 2020. LNCS, vol. 12543, pp. 52–69. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64881-7_4

    Chapter  Google Scholar 

  24. Rivest, R.L., Schapire, R.E.: Inference of finite automata using homing sequences. Inf. Comput. 103(2), 299–347 (1993)

    Article  MathSciNet  Google Scholar 

  25. Shahbaz, M., Groz, R.: Inferring mealy machines. In: Cavalcanti, A., Dams, D.R. (eds.) FM 2009. LNCS, vol. 5850, pp. 207–222. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-05089-3_14

    Chapter  Google Scholar 

  26. Tappler, M., Aichernig, B.K., Bacci, G., Eichlseder, M., Larsen, K.G.: \(L^*\)-based learning of Markov decision processes (extended version). Formal Aspects Comput. 33(4), 575–615 (2021). https://doi.org/10.1007/s00165-021-00536-5

    Article  MathSciNet  MATH  Google Scholar 

  27. Tappler, M., Aichernig, B.K., Bacci, G., Eichlseder, M., Larsen, K.G.: \(L^*\)-based learning of Markov decision processes. In: ter Beek, M.H., McIver, A., Oliveira, J.N. (eds.) FM 2019. LNCS, vol. 11800, pp. 651–669. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30942-8_38

    Chapter  Google Scholar 

  28. Tappler, M., Aichernig, B.K., Bloem, R.: Model-based testing IoT communication via active automata learning. In: 2017 IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, Tokyo, Japan, 13–17 March 2017, pp. 276–287. IEEE Computer Society (2017)

    Google Scholar 

  29. Volpato, M., Tretmans, J.: Approximate active learning of nondeterministic input output transition systems. ECEASST 72 (2015)

    Google Scholar 

Download references

Acknowledgments

This work has been supported by the “University SAL Labs” initiative of Silicon Austria Labs (SAL) and its Austrian partner universities for applied fundamental research for electronic based systems.

Author information

Authors and Affiliations

  1. Silicon Austria Labs, TU Graz - SAL DES Lab, Graz, Austria

    Martin Tappler, Edi Muškardin & Ingo Pill

  2. Institute of Software Technology, Graz University of Technology, Graz, Austria

    Martin Tappler, Edi Muškardin & Bernhard K. Aichernig

Authors
  1. Martin Tappler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edi Muškardin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bernhard K. Aichernig
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ingo Pill
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Edi Muškardin .

Editor information

Editors and Affiliations

  1. University of York, York, UK

    Radu Calinescu

  2. Carnegie Mellon University, Moffett Field, CA, USA

    Corina S. Păsăreanu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tappler, M., Muškardin, E., Aichernig, B.K., Pill, I. (2021). Active Model Learning of Stochastic Reactive Systems. In: Calinescu, R., Păsăreanu, C.S. (eds) Software Engineering and Formal Methods. SEFM 2021. Lecture Notes in Computer Science(), vol 13085. Springer, Cham. https://doi.org/10.1007/978-3-030-92124-8_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-92124-8_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-92123-1

  • Online ISBN: 978-3-030-92124-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation