Skip to main content
SpringerLink
Log in

Toward Implicit Learning for the Compositional Verification of Markov Decision Processes

  • Conference paper
  • First Online:
Verification and Evaluation of Computer and Communication Systems (VECoS 2018)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11181))

  • 316 Accesses

Abstract

In this paper, we propose an automated compositional verification using implicit learning to verify Markov Decision Process (MDP) against probabilistic safety properties. Our approach, denoted ACVuIL (Automatic Compositional Verification using Implicit Learning), starts by encoding implicitly the MDP components by using compact data structures. Then, we use a sound and complete symbolic assume-guarantee reasoning rule to establish the compositional verification process. This rule uses the CDNF learning algorithm to generate automatically the symbolic probabilistic assumptions. Experimental results suggest promising outlooks for our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://sourceforge.net/projects/bull/.

  2. 2.

    https://se.uni-konstanz.de/research1/tools/dipro/.

  3. 3.

    http://www.prismmodelchecker.org/casestudies/index.php.

References

  1. Abate, A., Prandini, M., Lygeros, J., Sastry, S.: Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems. Automatica 44(11), 2724–2734 (2008)

    Article  MathSciNet  Google Scholar 

  2. Baier, C., Katoen, J.-P.: Principles of Model Checking. MIT press, Cambridge (2008)

    MATH  Google Scholar 

  3. Baier, C., Kwiatkowska, M.: Model checking for a probabilistic branching time logic with fairness. Distrib. Comput. 11(3), 125–155 (1998)

    Article  Google Scholar 

  4. Benedikt, M., Lenhardt, R., Worrell, J.: LTL model checking of interval markov chains. In: Piterman, N., Smolka, S.A. (eds.) TACAS 2013. LNCS, vol. 7795, pp. 32–46. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36742-7_3

    Chapter  MATH  Google Scholar 

  5. Bouchekir, R., Boukhedouma, S., Boukala, M.C.: Automatic compositional verification of probabilistic safety properties for inter-organisational workflow processes. In: 2016 6th International Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH), pp. 1–10. IEEE (2016)

    Google Scholar 

  6. Bshouty, N.H.: Exact learning boolean functions via the monotone theory. Inf. Comput. 123(1), 146–153 (1995)

    Article  MathSciNet  Google Scholar 

  7. Burch, J.R., Clarke, E.M., McMillan, K.L., Dill, D.L., Hwang, L.-J.: Symbolic model checking: 1020 states and beyond. Inf. Comput. 98(2), 142–170 (1992)

    Article  Google Scholar 

  8. Chatterjee, K., Sen, K., Henzinger, T.A.: Model-checking \(\omega \)-regular properties of interval markov chains. In: Amadio, R. (ed.) FoSSaCS 2008. LNCS, vol. 4962, pp. 302–317. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-78499-9_22

    Chapter  MATH  Google Scholar 

  9. Chen, Y.-F., Clarke, E.M., Farzan, A., Tsai, M.-H., Tsay, Y.-K., Wang, B.-Y.: Automated assume-guarantee reasoning through implicit learning. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 511–526. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14295-6_44

    Chapter  Google Scholar 

  10. Ciesinski, F., Baier, C., Größer, M., Parker, D.: Generating compact MTBDD-representations from Probmela specifications. In: Havelund, K., Majumdar, R., Palsberg, J. (eds.) SPIN 2008. LNCS, vol. 5156, pp. 60–76. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-85114-1_7

    Chapter  Google Scholar 

  11. Cobleigh, J.M., Giannakopoulou, D., PĂsĂreanu, C.S.: Learning assumptions for compositional verification. In: Garavel, H., Hatcliff, J. (eds.) TACAS 2003. LNCS, vol. 2619, pp. 331–346. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-36577-X_24

    Chapter  MATH  Google Scholar 

  12. Debbi, H., Debbi, A., Bourahla, M.: Debugging of probabilistic systems using structural equation modelling. Int. J. Crit. Comput.-Based Syst. 6(4), 250–274 (2016)

    Article  Google Scholar 

  13. Duflot, M., Fribourg, L., Picaronny, C.: Randomized dining philosophers without fairness assumption. Distrib. Comput. 17(1), 65–76 (2004)

    Article  Google Scholar 

  14. Feng, L.: On learning assumptions for compositional verification of probabilistic systems. Ph.D. thesis, University of Oxford (2013)

    Google Scholar 

  15. Feng, L., Kwiatkowska, M., Parker, D.: Compositional verification of probabilistic systems using learning. In: 7th International Conference on Quantitative Evaluation of Systems (QEST 2010), p. 133 (2010)

    Google Scholar 

  16. Fujita, M., McGeer, P.C., Yang, J.C.-Y.: Multi-terminal binary decision diagrams: an efficient data structure for matrix representation. Form. Methods Syst. Des. 10(2–3), 149–169 (1997)

    Article  Google Scholar 

  17. Hart, S., et al.: Probabilistic temporal logics for finite and bounded models. In: Proceedings of the sixteenth annual ACM symposium on Theory of computing, pp. 1–13. ACM (1984)

    Google Scholar 

  18. Hasson, H., Jonsson, B.: A logic for reasoning about time and probability. Form. Asp. Comput. 6, 512–535 (1994)

    Article  Google Scholar 

  19. He, F., Gao, X., Wang, M., Wang, B.-Y., Zhang, L.: Learning weighted assumptions for compositional verification of markov decision processes. ACM Trans. Softw. Eng. Methodol. (TOSEM) 25(3), 21 (2016)

    Article  Google Scholar 

  20. Israeli, A., Jalfon, M.: Token management schemes and random walks yield self-stabilizing mutual exclusion. In: Proceedings of the Ninth Annual ACM Symposium on Principles of Distributed Computing, pp. 119–131. ACM (1990)

    Google Scholar 

  21. Jansen, N., et al.: Symbolic counterexample generation for large discrete-time markov chains. Sci. Comput. Program. 91, 90–114 (2014)

    Article  Google Scholar 

  22. Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22110-1_47

    Chapter  Google Scholar 

  23. Kwiatkowska, M., Norman, G., Parker, D., Qu, H.: Assume-guarantee verification for probabilistic systems. In: Esparza, J., Majumdar, R. (eds.) TACAS 2010. LNCS, vol. 6015, pp. 23–37. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12002-2_3

    Chapter  MATH  Google Scholar 

  24. Larsen, K.G., Pettersson, P., Yi, W.: Compositional and symbolic model-checking of real-time systems. In: Proceedings of 16th IEEE Real-Time Systems Symposium 1995, pp. 76–87. IEEE (1995)

    Google Scholar 

  25. Lehmann, D., Rabin, M.O.: On the advantages of free choice: a symmetric and fully distributed solution to the dining philosophers problem. In: Proceedings of the 8th ACM SIGPLAN-SIGACT Symposium on Principles of programming languages, pp. 133–138. ACM (1981)

    Google Scholar 

  26. Lehmann, D., Shelah, S.: Reasoning with time and chance. Inf. Control 53(3), 165–198 (1982)

    Article  MathSciNet  Google Scholar 

  27. McMillan, K.L.: Symbolic model checking. In: McMillan, K.L. (ed.) Symbolic Model Checking, pp. 25–60. Springer, Boston (1993). https://doi.org/10.1007/978-1-4615-3190-6_3

    Chapter  Google Scholar 

  28. Parker, D.A.: Implementation of symbolic model checking for probabilistic systems. Ph.D. thesis, University of Birmingham (2003)

    Google Scholar 

  29. Pasareanu, C.S., Giannakopoulou, D., Bobaru, M.G., Cobleigh, J.M., Barringer, H.: Learning to divide and conquer: applying the \(l^{*}\) algorithm to automate assume-guarantee reasoning. Form. Methods Syst. Des. 32, 175–205 (2008)

    Article  Google Scholar 

  30. Pnueli, A., Zuck, L.: Verification of multiprocess probabilistic protocols. Distrib. Comput. 1(1), 53–72 (1986)

    Article  Google Scholar 

  31. Segala, R.: Modeling and verification of randomized distributed real-time systems (1996)

    Google Scholar 

  32. Vardi, M.Y.: Automatic verification of probabilistic concurrent finite state programs. In: 26th Annual Symposium on Foundations of Computer Science (SFCS 1985) (FOCS), pp. 327–338, October 1985

    Google Scholar 

  33. Vardi, M.Y.: Probabilistic linear-time model checking: an overview of the automata-theoretic approach. In: Katoen, J.-P. (ed.) ARTS 1999. LNCS, vol. 1601, pp. 265–276. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-48778-6_16

    Chapter  Google Scholar 

  34. Vardi, M.Y., Wolper, P.: Reasoning about infinite computations. Inf. Comput. 115(1), 1–37 (1994)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. MOVEP, Computer Science Department, University of Science and Technology Houari Boumediene, BP 32 El-Alia, Algiers, Algeria

    Redouane Bouchekir & Mohand Cherif Boukala

Authors
  1. Redouane Bouchekir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohand Cherif Boukala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Redouane Bouchekir .

Editor information

Editors and Affiliations

  1. Uppsala University, Uppsala, Sweden

    Mohamed Faouzi Atig

  2. Grenoble Alpes University, St Martin d’Hères, France

    Saddek Bensalem

  3. Inria Lille - Nord Europe, Villeneuve d’Ascq, France

    Simon Bliudze

  4. École Nationale Superieure de Techniques Avancées, Palaiseau, France

    Bruno Monsuez

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bouchekir, R., Boukala, M.C. (2018). Toward Implicit Learning for the Compositional Verification of Markov Decision Processes. In: Atig, M., Bensalem, S., Bliudze, S., Monsuez, B. (eds) Verification and Evaluation of Computer and Communication Systems. VECoS 2018. Lecture Notes in Computer Science(), vol 11181. Springer, Cham. https://doi.org/10.1007/978-3-030-00359-3_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-00359-3_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-00358-6

  • Online ISBN: 978-3-030-00359-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation