Toward Implicit Learning for the Compositional Verification of Markov Decision Processes

  • Redouane BouchekirEmail author
  • Mohand Cherif Boukala
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11181)


In this paper, we propose an automated compositional verification using implicit learning to verify Markov Decision Process (MDP) against probabilistic safety properties. Our approach, denoted ACVuIL (Automatic Compositional Verification using Implicit Learning), starts by encoding implicitly the MDP components by using compact data structures. Then, we use a sound and complete symbolic assume-guarantee reasoning rule to establish the compositional verification process. This rule uses the CDNF learning algorithm to generate automatically the symbolic probabilistic assumptions. Experimental results suggest promising outlooks for our approach.


Probabilistic model checking Compositional verification Symbolic model checking Assume-guarantee paradigm Machine learning CDNF Learning 


  1. 1.
    Abate, A., Prandini, M., Lygeros, J., Sastry, S.: Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems. Automatica 44(11), 2724–2734 (2008)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Baier, C., Katoen, J.-P.: Principles of Model Checking. MIT press, Cambridge (2008)zbMATHGoogle Scholar
  3. 3.
    Baier, C., Kwiatkowska, M.: Model checking for a probabilistic branching time logic with fairness. Distrib. Comput. 11(3), 125–155 (1998)CrossRefGoogle Scholar
  4. 4.
    Benedikt, M., Lenhardt, R., Worrell, J.: LTL model checking of interval markov chains. In: Piterman, N., Smolka, S.A. (eds.) TACAS 2013. LNCS, vol. 7795, pp. 32–46. Springer, Heidelberg (2013). Scholar
  5. 5.
    Bouchekir, R., Boukhedouma, S., Boukala, M.C.: Automatic compositional verification of probabilistic safety properties for inter-organisational workflow processes. In: 2016 6th International Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH), pp. 1–10. IEEE (2016)Google Scholar
  6. 6.
    Bshouty, N.H.: Exact learning boolean functions via the monotone theory. Inf. Comput. 123(1), 146–153 (1995)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Burch, J.R., Clarke, E.M., McMillan, K.L., Dill, D.L., Hwang, L.-J.: Symbolic model checking: 1020 states and beyond. Inf. Comput. 98(2), 142–170 (1992)CrossRefGoogle Scholar
  8. 8.
    Chatterjee, K., Sen, K., Henzinger, T.A.: Model-checking \(\omega \)-regular properties of interval markov chains. In: Amadio, R. (ed.) FoSSaCS 2008. LNCS, vol. 4962, pp. 302–317. Springer, Heidelberg (2008). Scholar
  9. 9.
    Chen, Y.-F., Clarke, E.M., Farzan, A., Tsai, M.-H., Tsay, Y.-K., Wang, B.-Y.: Automated assume-guarantee reasoning through implicit learning. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 511–526. Springer, Heidelberg (2010). Scholar
  10. 10.
    Ciesinski, F., Baier, C., Größer, M., Parker, D.: Generating compact MTBDD-representations from Probmela specifications. In: Havelund, K., Majumdar, R., Palsberg, J. (eds.) SPIN 2008. LNCS, vol. 5156, pp. 60–76. Springer, Heidelberg (2008). Scholar
  11. 11.
    Cobleigh, J.M., Giannakopoulou, D., PĂsĂreanu, C.S.: Learning assumptions for compositional verification. In: Garavel, H., Hatcliff, J. (eds.) TACAS 2003. LNCS, vol. 2619, pp. 331–346. Springer, Heidelberg (2003). Scholar
  12. 12.
    Debbi, H., Debbi, A., Bourahla, M.: Debugging of probabilistic systems using structural equation modelling. Int. J. Crit. Comput.-Based Syst. 6(4), 250–274 (2016)CrossRefGoogle Scholar
  13. 13.
    Duflot, M., Fribourg, L., Picaronny, C.: Randomized dining philosophers without fairness assumption. Distrib. Comput. 17(1), 65–76 (2004)CrossRefGoogle Scholar
  14. 14.
    Feng, L.: On learning assumptions for compositional verification of probabilistic systems. Ph.D. thesis, University of Oxford (2013)Google Scholar
  15. 15.
    Feng, L., Kwiatkowska, M., Parker, D.: Compositional verification of probabilistic systems using learning. In: 7th International Conference on Quantitative Evaluation of Systems (QEST 2010), p. 133 (2010)Google Scholar
  16. 16.
    Fujita, M., McGeer, P.C., Yang, J.C.-Y.: Multi-terminal binary decision diagrams: an efficient data structure for matrix representation. Form. Methods Syst. Des. 10(2–3), 149–169 (1997)CrossRefGoogle Scholar
  17. 17.
    Hart, S., et al.: Probabilistic temporal logics for finite and bounded models. In: Proceedings of the sixteenth annual ACM symposium on Theory of computing, pp. 1–13. ACM (1984)Google Scholar
  18. 18.
    Hasson, H., Jonsson, B.: A logic for reasoning about time and probability. Form. Asp. Comput. 6, 512–535 (1994)CrossRefGoogle Scholar
  19. 19.
    He, F., Gao, X., Wang, M., Wang, B.-Y., Zhang, L.: Learning weighted assumptions for compositional verification of markov decision processes. ACM Trans. Softw. Eng. Methodol. (TOSEM) 25(3), 21 (2016)CrossRefGoogle Scholar
  20. 20.
    Israeli, A., Jalfon, M.: Token management schemes and random walks yield self-stabilizing mutual exclusion. In: Proceedings of the Ninth Annual ACM Symposium on Principles of Distributed Computing, pp. 119–131. ACM (1990)Google Scholar
  21. 21.
    Jansen, N., et al.: Symbolic counterexample generation for large discrete-time markov chains. Sci. Comput. Program. 91, 90–114 (2014)CrossRefGoogle Scholar
  22. 22.
    Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011). Scholar
  23. 23.
    Kwiatkowska, M., Norman, G., Parker, D., Qu, H.: Assume-guarantee verification for probabilistic systems. In: Esparza, J., Majumdar, R. (eds.) TACAS 2010. LNCS, vol. 6015, pp. 23–37. Springer, Heidelberg (2010). Scholar
  24. 24.
    Larsen, K.G., Pettersson, P., Yi, W.: Compositional and symbolic model-checking of real-time systems. In: Proceedings of 16th IEEE Real-Time Systems Symposium 1995, pp. 76–87. IEEE (1995)Google Scholar
  25. 25.
    Lehmann, D., Rabin, M.O.: On the advantages of free choice: a symmetric and fully distributed solution to the dining philosophers problem. In: Proceedings of the 8th ACM SIGPLAN-SIGACT Symposium on Principles of programming languages, pp. 133–138. ACM (1981)Google Scholar
  26. 26.
    Lehmann, D., Shelah, S.: Reasoning with time and chance. Inf. Control 53(3), 165–198 (1982)MathSciNetCrossRefGoogle Scholar
  27. 27.
    McMillan, K.L.: Symbolic model checking. In: McMillan, K.L. (ed.) Symbolic Model Checking, pp. 25–60. Springer, Boston (1993). Scholar
  28. 28.
    Parker, D.A.: Implementation of symbolic model checking for probabilistic systems. Ph.D. thesis, University of Birmingham (2003)Google Scholar
  29. 29.
    Pasareanu, C.S., Giannakopoulou, D., Bobaru, M.G., Cobleigh, J.M., Barringer, H.: Learning to divide and conquer: applying the \(l^{*}\) algorithm to automate assume-guarantee reasoning. Form. Methods Syst. Des. 32, 175–205 (2008)CrossRefGoogle Scholar
  30. 30.
    Pnueli, A., Zuck, L.: Verification of multiprocess probabilistic protocols. Distrib. Comput. 1(1), 53–72 (1986)CrossRefGoogle Scholar
  31. 31.
    Segala, R.: Modeling and verification of randomized distributed real-time systems (1996)Google Scholar
  32. 32.
    Vardi, M.Y.: Automatic verification of probabilistic concurrent finite state programs. In: 26th Annual Symposium on Foundations of Computer Science (SFCS 1985) (FOCS), pp. 327–338, October 1985Google Scholar
  33. 33.
    Vardi, M.Y.: Probabilistic linear-time model checking: an overview of the automata-theoretic approach. In: Katoen, J.-P. (ed.) ARTS 1999. LNCS, vol. 1601, pp. 265–276. Springer, Heidelberg (1999). Scholar
  34. 34.
    Vardi, M.Y., Wolper, P.: Reasoning about infinite computations. Inf. Comput. 115(1), 1–37 (1994)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.MOVEP, Computer Science DepartmentUniversity of Science and Technology Houari BoumedieneAlgiersAlgeria

Personalised recommendations