Abstract
This paper introduces Farkas certificates for lower and upper bounds on minimal and maximal reachability probabilities in Markov decision processes (MDP), which we derive using an MDP-variant of Farkas’ Lemma. The set of all such certificates is shown to form a polytope whose points correspond to witnessing subsystems of the model and the property. Using this correspondence we can translate the problem of finding minimal witnesses to the problem of finding vertices with a maximal number of zeros. While computing such vertices is computationally hard in general, we derive new heuristics from our formulations that exhibit competitive performance compared to state-of-the-art techniques. As an argument that asymptotically better algorithms cannot be hoped for, we show that the decision version of finding minimal witnesses is \({\text {NP}}\)-complete even for acyclic Markov chains.
This work was funded by DFG grant 389792660 as part of TRR 248, the Cluster of Excellence EXC 2050/1 (CeTI, project ID 390696704, as part of Germany’s Excellence Strategy), DFG-projects BA-1679/11-1 and BA-1679/12-1, and the Research Training Group QuantLA (GRK 1763).
Chapter PDF
Similar content being viewed by others
References
Ábrahám, E., Becker, B., Dehnert, C., Jansen, N., Katoen, J., Wimmer, R.: Counterexample generation for discrete-time Markov models: An introductory survey. In: 14th International School on Formal Methods for the Design of Computer, Communication, and Software Systems, SFM 2014. pp. 65–121 (2014), https://doi.org/10.1007/978-3-319-07317-0_3
de Alfaro, L.: Formal verification of probabilistic systems. Ph.D. thesis, Stanford University, Department of Computer Science (1997).
de Alfaro, L.: Temporal logics for the specification of performance and reliability. In: STACS 97. pp. 165–176. Springer, Berlin, Heidelberg (1997).
Aljazzar, H., Leitner-Fischer, F., Leue, S., Simeonov, D.: Dipro - A tool for probabilistic counterexample generation. In: Model Checking Software - 18th International SPIN Workshop 2011. pp. 183–187 (2011), https://doi.org/10.1007/978-3-642-22306-8_13
Aljazzar, H., Leue, S.: Extended directed search for probabilistic timed reachability. In: Formal Modeling and Analysis of Timed Systems, 4th International Conference, FORMATS 2006. pp. 33–51 (2006), https://doi.org/10.1007/11867340_4
Aljazzar, H., Leue, S.: Generation of counterexamples for model checking of Markov decision processes. In: Sixth International Conference on the Quantitative Evaluation of Systems, QEST 2009. pp. 197–206 (2009), https://doi.org/10.1109/QEST.2009.10
Aljazzar, H., Leue, S.: Directed explicit state-space search in the generation of counterexamples for stochastic model checking. IEEE Trans. Software Eng. 36(1), 37–60 (2010), https://doi.org/10.1109/TSE.2009.57
Amaldi, E., Kann, V.: On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theoretical Computer Science 209(1), 237–260 (1998), http://www.sciencedirect.com/science/article/pii/S0304397597001151
Andrés, M.E., D’Argenio, P.R., van Rossum, P.: Significant diagnostic counterexamples in probabilistic model checking. In: Hardware and Software: Verification and Testing, 4th International Haifa Verification Conference, HVC 2008. pp. 129–148 (2008), https://doi.org/10.1007/978-3-642-01702-5_15
Aspnes, J., Herlihy, M.: Fast randomized consensus using shared memory. Journal of Algorithms 11(3), 441–461 (1990), https://doi.org/10.1016/0196-6774(90)90021-6
Avis, D., Fukuda, K.: A pivoting algorithm for convex hulls and vertex enumeration of arrangements and polyhedra. Discrete & Computational Geometry 8, 295–313 (1992), https://doi.org/10.1007/BF02293050
Avis, D., Fukuda, K.: Reverse search for enumeration. Discrete Applied Mathematics 65, 21–46 (1993).
Baier, C., Katoen, J.P.: Principles of Model Checking (Representation and Mind Series). The MIT Press, Cambridge, MA (2008).
Balinski, M.L.: An algorithm for finding all vertices of convex polyhedral sets. Journal of the Society for Industrial and Applied Mathematics 9(1), 72–88 (1961), https://doi.org/10.1137/0109008
Bernasconi, A., Menghi, C., Spoletini, P., Zuck, L.D., Ghezzi, C.: From model checking to a temporal proof for partial models. In: Software Engineering and Formal Methods - 15th International Conference, SEFM 2017. pp. 54–69 (2017), https://doi.org/10.1007/978-3-319-66197-1_4
Bianco, A., de Alfaro, L.: Model checking of probabilistic and nondeterministic systems. In: Foundations of Software Technology and Theoretical Computer Science. pp. 499–513. Springer, Berlin, Heidelberg (1995).
Blum, M., Kannan, S.: Designing programs that check their work. Journal of the ACM 42(1), 269–291 (1995), https://doi.org/10.1145/200836.200880
Braitling, B., Wimmer, R., Becker, B., Jansen, N., Ábrahám, E.: Counterexample generation for Markov chains using SMT-based bounded model checking. In: Formal Techniques for Distributed Systems - Joint 13th IFIP WG 6.1 International Conference, FMOODS 2011, and 31st IFIP WG 6.1 International Conference, FORTE 2011. pp. 75–89 (2011), https://doi.org/10.1007/978-3-642-21461-5_5
Brázdil, T., Chatterjee, K., Chmelik, M., Fellner, A., Kretínský, J.: Counterexample explanation by learning small strategies in Markov decision processes. In: Computer Aided Verification - 27th International Conference, CAV 2015. pp. 158–177 (2015), https://doi.org/10.1007/978-3-319-21690-4_10
Brázdil, T., Chatterjee, K., Chmelík, M., Forejt, V., Křetínský, J., Kwiatkowska, M., Parker, D., Ujma, M.: Verification of Markov Decision Processes Using Learning Algorithms. In: Automated Technology for Verification and Analysis (ATVA 2014). pp. 98–114 (2014), https://doi.org/10.1007/978-3-319-11936-6_8
Bremner, D., Fukuda, K., Marzetta, A.: Primal–dual methods for vertex and facet enumeration. Discrete & Computational Geometry 20(3), 333–357 (1998), https://doi.org/10.1007/PL00009389
Bussieck, M.R., Lübbecke, M.E.: The vertex set of a 0/1 polytope is strongly \(\cal{P}\)-enumerable. Computational Geometry Theory and Applications 11(2), 103–109 (1998).
Ceska, M., Hensel, C., Junges, S., Katoen, J.: Counterexample-driven synthesis for probabilistic program sketches. In: Formal Methods - The Next 30 Years - Third World Congress, FM 2019. pp. 101–120 (2019), https://doi.org/10.1007/978-3-030-30942-8_8
Chadha, R., Viswanathan, M.: A counterexample-guided abstraction-refinement framework for Markov decision processes. ACM Transactions on Computational Logic 12(1), 1:1–1:49 (2010), https://doi.org/10.1145/1838552.1838553
Chatterjee, K., Chmelik, M., Daca, P.: CEGAR for qualitative analysis of probabilistic systems. In: Computer Aided Verification - 26th International Conference, CAV 2014. pp. 473–490 (2014), https://doi.org/10.1007/978-3-319-08867-9_31
Ciesinski, F., Baier, C., Größer, M., Klein, J.: Reduction techniques for model checking Markov decision processes. In: 2008 Fifth International Conference on Quantitative Evaluation of Systems. pp. 45–54 (2008). https://doi.org/10.1109/QEST.2008.45
Clarke, E.M., Grumberg, O., Jha, S., Lu, Y., Veith, H.: Counterexample-guided abstraction refinement for symbolic model checking. J. ACM 50(5), 752–794 (2003), https://doi.org/10.1145/876638.876643
Clarke, E.M., Jha, S., Lu, Y., Veith, H.: Tree-like counterexamples in model checking. In: 17th IEEE Symposium on Logic in Computer Science (LICS 2002). pp. 19–29 (2002), https://doi.org/10.1109/LICS.2002.1029814
Clarke, E.M., Veith, H.: Counterexamples revisited: Principles, algorithms, applications. In: Verification: Theory and Practice, Essays Dedicated to Zohar Manna on the Occasion of His 64th Birthday. pp. 208–224 (2003), https://doi.org/10.1007/978-3-540-39910-0_9
Colón, M., Sankaranarayanan, S., Sipma, H.: Linear invariant generation using non-linear constraint solving. In: Computer Aided Verification, 15th International Conference, CAV 2003. pp. 420–432 (2003).
Courcoubetis, C., Yannakakis, M.: Verifying temporal properties of finite-state probabilistic programs. In: Proceedings of the 29th Annual Symposium on Foundations of Computer Science. pp. 338–345. SFCS ’88, IEEE Computer Society (1988), https://doi.org/10.1109/SFCS.1988.21950
Courcoubetis, C., Yannakakis, M.: The complexity of probabilistic verification. Journal of the ACM 42(4), 857–907 (1995), http://doi.acm.org/10.1145/210332.210339
Damman, B., Han, T., Katoen, J.: Regular expressions for PCTL counterexamples. In: Fifth International Conference on the Quantitative Evaluaiton of Systems (QEST 2008). pp. 179–188 (2008), https://doi.org/10.1109/QEST.2008.11
D’Argenio, P.R., Jeannet, B., Jensen, H.E., Larsen, K.G.: Reachability analysis of probabilistic systems by successive refinements. In: Process Algebra and Probabilistic Methods, Performance Modeling and Verification: Joint International Workshop, PAPM-PROBMIV 2001. pp. 39–56 (2001), https://doi.org/10.1007/3-540-44804-7_3
Dyer, M.E.: The complexity of vertex enumeration methods. Mathematics of Operations Research 8(3), 381–402 (1983), https://doi.org/10.1287/moor.8.3.381
Dyer, M.E., Proll, L.G.: An algorithm for determining all extreme points of a convex polytope. Mathematical Programming 12(1), 81–96 (1977), https://doi.org/10.1007/BF01593771
Etessami, K., Kwiatkowska, M., Vardi, M.Y., Yannakakis, M.: Multi-Objective Model Checking of Markov Decision Processes. Logical Methods in Computer Science 4(4) (2008), https://lmcs.episciences.org/990
Farkas, J.: Theorie der einfachen ungleichungen. Journal für die reine und angewandte Mathematik 124, 1–27 (1902), http://eudml.org/doc/149129
Forejt, V., Kwiatkowska, M.Z., Norman, G., Parker, D., Qu, H.: Quantitative multi-objective verification for probabilistic systems. In: Tools and Algorithms for the Construction and Analysis of Systems - 17th International Conference, TACAS 2011. pp. 112–127 (2011), https://doi.org/10.1007/978-3-642-19835-9_11
Fukuda, K., Liebling, T.M., Margot, F.: Analysis of backtrack algorithms for listing all vertices and all faces of a convex polyhedron. Computational Geometry 8(1), 1–12 (1997), http://www.sciencedirect.com/science/article/pii/0925772195000496
Fukuda, K., Prodon, A.: Double description method revisited. In: Combinatorics and Computer Science, 8th Franco-Japanese and 4th Franco-Chinese Conference 1995. pp. 91–111 (1995), https://doi.org/10.1007/3-540-61576-8_77
Funke, F., Jantsch, S., Baier, C.: Farkas certificates and minimal witnesses for probabilistic reachability constraints (2019), https://arxiv.org/abs/1910.10636.
Gurobi Optimization LLC, L.: Gurobi optimizer reference manual (2019), http://www.gurobi.com.
Han, T., Katoen, J.: Counterexamples in probabilistic model checking. In: Tools and Algorithms for the Construction and Analysis of Systems (TACAS 2007). pp. 72–86 (2007), https://doi.org/10.1007/978-3-540-71209-1_8
Han, T., Katoen, J., Damman, B.: Counterexample generation in probabilistic model checking. IEEE Transactions on Software Engineering 35(2), 241–257 (2009), https://doi.org/10.1109/TSE.2009.5
Hart, S., Sharir, M., Pnueli, A.: Termination of probabilistic concurrent program. ACM Transactions on Programming Languages and Systems 5(3), 356–380 (1983), http://doi.acm.org/10.1145/2166.357214
Helmink, L., Sellink, M.P.A., Vaandrager, F.W.: Proof-checking a data link protocol. In: Types for Proofs and Programs, International Workshop TYPES’93. pp. 127–165 (1993), https://doi.org/10.1007/3-540-58085-9_75
Hermanns, H., Wachter, B., Zhang, L.: Probabilistic CEGAR. In: Computer Aided Verification, 20th International Conference, CAV 2008. pp. 162–175 (2008), https://doi.org/10.1007/978-3-540-70545-1_16
Jansen, N., Ábrahám, E., Katelaan, J., Wimmer, R., Katoen, J., Becker, B.: Hierarchical counterexamples for discrete-time Markov chains. In: Automated Technology for Verification and Analysis, 9th International Symposium, ATVA 2011. pp. 443–452 (2011), https://doi.org/10.1007/978-3-642-24372-1_33
Jansen, N., Ábrahám, E., Volk, M., Wimmer, R., Katoen, J., Becker, B.: The COMICS tool - computing minimal counterexamples for dtmcs. In: Automated Technology for Verification and Analysis - 10th International Symposium, ATVA 2012. pp. 349–353 (2012), https://doi.org/10.1007/978-3-642-33386-6_27
Jansen, N., Ábrahám, E., Zajzon, B., Wimmer, R., Schuster, J., Katoen, J., Becker, B.: Symbolic counterexample generation for discrete-time Markov chains. In: Formal Aspects of Component Software, 9th International Symposium, FACS 2012. pp. 134–151 (2012), https://doi.org/10.1007/978-3-642-35861-6_9
Jansen, N., Wimmer, R., Ábrahám, E., Zajzon, B., Katoen, J., Becker, B., Schuster, J.: Symbolic counterexample generation for large discrete-time Markov chains. Science of Computer Programming 91, 90–114 (2014), https://doi.org/10.1016/j.scico.2014.02.001
Jr., M.C., Jansen, N., Junges, S., Katoen, J.: Shepherding hordes of Markov chains. In: Tools and Algorithms for the Construction and Analysis of Systems - 25th International Conference, TACAS 2019. pp. 172–190 (2019), https://doi.org/10.1007/978-3-030-17465-1_10
Karp, R.M.: Reducibility among combinatorial problems. In: Complexity of Computer Computations: Proceedings of a symposium on the Complexity of Computer Computations, 1972. pp. 85–103. Springer, US, Boston, MA (1972).
Khachiyan, L., Boros, E., Borys, K., Elbassioni, K., Gurvich, V.: Generating all vertices of a polyhedron is hard. Discrete & Computational Geometry 39(1), 174–190 (2008), https://doi.org/10.1007/s00454-008-9050-5
Kuntz, M., Leitner-Fischer, F., Leue, S.: From probabilistic counterexamples via causality to fault trees. In: Proceedings of the 30th International Conference on Computer Safety, Reliability, and Security (SAFECOMP). pp. 71–84 (2011), https://doi.org/10.1007/978-3-642-24270-0_6
Kupferman, O., Vardi, M.Y.: From complementation to certification. In: Tools and Algorithms for the Construction and Analysis of Systems, 10th International Conference, TACAS 2004. pp. 591–606 (2004), https://doi.org/10.1007/978-3-540-24730-2_43
Kwiatkowska, M.Z., Norman, G., Parker, D.: PRISM 4.0: Verification of probabilistic real-time systems. In: Computer Aided Verification - 23rd International Conference, CAV 2011. pp. 585–591 (2011), https://doi.org/10.1007/978-3-642-22110-1_47
Kwiatkowska, M.Z., Norman, G., Parker, D.: The PRISM benchmark suite. In: Ninth International Conference on Quantitative Evaluation of Systems, QEST 2012. pp. 203–204 (2012), https://doi.org/10.1109/QEST.2012.14
Kwiatkowska, M.Z., Norman, G., Segala, R.: Automated verification of a randomized distributed consensus protocol using cadence SMV and PRISM. In: Computer Aided Verification, 13th International Conference, CAV 2001. pp. 194–206 (2001), https://doi.org/10.1007/3-540-44585-4_17
Kwiatkowska, M.Z., Norman, G., Sproston, J., Wang, F.: Symbolic model checking for probabilistic timed automata. Information and Computation 205(7), 1027–1077 (2007), https://doi.org/10.1016/j.ic.2007.01.004
Mangasarian, O.: Nonlinear Programming. Classics in Applied Mathematics, Society for Industrial and Applied Mathematics (1994).
Mattheiss, T.H.: An algorithm for determining irrelevant constraints and all vertices in systems of linear inequalities. Operations Research 21(1), 247–260 (1973), http://www.jstor.org/stable/169104
McConnell, R.M., Mehlhorn, K., Näher, S., Schweitzer, P.: Certifying algorithms. Computer Science Review 5(2), 119–161 (2011), https://doi.org/10.1016/j.cosrev.2010.09.009
Naiman, D.Q., Scheinerman, E.R.: Arbitrage and geometry. Preprint (2017), https://arxiv.org/abs/1709.07446.
Namjoshi, K.S.: Certifying model checkers. In: Computer Aided Verification, 13th International Conference, CAV 2001. pp. 2–13 (2001), https://doi.org/10.1007/3-540-44585-4_2
Peled, D.A., Pnueli, A., Zuck, L.D.: From falsification to verification. In: FST TCS 2001: Foundations of Software Technology and Theoretical Computer Science. pp. 292–304 (2001), https://doi.org/10.1007/3-540-45294-X_25
Provan, J.S.: Efficient enumeration of the vertices of polyhedra associated with network LP’s. Mathematical Programming 63(1), 47–64 (1994), https://doi.org/10.1007/BF01582058
Reiter, M.K., Rubin, A.D.: Crowds: Anonymity for web transactions. ACM Transactions on Information and System Security 1(1), 66–92 (1998), https://doi.org/10.1145/290163.290168
Schrijver, A.: Theory of Linear and Integer Programming. John Wiley & Sons Inc., New York, NY, USA (1986).
Schrijver, A.: A course in combinatorial optimization. Lecture notes (2017), https://homepages.cwi.nl/~lex/files/dict.pdf.
Shmatikov, V.: Probabilistic analysis of an anonymity system. Journal of Computer Security 12(3-4), 355–377 (2004).
Vardi, M.Y.: Automatic verification of probabilistic concurrent finite state programs. In: Proceedings of the 26th Annual Symposium on Foundations of Computer Science. pp. 327–338. SFCS ’85, IEEE Computer Society (1985), https://doi.org/10.1109/SFCS.1985.12
Vardi, M.Y., Wolper, P.: An automata-theoretic approach to automatic program verification (preliminary report). In: Proceedings of the Symposium on Logic in Computer Science (LICS 86). pp. 332–344 (1986).
Vohra, R.V.: The ubiquitous farkas lemma. In: Perspectives in Operations Research: Papers in Honor of Saul Gass’ 80th Birthday. pp. 199–210. Springer US, Boston, MA (2006), https://doi.org/10.1007/978-0-387-39934-8_11
Wimmer, R., Braitling, B., Becker, B.: Counterexample generation for discrete-time Markov chains using bounded model checking. In: Verification, Model Checking, and Abstract Interpretation, 10th International Conference, VMCAI 2009. pp. 366–380 (2009), https://doi.org/10.1007/978-3-540-93900-9_29
Wimmer, R., Jansen, N., Ábrahám, E., Becker, B., Katoen, J.: Minimal critical subsystems for discrete-time markov models. In: Tools and Algorithms for the Construction and Analysis of Systems - 18th International Conference, TACAS 2012. pp. 299–314 (2012), https://doi.org/10.1007/978-3-642-28756-5_21
Wimmer, R., Jansen, N., Ábrahám, E., Katoen, J., Becker, B.: Minimal counterexamples for linear-time probabilistic verification. Theoretical Computer Science 549, 61–100 (2014), https://doi.org/10.1016/j.tcs.2014.06.020
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2020 The Author(s)
About this paper
Cite this paper
Funke, F., Jantsch, S., Baier, C. (2020). Farkas Certificates and Minimal Witnesses for Probabilistic Reachability Constraints. In: Biere, A., Parker, D. (eds) Tools and Algorithms for the Construction and Analysis of Systems. TACAS 2020. Lecture Notes in Computer Science(), vol 12078. Springer, Cham. https://doi.org/10.1007/978-3-030-45190-5_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-45190-5_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-45189-9
Online ISBN: 978-3-030-45190-5
eBook Packages: Computer ScienceComputer Science (R0)