Counterexamples for Expected Rewards

  • Tim Quatmann
  • Nils Jansen
  • Christian Dehnert
  • Ralf Wimmer
  • Erika Ábrahám
  • Joost-Pieter Katoen
  • Bernd Becker
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9109)


The computation of counterexamples for probabilistic systems has gained a lot of attention during the last few years. All of the proposed methods focus on the situation when the probabilities of certain events are too high. In this paper we investigate how counterexamples for properties concerning expected costs (or, equivalently, expected rewards) of events can be computed. We propose methods to extract a minimal subsystem which already leads to costs beyond the allowed bound. Besides these exact methods, we present heuristic approaches based on path search and on best-first search, which are applicable to very large systems when deriving a minimum subsystem becomes infeasible due to the system size. Experiments show that we can compute counterexamples for systems with millions of states.


Model Check Target State Mixed Integer Linear Program Path Search Expected Reward 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Clarke, E.M.: The birth of model checking. In: Grumberg, O., Veith, H. (eds.) 25MC Festschrift 2008. LNCS, vol. 5000, pp. 1–26. Springer, Heidelberg (2008)Google Scholar
  2. 2.
    Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: Verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  3. 3.
    Norman, G., Parker, D., Kwiatkowska, M.Z., Shukla, S.K.: Evaluating the reliability of NAND multiplexing with PRISM. IEEE Trans. on CAD of Integrated Circuits and Systems 24(10), 1629–1637 (2005)CrossRefGoogle Scholar
  4. 4.
    Norman, G., Shmatikov, V.: Analysis of probabilistic contract signing. Journal of Computer Security 14(6), 561–589 (2006)Google Scholar
  5. 5.
    Kwiatkowska, M.Z., Norman, G., Parker, D.: Using probabilistic model checking in systems biology. SIGMETRICS Performance Evaluation Review 35(4), 14–21 (2008)CrossRefGoogle Scholar
  6. 6.
    Alur, R., Henzinger, T., Vardi, M.: Theory in practice for system design and verification. ACM Siglog News 2(1), 46–51 (2015)Google Scholar
  7. 7.
    Behrmann, G., Larsen, K.G., Rasmussen, J.I.: Priced timed automata: Algorithms and applications. In: de Boer, F.S., Bonsangue, M.M., Graf, S., de Roever, W.-P. (eds.) FMCO 2004. LNCS, vol. 3657, pp. 162–182. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  8. 8.
    Chatterjee, K., Doyen, L., Henzinger, T.A., Raskin, J.F.: Generalized Mean-payoff and Energy Games. In: Proc. of FSTTCS. LIPIcs, vol. 8, pp. 505–516. Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik (2010)Google Scholar
  9. 9.
    Boker, U., Henzinger, T.A., Radhakrishna, A.: Battery transition systems. In: Proc. of POPL, pp. 595–606. ACM Press (2014)Google Scholar
  10. 10.
    Howard, R.A.: Dynamic Probabilistic Systems; Volume I: Markov models. John Wiley & Sons (1971)Google Scholar
  11. 11.
    Baier, C., Hahn, E.M., Haverkort, B.R., Hermanns, H., Katoen, J.P.: Model checking for performability. Mathematical Structures in Computer Science 23(4), 751–795 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    Andova, S., Hermanns, H., Katoen, J.P.: Discrete-time rewards model-checked. In: Larsen, K.G., Niebert, P. (eds.) FORMATS 2003. LNCS, vol. 2791, pp. 88–104. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  13. 13.
    Ábrahám, E., Becker, B., Dehnert, C., Jansen, N., Katoen, J.-P., Wimmer, R.: Counterexample generation for discrete-time Markov models: An introductory survey. In: Bernardo, M., Damiani, F., Hähnle, R., Johnsen, E.B., Schaefer, I. (eds.) SFM 2014. LNCS, vol. 8483, pp. 65–121. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  14. 14.
    Aljazzar, H., Leue, S.: Directed explicit state-space search in the generation of counterexamples for stochastic model checking. IEEE Trans. on Software Engineering 36(1), 37–60 (2010)CrossRefGoogle Scholar
  15. 15.
    Jansen, N., Ábrahám, E., Katelaan, J., Wimmer, R., Katoen, J.-P., Becker, B.: Hierarchical counterexamples for discrete-time Markov chains. In: Bultan, T., Hsiung, P.-A. (eds.) ATVA 2011. LNCS, vol. 6996, pp. 443–452. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  16. 16.
    Gurobi Optimization, Inc.: Gurobi optimizer reference manual (2013),
  17. 17.
    Baier, C., Katoen, J.P.: Principles of Model Checking. The MIT Press (2008)Google Scholar
  18. 18.
    Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman & Co Ltd. (1979)Google Scholar
  19. 19.
    Schrijver, A.: Theory of Linear and Integer Programming. Wiley (1986)Google Scholar
  20. 20.
    Han, T., Katoen, J.P., Damman, B.: Counterexample generation in probabilistic model checking. IEEE Trans. on Software Engineering 35(2), 241–257 (2009)CrossRefGoogle Scholar
  21. 21.
    Wimmer, R., Jansen, N., Ábrahám, E., Katoen, J.P., Becker, B.: Minimal counterexamples for linear-time probabilistic verification. Theoretical Computer Science 549, 61–100 (2014)CrossRefzbMATHMathSciNetGoogle Scholar
  22. 22.
    Reiter, M.K., Rubin, A.D.: Crowds: Anonymity for web transactions. ACM Trans. on Information and System Security 1(1), 66–92 (1998)CrossRefGoogle Scholar
  23. 23.
    Herman, T.: Probabilistic self-stabilization. Information Processing Letters 35(2), 63–67 (1990)CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    Even, S., Goldreich, O., Lempel, A.: A randomized protocol for signing contracts. Communications of the ACM 28(6), 637–647 (1985)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Tim Quatmann
    • 1
  • Nils Jansen
    • 1
  • Christian Dehnert
    • 1
  • Ralf Wimmer
    • 2
  • Erika Ábrahám
    • 1
  • Joost-Pieter Katoen
    • 1
  • Bernd Becker
    • 2
  1. 1.RWTH Aachen UniversityAachenGermany
  2. 2.Albert-Ludwigs-UniversitätFreiburgGermany

Personalised recommendations