Probabilistic reachability for parametric Markov models

  • Ernst Moritz Hahn
  • Holger Hermanns
  • Lijun Zhang


Given a parametric Markov model, we consider the problem of computing the rational function expressing the probability of reaching a given set of states. To attack this principal problem, Daws has suggested to first convert the Markov chain into a finite automaton, from which a regular expression is computed. Afterwards, this expression is evaluated to a closed form function representing the reachability probability. This paper investigates how this idea can be turned into an effective procedure. It turns out that the bottleneck lies in the growth of the regular expression relative to the number of states (n Θ(log n)). We therefore proceed differently, by tightly intertwining the regular expression computation with its evaluation. This allows us to arrive at an effective method that avoids this blow up in most practical cases. We give a detailed account of the approach, also extending to parametric models with rewards and with non-determinism. Experimental evidence is provided, illustrating that our implementation provides meaningful insights on non-trivial models.


Parametric model analysis Markov chains Model checking Reachability 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abbott, J.: The design of CoCoALib. In: ICMS, pp. 205–215 (2006)Google Scholar
  2. 2.
    Baier C., Ciesinski F., Größer M.: ProbMela and verification of Markov decision processes. SIGMETRICS 32(4), 22–27 (2005)CrossRefGoogle Scholar
  3. 3.
    Baier, C., Hermanns, H.: Weak bisimulation for fully probabilistic processes. In: CAV, pp. 119–130 (1997)Google Scholar
  4. 4.
    Baier C., Katoen J.-P., Hermanns H., Wolf V.: Comparative branching-time semantics for Markov chains. Inf. Comput. 200(2), 149–214 (2005)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Bianco A., de Alfaro L.: Model checking of probabilistic and nondeterministic systems. FSTTCS 15, 499–513 (1995)Google Scholar
  6. 6.
    Bohnenkamp, H.C., van der Stok, P., Hermanns, H., Vaandrager, F.W.: Cost-optimization of the IPv4 zeroconf protocol. In: DSN, pp. 531–540 (2003)Google Scholar
  7. 7.
    Brzozowski J.A., Mccluskey E.J.: Signal flow graph techniques for sequential circuit state diagrams. IEEE Trans. Electron. Comp. EC 12, 67–76 (1963)zbMATHCrossRefGoogle Scholar
  8. 8.
    Chatterjee, K., Henzinger, T., Sen, K.: Model-checking omega-regular properties of interval Markov chains. In: FoSSaCS, pp. 302–317 (2008)Google Scholar
  9. 9.
    Damman, B., Han, T., Katoen, J.-P.: Regular expressions for PCTL counterexamples. In: QEST (2008)Google Scholar
  10. 10.
    Daws, C.: Symbolic and parametric model checking of discrete-time Markov Chains. In: ICTAC, pp. 280–294 (2004)Google Scholar
  11. 11.
    Derisavi S., Hermanns H., Sanders W.H.: Optimal state-space lumping in Markov chains. Inf. Process. Lett. 87(6), 309–315 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Fecher, H., Leucker, M., Wolf, V.: Don’t know in probabilistic Systems. In: SPIN, pp. 71–88 (2006)Google Scholar
  13. 13.
    Geddes K.O., Czapor S.R., Labahn G.: Algorithms for Computer Algebra. Kluwer, Dordrecht (1992)zbMATHCrossRefGoogle Scholar
  14. 14.
    Gruber, H., Johannsen, J.: Optimal lower bounds on regular expression size using communication complexity. In: FoSSaCS, pp. 273–286 (2008)Google Scholar
  15. 15.
    Hahn, E.M., Hermanns, H., Wachter, B., Zhang, L.: PARAM: a model checker for parametric Markov models. In: CAV, 2010 (to appear)Google Scholar
  16. 16.
    Han, T., Katoen, J.-P., Mereacre, A.: Approximate parameter synthesis for probabilistic time-bounded reachability. In: RTSS, pp. 173–182 (2008)Google Scholar
  17. 17.
    Hansson H., Jonsson B.: A logic for reasoning about time and reliability. FAC 6(5), 512–535 (1994)zbMATHCrossRefGoogle Scholar
  18. 18.
    Helmink, L., Sellink, A., Vaandrager, F.W.: Proof-checking a data link protocol. In: TYPES, vol. 806, pp. 127–165. Springer, Heidelberg (1994)Google Scholar
  19. 19.
    Hinton, A., Kwiatkowska, M.Z., Norman, G., Parker, D.: PRISM: a tool for automatic verification of probabilistic systems. In: TACAS, pp. 441–444 (2006)Google Scholar
  20. 20.
    Hopcroft J.E., Motwani R., Ullman J.D.: Introduction to automata theory, languages, and computation, 2nd edn. SIGACT News 32(1), 60–65 (2001)CrossRefGoogle Scholar
  21. 21.
    Hune T., Romijn J., Stoelinga M., Vaandrager F.W.: Linear parametric model checking of timed automata. J. Log. Algebra Program. 52(53), 183–220 (2002)CrossRefMathSciNetGoogle Scholar
  22. 22.
    Ibe O.C., Trivedi K.S.: Stochastic petri net models of polling systems. IEEE J. Selected Areas Commun. 8(9), 1649–1657 (1990)CrossRefGoogle Scholar
  23. 23.
    Jonsson, B., Larsen, K.G.: Specification and refinement of probabilistic processes. In: LICS, pp. 266–277. IEEE Computer Society, New York (1991)Google Scholar
  24. 24.
    Katoen, J.-P., Klink, D., Leucker, M., Wolf, V.: Three-valued abstraction for continuous-time markov chains. In: CAV, vol. 4590, pp. 311–324. Springer, Heidelberg (2007)Google Scholar
  25. 25.
    Kozine I., Utkin L.V.: Interval-valued finite Markov chains. Reliable Comput. 8(2), 97–113 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    Kwiatkowska, M.Z., Norman, G., Parker, D.: Stochastic model checking. In: SFM, pp. 220–270 (2007)Google Scholar
  27. 27.
    Lanotte R., Maggiolo-Schettini A., Troina A.: Parametric probabilistic transition systems for system design and analysis. FAC 19(1), 93–109 (2007)zbMATHCrossRefGoogle Scholar
  28. 28.
    Pnueli A., Zuck L.: Verification of multiprocess probabilistic protocols. Distrib. Comput. 1(1), 53–72 (1986)zbMATHCrossRefGoogle Scholar
  29. 29.
    Reiter M.K., Rubin A.D.: Crowds: anonymity for web transactions. ACM Trans. Inf. Syst. Secur. 1(1), 66–92 (1998)CrossRefGoogle Scholar
  30. 30.
    Sen, K., Viswanathan, M., Agha, G.: Model-checking Markov chains in the presence of uncertainties. In: TACAS, pp. 394–410 (2006)Google Scholar
  31. 31.
    Stewart W.J.: Introduction to the Numerical Solution of Markov Chains. Princeton University Press, Princeton (1994)zbMATHGoogle Scholar
  32. 32.
    Wimmer, R., Derisavi, S., Hermanns, H.: Symbolic partition refinement with dynamic balancing of time and space. In: QEST, pp. 65–74 (2008)Google Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • Ernst Moritz Hahn
    • 1
  • Holger Hermanns
    • 1
    • 2
  • Lijun Zhang
    • 3
  1. 1.Saarland UniversitySaarbrückenGermany
  2. 2.INRIA GrenobleRhône-AlpesFrance
  3. 3.DTU Informatics, Technical University of DenmarkCopenhagenDenmark

Personalised recommendations