On Reduction Criteria for Probabilistic Reward Models

  • Marcus Größer
  • Gethin Norman
  • Christel Baier
  • Frank Ciesinski
  • Marta Kwiatkowska
  • David Parker
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4337)


In recent papers, the partial order reduction approach has been adapted to reason about the probabilities for temporal properties in concurrent systems with probabilistic behaviours. This paper extends these results by presenting reduction criteria for a probabilistic branching time logic that allows specification of constraints on quantitative measures given by a reward or cost function for the actions of the system.


Model Check Markov Decision Process Reward Structure Reduction Criterion Discount Reward 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andova, S., Hermanns, H., Katoen, J.-P.: Discrete-time rewards model-checked. In: Larsen, K.G., Niebert, P. (eds.) FORMATS 2003. LNCS, vol. 2791, pp. 88–104. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Asadi, M., Huber, M.: Action dependent state space abstraction for hierarchical learning systems. In: Proc. IASTED, Insbruck, Austria (2005)Google Scholar
  3. 3.
    Baier, C., Ciesinski, F., Groesser, M.: Quantitative analysis of distributed randomized protocols. In: Proc. FMICS (2005)Google Scholar
  4. 4.
    Baier, C., D’Argenio, P., Größer, M.: Partial order reduction for probabilistic branching time. In: Proc. QAPL (2005)Google Scholar
  5. 5.
    Baier, C., Größer, M., Ciesinski, F.: Partial order reduction for probabilistic systems. In: QEST 2004 [28], pp. 230–239 (2004)Google Scholar
  6. 6.
    Bianco, A., de Alfaro, L.: Model checking of probabilistic and nondeterministic systems. In: Thiagarajan, P.S. (ed.) FSTTCS 1995. LNCS, vol. 1026, pp. 499–513. Springer, Heidelberg (1995)Google Scholar
  7. 7.
    D’Argenio, P.R., Niebert, P.: Partial order reduction on concurrent probabilistic programs. In: QEST 2004 [28], pp. 240–249 (2004)Google Scholar
  8. 8.
    de Alfaro, L.: Formal Verification of Probabilistic Systems. PhD thesis, Stanford University, Department of Computer Science (1997)Google Scholar
  9. 9.
    de Alfaro, L.: Temporal logics for the specification of performance and reliability. In: Reischuk, R., Morvan, M. (eds.) STACS 1997. LNCS, vol. 1200, pp. 165–179. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  10. 10.
    de Alfaro, L.: How to specify and verify the long-run average behavior of probabilistic systems. In: Proc. 13th LICS, pp. 454–465. IEEE Press, Los Alamitos (1998)Google Scholar
  11. 11.
    Dean, T., Givan, R., Leach, S.: Model reduction techniques for computing approximately optimal solutions for markov decision processes. In: Proc. 13th UAI, pp. 124–131. Morgan Kaufmann Publishers, San Francisco (1997)Google Scholar
  12. 12.
    Gerth, R., Kuiper, R., Peled, D., Penczek, W.: A partial order approach to branching time logic model checking. In: Proc. 3rd ISTCS 1995, pp. 130–139. IEEE Press, Los Alamitos (1995)Google Scholar
  13. 13.
    Godefroid, P.: Partial-Order Methods for the Verification of Concurrent Systems. LNCS, vol. 1032. Springer, Heidelberg (1996)Google Scholar
  14. 14.
    Griffioen, D., Vaandrager, F.: Normed simulations. In: Y. Vardi, M. (ed.) CAV 1998. LNCS, vol. 1427, pp. 332–344. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  15. 15.
    Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Aspects of Computing 6(5), 512–535 (1994)MATHCrossRefGoogle Scholar
  16. 16.
    Hinton, A., Kwiatkowska, M., Norman, G., Parker, D.: PRISM: A tool for automatic verification of probabilistic systems. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, pp. 441–444. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  17. 17.
    Holzmann, G.: The SPIN Model Checker, Primer and Reference Manual. Addison-Wesley, Reading (2003)Google Scholar
  18. 18.
    Jones, C.: Probabilistic Non-Determinism. PhD thesis, University of Edinburgh (1990)Google Scholar
  19. 19.
    Jonsson, B., Larsen, K.: Specification and refinement of probabilistic processes. In: Proc. LICS, pp. 266–277. IEEE CS Press, Los Alamitos (1991)Google Scholar
  20. 20.
    Kurshan, R.P., Levin, V., Minea, M., Peled, D.A., Yenigün, H.: Static partial order reduction. In: Steffen, B. (ed.) TACAS 1998. LNCS, vol. 1384, pp. 345–357. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  21. 21.
    Larsen, K., Skou, A.: Bisimulation through probabilistic testing. Information and Computation 94(1), 1–28 (1991)MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Namjoshi, K.S.: A simple characterization of stuttering bisimulation. In: Ramesh, S., Sivakumar, G. (eds.) FST TCS 1997. LNCS, vol. 1346, pp. 284–296. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  23. 23.
    Pekergin, N., Younès, S.: Stochastic model checking with stochastic comparison. In: Bravetti, M., Kloul, L., Zavattaro, G. (eds.) EPEW/WS-EM 2005. LNCS, vol. 3670, pp. 109–123. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  24. 24.
    Peled, D.: All from one, one for all: On model checking using representatives. In: Courcoubetis, C. (ed.) CAV 1993. LNCS, vol. 697, pp. 409–423. Springer, Heidelberg (1993)Google Scholar
  25. 25.
    Peled, D.: Partial order reduction: Linear and branching time logics and process algebras. In: [26], pp. 79–88 (1996)Google Scholar
  26. 26.
    Peled, D., Pratt, V., Holzmann, G. (eds.): Partial Order Methods in Verification. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol. 29(10). American Mathematical Society (1997)Google Scholar
  27. 27.
    Puterman, M.L.: Markov Decision Processes—Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc., New York (1994)MATHGoogle Scholar
  28. 28.
    Proceedings of the 1st International Conference on Quantitative Evaluation of SysTems (QEST 2004). Enschede, the Netherlands. IEEE Computer Society Press (2004)Google Scholar
  29. 29.
    Ravindran, B., Barto, A.G.: Model minimization in hierarchical reinforcement learning. In: Koenig, S., Holte, R.C. (eds.) SARA 2002. LNCS, vol. 2371, pp. 196–211. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  30. 30.
    Segala, R.: Modeling and Verification of Randomized Distributed Real-Time Systems. PhD thesis, Massachusetts Institute of Technology (1995)Google Scholar
  31. 31.
    Segala, R., Lynch, N.: Probabilistic simulations for probabilistic processes. Nordic Journal of Computing 2(2), 250–273 (1995)MATHMathSciNetGoogle Scholar
  32. 32.
    Valmari, A.: Stubborn set methods for process algebras. In: [26], pp. 79–88 (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Marcus Größer
    • 1
  • Gethin Norman
    • 2
  • Christel Baier
    • 1
  • Frank Ciesinski
    • 1
  • Marta Kwiatkowska
    • 2
  • David Parker
    • 2
  1. 1.Institut für Informatik IUniversität BonnGermany
  2. 2.School of Computer ScienceUniversity of BirminghamEdgbastonUnited Kingdom

Personalised recommendations