Cost vs. Time in Stochastic Games and Markov Automata

  • Hassan Hatefi
  • Bettina BraitlingEmail author
  • Ralf Wimmer
  • Luis María Ferrer Fioriti
  • Holger Hermanns
  • Bernd Becker
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9409)


Costs and rewards are important tools for analysing quantitative aspects of models like energy consumption and costs of maintenance and repair. Under the assumption of transient costs, this paper considers the computation of expected cost-bounded rewards and cost-bounded reachability for Markov automata and stochastic games. We give a transformation of this class of properties to expected time-bounded rewards and time-bounded reachability, which can be computed by available algorithms. We prove the correctness of the transformation and show its effectiveness on a number of case studies.


Stochastic Game Reward Structure Markovian Transition Fault Tree Analysis Cost Budget 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andova, S., Hermanns, H., Katoen, J.-P.: Discrete-time rewards model-checked. In: Larsen, K.G., Niebert, P. (eds.) FORMATS 2003. LNCS, vol. 2791, pp. 88–104. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  2. 2.
    Ash, R.B., Doléans-Dade, C.A.: Probability & Measure Theory. Academic Press, 2nd edn. (1999)Google Scholar
  3. 3.
    Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.-P.: On the logical characterisation of performability properties. In: Welzl, E., Montanari, U., Rolim, J.D.P. (eds.) ICALP 2000. LNCS, vol. 1853, p. 780. Springer, Heidelberg (2000) CrossRefGoogle Scholar
  4. 4.
    Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.: Reachability in continuous-time Markov reward decision processes. In: Logic and Automata: History and Perspectives [in Honor of Wolfgang Thomas]. Texts in Logic and Games, vol. 2, pp. 53–72. Amsterdam University Press (2008)Google Scholar
  5. 5.
    Baier, C., Katoen, J.P.: Principles of Model Checking. The MIT Press (2008)Google Scholar
  6. 6.
    Boudali, H., Crouzen, P., Stoelinga, M.: A rigorous, compositional, and extensible framework for dynamic fault tree analysis. IEEE Trans. Dependable Sec. Comput. 7(2), 128–143 (2010)CrossRefGoogle Scholar
  7. 7.
    Braitling, B., Ferrer Fioriti, L.M., Hatefi, H., Wimmer, R., Becker, B., Hermanns, H.: Abstraction-based computation of reward measures for markov automata. In: D’Souza, D., Lal, A., Larsen, K.G. (eds.) VMCAI 2015. LNCS, vol. 8931, pp. 172–189. Springer, Heidelberg (2015) Google Scholar
  8. 8.
    Brázdil, T., Forejt, V., Krcál, J., Kretínský, J., Kucera, A.: Continuous-time stochastic games with time-bounded reachability. Information and Computation 224, 46–70 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Bruno, J.L., Downey, P.J., Frederickson, G.N.: Sequencing tasks with exponential service times to minimize the expected flow time or makespan. Journal of the ACM 28(1), 100–113 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Buchholz, P., Schulz, I.: Numerical analysis of continuous time Markov decision processes over finite horizons. Computers & Operations Research 38(3), 651–659 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Cloth, L., Katoen, J., Khattri, M., Pulungan, R.: Model checking Markov reward models with impulse rewards. In: Proceedings of DSN, pp. 722–731. IEEE CS (2005)Google Scholar
  12. 12.
    Eisentraut, C., Hermanns, H., Katoen, J.-P., Zhang, L.: A semantics for every GSPN. In: Colom, J.-M., Desel, J. (eds.) PETRI NETS 2013. LNCS, vol. 7927, pp. 90–109. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  13. 13.
    Eisentraut, C., Hermanns, H., Zhang, L.: On probabilistic automata in continuous time. In: Proceedings of LICS, pp. 342–351. IEEE CS (2010)Google Scholar
  14. 14.
    Fu, H.: Maximal cost-bounded reachability probability on continuous-time markov decision processes. In: Muscholl, A. (ed.) FOSSACS 2014 (ETAPS). LNCS, vol. 8412, pp. 73–87. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  15. 15.
    Fu, H.: Verifying Probabilistic Systems: New Algorithms and Complexity Results. Ph.D. thesis, RWTH Aachen University (2014)Google Scholar
  16. 16.
    Guck, D., Han, T., Katoen, J.-P., Neuhäußer, M.R.: Quantitative timed analysis of interactive markov chains. In: Goodloe, A.E., Person, S. (eds.) NFM 2012. LNCS, vol. 7226, pp. 8–23. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  17. 17.
    Guck, D., Hatefi, H., Hermanns, H., Katoen, J.-P., Timmer, M.: Modelling, reduction and analysis of markov automata. In: Joshi, K., Siegle, M., Stoelinga, M., D’Argenio, P.R. (eds.) QEST 2013. LNCS, vol. 8054, pp. 55–71. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  18. 18.
    Guck, D., Hatefi, H., Hermanns, H., Katoen, J., Timmer, M.: Analysis of timed and long-run objectives for Markov automata. Logical Methods in Computer Science 10(3) (2014).
  19. 19.
    Guck, D., Timmer, M., Hatefi, H., Ruijters, E., Stoelinga, M.: Modelling and analysis of markov reward automata. In: Cassez, F., Raskin, J.-F. (eds.) ATVA 2014. LNCS, vol. 8837, pp. 168–184. Springer, Heidelberg (2014) Google Scholar
  20. 20.
    Hatefi, H., Braitling, B., Wimmer, R., Ferrer Fioriti, L.M., Hermanns, H., Becker, B.: Cost vs. time in stochastic games and Markov automata (extended version). Reports of SFB/TR 14 AVACS 113, SFB/TR 14 AVACS (2015).
  21. 21.
    Hatefi, H., Hermanns, H.: Model checking algorithms for Markov automata. ECEASST 53 (2012)Google Scholar
  22. 22.
    Hermanns, H. (ed.): Interactive Markov Chains. LNCS, vol. 2428. Springer, Heidelberg (2002) zbMATHGoogle Scholar
  23. 23.
    Johr, S.: Model checking compositional Markov systems. Ph.D. thesis, Saarland University, Germany (2008)Google Scholar
  24. 24.
    Miller, B.L.: Finite state continuous time Markov decision processes with a finite planning horizon. SIAM Journal on Control 6(2), 266–280 (1968)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Neuhäußer, M.R.: Model checking nondeterministic and randomly timed systems. Ph.D. thesis, RWTH Aachen University and University of Twente (2010)Google Scholar
  26. 26.
    Neuhäußer, M.R., Zhang, L.: Time-bounded reachability probabilities in continuous-time Markov decision processes. In: Proceedings of QEST, pp. 209–218. IEEE CS (2010)Google Scholar
  27. 27.
    Qiu, Q., Qu, Q., Pedram, M.: Stochastic modeling of a power-managed system-construction and optimization. IEEE Transactions on CAD of Integrated Circuits and Systems 20(10), 1200–1217 (2001)CrossRefGoogle Scholar
  28. 28.
    Segala, R.: A compositional trace-based semantics for probabilistic automata. In: Lee, I., Smolka, S.A. (eds.) CONCUR 1995. LNCS, vol. 962, pp. 234–248. Springer, Heidelberg (1995) CrossRefGoogle Scholar
  29. 29.
    Shapley, L.S.: Stochastic games. Proc. of the National Academy of Sciences of the United States of America 39(10), 1095 (1953)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Simunic, T., Benini, L., Glynn, P.W., Micheli, G.D.: Dynamic power management for portable systems. In: Proc. of MOBICOM, pp. 11–19 (2000)Google Scholar
  31. 31.
    Timmer, M., Katoen, J.-P., van de Pol, J., Stoelinga, M.I.A.: Efficient modelling and generation of markov automata. In: Koutny, M., Ulidowski, I. (eds.) CONCUR 2012. LNCS, vol. 7454, pp. 364–379. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  32. 32.
    Timmer, M., van de Pol, J., Stoelinga, M.I.A.: Confluence reduction for markov automata. In: Braberman, V., Fribourg, L. (eds.) FORMATS 2013. LNCS, vol. 8053, pp. 243–257. Springer, Heidelberg (2013) CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Hassan Hatefi
    • 1
  • Bettina Braitling
    • 2
    Email author
  • Ralf Wimmer
    • 2
  • Luis María Ferrer Fioriti
    • 1
  • Holger Hermanns
    • 1
  • Bernd Becker
    • 2
  1. 1.Saarland UniversitySaarbrückenGermany
  2. 2.Albert-Ludwigs-Universität FreiburgFreiburg im BreisgauGermany

Personalised recommendations