Advertisement

Energy and Mean-Payoff Parity Markov Decision Processes

  • Krishnendu Chatterjee
  • Laurent Doyen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6907)

Abstract

We consider Markov Decision Processes (MDPs) with mean-payoff parity and energy parity objectives. In system design, the parity objective is used to encode ω-regular specifications, while the mean-payoff and energy objectives can be used to model quantitative resource constraints. The energy condition requires that the resource level never drops below 0, and the mean-payoff condition requires that the limit-average value of the resource consumption is within a threshold. While these two (energy and mean-payoff) classical conditions are equivalent for two-player games, we show that they differ for MDPs. We show that the problem of deciding whether a state is almost-sure winning (i.e., winning with probability 1) in energy parity MDPs is in NP ∩ coNP, while for mean-payoff parity MDPs, the problem is solvable in polynomial time.

Keywords

Polynomial Time Energy Parity Markov Decision Process Priority Function Parity Game 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bianco, A., de Alfaro, L.: Model checking of probabilistic and nondeterministic systems. In: Thiagarajan, P.S. (ed.) FSTTCS 1995. LNCS, vol. 1026, pp. 499–513. Springer, Heidelberg (1995)Google Scholar
  2. 2.
    Bloem, R., Chatterjee, K., Henzinger, T.A., Jobstmann, B.: Better quality in synthesis through quantitative objectives. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, pp. 140–156. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. 3.
    Bouyer, P., Fahrenberg, U., Larsen, K.G., Markey, N., Srba, J.: Infinite runs in weighted timed automata with energy constraints. In: Cassez, F., Jard, C. (eds.) FORMATS 2008. LNCS, vol. 5215, pp. 33–47. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  4. 4.
    Brázdil, T., Brozek, V., Etessami, K., Kucera, A., Wojtczak, D.: One-counter Markov decision processes. In: Proc. of SODA, pp. 863–874. SIAM, Philadelphia (2010)Google Scholar
  5. 5.
    Brim, L., Chaloupka, J., Doyen, L., Gentilini, R., Raskin, J.-F.: Faster algorithms for mean-payoff games. Formal Methods in System Design (2010)Google Scholar
  6. 6.
    Chakrabarti, A., de Alfaro, L., Henzinger, T.A., Stoelinga, M.: Resource interfaces. In: Alur, R., Lee, I. (eds.) EMSOFT 2003. LNCS, vol. 2855, pp. 117–133. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  7. 7.
    Chatterjee, K., Doyen, L.: Energy parity games. In: Abramsky, S., Gavoille, C., Kirchner, C., Meyer auf der Heide, F., Spirakis, P.G. (eds.) ICALP 2010. LNCS, vol. 6199, pp. 599–610. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  8. 8.
    Chatterjee, K., Doyen, L.: Energy and mean-payoff parity Markov decision processes. Technical report, IST Austria (February 2011), http://pub.ist.ac.at/Pubs/TechRpts/2011/IST-2011-0001.pdf
  9. 9.
    Chatterjee, K., Henzinger, M.: Faster and dynamic algorithms for maximal end-component decomposition and related graph problems in probabilistic verification. In: Proc. of SODA. ACM SIAM (2011)Google Scholar
  10. 10.
    Chatterjee, K., Henzinger, T.A., Jobstmann, B., Singh, R.: Measuring and synthesizing systems in probabilistic environments. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 380–395. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    Chatterjee, K., Henzinger, T.A., Jurdziński, M.: Mean-payoff parity games. In: Proc. of LICS, pp. 178–187. IEEE Computer Society, Los Alamitos (2005)Google Scholar
  12. 12.
    Courcoubetis, C., Yannakakis, M.: The complexity of probabilistic verification. J. ACM 42(4), 857–907 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    de Alfaro, L.: Formal Verification of Probabilistic Systems. PhD thesis, Stanford University (1997)Google Scholar
  14. 14.
    Emerson, E.A., Jutla, C.: Tree automata, mu-calculus and determinacy. In: Proc. of FOCS, pp. 368–377. IEEE, Los Alamitos (1991)Google Scholar
  15. 15.
    Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer, Heidelberg (1997)zbMATHGoogle Scholar
  16. 16.
    Gimbert, H., Horn, F.: Solving simple stochastic tail games. In: Proc. of SODA, pp. 847–862 (2010)Google Scholar
  17. 17.
    Gimbert, H., Oualhadj, Y., Paul, S.: Computing optimal strategies for Markov decision processes with parity and positive-average conditions. Technical report, LaBRI, Université de Bordeaux II (2011)Google Scholar
  18. 18.
    Jurdziński, M.: Deciding the winner in parity games is in UP ∩ co-UP. Inf. Process. Lett. 68(3), 119–124 (1998)CrossRefGoogle Scholar
  19. 19.
    Kucera, A., Stražovský, O.: On the controller synthesis for finite-state markov decision processes. In: Proc. of FSTTCS, pp. 541–552 (2005)Google Scholar
  20. 20.
    Pacuk, A.: Hybrid of mean payoff and total payoff. In: Talk at the Workshop Games for Design and Verification, St Anne’s College Oxford (September 2010)Google Scholar
  21. 21.
    W. Thomas. Languages, automata, and logic. In: Handbook of Formal Languages, vol. 3, ch.7, pp. 389–455. Springer, Heidelberg (1997)Google Scholar
  22. 22.
    Vardi, M.Y.: Automatic verification of probabilistic concurrent finite-state systems. In: FOCS 1985. IEEE Computer Society Press, Los Alamitos (1985)Google Scholar
  23. 23.
    Zwick, U., Paterson, M.: The complexity of mean payoff games on graphs. Theor. Comput. Sci. 158(1&2), 343–359 (1996)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Berlin Heidelberg 2011

Authors and Affiliations

  • Krishnendu Chatterjee
    • 1
  • Laurent Doyen
    • 2
  1. 1.IST Austria (Institute of Science and Technology Austria)Austria
  2. 2.LSV, ENS Cachan & CNRSFrance

Personalised recommendations