Probabilistic Systems with LimSup and LimInf Objectives

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5489)


We give polynomial-time algorithms for computing the values of Markov decision processes (MDPs) with limsup and liminf objectives. A real-valued reward is assigned to each state, and the value of an infinite path in the MDP is the limsup (resp. liminf) of all rewards along the path. The value of an MDP is the maximal expected value of an infinite path that can be achieved by resolving the decisions of the MDP. Using our result on MDPs, we show that turn-based stochastic games with limsup and liminf objectives can be solved in NP ∩ coNP.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chatterjee, K., Henzinger, T.A.: Value iteration. In: Grumberg, O., Veith, H. (eds.) 25 Years of Model Checking. LNCS, vol. 5000, pp. 107–138. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  2. 2.
    Chatterjee, K., Jurdziński, M., Henzinger, T.A.: Simple stochastic parity games. In: Baaz, M., Makowsky, J.A. (eds.) CSL 2003. LNCS, vol. 2803, pp. 100–113. Springer, Heidelberg (2003)Google Scholar
  3. 3.
    Chatterjee, K., Jurdziński, M., Henzinger, T.A.: Quantitative stochastic parity games. In: Ian Munro, J. (ed.) SODA 2004, pp. 121–130. SIAM, Philadelphia (2004)Google Scholar
  4. 4.
    Courcoubetis, C., Yannakakis, M.: Markov decision processes and regular events. In: Paterson, M. (ed.) ICALP 1990. LNCS, vol. 443, pp. 336–349. Springer, Heidelberg (1990)CrossRefGoogle Scholar
  5. 5.
    de Alfaro, L.: Formal Verification of Probabilistic Systems. PhD thesis, Stanford University (1997)Google Scholar
  6. 6.
    Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer, Heidelberg (1997)zbMATHGoogle Scholar
  7. 7.
    Gimbert, H.: Jeux positionnels. PhD thesis, Université Paris 7 (2006)Google Scholar
  8. 8.
    Gimbert, H., Zielonka, W.: Games where you can play optimally without any memory. In: Abadi, M., de Alfaro, L. (eds.) CONCUR 2005. LNCS, vol. 3653, pp. 428–442. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  9. 9.
    Gimbert, H., Zielonka, W.: Perfect information stochastic priority games. In: Arge, L., Cachin, C., Jurdziński, T., Tarlecki, A. (eds.) ICALP 2007. LNCS, vol. 4596, pp. 850–861. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  10. 10.
    Liggett, T.A., Lippman, S.A.: Stochastic games with perfect information and time average payoff. SIAM Review 11, 604–607 (1969)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Maitra, A., Sudderth, W. (eds.): Discrete Gambling and Stochastic Games. Stochastic Modelling and Applied Probability, vol. 32. Springer, Heidelberg (1996)zbMATHGoogle Scholar
  12. 12.
    Martin, D.A.: The determinacy of Blackwell games. J. Symb. Log. 63(4), 1565–1581 (1998)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  1. 1.Jack Baskin School of EngineeringUniversity of California at Santa CruzSanta CruzUnited States of America
  2. 2.School of Computer and Communication SciencesÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland

Personalised recommendations