The Odds of Staying on Budget

  • Christoph HaaseEmail author
  • Stefan Kiefer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9135)


Given Markov chains and Markov decision processes (MDPs) whose transitions are labelled with non-negative integer costs, we study the computational complexity of deciding whether the probability of paths whose accumulated cost satisfies a Boolean combination of inequalities exceeds a given threshold. For acyclic Markov chains, we show that this problem is PP-complete, whereas it is hard for the PosSLP problem and in PSpace for general Markov chains. Moreover, for acyclic and general MDPs, we prove PSpace- and EXP-completeness, respectively. Our results have direct implications on the complexity of computing reward quantiles in succinctly represented stochastic systems.


Markov Chain Turing Machine Cost Process Markov Decision Process Arithmetic Circuit 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allender, E., Bürgisser, P., Kjeldgaard-Pedersen, J., Bro, P.: Miltersen. On the complexity of numerical analysis. SIAM J. Comput. 38(5), 1987–2006 (2009)zbMATHCrossRefGoogle Scholar
  2. 2.
    Baier, C., Dubslaff, C., Klüppelholz, S.: Trade-off analysis meets probabilistic model checking. In: Proc. CSL-LICS, pp. 1:1–1:10. ACM (2014)Google Scholar
  3. 3.
    Baier, C., Katoen, J.-P.: Principles of Model Checking. MIT Press (2008)Google Scholar
  4. 4.
    Baier, C., Klein, J., Klüppelholz, S., Märcker, S.: Computing conditional probabilities in markovian models efficiently. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014 (ETAPS). LNCS, vol. 8413, pp. 515–530. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  5. 5.
    Brázdil, T., Brožek, V., Etessami, K., Kučera, A., Wojtczak, D.: One-counter Markov decision processes. In: Proc. SODA, pp. 863–874. SIAM (2010)Google Scholar
  6. 6.
    Bruyère, V., Filiot, E., Randour, M., Raskin, J.-F.: Meet your expectations with guarantees: beyond worst-case synthesis in quantitative games. In: Proc. STACS, LIPIcs, vol. 25, pp. 199–213 (2014)Google Scholar
  7. 7.
    Etessami, K., Yannakakis, M.: Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations. J. ACM 56(1), 1:1–1:66 (2009)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Gill, J.: Computational complexity of probabilistic Turing machines. SIAM J. Comput. 6(4), 675–695 (1977)zbMATHMathSciNetCrossRefGoogle Scholar
  9. 9.
    Haase, C., Kiefer, S.: The odds of staying on budget (2014). Technical Report at
  10. 10.
    Haase, C., Kiefer, S.: The complexity of the \(K\)th largest subset problem and related problems (2015). Technical Report at
  11. 11.
    Jurdziński, M., Sproston, J., Laroussinie, F.: Model checking probabilistic timed automata with one or two clocks. Log. Meth. Comput. Sci. 4(3), 12 (2008)Google Scholar
  12. 12.
    Laroussinie, F., Sproston, J.: Model checking durational probabilistic systems. In: Sassone, V. (ed.) FOSSACS 2005. LNCS, vol. 3441, pp. 140–154. Springer, Heidelberg (2005) CrossRefGoogle Scholar
  13. 13.
    Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley and Sons (2008)Google Scholar
  14. 14.
    Randour, M., Raskin, J.-F., Sankur, O.: Percentile queries in multi-dimensional Markov decision processes. In: Proc. CAV, LNCS (2015)Google Scholar
  15. 15.
    Randour, M., Raskin, J.-F., Sankur, O.: Variations on the stochastic shortest path problem. In: D’Souza, D., Lal, A., Larsen, K.G. (eds.) VMCAI 2015. LNCS, vol. 8931, pp. 1–18. Springer, Heidelberg (2015) Google Scholar
  16. 16.
    Simon, J.: On the difference between one and many. In: Salomaa, A., Steinby, M. (eds.) ICALP 1977. LNCS, vol. 52, pp. 480–491. Springer, Heidelberg (1977) CrossRefGoogle Scholar
  17. 17.
    Toda, S.: PP is as hard as the polynomial-time hierarchy. SIAM J. Comput. 20(5), 865–877 (1991)zbMATHMathSciNetCrossRefGoogle Scholar
  18. 18.
    Travers, S.: The complexity of membership problems for circuits over sets of integers. Theor. Comput. Sci. 369(1–3), 211–229 (2006)zbMATHMathSciNetCrossRefGoogle Scholar
  19. 19.
    Ummels, M., Baier, C.: Computing quantiles in Markov reward models. In: Pfenning, F. (ed.) FOSSACS 2013 (ETAPS 2013). LNCS, vol. 7794, pp. 353–368. Springer, Heidelberg (2013) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Laboratoire Spécification et Vérification (LSV)CNRS and ENS de CachanCachan CedexFrance
  2. 2.Department of Computer ScienceUniversity of OxfordOxfordUK

Personalised recommendations