Advertisement

Variations on the Stochastic Shortest Path Problem

  • Mickael Randour
  • Jean-François Raskin
  • Ocan Sankur
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8931)

Abstract

In this invited contribution, we revisit the stochastic shortest path problem, and show how recent results allow one to improve over the classical solutions: we present algorithms to synthesize strategies with multiple guarantees on the distribution of the length of paths reaching a given target, rather than simply minimizing its expected value. The concepts and algorithms that we propose here are applications of more general results that have been obtained recently for Markov decision processes and that are described in a series of recent papers.

Keywords

Short Path Polynomial Time Target State Markov Decision Process Short Path Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Baier, C., Katoen, J.-P.: Principles of model checking. MIT Press (2008)Google Scholar
  2. 2.
    Bertsekas, D.P., Tsitsiklis, J.N.: An analysis of stochastic shortest path problems. Mathematics of Operations Research 16(3), 580–595 (1991)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Brihaye, T., Geeraerts, G., Haddad, A., Monmege, B.: To reach or not to reach? Efficient algorithms for total-payoff games. CoRR, abs/1407.5030 (2014)Google Scholar
  4. 4.
    Bruyère, V., Filiot, E., Randour, M., Raskin, J.-F.: Expectations or guarantees? I want it all! A crossroad between games and MDPs. In: Proc. of SR. EPTCS, vol. 146, pp. 1–8 (2014)Google Scholar
  5. 5.
    Bruyère, V., Filiot, E., Randour, M., Raskin, J.-F.: Meet your expectations with guarantees: Beyond worst-case synthesis in quantitative games. In: Proc. of STACS. LIPIcs, vol. 25, pp. 199–213. Schloss Dagstuhl - LZI (2014)Google Scholar
  6. 6.
    Chatterjee, K., Chmelik, M., Tracol, M.: What is decidable about partially observable Markov decision processes with omega-regular objectives. In: Proc. of CSL. LIPIcs, vol. 23, Schloss Dagstuhl - LZI (2013)Google Scholar
  7. 7.
    Chatterjee, K., Doyen, L., Randour, M., Raskin, J.-F.: Looking at mean-payoff and total-payoff through windows. In: Van Hung, D., Ogawa, M. (eds.) ATVA 2013. LNCS, vol. 8172, pp. 118–132. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  8. 8.
    Cherkassky, B.V., Goldberg, A.V., Radzik, T.: Shortest paths algorithms: Theory and experimental evaluation. Math. Programming 73(2), 129–174 (1996)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Courcoubetis, C., Yannakakis, M.: The complexity of probabilistic verification. J. ACM 42(4), 857–907 (1995)CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    de Alfaro, L.: Computing minimum and maximum reachability times in probabilistic systems. In: Baeten, J.C.M., Mauw, S. (eds.) CONCUR 1999. LNCS, vol. 1664, pp. 66–81. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  11. 11.
    Filiot, E., Gentilini, R., Raskin, J.-F.: Quantitative languages defined by functional automata. In: Koutny, M., Ulidowski, I. (eds.) CONCUR 2012. LNCS, vol. 7454, pp. 132–146. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  12. 12.
    Ehrenfeucht, A., Mycielski, J.: Positional strategies for mean payoff games. International Journal of Game Theory 8, 109–113 (1979)CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Etessami, K., Kwiatkowska, M.Z., Vardi, M.Y., Yannakakis, M.: Multi-objective model checking of Markov decision processes. LMCS 4(4) (2008)Google Scholar
  14. 14.
    Forejt, V., Kwiatkowska, M., Norman, G., Parker, D., Qu, H.: Quantitative multi-objective verification for probabilistic systems. In: Abdulla, P.A., Leino, K.R.M. (eds.) TACAS 2011. LNCS, vol. 6605, pp. 112–127. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  15. 15.
    Haase, C., Kiefer, S.: The odds of staying on budget. CoRR, abs/1409.8228 (2014)Google Scholar
  16. 16.
    Khachiyan, L., Boros, E., Borys, K., Elbassioni, K.M., Gurvich, V., Rudolf, G., Zhao, J.: On short paths interdiction problems: Total and node-wise limited interdiction, pp. 204–233 (2008)Google Scholar
  17. 17.
    Ohtsubo, Y.: Optimal threshold probability in undiscounted markov decision processes with a target set. Applied Math. and Computation 149(2), 519–532 (2004)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming, 1st edn. John Wiley & Sons, Inc., New York (1994)CrossRefzbMATHGoogle Scholar
  19. 19.
    Randour, M., Raskin, J.-F., Sankur, O.: Percentile queries in multi-dimensional Markov decision processes. CoRR, abs/1410.4801 (2014)Google Scholar
  20. 20.
    Raskin, J.-F., Sankur, O.: Multiple-environment Markov decision processes. CoRR, abs/1405.4733 (2014)Google Scholar
  21. 21.
    Raskin, J.-F., Sankur, O.: Multiple-environment Markov decision processes. In: Proc. of FSTTCS. LIPIcs, Schloss Dagstuhl - LZI (2014)Google Scholar
  22. 22.
    Sakaguchi, M., Ohtsubo, Y.: Markov decision processes associated with two threshold probability criteria. Journal of Control Theory and Applications 11(4), 548–557 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    Ummels, M., Baier, C.: Computing quantiles in Markov reward models. In: Pfenning, F. (ed.) FOSSACS 2013. LNCS, vol. 7794, pp. 353–368. Springer, Heidelberg (2013)Google Scholar
  24. 24.
    Vardi, M.: Automatic verification of probabilistic concurrent finite state programs. In: Proc. of FOCS, pp. 327–338. IEEE Computer Society (1985)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Mickael Randour
    • 1
  • Jean-François Raskin
    • 2
  • Ocan Sankur
    • 2
  1. 1.LSV, CNRS & ENS CachanFrance
  2. 2.Département d’InformatiqueUniversité Libre de Bruxelles (U.L.B.)Belgium

Personalised recommendations