Deciding the Value 1 Problem for Reachability in 1-Clock Decision Stochastic Timed Automata

  • Nathalie Bertrand
  • Thomas Brihaye
  • Blaise Genest
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8657)


We consider reachability objectives on an extension of stochastic timed automata (STA) with nondeterminism. Decision stochastic timed automata (DSTA) are Markov decision processes based on timed automata where delays are chosen randomly and choices between enabled edges are nondeterministic. Given a reachability objective, the value 1 problem asks whether a target can be reached with probability arbitrary close to 1. Simple examples show that the value can be 1 and yet no strategy ensures reaching the target with probability 1. In this paper, we prove that, the value 1 problem is decidable for single clock DSTA by non-trivial reduction to a simple almost-sure reachability problem on a finite Markov decision process. The ε-optimal strategies are involved: the precise probability distributions, even if they do not change the winning nature of a state, impact the timings at which ε-optimal strategies must change their decisions, and more surprisingly these timings cannot be chosen uniformly over the set of regions.


Markov Decision Process Pointed Region Reachability Problem Time Automaton Player State 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alur, R., Dill, D.L.: A Theory of Timed Automata. Theoretical Computer Science 126(2), 183–235 (1994)CrossRefzbMATHMathSciNetGoogle Scholar
  2. 2.
    Baier, C., Bertrand, N., Bouyer, P., Brihaye, T., Größer, M.: Probabilistic and Topological Semantics for Timed Automata. In: Arvind, V., Prasad, S. (eds.) FSTTCS 2007. LNCS, vol. 4855, pp. 179–191. Springer, Heidelberg (2007)Google Scholar
  3. 3.
    Baier, C., Bertrand, N., Bouyer, P., Brihaye, T., Größer, M.: Almost-Sure Model Checking of Infinite Paths in One-Clock Timed Automata. In: Proceedings of LICS 2008, pp. 217–226. IEEE Comp. Soc. Press (2008)Google Scholar
  4. 4.
    Baier, C., Katoen, J.-P.: Principles of model checking. MIT Press (2008)Google Scholar
  5. 5.
    Bertrand, N., Schewe, S.: Playing optimally on timed automata with random delays. In: Jurdziński, M., Ničković, D. (eds.) FORMATS 2012. LNCS, vol. 7595, pp. 43–58. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  6. 6.
    Bouyer, P., Brinksma, E., Larsen, K.G.: Optimal infinite scheduling for multi-priced timed automata. Formal Methods in System Design 32(1), 3–23 (2008)CrossRefzbMATHGoogle Scholar
  7. 7.
    Bouyer, P., Forejt, V.: Reachability in Stochastic Timed Games. In: Albers, S., Marchetti-Spaccamela, A., Matias, Y., Nikoletseas, S., Thomas, W. (eds.) ICALP 2009, Part II. LNCS, vol. 5556, pp. 103–114. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Brázdil, T., Krčál, J., Křetínský, J., Kučera, A., Řehák, V.: Stochastic Real-Time Games with Qualitative Timed Automata Objectives. In: Gastin, P., Laroussinie, F. (eds.) CONCUR 2010. LNCS, vol. 6269, pp. 207–221. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Chatterjee, K., de Alfaro, L., Henzinger, T.A.: Qualitative concurrent parity games. ACM Transactions on Computation Logic 12(4), 28 (2011)Google Scholar
  10. 10.
    Chatterjee, K., Tracol, M.: Decidable problems for probabilistic automata on infinite words. In: Proceedings of LICS 2012, pp. 185–194. IEEE (2012)Google Scholar
  11. 11.
    Chen, T., Han, T., Katoen, J.-P., Mereacre, A.: Reachability Probabilities in Markovian Timed Automata. In: Proceedings of CDC-ECC 2011, pp. 7075–7080. IEEE (2011)Google Scholar
  12. 12.
    de Alfaro, L., Henzinger, T.A., Kupferman, O.: Concurrent reachability games. Theoretical Computer Science 386(3), 188–217 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Fijalkow, N., Gimbert, H., Oualhadj, Y.: Deciding the value 1 problem for probabilistic leaktight automata. In: Proceedings of LICS 2012, pp. 295–304. IEEE (2012)Google Scholar
  14. 14.
    Gimbert, H., Oualhadj, Y.: Probabilistic automata on finite words: Decidable and undecidable problems. In: Abramsky, S., Gavoille, C., Kirchner, C., Meyer auf der Heide, F., Spirakis, P.G. (eds.) ICALP 2010. LNCS, vol. 6199, pp. 527–538. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  15. 15.
    Laroussinie, F., Markey, N., Schnoebelen, P.: Model checking timed automata with one or two clocks. In: Gardner, P., Yoshida, N. (eds.) CONCUR 2004. LNCS, vol. 3170, pp. 387–401. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  16. 16.
    Wolovick, N., Johr, S.: A Characterization of Meaningful Schedulers for Continuous-Time Markov Decision Processes. In: Asarin, E., Bouyer, P. (eds.) FORMATS 2006. LNCS, vol. 4202, pp. 352–367. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Nathalie Bertrand
    • 1
  • Thomas Brihaye
    • 2
  • Blaise Genest
    • 3
  1. 1.Inria, Team SUMO, UMR IRISARennesFrance
  2. 2.Université de MonsMonsBelgium
  3. 3.CNRS, Team SUMO, UMR IRISARennesFrance

Personalised recommendations