International Conference on Quantitative Evaluation of Systems

QEST 2015: Quantitative Evaluation of Systems pp 141-159 | Cite as

Optimizing Performance of Continuous-Time Stochastic Systems Using Timeout Synthesis

  • Tomáš Brázdil
  • Ľuboš Korenčiak
  • Jan Krčál
  • Petr Novotný
  • Vojtěch Řehák
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9259)


We consider parametric version of fixed-delay continuous-time Markov chains (or equivalently deterministic and stochastic Petri nets, DSPN) where fixed-delay transitions are specified by parameters, rather than concrete values. Our goal is to synthesize values of these parameters that, for a given cost function, minimise expected total cost incurred before reaching a given set of target states. We show that under mild assumptions, optimal values of parameters can be effectively approximated using translation to a Markov decision process (MDP) whose actions correspond to discretized values of these parameters. To this end we identify and overcome several interesting phenomena arising in systems with fixed delays.


  1. 1.
    Alur, R., Courcoubetis, C., Dill, D.: Model-checking for probabilistic real-time systems. In: Albert, J.L., Monien, B., Artalejo, M.R. (eds.) Automata, Languages and Programming. LNCS, vol. 510, pp. 115–136. Springer, Heidelberg (1991)CrossRefGoogle Scholar
  2. 2.
    Alur, R., Henzinger, T.A., Vardi, M.Y.: Parametric real-time reasoning. In: STOC, pp. 592–601. ACM (1993)Google Scholar
  3. 3.
    Audsley, N.C., Grigg, A.: Timing analysis of the ARINC 629 databus for real-time applications. Microprocess. Microsyst. 21(1), 55–61 (1997)CrossRefGoogle Scholar
  4. 4.
    Baier, C., Bertrand, N., Bouyer, P., Brihaye, T., Größer, M.: Probabilistic and topological semantics for timed automata. In: Arvind, V., Prasad, S. (eds.) FSTTCS 2007. LNCS, vol. 4855, pp. 179–191. Springer, Heidelberg (2007) CrossRefGoogle Scholar
  5. 5.
    Baier, C., Bertrand, N., Bouyer, P., Brihaye, T., Größer, M.: Almost-sure model checking of infinite paths in one-clock timed automata. In: LICS, pp. 217–226. IEEE (2008)Google Scholar
  6. 6.
    Brázdil, T., Forejt, V., Krčál, J., Křetínský, J., Kučera, A.: Continuous-time stochastic games with time-bounded reachability. Inf. Comput. 224, 46–70 (2013)CrossRefMATHGoogle Scholar
  7. 7.
    Brázdil, T., Korenčiak, Ĺ., Krčál, J., Novotný, P., Řehák, V.: Optimizing performance of continuous-time stochastic systems using timeout synthesis. CoRR, abs/1407.4777 (2014)Google Scholar
  8. 8.
    Brázdil, T., Krčál, J., Křetínský, J., Kučera, A., Řehák, V.: Stochastic real-time games with qualitative timed automata objectives. In: Gastin, P., Laroussinie, F. (eds.) CONCUR 2010. LNCS, vol. 6269, pp. 207–221. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  9. 9.
    Brázdil, T., Krčál, J., Křetínský, J., Řehák, V.: Fixed-delay events in generalized semi-Markov processes revisited. In: Katoen, J.-P., König, B. (eds.) CONCUR 2011. LNCS, vol. 6901, pp. 140–155. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  10. 10.
    Buchholz, P., Hahn, E.M., Hermanns, H., Zhang, L.: Model checking algorithms for CTMDPs. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 225–242. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  11. 11.
    Carnevali, L., Ridi, L., Vicario, E.: A quantitative approach to input generation in real-time testing of stochastic systems. IEEE Trans. Softw. Eng. 39(3), 292–304 (2013)CrossRefGoogle Scholar
  12. 12.
    Chen, T., Han, T., Katoen, J.-P., Mereacre, A.: Quantitative model checking of continuous-time Markov chains against timed automata specifications. In: LICS, pp. 309–318. IEEE (2009)Google Scholar
  13. 13.
    Choi, H., Kulkarni, V.G., Trivedi, K.S.: Transient analysis of deterministic and stochastic Petri nets. In: Marsan, M.A. (ed.) Application and Theory of Petri Nets, vol. 691, pp. 166–185. Springer, Heidelberg (1993)Google Scholar
  14. 14.
    Etessami, K., Wojtczak, D., Yannakakis, M.: Recursive stochastic games with positive rewards. In: Aceto, L., Damgård, I., Goldberg, L.A., Halldórsson, M.M., Ingólfsdóttir, A., Walukiewicz, I. (eds.) ICALP 2008, Part I. LNCS, vol. 5125, pp. 711–723. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  15. 15.
    Feller, W.: An Introduction to Probability Theory and Its Applications, vol. 1. Wiley, New York (1968)MATHGoogle Scholar
  16. 16.
    Guet, C.C., Gupta, A., Henzinger, T.A., Mateescu, M., Sezgin, A.: Delayed continuous-time Markov chains for genetic regulatory circuits. In: Madhusudan, P., Seshia, S.A. (eds.) CAV 2012. LNCS, vol. 7358, pp. 294–309. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  17. 17.
    Haase, C., Kreutzer, S., Ouaknine, J., Worrell, J.: Reachability in succinct and parametric one-counter automata. In: Bravetti, M., Zavattaro, G. (eds.) CONCUR 2009. LNCS, vol. 5710, pp. 369–383. Springer, Heidelberg (2009) CrossRefGoogle Scholar
  18. 18.
    Hahn, E.M., Hermanns, H., Zhang, L.: Probabilistic reachability for parametric Markov models. STTT 13(1), 3–19 (2011)CrossRefGoogle Scholar
  19. 19.
    Han, T., Katoen, J.P., Mereacre, A.: Approximate parameter synthesis for probabilistic time-bounded reachability. In: Real-Time Systems Symposium, pp. 173–182. IEEE (2008)Google Scholar
  20. 20.
    Jensen, P.G., Taankvist, J.H.: Learning optimal scheduling for time uncertain settings. Aalborg University, Student project (2014)Google Scholar
  21. 21.
    Jha, S.K., Langmead, C.J.: Synthesis and infeasibility analysis for stochastic models of biochemical systems using statistical model checking and abstraction refinement. TCS 412(21), 2162–2187 (2011)CrossRefMathSciNetMATHGoogle Scholar
  22. 22.
    Khaksari, M., Fischione, C.: Performance analysis and optimization of the joining protocol for a platoon of vehicles. In: ISCCSP, pp. 1–6. IEEE (2012)Google Scholar
  23. 23.
    Korenčiak, Ĺ., Krčál, J., Řehák, V.: Dealing with zero density using piecewise phase-type approximation. In: Horváth, A., Wolter, K. (eds.) EPEW 2014. LNCS, vol. 8721, pp. 119–134. Springer, Heidelberg (2014) Google Scholar
  24. 24.
    Kwiatkowska, M., Norman, G., Segala, R., Sproston, J.: Verifying quantitative properties of continuous probabilistic timed automata. In: Palamidessi, C. (ed.) CONCUR 2000. LNCS, vol. 1877, pp. 123–137. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  25. 25.
    Lindemann, C.: An improved numerical algorithm for calculating steady-state solutions of deterministic and stochastic Petri net models. Perform. Eval. 18(1), 79–95 (1993)CrossRefMathSciNetMATHGoogle Scholar
  26. 26.
    Marsan, M.A., Chiola, G.: On Petri nets with deterministic and exponentially distributed firing times. In: Rozenberg, G. (ed.) Advances in Petri Nets, pp. 132–145. Springer, Heidelberg (1987)Google Scholar
  27. 27.
    Neuhäusser, M.R., Zhang, L.: Time-bounded reachability probabilities in continuous-time Markov decision processes. In: QEST, pp. 209–218. IEEE (2010)Google Scholar
  28. 28.
    Neuts, M.F.: Matrix-geometric Solutions in Stochastic Models: An Algorithmic Approach. Courier Dover Publications, Mineola (1981)MATHGoogle Scholar
  29. 29.
    Norris, J.R.: Markov Chains. Cambridge University Press, Cambridge (1998)MATHGoogle Scholar
  30. 30.
    Obermaisser, R.: Time-Triggered Communication. CRC Press, Boca Raton (2011)Google Scholar
  31. 31.
    Puterman, M.L.: Markov Decision Processes. Wiley, Hoboken (1994)CrossRefMATHGoogle Scholar
  32. 32.
    Ramamritham, K., Stankovic, J.A.: Scheduling algorithms and operating systems support for real-time systems. Proc. IEEE 82(1), 55–67 (1994)CrossRefGoogle Scholar
  33. 33.
    Tiassou, K.B.: Aircraft operational reliability - a model-based approach and case studies. Ph.D. thesis, Universié de Toulouse (2013)Google Scholar
  34. 34.
    Wolovick, N., D’Argenio, P.R., Qu, V: Optimizing probabilities of real-time test case execution. In: ICST, pp. 446–455. IEEE (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Tomáš Brázdil
    • 1
  • Ľuboš Korenčiak
    • 1
  • Jan Krčál
    • 2
  • Petr Novotný
    • 3
  • Vojtěch Řehák
    • 1
  1. 1.Faculty of InformaticsMasaryk UniversityBrnoCzech Republic
  2. 2.Saarland University – Computer ScienceSaarbrückenGermany
  3. 3.IST AustriaKlosterneuburgAustria

Personalised recommendations