Quantitative Timed Analysis of Interactive Markov Chains

  • Dennis Guck
  • Tingting Han
  • Joost-Pieter Katoen
  • Martin R. Neuhäußer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7226)


This paper presents new algorithms and accompanying tool support for analyzing interactive Markov chains (IMCs), a stochastic timed \(1\frac{1}{2}\)-player game in which delays are exponentially distributed. IMCs are compositional and act as semantic model for engineering formalisms such as AADL and dynamic fault trees. We provide algorithms for determining the extremal expected time of reaching a set of states, and the long-run average of time spent in a set of states. The prototypical tool Imca supports these algorithms as well as the synthesis of ε-optimal piecewise constant timed policies for timed reachability objectives. Two case studies show the feasibility and scalability of the algorithms.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.-P.: Model-checking algorithms for continuous-time Markov chains. IEEE TSE 29, 524–541 (2003)Google Scholar
  2. 2.
    Bertsekas, D.P., Tsitsiklis, J.N.: An analysis of stochastic shortest path problems. Mathematics of Operations Research 16, 580–595 (1991)MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Böde, E., Herbstritt, M., Hermanns, H., Johr, S., Peikenkamp, T., Pulungan, R., Rakow, J., Wimmer, R., Becker, B.: Compositional dependability evaluation for STATEMATE. IEEE TSE 35, 274–292 (2009)Google Scholar
  4. 4.
    Boudali, H., Crouzen, P., Stoelinga, M.: A rigorous, compositional, and extensible framework for dynamic fault tree analysis. IEEE TSDC 7, 128–143 (2009)Google Scholar
  5. 5.
    Bozzano, M., Cimatti, A., Katoen, J.-P., Nguyen, V., Noll, T., Roveri, M.: Safety, dependability and performance analysis of extended AADL models. The Computer Journal 54, 754–775 (2011)CrossRefGoogle Scholar
  6. 6.
    Brázdil, T., Forejt, V., Krcál, J., Kretínský, J., Kucera, A.: Continuous-time stochastic games with time-bounded reachability. In: FSTTCS. LIPIcs, vol. 4, pp. 61–72. Schloss Dagstuhl (2009)Google Scholar
  7. 7.
    Buchholz, P., Hahn, E.M., Hermanns, H., Zhang, L.: Model Checking Algorithms for CTMDPs. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 225–242. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  8. 8.
    Buchholz, P., Schulz, I.: Numerical analysis of continuous time Markov decision processes over finite horizons. Computers & OR 38, 651–659 (2011)MathSciNetMATHCrossRefGoogle Scholar
  9. 9.
    Chatterjee, K., Henzinger, M.: Faster and dynamic algorithms for maximal end-component decomposition and related graph problems in probabilistic verification. In: Symp. on Discrete Algorithms (SODA), pp. 1318–1336. SIAM (2011)Google Scholar
  10. 10.
    Cloth, L., Haverkort, B.R.: Model checking for survivability. In: QEST, pp. 145–154. IEEE Computer Society (2005)Google Scholar
  11. 11.
    Coste, N., Garavel, H., Hermanns, H., Lang, F., Mateescu, R., Serwe, W.: Ten Years of Performance Evaluation for Concurrent Systems Using CADP. In: Margaria, T., Steffen, B. (eds.) ISoLA 2010, Part II. LNCS, vol. 6416, pp. 128–142. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  12. 12.
    Coste, N., Hermanns, H., Lantreibecq, E., Serwe, W.: Towards Performance Prediction of Compositional Models in Industrial GALS Designs. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, pp. 204–218. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  13. 13.
    de Alfaro, L.: Formal Verification of Probabilistic Systems. PhD thesis, Stanford University (1997)Google Scholar
  14. 14.
    de Alfaro, L.: How to specify and verify the long-run average behavior of probabilistic systems. In: LICS, pp. 454–465. IEEE CS Press (1998)Google Scholar
  15. 15.
    de Alfaro, L.: Computing Minimum and Maximum Reachability Times in Probabilistic Systems. In: Baeten, J.C.M., Mauw, S. (eds.) CONCUR 1999. LNCS, vol. 1664, pp. 66–81. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  16. 16.
    Eisentraut, C., Hermanns, H., Zhang, L.: On probabilistic automata in continuous time. In: LICS, pp. 342–351. IEEE Computer Society (2010)Google Scholar
  17. 17.
    Haverkort, B.R., Hermanns, H., Katoen, J.-P.: On the use of model checking techniques for dependability evaluation. In: SRDS, pp. 228–237. IEEE CS (2000)Google Scholar
  18. 18.
    Hermanns, H. (ed.): Interactive Markov Chains. LNCS, vol. 2428. Springer, Heidelberg (2002)MATHGoogle Scholar
  19. 19.
    Hermanns, H., Katoen, J.-P.: The How and Why of Interactive Markov Chains. In: de Boer, F.S., Bonsangue, M.M., Hallerstede, S., Leuschel, M. (eds.) FMCO 2009. LNCS, vol. 6286, pp. 311–338. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  20. 20.
    Johr, S.: Model Checking Compositional Markov Systems. PhD thesis, Saarland University (2007)Google Scholar
  21. 21.
    Knast, R.: Continuous-time probabilistic automata. Information and Control 15, 335–352 (1969)MathSciNetMATHCrossRefGoogle Scholar
  22. 22.
    López, G.G.I., Hermanns, H., Katoen, J.-P.: Beyond Memoryless Distributions: Model Checking Semi-Markov Chains. In: de Luca, L., Gilmore, S. (eds.) PAPM-PROBMIV 2001. LNCS, vol. 2165, pp. 57–70. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  23. 23.
    Norris, J.: Markov Chains. Cambridge University Press (1997)Google Scholar
  24. 24.
    Rabe, M.N., Schewe, S.: Finite optimal control for time-bounded reachability in CTMDPs and continuous-time Markov games. Acta Inf. 48, 291–315 (2011)MathSciNetMATHCrossRefGoogle Scholar
  25. 25.
    von Essen, C., Jobstmann, B.: Synthesizing systems with optimal average-case behavior for ratio objectives. In: iWIGP. EPTCS, vol. 50, pp. 17–32 (2011)Google Scholar
  26. 26.
    Yushtein, Y., Bozzano, M., Cimatti, A., Katoen, J.-P., Nguyen, V.Y., Noll, T., Olive, X., Roveri, M.: System-software co-engineering: Dependability and safety perspective. In: SMC-IT, pp. 18–25. IEEE Computer Society (2011)Google Scholar
  27. 27.
    Zhang, L., Neuhäußer, M.R.: Model Checking Interactive Markov Chains. In: Esparza, J., Majumdar, R. (eds.) TACAS 2010. LNCS, vol. 6015, pp. 53–68. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Dennis Guck
    • 1
  • Tingting Han
    • 2
  • Joost-Pieter Katoen
    • 1
  • Martin R. Neuhäußer
    • 3
  1. 1.RWTH Aachen UniversityGermany
  2. 2.University of OxfordUK
  3. 3.Saarland UniversityGermany

Personalised recommendations