Compositional Verification and Optimization of Interactive Markov Chains

  • Holger Hermanns
  • Jan Krčál
  • Jan Křetínský
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8052)


Interactive Markov chains (IMC) are compositional behavioural models extending labelled transition systems and continuous-time Markov chains. We provide a framework and algorithms for compositional verification and optimization of IMC with respect to time-bounded properties. Firstly, we give a specification formalism for IMC. Secondly, given a time-bounded property, an IMC component and the assumption that its unknown environment satisfies a given specification, we synthesize a scheduler for the component optimizing the probability that the property is satisfied in any such environment.


External Action Internal Transition Stochastic Game Parallel Composition Label Transition System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [AH96]
    Alur, R., Henzinger, T.A.: Reactive modules. In: LICS, pp. 207–218 (1996)Google Scholar
  2. [BF09]
    Bouyer, P., Forejt, V.: Reachability in stochastic timed games. In: Albers, S., Marchetti-Spaccamela, A., Matias, Y., Nikoletseas, S., Thomas, W. (eds.) ICALP 2009, Part II. LNCS, vol. 5556, pp. 103–114. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. [BHK+12]
    Brázdil, T., Hermanns, H., Krčál, J., Křetínský, J., Řehák, V.: Verification of open interactive markov chains. In: FSTTCS, pp. 474–485 (2012)Google Scholar
  4. [BHKH05]
    Baier, C., Hermanns, H., Katoen, J.-P., Haverkort, B.R.: Efficient computation of time-bounded reachability probabilities in uniform continuous-time Markov decision processes. Theor. Comp. Sci. 345(1), 2–26 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  5. [BS11]
    Buchholz, P., Schulz, I.: Numerical Analysis of Continuous Time Markov Decision processes over Finite Horizons. Computers and Operations Research 38, 651–659 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  6. [CD10]
    Chatterjee, K., Doyen, L.: The complexity of partial-observation parity games. In: Fermüller, C.G., Voronkov, A. (eds.) LPAR-17. LNCS, vol. 6397, pp. 1–14. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. [DLL+12]
    David, A., Larsen, K.G., Legay, A., Møller, M.H., Nyman, U., Ravn, A.P., Skou, A., Wasowski, A.: Compositional verification of real-time systems using ECDAR. STTT 14(6), 703–720 (2012)CrossRefGoogle Scholar
  8. [EKN+12]
    Esteve, M.-A., Katoen, J.-P., Nguyen, V.Y., Postma, B., Yushtein, Y.: Formal correctness, safety, dependability and performance analysis of a satellite. In: Proc. of ICSE. ACM and IEEE Press (2012)Google Scholar
  9. [GHKN12]
    Guck, D., Han, T., Katoen, J.-P., Neuhäußer, M.R.: Quantitative timed analysis of interactive markov chains. In: Goodloe, A.E., Person, S. (eds.) NFM 2012. LNCS, vol. 7226, pp. 8–23. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  10. [HH13]
    Hatefi, H., Hermanns, H.: Improving time bounded reachability computations in interactive Markov chains. In: Arbab, F., Sirjani, M. (eds.) FSEN 2013. LNCS, vol. 8161. Springer, Heidelberg (2013)Google Scholar
  11. [HK09]
    Hermanns, H., Katoen, J.-P.: The how and why of interactive Markov chains. In: de Boer, F.S., Bonsangue, M.M., Hallerstede, S., Leuschel, M. (eds.) FMCO 2009. LNCS, vol. 6286, pp. 311–338. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  12. [HKK13]
    Hermanns, H., Krčál, J., Křetínský, J.: Compositional verification and optimization of interactive markov chains. CoRR, abs/1305.7332 (2013)Google Scholar
  13. [HKR+10]
    Haverkort, B.R., Kuntz, M., Remke, A., Roolvink, S., Stoelinga, M.I.A.: Evaluating repair strategies for a water-treatment facility using Arcade. In: Proc. of DSN, pp. 419–424 (2010)Google Scholar
  14. [HMP01]
    Henzinger, T.A., Minea, M., Prabhu, V.S.: Assume-guarantee reasoning for hierarchical hybrid systems. In: Di Benedetto, M.D., Sangiovanni-Vincentelli, A.L. (eds.) HSCC 2001. LNCS, vol. 2034, pp. 275–290. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  15. [HNP+11]
    Hahn, E.M., Norman, G., Parker, D., Wachter, B., Zhang, L.: Game-based abstraction and controller synthesis for probabilistic hybrid systems. In: QEST, pp. 69–78 (2011)Google Scholar
  16. [KKLW07]
    Katoen, J.-P., Klink, D., Leucker, M., Wolf, V.: Three-valued abstraction for continuous-time Markov chains. In: Damm, W., Hermanns, H. (eds.) CAV 2007. LNCS, vol. 4590, pp. 311–324. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  17. [KKN09]
    Katoen, J.-P., Klink, D., Neuhäußer, M.R.: Compositional abstraction for stochastic systems. In: Ouaknine, J., Vaandrager, F.W. (eds.) FORMATS 2009. LNCS, vol. 5813, pp. 195–211. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  18. [KMvS94]
    Koller, D., Megiddo, N., von Stengel, B.: Fast algorithms for finding randomized strategies in game trees. In: STOC, pp. 750–759 (1994)Google Scholar
  19. [KV96]
    Kupferman, O., Vardi, M.: Module checking. In: Alur, R., Henzinger, T.A. (eds.) CAV 1996. LNCS, vol. 1102, pp. 75–86. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  20. [KZH+11]
    Katoen, J.-P., Zapreev, I.S., Hahn, E.M., Hermanns, H., Jansen, D.N.: The ins and outs of the probabilistic model checker MRMC. Performance Evaluation 68(2), 90–104 (2011)CrossRefGoogle Scholar
  21. [LT88]
    Larsen, K.G., Thomsen, B.: A modal process logic. In: LICS, pp. 203–210 (1988)Google Scholar
  22. [Mar11]
    Markovski, J.: Towards supervisory control of interactive Markov chains: Controllability. In: ACSD, pp. 108–117 (2011) Google Scholar
  23. [MC81]
    Misra, J., Mani Chandy, K.: Proofs of networks of processes. IEEE Trans. Software Eng. 7(4), 417–426 (1981)zbMATHCrossRefGoogle Scholar
  24. [RW89]
    Ramadge, P.J.G., Wonham, W.M.: The control of discrete event systems. Proceedings of the IEEE 77(1) (1989)Google Scholar
  25. [Spr11]
    Sproston, J.: Discrete-time verification and control for probabilistic rectangular hybrid automata. In: QEST, pp. 79–88 (2011)Google Scholar
  26. [TAKB96]
    Tasiran, S., Alur, R., Kurshan, R.P., Brayton, R.K.: Verifying abstractions of timed systems. In: Sassone, V., Montanari, U. (eds.) CONCUR 1996. LNCS, vol. 1119, pp. 546–562. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  27. [ZN10]
    Zhang, L., Neuhäußer, M.R.: Model checking interactive Markov chains. In: Esparza, J., Majumdar, R. (eds.) TACAS 2010. LNCS, vol. 6015, pp. 53–68. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Holger Hermanns
    • 1
  • Jan Krčál
    • 2
  • Jan Křetínský
    • 2
    • 3
  1. 1.Computer ScienceSaarland UniversitySaarbrückenGermany
  2. 2.Faculty of InformaticsMasaryk UniversityCzech Republic
  3. 3.Institut für InformatikTechnical University MunichGermany

Personalised recommendations