Model-Checking ω-Regular Properties of Interval Markov Chains

  • Krishnendu Chatterjee
  • Koushik Sen
  • Thomas A. Henzinger
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4962)


We study the problem of model checking Interval-valued Discrete-time Markov Chains (IDTMC). IDTMCs are discrete-time finite Markov Chains for which the exact transition probabilities are not known. Instead in IDTMCs, each transition is associated with an interval in which the actual transition probability must lie. We consider two semantic interpretations for the uncertainty in the transition probabilities of an IDTMC. In the first interpretation, we think of an IDTMC as representing a (possibly uncountable) family of (classical) discrete-time Markov Chains, where each member of the family is a Markov Chain whose transition probabilities lie within the interval range given in the IDTMC. We call this semantic interpretation Uncertain Markov Chains (UMC). In the second semantics for an IDTMC, which we call Interval Markov Decision Process (IMDP), we view the uncertainty as being resolved through non-determinism. In other words, each time a state is visited, we adversarially pick a transition distribution that respects the interval constraints, and take a probabilistic step according to the chosen distribution. We introduce a logic ω-PCTL that can express liveness, strong fairness, and ω-regular properties (such properties cannot be expressed in PCTL). We show that the ω-PCTL model checking problem for Uncertain Markov Chain semantics is decidable in PSPACE (same as the best known upper bound for PCTL) and for Interval Markov Decision Process semantics is decidable in coNP (improving the previous known PSPACE bound for PCTL). We also show that the qualitative fragment of the logic can be solved in coNP for the UMC interpretation, and can be solved in polynomial time for a sub-class of UMCs. We also prove lower bounds for these model checking problems. We show that the model checking problem of IDTMCs with LTL formulas can be solved for both UMC and IMDP semantics by reduction to the model checking problem of IDTMC with ω-PCTL formulas.


Markov Chain Model Check Markov Decision Process Existential Theory State Formula 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bianco, A., de Alfaro, L.: Model checking of probabilistic and nondeterministic systems. In: Thiagarajan, P.S. (ed.) FSTTCS 1995. LNCS, vol. 1026, Springer, Heidelberg (1995)Google Scholar
  2. 2.
    Canny, J.: Some algebraic and geometric computations in PSPACE. In: STOC 1988, pp. 460–467. ACM, New York (1988)CrossRefGoogle Scholar
  3. 3.
    Chatterjee, K., de Alfaro, L., Henzinger, T.: Trading memory for randomness. In: QEST 2004, IEEE, Los Alamitos (2004)Google Scholar
  4. 4.
    Chatterjee, K., Jurdziński, M., Henzinger, T.: Quantitative stochastic parity games. In: SODA 2004, ACM-SIAM (2004)Google Scholar
  5. 5.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 2nd edn. MIT Press and McGraw-Hill (1990)Google Scholar
  6. 6.
    Courcoubetis, C., Yannakakis, M.: The complexity of probabilistic verification. Journal of ACM 42(4), 857–907 (1995)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    de Alfaro, L.: Formal Verification of Probabilistic Systems. PhD thesis, Stanford University (1997)Google Scholar
  8. 8.
    Fecher, H., Leucker, M., Wolf, V.: Don’t know in probabilistic systems. In: Valmari, A. (ed.) SPIN 2006. LNCS, vol. 3925, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Aspects of Computing 6(5), 512–535 (1994)zbMATHCrossRefGoogle Scholar
  10. 10.
    Hopcroft, J.E., Ullman, J.D.: Introduction to Automata Theory, Languages and Computation. Addison-Wesley, Reading, MA (1979)zbMATHGoogle Scholar
  11. 11.
    Jonsson, B., Larsen, K.G.: Specification and refinement of probabilistic processes. In: LICS 1991, IEEE, Los Alamitos (1991)Google Scholar
  12. 12.
    Kemeny, J., Snell, J., Knapp, A.: Denumerable Markov chains. Springer, Heidelberg (1976)zbMATHGoogle Scholar
  13. 13.
    Kozine, I.O., Utkin, L.V.: Interval-valued finite Markov chains. Reliable Computing 8(2), 97–113 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Kuznetsov, V.P.: Interval statistical models. Radio and Communication (1991)Google Scholar
  15. 15.
    Rutten, J., Kwiatkowska, M., Norman, G., Parker, D.: Mathematical Techniques for Analyzing Concurrent and Probabilistic Systems. American Mathematical Society (2004)Google Scholar
  16. 16.
    Safra, S.: Complexity of automata on infinite objects. PhD thesis, Weizmann Institute of Science (1989)Google Scholar
  17. 17.
    Segala, R.: Modeling and Verification of Randomized Distributed Real-Time Systems. PhD thesis, MIT (1995)Google Scholar
  18. 18.
    Sen, K., Viswanathan, M., Agha, G.: Model-checking Markov chains in the presence of uncertainties. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  19. 19.
    Thomas, W.: Languages, automata, and logic. In: Handbook of Formal Languages. Beyond Words, ch. 7, vol. 3, Springer, Heidelberg (1997)Google Scholar
  20. 20.
    Vardi, M., Wolper, P.: An automata-theoretic approach to automatic program verification. In: LICS 1986, IEEE, Los Alamitos (1986)Google Scholar
  21. 21.
    Vardi, M.Y.: Automatic verification of probabilistic concurrent finite-state programs. In: FOCS 1985, IEEE, Los Alamitos (1985)Google Scholar
  22. 22.
    Walley, P.: Measures of uncertainty in expert systems. Artificial Intelligence 83, 1–58 (1996)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Krishnendu Chatterjee
    • 1
  • Koushik Sen
    • 1
  • Thomas A. Henzinger
    • 1
    • 2
  1. 1.University of CaliforniaBerkeleyUSA
  2. 2.EPFLSwitzerland

Personalised recommendations