Abstract
We study the problem of model checking Interval-valued Discrete-time Markov Chains (IDTMC). IDTMCs are discrete-time finite Markov Chains for which the exact transition probabilities are not known. Instead in IDTMCs, each transition is associated with an interval in which the actual transition probability must lie. We consider two semantic interpretations for the uncertainty in the transition probabilities of an IDTMC. In the first interpretation, we think of an IDTMC as representing a (possibly uncountable) family of (classical) discrete-time Markov Chains, where each member of the family is a Markov Chain whose transition probabilities lie within the interval range given in the IDTMC. We call this semantic interpretation Uncertain Markov Chains (UMC). In the second semantics for an IDTMC, which we call Interval Markov Decision Process (IMDP), we view the uncertainty as being resolved through non-determinism. In other words, each time a state is visited, we adversarially pick a transition distribution that respects the interval constraints, and take a probabilistic step according to the chosen distribution. We introduce a logic ω-PCTL that can express liveness, strong fairness, and ω-regular properties (such properties cannot be expressed in PCTL). We show that the ω-PCTL model checking problem for Uncertain Markov Chain semantics is decidable in PSPACE (same as the best known upper bound for PCTL) and for Interval Markov Decision Process semantics is decidable in coNP (improving the previous known PSPACE bound for PCTL). We also show that the qualitative fragment of the logic can be solved in coNP for the UMC interpretation, and can be solved in polynomial time for a sub-class of UMCs. We also prove lower bounds for these model checking problems. We show that the model checking problem of IDTMCs with LTL formulas can be solved for both UMC and IMDP semantics by reduction to the model checking problem of IDTMC with ω-PCTL formulas.
This research was supported in part by the AFOSR MURI grant F49620-00-1-0327, the NSF grant CNS-0720906, the NSF ITR grant CCR-0225610 and CCR-0234690, the Swiss National Science Foundation (NCCR MICS and Indo-Swiss Research Programme), and the ARTIST2 European Network of Excellence.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Bianco, A., de Alfaro, L.: Model checking of probabilistic and nondeterministic systems. In: Thiagarajan, P.S. (ed.) FSTTCS 1995. LNCS, vol. 1026, Springer, Heidelberg (1995)
Canny, J.: Some algebraic and geometric computations in PSPACE. In: STOC 1988, pp. 460–467. ACM, New York (1988)
Chatterjee, K., de Alfaro, L., Henzinger, T.: Trading memory for randomness. In: QEST 2004, IEEE, Los Alamitos (2004)
Chatterjee, K., Jurdziński, M., Henzinger, T.: Quantitative stochastic parity games. In: SODA 2004, ACM-SIAM (2004)
Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 2nd edn. MIT Press and McGraw-Hill (1990)
Courcoubetis, C., Yannakakis, M.: The complexity of probabilistic verification. Journal of ACM 42(4), 857–907 (1995)
de Alfaro, L.: Formal Verification of Probabilistic Systems. PhD thesis, Stanford University (1997)
Fecher, H., Leucker, M., Wolf, V.: Don’t know in probabilistic systems. In: Valmari, A. (ed.) SPIN 2006. LNCS, vol. 3925, Springer, Heidelberg (2006)
Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Aspects of Computing 6(5), 512–535 (1994)
Hopcroft, J.E., Ullman, J.D.: Introduction to Automata Theory, Languages and Computation. Addison-Wesley, Reading, MA (1979)
Jonsson, B., Larsen, K.G.: Specification and refinement of probabilistic processes. In: LICS 1991, IEEE, Los Alamitos (1991)
Kemeny, J., Snell, J., Knapp, A.: Denumerable Markov chains. Springer, Heidelberg (1976)
Kozine, I.O., Utkin, L.V.: Interval-valued finite Markov chains. Reliable Computing 8(2), 97–113 (2002)
Kuznetsov, V.P.: Interval statistical models. Radio and Communication (1991)
Rutten, J., Kwiatkowska, M., Norman, G., Parker, D.: Mathematical Techniques for Analyzing Concurrent and Probabilistic Systems. American Mathematical Society (2004)
Safra, S.: Complexity of automata on infinite objects. PhD thesis, Weizmann Institute of Science (1989)
Segala, R.: Modeling and Verification of Randomized Distributed Real-Time Systems. PhD thesis, MIT (1995)
Sen, K., Viswanathan, M., Agha, G.: Model-checking Markov chains in the presence of uncertainties. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, Springer, Heidelberg (2006)
Thomas, W.: Languages, automata, and logic. In: Handbook of Formal Languages. Beyond Words, ch. 7, vol. 3, Springer, Heidelberg (1997)
Vardi, M., Wolper, P.: An automata-theoretic approach to automatic program verification. In: LICS 1986, IEEE, Los Alamitos (1986)
Vardi, M.Y.: Automatic verification of probabilistic concurrent finite-state programs. In: FOCS 1985, IEEE, Los Alamitos (1985)
Walley, P.: Measures of uncertainty in expert systems. Artificial Intelligence 83, 1–58 (1996)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chatterjee, K., Sen, K., Henzinger, T.A. (2008). Model-Checking ω-Regular Properties of Interval Markov Chains. In: Amadio, R. (eds) Foundations of Software Science and Computational Structures. FoSSaCS 2008. Lecture Notes in Computer Science, vol 4962. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78499-9_22
Download citation
DOI: https://doi.org/10.1007/978-3-540-78499-9_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-78497-5
Online ISBN: 978-3-540-78499-9
eBook Packages: Computer ScienceComputer Science (R0)