Abstract
We investigate the problem of model checking Interval-valued Discrete-time Markov Chains (IDTMC). IDTMCs are discrete-time finite Markov Chains for which the exact transition probabilities are not known. Instead in IDTMCs, each transition is associated with an interval in which the actual transition probability must lie. We consider two semantic interpretations for the uncertainty in the transition probabilities of an IDTMC. In the first interpretation, we think of an IDTMC as representing a (possibly uncountable) family of (classical) discrete-time Markov Chains, where each member of the family is a Markov Chain whose transition probabilities lie within the interval range given in the IDTMC. This semantic interpretation we call Uncertain Markov Chains (UMC). In the second semantics for an IDTMC, which we call Interval Markov Decision Process (IMDP), we view the uncertainty as being resolved through non-determinism. In other words, each time a state is visited, we adversarially pick a transition distribution that respects the interval constraints, and take a probabilistic step according to the chosen distribution. We show that the PCTL model checking problem for both Uncertain Markov Chain semantics and Interval Markov Decision Process semantics is decidable in PSPACE. We also prove lower bounds for these model checking problems.
Chapter PDF
Similar content being viewed by others
References
Aziz, A., Singhal, V., Brayton, R.K., Sangiovanni-Vincentelli, A.L.: It usually works: The temporal logic of stochastic systems. In: Wolper, P. (ed.) CAV 1995, vol. 939, pp. 155–165. Springer, Heidelberg (1995)
Baier, C.: On algorithmic verification methods for probabilistic systems. Habilitation Thesis. Fakultät f’ür Mathematik and Informatik, Universität Mannheim (1998)
Baier, C., Kwiatkowska, M.Z.: Model checking for a probabilistic branching time logic with fairness. Distributed Computing 11(3), 125–155 (1998)
Bianco, A., de Alfaro, L.: Model checking of probabilistic and nondeterministic systems. In: Thiagarajan, P.S. (ed.) FSTTCS 1995. LNCS, vol. 1026, p. 515. Springer, Heidelberg (1995)
Blum, L., Shub, M., Smale, S.: On a theory of computation and complexity over real numbers: NP-completeness, recursive functions and universal machines. Bulletin of the American Mathematical Society 21, 1–46 (1989)
Canny, J.: Some algebraic and geometric computations in PSPACE. In: 20th ACM Symposium on Theory of Computing (STOC 1988), pp. 460–467 (1988)
Courcoubetis, C., Yannakakis, M.: Markov decision processes and regular events. In: Proceedings of the seventeenth international colloquium on Automata, languages and programming, pp. 336–349 (1990)
Courcoubetis, C., Yannakakis, M.: The complexity of probabilistic verification. Journal of ACM 42(4), 857–907 (1995)
Fukuda, M., Kojima, M.: Branch-and-cut algorithms for the bilinear matrix inequality eigenvalue problem. Comput. Optim. Appl. 19(1), 79–105 (2001)
Goh, K.C., Safonov, M.G., Papavassilopoulos, G.P.: Global optimization for the biaffine matrix inequality problem. Journal of Global Optimization 7, 365–380 (1995)
Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Aspects of Computing 6(5), 512–535 (1994)
Jonsson, B., Larsen, K.G.: Specification and refinement of probabilistic processes. In: Proceedings of the IEEE Symposium on Logic in Computer Science, pp. 266–277 (1991)
Kemeny, J., Snell, J., Knapp, A.: Denumerable Markov chains. Springer, Heidelberg (1976)
Kozine, I.O., Utkin, L.V.: Interval-valued finite markov chains. Reliable Computing 8(2), 97–113 (2002)
Kuznetsov, V.P.: Interval statistical models. Radio and Communication (1991)
PENbmi, http://www.penopt.com/
Puterman, M.: Markov decision processes: discrete stochastic dynamic programming. Wiley, New York (1994)
Renegar, J.: A faster pspace algorithm for deciding the existential theory of the reals. In: 29th Annual IEEE Symposium on Foundations of Computer Science, pp. 291–295 (1988)
Rutten, J., Kwiatkowska, M., Norman, G., Parker, D.: Mathematical Techniques for Analyzing Concurrent and Probabilistic Systems. CRM Monograph Series, vol. 23. American Mathematical Society (2004)
Segala, R.: Modeling and Verification of Randomized Distributed Real-Time Systems. PhD thesis. MIT, Cambridge (1995)
Segala, R., Lynch, N.A.: Probabilistic simulations for probabilistic processes. In: Jonsson, B., Parrow, J. (eds.) CONCUR 1994, vol. 836, pp. 481–496. Springer, Heidelberg (1994)
Sen, K., Viswanathan, M., Agha, G.: Statistical model checking of black-box probabilistic systems. In: Alur, R., Peled, D.A. (eds.) CAV 2004. LNCS, vol. 3114, pp. 202–215. Springer, Heidelberg (2004)
Sen, K., Viswanathan, M., Agha, G.: Model-checking markov chains in the presence of uncertainties. Technical Report UIUCDCS-R-2006-2677, UIUC (2006)
Toker, O., Oz̈bay, H.: On the NP-hardness of solving bilinear matrix in equalities and simultaneous stabilization with static output feedback. In: Proc. of American Control Conference (1995)
Vardi, M.Y.: Automatic verification of probabilistic concurrent finite-state programs. In: 26th Annual Symposium on Foundations of Computer Science, pp. 327–338. IEEE, Los Alamitos (1985)
Walley, P.: Measures of uncertainty in expert systems. Artificial Intelligence 83, 1–58 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sen, K., Viswanathan, M., Agha, G. (2006). Model-Checking Markov Chains in the Presence of Uncertainties. In: Hermanns, H., Palsberg, J. (eds) Tools and Algorithms for the Construction and Analysis of Systems. TACAS 2006. Lecture Notes in Computer Science, vol 3920. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11691372_26
Download citation
DOI: https://doi.org/10.1007/11691372_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33056-1
Online ISBN: 978-3-540-33057-8
eBook Packages: Computer ScienceComputer Science (R0)