Model-Checking Markov Chains in the Presence of Uncertainties

  • Koushik Sen
  • Mahesh Viswanathan
  • Gul Agha
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3920)


We investigate the problem of model checking Interval-valued Discrete-time Markov Chains (IDTMC). IDTMCs are discrete-time finite Markov Chains for which the exact transition probabilities are not known. Instead in IDTMCs, each transition is associated with an interval in which the actual transition probability must lie. We consider two semantic interpretations for the uncertainty in the transition probabilities of an IDTMC. In the first interpretation, we think of an IDTMC as representing a (possibly uncountable) family of (classical) discrete-time Markov Chains, where each member of the family is a Markov Chain whose transition probabilities lie within the interval range given in the IDTMC. This semantic interpretation we call Uncertain Markov Chains (UMC). In the second semantics for an IDTMC, which we call Interval Markov Decision Process (IMDP), we view the uncertainty as being resolved through non-determinism. In other words, each time a state is visited, we adversarially pick a transition distribution that respects the interval constraints, and take a probabilistic step according to the chosen distribution. We show that the PCTL model checking problem for both Uncertain Markov Chain semantics and Interval Markov Decision Process semantics is decidable in PSPACE. We also prove lower bounds for these model checking problems.


  1. 1.
    Aziz, A., Singhal, V., Brayton, R.K., Sangiovanni-Vincentelli, A.L.: It usually works: The temporal logic of stochastic systems. In: Wolper, P. (ed.) CAV 1995, vol. 939, pp. 155–165. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  2. 2.
    Baier, C.: On algorithmic verification methods for probabilistic systems. Habilitation Thesis. Fakultät f’ür Mathematik and Informatik, Universität Mannheim (1998)Google Scholar
  3. 3.
    Baier, C., Kwiatkowska, M.Z.: Model checking for a probabilistic branching time logic with fairness. Distributed Computing 11(3), 125–155 (1998)CrossRefGoogle Scholar
  4. 4.
    Bianco, A., de Alfaro, L.: Model checking of probabilistic and nondeterministic systems. In: Thiagarajan, P.S. (ed.) FSTTCS 1995. LNCS, vol. 1026, p. 515. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  5. 5.
    Blum, L., Shub, M., Smale, S.: On a theory of computation and complexity over real numbers: NP-completeness, recursive functions and universal machines. Bulletin of the American Mathematical Society 21, 1–46 (1989)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Canny, J.: Some algebraic and geometric computations in PSPACE. In: 20th ACM Symposium on Theory of Computing (STOC 1988), pp. 460–467 (1988)Google Scholar
  7. 7.
    Courcoubetis, C., Yannakakis, M.: Markov decision processes and regular events. In: Proceedings of the seventeenth international colloquium on Automata, languages and programming, pp. 336–349 (1990)Google Scholar
  8. 8.
    Courcoubetis, C., Yannakakis, M.: The complexity of probabilistic verification. Journal of ACM 42(4), 857–907 (1995)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Fukuda, M., Kojima, M.: Branch-and-cut algorithms for the bilinear matrix inequality eigenvalue problem. Comput. Optim. Appl. 19(1), 79–105 (2001)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Goh, K.C., Safonov, M.G., Papavassilopoulos, G.P.: Global optimization for the biaffine matrix inequality problem. Journal of Global Optimization 7, 365–380 (1995)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Aspects of Computing 6(5), 512–535 (1994)CrossRefMATHGoogle Scholar
  12. 12.
    Jonsson, B., Larsen, K.G.: Specification and refinement of probabilistic processes. In: Proceedings of the IEEE Symposium on Logic in Computer Science, pp. 266–277 (1991)Google Scholar
  13. 13.
    Kemeny, J., Snell, J., Knapp, A.: Denumerable Markov chains. Springer, Heidelberg (1976)CrossRefMATHGoogle Scholar
  14. 14.
    Kozine, I.O., Utkin, L.V.: Interval-valued finite markov chains. Reliable Computing 8(2), 97–113 (2002)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Kuznetsov, V.P.: Interval statistical models. Radio and Communication (1991)Google Scholar
  16. 16.
  17. 17.
    Puterman, M.: Markov decision processes: discrete stochastic dynamic programming. Wiley, New York (1994)CrossRefMATHGoogle Scholar
  18. 18.
    Renegar, J.: A faster pspace algorithm for deciding the existential theory of the reals. In: 29th Annual IEEE Symposium on Foundations of Computer Science, pp. 291–295 (1988)Google Scholar
  19. 19.
    Rutten, J., Kwiatkowska, M., Norman, G., Parker, D.: Mathematical Techniques for Analyzing Concurrent and Probabilistic Systems. CRM Monograph Series, vol. 23. American Mathematical Society (2004)Google Scholar
  20. 20.
    Segala, R.: Modeling and Verification of Randomized Distributed Real-Time Systems. PhD thesis. MIT, Cambridge (1995)Google Scholar
  21. 21.
    Segala, R., Lynch, N.A.: Probabilistic simulations for probabilistic processes. In: Jonsson, B., Parrow, J. (eds.) CONCUR 1994, vol. 836, pp. 481–496. Springer, Heidelberg (1994)CrossRefGoogle Scholar
  22. 22.
    Sen, K., Viswanathan, M., Agha, G.: Statistical model checking of black-box probabilistic systems. In: Alur, R., Peled, D.A. (eds.) CAV 2004. LNCS, vol. 3114, pp. 202–215. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  23. 23.
    Sen, K., Viswanathan, M., Agha, G.: Model-checking markov chains in the presence of uncertainties. Technical Report UIUCDCS-R-2006-2677, UIUC (2006)Google Scholar
  24. 24.
    Toker, O., Oz̈bay, H.: On the NP-hardness of solving bilinear matrix in equalities and simultaneous stabilization with static output feedback. In: Proc. of American Control Conference (1995)Google Scholar
  25. 25.
    Vardi, M.Y.: Automatic verification of probabilistic concurrent finite-state programs. In: 26th Annual Symposium on Foundations of Computer Science, pp. 327–338. IEEE, Los Alamitos (1985)Google Scholar
  26. 26.
    Walley, P.: Measures of uncertainty in expert systems. Artificial Intelligence 83, 1–58 (1996)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Koushik Sen
    • 1
  • Mahesh Viswanathan
    • 1
  • Gul Agha
    • 1
  1. 1.Department of Computer ScienceUniversity of Illinois at Urbana-ChampaignUSA

Personalised recommendations