Analysis of Recursive Probabilistic Models

  • Mihalis Yannakakis
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4218)


In this talk we will discuss recent work on the modeling and analysis of systems that involve recursion and probability. Both, recursion and probability, are fundamental constructs that arise in a wide variety of settings in computer science and other disciplines.


Markov Chain Model Check Markov Decision Process Stochastic Game Recursive Program 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alur, R., Yannakakis, M.: Model checking of hierarchical state machines. ACM Trans. Prog. Lang. Sys. 23(3), 273–303 (2001)CrossRefGoogle Scholar
  2. 2.
    Alur, R., Benedikt, M., Etessami, K., Godefroid, P., Reps, T.W., Yannakakis, M.: Analysis of recursive state machines. ACM Trans. Progr. Lang. Sys. 27, 786–818 (2005)CrossRefGoogle Scholar
  3. 3.
    Bouajjani, A., Esparza, J., Maler, O.: Reachability analysis of pushdown automata: Applications to model checking. In: Mazurkiewicz, A., Winkowski, J. (eds.) CONCUR 1997. LNCS, vol. 1243, pp. 135–150. Springer, Heidelberg (1997)Google Scholar
  4. 4.
    Brázdil, T., Brozek, V., Forejt, V., Kučera, A.: Reachability in recursive Markov decision processes. In: Baier, C., Hermanns, H. (eds.) CONCUR 2006. LNCS, vol. 4137, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    Brázdil, T., Kučera, A., Esparza, J.: Analysis and prediction of the long-run behavior of probabilistic sequential programs with recursion. In: Proc. of FOCS 2005, pp. 521–530 (2005)Google Scholar
  6. 6.
    Brázdil, T., Kučera, A., Stražovský, O.: Decidability of temporal properties of probabilistic pushdown automata. In: Diekert, V., Durand, B. (eds.) STACS 2005. LNCS, vol. 3404, Springer, Heidelberg (2005)CrossRefGoogle Scholar
  7. 7.
    Basu, S., Pollack, R., Roy, M.F.: On the combinatorial and algebraic complexity of quantifier elimination. J. ACM 43(6), 1002–1045 (1996)MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Canny, J.: Some algebraic and geometric computations in PSPACE. In: Prof. of 20th ACM STOC, pp. 460–467 (1988)Google Scholar
  9. 9.
    Condon, A.: The complexity of stochastic games. Inf. & Comp. 96(2), 203–224 (1992)MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Courcoubetis, C., Yannakakis, M.: The complexity of probabilistic verification. Journal of the ACM 42(4), 857–907 (1995)MATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Courcoubetis, C., Yannakakis, M.: Markov decision processes and regular events. IEEE Trans. on Automatic Control 43(10), 1399–1418 (1998)MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    de Alfaro, L., Kwiatkowska, M., Norman, G., Parker, D., Segala, R.: Symbolic model checking of probabilistic processes using MTBDDs and the kronecker representation. In: Schwartzbach, M.I., Graf, S. (eds.) ETAPS 2000 and TACAS 2000. LNCS, vol. 1785, pp. 395–410. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  13. 13.
    de Alfaro, L., Majumdar, R.: Quantitative solution of omega-regular games. J. Comp. Sys. Sc. 68(2), 374–397 (2004)MATHCrossRefGoogle Scholar
  14. 14.
    Esparza, J., Hansel, D., Rossmanith, P., Schwoon, S.: Efficient algorithms for model checking pushdown systems. In: Emerson, E.A., Sistla, A.P. (eds.) CAV 2000. LNCS, vol. 1855, pp. 232–247. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  15. 15.
    Esparza, J., Kučera, A., Mayr, R.: Model checking probabilistic pushdown automata. In: Proc. of 19th IEEE LICS 2004 (2004); Full version in Logical Methods in Computer Science, vol. 2(1) (2006)Google Scholar
  16. 16.
    Esparza, J., Kučera, A., Mayr, R.: Quantitative analysis of probabilistic pushdown automata: expectations and variances. In: Proc. of 20th IEEE LICS (2005)Google Scholar
  17. 17.
    Etessami, K., Yannakakis, M.: Recursive Markov chains, stochastic grammars, and monotone systems of non-linear equations. In: Diekert, V., Durand, B. (eds.) STACS 2005. LNCS, vol. 3404. Springer, Heidelberg (2005); Full expanded version available from: CrossRefGoogle Scholar
  18. 18.
    Etessami, K., Yannakakis, M.: Algorithmic verification of recursive probabilistic state machines. In: Halbwachs, N., Zuck, L.D. (eds.) TACAS 2005. LNCS, vol. 3440, pp. 253–270. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  19. 19.
    Etessami, K., Yannakakis, M.: Recursive Markov Decision Processes and Recursive Stochastic Games. In: Caires, L., Italiano, G.F., Monteiro, L., Palamidessi, C., Yung, M. (eds.) ICALP 2005. LNCS, vol. 3580, pp. 891–903. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  20. 20.
    Etessami, K., Yannakakis, M.: Checking LTL Properties of Recursive Markov Chains. In: Proc. 2nd Intl. Conf. on Quantitative Evaluation of Systems. IEEE, Los Alamitos (2005)Google Scholar
  21. 21.
    Etessami, K., Yannakakis, M.: Efficient Analysis of Classes of Recursive Markov Decision Processes and Stochastic Games. In: Durand, B., Thomas, W. (eds.) STACS 2006. LNCS, vol. 3884, pp. 634–645. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  22. 22.
    Etessami, K., Yannakakis, M.: Recursive concurrent stochastic games. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 324–335. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  23. 23.
    Everett, C.J., Ulam, S.: Multiplicative systems, part i., ii, and iii. Technical Report 683, 690, 707, Los Alamos Scientific Laboratory (1948)Google Scholar
  24. 24.
    Fagin, R., Karlin, A., Kleinberg, J., Raghavan, P., Rajagopalan, S., Rubinfeld, R., Sudan, M., Tomkins, A.: Random walks with “back buttons” (extended abstract). In: ACM Symp. on Theory of Computing, pp. 484–493 (2000)Google Scholar
  25. 25.
    Feinberg, E., Shwartz, A. (eds.): Handbook of Markov Decision Processes. Kluwer, Dordrecht (2002)MATHGoogle Scholar
  26. 26.
    Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer, Heidelberg (1997)MATHGoogle Scholar
  27. 27.
    Garey, M.R., Graham, R.L., Johnson, D.S.: Some NP-complete geometric problems. In: 8th ACM Symp. on Theory of Computing, pp. 10–22 (1976)Google Scholar
  28. 28.
    Haccou, P., Jagers, P., Vatutin, V.A.: Branching Processes: Variation, Growth, and Extinction of Populations. Cambridge University Press, Cambridge (2005)MATHCrossRefGoogle Scholar
  29. 29.
    Harris, T.E.: The Theory of Branching Processes. Springer, Heidelberg (1963)MATHGoogle Scholar
  30. 30.
    Hinton, A., Kwiatkowska, M.Z., Norman, G., Parker, D.: PRISM: A tool for automatic verification of probabilistic systems. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006 and ETAPS 2006. LNCS, vol. 3920, pp. 441–444. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  31. 31.
    Jagers, P.: Branching Processes with Biological Applications. Wiley, Chichester (1975)MATHGoogle Scholar
  32. 32.
    Kolmogorov, A.N., Sevastyanov, B.A.: The calculation of final probabilities for branching random processes. Dokl. Akad. Nauk SSSR 56, 783–786 (1947) (Russian)MATHGoogle Scholar
  33. 33.
    Kwiatkowska, M.: Model checking for probability and time: from theory to practice. In: 18th IEEE LICS, pp. 351–360 (2003)Google Scholar
  34. 34.
    Manning, C., Schütze, H.: Foundations of Statistical Natural Language Processing. MIT Press, Cambridge (1999)MATHGoogle Scholar
  35. 35.
    Paz, A.: Introduction to Probabilistic Automata. Academic Press, London (1971)MATHGoogle Scholar
  36. 36.
    Pnueli, A., Zuck, L.D.: Probabilistic verification. Inf. and Comp. 103(1), 1–29 (1993)MATHCrossRefMathSciNetGoogle Scholar
  37. 37.
    Puterman, M.L.: Markov Decision Processes. Wiley, Chichester (1994)MATHCrossRefGoogle Scholar
  38. 38.
    Renegar, J.: On the computational complexity and geometry of the first-order theory of the reals, parts I-III. J. Symb. Comp. 13(3), 255–352 (1992)MATHMathSciNetCrossRefGoogle Scholar
  39. 39.
    Sakakibara, Y., Brown, M., Hughey, R., Mian, I.S., Sjolander, K., Underwood, R., Haussler, D.: Stochastic context-free grammars for tRNA modeling. Nucleic Acids Research 22(23), 5112–5120 (1994)CrossRefGoogle Scholar
  40. 40.
    Sevastyanov, B.A.: The theory of branching processes. Uspehi Mathemat. Nauk 6, 47–99 (1951) (Russian)MathSciNetGoogle Scholar
  41. 41.
    Shapley, L.S.: Stochastic games. Proc. Nat. Acad. Sci. 39, 1095–1100 (1953)MATHCrossRefMathSciNetGoogle Scholar
  42. 42.
    Tiwari, P.: A problem that is easier to solve on the unit-cost algebraic ram. Journal of Complexity, 393–397 (1992)Google Scholar
  43. 43.
    Vardi, M.: Automatic verification of probabilistic concurrent finite-state programs. In: Proc. of 26th IEEE FOCS, pp. 327–338 (1985)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Mihalis Yannakakis
    • 1
  1. 1.Department of Computer ScienceColumbia University 

Personalised recommendations