Distributed Markov Chains

  • Ratul Saha
  • Javier Esparza
  • Sumit Kumar Jha
  • Madhavan Mukund
  • P. S. Thiagarajan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8931)


The formal verification of large probabilistic models is challenging. Exploiting the concurrency that is often present is one way to address this problem. Here we study a class of communicating probabilistic agents in which the synchronizations determine the probability distribution for the next moves of the participating agents. The key property of this class is that the synchronizations are deterministic, in the sense that any two simultaneously enabled synchronizations involve disjoint sets of agents. As a result, such a network of agents can be viewed as a succinct and distributed presentation of a large global Markov chain. A rich class of Markov chains can be represented this way.

We use partial-order notions to define an interleaved semantics that can be used to efficiently verify properties of the global Markov chain represented by the network. To demonstrate this, we develop a statistical model checking (SMC) procedure and use it to verify two large networks of probabilistic agents.

We also show that our model, called distributed Markov chains (DMCs), is closely related to deterministic cyclic negotiations, a recently introduced model for concurrent systems [10]. Exploiting this connection we show that the termination of a DMC that has been endowed with a global final state can be checked in polynomial time.


Markov Chain Probability Measure Global State Maximal Step Trajectory Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    van der Aalst, W.M.P., van Hee, K.M., ter Hofstede, A.H.M., Sidorova, N., Verbeek, H.M.W., Voorhoeve, M., Wynn, M.T.: Soundness of workflow nets: Classification, decidability, and analysis. Form. Asp. Comp. 23(3), 333–363 (2011)CrossRefzbMATHGoogle Scholar
  2. 2.
    Abbes, S., Benveniste, A.: True-concurrency probabilistic models: Branching cells and distributed probabilities for event structures. Info. and Comp. 204(2), 231–274 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Abbes, S., Benveniste, A.: True-concurrency probabilistic models: Markov nets and a law of large numbers. Theor. Comput. Sci. 390(2-3), 129–170 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Baier, C., Katoen, J.-P.: Principles of Model Checking. The MIT Press (2008)Google Scholar
  5. 5.
    Bogdoll, J., Ferrer Fioriti, L.M., Hartmanns, A., Hermanns, H.: Partial order Methods for statistical model checking and simulation. In: Bruni, R., Dingel, J. (eds.) FMOODS/FORTE 2011. LNCS, vol. 6722, pp. 59–74. Springer, Heidelberg (2011)Google Scholar
  6. 6.
    Boyer, B., Corre, K., Legay, A., Sedwards, S.: PLASMA-lab: A Flexible, sistributable statistical model checking library. In: Joshi, K., Siegle, M., Stoelinga, M., D’Argenio, P.R. (eds.) QEST 2013. LNCS, vol. 8054, pp. 160–164. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  7. 7.
  8. 8.
    Diekert, V., Rozenberg, G.: The book of traces. World Scientific (1995)Google Scholar
  9. 9.
    Esparza, J., Heljanko, K.: Unfoldings: A Partial-Order Approach to Model Checking, 1st edn. Springer Publishing Company (2008)Google Scholar
  10. 10.
    Esparza, J., Desel, J.: On negotiation as concurrency primitive II: Deterministic cyclic negotiations. In: Muscholl, A. (ed.) FOSSACS 2014. LNCS, vol. 8412, pp. 258–273. Springer, Heidelberg (2014)Google Scholar
  11. 11.
    Fokkink, W.: Variations on Itai-Rodeh leader election for anonymous rings and their analysis in PRISM. J. UCS, 12 (2006)Google Scholar
  12. 12.
    Groesser, M., Baier, C.: Partial order reduction for Markov decision processes: A survey. In: de Boer, F.S., Bonsangue, M.M., Graf, S., de Roever, W.-P. (eds.) FMCO 2005. LNCS, vol. 4111, pp. 408–427. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  13. 13.
    Itai, A., Rodeh, M.: Symmetry breaking in distributed networks. Info. and Comp. 88(1), 60–87 (1990)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Jesi, S., Pighizzini, G., Sabadini, N.: Probabilistic asynchronous automata. Math. Systems Theory 29(1), 5–31 (1996)CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    Jha, S.K., Clarke, E.M., Langmead, C.J., Legay, A., Platzer, A., Zuliani, P.: A Bayesian approach to model checking biological systems. In: Degano, P., Gorrieri, R. (eds.) CMSB 2009. LNCS, vol. 5688, pp. 218–234. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  16. 16.
    Clarke Jr., E.M., Grumberg, O., Peled, D.A.: Model Checking. The MIT Press, Cambridge (1999)Google Scholar
  17. 17.
    McMillan, K.L., Probst, D.K.: A technique of state space search based on unfolding. Form. Method. Syst. Des. 6(1), 45–65 (1995)CrossRefzbMATHGoogle Scholar
  18. 18.
    Pnueli, A., Zuck, L.D.: Verification of multiprocess probabilistic protocols. Distributed Computing 1(1), 53–72 (1986)CrossRefzbMATHGoogle Scholar
  19. 19.
    Saha, R., Esparza, J., Jha, S.K., Mukund, M., Thiagarajan, P.S.: Distributed Markov chains. Technical report (2014),
  20. 20.
    Varacca, D., Völzer, H., Winskel, G.: Probabilistic event structures and domains. In: Gardner, P., Yoshida, N. (eds.) CONCUR 2004. LNCS, vol. 3170, pp. 481–496. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  21. 21.
    Wald, A.: Sequential tests of statistical hypotheses. Ann. Math. Statist., 117–186Google Scholar
  22. 22.
    Lorens, H., Younes, S.: Verification and Planning for Stochastic Processes with Asynchronous Events. PhD thesis, Carnegie Mellon University, Pittsburgh, PA, USA (2004)Google Scholar
  23. 23.
    Zielonka, W.: Notes on finite asynchronous automata. ITA 21(2), 99–135 (1987)zbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Ratul Saha
    • 1
  • Javier Esparza
    • 2
  • Sumit Kumar Jha
    • 3
  • Madhavan Mukund
    • 4
  • P. S. Thiagarajan
    • 1
  1. 1.National University of SingaporeSingapore
  2. 2.Technische Universität MünchenGermany
  3. 3.University of Central FloridaUSA
  4. 4.Chennai Mathematical InstituteIndia

Personalised recommendations