Improved Reachability Analysis in DTMC via Divide and Conquer

  • Songzheng Song
  • Lin Gui
  • Jun Sun
  • Yang Liu
  • Jin Song Dong
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7940)


Discrete Time Markov Chains (DTMCs) are widely used to model probabilistic systems in many domains, such as biology, network and communication protocols. There are two main approaches for probability reachability analysis of DTMCs, i.e., solving linear equations or using value iteration. However, both approaches have drawbacks. On one hand, solving linear equations can generate accurate results, but it can be only applied to relatively small models. On the other hand, value iteration is more scalable, but often suffers from slow convergence. Furthermore, it is unclear how to parallelize (i.e., taking advantage of multi-cores or distributed computers) these two approaches. In this work, we propose a divide-and-conquer approach to eliminate loops in DTMC and hereby speed up probabilistic reachability analysis. A DTMC is separated into several partitions according to our proposed cutting criteria. Each partition is then solved by Gauss-Jordan elimination effectively and the state space is reduced afterwards. This divide and conquer algorithm will continue until there is no loop existing in the system. Experiments are conducted to demonstrate that our approach can generate accurate results, avoid the slow convergence problems and handle larger models.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ábrahám, E., Jansen, N., Wimmer, R., Katoen, J.-P., Becker, B.: DTMC Model Checking by SCC Reduction. In: QEST, pp. 37–46 (2010)Google Scholar
  2. 2.
    Althoen, S.C., McLaughlin, R.: Gauss - Jordan reduction: a brief history. The American Mathematical Monthly 94(2), 130–142 (1987)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Andrés, M.E., D’Argenio, P., van Rossum, P.: Significant Diagnostic Counterexamples in Probabilistic Model Checking. In: Chockler, H., Hu, A.J. (eds.) HVC 2008. LNCS, vol. 5394, pp. 129–148. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  4. 4.
    Aspnes, J., Herlihy, M.: Fast Randomized Consensus Using Shared Memory. Journal of Algorithms 15(1), 441–460 (1990)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Baier, C., Katoen, J.: Principles of Model Checking. The MIT Press (2008)Google Scholar
  6. 6.
    Ciesinski, F., Baier, C., Größer, M., Klein, J.: Reduction Techniques for Model Checking Markov Decision Processes. In: QEST, pp. 45–54 (2008)Google Scholar
  7. 7.
    Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. The MIT Press (1999)Google Scholar
  8. 8.
    Grenager, T., Powers, R., Shoham, Y.: Dispersion Games: General Definitions and Some Specific Learning Results. In: AAAI, pp. 398–403 (2002)Google Scholar
  9. 9.
    Itai, A., Rodeh, M.: Symmetry Breaking in Distributed Networks. Information and Computation 88, 150–158 (1981)MathSciNetGoogle Scholar
  10. 10.
    Katoen, J.-P., Khattri, M., Zapreev, I.S.: A Markov Reward Model Checker. In: QEST, pp. 243–244 (2005)Google Scholar
  11. 11.
    Katoen, J.-P., Zapreev, I.S., Hahn, E.M., Hermanns, H., Jansen, D.N.: The Ins and Outs of The Probabilistic Model Checker MRMC. In: QEST, pp. 167–176 (2009)Google Scholar
  12. 12.
    Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: Verification of Probabilistic Real-Time Systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  13. 13.
    Kwiatkowska, M.Z., Parker, D., Qu, H.: Incremental Quantitative Verification for Markov Decision Processes. In: DSN, pp. 359–370 (2011)Google Scholar
  14. 14.
    Stoer, J., Bulirsch, R.: Introduction to Numerical Analysis. Springer, Berlin (2002)zbMATHGoogle Scholar
  15. 15.
    Sun, J., Liu, Y., Dong, J.S., Pang, J.: PAT: Towards Flexible Verification under Fairness. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, pp. 709–714. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  16. 16.
    Sun, J., Song, S., Liu, Y.: Model Checking Hierarchical Probabilistic Systems. In: Dong, J.S., Zhu, H. (eds.) ICFEM 2010. LNCS, vol. 6447, pp. 388–403. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  17. 17.
    Tarjan, R.E.: Depth-First Search and Linear Graph Algorithms. SIAM J. Comput. 1(2), 146–160 (1972)MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Younes, H.L.S., Clarke, E.M., Zuliani, P.: Statistical Verification of Probabilistic Properties with Unbounded Until. In: Davies, J. (ed.) SBMF 2010. LNCS, vol. 6527, pp. 144–160. Springer, Heidelberg (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Songzheng Song
    • 1
  • Lin Gui
    • 1
  • Jun Sun
    • 2
  • Yang Liu
    • 3
  • Jin Song Dong
    • 1
  1. 1.National University of SingaporeSingapore
  2. 2.Singapore University of Technology and DesignSingapore
  3. 3.Nanyang Technological UniversitySingapore

Personalised recommendations