Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

A new parallel algorithm for solving large-scale Markov chains

  • 237 Accesses

  • 1 Citations


In this paper, we propose a new parallel sparse iterative method (PPSIA) for computing the stationary distribution of large-scale Markov chains. The PPSIA method is based on Markov chain state isolation and aggregation techniques. The parallel method conserves as much as possible the benefits of aggregation, and Gauss–Seidel effects contained in the sequential algorithm (SIA) using a pipelined technique. Both SIA and PPSIA exploit sparse matrix representation in order to solve large-scale Markov chains. Some Markov chains have been tested to compare the performance of SIA, PPSIA algorithms with other techniques such as the power method, and the generalized minimal residual GMRES method. In all the tested models, PPSIA outperforms the other methods and shows a super-linear speed-up.

This is a preview of subscription content, log in to check access.

Fig. 1
Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Algorithm 5


  1. 1.

    Gelenbe E, Labetoulle J, Marie R, Stewart WJ (1980) Reseaux de files d’attentes—modelisation et traitement numerique. In: Ed des hommes et technique de l’AFCET

  2. 2.

    Varga RS (1963) Matrix iteratives analysis. Printice Hall, Englewood Cliffs

  3. 3.

    Saad Y (1981) Krylov Subspace Methods for Solving Unsymetric Linear Systems. Math Comput 37:105–126

  4. 4.

    Philippe B, Saad Y, Stewart WJ (1992) Numerical methods in Markov chain modeling. Oper Res 40:1156–1179

  5. 5.

    Couturier R, Jezequel F (2010) Solving large-scale linear systems in grid environment using Java. In: Proceeding of IPDPS’10, Atlanta, USA

  6. 6.

    Touzene A (2012) A new parallel block aggregated algorithm for solving Markov chains. J Supercomput 62:573–587

  7. 7.

    Benzi M, Tuma M (2002) A parallel solver for large-scale Markov chains. Appl Numer Math 41:135–153

  8. 8.

    Benzi M, Ucar B (2007) Block triangular preconditioners for M-matrices and Markov chains. Electron Trans Numer Anal 26:209–227

  9. 9.

    Touzene A (1995) A new iterative method for solving large-scale Markov chains. In: Lecture notes in computer science, vol 977. Springer, Berlin, pp 180–193

  10. 10.

    Cao W-L, Stewart WJ (1985) Iterative aggregation/disaggregation techniques for nearly uncoupled Markov chains. J ACM 32(3):702–719

  11. 11.

    Koury R, McAllister DF, Stewart WJ (1984) Iterative methods for computing stationary distributions of nearly completely decomposable Markov chains. SIAM J Alg Discrete Math 5(2):164–186

  12. 12.

    Schweitzer PJ (1983) Aggregation methods for large Markov chains. In: International workshop on applied mathematics and performance reliability models of computer communication systems. University of Pisa, Pisa, pp 225–234

  13. 13.

    Stewart WJ, Wu W (1992) Numerical experiments with iteration and aggregation for Markov chains. ORSA J Comput 4:336–350

  14. 14.

    Stewart WJ. http://www4.ncsu.edu/billy/MARCA-Models/MARCA-Models.html

  15. 15.

    Dayar T, Stewart WJ (2000) Comparison of partitioning techniques for two-level iterative methods on large, sparse Markov chains. SIAM J Sci Comput 21:1691–1705

Download references


We thank Sultan Qaboos University for providing the High Performance Computer (HPC). We also thank Professor William Stewart for making the MARCA software available.

Author information

Correspondence to Abderezak Touzene.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Touzene, A. A new parallel algorithm for solving large-scale Markov chains. J Supercomput 67, 239–253 (2014). https://doi.org/10.1007/s11227-013-0997-5

Download citation


  • Performance evaluation
  • Markov chains
  • Iterative methods
  • Aggregation techniques