Encyclopedia of Operations Research and Management Science

2001 Edition
| Editors: Saul I. Gass, Carl M. Harris

Markov chain equations

Reference work entry
DOI: https://doi.org/10.1007/1-4020-0611-X_578


The use of mathematical models to analyze complex systems has a long history. With the advent of high powered workstations and cheap memory, these applications have greatly expanded. Frequently the characteristics of the system to be modeled are such that analytical solutions do not exist or are unknown so that systems engineers are obliged to turn to computing numerical solutions rather than analytical solutions. With Markov chain models, the numerical problem is not difficult to describe. The solution at any time t, π (t), is calculated from the Chapman-Kolmogorov differential equation,
This is a preview of subscription content, log in to check access.


  1. [1]
    Berman, A. and Plemmons, R.J. (1994). Nonnegative Matrices in the Mathematical Sciences, SIAM, Philadelphia.Google Scholar
  2. [2]
    Fernandes, P., Plateau, B., and Stewart, W.J. (1998). “Efficient Descriptor-Vector Multiplication in Stochastic Automata Networks,” Jl. Assoc. Comput. Mach., 45, 381–414.Google Scholar
  3. [3]
    Saad, Y. (1996). Iterative Solution of Sparse Linear Systems, PWS Publishing, New York.Google Scholar
  4. [4]
    Stewart, W.J. (1994). An Introduction to the Numerical Solution of Markov Chains, Princeton University Press, New Jersey.Google Scholar
  5. [5]
    Stewart, W.J. (1976). MARCA: Markov Chain Analyzer. IEEE Computer Repository, No. R76 232, 1976 (See the URL: http://www.csc.ncsu.edu/faculty/WStewart).Google Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  1. 1.North Carolina State UniversityRaleighUSA