Skip to main content
SpringerLink
Log in
Book cover

Encyclopedia of Operations Research and Management Science pp 477–480Cite as

Markov chain equations

  • Reference work entry
  • First Online:

  • 24 Accesses

INTRODUCTION

The use of mathematical models to analyze complex systems has a long history. With the advent of high powered workstations and cheap memory, these applications have greatly expanded. Frequently the characteristics of the system to be modeled are such that analytical solutions do not exist or are unknown so that systems engineers are obliged to turn to computing numerical solutions rather than analytical solutions. With Markov chain models, the numerical problem is not difficult to describe. The solution at any time t, π (t), is calculated from the Chapman-Kolmogorov differential equation,

(1)

Here Q is a square matrix of order n, the number of states in the Markov chain, and is called the infinitesimal generator. Its elements satisfy

The vector π (t) is also of length n and its ith component, πi(t), expresses the probability that the Markov chain is in state i at time t.

When the number of states in the Markov chain is small (e.g., less than two hundred), computing...

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   532.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Berman, A. and Plemmons, R.J. (1994). Nonnegative Matrices in the Mathematical Sciences, SIAM, Philadelphia.

    Google Scholar 

  2. Fernandes, P., Plateau, B., and Stewart, W.J. (1998). “Efficient Descriptor-Vector Multiplication in Stochastic Automata Networks,” Jl. Assoc. Comput. Mach., 45, 381–414.

    Google Scholar 

  3. Saad, Y. (1996). Iterative Solution of Sparse Linear Systems, PWS Publishing, New York.

    Google Scholar 

  4. Stewart, W.J. (1994). An Introduction to the Numerical Solution of Markov Chains, Princeton University Press, New Jersey.

    Google Scholar 

  5. Stewart, W.J. (1976). MARCA: Markov Chain Analyzer. IEEE Computer Repository, No. R76 232, 1976 (See the URL: http://www.csc.ncsu.edu/faculty/WStewart).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. North Carolina State University, Raleigh, USA

    William J. Stewart

Authors
  1. William J. Stewart
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Robert H. Smith School of Business, University of Maryland, College Part, Maryland, USA

    Saul I. Gass

  2. School of Information Technology & Engineering, George Mason University, Fairfax, Virginia, USA

    Carl M. Harris

Rights and permissions

Reprints and permissions

Copyright information

© 2001 Kluwer Academic Publishers

About this entry

Cite this entry

Stewart, W.J. (2001). Markov chain equations . In: Gass, S.I., Harris, C.M. (eds) Encyclopedia of Operations Research and Management Science. Springer, New York, NY. https://doi.org/10.1007/1-4020-0611-X_578

Download citation

  • DOI: https://doi.org/10.1007/1-4020-0611-X_578

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-0-7923-7827-3

  • Online ISBN: 978-1-4020-0611-1

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation