Discrete-Time Markov Chains

Part of the Springer Texts in Statistics book series (STS, volume 0)


Applied probability thrives on models. Markov chains are one of the richest sources of good models for capturing dynamical behavior with a large stochastic component [23, 24, 59, 80, 106, 107, 118]. In this chapter we give a few examples and a quick theoretical overview of discrete-time Markov chains. The highlight of our theoretical development, Proposition 7.4.1, relies on a coupling argument. Because coupling is one of the most powerful and intuitively appealing tools available to probabilists, we examine a few of its general applications as well. We also stress reversible Markov chains. Reversibility permits explicit construction of the long-run or equilibrium distribution of a chain when such a distribution exists. Chapter 8 will cover continuous-time Markov chains.


Markov Chain Simulated Annealing Equilibrium Distribution Detailed Balance Transition Probability Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Departments of Biomathematics, Human Genetics, and StatisticsUniversity of California, Los AngelesLos AngelesUSA

Personalised recommendations