Stochastic Processes (iii): Markov Chains

Abstract

Introductory aspects of the theory of Markov chains were discussed in Chapter 4. In the present chapter further details of the theory of Markov chains will be discussed, first for Markov chains with no absorbing states and second for Markov chains with absorbing states. Our analysis of finite Markov chain theory is often oversimplified, since an examination of some of the subtleties involved in the full theory is not appropriate for bioinformatics. A recent and more complete exposition of the theory can be found in Norris (1997).