Introduction to Markov Chains
The topic of Markov processes is huge. A number of volumes can be, and in fact were, written on this topic. We have no intentions to be complete in this area. What is given in this chapter is the minimum required in order to follow what is presented afterwards. In particular, we will refer at times to this chapter when results we present here are called for. For more comprehensive coverage of the topic of Markov chains and stochastic matrices, see [9, 19, 41] or .
- 9.Billingsley, P. (1995). Probability and measure (3rd ed.). New York: Wiley.Google Scholar
- 16.Denardo, E. V. (1982). Dynamic programming: models and applications. Englewood Cliffs: Prentice-Hall.Google Scholar
- 19.Feller, W. (1968). An introduction to probability theory and its applications (3rd ed.). New York: Wiley.Google Scholar
- 31.Kemeny, J. K., & Snell, J. L. (1961). Finite Markov chains. New York: D. Van Nostrand.Google Scholar
- 41.Ross, S. M. (1996). Stochastic processes (2nd ed.). New York: Wiley.Google Scholar
- 42.Seneta, E. (2006). Non-negative matrices and Markov Chains: revised prinitng. New York: Springer.Google Scholar