Abstract
This chapter was a compact introduction to both discrete-time and continuous-time Markov chains. The important concepts including the definition of discrete-time and continuous-time Markov chains, Chapman-Kolmogorov equations, reachability, communication, communication classes, recurrent and transient states, period of a state for a discrete-time Markov chain, the limiting probability of a state for a discrete-time and continuous-time Markov chain, and ergodic Markov chain were explained. Several examples and problems have been solved for the discrete-time Markov chains, and where relevant, state transition diagrams and tables have been used to facilitate the comprehension of the solutions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Bas, E. (2019). An Introduction to Markov Chains. In: Basics of Probability and Stochastic Processes. Springer, Cham. https://doi.org/10.1007/978-3-030-32323-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-32323-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32322-6
Online ISBN: 978-3-030-32323-3
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)