Abstract
Finite Markov chains are processes with finitely many (typically only a few) states on a nominal scale (with arbitrary labels).Time runs in discrete steps, such as day 1, day 2, …, and only the most recent state of the process affects its future development (the Markovian property).Our first objective is to compute the probability of being in a certain state after a specific number of steps.This is followed by investigating the process’s long-run behavior.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
M. S.Bartlett.An Introduction to Stochastic Processes, with Special Reference to Methods and Applications.Cambridge University Press, Cambridge/New York, 1980.
U. Narayan Bhat and G. K.Miller.Elements of Applied Stochastic Processes.Wiley-Interscience, Hoboken, N.J.2002.
R. Bronson.Schaum’s Outline of Theory and Problems of Matrix Operations.McGraw-Hill, New York, 1989.
W. Feller.An Introduction to Probability Theory and Its Applications.Wiley, New York, 1968.
S. Goldberg.Introduction to Difference Equations.Dover, New York, 1986.
S. Karlin.A First Course in Stochastic Processes.Academic, New York, 1975.
S. Karlin and H. M.Taylor.A Second Course in Stochastic Processes.Academic, New York, 1981.
J. G.Kemeny and J. L.Snell.Finite Markov Chains.Springer, New York, 1976.
J. Medhi.Stochastic Processes.Wiley, New York, 1994.
J. Medhi.Stochastic Models in Queueing Theory.Academic, Amsterdam, 2003.
S. Ross.Stochastic Processes.Wiley, New York, 1996.
A. Stuart.Kendall’s Advanced Theory of Statistics.Wiley, Chichester, 1994.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Vrbik, J., Vrbik, P. (2013). Finite Markov Chains. In: Informal Introduction to Stochastic Processes with Maple. Universitext. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4057-4_2
Download citation
DOI: https://doi.org/10.1007/978-1-4614-4057-4_2
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-4056-7
Online ISBN: 978-1-4614-4057-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)