Discrete-Time Markov Models

Abstract

Sequences of independent and identically distributed random variables are stochastic processes, but they are not always interesting as stochastic models because they behave more or less in the same way. In order to introduce more variability, one can allow for some dependence on the past, in the manner of deterministic recurrence equations. Discrete-time homogeneous Markov chains possess the required feature, since they can always be represented—at least distributionwise—by a stochastic recurrence equation X _n+1 = f(X _n , Z _n+1),where {Z _n } _n≥1 is an i.i.d sequence, independent of the initial state X ₀.