Continuous-Time Markov Chains
This chapter introduces the subject of continuous-time Markov chains [23, 52, 59, 80, 106, 107, 118, 152]. In practice, continuous-time chains are more useful than discrete-time chains. For one thing, continuous-time chains permit variation in the waiting times for transitions between neighboring states. For another, they avoid the annoyances of periodic behavior. Balanced against these advantages is the disadvantage of a more complex theory involving linear differential equations. The primary distinction between the two types of chains is the substitution of transition intensities for transition probabilities. Once one grasps this difference, it is straightforward to formulate relevant continuous-time models. Implementing such models numerically and understanding them theoretically then require the matrix exponential function. Kendall’s birth-death-immigration process, treated at the end of the chapter, involves an infinite number of states and transition intensities that depend on time.
KeywordsMarkov Chain Poisson Process Equilibrium Distribution Transition Intensity Markov Chain Model
Unable to display preview. Download preview PDF.