First Step Analysis
Starting with this chapter we introduce the systematic use of the first step analysis technique, in a general framework that covers the examples of random walks already treated in Chapters 3 and 4. The main applications of first step analysis are the computation of hitting probabilities, mean hitting and absorption times, mean first return times, and average number of returns to a given state.
KeywordsMarkov Chain Random Walk Transition Matrix Gambling Problem Monthly Income
- 11.Karlin, S., Taylor, H.M.: A Second Course in Stochastic Processes. Academic Press [Harcourt Brace Jovanovich Publishers], New York (1981) Google Scholar