Starting with this chapter we introduce the systematic use of the first step analysis technique, in a general framework that covers the examples of random walks already treated in Chapters  3 and  4. The main applications of first step analysis are the computation of hitting probabilities, mean hitting and absorption times, mean first return times, and average number of returns to a given state.


Markov Chain Random Walk Transition Matrix Gambling Problem Monthly Income 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 11.
    Karlin, S., Taylor, H.M.: A Second Course in Stochastic Processes. Academic Press [Harcourt Brace Jovanovich Publishers], New York (1981) Google Scholar

Copyright information

© Springer Science+Business Media Singapore 2013

Authors and Affiliations

  • Nicolas Privault
    • 1
  1. 1.School of Physical and Mathematical SciencesNanyang Technological UniversitySingaporeSingapore

Personalised recommendations