Construction of the Approximating Markov Chain

Abstract

In this chapter we develop some canonical methods for obtaining approximating Markov chains which are locally consistent with the controlled diffusion in the sense used in (4.1.3). We also deal with the controlled jump diffusion and reflected processes in Sections 5.6 and 5.7. The chapter outlines two classes of basic methods and a number of variations of each of them. One purpose is to describe methods which are readily programmable. But we also wish to show the versatility and intuitive nature of the general approach. There are many variations of the methods discussed. Once the general procedures are clear, the reader can adapt them to particular problems which might not be directly covered by the discussion. The development does not directly cover processes on manifolds such as the surface of a sphere or torus, but the various possibilities should be apparent.