Advertisement

Computational Methods for Controlled Markov Chains

  • Harold J. Kushner
  • Paul Dupuis
Chapter
Part of the Stochastic Modelling and Applied Probability book series (SMAP, volume 24)

Abstract

The chapter presents many of the basic ideas which are in current use for the solution of the dynamic programming equations for the optimal control and value function for the approximating Markov chain models. We concentrate on methods for problems which are of interest over a potentially unbounded time interval. Numerical methods for the ergodic problem will be discussed in Chapter 7, and are simple modifications of the ideas of this chapter. Some approaches to the numerical problem for the finite time problem will be discussed in Chapter 12.

Keywords

Coarse Grid Domain Decomposition Multigrid Method Policy Space Basic Feasible Solution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media New York 2001

Authors and Affiliations

  • Harold J. Kushner
    • 1
  • Paul Dupuis
    • 1
  1. 1.Division of Applied MathematicsBrown UniversityProvidenceUSA

Personalised recommendations