Advertisement

The Ergodic Cost Problem: Formulation and Algorithms

  • Harold J. Kushner
  • Paul Dupuis
Part of the Stochastic Modelling and Applied Probability book series (SMAP, volume 24)

Abstract

In this chapter, we reformulate some of the concepts in the last chapter so that they can be used on the ergodic cost problem. Before doing that it is useful to discuss the appropriate dynamic programming equations and some additional background material. The natural state spaces for control problems that are of interest over a long time interval are often unbounded, and they must be truncated for numerical purposes. One standard way of doing this involves a reflecting boundary, and this is the case dealt with in this chapter. Thus, there are no absorbing states. The basic process is the controlled diffusion (5.3.1) or jump diffusion (5.6.1). The approximating Markov chains {ξ n h , n <  } will be locally consistent with these processes. As in the previous chapters, S h denotes the state space, and ∂ G h +  ⊂ S h the set of reflecting states. Recall that the reflection is instantaneous and that we use Δt h (x,α) = 0 for x ∈ ∂ G h + .

Keywords

Feedback Control Invariant Measure Markov Chain Model Policy Space Jump Diffusion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media New York 2001

Authors and Affiliations

  • Harold J. Kushner
    • 1
  • Paul Dupuis
    • 1
  1. 1.Division of Applied MathematicsBrown UniversityProvidenceUSA

Personalised recommendations