Abstract
The chapter presents many of the basic ideas which are in current use for the solution of the dynamic programming equations for the optimal control and value function for the approximating Markov chain models. We concentrate on methods for problems which are of interest over a potentially unbounded time interval. Numerical methods for the ergodic problem will be discussed in Chapter 7, and are simple modifications of the ideas of this chapter. Some approaches to the numerical problem for the finite time problem will be discussed in Chapter 12.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer Science+Business Media New York
About this chapter
Cite this chapter
Kushner, H.J., Dupuis, P. (2001). Computational Methods for Controlled Markov Chains. In: Numerical Methods for Stochastic Control Problems in Continuous Time. Stochastic Modelling and Applied Probability, vol 24. Springer, New York, NY. https://doi.org/10.1007/978-1-4613-0007-6_7
Download citation
DOI: https://doi.org/10.1007/978-1-4613-0007-6_7
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4612-6531-3
Online ISBN: 978-1-4613-0007-6
eBook Packages: Springer Book Archive