Abstract
We introduce a revised simplex algorithm for solving a typical type of dynamic programming equation arising from a class of finite Markov decision processes. The algorithm also applies to several types of optimal control problems with diffusion models after discretization. It is based on the regular simplex algorithm, the duality concept in linear programming, and certain special features of the dynamic programming equation itself. Convergence is established for the new algorithm. The algorithm has favorable potential applicability when the number of actions is very large or even infinite.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bellman, R.,Dynamic Programming, Princeton University Press, Princeton, New Jersey, 1957.
Bertsekas, D., andShreve, S. E.,Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York, New York, 1978.
White, D. J.,Finite Dynamic Programming: An Approach to Finite Markov Decision Processes, Wiley-Interscience, New York, New York, 1978.
Derman, C.,Finite State Markovian Decision Processes, Academic Press, New York, New York, 1970.
Kallenberg, L. C. M.,Linear Programming and Finite Markovian Control Problems, Mathematisch Centrum, Amsterdam, Holland, 1983.
Sun, M.,Numerical Solutions of Singular Stochastic Control Problems in Bounded Intervals, Lectures in Applied Mathematics, American Mathematical Society, Providence, Rhode Island, Vol. 26, pp. 619–643, 1990.
Sun, M.,Singular Stochastic Control Problems Solved by a Sparse Simplex Method, IMA Journal of Mathematical Control and Information, Vol. 6, pp. 27–38, 1989.
Kushner, H. J.,Probability Limit Theorems and the Convergence of Finite Difference Approximations of Partial Differential Equations, Journal of Mathematical Analysis and Applications, Vol. 32, pp. 77–103, 1970.
Bensoussan, A.,Stochastic Control by Functional Analysis Methods, North-Holland, Amsterdam, Holland, 1982.
Bellman, R.,Adaptive Control Processes: A Guided Tour, Princeton University Press, Princeton, New Jersey, 1961.
Luenberger, D. G.,Linear and Nonlinear Programming, 2nd Edition, Addison-Wesley, Reading, Massachusetts, 1984.
Author information
Authors and Affiliations
Additional information
Communicated by D. G. Luenberger
Rights and permissions
About this article
Cite this article
Sun, M. Revised simplex algorithm for finite Markov decision processes. J Optim Theory Appl 79, 405–413 (1993). https://doi.org/10.1007/BF00940589
Issue Date:
DOI: https://doi.org/10.1007/BF00940589