Abstract
The study of infinite-horizon nonstationary dynamic programs using the operator approach is continued. The point of view here differs slightly from that taken by others, in that Denardo's local income function is not used as a starting point. Infinite-horizon values are defined as limits of finite-horizon values, as the horizons get long. Two important conditions of an earlier paper are weakened, yet the optimality equations, the optimality criterion, and the existence of optimal “structured” strategies are still obtained.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Shapley, L.,Stochastic Games, Proceedings of the National Academy of Sciences, Vol. 39, pp. 1095–1100, 1953.
Bellman, R.,Dynamic Programming, Princeton University Press, Princeton, New Jersey, 1957.
Blackwell, D.,Discounted Dynamic Programming, Annals of Mathematical Statistics, Vol. 36, pp. 226–235, 1965.
Blackwell, D.,Positive Dynamic Programming, Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, University of California Press, Berkeley, California, 1967.
Veinott, A., Jr.,Discrete Dynamic Programming with Sensitive Discount Optimality Criteria, Annals of Mathematical Statistics, Vol. 40, pp. 1635–1660, 1969.
Hinderer, K.,Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter, Springer-Verlag, New York, New York, 1970.
Blackwell, D., Freedman, D., andOrkin, M.,The Optimal Reward Operator in Dynamic Programming, Annals of Probability, Vol. 2, pp. 926–941, 1974.
Freedman, D.,The Optimal Reward Operator in Special Classes of Dynamic Programming Problems, Annals of Probability, Vol. 2, pp. 942–949, 1974.
Schäl, M.,Conditions for Optimality in Dynamic Programming and for the Limit of n-Stage Optimal Policies to Be Optimal, Zeitschrift Wahrscheinlichkeitstheorie verwandte Gebiete, Vol. 32, pp. 179–196, 1975.
Federgruen, A., Schweitzer, P., andTijms, H.,Contraction Mappings Underlying Undiscounted Markov Decision Problems, Journal of Mathematical Analysis and Applications, Vol. 65, pp. 711–730, 1978.
Shreve, S., andBertsekas, D.,Universally Measurable Policies in Dynamic Programming, Mathematics of Operations Research, Vol. 4, pp. 15–30, 1979.
Klein Haneveld, W.,On the Behavior of the Optimal Value Operator of Dynamic Programming, Mathematics of Operations Research, Vol. 5, pp. 308–320, 1980.
Denardo, E.,Contraction Mappings in the Theory Underlying Dynamic Programming, SIAM Review, Vol. 9, pp. 165–177, 1967.
Fox, B.,Finite-State Approximations to Denumerable-State Dynamic Programs, Journal of Mathematical Analysis and Applications, Vol. 34, pp. 665–670, 1971.
Porteus, E.,Some Bounds for Discounted Sequential Decision Processes, Management Science, Vol. 18, pp. 7–11, 1971.
Porteus, E.,On the Optimality of Structured Policies in Countable Stage Decision Processes, Management Science, Vol. 22, pp. 148–157, 1975.
Kreps, D., andPorteus, E.,On the Optimality of Structured Policies in Countable Stage Decision Processes, II: Positive and Negative Problems, SIAM Journal on Applied Mathematics, Vol. 32, pp. 457–466, 1977.
Bertsekas, D.,Monotone Mappings with Application in Dynamic Programing, SIAM Journal on Control and Optimization, Vol. 15, pp. 438–464, 1977.
Whitt, W.,Approximations of Dynamic Programs, I, Mathematics of Operations Research, Vol. 3, pp. 231–243, 1978.
Whitt, W.,Approximations of Dynamic Programs, II, Mathematics of Operations Research, Vol. 4, pp. 179–185, 1979.
Kreps, D., andPorteus, E.,Dynamic Choice Theory and Dynamic Programming, Econometrica, Vol. 47, pp. 91–100, 1979.
Schäl, M.,An Operator-Theoretical Treatment of Negative Dynamic Programming, Report, University of Bonn, Bonn, West Germany, 1978.
Wessels, J.,Markov Programming by Successive Approximations with Respect to Weighted Supremum Norms, Journal of Mathematical Analysis and Applications, Vol. 58, pp. 326–335, 1977.
Van Nunen, J., andWessels, J.,Markov Decision Processes with Unbounded Rewards, Markov Decision Theory, Edited by H. Tijms and J. Wessels, Mathematisch Centrum, Amsterdam, Holland, 1977.
Porteus, E.,Overview of Iterative Methods for Discounted Finite Markov and Semi-Markov Decision Chains, Recent Developments in Markov Decision Processes, Edited by R. Hartley, L. Thomas, and D. White, Academic Press, London, England, 1980.
Porteus, E.,Bounds and Transformations for Finite Markov Decision Processes, Operations Research, Vol. 23, pp. 761–784, 1975.
Porteus, E.,On Optimal Dividend, Reinvestment, and Liquidation Policies for the Firm, Operations Research, Vol. 25, pp. 818–834, 1977.
Author information
Authors and Affiliations
Additional information
Communicated by G. Nemhauser
Rights and permissions
About this article
Cite this article
Porteus, E. Conditions for characterizing the structure of optimal strategies in infinite-horizon dynamic programs. J Optim Theory Appl 36, 419–432 (1982). https://doi.org/10.1007/BF00934355
Issue Date:
DOI: https://doi.org/10.1007/BF00934355