Abstract.
This paper deals with a new optimality criterion consisting of the usual three average criteria and the canonical triplet (totally so-called strong average-canonical optimality criterion) and introduces the concept of a strong average-canonical policy for nonstationary Markov decision processes, which is an extension of the canonical policies of Herna´ndez-Lerma and Lasserre [16] (pages: 77) for the stationary Markov controlled processes. For the case of possibly non-uniformly bounded rewards and denumerable state space, we first construct, under some conditions, a solution to the optimality equations (OEs), and then prove that the Markov policies obtained from the OEs are not only optimal for the three average criteria but also optimal for all finite horizon criteria with a sequence of additional functions as their terminal rewards (i.e. strong average-canonical optimal). Also, some properties of optimal policies and optimal average value convergence are discussed. Moreover, the error bound in average reward between a rolling horizon policy and a strong average-canonical optimal policy is provided, and then a rolling horizon algorithm for computing strong average ε(>0)-optimal Markov policies is given.
Similar content being viewed by others
Author information
Authors and Affiliations
Additional information
Manuscript received: December 1999/Final version received: May 2000
Rights and permissions
About this article
Cite this article
Guo, X., Shi, P. & Zhu, W. A new strong optimality criterion for nonstationary Markov decision processes. Mathematical Methods of OR 52, 287–306 (2000). https://doi.org/10.1007/s001860000076
Issue Date:
DOI: https://doi.org/10.1007/s001860000076