Abstract
We control a discrete-time uniformly ergodic system, which depends on an unknown parameter α0 εA, a compact set. Our purpose is to minimize the long-run average-cost functional. We estimate the unknown parameter using the biased maximum likelihood estimator and apply the control which is almost optimal for the value of estimation. This way we construct strategies such that the value of the cost functional can be arbitrarily close to the optimal value obtained for α0.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Borkar V (1990) The Kumar-Becker-Lin scheme revisited. J Optim Theory Appl 66:289–309
Borkar V (to appear) Self-tuning control of diffusions without identifiability condition. J Optim Theory Appl
Borkar V, Bagchi A (1982) Parameter estimation in continuous time stochastic processes. Stochastics 8:193–212
Doob JL (1953) Stochastic Processes. Wiley, New York
Gubenko LG, Shtatland ES (1972) On discrete time Markov decision processes. Probab Theory Math Statist 7:51–64
Hernandez-Lerma O (1989) Adaptive Markov Control Processes. Springer-Verlag, Berlin
Kartashow NW (1984) Criteria for uniform ergodicity and strong stability of Markov chains in general state space. Probab Theory Math Statist 30:65–81
Kumar PR, Becker A (1982) A new family of optimal adaptive controllers for Markov chains. IEEE Trans Automat Control 27:137–146
Ueno T (1957) Some limit theorems for temporally discrete Markov processes. J Fac Sci Univ Tokyo 7:449–462
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Stettner, Ł. On nearly self-optimizing strategies for a discrete-time uniformly ergodic adaptive model. Appl Math Optim 27, 161–177 (1993). https://doi.org/10.1007/BF01195980
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF01195980