Abstract
Some problems of ergodic control and adaptive control are formulated and solved for stochastic differential delay systems. The existence and the uniqueness of invariant measures that are solutions of the stochastic functional differential equations for these systems are verified. For an ergodic cost criterion, almost optimal controls are constructed. For an unknown system, the invariant measures and the optimal ergodic costs are shown to be continuous functions of the unknown parameters. Almost self-optimizing adaptive controls are feasibly constructed by an approximate certainty equivalence principle.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bensoussan A.,Perturbation Methods in Optimal Control,J. Wiley, New York, New York, 1988.
Doob, J. L.,Stochastic Processes, J. Wiley, New York, New York, 1953.
Khasminskii, R. Z.,Stochastic Stability of Differential Equations. Sijthoff and Noordhoff, Alphen aan den Rijn, Holland, 1980 (Translation from Russian).
Kushner, H.,Approximation Methods for the Minimum Average Cost per Unit Time Problem with a Diffusion Model, Approximate Solutions of Random Equations. Edited by A. T. Barucha-Reid, North-Holland, Amsterdam, Holland, pp. 10–126, 1979.
Kushner, H. J.,Probability Methods for Approximation in Stochastic Control and for Elliptic Equations, Academic Press, New York, New York, 1977.
Duncan, T. E., Pasik-Duncan, B., andStettner, L.,Almost Self-Optimizing Strategies for Adaptive Control of Diffusion Processes, Journal of Optimization Theory and Applications, Vol. 81, pp. 479–507, 1994.
Stettner, L.,On Nearly Self-Optimizing Strategies for a Discrete-Time Uniformly Ergodic Adaptive Model, Journal of Applied Mathematics and Optimization, Vol. 27, pp. 161–177, 1993.
Kartashov, N. W.,Criteria for Uniform Ergodicity and Strong Stability of Markov Chains in General State Space, Probability Theory and Mathematical Statistics, Vol. 30, pp. 65–81, 1984.
Kumar, P. R., andBecker, A.,A New Family of Optimal Adaptive Controllers for Markov Chains, IEEE Transactions on Automatic Control, Vol. 27, pp. 137–146, 1982.
Borkar, V. S.,Self-Tuning Control of Diffusions without the Identifiability Condition, Journal of Optimization Theory and Applications, Vol. 68, pp. 117–138, 1991.
Borkar, V., andBagchi, A.,Parameter Estimation in Continuous-Time Stochastic Processes, Stochastics, Vol. 8, pp. 193–212, 1982.
Author information
Authors and Affiliations
Additional information
Communicated by R. Rishel
This research was partially supported by NSF Grants ECS-91-02714 and ECS91-13029.
Rights and permissions
About this article
Cite this article
Duncan, T.E., Pasik-Duncan, B. & Stettner, L. On the ergodic and the adaptive control of stochastic differential delay systems. J Optim Theory Appl 81, 509–531 (1994). https://doi.org/10.1007/BF02193098
Issue Date:
DOI: https://doi.org/10.1007/BF02193098