Some problems of ergodic control and adaptive control are formulated and solved for stochastic differential delay systems. The existence and the uniqueness of invariant measures that are solutions of the stochastic functional differential equations for these systems are verified. For an ergodic cost criterion, almost optimal controls are constructed. For an unknown system, the invariant measures and the optimal ergodic costs are shown to be continuous functions of the unknown parameters. Almost self-optimizing adaptive controls are feasibly constructed by an approximate certainty equivalence principle.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Bensoussan A.,Perturbation Methods in Optimal Control,J. Wiley, New York, New York, 1988.
Doob, J. L.,Stochastic Processes, J. Wiley, New York, New York, 1953.
Khasminskii, R. Z.,Stochastic Stability of Differential Equations. Sijthoff and Noordhoff, Alphen aan den Rijn, Holland, 1980 (Translation from Russian).
Kushner, H.,Approximation Methods for the Minimum Average Cost per Unit Time Problem with a Diffusion Model, Approximate Solutions of Random Equations. Edited by A. T. Barucha-Reid, North-Holland, Amsterdam, Holland, pp. 10–126, 1979.
Kushner, H. J.,Probability Methods for Approximation in Stochastic Control and for Elliptic Equations, Academic Press, New York, New York, 1977.
Duncan, T. E., Pasik-Duncan, B., andStettner, L.,Almost Self-Optimizing Strategies for Adaptive Control of Diffusion Processes, Journal of Optimization Theory and Applications, Vol. 81, pp. 479–507, 1994.
Stettner, L.,On Nearly Self-Optimizing Strategies for a Discrete-Time Uniformly Ergodic Adaptive Model, Journal of Applied Mathematics and Optimization, Vol. 27, pp. 161–177, 1993.
Kartashov, N. W.,Criteria for Uniform Ergodicity and Strong Stability of Markov Chains in General State Space, Probability Theory and Mathematical Statistics, Vol. 30, pp. 65–81, 1984.
Kumar, P. R., andBecker, A.,A New Family of Optimal Adaptive Controllers for Markov Chains, IEEE Transactions on Automatic Control, Vol. 27, pp. 137–146, 1982.
Borkar, V. S.,Self-Tuning Control of Diffusions without the Identifiability Condition, Journal of Optimization Theory and Applications, Vol. 68, pp. 117–138, 1991.
Borkar, V., andBagchi, A.,Parameter Estimation in Continuous-Time Stochastic Processes, Stochastics, Vol. 8, pp. 193–212, 1982.
This research was partially supported by NSF Grants ECS-91-02714 and ECS91-13029.
Communicated by R. Rishel
About this article
Cite this article
Duncan, T.E., Pasik-Duncan, B. & Stettner, L. On the ergodic and the adaptive control of stochastic differential delay systems. J Optim Theory Appl 81, 509–531 (1994). https://doi.org/10.1007/BF02193098
- Stochastic adaptive control
- ergodic control
- stochastic differential-delay systems
- almost self-optimizing adaptive controls