Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

On the ergodic and the adaptive control of stochastic differential delay systems


Some problems of ergodic control and adaptive control are formulated and solved for stochastic differential delay systems. The existence and the uniqueness of invariant measures that are solutions of the stochastic functional differential equations for these systems are verified. For an ergodic cost criterion, almost optimal controls are constructed. For an unknown system, the invariant measures and the optimal ergodic costs are shown to be continuous functions of the unknown parameters. Almost self-optimizing adaptive controls are feasibly constructed by an approximate certainty equivalence principle.

This is a preview of subscription content, log in to check access.


  1. 1.

    Bensoussan A.,Perturbation Methods in Optimal Control,J. Wiley, New York, New York, 1988.

  2. 2.

    Doob, J. L.,Stochastic Processes, J. Wiley, New York, New York, 1953.

  3. 3.

    Khasminskii, R. Z.,Stochastic Stability of Differential Equations. Sijthoff and Noordhoff, Alphen aan den Rijn, Holland, 1980 (Translation from Russian).

  4. 4.

    Kushner, H.,Approximation Methods for the Minimum Average Cost per Unit Time Problem with a Diffusion Model, Approximate Solutions of Random Equations. Edited by A. T. Barucha-Reid, North-Holland, Amsterdam, Holland, pp. 10–126, 1979.

  5. 5.

    Kushner, H. J.,Probability Methods for Approximation in Stochastic Control and for Elliptic Equations, Academic Press, New York, New York, 1977.

  6. 6.

    Duncan, T. E., Pasik-Duncan, B., andStettner, L.,Almost Self-Optimizing Strategies for Adaptive Control of Diffusion Processes, Journal of Optimization Theory and Applications, Vol. 81, pp. 479–507, 1994.

  7. 7.

    Stettner, L.,On Nearly Self-Optimizing Strategies for a Discrete-Time Uniformly Ergodic Adaptive Model, Journal of Applied Mathematics and Optimization, Vol. 27, pp. 161–177, 1993.

  8. 8.

    Kartashov, N. W.,Criteria for Uniform Ergodicity and Strong Stability of Markov Chains in General State Space, Probability Theory and Mathematical Statistics, Vol. 30, pp. 65–81, 1984.

  9. 9.

    Kumar, P. R., andBecker, A.,A New Family of Optimal Adaptive Controllers for Markov Chains, IEEE Transactions on Automatic Control, Vol. 27, pp. 137–146, 1982.

  10. 10.

    Borkar, V. S.,Self-Tuning Control of Diffusions without the Identifiability Condition, Journal of Optimization Theory and Applications, Vol. 68, pp. 117–138, 1991.

  11. 11.

    Borkar, V., andBagchi, A.,Parameter Estimation in Continuous-Time Stochastic Processes, Stochastics, Vol. 8, pp. 193–212, 1982.

Download references

Author information

Additional information

This research was partially supported by NSF Grants ECS-91-02714 and ECS91-13029.

Communicated by R. Rishel

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Duncan, T.E., Pasik-Duncan, B. & Stettner, L. On the ergodic and the adaptive control of stochastic differential delay systems. J Optim Theory Appl 81, 509–531 (1994).

Download citation

Key Words

  • Stochastic adaptive control
  • ergodic control
  • stochastic differential-delay systems
  • almost self-optimizing adaptive controls