Skip to main content
SpringerLink
Log in

Ergodic control of multidimensional diffusions, II: Adaptive control

  • Published:
Applied Mathematics & Optimization Aims and scope Submit manuscript

Abstract

The self-tuning scheme for the adaptive control of a diffusion process is studied with long-run average cost criterion and maximum likelihood estimation of parameters. Asymptotic optimality under a suitable identifiability condition is established under two alternative sets of hypotheses—a Lyapunov-type stability criterion and a condition on cost which penalizes instability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. D. G. Aronson, Bounds for the fundamental solution of a parabolic equation, Bull. Amer. Math. Soc. 73 (1967), pp. 890–896.

    Article  MathSciNet  MATH  Google Scholar 

  2. K. Aström, B. Wittenmark, On self-tuning regulators, Automatica 9 (1973), pp. 185–199.

    Article  MATH  Google Scholar 

  3. V. E. Beneš, Existence of optimal strategies based on specified information, for a class of stochastic decision problems, SIAM J. Control 8 (1970), pp. 179–188.

    Article  MathSciNet  MATH  Google Scholar 

  4. A. Bensoussan, Stochastic Control of Functional Analysis Methods, North-Holland, Amsterdam, 1982.

    MATH  Google Scholar 

  5. R. N. Bhattacharya, Asymptotic behaviour of several dimensional diffusions, in Stochastic Nonlinear Systems, L. Arnold and R. Lefever, eds., Springer-Verlag, New York, 1981, pp. 86–99.

    Google Scholar 

  6. V. S. Borkar, A topology for Markov controls, Appl. Math. Optim. 20 (1989), pp. 55–62.

    Article  MathSciNet  MATH  Google Scholar 

  7. V. S. Borkar, A. Bagchi, Parameter estimation in continuous-time stochastic processes, Stochastics 8 (1982), pp. 193–212.

    Article  MathSciNet  MATH  Google Scholar 

  8. V. S. Borkar, M. K. Ghosh, Ergodic control of multidimensional diffusions, I: the existence results, SIAM J. Control Optim. 26 (1988), pp. 112–126.

    Article  MathSciNet  MATH  Google Scholar 

  9. V. S. Borkar, P. Varaiya, Identification and adaptive control of Markov chains, SIAM J. Control Optim. 20 (1982), pp. 470–489.

    Article  MathSciNet  MATH  Google Scholar 

  10. R. M. Cox, Stationary and discounted control of diffusion processes, Ph.D. Thesis, Columbia University, 1984.

  11. W. H. Fleming, Generalized solutions in stochastic control, in Differential Games and Control Theory III, E. Roxin, P. T. Liu and R. L. Sternberg, eds., Marcel Dekker, New York, 1977, pp. 147–165.

    Google Scholar 

  12. D. Gilbarg, N. S. Trudinger, Elliptic Partial Differential Equations of Second Order, 2nd edn., Springer-Verlag, New York, 1983.

    MATH  Google Scholar 

  13. P. Grisvard, Elliptic Problems in Non-Smooth Domains, Pitman, Boston, 1965.

    Google Scholar 

  14. N. Ikeda, S. Watanabe, Stochastic Differential Equations and Diffusion Processes, North-Holland Kodansha, Amsterdam, 1981.

    MATH  Google Scholar 

  15. R. Z. Khas'Minskii, Ergodic properties of recurrent diffusion processes and stabilization of the solution to the Cauchy problem of parabolic equations, Theory Probab. Appl. 2 (1960), pp. 179–196.

    Article  MATH  Google Scholar 

  16. N. V. Krylov, Controlled Diffusion Processes, Springer-Verlag, New York, 1980.

    Book  MATH  Google Scholar 

  17. P. R. Kumar, Survey of results in stochastic adaptive control, SIAM J. Control Optim. 23 (1985), pp. 329–380.

    Article  MathSciNet  MATH  Google Scholar 

  18. P. R. Kumar, W. Lin, Optimal adaptive controllers for unknown Markov chains, IEEE Trans. Automat. Control (1982), pp. 765–774.

  19. H. J. Kushner, Existence results for optimal stochastic control, J. Optim. Theory Appl. 15 (1975), pp. 347–359.

    Article  MathSciNet  MATH  Google Scholar 

  20. O. A. Ladyzhenskaya, N. N. Ural'Tseva, Linear and Quasilinear Elliptic Equations, Academic Press, New York, 1968.

    MATH  Google Scholar 

  21. P. L. Lions, On the Hamilton-Jacobi-Bellman equations, Acta Appl. Math. 1 (1983), pp. 17–41.

    Article  MathSciNet  MATH  Google Scholar 

  22. R. S. Lipster, A. N. Shiryayev, Statistics of Random Processes I, Springer-Verlag, New York, 1977.

    Google Scholar 

  23. P. Mandl, Estimation and control of Markov chains, Adv. in Appl. Probab. 6 (1974), pp. 40–60.

    Article  MathSciNet  MATH  Google Scholar 

  24. P. Mandl, Self-optimizing control of Markov processes and Markov potential theory, Proceedings of ICM, Warsaw, 1983, vol. 2, pp. 1097–1105.

  25. M. Schäl, Estimation and control in discounted dynamic programming, Stochastics (1987), pp. 51–71.

  26. A. Ju. Veretennikov, On strong solutions and explicit formulas for solutions of stochastic integral equations, Math. USSR-Sb. 39 (1981), pp. 387–403.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tata Institute of Fundamental Research, Bangalore Centre, I.I.Sc. Campus, P.O. Box 1234, 560012, Bangalore, India

    Vivek S. Borkar & Mrinal K. Ghosh

Authors
  1. Vivek S. Borkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mrinal K. Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Communicated by S. K. Mitter

Rights and permissions

Reprints and permissions

About this article

Cite this article

Borkar, V.S., Ghosh, M.K. Ergodic control of multidimensional diffusions, II: Adaptive control. Appl Math Optim 21, 191–220 (1990). https://doi.org/10.1007/BF01445163

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01445163

Keywords

Search

Navigation