Skip to main content
SpringerLink
Log in

Event-driven stochastic approximation

  • Published:
Indian Journal of Pure and Applied Mathematics Aims and scope Submit manuscript

Abstract

We consider a Robbins-Monro type iteration wherein noisy measurements are event-driven and therefore arrive asynchronously. We propose a modification of step-sizes that ensures desired asymptotic behaviour regardless of this aspect. This generalizes earlier results on asynchronous stochastic approximation wherein the asynchronous behaviour is across different components, but not along the same component of the vector iteration, as is the case considered here.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Benaim, A dynamical system approach to stochastic approximation, SIAM Journal of Control and Optimization, 34 (1996), 437–472.

    Article  MathSciNet  MATH  Google Scholar 

  2. S. Bhatnagar, The Borkar-Meyn theorem for asynchronous stochastic approximation, Systems & Control Letters, 60 (2011), 472–478.

    Article  MathSciNet  MATH  Google Scholar 

  3. V. S. Borkar, Asynchronous stochastic approximation, SIAM Journal of Control and Optimization, 36 (1998), 840–851 (Correction note in: ‘Erratum: Asynchronous Stochastic Approximation’, SIAM Journal of Control and Optimization, 38 (2000), 662-663).

    Article  MathSciNet  MATH  Google Scholar 

  4. V. S. Borkar, Stochastic approximation: A dynamical systems viewpoint, Hindustan Publising Agency, New Delhi, and Cambridge University Press, Cambridge, UK, 2008.

    MATH  Google Scholar 

  5. V. S. Borkar and S. P. Meyn, The O.D.E. method for convergence of stochastic approximation and reinforcement learning, SIAM Journal of Control and Optimization, 38 (2000), 447–469.

    Article  MathSciNet  MATH  Google Scholar 

  6. L. Breiman, Probability, Addison-Wesley, Reading, Mass., 1968.

    MATH  Google Scholar 

  7. D. P. Derevitskii and A. I. Fradkov, Two models for analysing the dynamics of adaptation algorithms, Automation and Remote Control, 35 (1974), 59–67.

    MathSciNet  Google Scholar 

  8. R. Dwivedi and V. S. Borkar, Removing sampling bias in networked stochastic approximation, Proceedings of International Conference on Signal Processing and Communications (SPCOM), July 22-24, 2014, Bangalore.

    Google Scholar 

  9. N. N. Krasovskii, Stability of motion, Stanford Uni. Press, Stanford, CA, 1963.

    MATH  Google Scholar 

  10. L. Ljung, Analysis of recursive stochastic algorithms, IEEE Transactions on Automatic Control, 22 (1977), 551–575.

    Article  MathSciNet  MATH  Google Scholar 

  11. H. Robbins and J. Monro, A stochastic approximation method, Annals of Mathematical Statistics, 22 (1951), 400–407.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Author notes

  1. M. Ashok Vardhan

    Present address: Department of Electrical Engineering, University of Illinois at Urbana-Champaign, Urbana, Illinois, USA

Authors and Affiliations

  1. Department of Electrical Engineering, Indian Institute of Technology, Powai, Mumbai, 400 076, India

    Vivek S. Borkar, Neeraja Sahasrabudhe & M. Ashok Vardhan

Authors
  1. Vivek S. Borkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Neeraja Sahasrabudhe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Ashok Vardhan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vivek S. Borkar.

Additional information

Research of VSB supported in part by a J. C. Bose Fellowship and a grant for ‘Approximation of High Dimensional Optimization and Control Problems’ from Department of Science and Technology, Government of India.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Borkar, V.S., Sahasrabudhe, N. & Ashok Vardhan, M. Event-driven stochastic approximation. Indian J Pure Appl Math 47, 291–299 (2016). https://doi.org/10.1007/s13226-016-0188-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13226-016-0188-1

Key words

Search

Navigation