Dynamics of stochastic approximation algorithms

  • Michel Benaïm
Cours Spécialisés
Part of the Lecture Notes in Mathematics book series (LNM, volume 1709)


These notes were written for a D.E.A. course given at Ecole Normale Supérieure de Cachan during the 1996–97 and 1997–98 academic years and at University Toulouse III during the 1997–98 academic year. Their aim is to introduce the reader to the dynamical system aspects of the theory of stochastic approximations.


Periodic Orbit Global Attractor Stochastic Approximation Smooth Vector Field Stochastic Approximation Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Akin, E. (1993). The General Topology of Dynamical Systems. American Mathematical Society, Providence.zbMATHGoogle Scholar
  2. Arthur, B., Ermol'ev, Y., and Kaniovskii, Y. (1983). A generalized urn problem and its applications. Cybernetics, 19: 61–71.MathSciNetCrossRefGoogle Scholar
  3. Arthur, B. M. (1988). Self-reinforcing mechanisms in economics. In W. A. P., Arrow, K. J., and Pines, D., editors, The Economy as an Evolving Complex System, SFI Studies in the Sciences of Complexity. Addison-Wesley.Google Scholar
  4. Benaïm, M. (1996). A dynamical systems approach to stochastic approximations. SIAM Journal on Control and Optimization, 34: 141–176.CrossRefzbMATHGoogle Scholar
  5. Benaïm, M. (1997). Vertex reinforced random walks and a conjecture of Pemantle. The Annals of Probability, 25: 361–392.MathSciNetCrossRefzbMATHGoogle Scholar
  6. Benaïm, M. and Hirsch, M. W. (1994). Learning processes, mixed equilibria and dynamical systems arising from repeated games. Submitted.Google Scholar
  7. Benaïm, M. and Hirsch, M. W. (1995a). Chain recurrence in surface flows. Discrete and Continuous Dynamical Systems, 1(1): 1–16.MathSciNetzbMATHGoogle Scholar
  8. Benaïm, M. and Hirsch, M. W. (1995b). Dynamics of morse-smale urn processes. Ergodic Theory and Dynamical Systems, 15: 1005–1030.MathSciNetCrossRefzbMATHGoogle Scholar
  9. Benaïm, M. and Hirsch, M. W. (1996). Asymptotic pseudotrajectories and chain recurrent flows, with applications. J. Dynam. Differential Equations, 8: 141–176.MathSciNetCrossRefzbMATHGoogle Scholar
  10. Benaïm, M. and Schreiber, S. J. (1997). Weak asymptotic pseudotrajectories for semiflows: Ergodic properties. Preprint.Google Scholar
  11. Benveniste, A., Métivier, M., and Priouret, P. (1990). Stochastic Approximation and Adaptive Algorithms. Springer-Verlag, Berlin and New York.CrossRefzbMATHGoogle Scholar
  12. Bowen, R. (1975). Omega limit sets of Axiom A diffeomorphisms. J. Diff. Eq, 18: 333–339.MathSciNetCrossRefzbMATHGoogle Scholar
  13. Brandière, O. (1996). Autour des pièges des algorithmes stochastiques. Thèse de Doctorat, Université de Marne-la-Vallée.Google Scholar
  14. Brandière, O. (1997). Some pathological traps for stochastic approximation. SIAM Journal on Control and Optimization. To Appear.Google Scholar
  15. Brandière, O. and Duflo, M. (1996). Les algorithmes stochastique contournent ils les pièges. Annales de l'IHP, 32: 395–427.Google Scholar
  16. Conley, C. C. (1978). Isolated invariant sets and the Morse index. CBMS Regional conference series in mathematics. American Mathematical Society, Providence.CrossRefzbMATHGoogle Scholar
  17. Delyon, B. (1996). General convergence results on stochastic approximation. IEEE trans. on automatic control, 41: 1245–1255.MathSciNetCrossRefzbMATHGoogle Scholar
  18. Duflo, M. (1990). Méthodes Récursives Aléatoires. Masson. English Translation: Random Iterative Models, Springer Verlag 1997.Google Scholar
  19. Duflo, M. (1996). Algorithmes Stochastiques. Mathématiques et Applications. Springer-Verlag.Google Scholar
  20. Duflo, M. (1997). Cibles atteignables avec une probabilité positive d'après m. benaim. Unpublished manuscript.Google Scholar
  21. Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes, Characterization and Convergence. John Wiley and Sons, Inc.Google Scholar
  22. Fort, J. C., and Pages, G. (1994). Résaux de neurones: des méthodes connexionnistes d'apprentissage. Matapli, 37: 31–48.Google Scholar
  23. Fort, J. C. and Pages, G. (1996). Convergence of stochastic algorithms: From Kushner-Clark theorem to the lyapounov functional method. Adv. Appl. Prob, 28: 1072–1094.MathSciNetzbMATHGoogle Scholar
  24. Fort, J. C. and Pages, G. (1997). Stochastic algorithm with non constant step: a.s. weak convergence of empirical measures. Preprint.Google Scholar
  25. Fudenberg, D., and Kreps, K. (1993). Learning mixed equilibria. Games and Econom. Behav., 5: 320–367.MathSciNetCrossRefzbMATHGoogle Scholar
  26. Fudenberg, F., and Levine, D. (1998). Theory of Learning in Games. MIT Press, Cambridge, MA. In Press.zbMATHGoogle Scholar
  27. Hartman, P. (1964). Ordinary Differential Equationq. Wiley, New York.Google Scholar
  28. Hill, B. M., Lane, D., and Sudderth, W. (1980). A strong law for some generalized urn processes. Annals of Probability, 8: 214–226.MathSciNetCrossRefzbMATHGoogle Scholar
  29. Hirsch, M. W. (1976). Differential Topology. Springer-Verlag, Berlin, New York, Heidelberg.CrossRefzbMATHGoogle Scholar
  30. Hirsch, M. W. (1994). Asymptotic phase, shadowing and reaction-diffusion systems. In Differential equations, dynamical systems and control science, volume 152 of Lectures notes in pure and applied mathematics, pages 87–99. Marcel Dekker, New-York.Google Scholar
  31. Hirsch, M. W., and Pugh, C. C. (1988). Cohomology of chain recurrent sets. Ergodic Theory and Dynamical Systems, 8: 73–80.MathSciNetCrossRefzbMATHGoogle Scholar
  32. Kaniovski, Y., and Young, H. (1995). Learning dynamics in games with stochastic perturbations. Games and Econom. Behav., 11: 330–363.MathSciNetCrossRefzbMATHGoogle Scholar
  33. Kiefer, J., and Wolfowitz, J. (1952). Stochastic estimation of the maximum of a regression function. Ann. Math. Statis, 23: 462–466.MathSciNetCrossRefzbMATHGoogle Scholar
  34. Kushner, H. J., and Clarck, C. C. (1978). Stochastic Approximation for Constrained and Unconstrained Systems. Springer-Verlag, Berlin and New York.CrossRefGoogle Scholar
  35. Kushner, H. J., and Yin, G. G. (1997). Stochastic Approximation Algorithms and Applications. Springer-Verlag, New York.CrossRefzbMATHGoogle Scholar
  36. Ljung, L. (1977). Analysis of recursive stochastic algorithms. IEEE Trans. Automat. Control., AC-22: 551–575.MathSciNetCrossRefzbMATHGoogle Scholar
  37. Ljung, L. (1986). System Identification Theory for the User. Prentice Hall, Englewood Cliffs, NJ.zbMATHGoogle Scholar
  38. Ljung, L., and Sőderstrőm, T. (1983). Theory and Practice of Recursive Identification. MIT Press, Cambridge, MA.zbMATHGoogle Scholar
  39. Mañé, R. (1987). Ergodic Theory and Differentiable Dynamics. Springer-Verlag, New York.CrossRefzbMATHGoogle Scholar
  40. Métivier, M., and Priouret, P. (1987). Théorèmes de convergence presque sure pour une classe d'algorithmes stochastiques à pas décroissant. Probability Theory and Related Fields, 74: 403–428.MathSciNetCrossRefzbMATHGoogle Scholar
  41. Munkres, J. R. (1975). Topology a first course. Prentice Hall.Google Scholar
  42. Nevelson, M. B., and Khasminskii, R. Z. (1976). Stochastic Approximation and Recursive Estimation. Translation of Math. Monographs. American Mathematical Society, Providence.Google Scholar
  43. Pemantle, R. (1990). Nonconvergence to unstable points in urn models and stochastic approximations. Annals of Probability, 18: 698–712.MathSciNetCrossRefzbMATHGoogle Scholar
  44. Pemantle, R. (1992). Vertex reinforced random walk. Probability Theory and Related Fields, 92: 117–136.MathSciNetCrossRefzbMATHGoogle Scholar
  45. Robbins, H., and Monro, S. (1951). A stochastic approximation method. Ann. Math. Statis, 22: 400–407.MathSciNetCrossRefzbMATHGoogle Scholar
  46. Robinson, C. (1977). Stability theorems and hyperbolicity in dynamical systems. Rocky Journal of Mathematics, 7: 425–434.MathSciNetCrossRefzbMATHGoogle Scholar
  47. Robinson, C. (1995). Introduction to the Theory of Dynamical Systems. Studies in Advances Mathematics. CRC Press, Boca Raton.Google Scholar
  48. Schreiber, S. J. (1997). Expansion rates and Lyapunov exponents. Discrete and Conts. Dynam. Sys., 3: 433–438.MathSciNetCrossRefzbMATHGoogle Scholar
  49. Shub, M. (1987). Global Stability of Dynamical Systems. Springer-Verlag, Berlin, New York, Heidelberg.CrossRefzbMATHGoogle Scholar
  50. Stroock, D. W. (1993). Probability Theory. An analytic view. Cambridge University Press.Google Scholar
  51. White, H. (1992). Artificial Neural Networks: Approximation and Learning Theory. Blackwell, Cambridge, Massachussets. *** DIRECT SUPPORT *** A00I6C60 00002Google Scholar

Copyright information

© Springer-Verlag 1999

Authors and Affiliations

  • Michel Benaïm

There are no affiliations available

Personalised recommendations