Foundations of stochastic approximation

  • Harro Walk
Part of the DMV Seminar book series (OWS, volume 17)


Stochastic approximation or stochastic iteration concerns recursive estimation of quantities in connection with noise contaminated observations. Historical starting points are the papers of Robbins and Monro (1951) and of Kiefer and Wolfowitz (1952) on recursive estimation of zero and extremal points, resp., of regression functions, i.e. of functions whose values can be observed with zero expectation errors.


Regression Function Stochastic Approximation Invariance Principle Functional Central Limit Theorem Partial Summation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Albert, A.E., Gardner, L.A., Jr.: Stochastic Approximation and Nonlinear Regression. M.I.T. Press; Cambridge, Mass., 1967.zbMATHGoogle Scholar
  2. Benveniste, A., Métivier, M., Priouret, P.: Stochastic Approximations and Adaptive Algorithms. Springer; Berlin, Heidelberg, New York, 1990.zbMATHCrossRefGoogle Scholar
  3. Chen, Han-Fu: Recursive Estimation and Control for Stochastic Systems. Wiley; New York, London, 1985.Google Scholar
  4. Duflo, M.: Méthodes récursives aléatoires. Masson; Paris, 1990.zbMATHGoogle Scholar
  5. Kushner, H.J.: Approximation and Weak Convergence Methods for Random Processes, with Applications to Stochastic Systems Theory. M.I.T. Press; Cambridge, Mass., 1984.zbMATHGoogle Scholar
  6. Kushner, H.J., Clark, D.S.: Stochastic Approximation Methods for Constrained and Unconstrained Systems. Springer; Berlin, Heidelberg, New York 1978.CrossRefGoogle Scholar
  7. Ljung, L., Söderström, T.: Theory and Practice of Recursive Indentification. M.I.T. Press; Cambridge, Mass., 1983.Google Scholar
  8. Marti, K.: Approximationen stochastischer Optimierungsprobleme. Hain; Königstein/Ts., 1979.zbMATHGoogle Scholar
  9. Nevel’son, M.B., Has’minskii, R.Z.: Stochastic Approximation and Recursive Estimation. Translations of Math. Monographs, Vol. 47. American Mathematical Society; Providence. R.I., 1973/76.MathSciNetGoogle Scholar
  10. Schmetterer, L.: L’Approximation Stochastique. Université de Clermont-Ferrand, 2ème ed., 1972.Google Scholar
  11. Tsypkin, Ya.Z.: Adaption and Learning in Automatic Systems. Academic Press; New York, London, 1971.zbMATHGoogle Scholar
  12. Tsypkin, Ya.Z.: Foundations of the Theory of Learning Systems. Academic Press; New York, London, 1973.zbMATHGoogle Scholar
  13. Fabian, V.: Stochastic approximation. Optimizing Methods in Statistics (ed. J.S. Rustagi), 439–470. Academic Press; New York, London, 1971.Google Scholar
  14. Lai, T.L.: Stochastic approximation and sequential search for optimum. Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, Vol. II (eds. L. LeCam, R.A. Olshen), 557–577. Wadsworth; Monterey, Ca., 1985.Google Scholar
  15. Loginov, N.V.: Methods of stochastic approximation. Automat. Remote Control 27 (1966), 706–728.zbMATHGoogle Scholar
  16. Ruppert, D.: Stochastic approximation. Handbook of Sequential Analysis (eds. B.K. Ghosh, P.K. Sen), 503–529. Marcel Dekker; New York, 1991.Google Scholar
  17. Schmetterer, L.: Stochastic approximation. Proc. Fourth Berkeley Symp. Math. Statist. Prob., I (ed. J. Neyman), 587–609. Univ. of California Press; Berkeley, Los Angeles, 1961.Google Scholar
  18. Schmetterer, L.: Multidimensional stochastic approximation. Multivariate Analysis, II (ed. P.R. Krishnaiah), 443–460. Academic Press; New York, London, 1969.Google Scholar
  19. Schmetterer, L.: From stochastic approximation to the stochastic theory of optimization. 11. Steiermärk. Math. Symp., Stift Rein, 1979. Bericht Nr. 127 der Mathematisch-Statistischen Sektion im Forschungszentrum Graz.Google Scholar
  20. Abdelhamid, S.N.: Transformation of observations in stochastic approximation. Ann. Statist. 1 (1973), 1158–1174.MathSciNetzbMATHCrossRefGoogle Scholar
  21. Anbar, D.: On optimal estimation methods using stochastic approximation procedures. Ann. Statist. 1 (1973), 1175–1184.MathSciNetzbMATHCrossRefGoogle Scholar
  22. Anbar, D.: A stochastic Newton-Raphson method. J. Statist. Planning Inference 3 (1978), 153–163.MathSciNetCrossRefGoogle Scholar
  23. Arnold, L.: Stochastische Differentialgleichungen. Oldenbourg; München, 1973.zbMATHGoogle Scholar
  24. Becker, L, Greiner, G.: On the modulus of one-parameter semigroups. Semigroup Forum 34 (1986), 185–201.MathSciNetzbMATHCrossRefGoogle Scholar
  25. Berger, E.: Asymptotic behaviour of a class of stochastic approximation procedures. Probab. Th. Rel. Fields 71 (1986), 517–552.zbMATHCrossRefGoogle Scholar
  26. Billingsley, P.: Convergence of Probability Measures. Wiley; New York, London, 1968.zbMATHGoogle Scholar
  27. Billingsley, P.: Weak Convergence of Measures: Applications in Probability. Regional Conference Series in Applied Mathematics 5. SIAM; Philadelphia, Pa., 1971.Google Scholar
  28. Blum, J.R.: Multidimensional stochastic approximation methods. Ann. Math. Statist. 25 (1954), 737–744.zbMATHCrossRefGoogle Scholar
  29. Bouton, C: Approximation gaussienne dďalgorithmes stochastiques á dynamique markovienne. Ann. Inst. Henri Poincaré-Prob. Statist. 24 (1988), 131–155.MathSciNetzbMATHGoogle Scholar
  30. Chen, G.C., Lai, T.L., Wei, C.Z.: Convergence systems and strong consistency of least squares estimates in regression models. J. Multivariate Anal. 11 (1981), 319–333.MathSciNetzbMATHCrossRefGoogle Scholar
  31. Clark, D.S.: Necessary and sufficient conditions for the Robbins-Monro method. Stochastic Process. Appl. 17 (1984), 359–367.MathSciNetzbMATHCrossRefGoogle Scholar
  32. Cramér, H., Leadbetter, M.R.: Stationary and Related Stochastic Processes. Wiley; New York, 1967.zbMATHGoogle Scholar
  33. Daleckii, Ju. L., Krein, M.G.: Stability of Solutions of Differential Equations in Banach Space. Translations of Mathematical Monographs, Vol. 43. American Mathematical Society; Providence, R.I., 1970/74.Google Scholar
  34. Deheuvels, P.: Conditions nécessaires et suffisantes de convergence ponctuelle presque sûre et uniforme presque sûre des estimateurs de la densité. C.R. Acad. Sci. Paris Ser. A 278 (1974), 1217–1220.MathSciNetzbMATHGoogle Scholar
  35. Devroye, L.: On the pointwise and integral convergence of recursive kernel estimates of probabilty densities. Utilitas Math. 15 (1979), 113–128.MathSciNetzbMATHGoogle Scholar
  36. Dupač, V.: Stochastic approximation in the presence of trend. Czech. Math. J. 16(91) (1966), 454–462.Google Scholar
  37. Dupuis, P.: Large deviations analysis of reflected diffusions and constrained stochastic approximation algorithms in convex sets. Stochastics 21 (1987), 63–96.MathSciNetzbMATHCrossRefGoogle Scholar
  38. Dupuis, P., Kushner, H.J.: Stochastic approximations via large deviations: asymptotic properties. SIAM J. Control Optimization 23 (1985), 675–696.MathSciNetzbMATHCrossRefGoogle Scholar
  39. Dupuis, P., Kushner, H.J.: Asymptotic behavior of constrained stochastic approximations via the theory of large deviations. Probab. Th. Rel. Fields 75 (1987), 223–244.MathSciNetzbMATHCrossRefGoogle Scholar
  40. Dvoretzky, A.: On stochastic approximation. Proc. Third Berkeley Symp. Math. Statist. Prob., I (ed. J. Neyman), 39–55. Univ. of California Press; Berkeley, Los Angeles 1956.Google Scholar
  41. Eckhaus, W.: New approach to the asymptotic theory of nonlinear oscillations and wave-propagation. J. Math. Anal. Appl. 49 (1975), 575–611.MathSciNetzbMATHCrossRefGoogle Scholar
  42. Fabian, V.: On asymptotic normality in stochastic approximation. Ann. Math. Statist. 39 (1968), 1327–1332.MathSciNetzbMATHCrossRefGoogle Scholar
  43. Fabian, V.: Asymptotically efficient stochastic approximation; the RM case. Ann. Statist. 1 (1973), 486–495.MathSciNetzbMATHCrossRefGoogle Scholar
  44. Fabian, V.: A local asymptotic minimax optimality of an adaptive Robbins Monro stochastic approximation procedure. Mathematical Learning Models — Theory and Algorithms (eds. U. Herkenrath, D. Kalin, W. Vogel), 43–49. Springer; Berlin, Heidelberg, New York, 1983.Google Scholar
  45. Frees, E.W., Ruppert, D.: Estimation following a Robbins-Monro designed experiment. To appear in J. Amer. Statist. Assoc.Google Scholar
  46. Freidlin, M.I.: The averaging principle and theorems on large deviations. Russian Math. Surveys 33 (1978), 117–176.CrossRefGoogle Scholar
  47. Freidlin, M.I., Wentzell, A.D.: Random Perturbations of Dynamical Systems. Springer; Berlin, Heidelberg, New York, 1984.zbMATHCrossRefGoogle Scholar
  48. Fritz, J.: Stochastic approximation for finding local maximum of probability densities. Studia Sci. Math. Hungar. 8 (1973), 309–322.MathSciNetGoogle Scholar
  49. Gänssler, P., Stute, W.: Wahrscheinlichkeitstheorie. Springer; Berlin, Heidelberg, New York, 1977.zbMATHCrossRefGoogle Scholar
  50. Gladyshev, E.G.: On stochastic approximation. Theory Probability Appl. 10 (1965), 275–278.CrossRefGoogle Scholar
  51. Goldstein, L.: Minimizing noisy functionals in Hilbert space: an extension of the Kiefer-Wolfowitz procedure. J. Theor. Probab. 1 (1988), 189–204.zbMATHCrossRefGoogle Scholar
  52. Györfi, L.: Stochastic approximation from ergodic sample for linear regression. Z. Wahrscheinlichkeitstheorie verw. Gebiete 54 (1980), 47–55.zbMATHCrossRefGoogle Scholar
  53. Györfi, L.: Adaptive linear procedures under general conditions. IEEE Trans. Information Theory IT — 30 (1984), 262–267.CrossRefGoogle Scholar
  54. Härdie, W.K., Nixdorf, R.: Nonparametric sequential estimation of zeros and extrema of regression functions. IEEE Trans. Information Theory IT — 33 (1987), 367–372.CrossRefGoogle Scholar
  55. Hale, J.: Theory of Functional Differential Equations. Springer; Berlin, Heidelberg, New York, 1977.zbMATHCrossRefGoogle Scholar
  56. Henze, E.: Lernprozesse mit zeitabhängigen Wahrscheinlichkeiten. Zeitschr. angew. Math. Mech. 46 (1966), 297–302.MathSciNetzbMATHCrossRefGoogle Scholar
  57. Herkenrath, U.: On the speed of convergence of the Kiefer-Wolfowitz stochastic approximation procedure. Math. Operationsforsch. Statist., Ser. Statistics 12 (1981), 377–392.MathSciNetzbMATHGoogle Scholar
  58. Hiriart-Urruty, J.B.: Algorithms of penalization type and dual type for the solution of stochastic optimization problems with stochastic constraints. Recent Developments in Statistics (ed. J.R. Barra et al.), 183–219. North-Holland; Amsterdam, New York, Oxford, 1977.Google Scholar
  59. Kersting, G.: Almost sure approximation of the Robbins-Monro process by sums of independent random variables. Ann. Probab. 5 (1977), 954–965.MathSciNetzbMATHCrossRefGoogle Scholar
  60. Kiefer, J., Wolfowitz, J.: Stochastic estimation of the maximum of a regression function. Ann. Math. Statist. 23 (1952), 462–466.MathSciNetzbMATHCrossRefGoogle Scholar
  61. Kottmann, Th.: Learning procedures and rational expectations in linear models with forecast feedback. Diss. Univ. Bonn 1990.Google Scholar
  62. Kushner, H.J.: Stochastic approximation algorithms for the local optimization of functions with nonunique stationary points. IEEE Trans. Automatic Control AC-17 (1972), 646–654.MathSciNetCrossRefGoogle Scholar
  63. Kushner, H.J.: Asymptotic behavior of stochastic approximation and large deviations. IEEE Trans. Automatic Control AC-29 (1984), 984–990.MathSciNetCrossRefGoogle Scholar
  64. Kushner, H.J., Huang, H.: Rates of convergence for stochastic approximation type algorithms. SIAM J. Control Optim. 17 (1979), 607–617.MathSciNetzbMATHCrossRefGoogle Scholar
  65. Kushner, H.J., Sanvicente, E.: Stochastic approximation for constrained systems with observation noise on the system and constraints. Automatica 11 (1975), 375–380.MathSciNetzbMATHCrossRefGoogle Scholar
  66. Lai, T.L., Robbins, H.: Limit theorems for weighted sums and stochastic approximation processes. Proc. Nat. Acad. Sci. USA 75 (1978), 1068–1070.MathSciNetzbMATHCrossRefGoogle Scholar
  67. Lai, T.L., Robbins, H.: Consistency and asymptotic efficiency of slope estimates in stochastic approximation schemes. Z. Wahrscheinlichkeitstheorie verw. Geb. 56 (1981), 329–360.MathSciNetzbMATHCrossRefGoogle Scholar
  68. Ljung, L.: Analysis of recursive stochastic algorithms. IEEE Trans. Automatic Control AC-22 (1977), 551–575.MathSciNetCrossRefGoogle Scholar
  69. Ljung, L.: Strong convergence of a stochastic approximation algorithm. Ann. Statist. 6 (1978), 680–696.MathSciNetzbMATHCrossRefGoogle Scholar
  70. Mark, G.: Log-log-Invarianzprinzipien für Prozesse der stochastischen Approximation. Mitteilungen Math. Sem. Giessen 153 (1982).Google Scholar
  71. McLeish, D.L.: Functional and random central limit theorems for the Robbins-Monro process. J. Appl. Probab. 13 (1976), 148–154.MathSciNetzbMATHCrossRefGoogle Scholar
  72. Métivier, M., Priouret, P.: Applications of a Kushner and Clark lemma to general classes of stochastic algorithms. IEEE Trans. Information Theory IT-30 (1984), 140–151.CrossRefGoogle Scholar
  73. Métivier, M., Priouret, P.: Théorèmes de convergence presque sûre pour une classe dďalgorithmes stochastiques á pas décroissant. Probab. Th. Rel. Fields 74 (1987), 403–428.zbMATHCrossRefGoogle Scholar
  74. Milnor, J.W.: Topology from the Differentiable Viewpoint. Univ. Press of Virginia; Charlottesville, 1965/72.zbMATHGoogle Scholar
  75. Mohr, M.: Asymptotic theory for ordinary least squares estimators in regression models with forecast-feedback. Diss. Univ. Bonn 1990.Google Scholar
  76. Nazin, A.V., Polyak, B.T., Tsybakov, A.B.: Passive stochastic approximation. Automat. Remote Control 50 (1989), 1563–1569.MathSciNetzbMATHGoogle Scholar
  77. Nixdorf, R.: An invariance principle for a finite dimensional stochastic approximation method in a Hilbert space. J. Multivariate Analysis 15 (1984), 252–260.MathSciNetzbMATHCrossRefGoogle Scholar
  78. Pakes, A.: Some remarks on the paper by Theodorescu and Wolff: “ Sequential estimation of expectations in the presence of trend”, Austral. J. Statist. 24 (1982), 89–97.MathSciNetzbMATHCrossRefGoogle Scholar
  79. Parthasarathy, K.R.: Probability Measures on Metric Spaces. Academic Press; New York, London, 1967.zbMATHGoogle Scholar
  80. Parzen, E.: On estimation of a probabilty density function and mode. Ann. Math. Statist. 33 (1962), 1065–1076.MathSciNetzbMATHCrossRefGoogle Scholar
  81. Pechtl, A.: Ein Invarianzprinzip zu einem Gaußschen Markoff-Prozeß. Diplomarbeit Univ. Stuttgart 1988.Google Scholar
  82. Pflug, G.: Optimale sequentielle Zerlegung. Math. Operationsforsch. Statist., Ser. Statistics 11 (1980), 287–295.MathSciNetzbMATHGoogle Scholar
  83. Pflug, G.: On the convergence of a penalty-type stochastic approximation procedure. J. Information & Optimization Sciences 2 (1981), 249–258.MathSciNetzbMATHGoogle Scholar
  84. Polyak, B. T.: New method of stochastic approximation type. Automat. Remote Control 51 (1990), 937–946.MathSciNetzbMATHGoogle Scholar
  85. Polyak, B. T., Juditsky, A. B.: Acceleration of stochastic approximation by averaging. Technical Report, Institute for Control Sciences of USSR Acad. Sci. (1990).Google Scholar
  86. Prakasa Rao, B. L. S.: Nonparametric Functional Estimation. Academic Press; New York, London, 1983.zbMATHGoogle Scholar
  87. Révész, P.: The Laws of Large Numbers. Academic Press; New York, London. Akadémiai Kiadö; Budapest, 1968.zbMATHGoogle Scholar
  88. Révész, P.: Robbins-Monro procedure in a Hilbert space and its application in the theory of learning processes I. Studia Sci. Math. Hungar. 8 (1973), 391–398.MathSciNetGoogle Scholar
  89. Révész, P.: Robbins-Monro procedure in a Hilbert space II. Studia Sci. Math. Hungar. 8 (1973), 469–472.MathSciNetGoogle Scholar
  90. Révész, P.: How to apply the method of stochastic approximation in the non-parametric estimation of a regression function. Math. Operationsforschung Statist., Ser. Statistics 8 (1977), 119–126.zbMATHGoogle Scholar
  91. Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Statist. 22 (1951), 400–407.MathSciNetzbMATHCrossRefGoogle Scholar
  92. Robbins, H., Siegmund, D.: A convergence theorem for nonnegative almost supermartingales and some applications. Optimizing Methods in Statistics (ed. J.S. Rustagi) 233–257. Academic Press; New York, London, 1971.Google Scholar
  93. Rockafellar, R.T.: A dual approach to solving nonlinear programming problems by unconstrained optimization. Math. Progr. 5 (1973), 354–373.MathSciNetzbMATHCrossRefGoogle Scholar
  94. Rosenblatt, M.: Remarks on some nonparametric estimates of a density function. Ann. Math. Statist. 27 (1956), 832–837.MathSciNetzbMATHCrossRefGoogle Scholar
  95. Ruppert, D.: Almost sure approximations to the Robbins-Monro and Kiefer-Wolfowitz processes with dependent noise. Ann. Probab. 10 (1982), 178–187.MathSciNetzbMATHCrossRefGoogle Scholar
  96. Ruppert, D.: Efficient estimators from a slowly convergent Robbins-Monro process. Technical Report No. 781 (1988), School of Operations Research and Industrial Engineering, Cornell University Ithaca, New York.Google Scholar
  97. Sanchez-Palencia, E.: Méthode de centrage-estimation de l’erreur et comportement des trajectoires dans lľespace des phases, Int. J. Non-Linear Mechanics 11(176) (1976), 251–263.zbMATHCrossRefGoogle Scholar
  98. Sanders, J.A., Verhulst, F.: Averaging Methods in Nonlinear Dynamical Systems. Springer; Berlin, Heidelberg, New York, 1985.zbMATHGoogle Scholar
  99. Schmetterer, L.: Sur quelques résultats asymptotiques pour le processus de Robbins-Monro. Annales Scientifiques de lľUniversité de Clermont 58 (1976), 166–176.MathSciNetGoogle Scholar
  100. Schwabe, R.: Strong representation of an adaptive stochastic approximation procedure. Stoch. Processes Appl. 23 (1986), 115–130.MathSciNetzbMATHCrossRefGoogle Scholar
  101. Shwartz, A., Berman, N.: Abstract stochastic approximations and applications. Stoch. Processes Appl. 31 (1989), 133–149.MathSciNetzbMATHCrossRefGoogle Scholar
  102. Venter, J.H.: An extension of the Robbins-Monro procedure. Ann. Math. Statist. 38 (1967), 181–190.MathSciNetzbMATHCrossRefGoogle Scholar
  103. Walk, H.: An invariance principle for the Robbins-Monro process in a Hilbert space. Z. Wahrscheinlichkeitstheorie verw. Gebiete 39 (1977), 135–150.MathSciNetzbMATHCrossRefGoogle Scholar
  104. Walk, H.: Stochastic iteration for a constrained optimization problem. Commun. Statist. — Sequential Analysis 2 (1983-84), 369–385.Google Scholar
  105. Walk, H.: Limit behaviour of stochastic approximation processes. Statistics & Decisions 6 (1988), 109–128.MathSciNetzbMATHGoogle Scholar
  106. Walk, H., Zsidó, L.: Convergence of the Robbins-Monro method for linear problems in a Banach space. J. Math. Anal. Appl. 139 (1989), 152–177.MathSciNetzbMATHCrossRefGoogle Scholar
  107. Wei, C.Z.: Multivariate adaptive stochastic approximation. Ann. Statist. 15 (1987), 1115–1130.MathSciNetzbMATHCrossRefGoogle Scholar
  108. Wertz, W.: Sequential and recursive estimators of the probability density. Statistics 16 (1985), 277–295.MathSciNetzbMATHCrossRefGoogle Scholar
  109. Widrow, B., Hoff, M.E., Jr.: Adaptive switching circuits. IRE WESCON Convention Record, part 4 (1960), 96–104.Google Scholar
  110. Wolverton, C.T., Wagner, T.J.: Recursive estimates of probability densities. IEEE Trans. Systems Sci. Cybernet. SSC-5 (1969), 246–247.CrossRefGoogle Scholar
  111. Woodroofe, M.: Normal approximation and large deviations for the Robbins-Monro process. Z. Wahrscheinlichkeitstheorie verw. Geb. 21 (1972), 329–338.MathSciNetzbMATHCrossRefGoogle Scholar
  112. Yamato, H.: Sequential estimation of a continuous probability density function and mode. Bull. Math. Statist. 14 (1971), 1–12.MathSciNetzbMATHGoogle Scholar
  113. Yosida, K.: Functional Analysis. 2nd ed. Springer; Berlin, Heidelberg, New York, 1968.Google Scholar

Copyright information

© Springer Basel AG 1992

Authors and Affiliations

  • Harro Walk
    • 1
  1. 1.Mathematisches Institut AUniversity of StuttgartStuttgart 80Germany

Personalised recommendations