Skip to main content
SpringerLink
Log in

Foundations of stochastic approximation

  • Chapter
Stochastic Approximation and Optimization of Random Systems

Part of the book series: DMV Seminar ((OWS,volume 17))

Abstract

Stochastic approximation or stochastic iteration concerns recursive estimation of quantities in connection with noise contaminated observations. Historical starting points are the papers of Robbins and Monro (1951) and of Kiefer and Wolfowitz (1952) on recursive estimation of zero and extremal points, resp., of regression functions, i.e. of functions whose values can be observed with zero expectation errors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 29.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 39.95
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Albert, A.E., Gardner, L.A., Jr.: Stochastic Approximation and Nonlinear Regression. M.I.T. Press; Cambridge, Mass., 1967.

    MATH  Google Scholar 

  • Benveniste, A., Métivier, M., Priouret, P.: Stochastic Approximations and Adaptive Algorithms. Springer; Berlin, Heidelberg, New York, 1990.

    Book  MATH  Google Scholar 

  • Chen, Han-Fu: Recursive Estimation and Control for Stochastic Systems. Wiley; New York, London, 1985.

    Google Scholar 

  • Duflo, M.: Méthodes récursives aléatoires. Masson; Paris, 1990.

    MATH  Google Scholar 

  • Kushner, H.J.: Approximation and Weak Convergence Methods for Random Processes, with Applications to Stochastic Systems Theory. M.I.T. Press; Cambridge, Mass., 1984.

    MATH  Google Scholar 

  • Kushner, H.J., Clark, D.S.: Stochastic Approximation Methods for Constrained and Unconstrained Systems. Springer; Berlin, Heidelberg, New York 1978.

    Book  Google Scholar 

  • Ljung, L., Söderström, T.: Theory and Practice of Recursive Indentification. M.I.T. Press; Cambridge, Mass., 1983.

    Google Scholar 

  • Marti, K.: Approximationen stochastischer Optimierungsprobleme. Hain; Königstein/Ts., 1979.

    MATH  Google Scholar 

  • Nevel’son, M.B., Has’minskii, R.Z.: Stochastic Approximation and Recursive Estimation. Translations of Math. Monographs, Vol. 47. American Mathematical Society; Providence. R.I., 1973/76.

    MathSciNet  Google Scholar 

  • Schmetterer, L.: L’Approximation Stochastique. Université de Clermont-Ferrand, 2ème ed., 1972.

    Google Scholar 

  • Tsypkin, Ya.Z.: Adaption and Learning in Automatic Systems. Academic Press; New York, London, 1971.

    MATH  Google Scholar 

  • Tsypkin, Ya.Z.: Foundations of the Theory of Learning Systems. Academic Press; New York, London, 1973.

    MATH  Google Scholar 

  • Fabian, V.: Stochastic approximation. Optimizing Methods in Statistics (ed. J.S. Rustagi), 439–470. Academic Press; New York, London, 1971.

    Google Scholar 

  • Lai, T.L.: Stochastic approximation and sequential search for optimum. Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, Vol. II (eds. L. LeCam, R.A. Olshen), 557–577. Wadsworth; Monterey, Ca., 1985.

    Google Scholar 

  • Loginov, N.V.: Methods of stochastic approximation. Automat. Remote Control 27 (1966), 706–728.

    MATH  Google Scholar 

  • Ruppert, D.: Stochastic approximation. Handbook of Sequential Analysis (eds. B.K. Ghosh, P.K. Sen), 503–529. Marcel Dekker; New York, 1991.

    Google Scholar 

  • Schmetterer, L.: Stochastic approximation. Proc. Fourth Berkeley Symp. Math. Statist. Prob., I (ed. J. Neyman), 587–609. Univ. of California Press; Berkeley, Los Angeles, 1961.

    Google Scholar 

  • Schmetterer, L.: Multidimensional stochastic approximation. Multivariate Analysis, II (ed. P.R. Krishnaiah), 443–460. Academic Press; New York, London, 1969.

    Google Scholar 

  • Schmetterer, L.: From stochastic approximation to the stochastic theory of optimization. 11. Steiermärk. Math. Symp., Stift Rein, 1979. Bericht Nr. 127 der Mathematisch-Statistischen Sektion im Forschungszentrum Graz.

    Google Scholar 

  • Abdelhamid, S.N.: Transformation of observations in stochastic approximation. Ann. Statist. 1 (1973), 1158–1174.

    Article  MathSciNet  MATH  Google Scholar 

  • Anbar, D.: On optimal estimation methods using stochastic approximation procedures. Ann. Statist. 1 (1973), 1175–1184.

    Article  MathSciNet  MATH  Google Scholar 

  • Anbar, D.: A stochastic Newton-Raphson method. J. Statist. Planning Inference 3 (1978), 153–163.

    Article  MathSciNet  Google Scholar 

  • Arnold, L.: Stochastische Differentialgleichungen. Oldenbourg; München, 1973.

    MATH  Google Scholar 

  • Becker, L, Greiner, G.: On the modulus of one-parameter semigroups. Semigroup Forum 34 (1986), 185–201.

    Article  MathSciNet  MATH  Google Scholar 

  • Berger, E.: Asymptotic behaviour of a class of stochastic approximation procedures. Probab. Th. Rel. Fields 71 (1986), 517–552.

    Article  MATH  Google Scholar 

  • Billingsley, P.: Convergence of Probability Measures. Wiley; New York, London, 1968.

    MATH  Google Scholar 

  • Billingsley, P.: Weak Convergence of Measures: Applications in Probability. Regional Conference Series in Applied Mathematics 5. SIAM; Philadelphia, Pa., 1971.

    Google Scholar 

  • Blum, J.R.: Multidimensional stochastic approximation methods. Ann. Math. Statist. 25 (1954), 737–744.

    Article  MATH  Google Scholar 

  • Bouton, C: Approximation gaussienne dďalgorithmes stochastiques á dynamique markovienne. Ann. Inst. Henri Poincaré-Prob. Statist. 24 (1988), 131–155.

    MathSciNet  MATH  Google Scholar 

  • Chen, G.C., Lai, T.L., Wei, C.Z.: Convergence systems and strong consistency of least squares estimates in regression models. J. Multivariate Anal. 11 (1981), 319–333.

    Article  MathSciNet  MATH  Google Scholar 

  • Clark, D.S.: Necessary and sufficient conditions for the Robbins-Monro method. Stochastic Process. Appl. 17 (1984), 359–367.

    Article  MathSciNet  MATH  Google Scholar 

  • Cramér, H., Leadbetter, M.R.: Stationary and Related Stochastic Processes. Wiley; New York, 1967.

    MATH  Google Scholar 

  • Daleckii, Ju. L., Krein, M.G.: Stability of Solutions of Differential Equations in Banach Space. Translations of Mathematical Monographs, Vol. 43. American Mathematical Society; Providence, R.I., 1970/74.

    Google Scholar 

  • Deheuvels, P.: Conditions nécessaires et suffisantes de convergence ponctuelle presque sûre et uniforme presque sûre des estimateurs de la densité. C.R. Acad. Sci. Paris Ser. A 278 (1974), 1217–1220.

    MathSciNet  MATH  Google Scholar 

  • Devroye, L.: On the pointwise and integral convergence of recursive kernel estimates of probabilty densities. Utilitas Math. 15 (1979), 113–128.

    MathSciNet  MATH  Google Scholar 

  • Dupač, V.: Stochastic approximation in the presence of trend. Czech. Math. J. 16(91) (1966), 454–462.

    Google Scholar 

  • Dupuis, P.: Large deviations analysis of reflected diffusions and constrained stochastic approximation algorithms in convex sets. Stochastics 21 (1987), 63–96.

    Article  MathSciNet  MATH  Google Scholar 

  • Dupuis, P., Kushner, H.J.: Stochastic approximations via large deviations: asymptotic properties. SIAM J. Control Optimization 23 (1985), 675–696.

    Article  MathSciNet  MATH  Google Scholar 

  • Dupuis, P., Kushner, H.J.: Asymptotic behavior of constrained stochastic approximations via the theory of large deviations. Probab. Th. Rel. Fields 75 (1987), 223–244.

    Article  MathSciNet  MATH  Google Scholar 

  • Dvoretzky, A.: On stochastic approximation. Proc. Third Berkeley Symp. Math. Statist. Prob., I (ed. J. Neyman), 39–55. Univ. of California Press; Berkeley, Los Angeles 1956.

    Google Scholar 

  • Eckhaus, W.: New approach to the asymptotic theory of nonlinear oscillations and wave-propagation. J. Math. Anal. Appl. 49 (1975), 575–611.

    Article  MathSciNet  MATH  Google Scholar 

  • Fabian, V.: On asymptotic normality in stochastic approximation. Ann. Math. Statist. 39 (1968), 1327–1332.

    Article  MathSciNet  MATH  Google Scholar 

  • Fabian, V.: Asymptotically efficient stochastic approximation; the RM case. Ann. Statist. 1 (1973), 486–495.

    Article  MathSciNet  MATH  Google Scholar 

  • Fabian, V.: A local asymptotic minimax optimality of an adaptive Robbins Monro stochastic approximation procedure. Mathematical Learning Models — Theory and Algorithms (eds. U. Herkenrath, D. Kalin, W. Vogel), 43–49. Springer; Berlin, Heidelberg, New York, 1983.

    Google Scholar 

  • Frees, E.W., Ruppert, D.: Estimation following a Robbins-Monro designed experiment. To appear in J. Amer. Statist. Assoc.

    Google Scholar 

  • Freidlin, M.I.: The averaging principle and theorems on large deviations. Russian Math. Surveys 33 (1978), 117–176.

    Article  Google Scholar 

  • Freidlin, M.I., Wentzell, A.D.: Random Perturbations of Dynamical Systems. Springer; Berlin, Heidelberg, New York, 1984.

    Book  MATH  Google Scholar 

  • Fritz, J.: Stochastic approximation for finding local maximum of probability densities. Studia Sci. Math. Hungar. 8 (1973), 309–322.

    MathSciNet  Google Scholar 

  • Gänssler, P., Stute, W.: Wahrscheinlichkeitstheorie. Springer; Berlin, Heidelberg, New York, 1977.

    Book  MATH  Google Scholar 

  • Gladyshev, E.G.: On stochastic approximation. Theory Probability Appl. 10 (1965), 275–278.

    Article  Google Scholar 

  • Goldstein, L.: Minimizing noisy functionals in Hilbert space: an extension of the Kiefer-Wolfowitz procedure. J. Theor. Probab. 1 (1988), 189–204.

    Article  MATH  Google Scholar 

  • Györfi, L.: Stochastic approximation from ergodic sample for linear regression. Z. Wahrscheinlichkeitstheorie verw. Gebiete 54 (1980), 47–55.

    Article  MATH  Google Scholar 

  • Györfi, L.: Adaptive linear procedures under general conditions. IEEE Trans. Information Theory IT — 30 (1984), 262–267.

    Article  Google Scholar 

  • Härdie, W.K., Nixdorf, R.: Nonparametric sequential estimation of zeros and extrema of regression functions. IEEE Trans. Information Theory IT — 33 (1987), 367–372.

    Article  Google Scholar 

  • Hale, J.: Theory of Functional Differential Equations. Springer; Berlin, Heidelberg, New York, 1977.

    Book  MATH  Google Scholar 

  • Henze, E.: Lernprozesse mit zeitabhängigen Wahrscheinlichkeiten. Zeitschr. angew. Math. Mech. 46 (1966), 297–302.

    Article  MathSciNet  MATH  Google Scholar 

  • Herkenrath, U.: On the speed of convergence of the Kiefer-Wolfowitz stochastic approximation procedure. Math. Operationsforsch. Statist., Ser. Statistics 12 (1981), 377–392.

    MathSciNet  MATH  Google Scholar 

  • Hiriart-Urruty, J.B.: Algorithms of penalization type and dual type for the solution of stochastic optimization problems with stochastic constraints. Recent Developments in Statistics (ed. J.R. Barra et al.), 183–219. North-Holland; Amsterdam, New York, Oxford, 1977.

    Google Scholar 

  • Kersting, G.: Almost sure approximation of the Robbins-Monro process by sums of independent random variables. Ann. Probab. 5 (1977), 954–965.

    Article  MathSciNet  MATH  Google Scholar 

  • Kiefer, J., Wolfowitz, J.: Stochastic estimation of the maximum of a regression function. Ann. Math. Statist. 23 (1952), 462–466.

    Article  MathSciNet  MATH  Google Scholar 

  • Kottmann, Th.: Learning procedures and rational expectations in linear models with forecast feedback. Diss. Univ. Bonn 1990.

    Google Scholar 

  • Kushner, H.J.: Stochastic approximation algorithms for the local optimization of functions with nonunique stationary points. IEEE Trans. Automatic Control AC-17 (1972), 646–654.

    Article  MathSciNet  Google Scholar 

  • Kushner, H.J.: Asymptotic behavior of stochastic approximation and large deviations. IEEE Trans. Automatic Control AC-29 (1984), 984–990.

    Article  MathSciNet  Google Scholar 

  • Kushner, H.J., Huang, H.: Rates of convergence for stochastic approximation type algorithms. SIAM J. Control Optim. 17 (1979), 607–617.

    Article  MathSciNet  MATH  Google Scholar 

  • Kushner, H.J., Sanvicente, E.: Stochastic approximation for constrained systems with observation noise on the system and constraints. Automatica 11 (1975), 375–380.

    Article  MathSciNet  MATH  Google Scholar 

  • Lai, T.L., Robbins, H.: Limit theorems for weighted sums and stochastic approximation processes. Proc. Nat. Acad. Sci. USA 75 (1978), 1068–1070.

    Article  MathSciNet  MATH  Google Scholar 

  • Lai, T.L., Robbins, H.: Consistency and asymptotic efficiency of slope estimates in stochastic approximation schemes. Z. Wahrscheinlichkeitstheorie verw. Geb. 56 (1981), 329–360.

    Article  MathSciNet  MATH  Google Scholar 

  • Ljung, L.: Analysis of recursive stochastic algorithms. IEEE Trans. Automatic Control AC-22 (1977), 551–575.

    Article  MathSciNet  Google Scholar 

  • Ljung, L.: Strong convergence of a stochastic approximation algorithm. Ann. Statist. 6 (1978), 680–696.

    Article  MathSciNet  MATH  Google Scholar 

  • Mark, G.: Log-log-Invarianzprinzipien für Prozesse der stochastischen Approximation. Mitteilungen Math. Sem. Giessen 153 (1982).

    Google Scholar 

  • McLeish, D.L.: Functional and random central limit theorems for the Robbins-Monro process. J. Appl. Probab. 13 (1976), 148–154.

    Article  MathSciNet  MATH  Google Scholar 

  • Métivier, M., Priouret, P.: Applications of a Kushner and Clark lemma to general classes of stochastic algorithms. IEEE Trans. Information Theory IT-30 (1984), 140–151.

    Article  Google Scholar 

  • Métivier, M., Priouret, P.: Théorèmes de convergence presque sûre pour une classe dďalgorithmes stochastiques á pas décroissant. Probab. Th. Rel. Fields 74 (1987), 403–428.

    Article  MATH  Google Scholar 

  • Milnor, J.W.: Topology from the Differentiable Viewpoint. Univ. Press of Virginia; Charlottesville, 1965/72.

    MATH  Google Scholar 

  • Mohr, M.: Asymptotic theory for ordinary least squares estimators in regression models with forecast-feedback. Diss. Univ. Bonn 1990.

    Google Scholar 

  • Nazin, A.V., Polyak, B.T., Tsybakov, A.B.: Passive stochastic approximation. Automat. Remote Control 50 (1989), 1563–1569.

    MathSciNet  MATH  Google Scholar 

  • Nixdorf, R.: An invariance principle for a finite dimensional stochastic approximation method in a Hilbert space. J. Multivariate Analysis 15 (1984), 252–260.

    Article  MathSciNet  MATH  Google Scholar 

  • Pakes, A.: Some remarks on the paper by Theodorescu and Wolff: “ Sequential estimation of expectations in the presence of trend”, Austral. J. Statist. 24 (1982), 89–97.

    Article  MathSciNet  MATH  Google Scholar 

  • Parthasarathy, K.R.: Probability Measures on Metric Spaces. Academic Press; New York, London, 1967.

    MATH  Google Scholar 

  • Parzen, E.: On estimation of a probabilty density function and mode. Ann. Math. Statist. 33 (1962), 1065–1076.

    Article  MathSciNet  MATH  Google Scholar 

  • Pechtl, A.: Ein Invarianzprinzip zu einem Gaußschen Markoff-Prozeß. Diplomarbeit Univ. Stuttgart 1988.

    Google Scholar 

  • Pflug, G.: Optimale sequentielle Zerlegung. Math. Operationsforsch. Statist., Ser. Statistics 11 (1980), 287–295.

    MathSciNet  MATH  Google Scholar 

  • Pflug, G.: On the convergence of a penalty-type stochastic approximation procedure. J. Information & Optimization Sciences 2 (1981), 249–258.

    MathSciNet  MATH  Google Scholar 

  • Polyak, B. T.: New method of stochastic approximation type. Automat. Remote Control 51 (1990), 937–946.

    MathSciNet  MATH  Google Scholar 

  • Polyak, B. T., Juditsky, A. B.: Acceleration of stochastic approximation by averaging. Technical Report, Institute for Control Sciences of USSR Acad. Sci. (1990).

    Google Scholar 

  • Prakasa Rao, B. L. S.: Nonparametric Functional Estimation. Academic Press; New York, London, 1983.

    MATH  Google Scholar 

  • Révész, P.: The Laws of Large Numbers. Academic Press; New York, London. Akadémiai Kiadö; Budapest, 1968.

    MATH  Google Scholar 

  • Révész, P.: Robbins-Monro procedure in a Hilbert space and its application in the theory of learning processes I. Studia Sci. Math. Hungar. 8 (1973), 391–398.

    MathSciNet  Google Scholar 

  • Révész, P.: Robbins-Monro procedure in a Hilbert space II. Studia Sci. Math. Hungar. 8 (1973), 469–472.

    MathSciNet  Google Scholar 

  • Révész, P.: How to apply the method of stochastic approximation in the non-parametric estimation of a regression function. Math. Operationsforschung Statist., Ser. Statistics 8 (1977), 119–126.

    MATH  Google Scholar 

  • Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Statist. 22 (1951), 400–407.

    Article  MathSciNet  MATH  Google Scholar 

  • Robbins, H., Siegmund, D.: A convergence theorem for nonnegative almost supermartingales and some applications. Optimizing Methods in Statistics (ed. J.S. Rustagi) 233–257. Academic Press; New York, London, 1971.

    Google Scholar 

  • Rockafellar, R.T.: A dual approach to solving nonlinear programming problems by unconstrained optimization. Math. Progr. 5 (1973), 354–373.

    Article  MathSciNet  MATH  Google Scholar 

  • Rosenblatt, M.: Remarks on some nonparametric estimates of a density function. Ann. Math. Statist. 27 (1956), 832–837.

    Article  MathSciNet  MATH  Google Scholar 

  • Ruppert, D.: Almost sure approximations to the Robbins-Monro and Kiefer-Wolfowitz processes with dependent noise. Ann. Probab. 10 (1982), 178–187.

    Article  MathSciNet  MATH  Google Scholar 

  • Ruppert, D.: Efficient estimators from a slowly convergent Robbins-Monro process. Technical Report No. 781 (1988), School of Operations Research and Industrial Engineering, Cornell University Ithaca, New York.

    Google Scholar 

  • Sanchez-Palencia, E.: Méthode de centrage-estimation de l’erreur et comportement des trajectoires dans lľespace des phases, Int. J. Non-Linear Mechanics 11(176) (1976), 251–263.

    Article  MATH  Google Scholar 

  • Sanders, J.A., Verhulst, F.: Averaging Methods in Nonlinear Dynamical Systems. Springer; Berlin, Heidelberg, New York, 1985.

    MATH  Google Scholar 

  • Schmetterer, L.: Sur quelques résultats asymptotiques pour le processus de Robbins-Monro. Annales Scientifiques de lľUniversité de Clermont 58 (1976), 166–176.

    MathSciNet  Google Scholar 

  • Schwabe, R.: Strong representation of an adaptive stochastic approximation procedure. Stoch. Processes Appl. 23 (1986), 115–130.

    Article  MathSciNet  MATH  Google Scholar 

  • Shwartz, A., Berman, N.: Abstract stochastic approximations and applications. Stoch. Processes Appl. 31 (1989), 133–149.

    Article  MathSciNet  MATH  Google Scholar 

  • Venter, J.H.: An extension of the Robbins-Monro procedure. Ann. Math. Statist. 38 (1967), 181–190.

    Article  MathSciNet  MATH  Google Scholar 

  • Walk, H.: An invariance principle for the Robbins-Monro process in a Hilbert space. Z. Wahrscheinlichkeitstheorie verw. Gebiete 39 (1977), 135–150.

    Article  MathSciNet  MATH  Google Scholar 

  • Walk, H.: Stochastic iteration for a constrained optimization problem. Commun. Statist. — Sequential Analysis 2 (1983-84), 369–385.

    Google Scholar 

  • Walk, H.: Limit behaviour of stochastic approximation processes. Statistics & Decisions 6 (1988), 109–128.

    MathSciNet  MATH  Google Scholar 

  • Walk, H., Zsidó, L.: Convergence of the Robbins-Monro method for linear problems in a Banach space. J. Math. Anal. Appl. 139 (1989), 152–177.

    Article  MathSciNet  MATH  Google Scholar 

  • Wei, C.Z.: Multivariate adaptive stochastic approximation. Ann. Statist. 15 (1987), 1115–1130.

    Article  MathSciNet  MATH  Google Scholar 

  • Wertz, W.: Sequential and recursive estimators of the probability density. Statistics 16 (1985), 277–295.

    Article  MathSciNet  MATH  Google Scholar 

  • Widrow, B., Hoff, M.E., Jr.: Adaptive switching circuits. IRE WESCON Convention Record, part 4 (1960), 96–104.

    Google Scholar 

  • Wolverton, C.T., Wagner, T.J.: Recursive estimates of probability densities. IEEE Trans. Systems Sci. Cybernet. SSC-5 (1969), 246–247.

    Article  Google Scholar 

  • Woodroofe, M.: Normal approximation and large deviations for the Robbins-Monro process. Z. Wahrscheinlichkeitstheorie verw. Geb. 21 (1972), 329–338.

    Article  MathSciNet  MATH  Google Scholar 

  • Yamato, H.: Sequential estimation of a continuous probability density function and mode. Bull. Math. Statist. 14 (1971), 1–12.

    MathSciNet  MATH  Google Scholar 

  • Yosida, K.: Functional Analysis. 2nd ed. Springer; Berlin, Heidelberg, New York, 1968.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematisches Institut A, University of Stuttgart, Pfaffenwaldring 57, D-7000, Stuttgart 80, Germany

    Harro Walk

Authors
  1. Harro Walk
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 1992 Springer Basel AG

About this chapter

Cite this chapter

Walk, H. (1992). Foundations of stochastic approximation. In: Stochastic Approximation and Optimization of Random Systems. DMV Seminar, vol 17. Birkhäuser, Basel. https://doi.org/10.1007/978-3-0348-8609-3_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-0348-8609-3_1

  • Publisher Name: Birkhäuser, Basel

  • Print ISBN: 978-3-7643-2733-0

  • Online ISBN: 978-3-0348-8609-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation