Abstract
Based on the idea of averaging a new stochastic approximation algorithm has been proposed by Bather (1989), which shows a preferable performance for small to moderate sample sizes. In the present paper an almost sure representation is established for this procedure, which gives the optimal rate of convergence with minimal asymptotic variance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bather JA (1989) Stochastic approximation: A generalisation of the Robbins-Monro procedure. In: Mandl P, Hušková M (eds) Proc Fourth Prague Symp Asymptotic Statistics Charles Univ Prague August 29–September 2 1988. Charles Univ Prague 13–27
Blum JR (1954) Approximation methods which converge with probability one. Ann Math Statist 25:382–386
Chung KL (1954) On a stochastic approximation method. Ann Math Statist 25:463–483
Derman C, Sacks J (1959) On Dvoretzky’s stochastic approximation theorem. Ann Math. Statist 30:601–606
Fabian V (1968) On asymptotic normality in stochastic approximation. Ann Math Statist 39:1327–1332
Kersting G (1977) Almost sure approximation of the Robbins-Monro process by sums of independent random variables. Ann Probab 5:954–965
Ljung L (1978) Strong convergence of a stochastic approximation algorithm. Ann Statist 6:680–696
Loève M (1977) Probability theory I. 4th ed. Springer New York
Petrov VV (1975) Sums of independent random variables. Springer New York
Philipp W, Stout W (1975) Almost sure invariance principles for partial sums of weakly dependent random variables. Mem Am Math Soc 161
Polyak BT (1990) New method of stochastic approximation type. Autom Remote Control 51:937–946
Robbins H, Monro S (1951) A stochastic approximation method. Ann Math Statist 22:400–407
Ruppert D (1982) Almost sure approximations to the Robbins-Monro and Kiefer-Wolfowitz processes with dependent noise. Ann Probab 10:178–187
Ruppert D (1988) Efficient estimators from a slowly convergent Robbins-Monro process. Technical Report 781, School of Operations Research and Industrial Engineering, Cornell Univ Ithaca
Ruppert D (1991) Stochastic approximation. In: Ghosh BK, Sen PK (eds) Handbook of Sequential Analysis. Marcel Dekker New York 503–529
Sacks J (1958) Asymptotic distribution of stochastic approximation procedures. Ann Math Statist 29:373–405
Schwabe R (1986) Strong representation of an adaptive stochastic approximation procedure. Stochastic Process Appl 23:115–130
Schwabe R (1993) Stability results for smoothed stochastic approximation procedures. Z angew Math Mech 73:639–643
Schwabe R (1994) On Bather’s stochastic approximation procedure. Kybernetika 30:301–306
Venter JH (1967) An extension of the Robbins-Monro procedure. Ann Math Statist 38:181–190
Walk H (1992) Foundations of stochastic approximation. In: Ljung L, Pflug G, Walk H Stochastic approximation and optimization of random systems. DMV Seminar Blaubeuren, May 28–June 4, 1989 DMV Seminar Vol 17, Birkhäuser Basel 1–51
Yin G, Yin K (1994) Asymptotically optimal rate of convergence of smoothed stochastic recursive algorithms. Stochastics Stochastics Rep 47:21–46
Author information
Authors and Affiliations
Additional information
Work partly supported by the research grant Ku719/2-1 of the Deutsche Forschungsgemeinschaft
Rights and permissions
About this article
Cite this article
Schwabe, R., Walk, H. On a stochastic approximation procedure based on averaging. Metrika 44, 165–180 (1996). https://doi.org/10.1007/BF02614063
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF02614063