Abstract
The paper presents some results concerning the averaging approach in a “general” linear regression model in one dimension under suitable conditions about the martingale structure of errors. At first asymptotics of the primary and averaged estimators are discussed. Then it is shown that variances of estimators can be consistently estimated by appropriate integrated squared deviations functionals. Finally applications to the construction of confidence regions are considered.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ruppert D (1988) Efficient estimations from a slowly convergent Robbins-Monro process. Technical Report 781, Cornell Univ
Polyak BT (1990) New method of stochastic approximation type. Automate Remote Control 51:937–946
Yin G (1991) On extension of Polyak's averaging approach to stochastic approximation. Stochastics and Stochastic Reports 36:245–264
Polyak BT, Juditsky AB (1992) Acceleration of stochastic approximation by averaging. SIAM J Cont & Optim 30 (4):838–855
Delyon B, Juditsky AB (1992) Stochastic optimization with averaging of trajectories. Stochastics and Stochastic Reports 39:107–118
Juditsky AB (1992) On the recursive algorithm of estimation of autoregressive parameters. Automate Remote Control 7:118–127 (in russian)
Chen HF (1992) Continuous-time stochastic approximation: Convergence and asymptotic efficiency. (preprint)
Chen HF (1993) Asymptotically efficient stochastic approximation. Stochastics and Stochastic Reports 45:1–16
Le Breton A (1993) About the averaging approach in Gaussian schemes for stochastic approximation. Mathematical Methods of Statistics 2 (4):295–315
Le Breton A (1994) About Gaussian schemes in stochastic approximation. Stochastic Processes and their Applications 50:101–115
Le Breton A, Novikov AA (1993a) Averaging for estimating covariances in stochastic approximation. To appear in Mathematical Methods of Statistics
Le Breton A, Novikov AA (1993b) On stochastic approximation procedures with averaging. To appear in Proceedings of the Steklov Mathematical Institute.
Lai TL, Robbins H (1979) Adaptive design and stochastic approximation. Ann Stat 7:1196–1221
Lai TL, Robbins H (1981) Consistency and asymptotic efficiency of slope estimates in stochastic approximation schemes. Z Waht verw Gebiete 56:329–360
Lipcer RSH, Shiryaev AN (1989) Theory of martingales. Kluwer Academic Publishers, Dordrecht
Chow YS, Robbins H (1965) On the asymptotic theory of fixed-width confidence intervals for the mean. Ann Math Stat 36:457–462
Ghosh BK, Sen PK (1991) Handbook of sequential analysis. Marcel Dekker Inc, New York
Sielken R (1973) Stopping times for stochastic approximation procedures. Z Wahr verw Gebiete 26:67–75
Stroup DF, Braun HI (1982) On a new stopping time for stochastic approximation. Z Wahr verw Gebiete 60:535–554
Stroup DF, Braun HI (1984) Correction to “On a new stopping time for stochastic approximation”. Z Wahr verw Gebiete 67:237
Wei CZ (1985) Asymptotic properties of least squares estimates in stochastic regression models. Ann Stat 13:1498–1508
Pflug GCH (1985) Stepsize rules, stopping times and their implementation in stochastic quasigradient algorithms. In: Numerical Techniques for stochastic optimization Wets R Ermoliev Yu (Eds.). Springer Berlin-New York
Pflug GCH (1990) Non-asymptotic confidence bounds for stochastic approximation algorithms with constant step size. Monatshefte fur Math 110:297–314
Yin G (1988) A stopped stochastic approximation algorithm. Systems & Control Letters 11:107–115
Yin G (1990) A stopping rule for the Robbins-Monro method. J of Optim Theor and Appl 67: 151–173
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Le Breton, A., Novikov, A. Some results about averaging in stochastic approximation. Metrika 42, 153–171 (1995). https://doi.org/10.1007/BF01894297
Issue Date:
DOI: https://doi.org/10.1007/BF01894297