Abstract
This paper is concerned with the strong convergence of recursive estimators which are generalizations of the Robbins-Monro (1951) stochastic approximation procedure. The Robbins-Monro procedure and its generalizations have been investigated in mаnу contexts, for example recursive nonlinear regression (Albert and Gardner, 1967), recursive maximum likelihood estimation (Fabian, 1978), robust estimation of parameters for autoregressive process (Campbell, 1982), robust estimation of a location parameter (Martin and Masreliez, 1975 and Holst, 1980, 1982), control of physical processes (Comer, 1964 and Ruppert, 1979, 1981), and system identification (Kushner and Clark, 1978, section 2.6). In this paper, we present a rather general convergence theorem which allows the disturbances to be dependent and enter in a non-additive fashion. As examples, the theorem is applied to the nonlinear regression estimator of Albert and Gardner (1967), and to the author’s (Ruppert 1979, 1981) Robbins-Monro type procedures for use where the root of the unknown regression function varies with time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Albert, A.E. and Gardner, L.A., Jr. (1967). Stochastic Approximation and Nonlinear Regression. The M.I.T. Press, Cambridge, Mass.
Campbell, Katherine. (1982). Recursive computation of M-estimates for the parameters of a finite autoregressive process. Ann. Statist., 10. 442–453.
Comer, John P., Jr. (1964). Some stochastic approximation procedures for use in process control. Ann. Math. Statist.,35 1137–1146.
Derman, C. and Sacks, J. (1959). On Dvoretzky’s stochastic approximation theorem. Ann. Math. Statist., 30 601–605.
Fabian, V. (1978). On asymptotically efficient recursive estimation. Ann. Statist.,6 854–867.
Holst, Ulla (1980). Convergence of a recursive stochastic algorithm with m-dependent observations. Scand. J. Statist. 7 207–215.
Holst, Ulla (1982). Convergence of a recursive stochastic algorithm with strongly regular observations. Technical Report. Department of Mathematical Statistics. University of Lund and Lund Institute of Technology.
Kushner, H.J. and Clark, D.S. (1978). Stochastic Approximation Methods for Constrained and Unconstrained Systems. Springer-Verlag. New York. 260 p.
Martin, R.D. and Masreliez, C.J. (1975). Robust estimation via stochastic approximation. IEEE Trans. Inform. Theory IT-21 263–271.
McLeish, D.L. (1975). A maximal inequality and dependent strong laws. Ann. Prob. 3 829–839.
Parthasarathy, K.R. (1967). Probability Measures on Metric Spaces. Academic Press, New York.
Pham, Ruan D. and Tran, Lanh T. (1980). The strong mixing property of the autoregressive moving average time series model. Technical Report, Department of Mathematics, Indiana University.
Ranga, R.R. (1962). Relations between weak and uniform convergence of measures with applications. Ann. Math. Statist. 33 659–680.
Robbins, H. and Monro, S, (1951). A stochastic approximation method. Ann. Math. Statist. 22 400–407.
Robbins, H. and Siegmund, D. (1971). A Convergence Theorem for Nonnegative Almost Supermartingales and Some Applications. In Optimizing Methods in Statistics (J.S. Rustagi, ed.) 233–257. Academic Press, New York.
Ruppert, D. (1979). A new dynamic stochastic approximation procedure. Ann. Statist. 7 1179–1195.
Ruppert, D. (1981). Stochastic approximation of an implicitly defined function. Ann. Statist. 9 555–566.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1983 Springer-Verlag New York Inc.
About this paper
Cite this paper
Ruppert, D. (1983). Convergence of Stochastic Approximation Algorithms with Non-Additive Dependent Disturbances and Applications. In: Herkenrath, U., Kalin, D., Vogel, W. (eds) Mathematical Learning Models — Theory and Algorithms. Lecture Notes in Statistics, vol 20. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-5612-0_20
Download citation
DOI: https://doi.org/10.1007/978-1-4612-5612-0_20
Publisher Name: Springer, New York, NY
Print ISBN: 978-0-387-90913-4
Online ISBN: 978-1-4612-5612-0
eBook Packages: Springer Book Archive