Abstract
To estimate the root ϑ of an unknown regression function f: ℝ → ℝ the iterative Robbins-Monro method X n+1 = X n − a/nY n with noisy observations Y n = f(X n ) + V n of f(X n ) can be used. It is well known that X n − ϑ can be approximated by a weighted sum of the observation errors V n . As recently shown this approximation can be improved by adding quadratic and cubic forms in the observation errors. This paper presents valid Edgeworth expansions of the distribution function of the approximating sequence up to a remainder term of order o(1/√n) or even o(1/n).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
O. E. Barndorff-Nielsen and D. R. Cox, Asymptotic Techniques for Use in Statistics (Chapman and Hall, London, 1989).
V. Bentkus, F. Götze, V. Paulauskas, and A. Račkauskas, The Accuracy of Gaussian Approximation in Banach Spaces, Technical report, (Universität Bielefeld, Bielefeld, 1990).
R. N. Bhattacharya and R. R. Rao, Normal Approximation and Asymptotic Expansions (Wiley, New York, 1986).
P. J. Bickel, F. Götze, and W. R. van Zwet, “The Edgeworth Expansion for U-Statistics of Degree Two”, Ann. Statist. 14, 1463–1484 (1986).
H. Callaert, P. Janssen, and N. Veraverbeke, “An Edgeworth Expansion for U-Statistics”, Ann. Statist. 8, 299–312 (1980).
J. Dippon, “Higher Order Representations of the Robbins-Monro Process”, J. Multivar. Anal. 90, 301–326 (2004).
J. Dippon, “Moments and Cumulants in Stochastic Approximation” (in press).
J. Dippon, “Asymptotic Expansions of the Robbins-Monro Process”, Math. Methods Statist. (in press).
P. Hall, The Bootstrap and Edgeworth Expansion (Springer, New York, 1992).
R. Helmers, Edgeworth Expansions for Linear Combinations of Order Statistics, 2nd ed. (Mathematical Centre 105, Amsterdam, 1984).
M. Loeve, Probability Theory, 3rd ed. (Van Nostrand, Princeton, 1963).
V. V. Petrov, Sums of Independent Random Variables (Springer, Berlin, 1975).
V. V. Petrov, Limit Theorems of Probability Theory (Clarendon Press, Oxford, 1995).
B. T. Polyak and A. B. Juditsky, “Acceleration of Stochastic Approximation by Averaging”, SIAM J. Control and Optimization 30, 838–855 (1992).
H. Robbins and S. Monro, “A Stochastic Approximation Method”, Ann. Math. Statist. 22, 400–407 (1951).
H. Walk, “An Invariance Principle for the Robbins-Monro Process in a Hilbert Space”, Z. Wahrsch. verw. Gebiete 39, 135–150 (1977).
B. Zhidong and Z. Lincheng, “Edgeworth Expansions of Distribution Functions of Independent Random Variables”, Scientia Sinica 29, 1–22 (1984).
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Dippon, J. Edgeworth expansions for stochastic approximation theory. Math. Meth. Stat. 17, 44–65 (2008). https://doi.org/10.3103/S1066530708010043
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3103/S1066530708010043