Summary
Robbins-Monro stochastic approximation procedure \(x_{n + 1} = x_n - \frac{1}{{n + 1}}(A_{n + 1} x_n - y_{n + 1} )\) is used to solve the linear equation Ax=y in Hilbert space, where y n and A n are estimators such that their arithmetic means converge to y and A, respectively. Under some additional conditions it is shown that X n goes to the unique solution of this equation.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Ash, R.B.: Real Analysis and Probability. New York: Academic Press 1972
Csibi, S.: Statistical learning processes. Preprint. Technical University of Budapest 1973
Csibi, S.: Learning under computational constraints from weakly dependent samples. Problems of Control and Information Theory. 4, 3–21 (1975)
Fritz, J.: Learning from an ergodic training sequence. In Limit Theorems of Probability Theory; ed. P. Révész. Amsterdam: North-Holland 79–91 (1974)
Ljung, L.: Strong convergence of a stochastic approximation algorithm. Ann. Statist. 6, 680–696 (1978)
Révész, P.: Robbins-Monro procedure in a Hilbert space and its application in the theory of learning processes I. Studia Sci. Math. Hungar. 8, 391–398 (1973)
Saridis, G.N., Nikolic, Z.J., Fu, K.S.: Stochastic approximation algorithms for system indentification, estimation, and decomposition of mixtures. IEEE Trans. Systems Science and Cybernetics 5, 8–15 (1969)
Tsypkin, Y.A.: Foundations of the Theory of Learning Systems. In Russian. Moscow: Nauka 1970
Venter, J.H.: On Dvoretzky stochastic approximation theorems. Ann. Math. Statist. 37, 1534–1544 (1966)
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Györfi, L. Stochastic approximation from ergodic sample for linear regression. Z. Wahrscheinlichkeitstheorie verw Gebiete 54, 47–55 (1980). https://doi.org/10.1007/BF00535352
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00535352