Summary
The paper studies the pathwise asymptotic behaviour of stochastic algorithms of the following general form
the hypotheses allowing discontinuities on the adaptation termf. The process (Y n) n≧0 is a Markov chain “controlled by (θ n )”. For each θ fixed the Markov chain (Y θn ) n≧0 is essentially of positive recurrent type.
A first theorem generalizes to this situation a Ljung's theorem (L. Ljung, IEEE Trans. AC 22, 2, p. 551–575 (1977)).
An almost sure convergence theorem is proved under the existence of a global Lyapunov function for the associated deterministic differential equation.
whereh(θ) = ∫f(θ, y) Γ θ (d, y) and Γ θ is the invariant probability of the Markov chain (Y θn ) n≧0.
Références
Benveniste, A., Goursat, G., Ruget, G.: Analysis of stochastic approximation schemes with discontinuous dependent forcing terms. IEEE Trans. Autom. Control Vol. AC-25, 6 (1980)
Eweda, E., Macchi, O.: Convergence analysis of self adaptive equalizers. IEEE Trans. Information Theory. Vol. IT-30, 161–176 (1984)
Krasovski, N.N.: Stability of motion. Stanford: University Press (1963)
Kushner, H.: Stochastic approximation with discontinuous dynamics and state dependent noise: W.p 1 and weak convergence. J. Math. Anal. Appl.82, 527–542 (1981)
Kushner, H., Shwartz, A.: An invariant measure approach to the convergence of stochastic approximations with state dependent noise. SIAM J. Control Optimization22, 13–27 (1984)
Kushner, H., Clark, D.S.: Stochastic approximation for constrained and unconstrained systems. Appl. Math. Sci. Vol. 26. Berlin Heidelberg New York: Springer 1978
Ljung, L.: Analysis of recursive stochastic algorithms. IEEE Trans. Autom. Control Vol. AC-22, 551–575 (1977)
Ljung, L.: Analysis of stochastic gradient algorithms for linear regression problems. IEEE Trans. Inf. Theory. Vol. IT-30, 151–160 (1984)
Metivier, M., Priouret, P.: Applications of a Kushner and Clark lemma to general classes of stochastic algorithms. IEEE Trans. Inf. Theory. Vol. IT-30, 140–150 (1984)
Metivier, M., Priouret, P.: Convergence avec probabilité (1-ε) d'algorithmes stochastiques. Application au cas de l'égaliseur aveugle. Ann. Télécommunt. 41,56, 269–279 (1986)
Sunyach, C.: Une classe de chaînes récurrentes sur un espace métrique complet. Ann. Inst. H. Poincaré. Vol.HI, 325–343 (1975)
Freidlin, M.I., Wentzel, A.D.: Random perturbations of dynamical systems. Berlin Heidelberg New York: Springer 1984
Heyde, C.C., Hall, D.: Martingale limit theory and its applications. New York: Academic Press 1980
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Métivier, M., Priouret, P. Théorèmes de convergence presque sure pour une classe d'algorithmes stochastiques à pas décroissant. Probab. Th. Rel. Fields 74, 403–428 (1987). https://doi.org/10.1007/BF00699098
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF00699098