Summary
Let G be a bounded convex set, and Π G the projection onto G, and \(\{ \xi _j \} \) a bounded random process. Projected algorithms of the types \(X_{n + 1}^\varepsilon = \Pi _G (X_n^\varepsilon + \varepsilon b(X_n^\varepsilon ,\xi _n ))({\text{or }}X_{n + 1} = \Pi _G (X_n + a_n b(X_n ,\xi _n ))\), where 0<a n→0, Σ a n =∞) occur frequently in applications (among other places) in control and communications theory. The asymptotic convergence properties of {X ε n } as ε→0, εn→∞, have been well analyzed in the literature. Here, we use large deviations methods to get a more thorough understanding of the global behavior. Let Θ be a stable point of the algorithm in the sense that X ε n →Θ in distribution as ε→0, nε→∞. For the unconstrained case, rate of convergence results involve showing asymptotic normality of \(\{ (X_n^\varepsilon - {{\Theta )} \mathord{\left/ {\vphantom {{\Theta )} {\sqrt \varepsilon }}} \right. \kern-\nulldelimiterspace} {\sqrt \varepsilon }}\} \), and use linearizations about Θ. In the constrained case Θ is often on ∂G, and such methods are inapplicable. But the large deviations method yields an alternative which is often more useful in the applications. The action functionals are derived and their properties (lower semicontinuity, etc.) are obtained. The statistics (mean value, etc.) of the escape times from a neighborhood of Θ are obtained, and the global behavior on the infinite interval is described.
References
Kushner, H.J., Clark, D.S.: Stochastic approximation methods for constrained and unconstrained systems. Berlin Heidelberg New York: Springer 1978
Kushner, H.J., Shwartz, A.: An invariant measure approach to the convergence of stochastic approximations with state-dependent noise. SIAM J. on Control and Optim. 22, 13–27 (1984)
Ermoliov, Yu.: Methods of stochastic programming (in Russian). Moscow: Nauka 1976
Kushner, H.J., Schwarz, A.: Stochastic approximation in Hilbert space: Identification and optimization of linear continuous parameter systems. To appear in SIAM J. on Control and Optimization
Pflug, G.: Stochastic minimization with constant step-sizes. To appear in SIAM J. on Control and Optimization
Freidlin, M.I.: The averaging principle and theorems on large deviations. Russian Math. Surveys, 33, 117–176 (1978)
Kushner, H.J.: Robustness and approximation of escape times and large deviations estimates for systems with small noise effects. SIAM J. Appl. Math. 44, 160–182 (1984)
Dupuis, P., Kushner, H.J.: Stochastic approximation via large deviations: Asymptotic properties. To appear in SIAM J. on Control and Optimization September 1985
Varadhan, S.R.S.: Large deviations and applications. CBMS-NSF Regional Conference Series, SIAM, Philadelphia 1984
Freidlin, M.I., Ventsell, A.D.: Random perturbations of dynamical systems. Berlin Heidelberg New York: Springer 1984
Parthasarathy, T.: Selection theorems and their applications. Lect. Notes Math. 263. Berlin Heidelberg New York: Springer 1972
Kushner, H.J.: Approximation and weak convergence methods for random processes; with applications to stochastic systems theory. MIT Press, Cambridge, Mass. 1984
Korostelev, A.P.: Stochastic recurrent processes. Moscow: Nauka 1984
Author information
Authors and Affiliations
Additional information
Research has been supported in part by the US Army Research Office under Contract #DAAG 29-84-K-0082, and in part by the Office of Naval Research under Contract #N00014-83-K0542
Research has been supported in part by the National Science Foundation Grant #ECS 82-11476, and the Air Force Office of Scientific Research under Contract #AF-AFOSR 81-0116
Rights and permissions
About this article
Cite this article
Dupuis, P., Kushner, H.J. Asymptotic behavior of constrained stochastic approximations via the theory of large deviations. Probab. Th. Rel. Fields 75, 223–244 (1987). https://doi.org/10.1007/BF00354035
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/BF00354035
Keywords
- Asymptotic Behavior
- Statistical Theory
- Random Process
- Action Functional
- Convergence Property