Abstract
The stochastic gradient method has a rather long history. The method foundations were given by (Robbins and Monro, Annals of Mathematical Statistics, 22:400-407, (1951) [129]) on the one hand, and by (Kiefer and Wolfowitz, Annals of Mathematical Statistics, 23:462-466, (1952) [93]) on the other. Later on, (Polyak, Automation and Remote Control, 37(12):1858-1868, (1976) [120], Polyak and Tsypkin, Automation and Remote Control, 40(3):378-389, (1979) [123]) gave results about the convergence rate. Based on this work, (Dodu et al., Bulletin de la Direction des Études et Recherches EDF, (1981) [57]) studied the optimality of the stochastic gradient algorithm, that is, the asymptotic efficiency of the associated estimator. An important contribution by (Polyak, Automation and Remote Control, 51(7):937-946, (1990) [121], Polyak and Juditsky, SIAM Journal on Control and Optimization, 30(4):838-855, (1992) [122]) has been to combine stochastic gradient method and averaging techniques in order to reach the optimal efficiency.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Recall that a \(k\)-sample of
is a sequence 
of independent random variables with the same probability distribution as 
See Sect. B.7.2 for further details. - 2.
The positive sign in front of \(\epsilon ^{(k)}\) in the update formula (2.11) is explained later on.
- 3.
The symbol \({\mathrm {O}}\) corresponds to the “Big-O”O@Big-O notation notation: \(f(x)=\mathrm {O}\big (g(x)\big )\) as \(x\rightarrow x_{0}\) if and only if there exist a positive constant \(\alpha \) and a neighborhood \(V\) of \(x_{0}\) such that \(\left| f(x)\right| \le \alpha \left| g(x)\right| \), \(\forall x \in V\).
- 4.
See Sect. B.3.4 for this convergence notion and for the associated notation \(\mathop {\longrightarrow }\limits ^{\mathcal {D}}\).
- 5.
See [39] for a reference about the Auxiliary Problem Principle.
- 6.
See Definition 8.22. This implies that
is measurable \(\forall u \in U^{\mathrm {ad}}\). - 7.
Note that the semi-continuity of \(j(\cdot ,w)\) stems from the fact that \(j\) is a normal integrand.
- 8.
See (A.5) for the meaning of this term.
- 9.
In fact a better covariance matrix.
- 10.
Consistency in the terminology of Statistics.
- 11.
See [62, Sect. 4] for further details.
- 12.
From Assumption 2.9-5, the condition \(\alpha > 1/(2c)\) is required, \(c\) being the strong convexity modulus of \(J\). It is easy to produce a simple problem with extremely slow convergence in the case when this condition is not satisfied. For example, with \(j(u,w)=(1/2)u^{2}\) (deterministic cost function such that \(c=1\)), with \(\epsilon ^{(k)}=1/(5k)\) and starting from \(u^{(0)}=1\), the solution obtained after one billion iterations is about \(0.015\), hence relatively far from the optimal solution \(u^{\sharp }= 0\) (see [108] for details).
- 13.
There however exist extensions of the stochastic gradient method to closed-loop optimization problem: see [14] for further details.
- 14.
A random variable
is finite if 
. - 15.
A subset of \(\mathbb {U}\) is compact if it is closed and bounded, provided that \(\mathbb {U}\) is a finite-dimensional space. If \(\mathbb {U}\) is an infinite-dimensional Hilbert space, such a property remains true only in the weak topology, and the lower semi-continuity property of \(J\) is preserved in that topology because \(J\) is convex. See [64] for further details.
is a sequence 
of independent random variables with the same probability distribution as 
See Sect. B.7.2 for further details.
is measurable 
is finite if 
.Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Carpentier, P., Chancelier, JP., Cohen, G., De Lara, M. (2015). Open-Loop Control: The Stochastic Gradient Method. In: Stochastic Multi-Stage Optimization. Probability Theory and Stochastic Modelling, vol 75. Springer, Cham. https://doi.org/10.1007/978-3-319-18138-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-18138-7_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18137-0
Online ISBN: 978-3-319-18138-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)