Behavior of accelerated gradient methods near critical points of nonconvex functions

Abstract

We examine the behavior of accelerated gradient methods in smooth nonconvex unconstrained optimization, focusing in particular on their behavior near strict saddle points. Accelerated methods are iterative methods that typically step along a direction that is a linear combination of the previous step and the gradient of the function evaluated at a point at or near the current iterate. (The previous step encodes gradient information from earlier stages in the iterative process). We show by means of the stable manifold theorem that the heavy-ball method is unlikely to converge to strict saddle points, which are points at which the gradient of the objective is zero but the Hessian has at least one negative eigenvalue. We then examine the behavior of the heavy-ball method and other accelerated gradient methods in the vicinity of a strict saddle point of a nonconvex quadratic function, showing that both methods can diverge from this point more rapidly than the steepest-descent method.

A properties of the sequence \(\{t_k\}\) defined by (51)

In this “Appendix” we show that the following two properties hold for the sequence defined by (51):

$$\begin{aligned} \frac{t_{k-1}-1}{t_k} \;\; \text{ is } \text{ an } \text{ increasing } \text{ nonnegative } \text{ sequence } \end{aligned}$$

(52)

and

$$\begin{aligned} \lim _{k \rightarrow \infty } \frac{t_{k-1}-1}{t_k} = 1. \end{aligned}$$

(53)

We begin by noting two well known properties of the sequence $t_k$ (see for example [4, Section 3.7.2]):

$$\begin{aligned} t_k^2 - t_k = t_{k-1}^2 \end{aligned}$$

(54)

and

$$\begin{aligned} t_k \ge \frac{k+1}{2}. \end{aligned}$$

(55)

To prove that $\frac{t_{k-1}-1}{t_k}$ is monotonically increasing, we need

$$\begin{aligned} \frac{t_{k-1}-1}{t_k} = \frac{t_{k-1}}{t_k} - \frac{1}{t_k} \le \frac{t_k}{t_{k+1}} - \frac{1}{t_{k+1}} = \frac{t_k - 1}{t_{k+1}}, \quad k=1,2,\ldots . \end{aligned}$$

Since $t_{k+1} \ge t_k$ (which follows immediately from (51)), it is sufficient to prove that

$$\begin{aligned} \frac{t_{k-1}}{t_k} \le \frac{t_k}{t_{k+1}}. \end{aligned}$$

By manipulating this expression and using (54), we obtain the equivalent expression

$$\begin{aligned} t_{k-1} \le \frac{t_k^2}{t_{k+1}} = \frac{t_{k+1}^2 - t_{k+1}}{t_{k+1}} = t_{k+1} - 1. \end{aligned}$$

(56)

By definition of $t_{k+1}$, we have

$$\begin{aligned} t_{k+1} = \frac{\sqrt{4t_k^2 + 1} + 1}{2} \ge t_k + \frac{1}{2} = \frac{\sqrt{4t_{k-1}^2 + 1} + 1}{2} + \frac{1}{2} \ge t_{k-1} + 1. \end{aligned}$$

Thus (56) holds, so the claim (52) is proved. The sequence $\{ (t_{k-1}-1)/t_k \}$ is nonnegative, since $(t_0-1)/t_1 = 0$.

Now we prove (53). We can lower-bound $(t_{k-1}-1)/t_k$ as follows:

$$\begin{aligned} \frac{t_{k-1}-1}{t_k}&= \frac{2(t_{k-1} -1)}{\sqrt{4 t_{k-1}^2 +1} + 1}\ge \frac{2(t_{k-1} -1)}{\sqrt{4 t_{k-1}^2} + 2} \nonumber \\&= \frac{2(t_{k-1} -1)}{2 (t_{k-1} + 1)} = 1 - \frac{2}{t_{k-1}+1}. \end{aligned}$$

(57)

For an upper bound, we have from $t_k \ge t_{k-1}$ that

$$\begin{aligned} \frac{t_{k-1} -1}{t_k} \le \frac{t_{k-1}}{t_k} \le 1. \end{aligned}$$

(58)

Since $t_{k-1} \rightarrow \infty$ (because of (55)), it follows from (57) and (58) that (53) holds.