Computing Second-Order Points Under Equality Constraints: Revisiting Fletcher’s Augmented Lagrangian

Abstract

We address the problem of minimizing a smooth function under smooth equality constraints. Under regularity assumptions on these constraints, we propose a notion of approximate first- and second-order critical point which relies on the geometric formalism of Riemannian optimization. Using a smooth exact penalty function known as Fletcher’s augmented Lagrangian, we propose an algorithm to minimize the penalized cost function which reaches $\varepsilon$-approximate second-order critical points of the original optimization problem in at most ${\mathcal {O}}(\varepsilon ^{-3})$ iterations. This improves on current best theoretical bounds. Along the way, we show new properties of Fletcher’s augmented Lagrangian, which may be of independent interest.

Proof of Proposition 2.8

Proof

Define $\varphi (x) = \dfrac{1}{2}\left\| h(x)\right\| ^2$ and take any $x_0 \in {\mathcal {C}}= \lbrace x\in {\mathcal {E}}: \varphi (x) \le R^2/2\rbrace$. Consider the following differential system:

$$\begin{aligned} \left\{ \begin{aligned} \dfrac{\textrm{d}}{\textrm{d} t}x(t)&= - \nabla \varphi (x(t)) \\ x(0)&= x_0. \end{aligned} \right. \end{aligned}$$

(A.1)

The fundamental theorem of flows [28, Theorem A.42] guarantees the existence of a unique maximal integral curve starting at $x_0$ for (A.1). Let $z( \cdot ):I \rightarrow {\mathcal {E}}$ denote this maximal integral curve and $T>0$ be the supremum of the interval I on which $z(\cdot )$ is defined. We rely on the Escape Lemma [28, Lemma A.43] to show that z(t) is defined for all times $t\ge 0$. For $t< T$, we write $\ell = \varphi \circ z$ and find

$$\begin{aligned} \ell '(t)&= \textrm{D}\varphi (z(t))\left[ \frac{\textrm{d}}{\textrm{d}t}z(t)\right] = \left\langle {\nabla \varphi (z(t))},{\frac{\textrm{d}}{\textrm{d}t}z(t)}\right\rangle \end{aligned}$$

(A.2)

$$\begin{aligned}&= - \left\| \nabla \varphi (z(t))\right\| ^2 \end{aligned}$$

(A.3)

$$\begin{aligned}&= - \left\| \textrm{D}h(z(t))^*[h(z(t))]\right\| ^2 \le 0. \end{aligned}$$

(A.4)

This implies that $z(t) \in {\mathcal {C}}$ for all $0\le t < T$. We show that the trajectory z(t) has finite length. To that end, we note that

$$\begin{aligned} \dfrac{1}{2}\left\| \nabla \varphi (x)\right\| ^2 = \dfrac{1}{2} \left\| \textrm{D}h(x)^* [h(x)] \right\| ^2 \ge {\underline{\sigma }}^2 \dfrac{1}{2} \left\| h(x)\right\| ^2 = {\underline{\sigma }}^2 \varphi (x), \end{aligned}$$

(A.5)

for all $x\in {\mathcal {C}}$. The length of the trajectory from time $t=0$ to $t=T$ is bounded as follows, using a classical argument [30]:

$$\begin{aligned} \int _0^T \left\| \frac{\textrm{d}}{\textrm{d}t}z(t)\right\| \textrm{d} t&= \int _0^T \left\| - \nabla \varphi (z(t))\right\| \textrm{d} t \nonumber \\&= \int _0^T \dfrac{ \left\| \nabla \varphi (z(t))\right\| ^2}{ \left\| \nabla \varphi (z(t))\right\| } \textrm{d} t \nonumber \\&= \int _0^T \dfrac{ \left\langle {- \nabla \varphi (z(t))},{\frac{\textrm{d}}{\textrm{d}t}z(t)}\right\rangle }{ \left\| \nabla \varphi (z(t))\right\| } \textrm{d} t\nonumber \\&= \int _0^T \dfrac{ - (\varphi \circ z)'(t)}{ \left\| \nabla \varphi (z(t))\right\| } \textrm{d} t\nonumber \\&\le \int _0^T \dfrac{ - (\varphi \circ z)'(t)}{{\underline{\sigma }}\sqrt{2(\varphi \circ z)(t)}} \textrm{d} t\nonumber \\&= \dfrac{-\sqrt{2}}{{\underline{\sigma }}} \left[ \sqrt{\varphi (z(T))} - \sqrt{\varphi (z(0))}\right] \nonumber \\&\le \dfrac{\sqrt{2 \varphi (z(0))}}{{\underline{\sigma }}}. \end{aligned}$$

(A.6)

The length is bounded independently of T and therefore the flow has finite length. The Escape Lemma states that for a maximum integral curve $z(\cdot ) :I \rightarrow {\mathcal {E}}$, if I has a finite upper bound, then the curve $z(\cdot )$ must be unbounded. Since $z(\cdot )$ is contained in a compact set by (A.6), the converse ensures that the interval I does not have a finite upper bound and therefore, $I={\mathbb {R}}_+$. Since the trajectory z(t) is bounded for $t\ge 0$, it must have an accumulation point ${\bar{z}}$. From A1, we have $\sigma _\textrm{min}(\textrm{D}h(z(t)) \ge {\underline{\sigma }}>0$ for all $t \ge 0$. This gives the bound $\ell '(t) \le - {\underline{\sigma }}^2 \left\| h(z(t))\right\| ^2 = -2{\underline{\sigma }}^2 \ell (t)$. Gronwall’s inequality then yields

$$\begin{aligned} \ell (t) \le \varphi (x_0) e^{-2{\underline{\sigma }}^2 t}. \end{aligned}$$

(A.7)

Therefore $\ell (t) \rightarrow 0$ as $t \rightarrow \infty$, which implies $h(z(t))\rightarrow 0$ as $t\rightarrow \infty$. We conclude that the accumulation point satisfies $h({\bar{z}}) = 0$. Since ${\mathcal {C}}$ is closed, the point ${{\bar{z}}}$ is in ${\mathcal {C}}$. Therefore, ${{\bar{z}}}$ is both in ${\mathcal {M}}$ and in the connected component of ${\mathcal {C}}$ that contains $z(0) = x_0$. $\square$