On the geometric analysis of a quartic–quadratic optimization problem under a spherical constraint

Abstract

This paper considers the problem of solving a special quartic–quadratic optimization problem with a single sphere constraint, namely, finding a global and local minimizer of $\frac{1}{2}\mathbf {z}^{*}A\mathbf {z}+\frac{\beta }{2}\sum _{k=1}^{n}|z_{k}|^{4}$ such that $\Vert \mathbf {z}\Vert _{2}=1$. This problem spans multiple domains including quantum mechanics and chemistry sciences and we investigate the geometric properties of this optimization problem. Fourth-order optimality conditions are derived for characterizing local and global minima. When the matrix in the quadratic term is diagonal, the problem has no spurious local minima and global solutions can be represented explicitly and calculated in $O(n\log {n})$ operations. When A is a rank one matrix, the global minima of the problem are unique under certain phase shift schemes. The strict-saddle property, which can imply polynomial time convergence of second-order-type algorithms, is established when the coefficient $\beta$ of the quartic term is either at least $O(n^{3/2})$ or not larger than O(1). Finally, the Kurdyka–Łojasiewicz exponent of quartic–quadratic problem is estimated and it is shown that the largest exponent is at least 1/4 for a broad class of stationary points.

A Proof of Theorem 6.2

Proof

As in the proof of Theorem 6.1, the verification of Theorem 6.2 is mainly based on proper decompositions of $\Delta = \mathbf{y}-\mathbf{z}$ and $\mathrm{diag}(\tau )\mathbf{y}$ that allow to derive appropriate bounds of $|f(\mathbf{y}) - f(\mathbf{z})|$ and $\Vert \mathrm {grad\!\;}{f(\mathbf{y})}\Vert$ if $\mathbf{y}$ is close to $\mathbf{z}$ or if, equivalently, $\Delta$ is sufficiently small. In particular, we will discuss three cases that will allow to simplify and estimate the expressions for $|f(\mathbf{y}) - f(\mathbf{z})|$ and $\Vert \mathrm {grad\!\;}{f(\mathbf{y})}\Vert$ via identifying the different leading terms.

Without loss of generality we assume $\beta =1$. Let $\mathbf{y} \in \mathbb {S}^{n-1}$ be arbitrary and let us set $\Delta = \mathbf{y} - \mathbf{z}$, and

$$\begin{aligned}&\gamma _{1} = \sum _{k\in \mathcal {I}}z_{k}^{3}\Delta _{k},\quad \gamma _{2} = \sum _{k\in \mathcal {I}} z_{k}^2\Delta _{k}^2, \quad \gamma _{3}=\sum _{k\in \mathcal {I}} z_{k}\Delta _{k}^{3},\quad \gamma _{4}= \Vert \Delta \Vert _4^4. \end{aligned}$$

(A.1)

Based on the representation $\mathrm {grad\!\;}{f(\mathbf{y})} = P_\mathbf{y}^\bot [H + 2 \mathrm{diag}(\tau )]\mathbf{y}$, we now introduce the following decompositions

$$\begin{aligned}&2\mathrm{diag}(\tau )\mathbf{y} = \mathbf{w} + c_{1} \mathbf{y}, \quad \mathbf{w} = 2 P_\mathbf{y}^\bot \mathrm{diag}(\tau )\mathbf{y} , \quad c_{1} = 2 \mathbf{y}^T \mathrm{diag}(\tau )\mathbf{y} \nonumber \\&{\Delta } = \mathbf{u} + \mathbf{v}, \quad H\mathbf{u} = 0, \quad \mathbf{u}^T \mathbf{v} = 0, \quad \Vert H\mathbf{v}\Vert \ge \sigma _-(H)\Vert \mathbf{v}\Vert , \end{aligned}$$

(A.2)

where $\sigma _-(H)$ denotes the smallest positive singular value of H. Using this decomposition, (6.5), and $H\mathbf{y} = H\Delta$, we can express the norm of the Riemannian gradient as follows

$$\begin{aligned} \Vert \mathrm {grad\!\;}{f(\mathbf{y})}\Vert ^2&= \Vert P_\mathbf{y}^\bot [H + 2 \mathrm{diag}(\tau )]\mathbf{y}\Vert ^{2} = \Vert P_\mathbf{y}^\bot [H\mathbf{y} + \mathbf{w} + c_1\mathbf{y}]\Vert ^{2}\nonumber \\&= \Vert P_\mathbf{y}^\bot [H\Delta + \mathbf{w}]\Vert ^{2} = \Vert H\Delta + \mathbf{w}\Vert ^{2} - (\mathbf{y}^{T}H\Delta + \mathbf{y}^{T}{} \mathbf{w})^{2}\nonumber \\&= \Vert H{\Delta } + \mathbf{w}\Vert ^2 - ({\Delta }^T H{\Delta })^2 = \Vert H\mathbf{v} + \mathbf{w}\Vert ^2 - (\mathbf{v}^T H\mathbf{v})^2. \end{aligned}$$

(A.3)

Let $\lambda _+(H)$ be the largest eigenvalue of H. Then, by definition of $\mathbf{v}$, we obtain

$$\begin{aligned} |\mathbf{v}^{T}H\mathbf{v}|&\le \lambda _+(H) \Vert \mathbf{v}\Vert ^2 \le \lambda _+(H){\sigma _{-}(H)}^{-2} \Vert H\mathbf{v}\Vert ^2 =: {\bar{\sigma }}^{-1} \cdot \mathbf{v}^{T} H^{2} \mathbf{v}. \end{aligned}$$

(A.4)

Moreover, Lemma 6.1 yields

$$\begin{aligned} \Vert \tau \Vert ^\frac{3}{2} \le \eta _1 \Vert \mathbf{w}\Vert \end{aligned}$$

(A.5)

for some constant $\eta _{1}>0$ and for all $\mathbf{y} \in \mathbb {S}^{n-1}$ sufficiently close to $\mathbf{z}$. Throughout the proof, we will also repeatedly use the following facts:

$$\begin{aligned} 2 \mathbf{y}^{T}\Delta&= \Vert \mathbf{y}\Vert ^{2}+\Vert \Delta \Vert ^{2}-\Vert \mathbf{y}-\Delta \Vert ^{2} = \Vert \Delta \Vert ^{2},\nonumber \\ \mathbf{z}^{T}\Delta&=\mathbf{y}^{T}\Delta - \Vert \Delta \Vert ^{2}=-\frac{1}{2}\Vert \Delta \Vert ^{2}. \end{aligned}$$

(A.6)

Let $\epsilon \in (0,1]$ be an arbitrary, small positive constant. We now discuss three different cases.

Case 1. $\Vert \mathbf {w}\Vert \ge (1+\epsilon ) \Vert H\mathbf{v}\Vert$ or $\Vert \mathbf {w}\Vert \le (1-\epsilon )\Vert H\mathbf{v}\Vert$. We first assume $\mathbf{w} \ne 0$ and $\epsilon < 1$. Since the function $x \mapsto \varrho (x) := x + 1/x$ is monotonically decreasing on the interval $(0,1-\epsilon ]$ and monotonically increasing on $[1+\epsilon ,\infty )$, it follows

$$\begin{aligned} \varrho (x) \ge 1-\epsilon + \frac{1}{1-\epsilon } \quad x \in (0,1-\epsilon ] \quad \text {and} \quad \varrho (x) \ge 1+\epsilon + \frac{1}{1+\epsilon } \quad x \ge 1+\epsilon \end{aligned}$$

Thus, we obtain

$$\begin{aligned} \frac{\Vert \mathbf{w}\Vert }{\Vert H\mathbf{v}\Vert }+\frac{\Vert H\mathbf{v}\Vert }{\Vert \mathbf{w}\Vert } \ge \min \left\{ 1-\epsilon + \frac{1}{1-\epsilon },1+\epsilon + \frac{1}{1+\epsilon }\right\} = \frac{(1+\epsilon )^2+1}{1+\epsilon }, \end{aligned}$$

which further implies

$$\begin{aligned} \Vert H\mathbf{v} + \mathbf{w}\Vert ^{2}&\ge \Vert \mathbf{w}\Vert ^{2}+\Vert H\mathbf{v}\Vert ^{2} - 2\Vert \mathbf{w}\Vert \Vert H\mathbf{v}\Vert \\&= \left[ 1 - 2\left( \frac{\Vert \mathbf{w}\Vert }{\Vert H\mathbf{v}\Vert }+\frac{\Vert H\mathbf{v}\Vert }{\Vert \mathbf{w}\Vert }\right) ^{-1}\right] \cdot \left( \Vert \mathbf{w}\Vert ^{2}+\Vert H\mathbf{v}\Vert ^{2}\right) \\&\ge \frac{\epsilon ^{2}}{(1+\epsilon )^2+1} \left( \Vert \mathbf{w}\Vert ^{2}+\Vert H\mathbf{v}\Vert ^{2} \right) \ge \frac{\epsilon ^{2}}{5} \left( \Vert \mathbf{w}\Vert ^{2}+\Vert H\mathbf{v}\Vert ^{2} \right) . \end{aligned}$$

Next, we choose $\delta _\mathbf{z} > 0$ sufficiently small such that $\Vert \mathbf{v}\Vert ^2 \le \epsilon ^2 {\bar{\sigma }} / (10\lambda _+(H))$ and $|\mathbf{v}^TH\mathbf{v}| \le 1$. (This is possible due to the decomposition (A.2) and $\Vert \mathbf{v}\Vert \le \Vert \Delta \Vert$). Using (A.4), (A.5), the estimate $[\frac{1}{2} |a + b|]^{3/2} \le [|a|^{3/2} + |b|^{3/2}] / \sqrt{2}$, $a,b \in \mathbb {R}$, and setting ${\tilde{\eta }}_1 := \min \{\eta _1^{-2},\frac{{\bar{\sigma }}}{2}\}$, it follows

$$\begin{aligned} \Vert \mathrm {grad\!\;}{f(\mathbf {y})}\Vert ^{2}&\ge \frac{\epsilon ^{2}}{5}(\Vert \mathbf {w}\Vert ^{2}+\Vert H\mathbf{v}\Vert ^{2})-\lambda _{+}(H)\Vert \mathbf{v}\Vert ^{2} |\mathbf{v}^T H\mathbf{v}|\\&\ge \frac{\epsilon ^{2}}{10}(2\Vert \mathbf {w}\Vert ^{2}+2\Vert H\mathbf{v}\Vert ^{2})-\lambda _{+}(H) \cdot \epsilon ^2\bar{\sigma }/(10\lambda _+(H)) \cdot \bar{\sigma }^{-1}\Vert H\mathbf{v}\Vert ^2\\&=\frac{\epsilon ^{2}}{10}\left( 2\Vert \mathbf {w}\Vert ^{2}+\Vert H\mathbf{v}\Vert ^{2}\right) \ge \frac{\epsilon ^2}{5} \min \left\{ \frac{1}{\eta _{1}^{2}},\frac{{\bar{\sigma }}}{2} \right\} \left[ \Vert \tau \Vert ^3 + |\mathbf{v}^T H \mathbf{v}| \right] \\&\ge \frac{\epsilon ^2{\tilde{\eta }}_1}{5} \left[ \Vert \tau \Vert ^3 + |\mathbf{v}^T H \mathbf{v}|^\frac{3}{2} \right] \ge \frac{\epsilon ^2 {\tilde{\eta }}_1}{10} \left| \Vert \tau \Vert ^2 + \mathbf{v}^T H \mathbf{v} \right| ^{\frac{3}{2}}\\&= \frac{\epsilon ^2{\tilde{\eta }}_1\sqrt{2}}{5} | f(\mathbf{y}) - f(\mathbf{z}) |^\frac{3}{2}, \end{aligned}$$

where the last equality is a consequence of (6.4). Thus, we can infer that the largest KL exponent of problem (1.1) at $\mathbf{z}$ is at least $\frac{1}{4}$.

Case 2. $(2-{\epsilon })r_{+}^{2}\Vert \Delta _{\mathcal {I}} \Vert \ge \Vert \Delta \Vert ^2$ or $\gamma _1 \le (2-\epsilon )r_+^2 \Vert \Delta _\mathcal {I}\Vert ^2 \Vert \Delta \Vert ^{-2}$. First, utilizing the identity $\Delta ^T \mathbf{y} = \frac{1}{2}\Vert \Delta \Vert ^2$ in (A.6), we have $P_{\mathbf {y}}^{\perp }\Delta =\Delta -\frac{1}{2}\Vert \Delta \Vert ^{2}\mathbf {y}$. Defining $t_\Delta := \Delta ^T P_{\mathbf {y}}^{\perp }\Delta = \Vert \Delta \Vert ^2 - \frac{1}{4} \Vert \Delta \Vert ^4$, we will now work with the following additional decompositions

$$\begin{aligned}&H\Delta = \Delta ^{T}H\Delta \cdot \mathbf {y}+c_{2} P_{\mathbf {y}}^{\perp } \Delta +\mathbf {w}_{1}, \quad c_{2}=\frac{1-\frac{1}{2}\Vert \Delta \Vert ^{2}}{t_\Delta }\Delta ^{T}H\Delta ,\\&\mathbf {w}=c_{3} P_{\mathbf {y}}^{\perp }\Delta +\mathbf {w}_{2}, \quad c_{3}= \frac{1}{t_\Delta }[2\Delta ^T\mathrm{diag}(\tau )\mathbf{y} - \Vert \Delta \Vert ^2 \mathbf{y}^T\mathrm{diag}(\tau )\mathbf{y}]. \end{aligned}$$

We note that due to the choice of $c_2$ and $c_3$ and (A.2), the vectors $\mathbf {w}_{1}$ and $\mathbf {w}_{2}$ are orthogonal to $\mathbf {y}$ and $\Delta$. Hence, by (A.3), it holds that

$$\begin{aligned} \Vert \mathrm {grad\!\;}{f(\mathbf {y})}\Vert ^{2}&= \Vert H\Delta \Vert ^2 + 2\mathbf{w}^T H\Delta + \Vert \mathbf{w}\Vert ^2 - (\Delta ^T H \Delta )^2 \\&= (c_2^2 + 2 c_2c_3 + c_3^2)\Vert P_\mathbf{y}^\bot \Delta \Vert ^2 + \Vert \mathbf{w}_1\Vert ^2 + 2 \mathbf{w}_1^T\mathbf{w}_2 + \Vert \mathbf{w}_2\Vert ^2 \\&= (c_{2}+c_{3})^{2} \cdot t_\Delta +\Vert \mathbf {w}_{1}+\mathbf {w}_{2}\Vert ^{2} \ge (c_{2}+c_{3})^{2} \cdot t_\Delta . \end{aligned}$$

Recalling the definitions introduced in (A.1) and using $\tau _k = (y_k - z_k)(y_k + z_k) = \Delta _k (\Delta _k + 2z_k)$ and $y_k = \Delta _k + z_k$, we can express $t_\Delta \cdot c_{3}$ via

$$\begin{aligned} t_\Delta c_{3}&= 2\sum \limits _{k\in [n]} \Delta _k^2(\Delta _k+2z_k)(\Delta _k+z_k) \nonumber \\&\quad -\Vert \Delta \Vert ^{2}\sum \limits _{k\in [n]} \Delta _k(\Delta _{k}+2z_{k})(z_{k}+\Delta _{k})^{2} \nonumber \\&= \sum \limits _{k \in [n]} 2 [\Delta _k^4 + 3 \Delta _k^3z_k + 2\Delta _k^2z_k^2] \nonumber \\&\quad - \Vert \Delta \Vert ^2 [\Delta _k^4 + 4 \Delta _k^3z_k + 5\Delta _k^2z_k^2 + 2\Delta _kz_k^3] \nonumber \\&= (2-\Vert \Delta \Vert ^2) \gamma _4 + (6 - 4 \Vert \Delta \Vert ^2) \gamma _3 + (4-5\Vert \Delta \Vert ^2) \gamma _2 - 2 \Vert \Delta \Vert ^2 \gamma _1. \end{aligned}$$

(A.7)

Notice that we have $\gamma _2 \ge r_+^2 \Vert \Delta _\mathcal {I}\Vert ^2$ and $\gamma _1 \le 1$ provided that $\Vert \Delta \Vert \le 1$. Now, if $(2-{\epsilon })r_{+}^{2}\Vert \Delta _{\mathcal {I}}\Vert \ge \Vert \Delta \Vert ^{2}$, we obtain

$$\begin{aligned} (4-5\Vert \Delta \Vert ^2) \gamma _2 - 2\Vert \Delta \Vert ^2 \gamma _1&\ge (4 - 5\Vert \Delta \Vert ^2) r_+^2 \Vert \Delta _\mathcal {I}\Vert ^2 - 2\Vert \Delta \Vert ^2\\&\ge (2\epsilon - 5\Vert \Delta \Vert ^2) r_+^2 \Vert \Delta _\mathcal {I}\Vert ^2. \end{aligned}$$

and hence, it follows

$$\begin{aligned} t_\Delta \cdot c_{3} \ge \gamma _4 + \epsilon r_+^2 \Vert \Delta _\mathcal {I}\Vert ^2 + o(\Vert \Delta _\mathcal {I}\Vert ^2) \ge \Vert \Delta \Vert _4^4 + \frac{\epsilon r_+^2}{2} \Vert \Delta _\mathcal {I}\Vert ^2 \end{aligned}$$

(A.8)

for $\Delta$ sufficiently small. Otherwise, if $\gamma _1 \le (2-\epsilon )r_+^2 \Vert \Delta _\mathcal {I}\Vert ^2 \Vert \Delta \Vert ^{-2}$, then we also have $(4-5\Vert \Delta \Vert ^2) \gamma _2 - 2 \Vert \Delta \Vert ^2 \gamma _1 \ge (2\epsilon - 5\Vert \Delta \Vert ^2) r_+^2 \Vert \Delta _\mathcal {I}\Vert ^2$ and thus, (A.8) holds in both sub-cases. Consequently, due to the positive semidefiniteness of H, (A.8), and

$$\begin{aligned} \Vert \Delta \Vert ^2 = - 2 \Delta ^T \mathbf{z} = -2\Delta _\mathcal {I}^T z_\mathcal {I}\le 2 \Vert \Delta _\mathcal {I}\Vert \Vert z_\mathcal {I}\Vert \le 2 \Vert \Delta _\mathcal {I}\Vert , \end{aligned}$$

(A.9)

we can infer

$$\begin{aligned} \Vert \mathrm {grad\!\;}{f(\mathbf {y})}\Vert&\ge |c_{2}+c_{3}| \sqrt{t_\Delta } = \left[ \left( 1-\frac{1}{2}\Vert \Delta \Vert ^2\right) \Delta ^{T}H\Delta + t_\Delta c_{3}\right] t_\Delta ^{-1/2} \\&\ge [\Delta ^{T}H\Delta + 2 \Vert \Delta \Vert _4^4 +\epsilon r_{+}^{2}\Vert \Delta _{\mathcal {I}}\Vert ^{2}] \cdot (2\Vert \Delta \Vert )^{-1} \\&\ge [\Delta ^{T}H\Delta + 2 \Vert \Delta \Vert _4^4 +\epsilon r_{+}^{2}\Vert \Delta _{\mathcal {I}}\Vert ^{2}]^{\frac{3}{4}} \cdot \frac{\epsilon ^{\frac{1}{4}}\sqrt{r_{+} \Vert \Delta _{\mathcal {I}}\Vert }}{2\Vert \Delta \Vert }\\&\ge \eta _2 [\Delta ^{T}H\Delta + 2 \Vert \Delta \Vert _4^4 +\epsilon r_{+}^{2}\Vert \Delta _{\mathcal {I}}\Vert ^{2}]^{\frac{3}{4}}, \end{aligned}$$

where $\eta _2 := \epsilon ^{\frac{1}{4}}\sqrt{r_{+}} / 2\sqrt{2}$ and if $\Delta$ is chosen sufficiently small. Here, we also used the estimates $t_{\Delta }=\Vert \Delta \Vert ^{2}-\Vert \Delta \Vert ^{4}/4\le \Vert \Delta \Vert ^{2}$ and $1- \Vert \Delta \Vert ^2/2 \ge 1/2$ in the second inequality. The third inequality follows from $[\Delta ^{T}H\Delta + 2 \Vert \Delta \Vert _4^4 +\epsilon r_{+}^{2}\Vert \Delta _{\mathcal {I}}\Vert ^{2}]^\frac{1}{4}\ge \epsilon ^\frac{1}{4} \sqrt{r_{+}\Vert \Delta _{\mathcal {I}}\Vert }$. Next, utilizing (6.4) and $|\tau _k| \le 2 |\Delta _k|$ for all $k \in \mathcal {I}$, we finally obtain

$$\begin{aligned} |f(\mathbf {y})-f(\mathbf {z})|&=\frac{1}{2}[ \Delta ^{T}H\Delta + \Vert \tau \Vert ^2] \\&\le \frac{1}{2}\left[ \Delta ^{T}H\Delta + \Vert \Delta _\mathcal {A}\Vert _4^4 + 4\Vert \Delta _{\mathcal {I}}\Vert ^{2} \right] \le \eta _{3} \Vert \mathrm {grad\!\;}{f(\mathbf {y})}\Vert ^{\frac{4}{3}} \end{aligned}$$

for some constant $\eta _{3}>0$ and for all $\mathbf{y}$ sufficiently close to $\mathbf{z}$. Hence, the largest KL exponent is at least $\frac{1}{4}$ in this case.

Case 3. $(1-\epsilon )\Vert H\mathbf{v}\Vert \le \Vert \mathbf {w}\Vert \le (1+\epsilon )\Vert H\mathbf{v}\Vert$, $\gamma _{1}\ge (2-\epsilon )r_{+}^{2}\Vert \Delta _{\mathcal {I}} \Vert ^{2}\Vert \Delta \Vert ^{-2}$ and $\left( 2-\epsilon \right) r_{+}^{2}\Vert \Delta _{\mathcal {I}}\Vert \le \Vert \Delta \Vert ^{2}$. In this case, the inequality (A.9) implies $\Vert \Delta _{\mathcal {I}}\Vert =\Theta (\Vert \Delta \Vert ^{2})$ and setting $\nu = [(2-\epsilon )r_+^2]^{-1}$, the terms $\gamma _{i}$ for $i=1, \ldots ,3$ can be estimated as follows

$$\begin{aligned} (4\nu )^{-1} \Vert \Delta \Vert ^2&\le (2-\epsilon )r_{+}^{2}\Vert \Delta _{\mathcal {I}}\Vert ^{2}\Vert \Delta \Vert ^{-2} \le \gamma _{1}\le \Vert \Delta _{\mathcal {I}}\Vert _{1} \le |{\mathcal {I}}|\nu \Vert \Delta \Vert ^2,\nonumber \\&\quad \frac{r_+^2}{4} \Vert \Delta \Vert ^4 \le r_{+}^{2}\Vert \Delta _{\mathcal {I}}\Vert ^{2} \le \gamma _{2}\le \Vert \Delta _{\mathcal {I}}\Vert ^{2} \le \nu ^2 \Vert \Delta \Vert ^4,\nonumber \\&\quad -\nu ^3 \Vert \Delta \Vert ^6 \le -\Vert \Delta _{\mathcal {I}}\Vert _3^{3} \le \gamma _{3}\le \Vert \Delta _{\mathcal {I}}\Vert _3^{3} \le \nu ^3 \Vert \Delta \Vert ^6. \end{aligned}$$

(A.10)

Together with $\gamma _4 = \Vert \Delta \Vert _4^4$, this shows that

$$\begin{aligned} \gamma _{1} = \Theta (\Vert \Delta \Vert ^2) ,\quad \gamma _{2} = \Theta ( \Vert \Delta \Vert ^4), \quad \gamma _{3}=O(\Vert \Delta \Vert ^{6}),\quad \gamma _{4}=\Theta (\Vert \Delta \Vert ^{4}) \end{aligned}$$

(A.11)

for all $\Delta$. Let us set $m := |\mathcal {I}|$ and define $\sigma _{k}= \Delta _{k} z_{k}+\frac{1}{2m}\Vert \Delta \Vert ^{2}$ for all $k\in \mathcal {I}$ and $\sigma _k = 0$ for all $k \in \mathcal {A}$. Then, by (A.6), we have

$$\begin{aligned} \sum _{k \in [n]} \sigma _k = 0, \quad \Vert \sigma \Vert _1 \ge \sum _{k\in \mathcal {I}} z_{k}^{2}\sigma _{k} = \gamma _1 + \frac{1}{2m} \Vert \Delta \Vert ^2, \end{aligned}$$

(A.12)

and $\Vert \sigma \Vert = \Theta (\Vert \Delta \Vert ^2)$. We now express $\Vert \mathbf {w}\Vert ^{2}$ in terms of $\gamma _{1}$, $\gamma _2$, $\gamma _3$, and $\gamma _{4}$. Specifically, by utilizing (A.2), (A.11) and by mimicking the derivation of (A.7), we obtain

$$\begin{aligned} \frac{1}{4} \Vert \mathbf {w}\Vert ^{2}&= \Vert P_\mathbf{y}^\bot \mathrm{diag}(\tau )\mathbf{y}\Vert ^{2} = \mathbf{y}^T \mathrm{diag}(|\tau |^2)\mathbf{y} - (\mathbf{y}^T\mathrm{diag}(\tau )\mathbf{y})^2 \\&= \sum \limits _{k \in [n]} (z_k+\Delta _k)^2\Delta _k^2(\Delta _k+2z_k)^2 - (\Vert \Delta \Vert _4^4 + 4\gamma _3 + 5\gamma _2 + 2\gamma _1)^2 \\&= \sum \limits _{k \in [n]} \Delta _k^2 [\Delta _k^4 + 6\Delta _k^3z_k + 13\Delta _k^2z_k^2 + 12 \Delta _kz_k^3 + 4z_k^4] - 4\gamma _1^2 - 20\gamma _1\gamma _2 \\&\quad - 25\gamma _2^2 - 8\gamma _3(5\gamma _2+2\gamma _1) - 16 \gamma _3^2 - 2 \Vert \Delta \Vert _4^4(4\gamma _3+5\gamma _2+2\gamma _1) - \Vert \Delta \Vert _4^8 \\&= \sum \limits _{k \in \mathcal {I}} [4 \Delta _k^2 z_k^4 + 12 \Delta _k^3 z_k^3 + 13 \Delta _k^4 z_k^2 + 6\Delta _k^5z_k] - 4\gamma _1^2 + O(\Vert \Delta \Vert ^6) \\&= \sum \limits _{k \in \mathcal {I}} 4\Delta _k^2z_k^4 - 4\gamma _1^2 + O(\Vert \Delta \Vert ^6) = \sum \limits _{k, j \in \mathcal {I}} 4\Delta _k^2z_k^4z_j^2 - 4\gamma _1^2 + O(\Vert \Delta \Vert ^6) \\&= \sum \limits _{k, j \in \mathcal {I}} 2\Delta _k^2z_k^4z_j^2 + \sum \limits _{k, j \in \mathcal {I}} 2\Delta _j^2z_j^4z_k^2 - 4\left( \sum \limits _{k\in \mathcal {I}} \Delta _k z_k^3 \right) ^2 + O(\Vert \Delta \Vert ^6) \\&= 2 \sum \limits _{k, j \in \mathcal {I}} [z_k^4 z_j^2 \Delta _k^2 - 2 z_k^3 z_j^3 \Delta _k \Delta _j + z_k^2 z_j^4 \Delta _j^2] + O(\Vert \Delta \Vert ^6) \\&= 2 \sum \limits _{k, j \in \mathcal {I}} z_k^2 z_j^2 (\sigma _k - \sigma _j)^2 + O(\Vert \Delta \Vert ^6). \end{aligned}$$

We notice that the higher order terms $\sum _{k \in \mathcal {I}} \Delta _k^{3+\ell } z_k^{3-\ell }$, $\ell \in \{0,1,2\}$, can be discussed as in (A.10) and are all bounded by $O(\Vert \Delta \Vert ^6)$. Finally, applying (A.12) and $|z_k| \le 1$ and $z_k^2 \ge r_+^2$ for all $k \in \mathcal {I}$, we obtain

$$\begin{aligned} \sum \limits _{k, j \in \mathcal {I}} z_k^2 z_j^2 (\sigma _k - \sigma _j)^2 \ge r_+^4 \sum \limits _{k,j \in \mathcal {I}} (\sigma _k - \sigma _j)^2 = 2 m r_+^4 \Vert \sigma \Vert ^2 \end{aligned}$$

and ${\sum }_{k, j \in \mathcal {I}} z_k^2 z_j^2 (\sigma _k - \sigma _j)^2 \le 2m \Vert \sigma \Vert ^2$ which implies $\Vert \mathbf {w}\Vert =\Theta (\Vert \Delta \Vert ^{2})$ and

$$\begin{aligned} \Vert H\mathbf{v}\Vert =\Theta (\Vert \Delta \Vert ^{2}) \end{aligned}$$

(A.13)

for $\Delta \rightarrow 0$. As a consequence, by (A.4), we also get

$$\begin{aligned} |\mathbf{v}^{T}H\mathbf{v}|=O(\Vert \Delta \Vert ^{4}). \end{aligned}$$

(A.14)

Furthermore, by (6.4) and (A.1), it holds that

$$\begin{aligned} 2 |f(\mathbf {y})-f(\mathbf {z})|&\le |\Delta ^{T}H\Delta | + \Vert \tau \Vert ^{2} = |\mathbf{v}^{T}H\mathbf{v}| + \sum \limits _{k\in [n]}[\Delta _{k}^{2}+2z_{k}\Delta _{k}]^{2}\nonumber \\&=|\mathbf{v}^{T}H\mathbf{v}| + \Vert \Delta \Vert _4^4 +4\gamma _{3}+4\gamma _{2} = O(\Vert \Delta \Vert ^{4}) \end{aligned}$$

(A.15)

for $\Delta \rightarrow 0$. For some index sets ${\mathcal {K}}, {\mathcal {J}} \subset [n]$, let $H_{{\mathcal {K}}{\mathcal {J}}} \in \mathbb {R}^{|{\mathcal {K}}| \times |{\mathcal {J}}|}$ denote the submatrix $H_{{\mathcal {K}}{\mathcal {J}}} = (H_{kj})_{k \in {\mathcal {K}}, j \in {\mathcal {J}}}$. Notice that due to the positive semidefiniteness of H, we have $H_{\mathcal {I}\mathcal {I}}\succeq 0$ and $H_{\mathcal {A}\mathcal {A}}\succeq 0$. Moreover, due to (A.2), (A.3), and $|\mathbf{v}^{T}H\mathbf{v}|=O(\Vert \Delta \Vert ^{4})$, we obtain

$$\begin{aligned} \Vert \mathrm {grad\!\;}{f(\mathbf{y})}\Vert ^2&= \Vert H\Delta + \mathbf{w} \Vert ^2 + O(\Vert \Delta \Vert ^8) \nonumber \\&= \Vert H_{\mathcal {A}\cdot } \Delta + {w}_\mathcal {A}\Vert ^2 + \Vert H_{\mathcal {I}\cdot } \Delta + {w}_\mathcal {I}\Vert ^2 + O(\Vert \Delta \Vert ^8)\nonumber \\&= \Vert H_{\mathcal {I}\mathcal {A}}^T\Delta _\mathcal {I}+ H_{\mathcal {A}\mathcal {A}} \Delta _\mathcal {A}+ 2\mathrm{diag}(|\Delta _\mathcal {A}|^2) \Delta _\mathcal {A}- c_1 \Delta _\mathcal {A}\Vert ^2 \nonumber \\&\quad + \Vert H_{\mathcal {I}\mathcal {I}} \Delta _\mathcal {I}+ H_{\mathcal {I}\mathcal {A}} \Delta _\mathcal {A}+ {w}_\mathcal {I}\Vert ^2 + O(\Vert \Delta \Vert ^8), \end{aligned}$$

(A.16)

where $c_{1}=2\mathbf{y}^{T}\mathrm{diag}(\tau )\mathbf{y}$ was defined in (A.2) and we used the identities $z_{k}=0$ for $k\in \mathcal {A}$ and $w_{\mathcal {A}}=2\mathrm{diag}(\tau _\mathcal {A}){y}_\mathcal {A}-c_{1}{y}_\mathcal {A}=2\mathrm{diag}(|\Delta _\mathcal {A}|^2)\Delta _\mathcal {A}- c_1 \Delta _\mathcal {A}$. We set

$$\begin{aligned} \mathbf{h}&:= H_{\mathcal {I}\mathcal {I}} \Delta _\mathcal {I}+ H_{\mathcal {I}\mathcal {A}} \Delta _\mathcal {A}, \quad \mathbf{g}_1 := \mathbf{h}+ {w}_\mathcal {I}, \nonumber \\ \mathbf{g}_2&:= H_{\mathcal {I}\mathcal {A}}^T\Delta _\mathcal {I}+ H_{\mathcal {A}\mathcal {A}} \Delta _\mathcal {A}+ 2\mathrm{diag}(|\Delta _\mathcal {A}|^2)\Delta _\mathcal {A}- c_1 \Delta _\mathcal {A}\end{aligned}$$

(A.17)

and, let $\eta _{4},\eta _{5}>0$ and $\mu \in (0,\frac{1}{2})$ be given constants. Next, we discuss two separate sub-cases.

Sub-case 3.1. $\Delta _{\mathcal {I}}^{T}\mathbf{h}\ge -\eta _{4}\Vert \Delta \Vert ^{4+\mu }$ or $\Delta _\mathcal {I}^T \mathbf{h} \le -\eta _5 \Vert \Delta \Vert ^{4-\mu }$. Let us first assume $\Delta _{\mathcal {I}}^{T}{} \mathbf{h}\ge -\eta _{4}\Vert \Delta \Vert ^{4+\mu }$. Following the derivation of (A.7) and using (A.6), (A.2), and (A.11), we obtain $\frac{1}{2}c_1 = \Vert \Delta \Vert _4^4 + 4\gamma _3 + 5\gamma _2 + 2\gamma _1 = \Theta (\Vert \Delta \Vert ^2)$, $\Delta _\mathcal {I}^T y_\mathcal {I}= - \frac{1}{2}\Vert \Delta \Vert ^2$, and

$$\begin{aligned} \Delta _\mathcal {I}^T w_\mathcal {I}= 2\Delta _\mathcal {I}^T\mathrm{diag}(\tau _\mathcal {I}){y}_{\mathcal {I}} - c_1 \Delta _\mathcal {I}^T{y}_\mathcal {I}= 2\Vert \Delta _\mathcal {I}\Vert _4^4 + 6 \gamma _3 + 4\gamma _2 + {\textstyle \frac{1}{2}} \Vert \Delta \Vert ^2 c_1, \end{aligned}$$

which yields

$$\begin{aligned} \Delta _{\mathcal {I}}^{T}{} \mathbf{g}_{1}&\ge - \eta _4 \Vert \Delta \Vert ^{4+\mu } + 2 \Vert \Delta _\mathcal {I}\Vert _4^4 + 4\gamma _2 + \Vert \Delta \Vert ^2 [\Vert \Delta \Vert _4^4 + 5\gamma _2 + 2\gamma _1] \\&\quad + 6\gamma _3 + 4 \Vert \Delta \Vert ^2 \gamma _3 \\&\ge \Theta (\Vert \Delta \Vert ^{4}). \end{aligned}$$

Similarly, in the case $\Delta _{\mathcal {I}}^{T}{} \mathbf{h} \le -\eta _{5}\Vert \Delta \Vert ^{4-\mu }$ and if $\Vert \Delta \Vert$ is sufficiently small, we get

$$\begin{aligned} \Delta _{\mathcal {I}}^{T}{} \mathbf{g}_{1} \le -\frac{\eta _{5}}{2}\Vert \Delta \Vert ^{4-\mu }. \end{aligned}$$

Combining both cases, we can infer $\Vert \Delta _{\mathcal {I}}\Vert \Vert \mathbf{g}_{1}\Vert \ge |\Delta _{\mathcal {I}}^{T}{} \mathbf{g}_{1}| \ge \eta _6 \Vert \Delta \Vert ^4$ for some $\eta _6 > 0$ and for all $\mathbf{y}$ sufficiently close to $\mathbf{z}$. By the assumptions of case 3, this implies $\Vert \mathbf{g}_{1}\Vert \ge \nu ^{-1}\eta _6 \Vert \Delta \Vert ^2$ and hence, by (A.16), we have $\Vert \mathrm {grad\!\;}f(\mathbf{y})\Vert \ge \Theta (\Vert \Delta \Vert ^2)$. Considering (A.15), the Łojasiewicz inequality holds with $\theta = \frac{1}{2}$ in this sub-case.

Sub-case 3.2. $-\eta _{5}\Vert \Delta \Vert ^{4-\mu }\le \Delta _{\mathcal {I}}^{T}{} \mathbf{h} \le -\eta _{4}\Vert \Delta \Vert ^{4+\mu }$. Due to $\Delta _{\mathcal {I}}^{T}H_{\mathcal {I} \mathcal {I}}\Delta _{\mathcal {I}}\ge 0$, this directly yields $\Delta _{\mathcal {I}}^{T}H_{\mathcal {I}\mathcal {A}} \Delta _{\mathcal {A}}=\Delta _{\mathcal {I}}^T\mathbf{h}-\Delta _{\mathcal {I}}^{T}H_{\mathcal {I}\mathcal {I}} \Delta _{\mathcal {I}}<0$. Moreover, we can estimate that

$$\begin{aligned} \Delta _{\mathcal {I}}^{T}H_{\mathcal {I}\mathcal {A}}\Delta _{\mathcal {A}}&=\Delta _{\mathcal {I}}^T\mathbf{h}-\Delta _{\mathcal {I}}^{T}H_{\mathcal {I}\mathcal {I}}\Delta _{\mathcal {I}} \ge -\eta _5\Vert \Delta \Vert ^{4-\mu } - \lambda _-(H_{\mathcal {I}\mathcal {I}})\Vert \Delta _\mathcal {I}\Vert ^2\\&\ge -\eta _5\Vert \Delta \Vert ^{4-\mu } - \lambda _-(H_{\mathcal {I}\mathcal {I}}) \cdot (2-\epsilon )^{-2}r_+^{-4}\Vert \Delta \Vert ^4 \ge -2\eta _5 \Vert \Delta \Vert ^{4-\mu }, \end{aligned}$$

where $\lambda _-(H_{\mathcal {I}\mathcal {I}})$ is the maximal eigenvalue of $H_{\mathcal {I}\mathcal {I}}$. We note that the last inequality holds when $\Vert \Delta \Vert$ is small enough, since $\eta _5,r_+,\epsilon ,H_{\mathcal {I}\mathcal {I}}$ do not depend on $\Delta$ and can be considered as constants when $\Delta \rightarrow 0$. Also, we have

$$\begin{aligned} \Delta _{\mathcal {A}}^{T}H_{\mathcal {A}\mathcal {A}} \Delta _{\mathcal {A}}\ge \Delta _{\mathcal {A}}^{T} (H_{\mathcal {I}\mathcal {A}}^{T}\Delta _{\mathcal {I}} +H_{\mathcal {A}\mathcal {A}}\Delta _{\mathcal {A}})\ge \Delta ^{T}H\Delta = {\mathbf{v}^T H \mathbf{v}}. \end{aligned}$$

(A.18)

Utilizing the positive semidefiniteness of H, it holds that

$$\begin{aligned} \Vert H\mathbf{v}\Vert ^2 = (H^\frac{1}{2}\mathbf{v})^TH(H^\frac{1}{2}\mathbf{v}) \le \lambda _+(H) \Vert H^\frac{1}{2}\mathbf{v}\Vert = \lambda _+(H) \cdot \mathbf{v}^TH\mathbf{v}. \end{aligned}$$

Hence, the estimates $\Vert H\mathbf{v}\Vert =\Theta (\Vert \Delta \Vert ^2)$ in (A.13) and $|\mathbf{v}^TH\mathbf{v}| = O(\Vert \Delta \Vert ^4)$ in (A.14) lead to $\mathbf{v}^TH\mathbf{v} = \Theta (\Vert \Delta \Vert ^4)$. Consequently, if $\Delta _{\mathcal {A}}^{T}H_{\mathcal {A}\mathcal {A}} \Delta _{\mathcal {A}}\ge \eta _{7}\Vert \Delta \Vert ^{4-2\mu }$ for some constant $\eta _{7}>0$, then we can infer

$$\begin{aligned} \Delta _{\mathcal {A}}^{T}\mathbf{g}_{2}&=\Delta _{\mathcal {A}}^{T}(H_{\mathcal {I} \mathcal {A}}^{T}\Delta _{\mathcal {I}}+H_{\mathcal {A} \mathcal {A}}\Delta _{\mathcal {A}})+2\Vert \Delta _{\mathcal {A}} \Vert _{4}^{4}-c_1\Vert \Delta _{\mathcal {A}}\Vert ^{2}\\&\ge \eta _{7}\Vert \Delta \Vert ^{4-2\mu }-2\eta _{5} \Vert \Delta \Vert ^{4-\mu }+O(\Vert \Delta \Vert ^{4})\ge \frac{\eta _{7}}{2} \Vert \Delta \Vert ^{4-2\mu } \end{aligned}$$

for $\Delta \rightarrow 0$. As in sub-case 3.1, this allows to show $\Vert \mathrm {grad\!\;}{f(\mathbf{y})}\Vert \ge \Theta (\Vert \Delta \Vert ^{3-2\mu })$ and thus, by (A.15), the Łojasiewicz inequality holds with $\theta = \frac{1+2\mu }{4} \ge \frac{1}{4}$.

Finally, let us consider $\eta _{8}\Vert \Delta \Vert ^{4}\le \Delta _{\mathcal {A}}^{T} H_{\mathcal {A}\mathcal {A}}\Delta _{\mathcal {A}}\le \eta _{7}\Vert \Delta \Vert ^{4-2\mu }$, where $\eta _{8} > 0$ is chosen such that $\Delta ^{T}H\Delta \ge \eta _{8}\Vert \Delta \Vert ^{4}$ and let us define the final decompositions

$$\begin{aligned} \Delta _{\mathcal {A}}=\psi _{1}+\xi _{1},\quad \psi _{1}\in \mathrm {null}~H_{\mathcal {A}\mathcal {A}},\quad \xi _{1} \in [\mathrm {null}~H_{\mathcal {A}\mathcal {A}}]^\bot ,\\ \mathrm{diag}(|\Delta _\mathcal {A}|^2)\Delta _{\mathcal {A}}=\psi _{2}+\xi _{2}, \quad \psi _{2}\in \mathrm {null}~H_{\mathcal {A}\mathcal {A}}, \quad \xi _{2} \in [\mathrm {null}~H_{\mathcal {A}\mathcal {A}}]^\bot , \end{aligned}$$

where $\mathrm {null}~M$ is the null space of a matrix M. We then have $\Vert \xi _{1}\Vert =\Theta (\Vert H_{\mathcal {A}\mathcal {A}}\xi _{1}\Vert )= \Theta (\sqrt{\xi _{1}^{T}H_{\mathcal {A}\mathcal {A}}\xi _{1}})=O (\Vert \Delta \Vert ^{2-\mu })$ and $\Vert \psi _{1}\Vert =O(\Vert \Delta \Vert )$. Notice that such decompositions exist due to the symmetry of $H_{\mathcal {A}\mathcal {A}}$.

Since H is positive semidefinite, we can show that $\mathrm {null}\,H_{\mathcal {A}\mathcal {A}}\subset \mathrm {null}\, H_{\mathcal {I}\mathcal {A}}$. If $H_{\mathcal {I}\mathcal {A}}=0$, then this claim is certainly true. Otherwise, if we assume that the statement is false, the set $S = \mathrm {null}\,H_{\mathcal {A}\mathcal {A}}\cap [\mathrm {null}\,H_{\mathcal {I}\mathcal {A}}]^{\bot }$ does not only contain zero and we can select $\psi \in S\backslash \{0\}$ and $\xi \in [\mathrm {null}\,H_{\mathcal {I}\mathcal {A}}^{T}]^{\bot } \backslash \{0\} = [\mathrm {ran}\,H_{\mathcal {I}\mathcal {A}}] \backslash \{0\}$. Then it holds that

$$\begin{aligned} \begin{bmatrix}\xi ^T&\quad a\psi ^T\end{bmatrix}\begin{bmatrix} H_{\mathcal {I}\mathcal {I}} &{}\quad H_{\mathcal {I}\mathcal {A}} \\ H_{\mathcal {I}\mathcal {A}}^T &{}\quad H_{\mathcal {A}\mathcal {A}} \end{bmatrix} \begin{bmatrix}\xi \\ a\psi \end{bmatrix} =\xi ^{T}H_{\mathcal {I}\mathcal {I}}\xi +2a \psi ^{T}H_{\mathcal {I}\mathcal {A}}^{T}\xi \ge 0,\quad \forall ~a\in \mathbb {R}. \end{aligned}$$

But since $[H_{\mathcal {I}\mathcal {A}}\psi ]^{T}\xi \ne 0$, we can choose a such that $\xi ^{T}H_{\mathcal {I}\mathcal {I}}\xi +2a\psi ^{T}H_{\mathcal {I} \mathcal {A}}^{T}\xi <0$, which is a contradiction.

Hence and due to $H_{\mathcal {I}\mathcal {A}}^{T}\Delta _{\mathcal {I}} \in \mathrm {ran}\,H_{\mathcal {I}\mathcal {A}}^T = [\mathrm {null}\,H_{\mathcal {I}\mathcal {A}}]^\bot$, we can infer $H_{\mathcal {I}\mathcal {A}}^{T}\Delta _{\mathcal {I}} + H_{\mathcal {A}\mathcal {A}}\Delta _{\mathcal {A}}\in [\mathrm {null}\,H_{\mathcal {A}\mathcal {A}}]^{\perp }$. This implies that $\mathbf{g}_2$ can be written as $\mathbf{g}_2 = \mathbf{g}_3 + \mathbf{d}$ where $\mathbf{g}_3 \in [\mathrm {null}\,H_{\mathcal {A}\mathcal {A}}]^{\perp }$ and $\mathbf{d}=2\psi _{2}-c_{1}\psi _{1} \in \mathrm {null}\,H_{\mathcal {A}\mathcal {A}}$. If $\Vert \mathbf{d}\Vert \ge \frac{\eta _{8}}{2}\Vert \Delta \Vert ^{3}$, we obtain $\Vert \mathrm {grad\!\;}{f(\mathbf{y})}\Vert \ge \Vert \mathbf{g}_{2}\Vert \ge \Vert \mathbf{d}\Vert \ge \frac{\eta _{8}}{2}\Vert \Delta \Vert ^{3}$ by (A.16) and the Łojasiewicz inequality is satisfied with $\theta =\frac{1}{4}$ due to (A.15). Otherwise, if $\Vert \mathbf{d}\Vert \le \frac{\eta _{8}}{2}\Vert \Delta \Vert ^{3}$, it follows

$$\begin{aligned} 2\Vert \Delta _{\mathcal {A}}\Vert _{4}^{4}-c_{1}\Vert \Delta _{\mathcal {A}}\Vert ^{2}&= 2\psi _{1}^{T}\psi _{2}-c_{1}\Vert \psi _{1}\Vert ^{2}+2\xi _{1}^{T}\xi _{2}-c_{1}\Vert \xi _{1}\Vert ^{2}\\&=\psi _{1}^{T}\mathbf {d}+2\xi _{1}^{T}\xi _{2}-c_{1}\Vert \xi _{1}\Vert ^{2} \ge -\frac{\eta _{8}}{2}\Vert \Delta \Vert ^{4}+O(\Vert \Delta \Vert ^{5-\mu }), \end{aligned}$$

where we applied the estimates $\Vert \xi _2\Vert \le \Vert \Delta _\mathcal {A}\Vert _6^3 \le \Vert \Delta \Vert ^3$ and $c_1 = \Theta (\Vert \Delta \Vert ^2)$. Using (A.18), this shows

$$\begin{aligned} \Delta _{\mathcal {A}}^{T}{} \mathbf{g}_{2}&=\Delta _{\mathcal {A}}^{T}(H_{\mathcal {I}\mathcal {A}}^{T}\Delta _{\mathcal {I}}+H_{\mathcal {A}\mathcal {A}}\Delta _{\mathcal {A}})+2\Vert \Delta _{\mathcal {A}}\Vert _{4}^{4}-c_{1}\Vert \Delta _{\mathcal {A}}\Vert ^{2}\\&\ge \Delta ^{T}H\Delta -\frac{\eta _{8}}{2}\Vert \Delta \Vert ^{4}+O(\Vert \Delta \Vert ^{5-\mu }) \ge \frac{\eta _{8}}{2}\Vert \Delta \Vert ^{4}+O(\Vert \Delta \Vert ^{5-\mu })\ge \frac{\eta _{8}}{4}\Vert \Delta \Vert ^{4} \end{aligned}$$

for $\Delta \rightarrow 0$. Thus, we have $\Vert \mathrm {grad\!\;}{f(\mathbf{y})}\Vert \ge \Theta (\Vert \Delta \Vert ^{3})$ and as before we can infer that the Łojasiewicz inequality holds with $\theta =\frac{1}{4}$ in this case. $\square$