A Forward–Backward Algorithm With Different Inertial Terms for Structured Non-Convex Minimization Problems

Abstract

We investigate an inertial forward–backward algorithm in connection with the minimization of the sum of a non-smooth and possibly non-convex and a non-convex differentiable function. The algorithm is formulated in the spirit of the famous FISTA method; however, the setting is non-convex and we allow different inertial terms. Moreover, the inertial parameters in our algorithm can take negative values too. We also treat the case when the non-smooth function is convex, and we show that in this case a better step size can be allowed. Further, we show that our numerical schemes can successfully be used in DC-programming. We prove some abstract convergence results which applied to our numerical schemes allow us to show that the generated sequences converge to a critical point of the objective function, provided a regularization of the objective function satisfies the Kurdyka–Łojasiewicz property. Further, we obtain a general result that applied to our numerical schemes ensures convergence rates for the generated sequences and for the objective function values formulated in terms of the KL exponent of a regularization of the objective function. Finally, we apply our results to image restoration.

Appendices

Proofs of Abstract Convergence Results

In what follows we give full proofs for Lemma 3.1, Corollary 3.1, Theorem 3.1 and Theorem 3.2.

Proof of Lemma 3.1

We divide the proof into the following steps.

Step I. We show that $u_1\in B(u^*,\rho )$ and $F(u_1)< F(u^*)+\eta .$

Indeed, $u_0\in B(u^*,\rho )$ and (5) assures that $F(u_1)\ge F(u^*).$ Further, (H1) assures that

$$\begin{aligned} \Vert x_1-x_0\Vert \le \sqrt{\frac{F(u_0)-F(u_1)}{a}}\le \sqrt{\frac{F(u_0)-F(u^*)}{a}}. \end{aligned}$$

Since $\Vert x_1-x^*\Vert =\Vert (x_1-x_0)+(x_0-x^*)\Vert \le \Vert x_1-x_0\Vert +\Vert x_0-x^*\Vert$ and $F(u_1)\le F(u_0)$ the condition (6) leads to

$$\begin{aligned} \Vert x_1-x^*\Vert \le \Vert x_0-x^*\Vert + \sqrt{\frac{F(u_0)-F(u^*)}{a}}<\frac{\rho }{c_1+c_2}. \end{aligned}$$

Now, from (H3) we have $\Vert u_1-u^*\Vert \le c_1\Vert x_1-x^*\Vert +c_2\Vert x_{0}-x^*\Vert$ hence

$$\begin{aligned} \Vert u_1-u^*\Vert < c_1\frac{\rho }{c_1+c_2}+c_2\frac{\rho }{c_1+c_2}= \rho . \end{aligned}$$

Thus, $u_1\in B(u^*,\rho );$ moreover, (5) and (H1) provide that $F(u^*)\le F(u_2)\le F(u_1)\le F(u_0)< F(u^*)+\eta .$

Step II. Next we show that whenever for a $k\ge 1$ one has $u_k\in B(u^*,\rho ),\,F(u_k)<F(u^*)+\eta$ then it holds that

$$\begin{aligned} 3\Vert x_{k+1}-x_{k}\Vert\le & {} \Vert x_{k}-x_{k-1}\Vert +\Vert x_{k-1}-x_{k-2}\Vert \nonumber \\ {}{} & {} +\frac{9b}{4a}(\varphi (F(u_k)-F(u^*))-\varphi (F(u_{k+1})-F(u^*))). \end{aligned}$$

(28)

Hence, let $k\ge 1$ and assume that $u_k\in B(u^*,\rho ),\,F(u_k)<F(u^*)+\eta$. Note that from (H1) and (5) one has $F(u^*)\le F(u_{k+1})\le F(u_k)<F(u^*)+\eta ;$ hence,

$$\begin{aligned} F(u_k)-F(u^*),F(u_{k+1})-F(u^*)\in [0,\eta ); \end{aligned}$$

thus, (28) is well stated. Now, if $x_k=x_{k+1}$ then (28) trivially holds.

Otherwise, from (H1) and (5) one has

$$\begin{aligned} F(u^*)\le F(u_{k+1})<F(u_k)<F(u^*)+\eta . \end{aligned}$$

(29)

Consequently, $u_k\in B(u^*,\rho )\cap \{u\in \mathbb {R}^m: F(u^*)<F(u)<F(u^*)+\eta \}$ and $B(u^*,\rho )\subseteq B(u^*,\sigma )\subseteq U$; hence, by using the KL inequality we get

$$\begin{aligned} \varphi '(F(u_k)-F(u^*)){{\,\textrm{dist}\,}}(0, {\partial }F(u_k))\ge 1. \end{aligned}$$

Since $\varphi$ is concave, and (29) assures that $F(u_{k+1})-F(u^*)\in [0,\eta ),$ one has

$$\begin{aligned} \varphi (F(u_k)-F(u^*))-\varphi (F(u_{k+1})-F(u^*))\ge \varphi '(F(u_k)-F(u^*))(F(u_k)-F(u_{k+1})), \end{aligned}$$

consequently,

$$\begin{aligned} \varphi (F(u_k)-F(u^*))-\varphi (F(u_{k+1})-F(u^*))\ge \frac{F(u_k)-F(u_{k+1})}{{{\,\textrm{dist}\,}}(0, {\partial }F(u_k))}. \end{aligned}$$

Now, by using (H1) and (H2) we get that

$$\begin{aligned} \varphi (F(u_k)-F(u^*))-\varphi (F(u_{k+1})-F(u^*))\ge \frac{a\Vert x_{k+1}-x_{k}\Vert ^2}{b(\Vert x_{k}-x_{k-1}\Vert +\Vert x_{k-1}-x_{k-2}\Vert )}. \end{aligned}$$

Consequently,

$$\begin{aligned}&\Vert x_{k+1}-x_{k}\Vert \le \\&\quad \sqrt{\frac{b}{a}\left( \varphi (F(u_k)-F(u^*))-\varphi (F(u_{k+1})-F(u^*))\right) (\Vert x_{k}-x_{k-1}\Vert +\Vert x_{k-1}-x_{k-2}\Vert )} \end{aligned}$$

and by arithmetical-geometrical mean inequality we have

$$\begin{aligned} \Vert x_{k+1}-x_{k}\Vert&\le \frac{\Vert x_{k}-x_{k-1}\Vert +\Vert x_{k-1}-x_{k-2}\Vert }{3}\\&\quad +\frac{3b}{4a}(\varphi (F(u_k)-F(u^*))-\varphi (F(u_{k+1})-F(u^*))), \end{aligned}$$

which leads to (28).

Step III. Now we show by induction that (28) holds for every $k\ge 1.$ Indeed, Step II. can be applied for $k=1$ since according to Step I. $u_1\in B(u^*,\rho )$ and $F(u_1)< F(u^*)+\eta .$ Consequently, for $k=1$ the inequality (28) holds.

Assume that (28) holds for every $k\in \{1,2,...,n\}$ and we show also that (28) holds for $k=n+1.$ Arguing as at Step II., the condition (H1) and (5) assure that $F(u^*)\le F(u_{n+1})\le F(u_n)<F(u^*)+\eta ,$ hence it remains to show that $u_{n+1}\in B(u^*,\rho ).$ By using the triangle inequality and (H3), one has

$$\begin{aligned} \Vert u_{n+1}-u^*\Vert&\le c_1\Vert x_{n+1}-x^*\Vert +c_2\Vert x_n-x^*\Vert \nonumber \\&=c_1\Vert (x_{n+1}-x_n)+(x_n-x_{n-1})+\cdots +(x_0-x^*)\Vert \nonumber \\&+c_2\Vert (x_{n}-x_{n-1})+(x_{n-1}-x_{n-2})+\cdots +(x_0-x^*)\Vert \nonumber \\&\le c_1\Vert x_{n+1}-x_n\Vert +(c_1+c_2)\Vert x_0-x^*\Vert +(c_1+c_2)\sum _{k=1}^{n}\Vert x_{k}-x_{k-1}\Vert . \end{aligned}$$

(30)

By summing up (28) from $k=1$ to $k=n$ and using $x_{-1}=x_0$ we obtain

$$\begin{aligned} \sum _{k=1}^n \Vert x_k-x_{k-1}\Vert&\le 3\Vert x_{1}-x_{0}\Vert -3\Vert x_{n+1}-x_{n}\Vert -\Vert x_n-x_{n-1}\Vert \nonumber \\&+\frac{9b}{4a}(\varphi (F(u_1)-F(u^*))-\varphi (F(u_{n+1})-F(u^*))). \end{aligned}$$

(31)

Combining (30) and (31) and neglecting the negative terms, we get

$$\begin{aligned} \Vert u_{n+1}-u^*\Vert \le (3c_1+3c_2)\Vert x_{1}-x_0\Vert +(c_1+c_2)\Vert x_0-x^*\Vert \nonumber \\ +(c_1+c_2)\frac{9b}{4a}\varphi (F(u_1)-F(u^*)). \end{aligned}$$

But $\varphi$ is strictly increasing and $F(u_1)-F(u^*)\le F(u_0)-F(u^*)$, hence

$$\begin{aligned}{} & {} \Vert u_{n+1}-u^*\Vert \le (3c_1+3c_2)\Vert x_{1}-x_0\Vert +(c_1+c_2)\Vert x_0-x^*\Vert \nonumber \\{} & {} \quad +(c_1+c_2)\frac{9b}{4a}\varphi (F(u_0)-F(u^*)). \end{aligned}$$

According to (H1), one has

$$\begin{aligned} \Vert x_{1}-x_0\Vert \le \sqrt{\frac{F(u_{0})-F(u_1)}{a}}\le \sqrt{\frac{F(u_{0})-F(u^*)}{a}}; \end{aligned}$$

(32)

hence, from (6) we get

$$\begin{aligned} \Vert u_{n+1}-u^*\Vert \le (c_1+c_2)\left( \Vert x_0-x^*\Vert +3\sqrt{\frac{F(u_{0})-F(u^*)}{a}}+\frac{9b}{4a}\varphi (F(u_0)-F(u^*))\right) <\rho . \end{aligned}$$

Hence, we have shown so far that $u_n\in B(u^*,\rho )$ for all $n\in \mathbb {N}.$

Step IV. According to Step III, the relation (28) holds for every $k\ge 1.$ But this implies that (31) holds for every $n\ge 1.$ By using (32) and neglecting the non-positive terms, (31) becomes

$$\begin{aligned} \sum _{k=1}^n \Vert x_k-x_{k-1}\Vert \le 3\sqrt{\frac{F(u_{0})-F(u^*)}{a}}+\frac{9b}{4a}\varphi (F(u_1)-F(u^*)). \end{aligned}$$

(33)

Now letting $n\longrightarrow +\infty$ in (33), we obtain that $\sum _{k=1}^{\infty } \Vert x_k-x_{k-1}\Vert <+\infty .$

Obviously, the sequence $S_n=\sum _{k=1}^n\Vert x_k-x_{k-1}\Vert$ is Cauchy; hence, for all $\varepsilon >0$ there exists $N_{\varepsilon }\in \mathbb {N}$ such that for all $n\ge N_\varepsilon$ and for all $p\in \mathbb {N}$ one has $S_{n+p}-S_n\le \varepsilon .$

But

$$\begin{aligned} S_{n+p}-S_n=\sum _{k={n+1}}^{n+p}\Vert x_k-x_{k-1}\Vert \ge \left\| \sum _{k={n+1}}^{n+p}(x_k-x_{k-1})\right\| =\Vert x_{n+p}-x_n\Vert ; \end{aligned}$$

hence, the sequence $(x_n)_{n\in \mathbb {N}}$ is Cauchy and consequently is convergent. Let $\lim _{n\longrightarrow +\infty }x_n=\overline{x}$ and let $\overline{u}=(\overline{x},\overline{x}).$ Now, from (H3) we have

$$\begin{aligned} \lim _{n\longrightarrow +\infty }\Vert u_n-\overline{u}\Vert \le \lim _{n\longrightarrow +\infty }(c_1\Vert x_n-\overline{x}\Vert +c_2\Vert x_{n-1}-\overline{x}\Vert )=0, \end{aligned}$$

consequently $(u_n)_{n\in \mathbb {N}}$ converges to $\overline{u}.$ Further, $(u_n)_{n\in \mathbb {N}}\subseteq B(u^*,\rho )$ and $\rho <\sigma$, hence $\overline{u}\in B(u^*,\sigma ).$

Since $F(u^*)\le F(u_n)<F(u^*)+\eta$ for all $n\ge 1$ and the sequence $(F(u_n))_{n\ge 1}$ is decreasing, obviously $F(u^*)\le \lim _{n\longrightarrow +\infty } F(u_n)<F(u^*)+\eta .$ Assume that $F(u^*)< \lim _{n\longrightarrow +\infty } F(u_n).$ Then, one has $u_n\in B(u^*,\sigma )\cap \{z\in \mathbb {R}^m: F(u^*)<F(z)<F(u^*)+\eta \}$ and by using the KL inequality and the fact that $\varphi$ is concave, therefore $\varphi '$ is decreasing, we get

$$\begin{aligned} \varphi '(\lim _{n\longrightarrow +\infty }F(u_n)-F(u^*))\Vert {{\,\textrm{dist}\,}}(0,{\partial }F(u_n))\Vert \ge \varphi '(F(u_n)-F(u^*))\Vert {{\,\textrm{dist}\,}}(0,{\partial }F(u_n))\Vert \ge 1, \end{aligned}$$

for all $n\ge 1,$ impossible, since according to (H2) and the fact that $(x_n)_{n\in \mathbb {N}}$ converges one has

$$\begin{aligned} \lim _{n\longrightarrow +\infty }\Vert {{\,\textrm{dist}\,}}(0,{\partial }F(u_n))\Vert =0. \end{aligned}$$

Consequently, one has $\lim _{n\longrightarrow +\infty }F(u_n)=F(u^*).$ Since $u_n\longrightarrow \overline{u},\, n\longrightarrow +\infty$ and F is lower semi-continuous, it is obvious that $\lim _{n\longrightarrow +\infty } F(u_n)\ge F(\overline{u}).$ Hence,

$$\begin{aligned} \lim _{n\longrightarrow +\infty }F(u_n)=F(u^*)\ge F(\overline{u}). \end{aligned}$$

Assume now that (H4) also holds. Obviously, in this case

$$\begin{aligned} u_{n_j}\longrightarrow \overline{u} \text{ and } F(u_{n_j})\longrightarrow F(\overline{u}),\,j\longrightarrow +\infty . \end{aligned}$$

Consequently, one has $F(\overline{u})=F(u^*).$

From (H2) we have that there exists $W_{n_j}\in {\partial }F(u_{n_j})$ such that

$$\begin{aligned} \Vert W_{n_j}\Vert \le b(\Vert x_{{n_j}+1}-x_{{n_j}}\Vert +\Vert x_{n_j}-x_{{n_j}-1}\Vert ), \end{aligned}$$

consequently, $\lim _{j\longrightarrow +\infty }\Vert W_{n_j}\Vert =0.$

Now, one has $(u_{n_j},W_{n_j})\longrightarrow (\overline{u}, 0) \text{ and } F(u_{n_j})\longrightarrow F(\overline{u}),\,j\longrightarrow +\infty$; hence, by the closedness criterion of the graph of the limiting subdifferential we get $0\in {\partial }F(\overline{u}),$ which shows that $\overline{u}\in {{\,\textrm{crit}\,}}(F)$. $\square$

Next we prove Corollary 3.1.

Proof of Corollary 3.1

The claim that (H3) holds with $c_1=2+c$ and $c_2=c$ is an easy verification. We have to show that (5) holds, that is, $u_n\in B(u^*,\rho )$ implies $u_{n+1}\in B(u^*,\sigma )$ for all $n\in \mathbb {N}.$

According to (H1), the assumption that $F(u_n)\ge F(u^*)$ for all $n\ge 1$ and the hypotheses of Lemma 3.1, we have

$$\begin{aligned} \Vert x_n-x_{n-1}\Vert \le \sqrt{\frac{F(u_{n-1})-F(u_{n})}{a}}\le \sqrt{\frac{F(u_0) -F(u_{n})}{a}}\le \sqrt{\frac{F(u_0)-F(u^*)}{a}}<\sqrt{\frac{\eta }{a}} \end{aligned}$$

and

$$\begin{aligned} \Vert x_{n+1}-x_{n}\Vert \le \sqrt{\frac{F(u_{n})-F(u_{n+1})}{a}}\le \sqrt{\frac{F(u_0) -F(u_{n+1})}{a}}\le \sqrt{\frac{F(u_0)-F(u^*)}{a}}<\sqrt{\frac{\eta }{a}} \end{aligned}$$

for all $n\ge 1$.

Assume now that $n\ge 1$ and $u_n\in B(u^*,\rho ).$ Then, by using the triangle inequality we get

$$\begin{aligned} \Vert u_{n+1}-u^*\Vert =\Vert (u_{n+1}-u_n)+(u_n-u^*)\Vert \le \Vert u_{n+1}-u_n\Vert +\Vert u_n-u^*\Vert \le \Vert u_{n+1}-u_n\Vert +\rho . \end{aligned}$$

Further,

$$\begin{aligned} \Vert u_{n+1}-u_n\Vert&=\Vert (v_{n+1}-v_n,w_{n+1}-w_n)\Vert \\&\le \Vert x_{n+1}+\alpha _{n+1}(x_{n+1}-x_{n})-x_n-\alpha _n(x_n-x_{n-1})\Vert \\&+\Vert x_{n+1}+\beta _{n+1}(x_{n+1}-x_{n})-x_n-\beta _n(x_n-x_{n-1})\Vert \\&\le (2+|\alpha _{n+1}|+|\beta _{n+1}|)\Vert x_{n+1}-x_n\Vert +(|\alpha _n|+|\beta _n|)\Vert x_n-x_{n-1}\Vert \\&\le (2+c)\Vert x_{n+1}-x_n\Vert +c\Vert x_{n}-x_{n-1}\Vert , \end{aligned}$$

where $c=\sup _{n\in \mathbb {N}}(|\alpha _{n}|+|\beta _{n}|).$

Consequently, we have

$$\begin{aligned} \Vert u_{n+1}-u^*\Vert \le (2+c)\Vert x_{n+1}-x_n\Vert +c\Vert x_{n}-x_{n-1}\Vert +\rho <(2+2c)\sqrt{\frac{\eta }{a}}+\rho \le \sigma , \end{aligned}$$

which is exactly $u_{n+1}\in B(u^*,\sigma ).$ Further, arguing analogously as at Step I in the proof of Lemma 3.1, we obtain that $u_{1}\in B(u^*,\rho )\subseteq B(u^*,\sigma )$ and this concludes the proof. $\square$

Now we are ready to prove Theorem 3.1.

Theorem 3.1

We will apply Corollary 3.1. Since $u^*=(x^*,x^*)\in \omega ((u_n)_{n\in \mathbb {N}})$ there exists a subsequence $(u_{n_k})_{k\in \mathbb {N}}$ such that $u_{n_k}\longrightarrow u^*,\, k\longrightarrow +\infty .$

From (H1) we get that the sequence $(F(u_n))_{n\in \mathbb {N}}$ is decreasing and from (H4), which according to the hypotheses holds for $u^*$, one has $F(u_{n_k})\longrightarrow F(u^*),\, k\longrightarrow +\infty ,$ that implies

$$\begin{aligned} F(u_n)\longrightarrow F(u^*),\,n\longrightarrow +\infty \text{ and } F(u_n)\ge F(u^*),\,\text{ for } \text{ all } n\in \mathbb {N}. \end{aligned}$$

(34)

We show next that $x_{n_k}\longrightarrow x^*,\, k\longrightarrow +\infty .$ Indeed, from (H1) one has $a\Vert x_{n_k}-x_{{n_k}-1}\Vert ^2\le F(u_{n_k-1})-F(u_{{n_k}})$ and obviously the right side of the above inequality goes to 0 as $k\longrightarrow +\infty .$ Hence, $\lim _{k\longrightarrow +\infty }(x_{n_k}-x_{n_k-1})=0.$ Further, since the sequences $(\alpha _n)_{n\in \mathbb {N}},(\beta _n)_{n\in \mathbb {N}}$ are bounded we get $\lim _{k\longrightarrow +\infty }\alpha _{n_k}(x_{n_k}-x_{n_k-1})=0$ and $\lim _{k\longrightarrow +\infty }\beta _{n_k}(x_{n_k}-x_{n_k-1})=0.$ Finally, $u_{n_k}\longrightarrow u^*,\, k\longrightarrow +\infty$ is equivalent to $x_{n_k}-x^*+\alpha _{n_k}(x_{n_k}-x_{{n_k}-1})\longrightarrow 0,\, k\longrightarrow +\infty$ and $x_{n_k}-x^*+\beta _{n_k}(x_{n_k}-x_{{n_k}-1})\longrightarrow 0,\, k\longrightarrow +\infty ,$ which lead to the desired conclusion, that is

$$\begin{aligned} x_{n_k}\longrightarrow x^*,\, k\longrightarrow +\infty . \end{aligned}$$

(35)

The KL property around $u^*$ states the existence of quantities $\varphi$, U, and $\eta$ as in Definition 2.1. Let $\sigma > 0$ be such that $B(u^*, \sigma )\subseteq U$ and $\rho \in (0,\sigma ).$ If necessary we shrink $\eta$ such that $\eta < \frac{a(\sigma -\rho )^2}{4(1+c)^2},$ where $c=\sup _{n\in \mathbb {N}}(|\alpha _n|+|\beta _n|).$

Now, since the function $\varphi$ is continuous and $(F(u_n))$ is non-increasing, further $F(u_n)\longrightarrow F(u^*),\,n\longrightarrow +\infty$, $\varphi (0)=0$ and $u_{n_k}\longrightarrow u^*,\,x_{n_k}\longrightarrow x^*,\, k\longrightarrow +\infty$ we conclude that there exists $n_0\in \mathbb {N},\,n_0\ge 1$ such that $u_{n_0}\in B(u^*,\rho )$ and $F(u^*)\le F(u_{n_0})<F(u^*)+\eta ,$ moreover

$$\begin{aligned} \Vert x^*-x_{n_0}\Vert +3\sqrt{\frac{F(u_{n_0})-F(u^*)}{a}}+\frac{9b}{4a}\varphi (F(u_{n_0})-F(u^*))<\frac{\rho }{c_1+c_2}. \end{aligned}$$

Hence, Corollary 3.1 and consequently Lemma 3.1 can be applied to the sequence $({\mathcal {U}}_n)_{n\in \mathbb {N}},$ ${\mathcal {U}}_n=u_{n_0+n}.$

Thus, according to Lemma 3.1, $({\mathcal {U}}_n)_{n\in \mathbb {N}}$ converges to a point $(\overline{x},\overline{x})\in {{\,\textrm{crit}\,}}(F),$ consequently $(u_n)_{n\in \mathbb {N}}$ converges to $(\overline{x},\overline{x}).$ Then, since $\omega ((u_n)_{n\in \mathbb {N}})=\{(\overline{x},\overline{x})\}$ one has $x^*=\overline{x}.$ Hence, $(x_n)_{n\in \mathbb {N}}$ converges to $x^*$, $(u_n)_{n\in \mathbb {N}}$ converges to $u^*$ and $u^*\in {{\,\textrm{crit}\,}}(F).$ $\square$

Abstract convergence rates in terms of the KL exponent

The following lemma was established in [20] and will be crucial in obtaining our convergence rates (see also [5] for different techniques).

Lemma B.1

( [20] Lemma 15) Let $(e_n)_{n\in \mathbb {N}}$ be a monotonically decreasing positive sequence converging to 0. Assume further that there exist the natural numbers $l_0\ge 1$ and $n_0\ge l_0$ such that for every $n\ge n_0$ one has

$$\begin{aligned} e_{n-l_0}-e_n\ge C_0 e_n^{2\theta } \end{aligned}$$

(36)

where $C_0>0$ is some constant and $\theta \in [0,1).$ Then following statements are true:

• (i) if $\theta =0,$ then $(e_n)_{n\ge n_0}$ converges in finite time;

• (ii) if $\theta \in \left( 0,\frac{1}{2}\right]$, then there exists $C_1>0$ and $Q\in [0,1)$, such that for every $n\ge n_0$

  $$\begin{aligned} e_n\le C_1 Q^n; \end{aligned}$$

• (iii) if $\theta \in \left[ \frac{1}{2},1\right)$, then there exists $C_2>0$, such that for every $n\ge n_0+l_0$

  $$\begin{aligned} e_n\le C_2(n-l_0+1)^{-\frac{1}{2\theta -1}}. \end{aligned}$$

Now we are ready to prove Theorem 3.2.

Proof of Theorem 3.2

The fact that the sequence $(x_n)_{n\in \mathbb {N}}$ converges to $x^*$, $(u_n)_{n\in \mathbb {N}}$ converges to $u^*$ and $u^*\in {{\,\textrm{crit}\,}}(F)$ follows from Theorem 3.1. We divide the proof of the statements (a)-(c) into two cases.

Case I. Assume that there exists $\overline{n}\in \mathbb {N}$ such that $F(u_{\overline{n}})=F(u^*).$ According to (H4) there exists $(u_{n_j})\subseteq (u_n)$ such that $u_{n_j}\longrightarrow u^*,\,F(u_{n_j})\longrightarrow F(u^*),\,j\longrightarrow +\infty .$ Now, $F(u_{n_j})=F(u^*)$ for all $n_j\ge \overline{n}$ since the sequence $(F(u_{n_j}))_{j\in \mathbb {N}}$ is decreasing, and hence $F(u^*)=F(u_{\overline{n}})\ge F(u_{n_j})\ge \lim _{j\longrightarrow +\infty }F(u_{n_j})=F(u^*).$ Further, for every $n\ge \overline{n}$ there exists $j_0\in \mathbb {N}$ such that $n\le n_{j_0}$, consequently $F(u^*)=F(u_{\overline{n}})\ge F(n)\ge F(u_{n_{j_0}})=F(u^*).$ In other words, $F(u_n)=F(u^*)$ for all $n\ge \overline{n}.$ From (H1) we get that for all $n\ge \overline{n}$

$$\begin{aligned} \Vert x_{n+1}-x_n\Vert ^2\le \frac{1}{a}(F(u_n)-F(u_{n+1})=F(u^*)-F(u^*)=0; \end{aligned}$$

hence, $x_{n+1}=x_n$ for all $n\ge \overline{n}.$ But $x_n\longrightarrow x^*,\,n\longrightarrow +\infty$, hence $x_n=x^*$ for all $n\ge \overline{n}$.

Then, $u_n=(x_n,x_n)=(x^*,x^*)$ for all $n\ge \overline{n}$. Consequently, $(F(u_n))_{n\in \mathbb {N}},(x_n)_{n\in \mathbb {N}}$ and $(u_n)_{n\in \mathbb {N}}$ converge in a finite number of steps and this concludes $(a)-(c)$.

Case II. We assume that $F(u_n)>F(u^*)$ for all $n\in \mathbb {N}.$ Now, by using (H2) and (H1) we get

$$\begin{aligned} \mathop {{{\,\textrm{dist}\,}}}\nolimits ^2(0,{\partial }F(u_n))&\le b^2(\Vert x_n-x_{n-1}\Vert +\Vert x_{n-1}-x_{n-2}\Vert )^2\nonumber \\&\le 2b^2(\Vert x_n-x_{n-1}\Vert ^2+\Vert x_{n-1}-x_{n-2}\Vert ^2)\nonumber \\&\le \frac{2b^2}{a}((F(u_{n-1})-F(u_n))+(F(u_{n-2})-F(u_{n-1})))\nonumber \\&=\frac{2b^2}{a}((F(u_{n-2})-F(u^*))-(F(u_n)-F(u^*))), \end{aligned}$$

(37)

for all $n\ge 2.$

Now, according to (H4) there exists $(u_{n_j})\subseteq (u_n)$ such that

$$\begin{aligned} u_{n_j}\longrightarrow u^*,\,F(u_{n_j})\longrightarrow F(u^*),\,j\longrightarrow +\infty . \end{aligned}$$

Combining the above fact with the facts that $(F(u_n))$ is non-increasing and $u_n\longrightarrow u^*,\,n\longrightarrow +\infty$, we conclude that there exists $\overline{n}\in \mathbb {N},\,\overline{n}\ge 2$ such that $F(u^*)<F(u_n)<F(u^*)+\eta$ and $u_n\in B(u^*,\varepsilon )$ for all $n\ge \overline{n}.$ So, since the function F has the Kurdyka–Łojasiewicz property with an exponent $\theta \in [0,1)$ at $u^*$, we can apply the KL-inequality and we get

$$\begin{aligned} \mathop {{{\,\textrm{dist}\,}}}\nolimits ^2(0, {\partial }F(u_n)) \ge \frac{1}{K^2}(F(u_n)-F(u^*))^{2\theta }, \text{ for } \text{ all } n\ge \overline{n}. \end{aligned}$$

(38)

Hence, (37) and (38) yield

$$\begin{aligned} \frac{a}{2b^2K^2}(F(u_n)-F(u^*))^{2\theta }\le (F(u_{n-2})-F(u^*))-(F(u_n)-F(u^*)), \text{ for } \text{ all } n\ge \overline{n}.\nonumber \\ \end{aligned}$$

(39)

Further, using (H4) again, we have $F(u_{n_j})\longrightarrow F(u^*),\,j\longrightarrow +\infty$ and $(F(u_n))$ is non-increasing which leads to

$$\begin{aligned} \lim _{n\longrightarrow +\infty }F(u_n)-F(u^*)=0. \end{aligned}$$

Let us denote $e_n=F(u_n)-F(u^*).$ Then $(e_n)_{n\in \mathbb {N}}$ is a monotonically decreasing positive sequence converging to 0. Further from (39) we have that there exist the natural numbers $l_0=2$ and $\overline{n}\ge l_0$ such that for every $n\ge \overline{n}$ one has

$$\begin{aligned} e_{n-l_0}-e_n\ge C_0 e_n^{2\theta }, \end{aligned}$$

where $C_0=\frac{a}{2b^2K^2}>0.$ Consequently, Lemma B.1 can be applied. Let $\theta =0$. Then, the sequence $(F(u_n)-F(u^*))$ converges in a finite number of steps, that is $F(u_n)=F(u^*)$ after and index $n_1\in \mathbb {N}$. Then, according to Case I. $(x_n)$ and $(u_n)$ converges in a finite number of steps and this concludes (a).

Further, according to (28) we have

$$\begin{aligned} 3\Vert x_{k+1}-x_{k}\Vert \le \Vert x_{k}-x_{k-1}\Vert +\Vert x_{k-1}-x_{k-2}\Vert +\frac{9bK}{4a(1-\theta )}(e_k^{1-\theta }-e_{k+1}^{1-\theta }) \end{aligned}$$

for all $k\ge \overline{n}.$ Summing up the latter relation from $k=n\ge \overline{n}$ to $k=P>n$, we get $\sum _{k=1}^P\Vert x_{k+1}-x_{k}\Vert \le 2\Vert x_{n}-x_{n-1}\Vert +\Vert x_{n-1}-x_{n-2}\Vert -2\Vert x_{P+1}-x_{P}\Vert -\Vert x_{P}-x_{P-1}\Vert + \frac{9bK}{4a(1-\theta )}(e_n^{1-\theta }-e_{P+1}^{1-\theta }).$ Now, from the triangle inequality we have $\Vert x_n-x_{P+1}\Vert \le \sum _{k=1}^P\Vert x_{k+1}-x_{k}\Vert ,$ hence, $\Vert x_n-x_{P+1}\Vert \le 2\Vert x_{n}-x_{n-1}\Vert +\Vert x_{n-1}-x_{n-2}\Vert -2\Vert x_{P+1}-x_{P}\Vert -\Vert x_{P}-x_{P-1}\Vert + \frac{9bK}{4a(1-\theta )}(e_n^{1-\theta }-e_{P+1}^{1-\theta }).$ By neglecting the non-positive terms and letting $P\longrightarrow +\infty$, we get

$$\begin{aligned} \Vert x_n-x^*\Vert \le 2\Vert x_{n}-x_{n-1}\Vert +\Vert x_{n-1}-x_{n-2}\Vert +\frac{9bK}{4a(1-\theta )}e_n^{1-\theta }. \end{aligned}$$

(40)

Now, by using (H1) we get

$$\begin{aligned} 2\Vert x_{n}-x_{n-1}\Vert +\Vert x_{n-1}-x_{n-2}\Vert \le \sqrt{\frac{2(e_{n-1}-e_{n})}{a}}+\sqrt{\frac{e_{n-2}-e_{n-1}}{a}}. \end{aligned}$$

(41)

Finally, combining (40) and (41) we obtain

$$\begin{aligned} \Vert x_n-x^*\Vert \le \sqrt{\frac{2e_{n-1}}{a}}+\sqrt{\frac{e_{n-2}}{a}}+\frac{9bK}{4a(1-\theta )}e_n^{1-\theta }. \end{aligned}$$

(42)

Let $\theta \in \left( 0,\frac{1}{2}\right] .$ Then, there exists $C_1>0$ and $Q\in [0,1)$, such that for every $n\ge \overline{n}$ $e_n\le C_1 Q^n.$ Consequently, (42) yields that for all $n\ge \overline{n}+2$ one has

$$\begin{aligned} \Vert x_n-x^*\Vert \le \sqrt{\frac{2}{a}}\sqrt{C_1}Q^{\frac{n-1}{2}}+ \sqrt{\frac{1}{a}}\sqrt{C_1}Q^{\frac{n-2}{2}}+\frac{9bK}{4a(1-\theta )}C_1^{1-\theta }Q^{(1-\theta )n}. \end{aligned}$$

(43)

Now, since $\theta \le \frac{1}{2}$ and $Q\in [0,1)$ it is obvious that $Q^{(1-\theta )n}\le Q^{\frac{n}{2}},$ and hence, (43) yields

$$\begin{aligned} \Vert x_n-x^*\Vert \le \overline{C} Q^{\frac{n}{2}} \end{aligned}$$

for some $\overline{C}$ and for all $n\ge \overline{n}+2.$

Further, according to (H3)

$$\begin{aligned} \Vert u_n-u^*\Vert&\le (2+c)\Vert x_n-x^*\Vert +c\Vert x_{n-1}-x^*\Vert \le (2+c)C Q^{\frac{n}{2}}+cC Q^{\frac{n-1}{2}}= \overline{C}_1 Q^{\frac{n}{2}}, \end{aligned}$$

(44)

for all $n\ge \overline{n}+3$, where $c=\sup _{n\in \mathbb {N}}(|\alpha _n|+|\beta _n|).$ Hence, (b) is complete if one takes $A_1=\max (C_1,\overline{C},\overline{C}_1)$ and $\overline{k}=\overline{n}+3$.

Let $\theta \in \left[ \frac{1}{2},1\right)$. Then, there exists $C_2>0$, such that for every $n\ge \overline{n}+2$ $e_n\le C_2(n-1)^{-\frac{1}{2\theta -1}}.$ But $(n-1)^{-\frac{1}{2\theta -1}}\le 2^{\frac{1}{2\theta -1}}n^{-\frac{1}{2\theta -1}},$ hence $e_n\le C_2 2^{\frac{1}{2\theta -1}}n^{-\frac{1}{2\theta -1}}=\overline{C}_2 n^{-\frac{1}{2\theta -1}},$ for all $n\ge \overline{n}+2.$

Now, by using (42) we deduce that there exists $\overline{C}_3,\overline{C}_4,\overline{C}_5>0$ such that

$$\begin{aligned} \Vert x_n-x^*\Vert \le \overline{C}_3 n^{-\frac{\frac{1}{2}}{2\theta -1}}+\overline{C}_4 n^{-\frac{\frac{1}{2}}{2\theta -1}}+\overline{C}_5 n^{-\frac{1-\theta }{2\theta -1}}, \end{aligned}$$

or all $n\ge \overline{n}+4.$ Now, since $\theta >\frac{1}{2}$ one has $n^{-\frac{1-\theta }{2\theta -1}}\ge n^{-\frac{\frac{1}{2}}{2\theta -1}}$ we conclude that there exists $\overline{A}_2>0$ such that

$$\begin{aligned} \Vert x_n-x^*\Vert \le \overline{A}_2 n^{-\frac{1-\theta }{2\theta -1}}, \end{aligned}$$

or all $n\ge \overline{n}+4.$

By using the form of $u_n$, we argue as at the case $\theta \in \left( 0,\frac{1}{2}\right]$ in order to obtain that there exists $\overline{A}_3$ such that

$$\begin{aligned} \Vert u_n-u^*\Vert \le \overline{A}_3 n^{-\frac{1-\theta }{2\theta -1}}, \end{aligned}$$

or all $n\ge \overline{n}+5.$

Consequently, (c) holds with $A_2=\max (\overline{C}_2,\overline{A}_2,\overline{A}_3)$ and $\overline{k}=\overline{n}+5.$ $\square$