Convergence Results of a New Monotone Inertial Forward–Backward Splitting Algorithm Under the Local Hölder Error Bound Condition

Abstract

In this paper, we introduce a new monotone inertial Forward–Backward splitting algorithm (newMIFBS) for the convex minimization of the sum of a non-smooth function and a smooth differentiable function. The newMIFBS can overcome two negative effects caused by IFBS, i.e., the undesirable oscillations ultimately and extremely nonmonotone, which might lead to the algorithm diverges, for some special problems. We study the improved convergence rates for the objective function and the convergence of iterates under a local Hölder error bound (Local HEB) condition. Also, our study extends the previous results for IFBS under the Local HEB. Finally, we present some numerical experiments for the simplest newMIFBS (hybrid_MIFBS) to illustrate our results.

Appendices

Appendix A: Proof of Lemma 3.2

Proof

From (3.1) with $y: = {y_{k + 1}},x: = {x_k}$, we obtain that

$$\begin{aligned}&F\left( {{z_{k + 1}}} \right) - {F\left( {{x^*}} \right) } + \frac{1}{{2\lambda }}{\left\| {{z_{k + 1}} - {x_k}} \right\| ^2} + \frac{{1 - \mu }}{{2\lambda }}{\left\| {{z_{k + 1}} - {y_{k + 1}}} \right\| ^2} \nonumber \\&\quad \le F\left( {{x_k}} \right) - {F\left( {{x^*}} \right) } + \frac{1}{{2\lambda }}{ {\alpha _k^2} }{\left\| {{z_k} - {x_{k - 1}}} \right\| ^2} \end{aligned}$$

(A.1)

$$\begin{aligned}&\le F\left( {{z_k}} \right) - {F\left( {{x^*}} \right) } + \frac{1}{{2\lambda }}{\left\| {{z_k} - {x_{k - 1}}} \right\| ^2}. \end{aligned}$$

(A.2)

which means that $\sum \nolimits _{k = 1}^{ + \infty } {{{\left\| {{z_{k + 1}} - {y_{k + 1}}} \right\| }^2}} < + \infty ,$ i.e., $\mathop {\lim }\limits _{k \rightarrow \infty } {\left\| {{z_{k + 1}} - {y_{k + 1}}} \right\| ^2} = 0$ and $F\left( {{z_{k+1}}} \right) - {F\left( {{x^*}} \right) } \le \xi$ for $\xi = F\left( {{x_1}} \right) - {F\left( {{x^*}} \right) } + \frac{1}{{2\lambda }}{\left\| {{z_1} - {x_0}} \right\| ^2},$ i.e., ${x_{k + 1}} \in {S_\xi }.$ Based on the nonexpansiveness property of the proximal operator [16], $\nabla f$ is Lipschitz continuous and $\lambda = \frac{\mu }{{{L_f}}}$ with $\mu \in \left( {0,1} \right) ,$ we can deduce that

$$\begin{aligned}&\left\| {\mathrm{{pro}}{\mathrm{{x}}_{\lambda g}}\left( {{z_{k + 1}} - \lambda \nabla f\left( {{z_{k + 1}}} \right) } \right) - {z_{k + 1}}} \right\| \\&= \left\| {\mathrm{{pro}}{\mathrm{{x}}_{\lambda g}}\left( {{z_{k + 1}} - \lambda \nabla f\left( {{z_{k + 1}}} \right) } \right) - \mathrm{{pro}}{\mathrm{{x}}_{\lambda g}}\left( {{y_{k + 1}} - \lambda \nabla f\left( {{y_{k + 1}}} \right) } \right) } \right\| \\&\le \left( {1 + \lambda {L_f}} \right) \left\| {{z_{k + 1}} - {y_{k + 1}}} \right\| \le 2\left\| {{z_{k + 1}} - {y_{k + 1}}} \right\| . \end{aligned}$$

Using Lemma 2.3, we can conclude that

$$\begin{aligned} dist\left( {{z_{k + 1}},X^*} \right) \le {2^{\frac{\theta }{{1 - \theta }}}} \cdot \bar{\tau }{\left\| {{z_{k + 1}} - {y_{k + 1}}} \right\| ^{\frac{\theta }{{1 - \theta }}}}. \end{aligned}$$

(A.3)

Applying (3.1) with $y: = {y_{k + 1}},x: = {x^*}$ and using (A.3), we have

$$\begin{aligned}\begin{array}{*{20}{l}} {F\left( {{z_{k + 1}}} \right) - {F\left( {{x^*}} \right) } \le \frac{1}{{2\lambda }}{{\left\| {{y_{k + 1}} - {x^*}} \right\| }^2} - \frac{1}{{2\lambda }}{{\left\| {{z_{k + 1}} - {x^*}} \right\| }^2}}\\ { = \frac{1}{{2\lambda }}{{\left\| {{z_{k + 1}} - {y_{k + 1}}} \right\| }^2} + \frac{1}{\lambda }\left\langle {{y_{k + 1}} - {z_{k + 1}},{z_{k + 1}} - {x^*}} \right\rangle }\\ { \le \frac{1}{{2\lambda }}{{\left\| {{z_{k + 1}} - {y_{k + 1}}} \right\| }^2} + \frac{1}{\lambda }\left\| {{z_{k + 1}} - {y_{k + 1}}} \right\| \cdot dist\left( {{z_{k + 1}},{X^*}} \right) }\\ { \le \frac{1}{{2\lambda }}{{\left\| {{z_{k + 1}} - {y_{k + 1}}} \right\| }^2} + {2^{\frac{\theta }{{1 - \theta }}}}\frac{{\bar{\tau }}}{\lambda }{{\left\| {{z_{k + 1}} - {y_{k + 1}}} \right\| }^{\frac{1}{{1 - \theta }}}} \le \tilde{\tau }{{\left\| {{z_{k + 1}} - {y_{k + 1}}} \right\| }^{\frac{1}{{1 - \theta }}}},} \end{array}\end{aligned}$$

where $\tilde{\tau }= \frac{{1 + {2^{\frac{1}{{1 - \theta }}}}\bar{\tau }}}{{2\lambda }}.$ $\square$

Appendix B: Proof of Theorem 4.1

Proof

From (3.1) with $y: = {x_k},x: = {x^*}$, we obtain that

$$\begin{aligned} F\left( {{x_{k + 1}}} \right) \le F\left( {{x^ * }} \right) + \frac{1}{{2\lambda }}{\left\| {{x_k} - {x^ * }} \right\| ^2} - \frac{1}{{2\lambda }}{\left\| {{x_{k + 1}} - {x^ * }} \right\| ^2} - \frac{{1 - \mu }}{{2\lambda }}{\left\| {{x_{k + 1}} - {x_k}} \right\| ^2} \end{aligned}$$

(B.1)

and with $y: = {x_k},x: = {x_k},$ we obtain that

$$\begin{aligned} F\left( {{x_{k + 1}}} \right) \le F\left( {{x_k}} \right) - \frac{1}{{2\lambda }}{\left\| {{x_{k + 1}} - {x_k}} \right\| ^2} - \frac{{1 - \mu }}{{2\lambda }}{\left\| {{x_{k + 1}} - {x_k}} \right\| ^2} \end{aligned}$$

(B.2)

Similar with the proof of Lemma 3.2, we have

$$\begin{aligned} F\left( {{x_{k + 1}}} \right) - F\left( {{x^ * }} \right) \le \tau {\left\| {{x_{k + 1}} - {x_k}} \right\| ^{\frac{1}{{1 - \theta }}}} \end{aligned}$$

(B.3)

Consider the Local HEB condition with $\theta \in \left[ {0,\frac{1}{2}} \right) .$ Multiplying (B.2) by ${k^\alpha },$ where $\alpha \in \left[ {0,1} \right]$ is a constant, we obtain

$$\begin{aligned}&{\left( {k + 1} \right) ^\alpha }\left( {F\left( {{x_{k + 1}}} \right) - F\left( {{x^ * }} \right) } \right) + \frac{1}{{2\lambda }}{k^\alpha }{\left\| {{x_{k + 1}} - {x_k}} \right\| ^2} + \frac{{1 - \mu }}{{2\lambda }}{k^\alpha }{\left\| {{x_{k + 1}} - {x_k}} \right\| ^2} \nonumber \\&\quad \le {k^\alpha }\left( {F\left( {{x_k}} \right) - F\left( {{x^ * }} \right) } \right) + \left( {{{\left( {k + 1} \right) }^\alpha } - {k^\alpha }} \right) \left( {F\left( {{x_{k + 1}}} \right) - F\left( {{x^ * }} \right) } \right) \end{aligned}$$

(B.4)

where

$$\begin{aligned}\begin{array}{l} \left( {{{\left( {k + 1} \right) }^\alpha } - {k^\alpha }} \right) \left( {F\left( {{x_{k + 1}}} \right) - F\left( {{x^*}} \right) } \right) \le \tau \left( {{{\left( {1 + \frac{1}{k}} \right) }^\alpha } - 1} \right) {k^\alpha }{\left\| {{x_{k + 1}} - {x_k}} \right\| ^{\frac{1}{{1 - \theta }}}}\\ \le \frac{{1 - 2\theta }}{{2\left( {1 - \theta } \right) }}{\left( {\frac{\tau }{\varepsilon }} \right) ^{\frac{{2\left( {1 - \theta } \right) }}{{1 - 2\theta }}}} \cdot {\varPsi _k} + \frac{{{\varepsilon ^{2\left( {1 - \theta } \right) }}}}{{2\left( {1 - \theta } \right) }}{k^\alpha }\left\| {{x_{k + 1}} - {x_k}} \right\| ^2 \end{array}\end{aligned}$$

and ${\varPsi _k} = {\left( {{{\left( {1 + \frac{1}{k}} \right) }^\alpha } - 1} \right) ^{\frac{{2\left( {1 - \theta } \right) }}{{1 - 2\theta }}}} \cdot {k^\alpha } = O\left( {{k^{\alpha - 1}}} \right) .$ Hence, let $\frac{{{\varepsilon ^{2\left( {1 - \theta } \right) }}}}{{2\left( {1 - \theta } \right) }} = \frac{1}{{2\lambda }}$ and $\alpha = \frac{1}{{1 - 2\theta }},$ then, ${\varPsi _k} = O\left( {\frac{1}{k}} \right)$ and

$$\begin{aligned}&{k^\alpha }\frac{{1 - \mu }}{{2\lambda }}{\left\| {{x_{k + 1}} - {x_k}} \right\| ^2}\nonumber \\&\quad \le {k^\alpha }\left( {F\left( {{x_k}} \right) - F\left( {{x^ * }} \right) } \right) - {\left( {k + 1} \right) ^\alpha }\left( {F\left( {{x_{k + 1}}} \right) - F\left( {{x^ * }} \right) } \right) \nonumber \\&\quad \quad + \frac{{1 - 2\theta }}{{2\left( {1 - \theta } \right) }}{\left( {\frac{\tau }{\varepsilon }} \right) ^{\frac{{2\left( {1 - \theta } \right) }}{{1 - 2\theta }}}}{\varPsi _k}, \end{aligned}$$

(B.5)

which means that

$$\begin{aligned}\sum \nolimits _{k = 1}^{ N } {{k^\alpha }{{\left\| {{x_{k + 1}} - {x_k}} \right\| }^2}} \le C\ln \left( {N + 1} \right) .\end{aligned}$$

Using (B.3), then

$$\begin{aligned}\sum \nolimits _{k = 1}^{ N } {{k^\alpha }{{\left( {F\left( {{x_{k + 1}}} \right) - F\left( {{x^*}} \right) } \right) }^{2\left( {1 - \theta } \right) }}} \le C\ln \left( {N + 1} \right) .\end{aligned}$$

Further, using the monotonic of $\left\{ {F\left( {{x_k}} \right) - F\left( {{x^ * }} \right) } \right\} ,$ we obtain that

$$\begin{aligned}F\left( {{x_{N + 1}}} \right) - F\left( {{x^*}} \right) = O\left( {\root 2\left( {1 - \theta } \right) \of {{\frac{{\ln \left( {N + 1} \right) }}{{\sum \nolimits _{k = 1}^N {{k^\alpha }} }}}}} \right) = O\left( {\frac{{\root 2\left( {1 - \theta } \right) \of {{\ln \left( {N + 1} \right) }}}}{{{k^{\frac{1}{{1 - 2\theta }}}}}}} \right) \end{aligned}$$

The proof for the case that $\theta = \frac{1}{2}$ is more simple. The inequality (B.3) now become

$$\begin{aligned} F\left( {{x_{k + 1}}} \right) - F\left( {{x^ * }} \right) \le \tau {\left\| {{x_{k + 1}} - {x_k}} \right\| ^2}. \end{aligned}$$

(B.6)

Using above inequality to (B.2), we have

$$\begin{aligned}&\left( {1 + \frac{1}{\tau }} \right) \left( {F\left( {{x_{k + 1}}} \right) - F\left( {{x^ * }} \right) } \right) \le F\left( {{x_{k + 1}}} \right) - F\left( {{x^ * }} \right) \\&\quad + \frac{1}{{2\lambda }}{\left\| {{x_{k + 1}} - {x_k}} \right\| ^2} \le F\left( {{x_k}} \right) - F\left( {{x^ * }} \right) ,\end{aligned}$$

which means that the sequence $\left\{ {F\left( {{x_k}} \right) - F\left( {{x^ * }} \right) } \right\}$ generated by FBS is Q-linear convergent.

The point (iii) is trivially. $\square$