Inertial Forward–Backward Algorithms with Perturbations: Application to Tikhonov Regularization

Abstract

In a Hilbert space, we analyze the convergence properties of a general class of inertial forward–backward algorithms in the presence of perturbations, approximations, errors. These splitting algorithms aim to solve, by rapid methods, structured convex minimization problems. The function to be minimized is the sum of a continuously differentiable convex function whose gradient is Lipschitz continuous and a proper lower semicontinuous convex function. The algorithms involve a general sequence of positive extrapolation coefficients that reflect the inertial effect and a sequence in the Hilbert space that takes into account the presence of perturbations. We obtain convergence rates for values and convergence of the iterates under conditions involving the extrapolation and perturbation sequences jointly. This extends the recent work of Attouch–Cabot which was devoted to the unperturbed case. Next, we consider the introduction into the algorithms of a Tikhonov regularization term with vanishing coefficient. In this case, when the regularization coefficient does not tend too rapidly to zero, we obtain strong ergodic convergence of the iterates to the minimum norm solution. Taking a general sequence of extrapolation coefficients makes it possible to cover a wide range of accelerated methods. In this way, we show in a unifying way the robustness of these algorithms.

A Appendix

In this section, we present some auxiliary lemmas that are used throughout the paper.

Lemma A.1

([17, Lemma 5.14]) Let $(a_k)$ be a sequence of nonnegative numbers such that $a_k^2 \le c^2 + \sum _{j=1}^k \beta _j a_j$ for all $k\in {\mathbb {N}}$, where $(\beta _j )$ is a summable sequence of nonnegative numbers, and $c\ge 0$. Then, $\displaystyle a_k \le c + \sum \nolimits _{j=1}^{+\infty } \beta _j$ for all $k\in {\mathbb {N}}$.

Lemma A.2

([40, Opial’s Lemma]) Let S be a nonempty subset of ${\mathcal {H}}$, and $(x_k)$ a sequence of elements of ${\mathcal {H}}$. Assume that

(i):

every sequential weak cluster point of $(x_k)$, as $k\rightarrow +\infty$, belongs to S;

(ii):

for every $z\in S$, $\lim _{k\rightarrow +\infty }\Vert x_k-z\Vert$ exists.

Then $(x_k)$ converges weakly as $k\rightarrow +\infty$ to a point in S.

The following result allows us to establish the summability of a sequence $(a_k)$ satisfying some suitable inequality.

Lemma A.3

([1, Lemma 23]) Let us give a nonnegative sequence $(\alpha _k)$ satisfying $(K_0)$. Let $(t_k)$ be the sequence defined by $t_k=1+\sum _{i=k}^{+\infty }\prod _{j=k}^i\alpha _j$. Let $(a_k)$ and $(\omega _k)$ be two sequences of nonnegative numbers such that, for every $k\ge 0$, the following inequality is satisfied: $a_{k+1}\le \alpha _ka_k+\omega _k$.

Then, if $\sum _{k=0}^{+\infty }t_{k+1}\omega _k<+\infty$, we conclude that $\sum _{k=0}^{+\infty }a_k<+\infty$.

The next lemma provides an estimate of the convergence rate of a sequence that is summable with respect to weights.

Lemma A.4

([1, Lemma 22]) Let $(\tau _k)$ be a nonnegative sequence such that $\sum _{k=1}^{+\infty } \tau _{k}=+\infty$. Assume that $(\varepsilon _k)$ is a nonnegative and nonincreasing sequence satisfying $\sum _{k=1}^{+\infty } \tau _{k}\,\varepsilon _k<+\infty$.

Then, we have $\varepsilon _k=o\left( \frac{1}{\sum _{i=1}^k \tau _i}\right) \quad \text{ as } k\rightarrow +\infty .$