Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Abstract

We propose an inexact linearized proximal algorithm with an adaptive stepsize, together with its globalized version based on the backtracking line-search, to solve a convex composite optimization problem. Under the assumptions of local weak sharp minima of order $p\ (p\ge 1)$ for the outer convex function and a quasi-regularity condition for the inclusion problem associated to the inner function, we establish superlinear/quadratic convergence results for proposed algorithms. Compared to the linearized proximal algorithms with a constant stepsize proposed in Hu et al. (SIAM J Optim 26(2):1207–1235, 2016), our algorithms own broader applications and higher convergence rates, and the idea of analysis used in the present paper deviates significantly from that of Hu et al. (2016). Numerical applications to the nonnegative inverse eigenvalue problem and the wireless sensor network localization problem indicate that the proposed algorithms are more efficient and robust, and outperform the algorithms in Hu et al. (2016) and some popular algorithms for relevant problems.

Appendix

Proposition A Let $c>0$, $q>1$, and let $X^*\subseteq \mathbb {R}^{n}$ be closed. Let $\{x_{k}\}\subseteq \mathbb {R}^{n}$ be a sequence satisfying

$$\begin{aligned} \Vert x_{k+1}-x_k\Vert \le c \textrm{d}(x_{k},X^{*}) \quad \hbox {and}\quad \textrm{d}(x_{k+1},X^{*})\le c \textrm{d}^q(x_{k},X^{*}) \quad \text {for each}~ k\ge 0.\nonumber \\ \end{aligned}$$

(4.4)

If $\textrm{d}(x_{0},X^{*})< \left( \frac{1}{c}\right) ^\frac{1}{q-1}$, then $\{x_{k}\}$ converges to a point $x^*\in X^*$ at a rate of q.

Proof

Assume that $\textrm{d}(x_{0},X^{*})< \left( \frac{1}{c}\right) ^\frac{1}{q-1}$, and set $\tau := c \textrm{d}^{q-1}(x_{0},X^{*})$. Then $\tau <1$, and

$$\begin{aligned} c\textrm{d}^{q-1}(x_{k},X^{*})\le \tau \quad \hbox {and}\quad \textrm{d}(x_{k+1},X^{*})\le \tau \textrm{d}(x_{k},X^{*}) \quad \hbox {for each } k\ge 0 \end{aligned}$$

(4.5)

because, by the second inequality of (4.4), $\textrm{d}(x_{k},X^{*})\le c^{\frac{q^k-1}{q-1}} \textrm{d}^{q^k}(x_{0},X^{*})= c^{\frac{q^k-1}{q-1}}\left( \frac{\tau }{c}\right) ^\frac{q^k}{q-1}\le \left( \frac{\tau }{c}\right) ^{\frac{1}{q-1}}$ for each k. In particular, we have that $\textrm{d}(x_{k},X^{*})\rightarrow 0$.

Now fix $k\ge 1$. We have from (4.5) that $\textrm{d}(x_{k},X^{*})\le \textrm{d}(x_{k+1},X^{*})+\Vert x_{k+1}-x_k\Vert \le \tau \textrm{d}(x_{k},X^{*})+\Vert x_{k+1}-x_k\Vert$. It follows that $\textrm{d}(x_{k},X^{*})\le \frac{1}{1-\tau }\Vert x_{k+1}-x_k\Vert$. Thus, using (4.4), we check that

$$\begin{aligned} \Vert x_{k+1}-x_k\Vert \le c \textrm{d}( x_{k},X^{*})\le c^2\textrm{d}^{q}( x_{k-1},X^{*})\le \frac{ c^2}{(1-\tau )^{q}} \Vert x_{k}-x_{k-1}\Vert ^{q}. \end{aligned}$$

Set $c_k:=\frac{ c^2}{(1-\tau )^{q}} \Vert x_{k}-x_{k-1}\Vert ^{q-1}$. Then

$$\begin{aligned} c_{k+1}\le \left( \frac{ c^2}{(1-\tau )^{q}}\right) ^{q-1} \Vert x_{k}-x_{k-1}\Vert ^{(q-1)^2} c_k\quad \hbox {and}\quad \Vert x_{k+1}-x_k\Vert \le c_k\Vert x_{k}-x_{k-1}\Vert .\nonumber \\ \end{aligned}$$

(4.6)

Since $\textrm{d}(x_{k},X^{*})\rightarrow 0$, it follows from (4.4) that $\Vert x_{k}-x_{k-1}\Vert \rightarrow 0$, and so $c_k\rightarrow 0$. This, together with (4.6) implies that

$\{x_k\}$ is a Cauchy sequence and so converges to a point $x^*\in X^*$ (as $X^*$ is closed). Furthermore, without loss of generality, we may assume that $c_{k+1}\le c_k\le \frac{1}{2}$ (see the first inequality in (4.6)). Write $d_{k}:=x_{k+1}-x_k$ for simplicity. Then (4.6) implies that $\Vert d_{k+j}\Vert \le c_k^j\Vert d_k\Vert$ for each $j\ge 1$. Therefore, $\frac{\sum _{j=1}^{\infty }\Vert d_{k+j}\Vert }{\Vert d_{k}\Vert }\le \frac{c_k}{1-c_k}\rightarrow 0$, and so $\lim _{k\rightarrow \infty }\frac{\Vert \sum _{j=0}^{\infty } d_{k+j}\Vert }{\Vert d_{k} \Vert }=1,$ because

$$\begin{aligned} 1-\frac{\sum _{j=1}^{\infty }\Vert d_{k+j}\Vert }{\Vert d_{k}\Vert }\le \frac{\Vert \sum _{j=0}^{\infty } d_{k+j}\Vert }{\Vert d_{k}\Vert }\le 1+\frac{\sum _{j=1}^{\infty }\Vert d_{k+j}\Vert }{\Vert d_{k}\Vert }. \end{aligned}$$

Consequently, we conclude that

$$\begin{aligned} \limsup _{k\rightarrow \infty }\frac{\Vert x_{k+1}-x^*\Vert }{\Vert {x_{k}-x^*}\Vert ^{q}}=\limsup _{k\rightarrow \infty }\frac{\Vert \sum _{j=1}^{\infty }d_{k+j}\Vert }{\Vert \sum _{j=0}^{\infty }d_{k+j}\Vert ^{q}}=\limsup _{k\rightarrow \infty }\frac{\Vert d_{k+1}\Vert }{\Vert d_{k}\Vert ^{q}}\le \left( \frac{1}{1-\tau }\right) ^{q}c^2, \end{aligned}$$

which means that $\{x_k\}$ converges to $x^*$ at a rate of q. $\square$