Accelerated proximal stochastic variance reduction for DC optimization

Abstract

In this article, we study an important class of stochastic difference-of-convex (SDC) programming whose objective is given in the form of the sum of a continuously differentiable convex function, a simple convex function and a continuous concave function. Recently, a proximal stochastic variance reduction difference-of-convex algorithm (Prox-SVRDCA) (Xu et al., 2019) is developed for this problem. And, Prox-SVRDCA reduces to the proximal stochastic variance reduction gradient (Prox-SVRG) (Xiao and Zhang, 2014) as the continuous concave function is disappeared, and hence Prox-SVRDCA is potentially slow in practice. Inspired by recently proposed acceleration techniques, an accelerated proximal stochastic variance reduction difference-of-convex algorithm (AProx-SVRDCA) is proposed. Different from Prox-SVRDCA, an extrapolation acceleration step that involves the latest two iteration points is incorporated in AProx-SVRDCA. The experimental results show that, for a fairly general choice of the extrapolation parameter, the acceleration will be achieved for AProx-SVRDCA. Then, a rigorous theoretical analysis is presented. We first show that any accumulation point of the generated iteration sequences is a stationary point of the objective function. Furthermore, different from the traditional convergence analysis in the existing nonconvex stochastic optimizations, a global convergence property with respect to the generated sequences is established under the assumption: the Kurdyka-Łojasiewicz property together with the continuity and differentiability of the concave part in objective function. To the best of our knowledge, this is the first time that the acceleration trick is incorporated into nonconvex nonsmooth SDC programming. Finally, extended experimental results show the superiority of our proposed algorithm.

Appendices

Appendix

The proof of Lemma 3

Proof

By taking expectation with respect to $i_k$, and noting that $i_k$ is dependent of $x^s_k, x^s_{k-1}$ and ${\tilde{x}}^{s-1}$, we obtain

$$\begin{aligned} \begin{aligned}&{\mathbb {E}}_{i_k}||v^s_k-\nabla f(y^s_k)||^2\\&\quad ={\mathbb {E}}_{i_k}||\nabla f_{i_k}(y^s_k)-\nabla f_{i_k}({\tilde{x}}^{s-1}) +\nabla f({\tilde{x}}^{s-1})-\nabla f(y^s_k)||^2 \\&\quad \le {\mathbb {E}}_{i_k}||\nabla f_{i_k}(y^s_k)-\nabla f_{i_k}({\tilde{x}}^{s-1})||^2 \\&\quad \le L^2||y^s_k-{\tilde{x}}^{s-1}||^2 \\&\quad \le 2L^2||x^s_k-{\tilde{x}}^{s-1}||^2+2L^2\beta ^2_k||x^s_k-x^s_{k-1}||^2 \end{aligned} \end{aligned}$$

The first inequality holds by ${\mathbb {E}}[X-{\mathbb {E}}[X]]^2\le {\mathbb {E}}[X^2]$, X is a random variable. The second inequality follows from $\nabla f_i$ is Lipschitz continuously. The last inequality comes from the definition of $y^s_k$ and Cauchy–Schwarz Inequality. This completes the proof. $\square$

The proof of Lemma 4

Proof

Firstly, it follows from the strong convexity of minimization (2) that

$$\begin{aligned} \begin{aligned}&R_1(x^s_{k+1})+\frac{L}{2}||x^s_{k+1}-y^s_k||^2+\langle v^s_k -\partial R_2(x^s_k),x^s_{k+1}-x^s_k\rangle \\&\quad \le R_1(x^s_k)+\frac{L}{2}||x^s_k-y^s_k||^2-\frac{L}{2}||x^s_{k+1}-x^s_k||^2 \end{aligned} \end{aligned}$$

(16)

Then, we have

$$\begin{aligned} \begin{aligned}&F(x^s_{k+1}) \\&\quad = f(x^s_{k+1})+R_1(x^s_{k+1})-R_2(x^s_{k+1}) \\&\quad \le f(y^s_k)+\langle \nabla f(y^s_k)-v^s_k,x^s_{k+1}-x^s_k\rangle \\&\qquad +\frac{L}{2}||x^s_{k+1}-y^s_k||^2+R_1(x^s_{k+1}) \\&\qquad - R_2(x^s_k)+\langle \nabla f(y^s_k),x^s_k-y^s_k\rangle +\langle v^s_k -\partial R_2(x^s_k),x^s_{k+1}-x^s_k\rangle \\&\quad \le f(x^s_k)+\langle \nabla f(y^s_k)-v^s_k,x^s_{k+1}-x^s_k\rangle \\&\qquad +\frac{L}{2}||x^s_{k+1}-y^s_k||^2+R_1(x^s_{k+1}) \\&\qquad - R_2(x^s_k)+\langle v^s_k-\partial R_2(x^s_k),x^s_{k+1}-x^s_k\rangle \\&\quad \le F(x^s_k)+\langle \nabla f(y^s_k)-v^s_k,x^s_{k+1}-x^s_k\rangle \\&\qquad +\frac{L\beta ^2_k}{2}||x^s_k-x^s_{k-1}||^2 \\&\qquad -\frac{L}{2}||x^s_{k+1}-x^s_k||^2 \end{aligned} \end{aligned}$$

(17)

where the first inequality follows from the assumptions that f is L-smooth and the convexity of $R_2$. The second inequality comes from the assumption that the convexity of f. The last inequality holds by (15). To reflect the variance induced by stochastic sampling, we performing the following scaling on the second term of the last inequality in (16).

$$\begin{aligned} \begin{aligned}&\langle \nabla f(y^s_k)-v^s_k,x^s_{k+1}-x^s_k\rangle \\&\quad = \langle \nabla f(y^s_k)-v^s_k,x^s_{k+1}-{\bar{x}}^s_{k+1}\rangle \\&\qquad +\langle \nabla f(y^s_k)-v^s_k,{\bar{x}}^s_{k+1}-x^s_k\rangle \\&\quad \le ||\nabla f(y^s_k)-v^s_k||\cdot ||x^s_{k+1}-{\bar{x}}^s_{k+1}||\\&\qquad +\langle \nabla f(y^s_k)-v^s_k,{\bar{x}}^s_{k+1}-x^s_k\rangle \\&\quad \le \frac{1}{L}||\nabla f(y^s_k)-v^s_k||^{2}+\langle \nabla f(y^s_k) -v^s_k,{\bar{x}}^s_{k+1}-x^s_k\rangle \\ \end{aligned} \end{aligned}$$

(18)

where ${\bar{x}}^s_{k+1}=prox_{\frac{1}{L}R_1}\big (y^s_k-\frac{1}{L}(\nabla f(y^s_k)-\partial R_2(x^s_k))\big )$. The first inequality holds by Cauchy–Schwarz Inequality. The second inequality holds by the nonexpansiveness property of proximal operator. Note that $i_k$ is independent of $x^s_{k_1}, x^s_k$, and ${\tilde{x}}^{s-1}$. By substituting (17) in (16) and taking expectation over $i_k$ we have

$$\begin{aligned} \begin{aligned} {\mathbb {E}}_{i_k}[F(x^s_{k+1})]&\le F(x^s_k)+\frac{1}{L}{\mathbb {E}}_{i_k} ||\nabla f(y^s_k)-v^s_k||^2\\&\quad +\frac{L\beta ^2_k}{2}||x^s_k-x^s_{k-1}||^2 -\frac{L}{2}{\mathbb {E}}_{i_k}||x^s_{k+1}-x^s_k||^2 \\&\le F(x^s_k)+2L||x^s_k-{\tilde{x}}^{s-1}||^2\\&\quad +\frac{5L\beta ^2_k}{2} ||x^s_k-x^s_{k-1}||^2-\frac{L}{2}{\mathbb {E}}_{i_k}||x^s_{k+1}-x^s_k||^2 \end{aligned} \end{aligned}$$

where in the second inequality we use Lemma 3. This completes the proof. $\square$

The proof of Lemma 5

Proof

By Lemma 4, we have the following inequality

$$\begin{aligned} \begin{aligned} {\mathbb {E}}_{i_k}[F(x^s_{k+1})]&\le F(x^s_k)+2L||x^s_k-{\tilde{x}}^{s-1}||^2\\&\quad +\frac{5L\beta ^2_k}{2}||x^s_k -x^s_{k-1}||^2-\frac{L}{2}{\mathbb {E}}_{i_k}||x^s_{k+1}-x^s_k||^2 \end{aligned} \end{aligned}$$

By adding $c_{k+1}{\mathbb {E}}_{i_k}[||x^s_{k+1}-x^s_k||^2+||x^s_{k+1}-{\tilde{x}}^{s-1}||^2]$ on both sides of this inequality,

$$\begin{aligned} \begin{aligned}&{\mathbb {E}}_{i_k}[F(x^s_{k+1})+c_{k+1}\big (||x^s_{k+1}-x^s_k||^2+||x^s_{k+1} -{\tilde{x}}^{s-1}||^2\big )] \\&\quad \le F(x^s_k)+2L||x^s_k -{\tilde{x}}^{s-1}||^2+\frac{5L\beta ^2_k}{2}||x^s_k-x^s_{k-1}||^2 \\&\qquad +(c_{k+1}-\frac{L}{2}){\mathbb {E}}_{i_k}||x^s_{k+1}-x^s_k||^2 +c_{k+1}{\mathbb {E}}_{i_k}||x^s_{k+1}-{\tilde{x}}^{s-1}||^2 \end{aligned} \end{aligned}$$

(19)

We scale the last term on the right hand side of the above inequality,

$$\begin{aligned} \begin{aligned}&c_{k+1}{\mathbb {E}}_{i_k}||x^s_{k+1}-{\tilde{x}}^{s-1}||^2 \\&\quad = c_{k+1}[{\mathbb {E}}_{i_k}||x^s_{k+1}-x^s_k||^2\\&\qquad +2{\mathbb {E}}_{i_k}\langle x^s_{k+1}-x^s_k,x^s_k-{\tilde{x}}^{s-1}\rangle +||x^s_k-{\tilde{x}}^{s-1}||^2] \\&\quad \le c_{k+1}[{\mathbb {E}}_{i_k}||x^s_{k+1}-x^s_k||^2+\frac{1}{a} {\mathbb {E}}_{i_k}||x^s_{k+1}-x^s_k||^2\\&\qquad +a||x^s_k-{\tilde{x}}^{s-1}|^2+||x^s_k -{\tilde{x}}^{s-1}||^2] \\&\quad = c_{k+1}(1+\frac{1}{a}){\mathbb {E}}_{i_k}||x^s_{k+1}-x^s_k||^2\\&\qquad +c_{k+1}(1+a)||x^s_k-{\tilde{x}}^{s-1}||^2 \end{aligned} \end{aligned}$$

(20)

where $a>0$, the inequality comes from Cauchy–Schwarz Inequality and the Young’s Inequality. Substituting (19) in (18), then we have

$$\begin{aligned} \begin{aligned}&{\mathbb {E}}_{i_k}[F(x^s_{k+1})+c_{k+1}\big (||x^s_{k+1}-x^s_k||^2 +||x^s_{k+1}-{\tilde{x}}^{s-1}||^2\big )] \\&\quad \le F(x^s_k)+\frac{5L\beta ^2_k}{2}||x^s_k-x^s_{k-1}||^2\\&\qquad +[2L +c_{k+1}(1+a)]||x^s_k-{\tilde{x}}^{s-1}||^2 \\&\qquad +\left[ c_{k+1}\left( 2+\frac{1}{a}\right) -\frac{L}{2}\right] {\mathbb {E}}_{i_k}||x^s_{k+1}-x^s_k||^2 \\&\quad = F(x^s_k)+c_k[||x^s_k-x^s_{k-1}||^2+||x^s_k-{\tilde{x}}^{s-1}||^2]\\&\qquad -[2L+c_{k+1}(1+a)]||x^s_k-x^s_{k-1}||^2 \\&\qquad -\frac{5L\beta ^2_k}{2}||x^s_k-{\tilde{x}}^{s-1}||^2-\left[ \frac{L}{2} -c_{k+1}\left( 2+\frac{1}{a}\right) \right] \\&\qquad {\mathbb {E}}_{i_k}||x^s_{k+1}-x^s_k||^2 \end{aligned} \end{aligned}$$

(21)

By definition of $c_k$ and $c_k\le \frac{La}{2(1+2a)}$, $\forall ~k\in \{0,1,\ldots ,m-1\}$ in each epoch s

$$\begin{aligned} G^s_{k+1}\le G^s_k \end{aligned}$$

On the other hand, by $x^s_{-1}=x^s_0={\tilde{x}}^{s-1}=x^{s-1}_m$, for any s

$$\begin{aligned} \begin{aligned} G^s_m&={\mathbb {E}}[F(x^s_m)+c_m(||x^s_m-x^s_{m-1}||^2+||x^s_m-{\tilde{x}}^{s-1}||^2)] \\&\le {\mathbb {E}}[F(x^s_0)+c_0(||x^s_0-x^s_{-1}||^2+||x^s_0-{\tilde{x}}^{s-1}||^2] \\&={\mathbb {E}}[F({\tilde{x}}^{s-1})] \\&\le {\mathbb {E}}[F(x^{s-1}_m)+c_m(||x^{s-1}_m-x^{s-1}_{m-1}||^2 +||x^{s-1}_m-{\tilde{x}}^{s-2}||^2] \\&= G^{s-1}_m \end{aligned} \end{aligned}$$

(22)

Therefore, the sequence $\{G^s_k\}$ is nonincreasing $\forall k\in \{0,1,\ldots ,m-1\}$ and s. This together with the fact that it is also bounded from below by $\min f$. So, we can obtain that $\{G^s_k\}$ is convergent.

Next, we will present an upper bound on $c_0$. In view of the definition of $c_{k+1}$, we have

$$\begin{aligned} \begin{aligned} c_{k+1}&=\frac{1}{1+a}c_k-\frac{1}{1+a}\left( \frac{5L\beta ^2_k}{2}+2L\right) \\&=\frac{1}{1+a}\left[ \frac{1}{1+a}c_{k-1}-\frac{1}{1+a}\left( \frac{5L\beta ^2_k}{2} +2L\right) \right] \\&\quad -\frac{1}{1+a}\left( \frac{5L\beta ^2_k}{2}+2L\right) \\&=\frac{1}{(1+a)^2}c_{k-1} -\left[ \frac{1}{1+a}+\frac{1}{(1+a)^2}\right] \left( \frac{5L\beta ^2_k}{2}+2L\right) \\&=...... \\&=\frac{1}{(1+a)^{k+1}}c_0-\left[ \frac{1}{1+a}+... +\frac{1}{(1+a)^{k+1}}\right] \\&\quad \left( \frac{5L\beta ^2_k}{2}+2L\right) \\&=\frac{1}{(1+a)^{k+1}}c_0-\frac{1-\left( \frac{1}{1+a} \right) ^{k+1}}{a}\left( \frac{5L\beta ^2_k}{2}+2L\right) \end{aligned} \end{aligned}$$

Because of $c_m=0$, that is, when $k+1=m$, we have

$$\begin{aligned} c_m=0=\frac{1}{(1+a)^m}c_0-\frac{1 -\left( \frac{1}{1+a}\right) ^m}{a}\left( \frac{5L\beta ^2_k}{2}+2L\right) \end{aligned}$$

when $m=\frac{1}{a}$ and $a\rightarrow 0$, we have

$$\begin{aligned} c_0=\frac{(1+a)^m-1}{a}\left( \frac{5L\beta ^2_k}{2}+2L\right) \rightarrow \frac{e-1}{a}\left( \frac{5L\beta ^2_k}{2}+2L\right) \end{aligned}$$

As a result, we can obtain that $c_0\le \frac{e-1}{a}(\frac{5L\beta ^2_k}{2}+2L)$. There we use $(1+a)^\frac{1}{a}\rightarrow e$, when $a\rightarrow 0$. This completes the proof. $\square$