Distributed stochastic gradient tracking methods with momentum acceleration for non-convex optimization

Abstract

We consider a distributed non-convex optimization problem of minimizing the sum of all local cost functions over a network of agents. This problem often appears in large-scale distributed machine learning, known as non-convex empirical risk minimization. In this paper, we propose two accelerated algorithms, named DSGT-HB and DSGT-NAG, which combine the distributed stochastic gradient tracking (DSGT) method with momentum accelerated techniques. Under appropriate assumptions, we prove that both algorithms sublinearly converge to a neighborhood of a first-order stationary point of the distributed non-convex optimization. Moreover, we derive the conditions under which DSGT-HB and DSGT-NAG achieve a network-independent linear speedup. Numerical experiments for a distributed non-convex logistic regression problem on real data sets and a deep neural network on the MNIST database show the superiorities of DSGT-HB and DSGT-NAG compared with DSGT.

Appendices

Appendix A Proof of Lemma 8

Proof

Using $W_\infty W=W_\infty$ and (4c), we obtain

$$\begin{aligned}&\Vert \mathbf {s}_{k+1}-W_\infty \mathbf {s}_{k+1}\Vert ^2\nonumber \\&\quad =\Vert W(\mathbf {s}_{k}+ \mathbf {d}_{k+1}-\mathbf {d}_{k})-W_\infty W(\mathbf {s}_{k}+ \mathbf {d}_{k+1}-\mathbf {d}_{k})\Vert ^2\nonumber \\&\quad =\Vert W\mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2+\Vert (W-W_\infty )(\mathbf {d}_{k+1}- \mathbf {d}_{k})\Vert ^2\nonumber \\&\quad +2\langle (W-W_\infty )\mathbf {s}_k,(W-W_\infty )(\mathbf {d}_{k+1}- \mathbf {d}_{k})\rangle \nonumber \\&\quad \le \delta ^2\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2+\delta ^2\Vert \mathbf {d}_{k+1}- \mathbf {d}_{k}\Vert ^2 +2\langle (W-W_\infty )\mathbf {s}_k,(W-W_\infty )(\mathbf {d}_{k+1}- \mathbf {d}_{k})\rangle , \end{aligned}$$

(A1)

where the inequality follows from Lemma 2(c) and $\Vert W-W_\infty \Vert =\delta$. Notice that

$$\begin{aligned}&\mathbb {E}[\langle (W-W_\infty )\mathbf {s}_k,(W-W_\infty )(\mathbf {d}_{k+1}- \mathbf {d}_{k})\rangle \mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[\langle (W-W_\infty )\mathbf {s}_k,(W-W_\infty )(\nabla \mathbf {f}_{k+1}- \mathbf {d}_{k})\rangle \mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[\langle (W-W_\infty )\mathbf {s}_k,(W-W_\infty )(\nabla \mathbf {f}_{k+1}- \nabla \mathbf {f}_{k})\rangle \mid \mathcal {F}_k]\nonumber \\&\quad +\mathbb {E}[\langle (W-W_\infty )\mathbf {s}_k,(W-W_\infty )(\nabla \mathbf {f}_{k}- \mathbf {d}_{k})\rangle \mid \mathcal {F}_k]. \end{aligned}$$

(A2)

Taking the conditional expectation on (A1) and using (A2) lead to

$$\begin{aligned}&\mathbb {E}[\Vert \mathbf {s}_{k+1}-W_\infty \mathbf {s}_{k+1}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad \le \delta ^2\mathbb {E}[\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_k\Vert ^2\mid \mathcal {F}_k]+ \delta ^2\mathbb {E}[\Vert \mathbf {d}_{k+1}-\mathbf {d}_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +2\mathbb {E}[\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k}-\mathbf {d}_{k})\rangle \mid \mathcal {F}_k]\nonumber \\&\quad +2\mathbb {E}[\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k+1}-\nabla \mathbf {f}_{k})\rangle \mid \mathcal {F}_k]. \end{aligned}$$

(A3)

Consider the second term on the right-hand side of (A3). Note that

$$\begin{aligned}&\mathbb {E}[\Vert \mathbf {d}_{k+1}-\mathbf {d}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[\Vert \mathbf {d}_{k+1}-\nabla \mathbf {f}_{k+1}\Vert ^2\mid \mathcal {F}_k] +\mathbb {E}[\Vert \nabla \mathbf {f}_{k+1}-\mathbf {d}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad \le n\sigma ^2+\mathbb {E}[\Vert \nabla \mathbf {f}_{k+1}-\mathbf {d}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad =n\sigma ^2+\mathbb {E}[\Vert \nabla \mathbf {f}_{k+1}-\nabla \mathbf {f}_{k}\Vert ^2\mid \mathcal {F}_k]+ \mathbb {E}[\Vert \nabla \mathbf {f}_{k}-\mathbf {d}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +2\mathbb {E}[\langle \nabla \mathbf {f}_{k+1}-\nabla \mathbf {f}_{k},\nabla \mathbf {f}_{k}-\mathbf {d}_{k}\rangle \mid \mathcal {F}_k]\nonumber \\&\quad \le 2n\sigma ^2+L^2\mathbb {E}[\Vert \varvec{\theta }_{k+1}-\varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k] +2\mathbb {E}[\langle \nabla \mathbf {f}_{k+1},\nabla \mathbf {f}_{k}-\mathbf {d}_{k}\rangle \mid \mathcal {F}_k], \end{aligned}$$

(A4)

where the last inequality follows from Assumption 1 and $\mathbb {E}[\langle \nabla \mathbf {f}_{k},\nabla \mathbf {f}_{k}-\mathbf {d}_{k}\rangle \mid \mathcal {F}_k]=0$. We note that

$$\begin{aligned}&\mathbb {E}[\Vert \varvec{\theta }_{k+1}-\varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[\Vert W(\varvec{\theta }_k-\alpha (1-\beta )\mathbf {s}_k-\alpha \beta \mathbf {u}_k)-\varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[\Vert (W-I_{np})(\varvec{\theta }_k-W_\infty \varvec{\theta }_{k})-\alpha (1-\beta ) W \mathbf {s}_{k}-\alpha \beta W\mathbf {u}_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad \le 2\Vert W-I_{np}\Vert ^2\mathbb {E}[\Vert \varvec{\theta }_k-W_\infty \varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k]+4\alpha ^2(1-\beta )^2\mathbb {E}[\Vert W\mathbf {s}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +4\alpha ^2\beta ^2\mathbb {E}[\Vert \mathbf {u}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad \le 8\mathbb {E}[\Vert \varvec{\theta }_k-W_\infty \varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k]+4\alpha ^2\beta ^2\mathbb {E}[\Vert \mathbf {u}_{k}\Vert ^2\mid \mathcal {F}_k] +8\alpha ^2(1-\beta )^2\delta ^2\mathbb {E}[\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +8\alpha ^2(1-\beta )^2n\mathbb {E}[\Vert h_k\Vert ^2\mid \mathcal {F}_k] +8\alpha ^2(1-\beta )^2\sigma ^2, \end{aligned}$$

(A5)

where the first inequality uses Lemma 1(c) with $\eta =1$ and the last inequality uses $\Vert W-I_{np}\Vert \le 2$ and (13). Next, we consider the last term on the right-hand side of (A4). From (4), it follows that

$$\begin{aligned} \nabla \mathbf {f}_{k+1}&=\nabla \mathbf {f}(W(\varvec{\theta }_k-\alpha (1-\beta ) \mathbf {s}_k-\alpha \beta \mathbf {u}_{k}))\nonumber \\&=\nabla \mathbf {f}(W(\varvec{\theta }_k-\alpha (1-\beta ) W(\mathbf {s}_{k-1}+\mathbf {d}_{k}-\mathbf {d}_{k-1})-\alpha \beta \mathbf {u}_{k})). \end{aligned}$$

(A6)

Denote

$$\begin{aligned} \underline{\nabla \mathbf {f}}_{k+1}=\nabla \mathbf {f}(W(\varvec{\theta }_k-\alpha (1-\beta ) W(\mathbf {s}_{k-1}+\nabla \mathbf {f}_{k}-\mathbf {d}_{k-1})-\alpha \beta \mathbf {u}_{k})). \end{aligned}$$

(A7)

By Assumption 3,

$$\begin{aligned} \mathbb {E}[\langle \underline{\nabla \mathbf {f}}_{k+1},\nabla \mathbf {f}_k-\mathbf {d}_k\rangle \mid \mathcal {F}_k]=0. \end{aligned}$$

(A8)

In light of Assumption 1 and $\Vert W^2\Vert =1$,

$$\begin{aligned} \Vert \nabla \mathbf {f}_{k+1}-\underline{\nabla \mathbf {f}}_{k+1}\Vert&\le \alpha (1-\beta ) L\Vert W^2(\mathbf {d}_k-\nabla \mathbf {f}_k)\Vert \nonumber \\&\le \alpha (1-\beta ) L\Vert \mathbf {d}_k-\nabla \mathbf {f}_k\Vert . \end{aligned}$$

(A9)

Using (A6)-(A9), one has

$$\begin{aligned} \mathbb {E}[\langle \nabla \mathbf {f}_{k+1},\nabla \mathbf {f}_k-\mathbf {d}_k\rangle \mid \mathcal {F}_k]&=\mathbb {E}[\langle \nabla \mathbf {f}_{k+1}-\underline{\nabla \mathbf {f}}_{k+1},\nabla \mathbf {f}_k-\mathbf {d}_k\rangle \mid \mathcal {F}_k]\nonumber \\&\le \mathbb {E}[\Vert \nabla \mathbf {f}_{k+1}-\underline{\nabla \mathbf {f}}_{k+1}\Vert \Vert \nabla \mathbf {f}_k-\mathbf {d}_k\Vert \mid \mathcal {F}_k]\nonumber \\&\le \alpha (1-\beta ) L\mathbb {E}[\Vert \nabla \mathbf {f}_k-\mathbf {d}_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\le \alpha (1-\beta ) Ln\sigma ^2. \end{aligned}$$

(A10)

Substituting (A5) and (A10) into (A4) yields

$$\begin{aligned}&\mathbb {E}[\Vert \mathbf {d}_{k+1}-\mathbf {d}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad \le 8L^2\mathbb {E}[\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k] +4\alpha ^2\beta ^2 L^2\mathbb {E}[\Vert \mathbf {u}_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +8\alpha ^2(1-\beta )^2 L^2\delta ^2\mathbb {E}[\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_k\Vert ^2\mid \mathcal {F}_k] +8\alpha ^2(1-\beta )^2L^2n\mathbb {E}[\Vert h_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +2(1+\alpha (1-\beta ) L)n\sigma ^2+8\alpha ^2(1-\beta )^2L^2\sigma ^2. \end{aligned}$$

(A11)

Consider the third term on the right-hand side of (A3). Using $W_\infty (W-W_\infty )=0$ and (4c), we have

$$\begin{aligned}&\mathbb {E}[\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k}-\mathbf {d}_{k})\rangle \mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[\langle W\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k}-\mathbf {d}_{k})\rangle \mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[\langle W^2(\mathbf {s}_{k-1}+\mathbf {d}_{k}-\mathbf {d}_{k-1}),(W\!-\!W_\infty )(\nabla \mathbf {f}_{k}-\mathbf {d}_{k})\rangle \mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[\langle W^2\mathbf {d}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k}-\mathbf {d}_{k})\rangle \mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[\langle W^2(\mathbf {d}_{k}-\nabla \mathbf {f}_{k}),(W-W_\infty )(\nabla \mathbf {f}_{k}-\mathbf {d}_{k})\rangle \mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[(\mathbf {d}_{k}-\nabla \mathbf {f}_{k})^T(W_\infty -W^TW^2)(\mathbf {d}_{k}-\nabla \mathbf {f}_{k})\mid \mathcal {F}_k], \end{aligned}$$

(A12)

where the second and third equalities use $\mathbb {E}[\mathbf {d}_k\mid \mathcal {F}_k]=\nabla \mathbf {f}_k$. Since, for $\forall i\ne j\in \mathcal {V}$, it holds that

$$\begin{aligned} \mathbb {E}[\langle d_i(\theta ^i_k,\xi ^i_k)-\nabla f_i(\theta ^i_k), d_j(\theta ^j_k,\xi ^j_k)-\nabla f_j(\theta ^j_k)\rangle \mid \mathcal {F}_k]=0, \end{aligned}$$

we get

$$\begin{aligned}&\mathbb {E}[ (\mathbf {d}_{k}-\nabla \mathbf {f}_{k})^T(W_\infty -W^TW^2)(\mathbf {d}_{k}-\nabla \mathbf {f}_{k})\mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[(\mathbf {d}_{k}-\nabla \mathbf {f}_{k})^T\text {diag}(W_\infty -W^TW^2)(\mathbf {d}_{k}-\nabla \mathbf {f}_{k})\mid \mathcal {F}_k]\nonumber \\&\quad \le \mathbb {E}[(\mathbf {d}_{k}-\nabla \mathbf {f}_{k})^T\text {diag}(W_\infty )(\mathbf {d}_{k}-\nabla \mathbf {f}_{k})\mid \mathcal {F}_k]\nonumber \\&\quad =(1/n)\mathbb {E}[\Vert \mathbf {d}_{k}-\nabla \mathbf {f}_{k}\Vert ^2\mid \mathcal {F}_k], \end{aligned}$$

(A13)

where the inequality uses that $\text {diag}(W^TW^2)$ is nonnegative. Using Assumption 3 and combining (A13) with (A12) obtain

$$\begin{aligned} \mathbb {E}[\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k}-\mathbf {d}_{k})\rangle \mid \mathcal {F}_k]\le \sigma ^2. \end{aligned}$$

(A14)

Consider the fourth term on the right-hand side of (A3). By the Cauchy-schwarz inequality, we have

$$\begin{aligned}&\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k+1}-\nabla \mathbf {f}_{k})\rangle \nonumber \\&\quad \le \Vert W\mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert \Vert (W-W_\infty )(\nabla \mathbf {f}_{k+1}-\nabla \mathbf {f}_{k})\Vert \nonumber \\&\quad \le \delta ^2L\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert \Vert \varvec{\theta }_{k+1}-\varvec{\theta }_{k}\Vert , \end{aligned}$$

(A15)

where the last inequality applies $\Vert W-W_\infty \Vert =\delta$, Lemma 2(c) and Assumption 1. Next, we bound $\Vert \varvec{\theta }_{k+1}-\varvec{\theta }_{k}\Vert$. Note that

$$\begin{aligned}&\Vert \varvec{\theta }_{k+1}-\varvec{\theta }_{k}\Vert \nonumber \\&\quad =\Vert W(\varvec{\theta }_{k}-\alpha (1-\beta )\mathbf {s}_k-\alpha \beta \mathbf {u}_{k})-\varvec{\theta }_{k}\Vert \nonumber \\&\quad =\Vert (W-I_{np})(\varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k})-\alpha (1-\beta )W\mathbf {s}_k-\alpha \beta W\mathbf {u}_k\Vert \nonumber \\&\quad \le 2\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert +\alpha \beta \Vert \mathbf {u}_{k}\Vert +\alpha (1-\beta )\Vert W\mathbf {s}_k\Vert \nonumber \\&\quad \le 2\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert +\alpha \beta \Vert \mathbf {u}_{k}\Vert +\alpha (1-\beta )\delta \Vert \mathbf {s}_k-W_\infty \mathbf {s}_k\Vert +\alpha (1-\beta ) \sqrt{n}\Vert \overline{d}_k\Vert , \end{aligned}$$

(A16)

where the first inequality uses $\Vert W-I_{np}\Vert \le 2$ and $\Vert W\Vert =1$. Substituting (A16) into (A15) yields

$$\begin{aligned}&\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k+1}-\nabla \mathbf {f}_{k})\rangle \nonumber \\&\quad \le \alpha (1-\beta ) L\delta ^3\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2+2\delta \Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert \delta L\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert \nonumber \\&\quad +\delta \Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert \alpha \beta \delta L\Vert \mathbf {u}_k\Vert +\delta \Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert \alpha (1-\beta ) L\delta \sqrt{n}\Vert \overline{d}_k\Vert \nonumber \\&\quad \le \alpha (1-\beta ) L\delta ^3\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2+\eta _1\delta ^2\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2 +\eta _1^{-1}\delta ^2L^2\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert ^2\nonumber \\&\quad +0.5\eta _2\delta ^2\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2 +0.5\eta _2^{-1}\alpha ^2\beta ^2\delta ^2 L^2\Vert \mathbf {u}_k\Vert ^2\nonumber \\&\quad +0.5\eta _3\delta ^2\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2 +0.5\eta _3^{-1}\alpha ^2(1-\beta )^2L^2\delta ^2n\Vert \overline{d}_k\Vert ^2, \end{aligned}$$

(A17)

where $\eta _i>0, i=1,2,3$ and the last inequality uses Lemma 1(a) with $p=1$. Taking the conditional expectation on (A17) and using (13) lead to

$$\begin{aligned}&\mathbb {E}[\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k+1}-\nabla \mathbf {f}_{k})\rangle \mid \mathcal {F}_k]\nonumber \\&\quad \le (\alpha (1-\beta ) L\delta +\eta _1+0.5\eta _2+0.5\eta _3)\delta ^2\mathbb {E}[\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +\eta _1^{-1}L^2\delta ^2\mathbb {E}[\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k] +0.5\eta _2^{-1}\alpha ^2L^2\beta ^2\delta ^2\mathbb {E}[\Vert \mathbf {u}_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +0.5\eta _3^{-1}\alpha ^2(1-\beta )^2L^2\delta ^2n\mathbb {E}[\Vert h_k\Vert ^2\mid \mathcal {F}_k]+0.5\eta _3^{-1}\alpha ^2(1-\beta )^2L^2\delta ^2\sigma ^2. \end{aligned}$$

(A18)

Applying (A11), (A14) and (A18) to (A3) and then taking the total expectation, we further obtain

$$\begin{aligned}&\mathbb {E}[\Vert \mathbf {s}_{k+1}-W_\infty \mathbf {s}_{k+1}\Vert ^2]\nonumber \\&\quad \le (8+2\eta _1^{-1})L^2\delta ^2\mathbb {E}[\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert ^2]\nonumber \\&\quad +(1+2\alpha (1-\beta ) L\delta +8\alpha ^2(1-\beta )^2 L^2\delta ^2+2\eta _1+\eta _2+\eta _3)\delta ^2\mathbb {E}[\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2]\nonumber \\&\quad +(4+\eta _2^{-1})\alpha ^2L^2\beta ^2\delta ^2\mathbb {E}[\Vert \mathbf {u}_k\Vert ^2]+(8+\eta _3^{-1})\alpha ^2(1-\beta )^2L^2\delta ^2n\mathbb {E}[\Vert h_k\Vert ^2]\nonumber \\&\quad +(8+\eta _3^{-1})\alpha ^2(1-\beta )^2L^2\delta ^2\sigma ^2+2((1+\alpha (1-\beta ) L)n\delta ^2+1)\sigma ^2. \end{aligned}$$

(A19)

We set $\eta _1=\frac{1-\delta ^2}{8\delta ^2}$, $\eta _2=\frac{1-\delta ^2}{32\delta ^2}$ and $\eta _3=\frac{1-\delta ^2}{16\delta ^2}$. If $0<\alpha \le \frac{1-\delta ^2}{16(1-\beta )L\delta }$, then we get

$$\begin{aligned} 1+2\alpha (1-\beta ) L\delta +8\alpha ^2(1-\beta )^2 L^2\delta ^2+2\eta _1+\eta _2+\eta _3\le \frac{1+\delta ^2}{2\delta ^2}. \end{aligned}$$

(A20)

If $0<\alpha \le \min \left\{ \frac{1-\delta ^2}{8\delta },1\right\} \frac{1}{2(1-\beta )L}$, then the desired result follows from applying (A20) and the values of $\eta _1$, $\eta _2$ and $\eta _3$ to (A19). $\square$

Appendix B Proof of Lemma 17

Proof

First, using the same argument as that in the proof of Lemma 8, we have

$$\begin{aligned}&\mathbb {E}[\Vert \mathbf {s}_{k+1}-W_\infty \mathbf {s}_{k+1}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad \le \delta ^2\mathbb {E}[\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_k\Vert ^2]+ \delta ^2\mathbb {E}[\Vert \mathbf {d}_{k+1}-\mathbf {d}_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +2\mathbb {E}[\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k}-\mathbf {d}_{k})\rangle \mid \mathcal {F}_k]\nonumber \\&\quad +2\mathbb {E}[\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k+1}-\nabla \mathbf {f}_{k})\rangle \mid \mathcal {F}_k]. \end{aligned}$$

(B21)

Consider the second term on the right-hand side of (B21). Following the same steps as (A4), there holds

$$\begin{aligned}&\mathbb {E}[\Vert \mathbf {d}_{k+1}-\mathbf {d}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad \le \!2n\sigma ^2+L^2\mathbb {E}[\Vert \varvec{\theta }_{k+1}-\varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k] +2\mathbb {E}[\langle \nabla \mathbf {f}_{k+1},\nabla \mathbf {f}_{k}-\mathbf {d}_{k}\rangle \mid \mathcal {F}_k]. \end{aligned}$$

(B22)

Then, let us consider the second term on the right-hand side of (B22).

$$\begin{aligned}&\mathbb {E}[\Vert \varvec{\theta }_{k+1}-\varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[\Vert W(\varvec{\theta }_k-\alpha (1-\beta ^2)\mathbf {s}_k -\alpha \beta ^2\mathbf {u}_k)-\varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad =\mathbb {E}[\Vert (W-I_{np})(\varvec{\theta }_k-W_\infty \varvec{\theta }_{k}) -\alpha (1-\beta ^2) W \mathbf {s}_{k}\!-\!\alpha \beta ^2W\mathbf {u}_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad \le 2\Vert W-I_{np}\Vert ^2\mathbb {E}[\Vert \varvec{\theta }_k-W_\infty \varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k] +4\alpha ^2(1-\beta ^2)^2\mathbb {E}[\Vert W\mathbf {s}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +4\alpha ^2\beta ^4\mathbb {E}[\Vert \mathbf {u}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad \le 8\mathbb {E}[\Vert \varvec{\theta }_k-W_\infty \varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k] +4\alpha ^2\beta ^4\mathbb {E}[\Vert \mathbf {u}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +8\alpha ^2(1-\beta ^2)^2\delta ^2\mathbb {E}[\Vert \mathbf {s}_{k} -W_\infty \mathbf {s}_k\Vert ^2\mid \mathcal {F}_k] +8\alpha ^2(1-\beta ^2)^2n\mathbb {E}[\Vert h_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +8\alpha ^2(1-\beta ^2)^2\sigma ^2, \end{aligned}$$

(B23)

where the first inequality applies Lemma 1(c) with $\eta =1$ and the last inequality uses $\Vert W-I_{np}\Vert \le 2$ and (13). Next, we consider the last term on the right-hand side of (B22). From (5), it follows that

$$\begin{aligned} \nabla \mathbf {f}_{k+1}&=\nabla \mathbf {f}(W(\varvec{\theta }_k-\alpha (1-\beta ^2) \mathbf {s}_k-\alpha \beta ^2\mathbf {u}_{k}))\\&=\nabla \mathbf {f}(W(\varvec{\theta }_k-\alpha (1-\beta ^2)W(\mathbf {s}_{k-1}+\mathbf {d}_{k}-\mathbf {d}_{k-1})-\alpha \beta ^2\mathbf {u}_{k})). \end{aligned}$$

Denote

$$\begin{aligned} \underline{\nabla \mathbf {f}}_{k+1} =\nabla \mathbf {f}(W(\varvec{\theta }_k -\alpha (1-\beta ^2) W(\mathbf {s}_{k-1}+\nabla \mathbf {f}_{k} -\mathbf {d}_{k-1})-\alpha \beta ^2\mathbf {u}_{k})). \end{aligned}$$

By Assumption 3,

$$\begin{aligned} \mathbb {E}[\langle \underline{\nabla \mathbf {f}}_{k+1},\nabla \mathbf {f}_k-\mathbf {d}_k\rangle \mid \mathcal {F}_k]=0. \end{aligned}$$

(B24)

In light of Assumption 1 and $\Vert W^2\Vert =1$,

$$\begin{aligned} \Vert \nabla \mathbf {f}_{k+1}-\underline{\nabla \mathbf {f}}_{k+1}\Vert\le & {} \,\alpha (1-\beta ^2) L\Vert W^2(\mathbf {d}_k-\nabla \mathbf {f}_k)\Vert \nonumber \\\le & {} \,\alpha (1-\beta ^2) L\Vert \mathbf {d}_k-\nabla \mathbf {f}_k\Vert . \end{aligned}$$

(B25)

Using (B24) and (B25), we have

$$\begin{aligned} \mathbb {E}[\langle \nabla \mathbf {f}_{k+1},\nabla \mathbf {f}_k-\mathbf {d}_k\rangle \mid \mathcal {F}_k]\le & {}\, \mathbb {E}[\Vert \nabla \mathbf {f}_{k+1}-\underline{\nabla \mathbf {f}}_{k+1}\Vert \Vert \nabla \mathbf {f}_k-\mathbf {d}_k\Vert \mid \mathcal {F}_k]\nonumber \\\le & {} \,\alpha (1-\beta ^2) L\mathbb {E}[\Vert \nabla \mathbf {f}_k-\mathbf {d}_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\\le & {}\, \alpha (1-\beta ^2) Ln\sigma ^2. \end{aligned}$$

(B26)

Combining (B23), (B26) with (B22) gets

$$\begin{aligned}&\mathbb {E}[\Vert \mathbf {d}_{k+1}-\mathbf {d}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad \le \! 8L^2\mathbb {E}[\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k] +4\alpha ^2\beta ^4L^2\mathbb {E}[\Vert \mathbf {u}_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +8\alpha ^2(1-\beta ^2)^2 L^2\delta ^2\mathbb {E}[\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_k\Vert ^2\mid \mathcal {F}_k] +8\alpha ^2(1-\beta ^2)^2L^2n\mathbb {E}[\Vert h_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad +2(1+\alpha (1-\beta ^2) L)n\sigma ^2 +8\alpha ^2(1-\beta ^2)^2L^2\sigma ^2. \end{aligned}$$

(B27)

Consider the third term on the right-hand side of (B21). It has the same bound as the corresponding term of (A3), that is,

$$\begin{aligned} \mathbb {E}[\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k}-\mathbf {d}_{k})\rangle \mid \mathcal {F}_k]\le \sigma ^2. \end{aligned}$$

(B28)

Consider the fourth term on the right-hand side of (B21). Following the same steps as (A15), we have

$$\begin{aligned}&\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k+1}-\nabla \mathbf {f}_{k})\rangle \nonumber \\&\quad \le \!\delta ^2L\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert \Vert \varvec{\theta }_{k+1}-\varvec{\theta }_{k}\Vert . \end{aligned}$$

(B29)

Moreover,

$$\begin{aligned}&\Vert \varvec{\theta }_{k+1}-\varvec{\theta }_{k}\Vert \nonumber \\&\quad =\!\Vert W(\varvec{\theta }_{k}-\alpha (1-\beta ^2)\mathbf {s}_k-\alpha \beta ^2\mathbf {u}_{k})-\varvec{\theta }_{k}\Vert \nonumber \\&\quad =\!\Vert (W-I_{np})(\varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k})-\alpha (1-\beta ^2) W\mathbf {s}_k-\alpha \beta ^2 W\mathbf {u}_k\Vert \nonumber \\&\quad \le \!2\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert +\alpha \beta ^2\Vert \mathbf {u}_{k}\Vert +\alpha (1-\beta ^2)\Vert W\mathbf {s}_k\Vert \nonumber \\&\quad \le \!2\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert +\alpha \beta ^2\Vert \mathbf {u}_{k}\Vert +\alpha (1-\beta ^2)\delta \Vert \mathbf {s}_k-W_\infty \mathbf {s}_k\Vert +\alpha (1-\beta ^2)\sqrt{n}\Vert \overline{d}_k\Vert ,\,\,\,\,\,\,\, \end{aligned}$$

(B30)

where the first inequality uses $\Vert W-I_{np}\Vert \le 2$ and $\Vert W\Vert =1$. Substituting (B30) into (B29) yields

$$\begin{aligned}&\!\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k+1}-\nabla \mathbf {f}_{k})\rangle \nonumber \\&\quad \le \!\alpha (1-\beta ^2) L\delta ^3\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2 +2\delta \Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert \delta L\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert \nonumber \\&\quad +\delta \Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert \alpha \beta ^2\delta L\Vert \mathbf {u}_k\Vert +\delta \Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert \alpha (1-\beta ^2) L\delta \sqrt{n}\Vert \overline{d}_k\Vert \nonumber \\&\quad \le \alpha (1-\beta ^2) L\delta ^3\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2+\eta _1\delta ^2\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2 +\eta _1^{-1}\delta ^2L^2\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert ^2\nonumber \\&\quad \!+0.5\eta _2\delta ^2\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2 +0.5\eta _2^{-1}\alpha ^2\beta ^4\delta ^2 L^2\Vert \mathbf {u}_k\Vert ^2\nonumber \\&\quad \!+0.5\eta _3\delta ^2\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2 +0.5\eta _3^{-1}\alpha ^2(1-\beta ^2)^2 L^2\delta ^2n\Vert \overline{d}_k\Vert ^2, \end{aligned}$$

(B31)

where $\eta _i>0, i=1,2,3$ and the last inequality uses Lemma 1(a) with $p=1$. Taking the conditional expectation on (B31) and using (13) lead to

$$\begin{aligned}&\!\mathbb {E}[\langle (W-W_\infty )\mathbf {s}_{k},(W-W_\infty )(\nabla \mathbf {f}_{k+1}-\nabla \mathbf {f}_{k})\rangle \mid \mathcal {F}_k]\nonumber \\&\quad \le \! (\alpha (1-\beta ^2) L\delta +\eta _1+0.5\eta _2+0.5\eta _3)\delta ^2\mathbb {E}[\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad \!+\eta _1^{-1}L^2\delta ^2\mathbb {E}[\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert ^2\mid \mathcal {F}_k] +0.5\eta _2^{-1}\alpha ^2L^2\beta ^4\delta ^2\mathbb {E}[\Vert \mathbf {u}_k\Vert ^2\mid \mathcal {F}_k]\nonumber \\&\quad \!+0.5\eta _3^{-1}\alpha ^2(1-\beta ^2)^2L^2\delta ^2n\mathbb {E}[\Vert h_k\Vert ^2\mid \mathcal {F}_k] +0.5\eta _3^{-1}\alpha ^2(1-\beta ^2)^2L^2\delta ^2\sigma ^2. \end{aligned}$$

(B32)

Applying (B27), (B28) and (B32) to (B21) and then taking the total expectation, we further obtain

$$\begin{aligned}&\mathbb {E}[\Vert \mathbf {s}_{k+1}-W_\infty \mathbf {s}_{k+1}\Vert ^2]\nonumber \\&\quad \le \!(8+2\eta _1^{-1})L^2\delta ^2\mathbb {E}[\Vert \varvec{\theta }_{k}-W_\infty \varvec{\theta }_{k}\Vert ^2]\nonumber \\&\quad \!+(1+2\alpha (1-\beta ^2) L\delta +8\alpha ^2(1-\beta ^2)^2 L^2\delta ^2+2\eta _1+\eta _2+\eta _3)\delta ^2\mathbb {E}[\Vert \mathbf {s}_{k}-W_\infty \mathbf {s}_{k}\Vert ^2]\nonumber \\&\quad \!+(4+\eta _2^{-1})\alpha ^2L^2\beta ^4\delta ^2\mathbb {E}[\Vert \mathbf {u}_k\Vert ^2] +(8+\eta _3^{-1})\alpha ^2(1-\beta ^2)^2L^2\delta ^2n\mathbb {E}[\Vert h_k\Vert ^2]\nonumber \\&\quad \!+(8+\eta _3^{-1})\alpha ^2(1-\beta ^2)^2L^2\delta ^2\sigma ^2 +2((1+\alpha (1-\beta ^2) L)n\delta ^2+1)\sigma ^2. \end{aligned}$$

(B33)

We set $\eta _1=\frac{1-\delta ^2}{8\delta ^2}$, $\eta _2=\frac{1-\delta ^2}{32\delta ^2}$ and $\eta _3=\frac{1-\delta ^2}{16\delta ^2}$. It can then be verified that if $0<\alpha \le \frac{1-\delta ^2}{16(1-\beta ^2)L\delta }$,

$$\begin{aligned} 1+2\alpha (1-\beta ^2) L\delta +8\alpha ^2(1-\beta ^2)^2 L^2\delta ^2+2\eta _1+\eta _2+\eta _3 \le \frac{1+\delta ^2}{2\delta ^2}. \end{aligned}$$

(B34)

If $0<\alpha \le \min \{\frac{1-\delta ^2}{8\delta },1\}\frac{1}{2(1-\beta ^2)L}$, then the desired result follows from applying (B34) and the values of $\eta _1$, $\eta _2$ and $\eta _3$ to (B33). $\square$