A Projected Extrapolated Gradient Method with Larger Step Size for Monotone Variational Inequalities

Abstract

A projected extrapolated gradient method is designed for solving monotone variational inequality in Hilbert space. Requiring local Lipschitz continuity of the operator, our proposed method improves the value of the extrapolated parameter and admits larger step sizes, which are predicted based a local information of the involved operator and corrected by bounding the distance between each pair of successive iterates. The correction will be implemented when the distance is larger than a given constant and its main cost is to compute a projection onto the feasible set. In particular, when the operator is the gradient of a convex function, the correction step is not necessary. We establish the convergence and ergodic convergence rate in theory under the larger range of parameters. Related numerical experiments illustrate the improvements in efficiency from the larger step sizes.

The Details on Remark 3.4

For the case of $\delta \in (\frac{\sqrt{5}-1}{2},1]$ presented in Remark 3.4, by Fact 2.4 with $\varepsilon _1>0$, we have

$$\begin{aligned} 2\alpha \Vert y_n-y_{n-1}\Vert \Vert y_n-x_{n+1}\Vert\le & {} \alpha (\frac{1}{\varepsilon _1 }\Vert y_n-y_{n-1}\Vert ^2+\varepsilon _1\Vert x_{n+1}-y_n\Vert ^2). \end{aligned}$$

Meanwhile, for any $\varepsilon _2>0$ we deduce

$$\begin{aligned} \Vert y_n-y_{n-1}\Vert ^2= & {} \Vert y_n-x_n\Vert ^2+\Vert x_n-y_{n-1}\Vert ^2+2\langle y_n-x_n,x_n-y_{n-1}\rangle \\\le & {} \Vert y_n-x_n\Vert ^2+\Vert x_n-y_{n-1}\Vert ^2+2\Vert y_n-x_n\Vert \Vert x_n-y_{n-1}\Vert \\\le & {} (1+\frac{1}{\varepsilon _2})\Vert y_n-x_n\Vert ^2+(1+\varepsilon _2) \Vert x_n-y_{n-1}\Vert ^2. \end{aligned}$$

Hence, Lemmas 3.3 and 3.4 can be improved as following lemmas.

Lemma A.1

Let $\{x_n\}$, $\{y_n\}$ be two sequences generated by Algorithm 3.1 and ${\bar{x}}\in {{\mathcal {S}}}$. Then, for any $\varepsilon _1,\varepsilon _2>0$, we have

$$\begin{aligned}&\Vert x_{n+1}-{\bar{x}}\Vert ^2+2\lambda _n(1 +\delta ) \varPhi ({\bar{x}}, x_n) \\&\quad \le \Vert x_n-{\bar{x}}\Vert ^2+ 2\lambda _{n-1}(1 +\delta ) \varPhi ({\bar{x}}, x_{n-1})\\&\qquad +\left[ \frac{1}{\varepsilon _1}\left( 1+\frac{1}{\varepsilon _2}\right) \alpha -\frac{\lambda _n}{\delta \lambda _{n-1}} \right] \Vert x_n-y_n\Vert ^2\\&\qquad + \left( \frac{\lambda _n}{\delta \lambda _{n-1}}-1 \right) \Vert x_{n+1}-x_n\Vert ^2 \\&\qquad +\left( \varepsilon _1 \alpha -\frac{\lambda _n}{\delta \lambda _{n-1} }\right) \Vert x_{n+1}-y_n\Vert ^2+ \frac{1+\varepsilon _2}{\varepsilon _1}\alpha \Vert x_n-y_{n-1}\Vert ^2, \end{aligned}$$

where $\varPhi (\cdot ,\cdot )$ is defined as in (7).

Lemma A.2

Let $\{x_n\}$, $\{y_n\}$ be two sequences generated by Algorithm 3.1 and ${\bar{x}}\in {{\mathcal {S}}}$. Then, for any $\varepsilon _1,\varepsilon _2>0$, we have

$$\begin{aligned} a_{n+1}\le & {} a_n-b_n,~~n\ge 2, \end{aligned}$$

(36)

where

$$\begin{aligned} \left\{ \begin{array}{rcl} a_n&{}=&{}\Vert x_n-{\bar{x}}\Vert ^2+ 2\lambda _{n-1}(1 +\delta ) \varPhi ({\bar{x}}, x_{n-1})+\frac{1+\varepsilon _2}{\varepsilon _1}\alpha \Vert x_n-y_{n-1}\Vert ^2, \\ b_n&{}=&{}\left[ \frac{\lambda _n}{\delta \lambda _{n-1} }-\left( \varepsilon _1+\frac{1+\varepsilon _2}{\varepsilon _1}\right) \alpha \right] \Vert x_{n+1}-y_n\Vert ^2\\ &{}&{}+\left[ \frac{\lambda _n}{\delta \lambda _{n-1}}-\frac{1}{\varepsilon _1}\left( 1+\frac{1}{\varepsilon _2}\right) \alpha \right] \Vert x_n-y_n\Vert ^2 +\left( 1- \frac{\lambda _n}{\delta \lambda _{n-1}} \right) \Vert x_{n+1}-x_n\Vert ^2. \end{array}\right. \end{aligned}$$

(37)

For any $\delta \in (\frac{\sqrt{5}-1}{2},1]$, we have $\delta ^2+\delta -1>0$ and define a function $\kappa (\delta )$ as

$$\begin{aligned} \kappa (\delta ):=\frac{1}{\delta }\max \limits _{\varepsilon _1>0,\varepsilon _2>0}\min \left\{ \frac{\varepsilon _1}{\varepsilon _1^2+\varepsilon _2+1}, ~\frac{(\delta ^2+\delta -1)\varepsilon _1\varepsilon _2}{\delta ^2(1+\varepsilon _2)} \right\} . \end{aligned}$$

(38)

Noting that the structure of (38) and $\kappa (\delta )$ is a maximum value, so

$$\begin{aligned} \frac{\varepsilon _1}{\varepsilon _1^2+\varepsilon _2+1}= ~\frac{(\delta ^2+\delta -1)\varepsilon _1\varepsilon _2}{\delta ^2(1+\varepsilon _2)}, \end{aligned}$$

which together with $a=\frac{\delta ^2}{\delta ^2+\delta -1}$ and $\varepsilon _1>0$ yields

$$\begin{aligned} \varepsilon _1=\sqrt{\frac{(1+\varepsilon _2)(a-\varepsilon _2)}{\varepsilon _2}}, \end{aligned}$$

and then

$$\begin{aligned} \kappa (\delta )=\frac{1}{\delta }\max \limits _{\varepsilon _2>0} \frac{\sqrt{a\varepsilon _2+(a-1) \varepsilon _2^2-\varepsilon _2^3}}{a(1+\varepsilon _2)}. \end{aligned}$$

(39)

It follows from the first-order optimality condition of problem (39) that

$$\begin{aligned} \kappa (\delta )=\frac{\sqrt{a+1}}{\delta (a+1+\sqrt{a+1})}, \end{aligned}$$

when $\varepsilon _1=\sqrt{a+1}$ and $\varepsilon _2=\sqrt{a+1}-1$.

Consequently, we can obtain the following convergence result on Algorithm 3.1 with $\delta \in (\frac{\sqrt{5}-1}{2},1]$ and $\alpha \in (0,\kappa (\delta ))$.

Theorem A.1

Let $\{x_n\}$ be the sequence generated by Algorithm 3.1 with $\delta \in (\frac{\sqrt{5}-1}{2},1]$ and $\alpha \in (0,\kappa (\delta ))$. Then, $\{x_n\}$ converges weakly to a solution of problem (1).

Proof

Firstly, $\delta \in (\frac{\sqrt{5}-1}{2},1]$ gives $\frac{1}{\delta }\ge \frac{\delta ^2 +\delta -1}{\delta ^3}>0$. Note that $\lim \limits _{n\rightarrow \infty }\frac{\lambda _n}{\lambda _{n-1}}=1$, by taking the limit, $\alpha <\kappa (\delta )$ and the definition of $\kappa (\alpha )$ in (30), we have

$$\begin{aligned} \left. \begin{array}{r} \lim \limits _{n\rightarrow \infty }\left[ \left( \varepsilon _1+\frac{1+\varepsilon _2}{\varepsilon _1} \right) \alpha -\frac{\lambda _n}{\delta \lambda _{n-1} }\right] =\left( \frac{\varepsilon _1^2+\varepsilon _2+1}{\varepsilon _1}\right) \alpha -\frac{1}{\delta }<0,\\ \lim \limits _{n\rightarrow \infty }\left[ \frac{1}{\varepsilon _1}\left( 1+\frac{1}{\varepsilon _2}\right) \alpha -\frac{\lambda _{n}}{\delta \lambda _{n-1} }\right] =\frac{1}{\varepsilon _1}\left( 1+\frac{1}{\varepsilon _2}\right) \alpha -\frac{1}{\delta }<0,\\ \lim \limits _{n\rightarrow \infty }\left[ \frac{1}{\varepsilon _1} \left( 1+\frac{1}{\varepsilon _2}\right) \alpha -\frac{\lambda _n}{\delta \lambda _{n-1} }+\frac{1}{\delta ^2}\left( \frac{\lambda _{n-1}}{\delta \lambda _{n-2}}-1 \right) \right] =\frac{1}{\varepsilon _1}\left( 1+\frac{1}{\varepsilon _2}\right) \alpha - \frac{\delta ^2 +\delta -1}{\delta ^3} <0, \end{array} \right. \end{aligned}$$

for any $\delta \in (\frac{\sqrt{5}-1}{2},1]$. Thus, there exists an integer $N>2,$ such that for any $n>N$,

$$\begin{aligned} \left. \begin{array}{r} \left( \frac{\varepsilon _1^2+\varepsilon _2+1}{\varepsilon _1}\right) \alpha -\frac{\lambda _n}{\delta \lambda _{n-1} }<0,\\ \frac{1}{\varepsilon _1}\left( 1+\frac{1}{\varepsilon _2}\right) \alpha -\frac{\lambda _{n}}{\delta \lambda _{n-1} }<0,\\ \frac{1}{\varepsilon _1}\left( 1+\frac{1}{\varepsilon _2}\right) \alpha -\frac{\lambda _n}{\delta \lambda _{n-1} }+\frac{1}{\delta ^2}\left( \frac{1}{\delta }-1 \right) <0. \end{array} \right. \end{aligned}$$

Consequently, by the similar processing as in the proof of Theorem 3.1, the result can be proved. We omit the details. $\square$