Optimal stochastic extragradient schemes for pseudomonotone stochastic variational inequality problems and their variants

Abstract

We consider the stochastic variational inequality problem in which the map is expectation-valued in a component-wise sense. Much of the available convergence theory and rate statements for stochastic approximation schemes are limited to monotone maps. However, non-monotone stochastic variational inequality problems are not uncommon and are seen to arise from product pricing, fractional optimization problems, and subclasses of economic equilibrium problems. Motivated by the need to address a broader class of maps, we make the following contributions: (1) we present an extragradient-based stochastic approximation scheme and prove that the iterates converge to a solution of the original problem under either pseudomonotonicity requirements or a suitably defined acute angle condition. Such statements are shown to be generalizable to the stochastic mirror-prox framework; (2) under strong pseudomonotonicity, we show that the mean-squared error in the solution iterates produced by the extragradient SA scheme converges at the optimal rate of ${{\mathcal {O}}}\left( \frac{1}{{K}}\right)$, statements that were hitherto unavailable in this regime. Notably, we optimize the initial steplength by obtaining an $\epsilon$-infimum of a discontinuous nonconvex function. Similar statements are derived for mirror-prox generalizations and can accommodate monotone SVIs under a weak-sharpness requirement. Finally, both the asymptotics and the empirical rates of the schemes are studied on a set of variational problems where it is seen that the theoretically specified initial steplength leads to significant performance benefits.

Appendix

Lemma 7

Consider the function $t_0(\gamma _0)$ defined as

$$\begin{aligned} t_0(\gamma _0) \triangleq \frac{\gamma _0^2}{2\sigma \gamma _0 - \lfloor 2\sigma \gamma _0 \rfloor }, \end{aligned}$$

where $\sigma _0$ denotes the strong pseudomonotonicity constant. Then the following hold:

• (a) A minimizer of $t_0(\gamma _0)$ cannot exist in an interval $\lfloor 2\sigma \gamma _0 \rfloor \in [n,n+1]$ where $n > 1$.

• (b) The infimum of $t_0(\gamma _0)$ is given by the following : 

  $$\begin{aligned} {{ f^* \triangleq }} \inf _{\gamma _0} \left\{ t_0(\gamma _0) \mid 1< \ 2\sigma \gamma _0 \ < 2 \right\} = \frac{1}{\sigma ^2}. \end{aligned}$$

• (c) Suppose an $\epsilon$-infimum of $t_0(\gamma _0)$, denoted by $f^*_{\epsilon }$, satisfies $f^*_{\epsilon } \le f^* + \beta \epsilon$ for some $\beta > 0$. Then $f^*_{\epsilon }$ is achieved by $\gamma _0 = \frac{2-\epsilon }{2\sigma }$ and satisfies ${{f_{\epsilon }^*}} \le f^* + \frac{2\epsilon }{\sigma ^2}$, where $\epsilon \in (0,1/2)$.

Proof

• (a) We begin by observing that if $2\sigma \gamma _0 \in \mathbb {Z}_+$, then $t_0(\gamma _0) = +\infty .$ Consequently, any minimizer of $t_0(\gamma _0)$ has to satisfy $2\gamma _0 \sigma \not \in \mathbb {Z}_+$. We proceed to show that $t_0(\gamma _0)$ does not admit a minimizer in $(n,n+1)$ where $n > 1$. Assume this is false and suppose there exists a minimizer $\gamma _0^*$ satisfying $2\gamma _0^*\sigma \in (n,n+1)$. But there exists a $\tilde{\gamma }_0$ such that $2\sigma {\tilde{\gamma }}_0 \in (n-1,n)$. In fact, $t_0({{\tilde{\gamma }}}_0) < t_0(\gamma _0^*)$ as we show next and our claim follows.

  $$\begin{aligned} t_0({\tilde{\gamma }}_0) = \left( \frac{{\tilde{\gamma }}_0^2}{2\sigma {\tilde{\gamma }}_0 - \lfloor 2 \sigma {\tilde{\gamma }}_0 \rfloor } \right) < \left( \frac{(\gamma ^*_0)^2}{2\sigma \gamma ^*_0 - \lfloor 2 \sigma \gamma ^*_0 \rfloor } \right) = t_0(\gamma _0^*). \end{aligned}$$

• (b) From (a), it follows that if a minimizer exists, it has to satisfy $2\gamma _0^* \sigma \in (1,2)$. It follows that $t_0(\gamma _0)$ reduces to $\gamma _0^2/(2\sigma \gamma _0 - 1).$ Consider the following optimization problem:

  $$\begin{aligned} \inf _{ \gamma _0} \left\{ \frac{\gamma _0^2}{(2\sigma \gamma _0 - 1)} \mid 1< 2\sigma \gamma _0 < 2 \right\} . \end{aligned}$$

  We observe that $t_0(\gamma _0)$ is a strictly decreasing function by noting that

  $$\begin{aligned} t'_0(\gamma _0)= & {} \frac{2\gamma _0}{2\sigma \gamma _0-1} -\frac{ 2\sigma \gamma _0^2 }{(2\sigma \gamma _0 -1)^2} = \frac{2\gamma _0}{2\sigma \gamma _0-1} \left( 1-\frac{\sigma \gamma _0}{2\sigma \gamma _0-1} \right) \\= & {} \frac{2\gamma _0}{2\sigma \gamma _0-1} \left( \frac{\sigma \gamma _0-1}{2\sigma \gamma _0-1} \right) < 0, \end{aligned}$$

  since $\sigma \gamma _0 < 1$. It follows that the infimum is at the end-point given by $\sigma \gamma _0 = 1$ implying that

  $$\begin{aligned} \inf _{ \gamma _0} \left\{ \frac{\gamma _0^2}{(2\sigma \gamma _0 - 1)} \mid 1< 2\sigma \gamma _0 < 2 \right\} = \frac{1}{\sigma ^2}. \end{aligned}$$

• (c) Suppose ${\widehat{\gamma }}_0 = \frac{2-\epsilon }{2\sigma }$ where $\epsilon \in (0,1/2)$. Then we have that

  $$\begin{aligned} f^*_{\epsilon } - f^* = \frac{(2-\epsilon )^2}{4\sigma ^2 (1-\epsilon )} - \frac{1}{\sigma ^2} \le \frac{1}{\sigma ^2} \left( \frac{1}{1-\epsilon } - 1\right) = \frac{1}{\sigma ^2}\frac{\epsilon }{1-\epsilon } = \frac{2}{\sigma ^2} \epsilon . \end{aligned}$$

  It follows that ${{f^*_{\epsilon } \le f^* + \frac{2\epsilon }{\sigma ^2}}}.$$\square$

Example 1

Unfortunately, while one can derive an infimum of the above discontinuous optimization problem, this infimum cannot be achieved and the problem lacks a minimizer as proved in the above result. Yet, this infimum is informative in developing an approximate $\epsilon$-solution as part (c) shows. We proceed to use this $\epsilon$-infimum in deriving rate statements and demonstrate this result through an example. Suppose $\sigma = 0.1\sqrt{1.3}$. Then $t_0(\gamma _0)$ is shown as a solid line with discontinuities in Fig. 1 while the dashed flat line displays the infimum $1/\sigma ^2$.

Fig. 1

A schematic of $t_0(\gamma _0)$

Lemma 8

Consider the following recursion: $a_{k+1} \le (1-2c\theta /k) a_k + {\textstyle {1\over 2}}\theta ^2 M^2/k^2,$ where $\theta$ and M are positive constants, $a_k \ge 0$, and $(1-2c\theta ) < 0$. Then for $k \ge 1$, we have that

$$\begin{aligned} 2a_k \le \frac{\max \left( \frac{\theta ^2}{2c\theta - \lfloor 2c\theta \rfloor } M^2,2a_1\right) }{k}. \end{aligned}$$

Proof

We begin by noting that ${\bar{\epsilon }} > 0$ and $\kappa >1$ as seen next.

$$\begin{aligned} \kappa = \left( 1+\frac{\bar{k}-1}{2c\theta -\bar{k}}\right) = \left( \frac{2c\theta -1}{2c\theta -\lfloor 2c\theta \rfloor }\right) > 1. \end{aligned}$$

We consider the following cases for k.

• Case 1: Consider $k = 1$. Then the following holds: $a_{2} \le (1-2c\theta )a_1 + \frac{1}{2}\theta ^2 M^2.$ If $(2c\theta -1) > 0$, we may rearrange the inequalities as follows:

  $$\begin{aligned} (2c\theta -1)a_1&\le -a_2 + \frac{1}{2} \theta ^2M^2 \le \frac{1}{2} \theta ^2M^2 \; \text {or} \; 2a_1 \le \frac{1}{2c\theta -1} \theta ^2M^2 \\ \Longrightarrow 2a_1&\le \max \left( (2c\theta -1)^{-1}\theta ^2M^2, 2a_1\right) \\&\le \max \left( (2c\theta -1)^{-1}\theta ^2M^2\kappa ,2a_1 \right) , \end{aligned}$$

  where $\kappa > 1$.

• Case 2: $1 \ < \ k \le \bar{k}$. Recall that when $k \le \bar{k}$, we have that

  $$\begin{aligned} (1-2c\theta /k) \le (1-2c\theta /\bar{k}) = \left( 1-(2c\theta ) /(\lceil 2c\theta \rceil )\right) < 0. \end{aligned}$$

  Then the following holds:

  $$\begin{aligned} a_{k+1}&\le \left( \left( 1-\frac{2c\theta }{k}\right) a_{k} + \frac{1}{2}\frac{\theta ^2 M^2}{k^2} \right) \\ a_k \left( \frac{2c\theta }{k}-1\right)&\le -a_{k+1} + \frac{\theta ^2M^2}{2k^2} \le \frac{\theta ^2 M^2}{2k^2}\\ \Longrightarrow a_k&\le \frac{\theta ^2M^2}{2k^2}\left( \frac{k}{2c\theta -k}\right) = \frac{\theta ^2M^2}{2k(2c\theta -k)}. \end{aligned}$$

  By the definition of $\kappa$ and $u \triangleq \max \left( \frac{\kappa \theta ^2 M^2}{(2c\theta -1)}, {2a_1}\right)$, we may conclude the following:

  $$\begin{aligned} 2a_{k} \le \frac{\theta ^2M^2}{k(2c\theta -k)}&= \frac{\theta ^2M^2}{k(2c\theta -1)}\left( \frac{2c\theta -1}{2c\theta -k}\right) \\&\le \frac{\theta ^2M^2}{k(2c\theta -1)}\left( \frac{2c\theta -k+\bar{k}-1}{2c\theta -k}\right) \\&\le \frac{\theta ^2M^2}{k(2c\theta -1)} \left( 1+\frac{\bar{k}-1}{2c\theta -k}\right) \\&\le \frac{\theta ^2M^2}{k(2c\theta -1)} \left( 1+\frac{\bar{k}-1}{2c\theta -{\bar{k}} }\right) = \frac{\theta ^2M^2}{k(2c\theta -1)} \kappa \\&\le {1\over k}\max \left( \frac{\kappa \theta ^2 M^2}{(2c\theta -1)}, {2a_1}\right) = \frac{u}{k}, \end{aligned}$$

• Case 3: $k > \bar{k}$. Suppose, this holds for $k > \bar{k}$, implying that $2a_k \le \frac{\max (\theta ^2M^2(2c\theta -1)^{-1}\kappa , 2a_1)}{k}$. We proceed to show that this holds for $k: = k+1$ where $u \triangleq \max \left( \theta ^2M^2 (2c\theta -1)^{-1}\kappa , 2a_1 \right)$ and $(1-\frac{2c\theta }{k}) > 0$ since $k > \bar{k}$:

  $$\begin{aligned} a_{k+1}&\le \left( 1-\frac{2c\theta }{k}\right) \frac{u}{2k} + \frac{1}{2}\left( \frac{\theta ^2M^2}{k^2}\right) \\&= \left( 1-\frac{2c\theta }{k}\right) \frac{u}{2k} + \frac{(2c\theta -1)}{2k} \left( \frac{\theta ^2M^2}{(2c\theta -1)k}\right) \\&\le \left( 1-\frac{2c\theta }{k}\right) \frac{u}{2k} + \frac{(2c\theta -1)}{2k} \left( \frac{\theta ^2M^2\kappa }{(2c\theta -1)k}\right) \\&\le \left( \frac{u}{2k}-\left( \frac{2c\theta }{k}\right) \frac{u}{2k}\right) + \left( \frac{2c\theta -1}{2k}\right) \frac{u}{k} \\&= \frac{u}{2k}-\frac{1}{k}\left( \frac{u}{2k}\right) \le \frac{u}{2k} -\frac{1}{k+1}\left( \frac{u}{2k}\right) = \frac{u}{2(k+1)}. \end{aligned}$$

$\square$

Lemma 9

Consider the following problem: $\min \left\{ h(\gamma _0) g(z) \mid \gamma _0 \in \Gamma _0, z \in {{\mathcal {Z}}}\right\} ,$ where h and g are positive functions over $\Gamma _0$ and ${\mathcal {Z}}$, respectively. If ${\bar{\gamma }}_0$ and ${\bar{z}}$ denote global minimizers of $h(\gamma _0)$ and g(z) over $\Gamma _0$ and ${{\mathcal {Z}}}$, respectively, then the following holds:

$$\begin{aligned} {{\min _{\gamma _0 \in \Gamma _0, z \in {\mathcal {Z}}} h(\gamma _0) g(z) = h({\bar{\gamma }}_0) g({\bar{z}}).}} \end{aligned}$$

Proof

The proof has two steps. First, we note that $\min _{\gamma _0 \in \Gamma _0, z \in {\mathcal {Z}}} h(\gamma _0) g(z) \ge h({\bar{\gamma }}_0) g({\bar{z}}),$ implying that at any global minimizer $(\gamma _0^*,z^*)$,

$$\begin{aligned} h(\gamma _0^*)g(z^*) \ge h({\bar{\gamma }}_0) g({\bar{z}}). \end{aligned}$$

(35)

Second, since $({\bar{\gamma }}_0, {\bar{z}}) \in \Gamma _0 \times {\mathcal {Z}}$, we have that $h(\gamma _0^*)g(z^*)$ has an optimal value that is no smaller than that the value associated with any feasible solution or

$$\begin{aligned} h(\gamma _0^*) g(z^*) = \min _{\gamma _0 \in \Gamma _0, z \in {\mathcal {Z}}} h(\gamma _0) g(z) \le h({\bar{\gamma }}_0) g({\bar{z}}). \end{aligned}$$

(36)

By combining (35) and (36), the result follows. $\square$