On the computation of equilibria in monotone and potential stochastic hierarchical games

Abstract

We consider a class of noncooperative hierarchical ${\textbf{N}}$-player games where the ith player solves a parametrized stochastic mathematical program with equilibrium constraints (MPEC) with the caveat that the implicit form of the ith player’s in MPEC is convex in player strategy, given rival decisions. Few, if any, general purpose schemes exist for computing equilibria, motivating the development of computational schemes in two regimes: (a) Monotone regimes. When player-specific implicit problems are convex, then the necessary and sufficient equilibrium conditions are given by a stochastic inclusion. Under a monotonicity assumption on the operator, we develop a variance-reduced stochastic proximal-point scheme that achieves deterministic rates of convergence in terms of solving proximal-point problems in monotone/strongly monotone regimes with optimal or near-optimal sample-complexity guarantees. Finally, the generated sequences are shown to converge to an equilibrium in an almost-sure sense in both monotone and strongly monotone regimes; (b) Potentiality. When the implicit form of the game admits a potential function, we develop an asynchronous relaxed inexact smoothed proximal best-response framework, requiring the efficient computation of an approximate solution of an MPEC with a strongly convex implicit objective. To this end, we consider an $\eta$-smoothed counterpart of this game where each player’s problem is smoothed via randomized smoothing. In fact, a Nash equilibrium of the smoothed counterpart is an $\eta$-approximate Nash equilibrium of the original game. Our proposed scheme produces a sequence and a relaxed variant that converges almost surely to an $\eta$-approximate Nash equilibrium. This scheme is reliant on resolving the proximal problem, a stochastic MPEC whose implicit form has a strongly convex objective, with increasing accuracy in finite-time. The smoothing framework allows for developing a variance-reduced zeroth-order scheme for such problems that admits a fast rate of convergence. Numerical studies on a class of multi-leader multi-follower games suggest that variance-reduced proximal schemes provide significantly better accuracy with far lower run-times. The relaxed best-response scheme scales well with problem size and generally displays more stability than its unrelaxed counterpart.

Appendix

1.1 Variational inequality problems, Inclusions, and monotonicity

(a) Variational inequality problems and Inclusions. Consider a variational inequality problem VI$({\mathcal {X}}, F)$ where ${\mathcal {X}}$ is a closed and convex set and $F: \mathbb {R}^n \rightarrow \mathbb {R}^n$ is a single-valued continuous map. Such a problem requires an $\textbf{x}$ such that

$$\begin{aligned} ({\tilde{\textbf{x}}}-\textbf{x})^{\textsf{T}} F(\textbf{x}) \ge 0, \qquad \forall {\tilde{\textbf{x}}} \in \mathcal{X}. \end{aligned}$$

Furthermore, VI$(\mathcal{X},F)$ can also be written as an inclusion problem, i.e.

$$\begin{aligned} \textbf{x}\text { solves } \text {VI}(\mathcal{X},F) \qquad \iff \qquad 0 \in F(\textbf{x}) + {\mathcal {N}}_{\mathcal{X}}(\textbf{x}). \end{aligned}$$

Consider an ${\textbf{N}}$-player game ${\mathscr {G}}$ where for $i=1, \cdots , {\textbf{N}}$, the ith player minimizes the parametrized smooth convex optimization problem defined as

By convexity assumptions, the set of Nash equilibria of ${\mathscr {G}}$ is equivalent to the solution set of the variational inequality problem VI$({\mathcal {X}}, F)$ where ${\mathcal {X}} \triangleq \prod _{i=1}^{\textbf{N}}\mathcal{X}_i$ and

$$\begin{aligned} F(\textbf{x}) \triangleq \begin{pmatrix} \nabla _{\textbf{x}^1} f_1(\textbf{x}) \\ \vdots \\ \nabla _{\textbf{x}^{\textbf{N}}} f_{\textbf{N}}(\textbf{x}) \end{pmatrix}.\end{aligned}$$

(39)

If f is a nonsmooth convex function, then the subdifferential $\partial f$ is also a monotone set-valued (or multi-valued) map on $\mathcal{X}$. In addition, if $f_i(\bullet , \textbf{x}^{-i})$ is not necessarily smooth, then the associated set of equilibria are given by the solution of VI$(\mathcal{X}, T)$ where

$$\begin{aligned} T(\textbf{x}) \triangleq \prod _{i=1}^{\textbf{N}}\partial _{\textbf{x}^i} f_i(\textbf{x}^i,\textbf{x}^{-i}). \end{aligned}$$

(40)

(b) Monotonicity properties. Consider VI$(\mathcal{X}, F)$. Then the map F is monotone on ${\mathcal {X}}$ if $(F(\textbf{x})-F(\textbf{y}))^{\textsf{T}}(\textbf{x}-\textbf{y}) \ge 0$ for all $\textbf{x}, \textbf{y}\in \mathcal{X}$. Monotonicity may also emerge in the context of ${\textbf{N}}$-player noncooperative games. In particular, one may view ${\mathscr {G}}$ as being monotone if and only if the associated map F, defined as (39), is monotone on $\mathcal{X}$. In the special case when ${\textbf{N}}= 1$, this reduces to the gradient map of a smooth convex function f, denoted by $\nabla f$, being monotone. This can also be generalized to set-valued regimes. For instance, the map T, defined as (40), arising from a noncooperative game ${\mathscr {G}}$ with nonsmooth player-specific objectives is said to be monotone if for any $\textbf{x}, \textbf{y}\in \mathcal{X}$ and any $u \in T(\textbf{x})$ and $v \in T(\textbf{y})$, we have $(u-v)^{\textsf{T}}(\textbf{x}-\textbf{y}) \ge 0$.

(c) Monotonicity in the context of single-leader single-follower. Consider a single-leader single-follower problem in which the follower’s objective $g(\textbf{x},\bullet )$ is a strongly convex function on $\mathcal{Y}$, a closed and convex set while $\mathcal{X}$ is also a closed and convex.

$$\begin{aligned} \min _{\textbf{x}\in \mathcal{X}} \&f(\textbf{x},\textbf{y}(\textbf{x})), \text { where } \end{aligned}$$

(Leader)

$$\begin{aligned} \textbf{y}(\textbf{x}) = \text {arg}\min _{\textbf{y}\in \mathcal{Y}} \ {}&g(\textbf{x},\textbf{y}). \end{aligned}$$

(Follower)

There are many instances when $f(\bullet , \textbf{y}(\bullet ))$ is a convex function on $\mathcal{X}$ (see [5, 6, 10, 91] for some instances) implying that $\partial f(\bullet , \textbf{y}(\bullet ))$ is a monotone map on $\mathcal{X}$. In other words, the implicit problem in leader-level decisions can be seen to be characterized by a convex objective with a monotone map. However, when viewing the problem in the full space of $\textbf{x}$ and $\textbf{y}$, i.e.

$$\begin{aligned} \begin{aligned} \min _{\textbf{x}\in \mathcal{X}, \textbf{y}} \ f(\textbf{x},\textbf{y})&\\ ({\tilde{\textbf{y}}}-\textbf{y})^{\textsf{T}} \nabla _{\textbf{y}} g(\textbf{x},\textbf{y})&\ge 0, \qquad \forall {\tilde{\textbf{y}}} \ \in \ \mathcal{Y}. \end{aligned} \end{aligned}$$

In the full space of $\textbf{x}$ and $\textbf{y}$, this is indeed a nonconvex optimization problem [15]; however, the implicit problem in x may be convex under some assumptions and the resulting subdifferential map is then monotone.

1.2 Proofs

Proof of Proposition 1

In both cases, it is not difficult to see that H is a monotone map where $H(\textbf{x}) \triangleq \prod _{i=1}^{\textbf{N}}\partial _{\textbf{x}^i} h_i(\textbf{x}^i,\textbf{x}^{-i})$. Consequently, if T is defined as $T(\textbf{x})=G(\textbf{x})+H(\textbf{x})$, then T is a monotone map which follows from the monotonicity of G, defined as $G(\textbf{x}) \triangleq \prod _{i=1}^{\textbf{N}}\partial _{\textbf{x}^i} g_i(\textbf{x}^i,\textbf{x}^{-i})$. $\square$

Proof of Proposition 2

For (a), potentiality follows by noting that for any $\textbf{x}^i, {\tilde{\textbf{x}}}^i \in \mathcal{X}_i$ and $\textbf{x}^{-i} \in \mathcal{X}^{-i}$, we have

$$\begin{aligned} {\widehat{P}}(\textbf{x}^i,\textbf{x}^{-i}) - {\widehat{P}}({\tilde{\textbf{x}}}^i,\textbf{x}^{-i})&= P(\textbf{x}^i,\textbf{x}^{-i}) - P({\tilde{\textbf{x}}}^i,\textbf{x}^{-i}) + h_i(\textbf{x}^i,\textbf{y}^i(\textbf{x}^i)) - h_i({\tilde{\textbf{x}}}^i,\textbf{y}^i({\tilde{\textbf{x}}}^i)) \\&= g_i(\textbf{x}^i,\textbf{x}^{-i})+h_i(\textbf{x}^i,\textbf{y}^i(\textbf{x}^i,\textbf{x}^{-i}))\\&\quad -(g_i({\tilde{\textbf{x}}}^i,\textbf{x}^{-i})+h_i({\tilde{\textbf{x}}}^i,\textbf{y}^i({\tilde{\textbf{x}}}^i,\textbf{x}^{-i}))). \end{aligned}$$

For (b), proceeding in a similar fashion, it follows that for any $\textbf{x}^i, {\tilde{\textbf{x}}}^i \in \mathcal{X}_i$ and $\textbf{x}^{-i} \in \mathcal{X}^{-i}$, we have

$$\begin{aligned} {\widehat{P}}(\textbf{x}^i,\textbf{x}^{-i}) - {\widehat{P}}({\tilde{\textbf{x}}}^i,\textbf{x}^{-i})&= P(\textbf{x}^i,\textbf{x}^{-i}) - P({\tilde{\textbf{x}}}^i,\textbf{x}^{-i}) + h(\textbf{x}^i,\textbf{x}^{-i}) - h({\tilde{\textbf{x}}}^i,\textbf{x}^{-i})\\&= g_i(\textbf{x}^i,\textbf{x}^{-i})+h_i(\textbf{x}^i,\textbf{y}^i(\textbf{x}^i,\textbf{x}^{-i}))\\&\quad -(g_i({\tilde{\textbf{x}}}^i,\textbf{x}^{-i})+h_i({\tilde{\textbf{x}}}^i,\textbf{y}^i({\tilde{\textbf{x}}}^i,\textbf{x}^{-i}))). \end{aligned}$$

$\square$

Proof of Lemma 2

Suppose $J_2$ denotes a positive integer such that $(1-2c\alpha _j) \ge 0$ for $j \ge J_2$, i.e. $J_2 = \lceil 2c\theta \rceil \ge 2c\theta .$ Let $J \triangleq \max \{J_1,J_2\}$ and ${\mathcal {D}} \triangleq \max \left\{ \tfrac{{\mathcal {M}}^2 \theta ^2}{2(2c\theta -1)},J {\mathcal {A}}_J\right\} .$ For $j = J$, the inductive hypothesis holds trivially. If it holds for some $j > J$,

$$\begin{aligned} {\mathcal {A}}_{j+1}&\le (1-2c\alpha _j) {\mathcal {A}}_j + \tfrac{\alpha _j^2 {\mathcal {M}}^2}{2} \le (1-2c\alpha _j) \tfrac{{\mathcal {D}}}{j} + \tfrac{\alpha _j^2 {\mathcal {M}}^2}{2}\\&= (1-2c\alpha _j) \tfrac{{\mathcal {D}}}{j} + \tfrac{2(2c\theta -1)}{2j}\tfrac{\theta ^2 {\mathcal {M}}^2}{2(2c\theta -1)j} \le (1-2c\alpha _j) \tfrac{{\mathcal {D}}}{j} + \tfrac{2c\theta -1}{j}\tfrac{{\mathcal {D}}}{j} \\&\le (1-\tfrac{2c\theta }{j}) \tfrac{{\mathcal {D}}}{j} + \tfrac{2c\theta -1}{j}\tfrac{{\mathcal {D}}}{j} = \tfrac{{\mathcal {D}}}{j} - \tfrac{2c\theta {\mathcal {D}}}{j^2} + \tfrac{2c\theta {\mathcal {D}}}{j^2} - \tfrac{{\mathcal {D}}}{j^2} \le \tfrac{{\mathcal {D}}}{j} - \tfrac{{\mathcal {D}}}{j(j+1)} = \tfrac{{\mathcal {D}}}{(j+1)}. \end{aligned}$$

It remains to get a bound on ${\mathcal {A}}_J$.

$$\begin{aligned} {\mathcal {A}}_J&\le (1-2c\alpha _{J-1}){\mathcal {A}}_{{J-1}} + \tfrac{\alpha _{J-1}^2 {\mathcal {M}}^2}{2} \le {\mathcal {A}}_{{J-1}} + \tfrac{\alpha _{J-1}^2 {\mathcal {M}}^2}{2}\nonumber \\&\le \left( (1-2c\alpha _{J-2}) {\mathcal {A}}_{J-2} + \tfrac{\alpha _{J-2}^2{\mathcal {M}}^2}{2}\right) + \tfrac{\alpha _{J-1}^2{\mathcal {M}}^2}{2} \le {\mathcal {A}}_{{1}} + {\mathcal {M}}^2\sum _{\ell = 1}^{J-1} \tfrac{\alpha _{\ell }^2 }{2} \nonumber \\&\le {\mathcal {A}}_{{1}} + \tfrac{{\mathcal {M}}^2\theta ^2\pi ^2}{12} \triangleq {{\mathcal {A}}_1 + B {\mathcal {M}}^2}, {\text { since } \sum _{\ell =0}^{J-1} \tfrac{1}{\ell ^2} \le \tfrac{ \pi ^2}{6}.} \end{aligned}$$

(41)

Consequently, for $j \ge J$, ${\mathcal {A}}_{j} \le \frac{\max \left\{ \tfrac{{\mathcal {M}}^2\theta ^2}{2(2c\theta -1)}, J {\mathcal {A}}_{J}\right\} }{2j} \le \frac{\tfrac{{\mathcal {M}}^2\theta ^2}{2(2c\theta -1)}+ J ({{\mathcal {A}}_{{1}} + B{\mathcal {M}}^2})}{2j}.$ $\square$

Proof of Proposition 5

Throughout this proof, we refer to $J_{\lambda }^T(\textbf{x}^k)$ by $\textbf{z}^{k,*}$ to ease the exposition. Consider the update rule given by (SA), given $\textbf{z}_{0} = \textbf{x}^k$. We have that

$$\begin{aligned} \Vert \textbf{z}_{j+1}-\textbf{z}^{k,*}\Vert ^2 \! =\! \Vert \textbf{z}_j \!-\! \alpha _j u_j - \textbf{z}^{k,*}\Vert ^2 \!=\! \Vert \textbf{z}_j - \textbf{z}^{k,*}\Vert ^2 + \alpha _j^2 \Vert u_j\Vert ^2 \!-\! 2\alpha _j u_j^{\textsf{T}}(\textbf{z}_j-\textbf{z}^{k,*}). \end{aligned}$$

Taking expectations on both sides, we obtain that

$$\begin{aligned}&{\mathbb {E}}[\Vert \textbf{z}_{j+1} - \textbf{z}^{k,*}\Vert ^2 \mid {\mathcal {F}}_{k,j}] = \Vert \textbf{z}_j - \textbf{z}^{k,*}\Vert ^2 + \alpha _j^2 {\mathbb {E}}[\Vert u_j\Vert ^2 \mid {\mathcal {F}}_{k,j}]\\&\qquad - 2\alpha _j {\mathbb {E}}[u_j^{\textsf{T}}(\textbf{z}_j-\textbf{z}^{k,*}) \mid {\mathcal {F}}_{k,j}] \\&\quad = \Vert \textbf{z}_j - \textbf{z}^{k,*}\Vert ^2 + \alpha _j^2 {\mathbb {E}}[\Vert u_j\Vert ^2 \mid {\mathcal {F}}_{k,j}] - 2\alpha _j{\mathbb {E}}[u_j \mid {\mathcal {F}}_{k,j}]^{\textsf{T}}(\textbf{z}_j-\textbf{z}^{k,*}) \\&\quad = \Vert \textbf{z}_j - \textbf{z}^{k,*}\Vert ^2 + \alpha _j^2 {\mathbb {E}}[\Vert u_j\Vert ^2\mid {\mathcal {F}}_{k,j}] - 2\alpha _j {\bar{u}}_j^{\textsf{T}}(\textbf{z}_j-\textbf{z}^{k,*}) \\&\quad - 2\underbrace{{\mathbb {E}}[\alpha _j(u_j-{\bar{u}}_j)^{\textsf{T}}(\textbf{z}_j-\textbf{z}^{k,*})\mid {\mathcal {F}}_{k,j}]}_{\ = \ 0}\\&\quad = \Vert \textbf{z}_j - \textbf{z}^{k,*}\Vert ^2 + \alpha _j^2 {\mathbb {E}}[\Vert u_j\Vert ^2 \mid {\mathcal {F}}_{k,j}] - 2\alpha _j ({\bar{u}}_j-{\bar{u}}_{k}^*)^{\textsf{T}}(\textbf{z}_j-\textbf{z}^{k,*}), \end{aligned}$$

where ${\bar{u}}_j \in F_k(\textbf{z}_j)$, ${\mathbb {E}}[\alpha _j(u_j-{\bar{u}}_j)^{\textsf{T}}(\textbf{z}_j-\textbf{z}^{k,*})\mid {\mathcal {F}}_{k,j}] = \alpha _j({\mathbb {E}}[u_j \mid {\mathcal {F}}_{k,j}]-{\bar{u}}_j)^{\textsf{T}}(\textbf{z}_j-\textbf{z}^{k,*}) = 0$, $0 = {\bar{u}}_k^* \in F_k(\textbf{z}^{k,*})$ and $({\bar{u}}_j-{\bar{u}}_{k}^*)^{\textsf{T}}(\textbf{z}_j-\textbf{z}^{k,*}) \ge \tfrac{1}{\lambda }\Vert \textbf{z}_j-\textbf{z}^{k,*}\Vert ^2$ by the $\tfrac{1}{\lambda }$-strong monotonicity of $F_k$. Consequently, we have that

$$\begin{aligned}&{\mathbb {E}}[\Vert \textbf{z}_{j+1} - \textbf{z}^{k,*}\Vert ^2 \mid {\mathcal {F}}_{k,j}] \le (1-\tfrac{2\alpha _j}{\lambda })\Vert \textbf{z}_j - \textbf{z}^{k,*}\Vert ^2 + \alpha _j^2 {\mathbb {E}}[\Vert u_j\Vert ^2 \mid {\mathcal {F}}_{k,j}] \\&\quad \overset{{(5)}}{\le } (1-\tfrac{2\alpha _j}{\lambda })\Vert \textbf{z}_j - \textbf{z}^{k,*}\Vert ^2 + \alpha _j^2 (4M_1^2 \Vert \textbf{x}^k\Vert ^2 + 2M_2^2 + (4M_1^2+\tfrac{2}{\lambda ^2})\Vert \textbf{z}_j-\textbf{x}^k\Vert ^2) \\&\quad \le (1-\tfrac{2\alpha _j}{\lambda })\Vert \textbf{z}_j - \textbf{z}^{k,*}\Vert ^2 + \alpha _j^2 (({8}M_1^2+\tfrac{{4}}{\lambda ^2}) \Vert \textbf{z}_j-\textbf{z}^{k,*}\Vert ^2 + 4M_1^2\Vert \textbf{x}^k\Vert ^2 \\&\qquad + 2M_2^2 + (8M_1^2+\tfrac{4}{\lambda ^2})\Vert \textbf{z}^{k,*}-\textbf{x}^k\Vert ^2) \\&\quad \le (1-2\alpha _j(\tfrac{1}{\lambda }-\alpha _j (4M_1^2+\tfrac{2}{\lambda ^2}))\Vert \textbf{z}_j - \textbf{z}^{k,*}\Vert ^2\\&\qquad + \alpha _j^2 (4M_1^2\Vert \textbf{x}^k\Vert ^2+ 2M_2^2 + (8M_1^2+\tfrac{4}{\lambda ^2})\Vert \textbf{z}^{k,*}-\textbf{x}^k\Vert ^2) \\&\quad \le (1-\tfrac{\alpha _j}{\lambda })\Vert \textbf{z}_j - \textbf{z}^{k,*}\Vert ^2+ \alpha _j^2 (4M_1^2\Vert \textbf{x}^k\Vert ^2+ 2M_2^2 + (8M_1^2+\tfrac{4}{\lambda ^2})\Vert \textbf{z}^{k,*}-\textbf{x}^k\Vert ^2), \end{aligned}$$

where the last inequality follows from $\alpha _j ({8}M_1^2+\tfrac{{4}}{\lambda ^2}) \le \tfrac{1}{2\lambda }$ for $j \ge J_1$ where $j \ge J_1 \triangleq \lceil 2\lambda \theta ({8}M_1^2+\tfrac{{4}}{\lambda ^2})\rceil$. Taking expectations conditioned on ${\mathcal {F}}_k$ and recalling that ${\mathbb {E}}[[\Vert \textbf{z}_{j+1} - \textbf{z}^{k,*}\Vert ^2 \mid {\mathcal {F}}_{k,j}]\mid {\mathcal {F}}_k] = {\mathbb {E}}[\Vert \textbf{z}_{j+1} - \textbf{z}^{k,*}\Vert ^2\mid {\mathcal {F}}_k]$ since ${\mathcal {F}}_k \subset {\mathcal {F}}_{k,j}$, we obtain the following inequality for $j \ge J_1$,

$$\begin{aligned} {\mathbb {E}}[\Vert \textbf{z}_{j+1} - \textbf{z}^{k,*}\Vert ^2 \mid {\mathcal {F}}_k]&\le (1-\tfrac{\alpha _j}{\lambda }){\mathbb {E}}[\Vert \textbf{z}_j - \textbf{z}^{k,*}\Vert ^2\mid {\mathcal {F}}_k]\\&\quad + \alpha _j^2 (4M_1^2{\mathbb {E}}[\Vert \textbf{x}^k\Vert ^2 \mid {\mathcal {F}}_k]+ 2M_2^2 \\&\quad + (8M_1^2+\tfrac{4}{\lambda ^2}){\mathbb {E}}[\Vert \textbf{z}^{k,*}-\textbf{x}^k\Vert ^2 \mid {\mathcal {F}}_k]). \end{aligned}$$

Consequently, if $\alpha _j = \tfrac{\theta }{j}$, we have a recursion given by

$$\begin{aligned} {\mathcal {A}}_{j+1} \le (1-2c \alpha _j) {\mathcal {A}}_j + \tfrac{\alpha _j^2 {\mathcal {M}}^2}{2}, \quad j \ge J_1 \end{aligned}$$

where ${\mathcal {A}}_j \triangleq {\mathbb {E}}[\Vert \textbf{z}_{j}-\textbf{z}^{k,*}\Vert ^2 \mid {\mathcal {F}}_k]$, $c = \tfrac{1}{2\lambda }$, $\alpha _j = \tfrac{\theta }{j}$, and ${\mathcal {M}}^2/2 = 4M_1^2{\mathbb {E}}[\Vert \textbf{x}^k\Vert ^2\mid {\mathcal {F}}_k]+ 2M_2^2 + (8M_1^2+\tfrac{4}{\lambda ^2}){\mathbb {E}}[\Vert \textbf{z}^{k,*}-\textbf{x}^k\Vert ^2\mid {\mathcal {F}}_k]$. By Lemma 2, we have that

$$\begin{aligned} {\mathcal {A}}_j \le \frac{\tfrac{{\mathcal {M}}^2\theta ^2}{2(2c\theta -1)}+ J {({\mathcal {A}}_1+B{\mathcal {M}}^2)}}{2j}, \quad j \ge J \end{aligned}$$

(42)

where $J \triangleq \max \{J_1,J_2\}$, $J_2 \triangleq \lceil 2c\theta \rceil$, and ${B \triangleq \tfrac{\theta ^2 \pi ^2}{12}}$. Since ${\mathcal {A}}_1 = {\mathbb {E}}[\Vert \textbf{z}^{k,*}-\textbf{x}^k\Vert ^2 \mid {\mathcal {F}}_k]$, the numerator in (42) may be further bounded as follows.

$$\begin{aligned}&\tfrac{{\mathcal {M}}^2\theta ^2}{2(2c\theta -1)}+J {({\mathcal {A}}_1+B{\mathcal {M}}^2)} = \left( \tfrac{\theta ^2 }{2(2c\theta -1)}+J {B}\right) {\mathcal {M}}^2 + {J{\mathcal {A}}_1} \nonumber \\&\quad \le \left( \tfrac{\theta ^2}{2(2c\theta -1)}+J{B}\right) (8M_1^2\Vert \textbf{x}^k\Vert ^2+ 4M_2^2 + (16M_1^2+\tfrac{8}{\lambda ^2}){\mathbb {E}}[\Vert \textbf{z}^{k,*}-\textbf{x}^k\Vert ^2\mid {\mathcal {F}}_k]) \nonumber \\&\qquad + J{\mathbb {E}}[\Vert \textbf{z}^{k,*}-\textbf{x}^k\Vert ^2 \mid {\mathcal {F}}_k] \nonumber \\&\quad = \left( \tfrac{\theta ^2}{2(2c\theta -1)}+J {B}\right) (8M_1^2\Vert \textbf{x}^k\Vert ^2+ 4M_2^2) \nonumber \\&\qquad + \left( \left( \tfrac{\theta ^2}{2(2c\theta -1)}+J{B}\right) \left( 16M_1^2+\tfrac{8}{\lambda ^2}\right) +J\right) {\mathbb {E}}[\Vert \textbf{z}^{k,*}-\textbf{x}^k\Vert ^2\mid {\mathcal {F}}_k]. \end{aligned}$$

(43)

We have that

$$\begin{aligned}&{\mathbb {E}}[\Vert \textbf{z}^{k,*}-\textbf{x}^k\Vert ^2 \mid {\mathcal {F}}_k] \le 2\Vert \textbf{x}^k - \textbf{x}^*\Vert ^2 + 2{\mathbb {E}}[\Vert \textbf{z}^{k,*} - \textbf{x}^*\Vert ^2 \mid {\mathcal {F}}_k] \\&\qquad = 2\Vert \textbf{x}^k - \textbf{x}^*\Vert ^2 + 2{\mathbb {E}}[\Vert J_{\lambda }^T(\textbf{x}^k) - \textbf{x}^*\Vert ^2 \mid {\mathcal {F}}_k] \\&\qquad \le 2\Vert \textbf{x}^k - \textbf{x}^*\Vert ^2 + 2\Vert \textbf{x}^k - \textbf{x}^*\Vert ^2 \le 8 \Vert \textbf{x}^k\Vert ^2+8 \Vert \textbf{x}^*\Vert ^2, \end{aligned}$$

where the second inequality follows from $\Vert J_{\lambda }^T(\textbf{x}^k)-\textbf{x}^*\Vert = \Vert J^T_{\lambda }(\textbf{x}^k)-J^T_{\lambda }(\textbf{x}^*)\Vert \le \Vert \textbf{x}^k-\textbf{x}^*\Vert .$ Consequently, from (43), $\tfrac{{\mathcal {M}}^2\theta ^2}{2(2c\theta -1)}+J {({\mathcal {A}}_1+B{\mathcal {M}}^2)} \le \nu _1^2 \Vert \textbf{x}^k\Vert ^2 + \nu _2^2$, where

$$\begin{aligned} \nu _1^2&\triangleq \left( \left( \tfrac{\theta ^2}{2(2c\theta -1)}+JB\right) \left( {136}M_1^2+\tfrac{64}{\lambda ^2}\right) +{8J}\right) \text { and } \\ \nu _2^2&\triangleq 4\left( \tfrac{\theta ^2}{2(2c\theta -1)}+J {B}\right) M_2^2 + 8\left( \left( \tfrac{\theta ^2}{2(2c\theta -1)}+J{B}\right) \left( 16M_1^2+\tfrac{8}{\lambda ^2}\right) +J\right) \Vert \textbf{x}^*\Vert ^2. \end{aligned}$$

$\square$

Proposition 15

For $i=1,\cdots , {\textbf{N}}$, consider the problem (Player$_i(\textbf{x}^{-i}$)). Suppose for $i =1, \cdots , {\textbf{N}}$, (a.i) and (a.ii) hold.

(a.i) $\mathcal{X}_i \subseteq \mathbb {R}^{n_i}$ and $\mathcal{Y}_i \subseteq \mathbb {R}^{m_i}$ are closed and convex sets.

(a.ii) $F_i(\textbf{x},\bullet ,\omega )$ is a $\mu _F(\omega )$-strongly monotone and $L_F(\omega )$-Lipschitz continuous map on $\mathcal{Y}$ uniformly in $\textbf{x}\in \mathcal{X}$ for every $\omega \in \Omega$, and there exist scalars $\mu _F,L_F > 0$ such that $\inf _{\omega \in \Omega }\mu _F(\omega ) \ge \mu _F$ and $\sup _{\omega \in \Omega }L_F(\omega ) \le L_F$.

Suppose ${\tilde{f}}_i(\textbf{x},\textbf{y}_i,\omega )$ is continuously differentiable on $\mathcal{C}\times \mathbb {R}^{m_i}$ for every $\omega \in \Omega$ where $\mathcal{C}$ is an open set containing $\mathcal{X}$ and $\mathcal{X}$ is bounded. Then the function $f_i^\textbf{imp}$, defined as ${f_i^{\textbf{imp}}(\textbf{x})} \triangleq {\mathbb {E}}[{\tilde{f}}(\textbf{x},\textbf{y}(\textbf{x},\omega ), \omega )]$, is Lipschitz continuous and directionally differentiable on $\mathcal{X}$.