Nash Equilibrium Seeking for Dynamic Systems with Non-quadratic Payoffs

Abstract

We consider general, stable nonlinear differential equations with N inputs and N outputs, where in the steady state, the output signals represent the payoff functions of a noncooperative game played by the steady-state values of the input signals. To achieve locally stable convergence to the resulting steady-state Nash equilibria, we introduce a non-model-based approach, where the players determine their actions based only on their own payoff values. This strategy is based on the extremum seeking approach, which has previously been developed for standard optimization problems and employs sinusoidal perturbations to estimate the gradient. Since non-quadratic payoffs create the possibility of multiple, isolated Nash equilibria, our convergence results are local. Specifically, the attainment of any particular Nash equilibrium is not assured for all initial conditions, but only for initial conditions in a set around that specific stable Nash equilibrium. For non-quadratic costs, the convergence to a Nash equilibrium is not perfect, but is biased in proportion to the perturbation amplitudes and the higher derivatives of the payoff functions. We quantify the size of these residual biases.

Appendix

The following integrals are computed to obtain (9.21), where we have assumed the frequencies satisfy ω _i ≠ω _j , 2ω _i ≠ω _j , 3ω _i ≠ω _j , ω _i ≠ω _j  + ω _k , ω _i ≠2ω _j  + ω _k , 2ω _i ≠ω _j  + ω _k , for distincti, j, k ∈ { 1, …, N} and defined γ _i  = ω _i  ∕ min _i {ω _i }:

$$\begin{array}{rl} \mathop{\lim}\limits_{T\rightarrow \infty }\frac{1}{T}{\int }_{0}^{T}{\mu }_{i}(\tau )\,\mathrm{d}\tau & = \mathop{\lim}\limits_{T\rightarrow \infty }\frac{{a}_{i}} {T}{\int }_{0}^{T}\sin ({\gamma }_{ i}\tau + {\varphi }_{i})\,\mathrm{d}\tau \\ & = 0,\\ \end{array}$$

(9.55)

$$\begin{array}{rl}\mathop{\lim}\limits_{T\rightarrow \infty }\frac{1} {T}{\int }_{0}^{T}{\mu }_{ i}^{2}(\tau )\,\mathrm{d}\tau & = \mathop{\lim}\limits_{ T\rightarrow \infty }\frac{{a}_{i}^{2}} {2T}{\int }_{0}^{T}[1 -\cos (2{\gamma }_{ i}\tau + 2{\varphi }_{i})]\,\mathrm{d}\tau ,\\ & = \frac{{a}_{i}^{2}}{2} ,\\ \end{array}$$

(9.56)

$$\begin{array}{rl} \mathop{\lim}\limits_{T\rightarrow \infty }\frac{1} {T}{\int }_{0}^{T}{\mu }_{i}^{3}(\tau )\,\mathrm{d}\tau & = \mathop{\lim}\limits_{T\rightarrow \infty }\frac{{a}_{i}^{3}} {4T}{\int}_{0}^{T}[3\sin ({\gamma }_{i}\tau + {\varphi }_{i}) -\sin (3{\gamma }_{i}\tau + 3{\varphi }_{i})]\,\mathrm{d}\tau , \\ & = 0,\\ \end{array}$$

(9.57)

$$\begin{array}{rl}{\lim} _{T\rightarrow \infty }\frac{1} {T}{\int }_{0}^{T}{\mu }_{i}^{4}(\tau )\,\mathrm{d}\tau & = \mathop{\lim}\limits_{ T\rightarrow \infty }\frac{{a}_{i}^{4}} {8T}{\int }_{0}^{T}[3 - 4\cos (2{\gamma }_{ i}\tau + 2{\varphi }_{i})\\ &\quad +\cos (4{\gamma }_{i}\tau + 4{\varphi }_{i})]\,\mathrm{d}\tau ,\\ & =\frac{3{a}_{i}^{4}} {8} ,\\ \end{array}$$

(9.58)

$$ \begin{array}{rl} \mathop{\lim}\limits_{T\rightarrow \infty }\frac{1} {T}{\int}_{0}^{T}{\mu }_{i}(\tau ){\mu }_{j}(\tau )\,\mathrm{d}\tau & = \frac{{a}_{i}{a}_{j}} {2T} {\int}_{0}^{T}[\cos (({\gamma }_{i} - {\gamma }_{j})\tau + {\varphi }_{i} - {\varphi }_{j})\\ {} & \quad -\cos (({\gamma }_{i} + {\gamma }_{j})\tau + {\varphi }_{i} + {\varphi }_{j})]\,\mathrm{d}\tau ,\\ & = 0, \\ \end{array}$$

(9.59)

$$\begin{array}{rl} \mathop{\lim}\limits_{T\rightarrow \infty }\frac{1} {T}{\int}_{0}^{T}{\mu }_{i}^{2}(\tau ){\mu }_{ j}(\tau )\,\mathrm{d}\tau & = \frac{{a}_{i}^{2}{a}_{j}} {2T} {\int}_{0}^{T}[\sin ({\gamma }_{j}\tau + {\varphi }_{j})\\ &\quad -\cos (2{\gamma }_{i}\tau + 2{\varphi }_{i})\sin ({\gamma }_{j}\tau + {\varphi }_{j})]\,\mathrm{d}\tau ,\\ & = \frac{{a}_{i}^{2}{a}_{j}}{4T} {\int}_{0}^{T}[2\,\sin ({\gamma }_{j}\tau + {\varphi }_{j})\\ & \quad -\sin ((2{\gamma }_{i} + {\gamma }_{j})\tau + 2{\varphi }_{i} + {\varphi }_{j})\\ &\quad+\sin ((2{\gamma }_{i} - {\gamma }_{j})\tau + 2{\varphi }_{i} - {\varphi }_{j})]\,\mathrm{d}\tau,\\ & = 0, \\ \end{array}$$

(9.60)

$$ \begin{array}{rl} \mathop{\lim}\limits_{T\rightarrow \infty }\frac{1} {T}{\int}_{0}^{T}{\mu }_{i}^{3}(\tau ){\mu }_{ j}(\tau )\,\mathrm{d}\tau & = \frac{{a}_{i}^{3}{a}_{j}} {4T} {\int}_{0}^{T}[3\sin ({\gamma }_{i}\tau + {\varphi }_{j})\sin ({\gamma }_{j}\tau + {\varphi }_{j})\\ & \quad -\sin (3{\gamma }_{i}\tau + 3{\varphi }_{i})\sin ({\gamma }_{j}\tau + {\varphi }_{j})]\,\mathrm{d}\tau ,\\ & = \frac{{a}_{i}^{3}{a}_{j}}{8T} {\int}_{0}^{T}[3\cos (({\gamma }_{i} - {\gamma }_{j})\tau + {\varphi }_{i} - {\varphi }_{j})\\ & \quad -3\cos (({\gamma }_{i} + {\gamma }_{j})\tau + {\varphi }_{i} + {\varphi }_{j})\\ & \quad-\cos ((3{\gamma }_{i} - {\gamma }_{j})\tau + 3{\varphi }_{i} - {\varphi }_{j})\\ & \quad +\cos ((3{\gamma }_{i} + {\gamma }_{j})\tau + 3{\varphi }_{i} + {\varphi }_{j})]\,\mathrm{d}\tau ,\\ & =0, \\ \end{array}$$

(9.61)

$$\begin{array}{rl} \mathop{\lim}\limits_{T\rightarrow \infty }\frac{1}{T}{\int}_{0}^{T}{\mu }_{ i}^{2}(\tau ){\mu }_{j}^{2}(\tau )\,\mathrm{d}\tau & = \mathop{\lim}\limits_{T\rightarrow \infty }\frac{{a}_{i}^{2}{a}_{j}^{2}} {8T} {\int}_{0}^{T}[2 - 2\cos (2{\gamma }_{i}\tau + 2{\varphi }_{i})\\ & \quad -2\,\cos (2{\gamma }_{j}\tau + 2{\varphi }_{j})\\ & \quad +\cos (2({\gamma }_{i} - {\gamma }_{j})\tau + 2({\varphi }_{i} - {\varphi }_{j}))\\ &\quad +\cos (2({\gamma }_{i} + {\gamma }_{j})\tau + 2({\varphi }_{i} + {\varphi }_{j}))]\,\mathrm{d}\tau ,\\ & =\frac{{a}_{i}^{2}{a}_{j}^{2}}{4} , \\ \end{array} $$

(9.62)

$$\begin{array}{rl} \mathop{\lim}\limits_{T\rightarrow \infty }\frac{1}{T}{\int}_{0}^{T}{\mu }_{ i}(\tau ){\mu }_{j}(\tau ){\mu }_{k}(\tau )\,\mathrm{d}\tau ,& = \mathop{\lim}\limits_{T\rightarrow \infty }\frac{{a}_{i}{a}_{j}{a}_{k}} {2T} {\int}_{0}^{T}[\cos (({\gamma }_{i} - {\gamma }_{j})\tau + {\varphi }_{i} - {\varphi }_{j})\\ & \quad -\cos (({\gamma }_{i} + {\gamma }_{j})\tau + {\varphi }_{i} + {\varphi }_{j})]\sin ({\gamma }_{k}\tau + {\varphi }_{k})\,\mathrm{d}\tau ,\\ & = \mathop{\lim}\limits_{T\rightarrow \infty }\frac{{a}_{i}{a}_{j}{a}_{k}}{4T}\\ & \quad \times {\int}_{0}^{T}[\sin (({\gamma }_{i} - {\gamma }_{j} + {\omega }_{k})\tau + {\varphi }_{i} - {\varphi }_{j} + {\varphi }_{k})\\ & \quad -\sin (({\gamma }_{i} - {\gamma }_{j} - {\gamma }_{k})\tau + {\varphi }_{i} - {\varphi }_{j} - {\varphi }_{k})\\ & \quad -\sin (({\gamma }_{i} + {\gamma }_{j} + {\gamma }_{k})\tau + {\varphi }_{i} + {\varphi }_{j} + {\varphi }_{k})\\ & \quad +\sin (({\gamma }_{i} + {\gamma }_{j} - {\gamma }_{k})\tau + {\varphi }_{i} + {\varphi }_{j} - {\varphi }_{k})]\,\mathrm{d}\tau ,\\ & = 0, \\ \end{array}$$

(9.63)

$$\begin{array}{rl} \mathop{\lim}\limits_{T\rightarrow \infty }\frac{1}{T}{\int}_{0}^{T}{\mu }_{i}(\tau ){\mu }_{j}^{2}(\tau ){\mu }_{k}(\tau )\,\mathrm{d}\tau ,& = \mathop{\lim}\limits_{T\rightarrow \infty }\frac{{a}_{i}{a}_{j}^{2}{a}_{k}} {2T} {\int}_{0}^{T}\sin ({\gamma }_{i}\tau + {\varphi }_{i})\\ & \quad \times (1 -\cos (2{\gamma }_{j}\tau + 2{\varphi }_{j}))\sin ({\gamma }_{k}\tau + {\varphi }_{k})\,\mathrm{d}\tau ,\\ & = \mathop{\lim}\limits_{T\rightarrow \infty }\frac{{a}_{i}{a}_{j}^{2}{a}_{k}}{4T} {\int}_{0}^{T}[\cos (({\gamma }_{i} - {\gamma }_{k})\tau + {\varphi }_{i} - {\varphi }_{k})\\ & \quad -\cos (({\gamma }_{i} + {\gamma }_{k})\tau + {\varphi }_{i} + {\varphi }_{k})]\\ & \quad \times (1 -\cos (2{\gamma }_{j}\tau + 2{\varphi }_{j}))\,\mathrm{d}\tau\\ & = \mathop{\lim}\limits_{T\rightarrow \infty }\frac{{a}_{i}{a}_{j}^{2}{a}_{k}}{8T} {\int}_{0}^{T}[2\,\cos (({\gamma }_{i} - {\gamma }_{k})\tau + {\varphi }_{i} - {\varphi }_{k})\\ & \quad -\cos (({\gamma }_{i} - 2{\gamma }_{j} - {\gamma }_{k})\tau + {\varphi }_{i} - 2{\varphi }_{j} - {\varphi }_{k})\\ & \quad -\cos (({\gamma }_{i} + 2{\gamma }_{j} - {\gamma }_{k})\tau + {\varphi }_{i} + 2{\varphi }_{j} - {\varphi }_{k})\\ & \quad +\cos (({\gamma }_{i} - 2{\gamma }_{j} + {\gamma }_{k})\tau + {\varphi }_{i} - 2{\varphi }_{j} + {\varphi }_{k})\\ & \quad +\cos (({\gamma }_{i} + 2{\gamma }_{j} + {\gamma }_{k})\tau + {\varphi }_{i} + 2{\varphi }_{j} + {\varphi }_{k})\\ & \quad -2\cos (({\gamma }_{i} + {\gamma }_{k})\tau + {\varphi }_{i} + {\varphi }_{k})]\,\mathrm{d}\tau ,\\ & =0. \\ \end{array}$$

(9.64)

The conditions 3ω _i ≠ω _j , ω _i ≠2ω _j  + ω _k , and 2ω _i ≠ω _j  + ω _k , arise due to the payoff functions being non-quadratic and are not required for quadratic payoff functions.