Pareto Improvements of Nash Equilibria in Differential Games

Abstract

Frequently controls forming Nash equilibria in differential games are not Pareto optimal. This paper presents conditions that can be used to show the existence of strict Pareto improvements of Nash equilibria in such games. The conditions are based on standard tools in control theory.

Appendix

In this Appendix we consider a standard control system

$$ \dot{x}(t)=f\bigl(t,x(t),u(t)\bigr),\qquad x(0)=x_{0},\quad x_{0}\in \mathbb{R}^{n}\text{ fixed},\ u(t)\in U\ \text{for all }t, $$

(26)

where x ^u(.) is the solution corresponding to u=u(.), U is a fixed bounded set, f and f _x are continuous. The criterion is ax(T), with a a given nonzero vector.

Let U′ be the set of measurable functions with values in U, let $\hat{u}(.)$ ∈U′ be a given control with corresponding solution (of (26)) denoted $\hat{x}(t)$, assumed to exist. For u,u′∈U′, let σ(u,u′):=meas{t:u(t)≠u′(t)}.

Note that $\hat{x}([0,T])$ is compact, so there exists an open bounded set $B\subset \mathbb{R}^{n}$ containing $\hat{x}([0,T])$ such that f and f _x are bounded on [0,T]×B×U by some constant M. For u=u(.) σ—close to $\hat{u}(.)$, x ^u(.) exists and x ^u(t)∈B for all t. Let {t ^j} be a finite set of distinct Lebesgue points of $d\hat{x}/dt$ in (0,T), let u ^j∈U, and let $u_{\delta }(t):=\sum_{j}u^{j}1_{[t^{j}-\delta s^{j},t^{j}]}(t)+\hat{u}(t)(1-\sum_{j}1_{[t^{j}-\delta s^{j},t^{j}]}(t))$, s ^j given positive numbers, δ≥0.

Let C(T,t) be the resolvent of the linear equation $\dot{q}(t)=f_{x}(t, \hat{x}(t),\hat{u}(t))q(t)$, and for any u∈U,t∈(0,T), let q ^u,t(T)= $C(T,t)[f(t,\hat{x}(t),u)-f(t,x(t),\hat{u}(t))]$.

Lemma 1

For δ small, the first-order change in $x^{u_{\delta }}(T)$ can be calculated by means of the expression

$$\bigl[dx^{u_{\delta }}(T)/d\delta\bigr]_{\delta =0}=\sum _{j}s^{j}C\bigl(T,t^{j}\bigr)\bigl[f \bigl(t^{j},\hat{x}\bigl(t^{j}\bigr),u^{j}\bigr)-f \bigl(t^{j},\hat{x}\bigl(t^{j}\bigr),\hat{u} \bigl(t^{j}\bigr)\bigr)\bigr] $$

so in the free end case, for p(t)=aC(T,t), the criterion changes according to

$$\bigl[dax^{u_{\delta }}(T)/d\delta \bigr]_{\delta =0}=\sum _{j}s^{j}p \bigl(t^{j} \bigr) \bigl[f \bigl(t^{j},\hat{x} \bigl(t^{j} \bigr),u^{j} \bigr)-f \bigl(t^{j},\hat{x}\bigl(t^{j} \bigr), \hat{u} \bigl(t^{j} \bigr) \bigr) \bigr]. $$

The derivatives do exist, as δ≥0 they are actually a right derivative.

Comment on Proof

For the first equality, see the classic work of Pontryagin et al. [12] (or for $\hat{u}(.)$ piecewise continuous, e.g. Lemma 14.1 in [6]). □

Lemma 2

Let $\pi :(x_{1},\ldots,x_{n})\rightarrow (x_{1},\ldots,x_{n^{\ast }})$, n ^∗≤n, and let $B(\bar{p},4\varepsilon )\subset \mathrm{clco}\{\pi q^{u,t}(T):u\in U,\ t\ \mbox{\textit{a Lebesgue point of}}\ d\hat{x}/dt\ \mbox{\textit{in}}\ (0,T)\}$, ε>0. Then for some d′>0, for all d∈(0,d′], $\pi \hat{x}(T)+d\bar{p}=\pi x^{u_{d}}(T)$ for some u _d ∈U′.

Proof

For a finite number of elements $(u_{i},t_{i}),i=1,\ldots,i^{\ast },B(\bar{p},3\varepsilon )\subset \mathrm{co} \{\pi q^{u_{i},t_{i}}(T):i=1,\ldots,i^{\ast }\}$ where we can assume that all t _i are distinct and <T. Then a standard result, contained in many proofs of the maximum principle, says that for some d′>0, for all d∈(0,d′], $d\bar{p}$ equals $\pi (x^{u_{d}}(T)-\hat{x}(T))$ for some control u _d .⁵ □

Comment on the Condition p ⁱ(t)=lim _T→∞ p ⁱ(t,T)

Assume in connection with (26) that f=(f ₁,…,f _n ) does not depend on x ₁, that a=(1,0,…,0) and that $|\partial f_{1}(t,x,\hat{u}(t))/\partial x_{k}|\leq B\exp (-bt)$ for all k≥2 and $|\partial f_{i}(t,x,\hat{u}(t))/ \partial x_{k}|\leq \hat{k}$ for all i≥2, k≥2, where $\hat{k}(n-1)<b$. In the free-end, finite-horizon case it is well known that when $\hat{x}(.),\hat{u}(.)$ is optimal, then p(t)=aC(T,t), so the maximum condition is $0\geq p(t)[f(t,\hat{x}(t),u)-f(t,\hat{x}(t), \hat{u}(t),\hat{x}(t))]=aC(T,t)[f(t,\hat{x}(t),u)-f(t,\hat{x}(t),\hat{u}(t), \hat{x}(t))]$, obtained simply by differentiating the criterion with respect to δ for a perturbation (replacement of $\hat{u}(.)$ by) u on [t,t+δ]. This first-order condition reads $aC(\infty ,t)[f(t,\hat{x} (t),u)-f(t,\hat{x}(t),\hat{u}(t),\hat{x}(t))]\leq 0$ when T=∞, the growth condition taking care of both that aC(∞,t) exists, and that this first condition can be derived. If we for a moment write p(t,T)=aC(T,t) in the finite-horizon case, then letting p(t)=lim _T→∞ p(t,T)=aC(∞,t) we have, in the infinite-horizon case, $p(t)[f(t,\hat{x}(t),u)-f(t,\hat{x}(t),\hat{u}(t),\hat{x}(t))]\leq 0$ for this limit p(t).

Comment on Remark 3

One way to prove Remark 3 is to replace [0,T] by an enlarged interval [0,T+1], letting f ⁱ=S ⁱ(x), g=0 on (T,T+1], having only integrals $\int_{0}^{T+1}f^{i}$ as criteria. With $a^{i}=S_{x}^{i}(\hat{x}(T))$, the maximum condition again holds as before on [0,T], with p(t) on [0,T] defined as before. Carrying out the perturbations $\bar{u}^{i}$ at (near) T as before, the same proof works, as soon as we note that, in Remark 3(a), (b) in Theorem 4 is meant to hold for the present definition of a ⁱ. Note, by the way, that we could have weakened (b) to read: For all i, $f^{i}(T,\hat{x}(T),u)$ and $g_{k}(T,\hat{x}(T),u)$, $k\in I\cup I^{a^{i}}$, are independent of u _j , j≠i.