Continuous Time Learning Algorithms in Optimization and Game Theory

Abstract

The purpose of this work is the comparison of learning algorithms in continuous time used in optimization and game theory. The first three are issued from no-regret dynamics and cover in particular “Replicator dynamics” and “Local projection dynamics”. Then we study “Conditional gradient” versus “Global projection” dynamics and finally “Frank-Wolfe” versus “Best reply” dynamics. Important similarities occur when considering potential or dissipative games.

Appendix

Assume g continuous and dissipative and recall that X is convex and compact. Let us prove that $S_{\mathrm{ext}}$ is non-empty. Define :

$$\begin{aligned} S_{\mathrm{ext}}^y = \{x \in X; \langle g (y) | x - y \rangle \ge 0 \} \end{aligned}$$

so that $S_{\mathrm{ext}} = \cap _{y \in X} S_{\mathrm{ext}}^y$. Hence by compactness (weak-compactness in an Hilbert framework) it is enough to establish the following:

Claim

For any finite collection $y_i \in X, i \in I$, there exists $x \in co \{ y_i, i \in I\}$ such that:

$$\begin{aligned} \langle g (y_i) | x - y_i \rangle \ge 0, \qquad \forall i \in I. \end{aligned}$$

(47)

Consider the finite two-person zero-sum game defined by the following $I \times I$ matrix A:

$$\begin{aligned} A_{ij}= \langle g (y_j) | y_i - y_j \rangle . \end{aligned}$$

Introduce $B = \frac{1}{2}[ A +\, ^tA]$ and $C = \frac{1}{2}[ A -\, ^tA]$.

The crucial point is that B has non negative coefficients since:

$$\begin{aligned} B_{ij}= \langle g (y_j) | y_i - y_j \rangle + \langle g (y_i) | y_j - y_i \rangle = \langle g (y_j) - g(y_i) | y_i - y_j \rangle \ge 0. \end{aligned}$$

Hence an optimal strategy $u \in \Delta (I)$ in the game C (which has value 0) gives $uA_j \ge 0, \forall j \in I$. Letting $x = \sum _i u_i y_i$ this writes as (47).