Solution for a class of closed-loop leader-follower games with convexity conditions on the payoffs

Abstract

In the present paper, a recent deterministic continuum-strategy two-player discrete-time dynamic leader-follower game with fixed finite time duration and closed-loop information structure is studied. The types of the considered payoff functions can be widely used in different applications (mainly in conflicts of consuming a limited resource, where one player, called the leader, is a superior authority choosing a strategy choice first, and another player, called the follower, chooses after). In case of certain payoff convexity, explicit conditions are given, when it can be known in advance that an equilibrium exists and consists of only two possible choices of both players at each step. The sub-game equilibrium from a given step may depend on the former selections of the players. Thus the continuum-strategy problem has been reduced to a general finite game of two possible choices corresponding to both players. Such type of games could be solved in a standard way with dynamic programming using a computer. Nevertheless, the game can be further simplified, and then an equilibrium can be directly determined, such decreasing the computational demand to a great extent. A solution algorithm and practical examples are also given to support the real-life application of the results.

Appendix

Lemma 1

Suppose that

$$\begin{aligned} {\pi }''\left( \rho \right) \le \frac{2}{R}\left| {{\pi }'\left( \rho \right) } \right| , \quad (\rho \in ]0,R[) \end{aligned}$$

(7)

where $\pi$ is the price function introduced in the game of Sect. 3.

Then $\frac{\partial ^{2}f_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) ^{2}}\ge 0$ and $\frac{\partial ^{2}g_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) ^{2}}\ge 0$ for any $x_1 ,y_1 ,...,x_t ,y_t \in \left[ {0,1} \right]$ ($t=2,...,T$,$s=1,...,t-1)$.

Proof

Let $t=2,...,T$, $s=1,...,t-1$, then for the game of Sect. 3 (for any $x_1 ,y_1 ,...,x_t ,y_t \in \left[ {0,1} \right] )$, it holds that:

$$\begin{aligned}&\frac{\partial f_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) }\nonumber \\&\quad =x_t \left[ {-{\pi }'\left[ {R-\sum _{i=1}^{t-1} {\left( {1-\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } \right) } r_i } \right] } \right. \nonumber \\&\qquad \quad \times \frac{\partial \sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } r_i } }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) }\cdot \left( {\sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } r_i +r_t } \right) \nonumber \\&\qquad +\left( {\alpha -\pi \left[ {R-\sum _{i=1}^{t-1} {\left( {1-\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } \right) } r_i } \right] } \right) \nonumber \\&\qquad \quad \cdot \left. {\frac{\partial \sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } r_i } }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) }} \right] \nonumber \\&\qquad +\,qy_t \left( {1-x_t } \right) \left[ {-{\pi }'\left[ {R-\sum _{i=1}^{t-1} {\left( {1-\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } \right) } r_i } \right] } \right. \nonumber \\&\qquad \quad \cdot \frac{\partial \sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } r_i } }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) }\cdot \left( {\sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } r_i +r_t } \right) \nonumber \\&\qquad +\left( {\gamma _t -\pi \left[ {R-\sum _{i-1}^{t-1} {\left( {1-} \right. \prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) } } } \right. } \right. \nonumber \\&\qquad \quad \left. {\left. {\left. {\left. {\cdot \left( {1-x_{t-k} } \right) } \right) r_i } \right] } \right) \frac{\partial \sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } r_i } }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) }} \right] \end{aligned}$$

(8)

(It should be noted that $\frac{\partial ^{2}\sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } r_i } }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) ^{2}}=0.)$

$$\begin{aligned}&\frac{\partial ^{2}f_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) ^{2}}=x_t \left[ {-{\pi }''\left[ {R-\sum _{i=1}^{t-1} {\left( {1-\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } \right) } r_i } \right] } \right. \nonumber \\&\qquad \cdot \left( {\frac{\partial \sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } r_i } }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) }} \right) ^{2}\left( {\sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } r_i +r_t } \right) \nonumber \\&\quad -2{\pi }'\left[ {R-\sum _{i=1}^{t-1} {\left( {1-\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) } } \right. } } \right. \nonumber \\&\qquad \left. {\left. {\left. {\cdot \left( {1-x_{t-k} } \right) } \right) r_i } \right] \left( {\frac{\partial \sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } r_i } }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) }} \right) ^{2}} \right] \nonumber \\&\quad +qy_t \left( {1-x_t } \right) \left[ {-{\pi }''} \right. \left[ {R-\sum _{i=1}^{t-1} {\left( {1-\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } \right) } \cdot } \right. \left. {r_i } \right] \nonumber \\&\qquad \left( {\frac{\partial \sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } r_i } }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) }} \right) ^{2}\nonumber \\&\qquad \left( {\sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } r_i +r_t } \right) -2{\pi }'\left[ {R-\sum _{i=1}^{t-1} {\left( {1-\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) } } \right. } } \right. \nonumber \\&\qquad \left. {\left. {\left. {\cdot \left( {1-x_{t-k} } \right) } \right) r_i } \right] \left( {\frac{\partial \sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } r_i } }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) }} \right) ^{2}} \right] , \end{aligned}$$

(9)

so $\frac{\partial ^{2}f_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) ^{2}}\ge 0$ if

$$\begin{aligned}&-{\pi }''\left[ {R-\sum _{i=1}^{t-1} {\left( {1-\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } \right) } r_i } \right] \left( {\sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } r_i +r_t } \right) \nonumber \\&\quad -2{\pi }'\left[ {R-\sum _{i=1}^{t-1} {\left( {1-} \right. } } \right. -\left. {\left. {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } \right) r_i } \right] \ge 0, \end{aligned}$$

(10)

that is if

$$\begin{aligned}&{\pi }''\left[ {R-\sum _{i=1}^{t-1} {\left( {1-\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } \right) } r_i } \right] \nonumber \\&\quad \le \frac{-2{\pi }'\left[ {R-\sum _{i=1}^{t-1} {\left( {1-\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } \right) } r_i } \right] }{r_t +\sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } r_i }, \end{aligned}$$

(11)

assuming $r_t +\sum _{i=1}^{t-1} {\prod _{k=1}^{t-i} {\left( {1-y_{t-k} } \right) \left( {1-x_{t-k} } \right) } } r_i \ne 0$, otherwise, (A4) already holds, because of ${\pi }'\left( \rho \right) \le 0$ ($\rho \in ] {0,R} ])$. The numerator is the absolute value of itself. The positive denominator can be estimated with R from above. It follows that the following condition is sufficient for the second derivative to be nonnegative: ${\pi }''\left( \rho \right) \le \frac{2}{R}\left| {{\pi }'\left( \rho \right) } \right|$ ($\rho \in ] {0,R} ])$. $\frac{\partial ^{2}g_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) ^{2}}\ge 0$ (for any $x_1 ,y_1 ,...,x_t ,y_t \in \left[ {0,1} \right] )$ ($t=2,...,T$, $s=1,...,t-1)$ can be similarly justified. Thus Lemma 1 has been proved. $\square$

Remark 5.1

(Kicsiny et al. 2014b) Consider the following type of price functions in the game of Sect. 3:

$$\begin{aligned} \pi \left( \rho \right) =\frac{c}{(\rho +\rho _0 )^{n}} (\rho \in \left[ {0,R} \right] ), \end{aligned}$$

where c, $\rho _0$ and n are positive constants. The graph of this function can be seen in Fig. 8.

Fig. 8

Convex price function type

Then (A1) is satisfied for any $\rho \in \left[ {0,R} \right]$ if

$$\begin{aligned} \rho _0 \ge \frac{\left( {n+1} \right) R}{2}. \end{aligned}$$

(12)

Lemma 2

Let $\frac{\partial ^{2}f_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) ^{2}}\ge 0$ for any $x_1 ,y_1 ,...,x_t ,y_t \in \left[ {0,1} \right]$ ($t=2,...,T$, $s=1,...,t-1)$ for function (3) in the game of Sect. 3.

Then $\frac{\partial ^{2}f_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial x_{t-s}^2 }\ge 0$ for any $x_1 ,y_1 ,...,x_t ,y_t \in \left[ {0,1} \right]$ ($t=2,...,T$, $s=1,...,t-1)$.

Proof

$$\begin{aligned} \begin{array}{l} \frac{\partial f_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial x_{t-s} }=\frac{\partial f_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) }\frac{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) }{\partial x_{t-s} }=\frac{\partial f_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) }\cdot \\ \left( {-\left( {1-y_{t-s} } \right) } \right) , \\ \end{array} \end{aligned}$$

(13)

from which

$$\begin{aligned} \begin{array}{l} \frac{\partial ^{2}f_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial x_{t-s}^2 }=\left( {-\left( {1-y_{t-s} } \right) } \right) \frac{\partial ^{2}f_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) ^{2}}\left( {-\left( {1-y_{t-s} } \right) } \right) =\left( {1-y_{t-s} } \right) ^{2}\cdot \\ \frac{\partial ^{2}f_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) ^{2}} \\ \end{array} \end{aligned}$$

(14)

for any $x_1 ,y_1 ,...,x_t ,y_t \in \left[ {0,1} \right]$, so Lemma 2 has been proved, since $\left( {1-y_{t-s} } \right) ^{2}\ge 0$. $\square$

Remark 5.2

$\frac{\partial ^{2}f_\tau \left( {x_1 ,...,x_\tau ,y_1 ,...,y_\tau } \right) }{\partial x_\tau ^2 }=0$ for any $x_1 ,y_1 ,...,x_\tau ,y_\tau \in \left[ {0,1} \right]$ ($\tau =1,...,T)$, since $f_\tau$ is linear in $x_\tau$, so $\frac{\partial ^{2}f_\tau \left( {x_1 ,...,x_\tau ,y_1 ,...,y_\tau } \right) }{\partial x_\upsilon ^2 }\ge 0$ for any $x_1 ,y_1 ,...,x_\tau ,y_\tau \in \left[ {0,1} \right]$ ($\tau =1,...,T$; $\upsilon =1,...,\tau )$ if the condition of Lemma 2 is satisfied.

Corollary 1

From Lemma 2 and Remark 5.2, it follows that $f_{\omega +1} +...+f_T$ is convex with respect to $x_\omega$ ($\omega =1,...,T-1)$ (for any $x_1 ,y_1 ,...,x_T ,y_T \in \left[ {0,1} \right] )$ if the condition of Lemma 2 is satisfied.

Remark 5.3

It can be similarly shown that $g_{\omega +1} +...+g_T$ is convex with respect to $x_\omega$ and $y_\omega$ ($\omega =1,...,T-1)$ (for any $x_1 ,y_1 ,...,x_T ,y_T \in \left[ {0,1} \right] )$ if $\frac{\partial ^{2}g_t \left( {x_1 ,...,x_t ,y_1 ,...,y_t } \right) }{\partial \left( {\left( {1-y_{t-s} } \right) \left( {1-x_{t-s} } \right) } \right) ^{2}}\ge 0$ ($t=2,...,T$, $s=1,...,t-1)$ for function (4) in the game of Sect. 3.

Lemma 3

Consider the game defined in Sect. 3. It can be stated that F gains not less final payoff with delaying any consumption from step v ($v=1,...,T-1)$ to step w ($w=2,...,T$, v < w) if it may occur for v that $\alpha -\pi \left[ \cdot \right] \ge q\left( {\gamma _v -\pi \left[ \cdot \right] } \right)$ (that is if $\alpha -\pi \left[ R \right] \ge q\left( {\gamma _v -\pi \left[ R \right] } \right) )$ and it holds for w that $\alpha -\pi \left[ R \right] \le q\left( {\gamma _w -\pi \left[ R \right] } \right)$ (supposing that L does not consume at steps from (v+1) to w).

Proof

It is surely more advantageous for F to consume the resource to be consumed at step v (as well) later at step w because of the following: Suppose indirectly that F would get higher final payoff if he consumed the amount $r_v$ at step v and the amount $r_w$ at step w than in case of consuming all $r_v +r_w$ at step w, that is: $q\left( {\gamma _v -\pi \left[ r \right] } \right) r_v +q\left( {\gamma _w -\pi \left[ {r-r_v } \right] } \right) r_w >q\left( {\gamma _w -\pi \left[ r \right] } \right) \left( {r_v +r_w } \right)$. (r is the resource remaining from R at step v.)

It follows that $q\left( {\gamma _v -\pi \left[ r \right] } \right) r_v >q\left( {\gamma _w -\pi \left[ r \right] } \right) r_v$. This is a contradiction, since $q\left( {\gamma _v -\pi \left[ R \right] } \right) \le \alpha -\pi \left[ R \right] \le q\left( {\gamma _w -\pi \left[ R \right] } \right) \Rightarrow q\left( {\gamma _v -\pi \left[ r \right] } \right) \le q\left( {\gamma _w -\pi \left[ r \right] } \right)$, which follows from the fact that function $q\left( {\gamma _v -\pi \left[ \cdot \right] } \right)$ and $q\left( {\gamma _w -\pi \left[ \cdot \right] } \right)$ differ only by a constant, so the relative position of their values is fixed. Thus Lemma 3 has been proved. $\square$

Lemma 4

Let us consider the game defined in Sect. 3. It can be stated that L gains higher final payoff with delaying any consumption for as long as possible (if F does not consume meanwhile).

Proof

Keeping the notation of Lemma 3, it holds that

$$\begin{aligned} \left( {\alpha -\pi \left[ r \right] } \right) r_v +\left( {\alpha -\pi \left[ {r-r_v } \right] } \right) r_w \le \left( {\alpha -\pi \left[ r \right] } \right) r_v +\left( {\alpha -\pi \left[ r \right] } \right) r_w =\left( {\alpha -\pi \left[ r \right] } \right) \left( {r_v +r_w } \right) ,\nonumber \\ \end{aligned}$$

(15)

from which Lemma 4 follows. $\square$

Lemma 5

(Pardalos et al 1995) Consider the deterministic two-player discrete-time leader-follower game in Definition 1 with the following: the game has only one step, $X_1 \subset \mathbf{R}^{m}$ and $Y_1 \subset \mathbf{R}^{n}$ are compact sets ($m,n\in \mathbf{N})$ and $F,G\in C\left[ {X_1 \times Y_1 } \right]$. Then there is a leader-follower equilibrium of the game. $\square$

Lemma 6

(Kuhn and Tucker 1953) Consider the deterministic two-player discrete-time leader-follower game in Definition 1 with finite strategy sets. Then there is a leader-follower equilibrium of the game.