Abstract
Recursive game theory provides theoretic procedures for computing the equilibrium payoff or value sets of repeated games and the equilibrium payoff or value correspondences of dynamic games. In this paper, we propose and implement outer and inner approximation methods for equilibrium value correspondences that naturally occur in the analysis of dynamic games. The procedure utilizes set-valued step functions. We provide an application to a bilateral insurance game with storage.
Similar content being viewed by others
Notes
The usefulness of inner and outer approximations in providing error bounds was first emphasized by Judd et al. [12] in the context of repeated games.
Thus, we allow for an uncountable number of endogenous states. The restriction to a finite set \(S\) avoids measure-theoretic complications.
For a correspondence \(C:S\times K\rightrightarrows X\), Graph \(C(s)=\{(k,x):x\in C(s,k)\}\) and Graph \(C=\{(s,k,x):x\in C(s,k)\}\).
However, each Graph \(V^{L*}(s)\) is not generally convex.
Geometrically, for \(k\in K_q\), \(\omega ^{\mathbb O}_{s,q}(k)\) equals the intersection of a finite number of half-spaces containing and with hyperplanes tangent to co \(\cup _{k^{\prime }\in K_{q}}V(s,k^{\prime })\).
The stronger assumption of a non-empty interior is needed in this case.
The surface of the step correspondence contains discontinuities at the joins of the partition elements. Application of a nonlinear optimizer requires either that the correspondence is smoothed across these joins, or that the optimizations are split into a collection of sub-optimizations in each of which the continuation state variable is restricted to an element of the partition.
References
Abreu D, Pearce D, Stacchetti E (1986) Optimal cartel equilibria with imperfect monitoring. J Econ Theory 39(1): 251–269
Abreu D, Pearce D, Stacchetti E (1990) Toward a theory of discounted repeated games with imperfect monitoring. Econometrica 58(5):1041–1063
Atkeson A (1991) International lending with moral hazard and risk of repudiation. Econometrica 59:1069–1089
Balbus L, Reffett K, Wozny L (2014) A constructive study of markov equilibria in stochastic games with strategic complementarities. J Econ Theory 150:815–840
Baldauf M, Judd K, Mollner J, Yeltekin Ş (2013) Supergames with states. Stanford University, Mimeo
Beer G (1980) The approximation of upper semicontinuous multifunctions by step multifunctions. Pac J Math 87:11–19
Chang R (1998) Credible monetary policy in an infinite horizon model: recursive approaches. J Econ Theory 81:431–461
Cronshaw M, Luenberger D (1994) Strongly symmetric subgame perfect equilibrium in infinitely repeated games with perfect monitoring and discounting. Games Econ Behav 6:220–237
Feng Z, Miao J, Peralta-Alva A, Santos M (2014) Numerical simulation of nonoptimal dynamic equilibrium models. Int Econ Rev 55(1):83–110
Fernandez-Villaverde J, Tsyvinski A (2002) Optimal fiscal policy in a business cycle model without commitment. University of Pennsylvania, Mimeo
Herings PJ-J, Peeters R (2004) Stationary equilibria in stochastic games: structure, selection and computation. J Econ Theory 118:32–60
Judd K, Yeltekin Ş, Conklin J (2003) Computing supergame equilibria. Econometrica 71:1239–1255
Kocherlakota N (1996) Efficient bilateral risk sharing without commitment. Rev Econ Stud 63:595–609
Phelan C, Stacchetti E (2001) Subgame perfect equilibria in a ramsey taxes model. Econometrica 69:1491–1518
Sleet C (1996) New recursive methods for government policy problems. Stanford University, Mimeo
Yeltekin Ş, Judd K, Cai Y (2014) Computing equilibria of dynamic games. Working Paper. Carnegie Mellon University
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
The following algorithms provide the details of the computational procedure for the outer and inner monotone approximations of \(B(V)(s,k)\) using hyperplanes. The correspondence \(V(s,k)\) is a candidate for the equilibrium value correspondence.
______________________________________
Outer Approximation Algorithm
-
1.
Fixed Inputs:
-
(a)
Exogenous parameters.
-
(b)
Partition \(\mathcal {K} = \{K_q\}\).
-
(c)
Search directions: \({\mathcal {R}}=\{r_1,\ldots , r_{\overline{\ell }_r} \}\), with each \(\{r_{\ell }\in {\mathbb {R}}^I \ : \ ||r_{\ell }|| =1\}\).
-
(d)
Convergence criterion \(\varepsilon >0\).
-
(a)
-
2.
First Iteration Inputs:
-
(a)
Set of boundary points \(\{v_{\ell }(s,q)|\ell = 1,\ldots , \overline{\ell }\}\) for each \((s,q)\).
-
(b)
Set of corresponding normals: \({\mathcal {Z}}=\{z_{1},\ldots ,{z}_{\overline{\ell }}\}\). Together (a) and (b) define the initial outer approximation \(V^{\mathbb {O}}(s,k)=\cup _{q} \omega ^{\mathbb {O}}_{s,q}(k)\) where
$$\begin{aligned} \omega ^{\mathbb {O}}_{s,q}(k) = {\left\{ \begin{array}{ll} \cap _{\ell =1}^{\overline{\ell }}\{v\;|\;z_{\ell }\cdot v\le z_{\ell }\cdot v_\ell (s,q)=c_{\ell }(s,q)\}, &{} \text {if}\,k \in K_{q}, \\ \emptyset &{} \text {otherwise}. \end{array}\right. } \end{aligned}$$for all \(v\in \) co \( \cup _{k^{\prime }\in K_{q}}V(s,k^{\prime })\).
-
(a)
-
3.
Optimization: Compute the new subgradient and boundary point sets that together represent an outer approximation to \(B^L({V})\). For each \(s \in S\), each partition element \(K_{q}\) and each \({r_{\ell }}\in {{\mathcal {R}}}\)
$$\begin{aligned} c_{\ell }^{+}(s,q)&= {\max _{a,v^{\prime },k \in K_{q}}}\ r_{\ell }\cdot [(1-\beta )u(s,k,a)+ \beta \sum _{S} v^{\prime }(s^{\prime })\Pi (s^{\prime }|s)] \;\; \text {s.t.} \\&(i)\;v^{\prime }(s^{\prime })\in V^{\mathbb {O}}(s^{\prime },f(s,k,a)), \ a \in A(s,k), \nonumber \\&(ii)\;\text {for each } i \in \{1,\ldots ,I\}, \nonumber \\&\;\;\;\;\ (1-\beta )u^{i}(s,k,a)+\beta \sum _{S} v^{i\prime }(s^{\prime })\Pi (s^{\prime }|s)\ge \nonumber \\&\;\;\;\;\; \;\;\;\;\;\;\;\;\; \max _{a_{-i}\in A^{i}(s,k)} \min _{\tilde{v}^i(s^{\prime })\in V^{\mathbb {O}}(s^{\prime },f(s,k,a_{-i},a^{\prime }))} \ \ (1-\beta )u^{i}(s,k,a_{-i},a^{\prime })\nonumber \\ {}&+\,\, \beta \sum _{S}\tilde{v}^{i}(s^{\prime })\Pi (s^{\prime }|s). \nonumber \end{aligned}$$(6) -
4.
Output: \(V^{+}(k,s)={\hat{B}}(V)(s,k)=\cup _{q}\omega _{s,q}^{{\mathbb {O}}^{+}}(k)\), where
$$\begin{aligned} \omega _{s,q}^{{\mathbb {O}}^{+}}(k) = {\left\{ \begin{array}{ll} \cap _{\ell =1}^{\overline{\ell }_{r}}\{v\;|\;r_{\ell }\cdot v\le c_{\ell }^{+}(s,q)\} &{} \text {if}\,k \in K_{q}, \\ \emptyset &{} \text {otherwise.} \end{array}\right. } \end{aligned}$$ -
5.
Check for convergence: Stop if \(d_{V} ({V}^{+}, {V}) < \varepsilon \). Else set \({V}^{\mathbb {O}}={V}^{+}\), \({\mathcal {Z}}=R\), and \(\overline{\ell } = \overline{\ell }_r\) and return to Step 3.
______________________________________
The main computational work is done in the optimization step. In this step, for each search direction \(r_{\ell }\) and element of the partition \(\mathcal {K}\), we find an action profile \(a\), continuation values \(v^{\prime }(s^{\prime }) \in V(s^{\prime },f(s,k,a))\), and current state \(k\) in \(K_{q}\) which makes \(a\) incentive compatible and maximizes a weighted sum of player payoffs where the weights are given by \(r_{\ell }\).Footnote 9
In the outer approximation case, the set of search directions, \({{\mathcal {R}}}\), and the set of hyperplane normals for the approximation of \(V(s,k)\), \({\mathcal {Z}}\), coincide after the initial iteration. Correspondences are represented by sets of bounding hyperplanes, one set for each block in the partition and each state \(s\). In this way, an approximated correspondence can be reduced to a collection \( \{c_{\ell }(s,q)\}\) in \({\mathbb {R}}^{SQI\overline{\ell }}\), with each \(c_{\ell }(s,q)\) determining the position of the \(\ell \)-th hyperplane on the \(q\)-th block in state \(s\).
For inner approximations, the set of hyperplane normals vary from iteration to iteration, depending on the number and position of the points on the convex hull of \(V(s,k)\). The bounding hyperplanes are constructed from the line segments that connect the extreme points belonging to the convex hull of \(V(s,k)\). In this case, an approximated correspondence cannot be reduced to a collection of \(\{c_{\ell }(s,q)\}\) in \({\mathbb {R}}^{SQI\overline{\ell }}\) but can be reduced to a collection of \(\{v_{\ell }(s,q)\}\) in \(\cap _{k^{\prime }\in K_{q}}V(s,k^{\prime })\) for each \((s,q)\). This requires a slight modification to the output and updating used in the outer approximation algorithm. Below is the step by step description of the algorithm for computing an inner approximation to \(B(V)\).
______________________________________
Inner Approximation Algorithm
-
1.
Fixed Inputs:
-
(a)
Exogenous parameters
-
(b)
Partition \({\mathcal {K}} = \{K_q\}\).
-
(c)
Search directions: \({{\mathcal {R}}}=\{r_1,\ldots , r_{\overline{\ell }_r} \}\), with each \(\{r_{\ell } \in {\mathbb {R}}^I \ : \ ||r_{\ell }|| =1\}\).
-
(d)
Convergence criterion: \(\varepsilon >0\).
-
(a)
-
2.
First Iteration Inputs:
-
(a)
Set of points \(\{v_{\ell }(s,q)|\ell = 1,\ldots , \overline{\ell }\}\) in \( \cap _{k^{\prime }\in K_{q}}V(s,k^{\prime })\), for each \((s,q)\). These points define the initial inner approximation \(V^{\mathbb {I}}(s,k)=\cap _{q} \omega ^{\mathbb {I}}_{s,q}(k)\) where
$$\begin{aligned} \omega ^{\mathbb {I}}_{s,q}(k) = {\left\{ \begin{array}{ll} \text {co} \ \{v_{\ell }(s,q)|\ell = 1,\ldots , \overline{\ell }\} &{} \text {if}\,k \in K_{q}, \\ {\mathbb {R}}^{I} &{} \text {otherwise}. \end{array}\right. } \end{aligned}$$
-
(a)
-
3.
Optimization: Compute the new sets of point from which the convex hulls that together represent an inner approximation to \(B^L({V})\) are constructed. For each \(s \in S\), each partition element \(K_{q}\) and each \({r_{\ell }}\in {{\mathcal {R}}}\)
$$\begin{aligned} \{a,v^{\prime },k\}_{\ell }^{+}(s,q)&= \mathop {{{\mathrm{argmax}}}}\limits _{a,v^{\prime },k \in K_{q}}\;\;\; r_{\ell }\cdot [(1-\beta )u(s,k,a)+ \beta \sum _{S} v^{\prime }(s^{\prime })\Pi (s^{\prime }|s)] \ \text {s.t} \\&(i)\;v^{\prime }(s^{\prime })\in V^{\mathbb {I}}(s^{\prime },f(s,k,a)), \ a \in A(s,k), \nonumber \\&(ii)\;\text {for each } i \in \{1,\ldots ,I\}, \nonumber \\&\;\;\;\;\ (1-\beta )u^{i}(s,k,a)+\beta \sum _{S} v^{i\prime }(s^{\prime })\Pi (s^{\prime }|s)\ge \nonumber \\&\;\;\;\;\; \ \max _{a_{-i}\in A^{i}(s,k)} \min _{\tilde{v}^i(s^{\prime })\in V^{\mathbb {I}}(s^{\prime },f(s,k,a_{-i},a^{\prime }))} \ (1-\beta )u^{i}(s,k,a_{-i},a^{\prime })\nonumber \\&\quad +\, \beta \sum _{S}\tilde{v}^{i}(s^{\prime })\Pi (s^{\prime }|s). \nonumber \end{aligned}$$(7) -
4.
Output: \(V^{+}(k,s)={\hat{B}}(V)(s,k)=\cap _{q}\omega _{s,q}^{{\mathbb {I}}^{+}}(k)\), with
$$\begin{aligned} \omega _{s,q}^{{\mathbb {I}}^{+}}(k) = {\left\{ \begin{array}{ll} \text {co\ } \{v^{+}_{\ell }(s,q)|\ell = 1,\ldots , \overline{\ell }\} &{} \text {if}\,k \in K_{q}, \\ {\mathbb {R}}^{I} &{} \text {otherwise.} \end{array}\right. } \end{aligned}$$where \(v^{+}_{\ell }(s,q)\) is equal to \([(1-\beta )u(s,k,a)+ \beta \sum _{S} v^{\prime }(s^{\prime })\Pi (s^{\prime }|s)]\) evaluated at \(\{a,v^{\prime },k\}_{\ell }^{+}(s,q)\).
-
5.
Check for convergence: Stop if \(d_V ({V}^{+}, {V}) < \varepsilon \). Else set \({V}^{\mathbb {I}}(s,k)={V}^{+}(s,k)\) and go back to Step 3.
See Judd et al. [12] and Yeltekin et al. [16] for the details of constructing convex hulls similar to those required in Step 4.
Rights and permissions
About this article
Cite this article
Sleet, C., Yeltekin, Ş. On the Computation of Value Correspondences for Dynamic Games. Dyn Games Appl 6, 174–186 (2016). https://doi.org/10.1007/s13235-015-0139-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13235-015-0139-1