Abstract
An important aspect of network dynamics that has been missing from our understanding of network dynamics in various applied settings is the influence of strategic behavior in determining equilibrium network dynamics. Our main objective hear to say what we can regarding the emergence of stable club networks—and therefore, stable coalition structures—based on the stability properties of strategically determined equilibrium network formation dynamics. Because club networks are layered networks, our work here can be thought of as a first work on the strategic dynamics of layered networks. In addition to constructing a discounted stochastic game model (i.e., a DSG model) of club network formation, (1) we show that our DSG of network formation possesses a stationary Markov perfect equilibrium in players’ membership-action strategies; (2) we identify the assumptions on primitives which ensure that the induced equilibrium Markov process of layered club network formation satisfies the Tweedie Stability Conditions (Tweedie in Stoch Process Appl 92:345–354, 2001) and (3) we show that, as a consequence, the equilibrium Markov network formation process generates a unique decomposition of the set of state-network pairs into a transient set together with finitely many basins of attraction. Moreover, we show that if there is a basin containing a vio set (a visited infinitely often set) of club networks sufficiently close together, then the coalition structures across club networks in the vio set will be the same (i.e., closeness across networks in a vio set leads to invariance in coalition structure across networks in a vio set).
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
A coalition is a group of players who, through their own actions, can realize some set of outcomes for its own members [36]. Here, we will be interested in the equilibrium dynamics governing the formation and evolution of coalitions as well as the strategic forces which give rise to these dynamics. We will think of a coalition as a group of players belonging to the same club, and we will represent the prevailing club membership structure as a labeled, directed bipartite network. Because we will allow each player to be a member of multiple clubs, each player can be a member of multiple coalitions (see [32]).Footnote 1 Each club network consists of three primitives: a set of players, a set of clubs, and a set of arc labels. In our network model, a player’s club membership is represented by a labeled directed arc from the node representing the player to the node representing the club. The arc label, which must be feasible for that player in that club, indicates the action chosen by the player to be taken in the chosen club. Thus, a player establishes a directed connection by choosing a club and a feasible club action. The set of all such player-specific directed club connections is the player’s club network and together the union of these player club networks constitute the club network. At each of infinitely many time points, players, in light of the prevailing state and club network, are free to noncooperatively alter their club memberships as well as their corresponding club action profiles in accordance with the rules of network formation. We will assume that after players have altered their own club networks, each player receives a stage payoff, a function of the prevailing state-network pair, then given the prevailing state and the new club network chosen by the players, a new state is generated in accordance with the law of motion. We will assume that players, in making their membership-action choices through discrete time, seek to maximize the discounted sum of their expected future payoffs. In particular, we will assume that players in forming their club networks, play a discounted stochastic game of club network formation in which they seek to choose stationary Markov perfect membership-action strategies that maximize the discounted sum of their expected payoffs. Taken together, the players’ noncooperative network formation strategies (i.e., membership and club specific action strategies) determine a network formation process. We say that this process is an equilibrium network formation process if in the underlying discounted stochastic game (DSG) of network formation players noncooperatively choose stationary Markov perfect membership and club action strategies forming a Nash equilibrium.
Our main contributions, in addition to constructing a DSG model of layered club network formation, are (1) to show that our DSG model of network formation is approximable, and that as a consequence, it possesses a stationary Markov perfect equilibrium in players’ membership-action strategies; (2) to identify the assumptions on primitives which ensure that the induced equilibrium Markov state and club network formation process satisfies the Tweedie Conditions [35] and (3) to show that as a consequence of satisfying the Tweedie Conditions, the equilibrium Markov perfect network formation process generates a unique decomposition of the set of state-network pairs into a transient set together with finitely many basins of attraction. We then show that each basin, upon which resides a unique ergodic probability measure, has the property that if the Markov state-network process enters the basin, then the process will remain there for all future periods, visiting some unique, further subset of state-network pairs infinitely often. From a macroscopic perspective, it is these basin-specific club networks, visited infinitely often by the process (i.e., vio sets) that form the set of viable candidates for stable equilibrium club network. We show that if there is a basin having a vio set of states generating club networks each of whose induced coalition structure is invariant across the networks generated by the vio states, then this vio set of states generates club networks having a stable coalition structure.Footnote 2 This is what we mean by coalitional stability: persistent club network structures in which the underlying coalition structure is invariant across club networks generated by the set of vio states. We show that whether or not the vio states contained in a basin of attraction generate an invariant coalition structure depends on the distance between the club networks generated by the vio states (i.e., it depends on the distance between the vio networks). In particular, we show that closeness (across networks) leads to invariance in coalition structure across club networks. Thus, if each of finitely many basins has network vio sets containing club networks that are sufficiently close together (i.e., if networks are densely packed), then the equilibrium Markov process of club network formation will, in finite time with probability one, generate a stable coalition structure. While the coalition structures generated by the vio states in each basin are the same across the basin’s densely packed club networks, these signature coalition structures can differ across basins. In the long run, whether or not the equilibrium Markov process of network formation generates a stable coalition structure depends on whether or not each basin’s network vio set is densely packed. If all basin-specific network vio sets are sufficiently dense, then with probability 1 in finite time, the equilibrium network formation process will arrive at a basin specific stable coalition structure. However, if some basins have network vio sets in which the club networks are not sufficiently close, while other basins possess densely packed network vio sets, then in equilibrium, there is a positive probability that a stable coalition structure will never be reached. What we conclude here is that if a basin’s vio set of networks is chainable, then a stable coalition structure will emerge.Footnote 3 Thus, if club network vio sets are chainable, then in finite time with probability 1, there will emerge from the equilibrium process of club network formation, a basin specific stable coalition structure.
As a running example, we consider a DSG over time allocation networks (a class of club networks sufficiently simple to allow us to illustrate the ideas and results we obtain for our general DSG of network formation and equilibrium network dynamics). We show that because players’ payoff functions are naturally affine over the convex, compact feasible set of time allocation networks, players’ stationary Markov perfect equilibrium network formation strategies are bang-bang. Thus, rather than diversifying their club time across several clubs, each player in each state spends all club time in one and only one club. Moreover, we show that all that is required for the equilibrium time allocation process induced by players’ bang-bang time allocation strategies to satisfy the Tweedie conditions is for the state space to be compact and the conditional densities over coming states to be continuous in the current state and time allocation network for almost all coming states (i.e., except potential coming states having probability zero of occurring).
2 Layered Club Networks
We begin with a formal definition of club networks and a discussion of their properties. The discussion here is based, in part, on prior work by the second author with M. Wooders (see [29,30,31,32, 36]).Footnote 4Multiple membership club networks, as defined in [32], are examples of layered networks in which connections between layers is brought about by overlapping club memberships. In a club network where each player is the member of one and only one club, the induced club membership structure partitions the set of active players, making each club layer isolated—having no connections, via overlapping memberships, to other layers in the network.
In the club network model, we construct here the feasible action sets available to the players who are active in a particular club layer are subsets of a compact metric space of actions specific to that club layer—and these club specific action spaces can differ across layers. In [32], the underlying metric space of player actions—whose subsets form the various player feasible sets—is the same across club layers. Here, the heterogeneity of player action sets across club layers makes defining a metric to measure the distance between club networks a much more delicate task—but we do accomplish this, thereby providing us with a compact metric hyperspace of club networks in which to carry out our game theoretic analysis of the emergence of equilibrium layered club network dynamics.
Finally, here unlike in [29,30,31,32], the game of network formation is dynamic with the equilibrium network dynamics being determined by the law of motion and the stationary Markov perfect equilibrium in behavioral network formation strategies which emerge from the game of network formation.
We begin by defining connections, layers and networks.
2.1 Connections
[N-1] Assume the following:
-
(1)
D is a finite set of n players equipped with the discrete metric \(\eta _{D}\), having typical element d.Footnote 5
-
(2)
C is a finite set of m clubs equipped with discrete metric \(\eta _{C}\) having typical element c.
-
(3)
\(A_{c}\) is a compact metric space of arc types, equipped with metric \(\rho _{c}\), having typical element a.
-
(4)
\(2^{A_{c}}\) is the collection of all closed subsets of \( A_{c}\) (including the empty set).
-
(5)
\((d,c)\longrightarrow A(d,c)\in 2^{A_{c}}\) is the feasible arc correspondence, with \(A(d,c)\subset A_{c}\) for all c , with c-domain given by
$$\begin{aligned} \mathcal {D}(A(\cdot ,c)):=\{d\in D:A(d,c)\text { }\ne \varnothing \}. \end{aligned}$$specifying for each club, the set of players who can join club c by indicating a nonempty feasible set of arcs club c will allow player d to use in connecting to club c.
A connection is an ordered pair, (a, (d, c)), consisting of an arc label, a, indicating player \(d^{\prime }s\) action (arc) choice and a player-club pair, (d, c), indicating player \(d^{\prime }s\) club membership choice c. The action choice made by a particular player d must be feasible for the club player d has chosen, a requirement satisfied provided \(a\in A(d,c)\subset A_{c}\).
We will refer to a player-club pair, (d, c), as a preconnection. A preconnection, (d, c), acquires the status of a connection only when player d has chosen a feasible arc and (a, (d, c)) becomes part of a network. For preconnection, (d, c), the set of all feasible connections is given by
The collection, \(\{K_{dc}:(d,c)\in D\times C\}\), contains the basic building block of layered networks. Let \(2^{K_{dc}}\) denote the collection of all closed subsets of \(K_{dc}\) (including the empty set), and let \(\mathbb {K} _{dc}\subset 2^{K_{dc}}\) be the collection of subsets of \(K_{dc}\) of size at most 1.
[N-2] Assume the following:
-
(1)
Each player, in considering whether or not to connect to a particular club either does not connect or connects in one and only one way.
-
(2)
Each player or group of players can move freely and unilaterally from one club to another via feasible connections. Thus, a player can drop his membership in any given club and, if allowed by the feasible arc correspondence, join any other club without bargaining and without seeking the permission of any player or group of players.Footnote 6
2.2 Layered Networks
If \(G_{cd}\in \) \(\mathbb {K}_{dc}\), then
The feasible set of club networks, \(\mathbb {K}:=(\mathbb {K}_{dc})_{dc}\), is given by an \(n\times m\) array of feasible player-club connections (see, 1), where \(\mathbb {K}_{c}:=(\mathbb {K}_{dc})_{d}\) is the feasible set of all possible c-layers (or club c layers), and where \( \mathbb {K}^{d}:=(\mathbb {K}_{dc})_{c}\) is the feasible set of all possible d-layers (or player d layers). In array form, we have
Formally, we have the following definition of a feasible club network, \(G\in \mathbb {K}:=(\mathbb {K}_{dc})_{dc}\), as an \(n\times m\) array of feasible player-club connections.
Definition 1
(Feasible Club Networks, c-Layers, d-Layers, and Connection Arrays) A feasible club network \(G\in \mathbb {K}\) is an \(n\times m\) array of feasible player-club connections given by
where for each player-club pair, (d, c),
is player \(d^{\prime }s\) part of c-layer, \(G_{c}\in \mathbb {K}_{c}\), given by the c th column in the club network, G, above, and where \(G_{dc}\in \mathbb {K}_{dc}\) if and only if
and where \(G^{d}\in \mathbb {K}^{d}\) is player \(d^{\prime }s\) connection choice profile, given by the d th row in the club network, G, above.
If we agree to the notational convention that \(G_{dc}:=a_{dc}\) then club networks in \(\mathbb {K}=(\mathbb {K}_{dc})_{dc}\) can be given a reduced form array representation—as an array of arc types (without loss of information)—as follows:
A club network, \(G\in \mathbb {K}\) implicitly determines an arc selection, \((d,c)\longrightarrow G(d,c)\) given by
with domain given by
If \(G(d,c)=\varnothing \), then in network G, the preconnection (d, c) is not elevated to the status of a connection (i.e., player d is not a member of club c in network G either because d chose not to join club c or because player d was not allowed to join club c—i.e., \( A(d,c)=\varnothing \)). Alternatively, if \(G(d,c)\ne \varnothing \), then preconnection (d, c) has been elevated to a connection. We note that \( G(d,c)\subset A(d,c)\) is a singleton set for all preconnections \((d,c)\in D\times C\) and for all preconnections (d, c) in \(\mathcal {D}(G)\subset \mathcal {D}(A(d,c))\), \(G(d,c)\ne \varnothing \). Thus, a club network’s arc selection, \(G(\cdot ,\cdot )\), is a selection from the feasible arc correspondence, \((d,c)\longrightarrow A(d,c)\) correspondence.
2.3 Coalition Structure
Each feasible club network, with its implied arc selection determines a coalition structure. In particular, given arc selection \(G(\cdot ,\cdot )\) determined by club network, \(G\in \mathbb {K}\), each club c has a membership coalition given by the domain of the mapping \(d\longrightarrow G(d,c)\) for fixed c. In particular, club c has membership coalition in network G, denoted by, \(S_{cG}\), and given by
where \(2^{D}\) is the collection of all subsets of D, including the empty set. Thus, the coalition structure determined by club network \(G\in \mathbb {K }\) is given by
In a club network, connections between layers are made through overlapping club memberships. Without this, each club layer is isolated. For example, in layered club network, \(G\in \mathbb {K}\), if \(S_{cG}\cap S_{c^{\prime }G}\ne \varnothing \), then each player \(d\in \) \(S_{cG}\cap S_{c^{\prime }G}\) is a member of club c as well as a member of club \( c^{\prime }\). In this way, club layers \(G_{c}\) and \(G_{c^{\prime }}\) are connected in network G. Note that if in club network, G, players are members of one and only one club then if the network has multiple nonempty layers, there are no connections between these layers—each layer is isolated precisely because there are no overlapping club memberships. In this case, the coalition structure induced by club network G given by
is a partition of the active club members—and players are siloed by their club memberships.
3 Measuring the Distance Between Club Networks
In order to analyze the co-evolution of strategic behavior, club network structure and equilibrium dynamics, we require a topology for the space of club networks that is simultaneously coarse enough to guarantee compactness of the set of networks and fine enough to discriminate between differences across networks that are due to differences in the ways nodes are connected (via differing arc types) versus differences across networks that are due to the complete absence of connections. We resolve this topological dilemma by equipping the space club networks, \( \mathbb {K},\) with the Hausdorff metric \(h_{K}\)—making the space of feasible club network connection arrays, a compact metric space (see Appendix 1 on the Hausdorff metric on the hyperspace of layered networks).
It is easy to show that if the Hausdorff distance between any pair club networks G and \(G^{\prime }\) is less than \(\varepsilon \in (0,1)\), then the networks can differ only in the ways a given set of player-club pairs are connected—and not in the set of player-club pairs that are connected. In particular, if for networks G and \(G^{\prime }\) , \(h_{K}(G,G^{\prime })<\varepsilon <1\), then
for arcs a and \(a^{\prime }\) with \(\rho _{A_{c}}(a,a^{\prime })<\varepsilon \). Thus, if two club networks are at \(h_{K}\)-distance \( \varepsilon <1\), then both club networks G and \(G^{\prime }\) have the same coalition structures, i.e.,
Such closeness will often occur and can only persist in network vio sets (sets of networks visited infinitely often by the equilibrium stochastic process of network formation) i.e., sets belonging to basins of attraction generated by the equilibrium dynamics governing the movements of club networks—all discussed in detail below.
We will equip the hyperspace of feasible club network connection arrays, \( \mathbb {K}=(\mathbb {K}_{dc})_{dc}\), with the Hausdorff metric given by
where
and where, due to the single arc connection rule,
4 Example: Time Allocation Networks
Consider a space of club networks in which the set of arc types, \(A_{c}\), is given by \(A_{c}=[0,1]\) for all clubs, c, and the feasible arc correspondence, \((d,c)\longrightarrow A(d,c)\), is given by \(A(d,c)=[0,1]\) for all player-clubs pairs. We have for each player-club pair, (d, c), that \(K_{dc}:=[0,1]\times (\{d\}\times \{c\})\) and \(2^{K_{dc}}\) is the hyperspace of all closed subsets of \(K_{dc}\) (including the empty set) and \(\mathbb {K} _{dc}\) is the collection of all subsets of \(K_{dc}\) of size 1. Thus, \( G_{dc}\in \mathbb {K}_{dc}\) if and only if
But now we add a further condition given by the time allocation constraint. We will require that the d-layers, \(\mathbb {K}^{d}\), in each club network \( G\in \) \(\mathbb {K}:=(\mathbb {K}_{dc})_{dc}\), having arc selection, \( (d,c)\longrightarrow G(d,c)\), be such that
where \(\mathcal {D}(G(d,\cdot ))\) is the club domain of player d—the clubs to which d belongs. The feasible set of club networks is,
Thus, in a time allocation network, players freely and noncooperatively choose which clubs they want to join as well as their levels of participation in each club they join. Their levels of participation in each club are then expressed as fractions of their total club time—and these fractions must sum to one.
Figure 1 depicts a time allocation in which 5 players, \( D=\{d_{1},d_{2},d_{3},d_{4},d_{5}\}\) allocate their club time across 3 clubs, \(C=\{c_{1},c_{2},c_{3}\}\).
Following the notational convention that
the connections array in reduced form for network G in Fig. 1 is given by
Because the d-layers (the rows) must have arc labels, \(a_{dc}\in [0,1]\), that sum to 1, and because players \(d_{1},d_{3},d_{4},\) and \(d_{5}\) spend all of their time in one and only one club, we know that in the case of player \(d_{1}\), for example, \(a_{11}=1\), while \(a_{12}=a_{13}=0\). Player \( d_{2}\) splits all of his club time between clubs \(c_{2}\) and \(c_{3}\). Thus, in the case of player \(d_{2}\), we have that \(a_{22}\) and \(a_{23}\) are positive with \(a_{22}+a_{23}=1\). Club \(c_{2}\) has 3 members, \( S_{c_{2}G}=\{d_{2},d_{3},d_{5}\}\), while club \(c_{1}\) has 2 members, \( S_{c_{1}G}=\{d_{1},d_{4}\}\), and club \(c_{3}\) has 1 member, \( S_{c_{3}G}=\{d_{2}\}\).
Again following the notational convention (8) and writing out the connections long hand, the connections in the d-layers are given by
while the connections in the c-layers are given by
We note that the time allocation club networks in \(\mathbb {K}_{TA}\) and in general the club networks in \(\mathbb {K}\) satisfy assumptions [N-1]. We will assume for the remainder of the paper that [N-2] holds. We will return to the example above later in the paper.
5 Discounted Stochastic Games of Club Network Formation
In order to address the questions of whether or not and under what conditions the strategic formation of club networks will lead to the emergence of dynamically stable coalition structures, we will show that our discounted stochastic game (DSG) of club network formation possesses Nash equilibria in stationary Markov perfect behavioral club network formation strategies. It is the players’ equilibrium behavioral network formation strategies which determine the equilibrium dynamics of club network formation. By identifying conditions under which stationary Markov perfect equilibria (SMPE) exist in such behavioral strategies and by showing that the resulting equilibrium state and club network dynamics are stable, we will be able to establish the conditions under which stable coalition structures will emerge and persist in the form of stable club networks. In this section, we will construct a DSG model of club network formation and show that our model possesses SMPE in behavioral club network formation strategies. The SMPE existence problem in the setting considered here (with uncountable states and compact metric action spaces) is quite difficult and its solution—and counterexamples—are of independent interest (for details see [15, 22, 23, 27, 28]).
An m-player, nonzero sum, discounted stochastic game, DSG, over the product space of probability measures over player club networks (i.e., the convex, compact metric space of behavioral actions),
is given by the following primitives:
where \(\Omega \) is the state space, \(B_{\Omega }\) is the Borel \(\sigma \) -field of events, and \(\mu \) is a probability measure. For each player d, \( \mathbb {K}^{d}\) is the set of all possible player club networks available to player d, while \(\Delta (\Phi _{d}(\omega ))\) is the convex, compact feasible set of behavioral action available to player d in state \(\omega \) . A feasible behavioral action available to player d in state \( \omega \) is a probability measure \(\sigma _{d}\in \Delta (\mathbb {K}^{d})\) with support contained in \(\Phi _{d}(\omega )\) (i.e., \(\sigma _{d}\in \Delta (\Phi _{d}(\omega ))\) if and only if \(\sigma _{d}(\Phi _{d}(\omega ))=1\)). Finally, \(\beta _{d}\in (0,1)\) is player \(d^{\prime }s\) discount rate and \( U_{d}(\omega ,v_{d},\cdot )\) is player \(d^{\prime }s\) payoff function in state \(\omega \) given valuations (or prices) \(v_{d}\), and \(q(\cdot |\omega ,\cdot )\) is the law of motion in state \(\omega \). If players holding value function profile \(v=(v_{1}\ldots ,v_{n})\) choose feasible profile of behavioral actions,
in state \(\omega \), then the next state \(\omega ^{\prime }\) is chosen in accordance with probability measure \(q(\cdot |\omega ,G)\in \Delta (\Omega )\) and player \(d^{\prime }s\) stage payoff is given by
Here, \(\pi \sigma (\mathrm{d}G):=\pi (\sigma _{1}(\mathrm{d}G^{1}),\ldots ,\sigma _{n}(\mathrm{d}G^{n})):=\otimes _{d=1}^{n}\sigma _{d}(\mathrm{d}G^{d})\) is the product probability measure representing the random club network determined by the n-tuple of random player club networks, \((\sigma _1(dG^1),..., \sigma _n(dG^n))\), chosen by the players.
We will denote by, \(\mathcal {G}_{{\small (\omega ,v)}}:=\left( \Delta (\Phi _{d}(\omega )),U_{d}(\omega ,v_{d},\cdot )\right) _{d\in D}\), the n-player \({\small (\omega ,v)}\)-game in state \(\omega \) underlying the DSG when players hold valuations (or state-contingent prices), \(v:=(v_{1},\ldots ,v_{n})\).
5.1 Primitives and Assumptions
We will maintain the following assumptions throughout. Label these (i.e., assumptions (1)–(15)) as [A-1]:
-
(1)
\(D=\) the set of players, consisting of n players indexed by \(d=1,2,\ldots ,n\) and each having discount rate given by \(\beta _{d}\in (0,1)\).
-
(2)
\((\Omega ,B_{\Omega },\mu )\), the state space where \( \Omega \) is a complete separable metric spaces with metric \(\rho _{\Omega }\), equipped with the Borel \(\sigma \)-field, \( B_{\Omega }\), upon which is defined a probability measure, \(\mu \) .
-
(3)
\(Y:=Y_{1}\times \cdots \times Y_{n}\), the space of all possible players’ payoff profiles, \(U:=(U_{1},\ldots ,U_{n})\), such that for each player d, \(Y_{d}:=[-M,M]\) and is equipped with the absolute value metric, \(\rho _{Y_{d}}(U_{d},U_{d}^{\prime }):=\left| U_{d}-U_{d}^{\prime }\right| \) and Y is equipped with the sum metric, \(\rho _{Y}:=\sum _{d}\rho _{Y_{d}}\).
-
(4)
\(\mathbb {K}:=\mathbb {K}^{1}\times \cdots \times \mathbb {K}^{n},\) the set of player pure action profiles, \(G:=(G^{1},\ldots ,G^{n})\) , where for each player d,
$$\begin{aligned} G^{d}:=(G_{d1},\ldots ,G_{dc},\ldots ,G_{dm}) \end{aligned}$$is player \(d^{\prime }s\) club network and where \(\mathbb {K} ^{d}:=\mathbb {K}_{d1}\times \cdots \times \mathbb {K}_{dc}\times \cdots \times \mathbb {K}_{dm}\) is a compact metric space of feasible player club networks with typical element, \(G^{d}\), equipped with metric, \( h_{K^{d}}:=\sum _{c}h_{K_{dc}}\), and \(\mathbb {K}\) is equipped with the sum metric, \(h_{K}:=\sum _{d=1}^{n}\sum _{c=1}^{m}h_{K_{dc}}.\)
-
(5)
\(\Delta (\mathbb {K}^{d})\)is the space of all probability measures, \(\sigma _{d}\), with supports contained in player \( d^{\prime }s\)set of club networks, \(\mathbb {K}^{d}\), equipped with the compact metrizable weak star topology (a topology denoted by \( w_{ca}^{*d}\)) inherited from \(ca(\mathbb {K}^{d})\), the Banach space of finite signed Borel measures on \(\mathbb {K}^{d}\) with the total variation norm.Footnote 7 We will equip \(\Delta (\mathbb {K}^{d})\) with a metric, \(\rho _{w_{ca}^{*d}}\), compatible with the relative \(w_{ca}^{*d}\)-topology on \(\Delta (\mathbb {K}^{d})\) inherited from \(ca(\mathbb {K}^{d})\) and we will refer to \( \sigma _{d}\) as player \(d^{\prime }s\) random player club network.
-
(6)
\(\Delta (\mathbb {K}):=\Delta (\mathbb {K}^{1})\times \cdots \times \Delta (\mathbb {K}^{n})\), the space of player behavioral action profiles, \(\sigma :=(\sigma _{1},\ldots ,\sigma _{n})\), equipped with the sum metric, \(\rho _{w_{ca}^{*}}:=\sum _{d}\rho _{w_{ca}^{*d}}\) , a metric compatible with the relative \(w_{ca}^{*}\)-product topology on
$$\begin{aligned} \Delta (\mathbb {K}^{1})\times \cdots \times \Delta (\mathbb {K}^{n}) \end{aligned}$$inherited from \(ca(\mathbb {K}^{1})\times \cdots \times ca(\mathbb {K} ^{n})\).
-
(7)
\(\omega \longrightarrow \Phi _{d}(\omega )\), is player \( d^{\prime }s\) measurable action constraint correspondence, defined on \(\Omega \) taking nonempty \(h_{K^{d}}\)-closed (and hence \( h_{K^{d}}\)-compact) network values in \(\mathbb {K}^{d}\).
-
(8)
\(\omega \longrightarrow \Phi (\omega ):=\Phi _{1}(\omega )\times \cdots \times \Phi _{n}(\omega )\), players’ measurable action profile constraint correspondence, defined on \(\Omega \) taking nonempty \( h_{K}\)-closed (and hence compact) network values in \(\mathbb {K}.\)
-
(9)
\(\omega \longrightarrow \Delta (\Phi _{d}(\omega ))\), is player \(d^{\prime }s\) measurable behavioral action constraint correspondence, defined on \(\Omega \) taking nonempty \(w_{ca}^{*d}\)-closed (and hence \(w_{ca}^{*d}\)-compact), convex random network values in \(\Delta (\mathbb {K}^{d})\), containing all probability measures, \(\sigma _{d}(\omega )\), with supports contained in player \(d^{\prime }s\) feasible set of actions, \(\Phi _{d}(\omega )\subset \mathbb {K}^{d}\), in state \(\omega \).
-
(10)
\(\omega \longrightarrow \Delta (\Phi (\omega )):=\Delta (\Phi _{1}(\omega ))\times \cdots \times \Delta (\Phi _{n}(\omega ))\), players’ measurable behavioral action profile constraint correspondence, defined on \(\Omega \) taking nonempty \(w_{ca}^{*}\)-closed (and hence \(w_{ca}^{*}\)-compact), convex values in \(\Delta ( \mathbb {K}^{1})\times \cdots \times \Delta (\mathbb {K}^{n}).\)
-
(11)
\(\mathcal {L}_{Y_{d}}^{\infty }\), the Banach space of all \( \mu \)-equivalence classes of measurable (value) functions, \( v_{d}(\cdot ),\) defined on \(\Omega \) with values in \(Y_{d}\) a.e. \([\mu ]\), equipped with metric \(\rho _{w_{d}^{*}}\) compatible with the weak star topology inherited from \(\mathcal {L} _{R}^{\infty }\).Footnote 8
-
(12)
\(\mathcal {L}_{Y}^{\infty }:=\mathcal {L}_{Y_{1}}^{\infty }\times \cdots \times \mathcal {L}_{Y_{m}}^{\infty }\subset \mathcal {L} _{R^{m}}^{\infty }\), the Banach space of all \(\mu \)-equivalence classes of measurable (value) function profiles, \(v(\cdot ):=(v_{1}(\cdot ),\ldots ,v_{m}(\cdot ))\), defined on \(\Omega \) with values in Y a.e. \([\mu ]\), equipped with the sum metric \(\rho _{w^{*}}:=\sum _{d}\rho _{w_{d}^{*}}\) compatible with the weak star product topology inherited from \(\mathcal {L}_{R^{m}}^{\infty }\).
-
(13)
\(\mathcal {S}^{\infty }(\Delta (\Phi _{d}(\cdot )))\), the set of all \(\mu \)-equivalence classes of measurable functions (selections), \(\sigma _{d}(\cdot )\in \mathcal {L}_{\Delta (\mathbb { K}^{d})}^{\infty },\) defined on \(\Omega \) such that in \( \sigma _{d}(\omega )\in \Delta (\Phi _{d}(\omega ))\) a.e. \([\mu ]\) , and
$$\begin{aligned} \mathcal {S}^{\infty }(\Delta (\Phi (\cdot )))=\mathcal {S}^{\infty }(\Delta (\Phi _{1}(\cdot )))\times \cdots \times \mathcal {S}^{\infty }(\Delta (\Phi _{n}(\cdot ))) \end{aligned}$$(13)the set of all \(\mu \)-equivalence classes of measurable profiles (selection profiles), \(\sigma (\cdot )=(\sigma _{1}(\cdot ),\ldots ,\sigma _{n}(\cdot ))\), defined on \(\Omega \) such that in \( \sigma (\omega )\in \Delta (\Phi (\omega )):=\Delta (\Phi _{1}(\omega ))\times \cdots \times \Delta (\Phi _{n}(\omega ))\) a.e. \([\mu ]\).
-
(14)
\(r_{d}(\cdot ,\cdot ):\Omega \times \mathbb {K}\longrightarrow Y_{d}\) is player \(d^{\prime }s\) Caratheodory stage payoff function (i.e., for all \((\omega ,G)\in \Omega \times \mathbb {K}\), \( r_{d}(\omega ,\cdot )\) is \(h_{K}\)-continuous on \(\mathbb {K}\) and \(r_{d}(\cdot ,G)\) is \((B_{\Omega },B_{Y_{d}})\) -measurable on \(\Omega \)).
-
(15)
\(q(\cdot |\cdot ,\cdot ):\Omega \times \mathbb {K}\longrightarrow \Delta (\Omega )\) is the law of motion defined on \(\Omega \times \mathbb {K}\) taking values in the space of probability measures on \( \Omega \), having the following properties: (i) each probability measure, \(q(\cdot |\omega ,G)\), in the collection
$$\begin{aligned} Q(\Omega \times \mathbb {K}):=\left\{ q(\cdot |\omega ,G):(\omega ,G)\in \Omega \times \mathbb {K}\right\} \end{aligned}$$is absolutely continuous with respect to \(\mu \)(denoted \( Q(\Omega \times \mathbb {K})<<\mu \)), (ii) for each \(E\in B_{\Omega }\) , \(q(E|\cdot ,\cdot )\) is measurable on \(\Omega \times \mathbb {K}\), (iii) the collection of probability density functions,
$$\begin{aligned} H_{\mu }:=\left\{ h(\cdot |\omega ,G):(\omega ,G)\in \Omega \times \mathbb {K} \right\} , \end{aligned}$$(14)of \(q(\cdot |\omega ,G)\) with respect to \(\mu \) is such that for each state \(\omega \), the function
$$\begin{aligned} G\longrightarrow h(\omega ^{\prime }|\omega ,G) \end{aligned}$$(15)is \(h_{\mathbb {K}}\)-continuous in G a.e. \([\mu ]\) in \(\omega ^{\prime }.\)
We note that the assumptions are above are the usual assumptions underlying discounted stochastic game models (see the Appendix 2 for technical notes on these assumptions).
6 Stationary Markov Perfect Equilibria in Club Network Formation Strategies
6.1 Existence
Let DSG be a discounted stochastic game of club network formation satisfying assumptions [A-1], with one-shot game,
Definition 2
(Nash Equilibria in Behavioral Strategies) A feasible profile of probability measures over player club networks, \(\sigma ^{*}:=(\sigma _{1}^{*},\ldots ,\sigma _{m}^{*})\in \Delta (\Phi (\omega ))\) is said to be a Nash equilibrium of the one-shot network formation game, \(\left( \Delta (\Phi _{d}(\omega )),U_{d}(\omega ,v_{d},\cdot )\right) _{d}\), provided that for each player d,
Denote by \(\mathcal {N}(\omega ,v)\) the set of all Nash equilibria belonging to \(\left( \Delta (\Phi _{d}(\omega )),U_{d}(\omega ,v_{d},\cdot )\right) _{d}\), and by \(\mathcal {P}(\omega ,v)\) the set of all Nash equilibria payoffs belonging to \(\left( \Delta (\Phi _{d}(\omega )),U_{d}(\omega ,v_{d},\cdot )\right) _{d}\). Thus,
Under assumptions, [A-1], we know that \(\mathcal {N}(\omega ,v)\) is nonempty and \(\rho _{w_{ca}^{*}}\)-compact and therefore we know that \(\mathcal {P} (\omega ,v)\) is nonempty and \(\rho _{Y}\)-compact. Moreover, applying optimal measurable selection results (e.g., [19]) and Berge’s Maximum Theorem (e.g., see 17.31 in [1]), we can show that the Nash correspondences, \(\mathcal {N}(\cdot ,\cdot ) \) and \(\mathcal {P}(\cdot ,\cdot )\), are upper Caratheodory (also, see Proposition 4.2 in Page [26]). In particular, the Nash correspondence, \( \mathcal {N}(\cdot ,\cdot )\), is jointly measurable in \((\omega ,v)\) and \( \mathcal {N}(\omega ,\cdot )\), is upper semicontinuous in v for each \( \omega \), and the Nash payoff correspondence, \(\mathcal {P}(\cdot ,\cdot )\), is jointly measurable in \((\omega ,v)\) and \(\mathcal {P}(\omega ,\cdot )\), is upper semicontinuous in v for each \(\omega \).
Our main existence result is the following:
Theorem 1
(All DSGs satisfying [A-1] have SMPE in behavioral strategies) Let
be a discounted stochastic game of club network formation satisfying assumptions [A-1], with upper Caratheodory (uC) Nash correspondences, \(\mathcal {N}(\cdot ,\cdot ) \) and \(\mathcal {P}(\cdot ,\cdot )\).
There exists a pair, \((v^{*},\sigma ^{*}(\cdot ))\in \mathcal { L}_{Y}^{\infty }\times \mathcal {S}^{\infty }(\Delta (\Phi (\cdot )))\) such that \(\sigma ^{*}(\cdot )\in \mathcal {S}^{\infty }(\mathcal {N} _{v^{*}})\) is a Stationary Markov Perfect Equilibrium (SMPE) in behavioral club network formation strategies supported by Bellman prices \(v^{*}\in \mathcal {S}^{\infty }(\mathcal {P}_{v^{*}})\)—i.e., there exists a pair, \((v^{*},\sigma ^{*}(\cdot ))\in \mathcal {L}_{Y}^{\infty }\times \mathcal {S}^{\infty }(\Delta (\Phi (\cdot )))\) such that a.e. \([\mu ]\)
For a formal proof, we refer the reader to [15]. Informally, the proof proceeds along the following lines: Let \(\mathcal {S}^{\infty }( \mathcal {P}(\cdot ,v))\) be the set of all \(\mu \)-equivalence classes of measurable functions, \(u\in \mathcal {L}_{Y}^{\infty }\), such that \(u(\omega )\in \) \(\mathcal {P}(\omega ,v)\) a.e. \([\mu ]\). It follows from Blackwell’s Theorem [9], extended to DSGs, that our discounted stochastic game of club network formation (10) will have stationary Markov perfect equilibria in network formation strategies if and only if there exists a value function profile, \(v^{*}\in \mathcal {L} _{Y}^{\infty }\), such that
or equivalently if and only if the Nash payoff selection correspondence,
has fixed points, i.e., has at least one value function profile, \(v^{*}\in \mathcal {L}_{Y}^{\infty }\), such that \(v^{*}\in \mathcal {S}^{\infty }(\mathcal {P}_{v^{*}})\). This is a very difficult fixed point problem because the measurable selection valued correspondence, \(\mathcal {S}^{\infty }(\mathcal {P}_{(\cdot )})\), is neither convex valued nor closed value—nor of course is it upper semicontinuous. Until [15], no results were available to establish the existence of a fixed point for Nash payoff selection correspondences. Essentially what [15] show is that while the Nash payoff selection correspondence, \(v\longrightarrow \mathcal {S} ^{\infty }(\mathcal {P}_{v})\), is badly behaved, under assumptions [A-1], its underlying upper Caratheodory Nash payoff correspondence, \(\mathcal {P}(\cdot ,\cdot )\), always contains an upper Caratheodory sub-correspondence, \( p(\cdot ,\cdot )\), that has \(\varepsilon \)-approximate Caratheodory selections for all \(\varepsilon >0\), implying that the Nash payoff selection sub-correspondence, \(v\longrightarrow \mathcal {S}^{\infty }(p_{v})\), has fixed points. Thus, [15] show that, in general, there exists a Nash payoff sub-correspondence, \(\mathcal {S}^{\infty }(p_{(\cdot )})\), having fixed points, i.e., that there exists \(v^{*}\in \mathcal {L} _{Y}^{\infty }\), such that \(v^{*}\in \mathcal {S}^{\infty }(p_{v^{*}})\subset \mathcal {S}^{\infty }(\mathcal {P}_{v^{*}})\). More fundamentally, [15] are able to show that the upper Caratheodory Nash payoff sub-correspondence, \(p(\cdot ,\cdot )\), has \( \varepsilon \)-approximate Caratheodory selections for all \(\varepsilon >0\), because under assumption [A-1] the underlying upper Caratheodory Nash correspondence, \(\mathcal {N}(\cdot ,\cdot )\), always contains an upper Caratheodory Nash sub-correspondence, \(\eta (\cdot ,\cdot )\), that takes closed connected values. Thus, because
the continuity of \(U_{d}(\omega ,v_{d},\cdot )\) in behavioral actions \( \sigma \) for each player \(d=1,2,\ldots ,n\) together with the closed connectedness of \(\eta (\omega ,v)\) implies that
is upper Caratheodory and interval valued for each player \( d=1,2,\ldots ,n\). It then follows from Corollary 4.3 in [21] that each player’s Nash payoff sub-correspondence, \(p_{d}(\cdot ,\cdot )\), is Caratheodory approximable and this together with the unusual properties of Komlos convergence [20] of value functions allow us to show that there exists a value function profile, , \(v^{*}\in \mathcal {L} _{Y}^{\infty }\), such that \(v^{*}(\omega )\in p(\omega ,v^{*})\) a.e. \([\mu ]\) or equivalently such that
In order to complete our informal argument for existence we need only note that by implicit measurable selection (e.g., see Theorem 7.1 in Himmelberg [18]), there exists a profile, \(\sigma ^{*}(\cdot )=(\sigma _{1}^{*}(\cdot ),\ldots ,\sigma _{n}^{*}(\cdot ))\), of a.e. measurable selections of \(\omega \longrightarrow \eta (\omega ,v^{*})\), such that for each player \(d=1,2,\ldots ,n\),
Thus, for each player d, the state-contingent prices given by value function, \(v_{d}^{*}(\cdot )\in \mathcal {L}_{Y_{d}}^{\infty }\), incentivizes the continued choice by each player d, of behavioral strategy, \(\sigma _{d}^{*}(\cdot )\), and we have for the value function-strategy profile pair, \((v^{*},x^{*}(\cdot ))\in \mathcal {S} ^{\infty }(p_{v^{*}})\times \mathcal {S}^{\infty }(\eta _{v^{*}})\), that
implying that
Thus, for value function-behavioral strategy profile pair, \((v^{*},\sigma ^{*}(\cdot ))\), we have for each player \(d=1,2,\ldots ,n\) and for \(\omega \) a.e. \([\mu ]\), that \((v^{*},\sigma ^{*}(\cdot ))\) satisfies the Bellman equation (1) and the Nash condition (2),
Thus, \(\sigma ^{*}(\cdot )\in \) \(\mathcal {S}^{\infty }(\mathcal {N} _{v^{*}})\) is a stationary Markov perfect equilibrium of a DSG satisfying assumptions [A-1], incentivized by state-contingent prices, \( v^{*}\in \) \(\mathcal {S}^{\infty }(\mathcal {P}_{v^{*}})\).
Thus here we have argued informally—and shown formally in [15]—that under the usual assumptions specifying a discounted stochastic game (in this case a club network formation DSG), while the \( DSG^{\prime }s\) Nash payoff selection is badly behaved, nonetheless, it naturally possesses (without additional assumptions) an underlying Nash payoff correspondence, \(\mathcal {P}(\cdot ,\cdot )\), containing sub-correspondences, \(p(\cdot ,\cdot )\), which are Caratheodory approximable implying that its induced selection correspondence, \(v\longrightarrow \mathcal {S}^{\infty }(p_{v^{*}})\) has fixed points. [16], by making an additional assumption (that the DSG is \(\mathcal {G}\) -nonatomic or has a coarser transition kernel) guarantee that \( DSG^{\prime }s\) Nash payoff selection correspondence has a convex valued sub-correspondence—and therefore an approximable sub-correspondence.Footnote 9 Moreover, [16] show that [12] accomplishes the same thing by assuming that the DSG has a noisy state. In the negative direction, [22, 23] construct counterexamples showing that not all uncountable-finite DSGs have stationary Markov perfect equilibria. They accomplish this by constructing counterexamples in which the Nash correspondences are not approximable—which follows from the fact that in their counterexamples, there is an absence of fixed points. Because the club network formation DSG we analyze here is approximable, we avoid the Levy-McLennan counterexamples.
6.2 Example: DSGs over Time Allocation Networks
The one-shot game for a DSG satisfying assumptions [A-1] over time allocation networks, \(\mathbb {K}_{TA}\), is given by
In a \((\omega ,v)\)-game each player d has preferences over the d-layer networks in \(\mathbb {K}^{d}\) given by,
where \(G^{d}=(G_{d1},\ldots ,G_{dm})\) and \(G_{dc}=\{(a_{dc},(d,c))\}\) for \( c\in \mathcal {D}(G(d,\cdot ))\), and \(G_{dc}=\varnothing \) for \(c\notin \mathcal {D}(G(d,\cdot ))\). Rewriting these preferences as preferences over d-layer networks in reduced from, we have
where
and where
with notational convention, \(a_{dc_{k}}\in (0,1]\Longleftrightarrow G_{dc_{k}}=\{(a_{dc_{k}},(d,c_{k}))\}\) and \(a_{dc_{k}}=0\Longleftrightarrow G_{dc_{k}}=\varnothing \).
We will assume that for each player d and for each \((\omega ,v)\in \Omega \times \mathcal {L}_{Y}^{\infty }\), \(u_{d}(\omega ,v_{d},\cdot )\) is affine on \(\mathbb {K}_{TA}\). Thus, for time allocation networks, G and \( G^{\prime }\) in \(\mathbb {K}_{TA}\), we have for any \(\lambda \in [0,1]\) ,
where
We will also assume that players’ one-shot payoff functions,
are measurable in \(\omega \) for each \((v_{d},a^{d},a^{-d})\) and jointly continuous in \((v_{d},a^{d},a^{-d})\) for each \(\omega \).
By Theorem 1 (also see [15]), there exists a value function profile, \(v^{*}\in \mathcal {L}_{Y}^{\infty }\) and an n-tuple of measurable function
where by notational convention,
and where,
In fact, given the assumptions that players’ one-shot payoff functions are affine over the convex set of time allocation networks (see 28 and 29), we know by Corollary 1.4 in [4] that the m-tuple,
consists of \(1^{\prime }s\) and \(0^{\prime }s\), because for each club \(c_{k}\) , \(a_{d_{i}c_{k}}^*(\omega )\) is an extreme point of the interval [0, 1] a.e. \([\mu ]\). Thus, in each state a player spends \(100\%\) of his time in one and only one club—i.e., each player’s stationary Markov perfect time allocation strategy is bang-bang. Thus, we have by Theorem 1 that
where by Corollary 1.3 in [4], a.e. \([\mu ]\) in \(\omega \), the stationary Markov perfect time allocation network, given by the arrays,
has a reduced form \(n\times m\) array, \((a_{dc}^{*}(\omega ))_{dc}\), consisting of \(0^{\prime }s\) and \(1^{\prime }s\). Moreover, in equilibrium, the state-contingent coalition of members of any club \(c_{k}\) determined by the equilibrium club network \(G^{*}(\cdot )\) is given by
7 Equilibrium State-Network Dynamics and Stable Coalitions
Returning to our general model, under the profile of stationary Markov perfect equilibrium behavioral strategies
the equilibrium state and network formation process is given by,
with underlying probability space \((\Omega \times \mathbb {K},B_{\Omega }\times B_{\mathbb {K}},\mu \otimes \sigma ^{*}):=(\mathbb {Z},B_{\mathbb {Z}},P^{*})\) consists of random objects,
such that in each state \(z=(\omega ,G)\in \Omega \times \mathbb {K}\),
and where for any measurable set \(E\in B_{\Omega }\times B_{\mathbb {K}}\),
\(E_{\omega }:=\{G\in \) \(\mathbb {K}:(\omega ,G)\in E\}\) (see 2.6.2, the Product Measure Theorem, in [3]).
The movements of the process, \(\left\{ Z_{t}^{*}\right\} _{t=0}^{\infty } \), are governed by the equilibrium Markov transition kernel,
For each \((\omega ,G)\in \Omega \times \mathbb {K}\), \(\pi ^{*}(d(\omega ^{\prime },G^{\prime })|\omega ,G)\) is given by the product probability measure, \(q(\cdot |\omega ,G)\otimes Q^{*}(\cdot |\omega ,G)\), where for each measurable rectangle, \(R\times \mathbb {R}\in B_{\Omega }\times B_{ \mathbb {K}}\),
Thus, if the current state-club network is \((\omega ,G)=(\omega ,G^{1},\ldots ,G^{n})\), with d-layers \((G^{d})_{d=1}^{n}\), then the probability that the coming state and coming club network lie in the set, \( R\times \mathbb {R}\in B_{\Omega }\times B_{\mathbb {K}}\), is given by
Whether or not a stable club network emerges depends on the stability properties of equilibrium state-network dynamics underlying club network formation. Our main objective is to say what we can regarding the emergence of stable club structures—and hence stable coalition structures—based on the stability properties of the equilibrium state-network formation dynamics. We would argue that this is one of the main aspects of network dynamics that has been missing from our understanding of network dynamics in various applied settings—the influence of strategic behavior on network dynamics. Here, we present a first attempt. We will proceed as follows: First we state the Tweedie (Stability) Conditions [35] guaranteeing Markov stability. Second, we show that if we slightly strengthen assumptions [A-1] (2) and (15) in our discounted stochastic game model, then the stationary Markov perfect equilibrium (SMPE) of our discounted stochastic game of network formation gives rise to a Markov stable equilibrium state-network formation process satisfying the Tweedie Conditions. Finally, we summarize some of the main implications of the Tweedie Conditions and the resulting Markov stability properties of the equilibrium state-network formation process—such as the existence of finitely many basins of attraction and ergodic probabilities, and the implications of Markov stability for the emergence of stable coalition structures.
We begin by stating the Tweedie Conditions, and after strengthening assumptions [A-1] (2) and (15)(iii) in our discounted stochastic game model, showing that our strengthened discounted stochastic game model, in general, gives rise to an equilibrium state-network process satisfying the Tweedie conditions.
7.1 The Tweedie Conditions and Discounted Stochastic Games of Club Network Formation
We say that the Markov transition, \(\pi ^{*}(\cdot |\cdot )\), satisfies the Tweedie conditions, [T], provided,
-
(1)
(Drift Condition) There exists (i) a nonnegative-valued measurable function, \(V(\cdot ):\Omega \times \mathbb {K}\longrightarrow [0,+\infty ]\), with \(V(z^{0})<+\infty \) for some \(z^{0}\in \Omega \times \mathbb {K}\); (ii) a subset \(C\subset \Omega \times \mathbb {K}\), and (iii) a finite real number, \(-\infty<b<+\infty \) , such that we have
$$\begin{aligned} \int _{\Omega \times \mathbb {K}}V(z^{\prime })\pi ^{*}(\mathrm{d}z^{\prime }|z)\le V(z)-1+bI_{C}(z). \end{aligned}$$(36) -
(2)
(Uniform Countable Additivity) For any sequence of measurable sets, \(\{S_{n}\}_{n}\subset B_{\Omega }\times B_{\mathbb {K}}\) with \( S_{n}\downarrow \varnothing \),
$$\begin{aligned} \lim _{n\longrightarrow +\infty }\sup _{z\in C}\pi ^{*}(S_{n}|z)=0. \end{aligned}$$(37)
The intuition behind the Tweedie conditions can be described as follows: Rewriting the first condition (36)—the drift condition—we have for each \(z\in \Omega \times \mathbb {K}\)
where
is the drift operator evaluated at \(z=(\omega ,G)\in \Omega \times \mathbb {K} \). The drift operator measures the expected drift (or movement) of the process away from states in C starting from state z. Drift is measured by the value taken by the drift operator, \(\triangle V(\cdot )\), at z. We see from the state-contingent bound on the right-hand side of (38) that if we are measuring expected drift from a state z in C and if \(b>1\), so that \(b-1>0\), then condition (36) will tolerate some drift away from C. However, if we are measuring expected drift from a state z not in C, then condition (36) requires that the expected drift be back toward C. The first Tweedie condition requires that the equilibrium process be such that there exist such a function, \( V(\cdot ):\Omega \times \mathbb {K}\longrightarrow [0,+\infty ]\), finite valued at least at one point, a set C, and a bounding constant, b , such (36) is satisfied. The second Tweedie condition requires that the collection of probability measures,
determined by discounted stochastic game’s strengthened law of motion and the equilibrium behavioral strategies of the players be uniformly countably additive (37)—guaranteeing therefore that the expected drift of the process is a continuous function of state-network pairs (i.e., that there are no discontinuous jumps in the expected magnitude of the drift).
The required strengthening of our stochastic continuity assumptions [A-1](15) is the following:
[A-1](15)(iii)\(^{*}\)In addition to [A-1](15)(iii), we will assume that for any sequence of state-club network pairs, \(\{(\omega ^{n},G^{n})\}_{n}\), converging to \((\omega ^{*},G^{*})\) under the metric \(\rho _{\Omega \times \mathbb {K}}:=\rho _{\Omega }+h_{K}\) on the product of the state space \(\Omega \) and the club network space \(\mathbb {K}\),
for any \(R \in B_\Omega \), i.e., the sequence of probability measures, \(\{q(\cdot |\omega ^{n},G^{n})\}_{n}\) converges setwise to \(q(\cdot |\omega ^{*},G^{*})\).
Under the strengthened stochastic continuity assumptions, labeled [A-1](15)\( ^{*}\), because the sequence, \(\{q(R|\omega ^{n},G^{n})\}_{n}\), converges to \(q(R|\omega ^{*},G^{*})\) uniformly in \(R\in B_{\Omega }\) and because \(\sigma ^{*}(\mathbb {R}|\cdot )\) is a bounded measurable function for any \(\mathbb {R\in }B_{\mathbb {K}}\), it follows from Delbaen’s Lemma [11] that \(\{Q^{*}(\cdot |\omega ^{n},G^{n})\}_{n}\) setwise converges to \(Q^{*}(\cdot |\omega ^{*},G^{*})\), and in particular that
for any \(\mathbb {R\in }B_{\mathbb {K}}\). Thus, given that \(\{q(\cdot |\omega ^{n},G^{n})\}_{n}\) and \(\{Q^{*}(\cdot |\omega ^{n},G^{n})\}_{n}\) setwise converge to \(q(\cdot |\omega ^{*},G^{*})\) and \(Q^{*}(\cdot |\omega ^{*},G^{*})\) for any sequence of state-club network pairs, \( \{(\omega ^{n},G^{n})\}_{n}\), converging to \((\omega ^{*},G^{*})\), we have that
for any measurable rectangle, \(R\times \) \(\mathbb {R\in B}_{\Omega }\times B_{ \mathbb {K}}\) i.e., that \(\{\pi ^{*}(\cdot |\omega ^{n},G^{n})\}_{n}\), setwise converges to \(\pi ^{*}(\cdot |\omega ^{*},G^{*})\), for any sequence, \(\{(\omega ^{n},G^{n})\}_{n}\), converging to \((\omega ^{*},G^{*})\). Moreover, we know that if \(\{\pi ^{*}(\cdot |z^{n})\}_{n}\) setwise converges to \(\pi ^{*}(\cdot |z^{*})\) for any sequence \(\{z^{n}\}_{n}=\{(\omega ^{n},G^{n})\}_{n}\), converging to \( z^{*}=(\omega ^{*},G^{*})\), then for any bounded and continuous function, \(c(\cdot )\in C_{b}(\Omega \times \mathbb {K})\)—or any bounded measurable function—
i.e., \(\{\pi ^{*}(\cdot |z^{n})\}_{n}\) converges weak star to \( \pi ^{*}(\cdot |z^{*})\).
Up until now, we have assumed that the set of states, \(\Omega \), is Polish (complete, separable, metric). Our strengthening of [A-1](2) is to assume that
[A-1](2)\(^{*}\) In the probability space of states, \( (\Omega ,B_{\Omega },\mu )\), \(\Omega \), is a compact metric space.
Because the hyperspace of club networks, \(\mathbb {K}\), is compact, with the strengthening of [A-1](2) making \(\Omega \) compact, we have for any sequence, \(\{(\omega ^{n},G^{n})\}_{n}\), in \(\Omega \times \mathbb {K}\) that there is a convergent subsequence \(\{(\omega ^{n_{k}},G^{n_{k}})\}_{k} \) with limit \((\omega ^{*},G^{*})\in \Omega \times \mathbb {K}\). Given the arguments and observations immediately above this means that the set of probability measures,
is weak star compact—because \(Z:=\Omega \times \mathbb {K}\) is compact, any sequence \(\{z^{n}\}_{n}\) has a subsequence converging to some \(z^{*}\in Z \) , implying that any sequence of probability measures, \(\{\pi ^{*}(\cdot |z^{n})\}_{n}\), has a subsequence, \(\{\pi ^{*}(\cdot |z^{n_{k}})\}_{k}\), setwise converging and therefore weak star converging to \(\pi ^{*}(\cdot |z^{*})\). Moreover, by Proposition 1.4.2(a)(ii) in Hernandez-Lerma and Laserre [17], because \(\Pi :=\{\pi ^{*}(\cdot |z):z\in \Omega \times \mathbb {K}\}\) is weak star compact, the equilibrium Markov transition kernel, \(z\longrightarrow \pi ^{*}(\mathrm{d}z^{\prime }|z)\), is uniformly countably additive on any closed subset C of \(\Omega \times \mathbb {K}\). Thus, the equilibrium Markov kernel, \(\pi ^{*}(\cdot |\cdot )\), governing the state-network process, \(Z_{t},\) is such that for any closed subset C of \(\Omega \times \mathbb {K}\),
for any sequence of states \(\{S_{n}\}\subset B_{\Omega }\times B_{\mathbb {K} } \) with \(S_{n}\downarrow \varnothing \).
We will denote by [A-1]\(^{*}\) our list of assumptions [A-1] but with [A-1](2) and [A-1](15)(iii) strengthened as above. Our next result, which follows directly from the arguments above, states that any equilibrium state-network process determined by the stationary Markov perfect equilibrium of a DSG of club network formation satisfying assumptions [A-1] \(^{*}\) satisfies the Tweedie conditions.
Theorem 2
(All DSGs satisfying [A-1] \(^{*}\) induce equilibrium processes satisfying the Tweedie conditions) Any state-network Markov transition,
determined by a stationary Markov perfect behavioral club network formation strategy,
of a discounted stochastic game satisfying assumptions [A-1] \(^{*}\) satisfies the Tweedie conditions. In particular, for \(V(z)=1\) for all \(z\in \Omega \times \mathbb {K}\), \(b=2\), and \(C=\Omega \times \mathbb {K}\) we have
and for any sequence of sets of states \(\{S_{n}\}_{n}\subset B_{\Omega }\times B_{\mathbb {K}}\) with \(S_{n}\downarrow \varnothing \)
In our DSG over time allocation networks discussed above, if we strengthen the stochastic continuity assumptions [A-1](15)(iii) to [A-1](15)(iii)\( ^{*}\) and if we assume that the state space, \(\Omega \), is compact, then the equilibrium state-network process induced by players’ bang-bang strategies will satisfy the Tweedie conditions. Moreover, if \(\Omega \) is compact and we assume that the set of conditional densities,
of \(q(\cdot |\omega ,G)\) with respect to \(\mu \) is such that the function
is \(\rho _{\Omega \times \mathbb {K}}\)-continuous (\(\rho _{\Omega \times \mathbb {K}}:=\rho _{\Omega }+h_{K}\)) in \(\omega \) and G a.e. \([\mu ]\) in \(\omega ^{\prime }\), then the Tweedie conditions will be satisfied in our DSG over time allocation networks.
8 Strategically Stable Club Networks and Coalition Structures
By Theorem 2, the equilibrium state-network transition, \(\pi ^{*}(\cdot |\cdot )\), governing the movements of the state-network process, \( \{Z_{t}^{*}\}_{t}\), through the space, \(\Omega \times \mathbb {K}\), satisfies the Tweedie conditions. As a consequence, as shown by [35], the space \(\Omega \times \mathbb {K}\) can be decomposed in a unique way into a transient set of state-network pairs and a finite collection of largest absorbing sets (i.e., basins of attraction given by Harris sets) consisting of state-network pairs that might persist in the long run (and will persist in the long run if there is only one basin of attraction) and that exhaust all the possibilities for what will happen in the future with regard to the path of the network formation process.Footnote 10
Restating Tweedie’s decomposition result for the strategically determined equilibrium state-network transition, \(\pi ^{*}(\cdot |\cdot )\), we have the following:
Theorem 3
[35, Theorem 2] Let \(\{Z_{t}^{*}\}_{t}\) be an equilibrium state-network Markov process governed by the transition kernel, \(\pi ^{*}(\mathrm{d}z^{\prime }|z)\), satisfying [T]. Then, \(\pi ^{*}(\mathrm{d}z^{\prime }|z)\) uniquely partitions the space \(\mathbb {Z}:=\Omega \times \mathbb {K}\) into a finite number of maximal Harris sets, \(H_{i}\subset \mathbb {Z}\), and a transient set, \(E\subset \mathbb {Z}\), and in particular,
The union of the Harris sets, \(H_{I}:=\cup _{i=1}^{I}H_{i}\), has the property that starting from any transient \(z\in E\), the state-network process \(Z_{t}^{*}\) reaches \(H_{I}\) (i.e., reaches some Harris set) in finite time with probability 1, and each Harris set, \(H_{i}\) has an ergodic probability measure, \(\lambda ^{i}\).
We denote the fact that starting from any transient state-network pair, \( z=(\omega ,G)\in E\), the state-network process, \(Z_{t}^{*}\), reaches \( H_{I}\) in finite time with probability 1 by writing, \(L(z,H_{I})=1\), where
denotes the first entry probability and where
is the first enter time.
Because we are concerned with stability, one of the questions we want to answer is for any given subset S of state-network pairs, \(z:=(\omega ,G)\), contained in a Harris set, \(H_{i}\), how often does the state-network process, \(Z_{t}^{*}\), visit state-network pairs in S? Let \(\eta _{S}:=\sum _{t=1}^{\infty }I_{S}(Z_{t}^{*}(z))\) denote the number of visits of the process, \(Z_{t}^{*}\), to the set of state-network pairs, S, where \(I_{S}(\cdot )\) is the indicator function of the set S. We say that a state-network set, \(S\subset \mathbb {Z}\), is a vio set (i.e., a set visited infinitely often) if the probability that \(Z_{t}^{*}\), leaving from z, visits S infinitely often is 1—a fact denoted by writing
where Q(z, S) denotes the vio set probability.
In order to answer the question of how many times the process visits various subsets of state-networks contained in a Harris set, \(H_{i}\), the following further refinement of our partition is useful. To begin, because each Harris set \(H_{i}\) is a largest absorbing set, if we restrict the process to the maximal Harris set, \(H_{i}\), giving us a sub-process, \( Z_{t}^{*H_{i}}:=Z_{t}^{*i}\), governed by the Markov kernel, \(\pi _{H_{i}}^{*}(\cdot |\cdot ):=\pi _{i}^{*}(\cdot |\cdot )\), then the process, restricted to \(H_{i}\), is \(\lambda ^{i}(\cdot )\)-irreducible (i.e., for all \(S\subset H_{i}\) with \(S\in B_{H_{i}}\) and \(\lambda ^{i}(S)>0\), \( L(z,S)>0\) for all \(z\in H_{i}\)). Because the process, \(Z_{t}^{*}\), generates a finite set of largest absorbing sets with each having an ergodic measure, it follows from Theorem 2.18 and Corollary 2.19 in [10] that \(Z_{t}^{*i}\), governed by the kernel, \(\pi _{i}^{*}(\cdot |\cdot )\), is a \(\lambda ^{i}(\cdot )\)-irreducible, T-process (see p. 124 in [24], and Definitions 2.1–2.4 in [10]). By Theorem 9.3.6, in [24], each Harris set \(H_{i}\) in expression (44) is positive Harris recurrent and by Theorem 9.3.5, in [24] can be further decomposed as,
where \(R_{i}\) is the set of topological Harris recurrent state-network pairs and \(E_{i}\) is topologically transient (i.e., \(z^{*i}\in E_{i}\) if and only if \(E_{z^{*i}}(\eta _{O_{z^{*i}}})<\infty \)).Footnote 11 Therefore, a state-network pair \(z^{*i}=(\omega ^{*i},G^{*i})\in R_{i}\) if and only if
(i.e., if and only if the process visits each state-network pair contained in any neighborhoods of \(z^{*i}\in R_{i}\subset H_{i}\) infinitely often).Footnote 12 Essentially, when the process, \(Z_{t}^{*}\), enters the maximal Harris set \(H_{i}\) it stays there for all future periods and becomes a \(\lambda ^{i}(\cdot )\) -irreducible, T-process, visiting the topological Harris recurrent state-network pairs, \(z^{*i}\in R_{i}\), infinitely often—passing through state-network pairs in \(E_{i}\) on its way to state-network pairs in \( R_{i}\). Thus, a refinement of the decomposition in Theorem 3 is given by
If the process begins at some state-network pair in E, then in finite time with probability 1 the process will leave E and enter one of the basins, \( H_{i}=R_{i}\cup E_{i}\), where it will remain, visiting each state-network pair in \(R_{i}\) infinitely often with probability 1, and if the process visits a state-network pair in \(E_{i}\) then with probability 1 it will leave that state-network pair in finite time never to return, perhaps visiting a different state-network pair in \(E_{i}\) and with probability 1 leaving that state-network pair in finite time never to return.
Because the equilibrium Markov transition, \(\pi ^{*}(\cdot |\cdot )\), governing the movements of the state-network process, \(Z_{t}^{*}\), is strongly stochastically continuous (i.e., because \(z^{n}\) converging to \( z^{*}\) implies that \(\pi ^{*}(\cdot |z^{n})\) converges setwise to \( \pi ^{*}(\cdot |z^{*})\)), \(\pi ^{*}(\cdot |\cdot )\) has the strong Feller property (e.g., see [24, Section 6.1]). Moreover, because, \(Z_{t}^{*i}\) restricted to \(H_{i}\) is a \(\lambda ^{i}(\cdot )\)-irreducible, T-process, governed by the Markov kernel, \(\pi _{i}^{*}(\cdot |\cdot )\), having the strong Feller property, it follows from Theorem 7.1(iii) in [34] that because the state-network space is compact (see [A-2]\(^{*}\)), the set of topologically Harris recurrent (THR) state-network pairs, \(R_{i}\) is closed (and hence compact) and \(E_{i}\) is open.
Each set of topological Harris recurrent state-network pairs, \(R_{i}\subset \mathbb {Z}:=\Omega \times \mathbb {K}\), determines a correspondence,
with range given by
For club network \(G\in \mathbb {R}_{i}\), there is at least one state \(\omega \) (and possibly many states) such that \((\omega ,G)\in R_{i}\). Because \(R_{i}\) is compact, \(\mathbb {R}_{i}\) is compact. Thus, for each \(0<\varepsilon <1\), there is a finite set of club network contained in \(\mathbb {R}_{i}\),
and a corresponding finite set of open balls,
covering \(\mathbb {R}_{i}\) such that any club network \(G\in \mathbb {R}_{i}\) is contained in an open ball of networks, \(B_{h_{K}}(\varepsilon ,G_{\varepsilon h}^{*i})\), with each network in \(B_{h_{K}}(\varepsilon ,G_{\varepsilon h}^{*i})\) being at \(h_{K}\)-distance less than \( \varepsilon \in (0,1)\) from network \(G_{\varepsilon h}^{*i}\) for some \( h=1,2,\ldots ,N_{\varepsilon }^{i}\). Given our prior observation concerning the properties of the Hausdorff metric on the hyperspace of club networks, we know that for \(\varepsilon \in (0,1)\), all club networks contained in the open ball \(B_{h_{K}}(\varepsilon ,G_{\varepsilon h}^{*i})\) have the same coalition structure as club network \(G_{\varepsilon h}^{*i}\). Thus, for club networks G and \(G^{\prime }\) contained in \(B_{h_{K}}(\varepsilon ,G_{\varepsilon h}^{*i})\), for \(\varepsilon \in (0,1)\),
where recall for each club-network pair, (c, G),
What conditions are sufficient to guarantee that the finite set of networks,
underlying the finite \(\varepsilon \)-covering,
have the same coalition structure? Consider the following definition.
Definition 3
(Chainable THR Sets) Let \(R_{i}\) be a compact topologically Harris recurrent (THR) set of state-network pairs with network part \(\mathbb {R}_{i}\). We say that \(\mathbb {R}_{i}\) is chainable if there is an \(\varepsilon \in (0,1)\) and a finite \(\varepsilon \) -covering
with underlying set of networks, \(\mathbb {G}_{\varepsilon }^{*i}:=\{G_{\varepsilon h}^{*i}:h=1,2,\ldots ,N_{\varepsilon }^{i}\}\subset \mathbb {R}_{i},\) such that for any two club network, G and \(G^{\prime }\) in \(\mathbb {R}_{i}\) there is a subset of the finite \(\varepsilon \)-covering, \(\{B_{h_{K}}(\varepsilon ,G_{\varepsilon h}^{*i}):h=1,2,\ldots ,N_{\varepsilon }^{i}\}\) given by
with \(G\in B_{h_{K}}(\varepsilon ,G_{\varepsilon h_{1}}^{*i})\) and \(G^{\prime }\in B_{h_{K}}(\varepsilon ,G_{\varepsilon h_{k^{\prime }}}^{*i})\) such that
Thus, for any two club networks in \(\mathbb {R}_{i}\), there is a \( \varepsilon \)-open ball path from G to \(G^{\prime }\) consisting of pairwise intersecting open balls. Because consecutive open balls are intersecting, that is, because,
all the networks contained in \(\cup _{k}B_{h_{K}}(\varepsilon ,G_{\varepsilon h_{k}}^{*i})\) have the same coalition structure. Because \(\mathbb {R}_{i}\) is chainable such an \(\varepsilon \)-open ball path with intersecting consecutive open balls can be found between any two club networks in \(\mathbb {R}_{i}\), implying that all the club networks in
have the same coalition structure.
We can distill the arguments and conclusions above in the following Theorem:
Theorem 4
(Topological Harris Recurrence and Coalitional Homogeneity) Let \(\{Z_{t}^{*}\}_{t}\) be an equilibrium state-network Markov process governed by the transition kernel, \(\pi ^{*}(\mathrm{d}z^{\prime }|z)\), satisfying [T] induced by the stationary Markov perfect equilibrium behavioral strategy profile, \(\sigma ^{*}(\cdot )\), of a discounted stochastic game of club network formation satisfying assumptions, [A-1]\(^{*}\). Let
be the decomposition of the space \(\mathbb {Z}:=\Omega \times \mathbb {K }\) generated by equilibrium state-network process, \(\{Z_{t}^{*}\}_{t}\) into a finite number of maximal Harris sets, \( H_{i}=R_{i}\cup E_{i}\subset \mathbb {Z}\), and a transient set, \(E\subset \mathbb {Z}\) with \(H_{i}=R_{i}\cup E_{i}\) being given by the union of an open topologically transient set \(E_{i}\) and a compact topologically Harris recurrent (THR) set, \(R_{i}\), with network part
.
If \(\mathbb {R}_{i}\) is chainable, then all club networks in \( \mathbb {R}_{i}\) have the same underlying coalition structure. Moreover, all club networks in any chainable subset \(\mathbb {C}_{i}\) of \(\mathbb {R}_{i}\) contain networks with the same underlying coalition structure.
Proof
If \(\mathbb {R}_{i}\) is chainable, then for any two club networks in \(\mathbb {R}_{i}\), there is a \(\varepsilon \)-open ball path from G to \(G^{\prime }\) consisting of pairwise intersecting open balls. Because consecutive open balls are intersecting, that is, because,
all the networks contained in \(\cup _{k}B_{h_{K}}(\varepsilon ,G_{\varepsilon h_{k}}^{*i})\) have the same coalition structure. Because \(\mathbb {R}_{i}\) is chainable such an \(\varepsilon \)-open ball path with intersecting consecutive open balls can be found between any two club networks in \(\mathbb {R}_{i}\), implying that all the club networks in
have the same coalition structure. Q.E.D. \(\square \)
9 Summary and Conclusions
If we strengthen the assumptions underlying our discounted stochastic game of club network formation (from [A-1] to [A-1]\(^{*}\)), then because the equilibrium state-network transition kernel, \(\pi ^{*}(\mathrm{d}z^{\prime }|z)\), induced by the SMPE strategy profile, \(\sigma ^{*}(\cdot )\), of the DSG satisfies the Tweedie conditions [T], we know by Theorem 2 in [35] and further results by [24] which build on [10, 34, 35], that the state-network space, \(\mathbb {Z}:=\Omega \times \mathbb {K}\), can be uniquely partitioned into a finite number of maximal Harris sets, \(H_{i}\subset \mathbb {Z}\), and a transient set, \(E\subset \mathbb {Z}\), and in particular, we know that
where \(H_{i}=R_{i}\cup E_{i}\subset \mathbb {Z}\), \(E_i\) being an open topologically transient set and \(R_i\) being a compact topologically Harris recurrent (THR) set with network part given by
These conclusions all flow from the discounted stochastic game of club network formation under assumptions [A-1]\(^{*}\). Our last result above tells us that if, in addition, the club network vio sets, \(\mathbb {R} _{i}\), are chainable, then all the club networks in any given vio set \(\mathbb {R}_{i}\) have the same coalition structure.
Change history
26 April 2023
A Correction to this paper has been published: https://doi.org/10.1007/s13235-022-00483-7
Notes
The paper by [2], is closely related to our paper—but differs in two important respects: (i) in Arnold and Wooders, players are allowed to join only one club, whereas here we allow players to have multiple club memberships, and (ii) in Arnold and Wooders, players are myopic, whereas here players are farsighted in the sense of discounted stochastic games.
vio stands for “visited infinitely often.”
The idea of a chainable set is borrowed from continuum theory [25]. A set of networks is chainable if there is a finite sequence of pairwise intersecting open balls of radius less than 1 forming an open ball path from one network to any other network in the vio set.
Under the discrete metric the distance between two nodes d and \(d^{\prime } \) in D is given by
$$\begin{aligned} \begin{array}{cc} \eta _{D}(d,d^{\prime }):= &{} \left\{ \begin{array}{cc} 1 &{} \text {if }d\ne d^{\prime } \\ 0 &{} \text {if }d=d^{\prime }. \end{array} \right. \end{array} \end{aligned}$$It is interesting to note that strategic network formation with club memberships being determined via bargaining with or voting by the club’s existing membership can be treated via a discounted stochastic game, but a game played over stationary semi-Markov behavioral strategies or a game in which the state space consists of network-coalition pairs. Here, we will focus on stationary Markov behavioral strategies—leaving for future research the semi-Markov case.
Recall that the support of (a regular Borel) probability measure, \(\sigma _{d}\in \Delta (\mathbb {K}^{d})\), is the unique closed subset, \(supp\sigma _{d}\), of \(\mathbb {K}^{d}\) such that \(\sigma _{d} (\mathbb {K}^{d}\backslash supp\sigma _{d})=0\), with the property that for any open set \(\mathbb {E} ^{d}\subset \mathbb {K}^{d}\) such that
$$\begin{aligned} \mathbb {E}^{d}\cap supp\sigma _{d}\ne \varnothing \end{aligned}$$\(\sigma _{d}(\mathbb {E}^{d}\cap supp\sigma _{d})>0\). Also, note that \(ca(\mathbb {K}^{d})\) is a locally convex Hausdorff topological vector space with \(\Delta (\mathbb {K}^{d})\) a convex, \(\rho _{w_{ca}^{*d}}\)-compact subset of \(ca(\mathbb {K}^{d})\).
\(\mathcal {L}_{R}^{\infty }\) is the Banach space of \(\mu \)-equivalence classes of \(\mu \)-essentially bounded functions, \(v:\Omega \longrightarrow R\) with norm
$$\begin{aligned} \begin{array}{c} \left\| v\right\| _{\infty }:=esssup[v]:=\inf \left\{ y\in R:\mu \{\omega :\left| v(\omega )\right| >y\}=0\right\} . \end{array} \end{aligned}$$The space of \(\mu \)-equivalence classes of functions \(\mathcal {L} _{R}^{\infty }\) is the separable norm dual of the space of \(\mu \) -equivalence classes of \(\mu \)-integrable functions, \(\mathcal {L}_{R}^{1}\). Because the Borel \(\sigma \)-field \(B_{\Omega }\) is countably generated, \( \mathcal {L}_{R}^{1}\), is separable. As a consequence, the subset of value function \(\mu \)-equivalence classes, \(\mathcal {L}_{Y_{d}}^{\infty }\), is a compact, convex, and metrizable subset of \(\mathcal {L}_{R}^{\infty }\) for the weak star topology.
In our club network formation model, we have assumed that the state space is a Polish space, \(\Omega ,\) equipped with the Borel \(\sigma \)-field, \( B_{\Omega }\), and a probability measure, \(\mu \), defined on \(B_{\Omega }\). Suppose that \(\mathcal {G}\) is a sub-\(\sigma \)-field of \(B_{\Omega }\). Denote by \(\mu ^{\mathcal {G}}(\cdot )\) a regular \(\mathcal {G}\)-conditional probability given sub-\(\sigma \)-field \(\mathcal {G}\). Following [13], \(A\in B_{\Omega }\) is \(\mathcal {G}\)-atom if \(\mu (A)>0\) and for any \(B\in B_{\Omega }\) such that \(B\subset A\)
$$\begin{aligned} \mu \left\{ \omega \in \Omega :0<\mu ^{\mathcal {G}}(B)(\omega )<\mu ^{ \mathcal {G}}(A)(\omega )\right\} =0. \end{aligned}$$Recall that a set of \(H\in B_{\Omega }\times B_{\mathbb {K}}\) is an absorbing set with respect to the transition kernel, \(\pi ^{*}(\cdot |\cdot )\), if \(\pi ^{*}(H|z)=1\) for all \(z\in H.\)
Recall that \(\eta _{O_{z^{*i}}}:=\sum _{t=1}^{\infty }I_{O_{z^{*i}}}(Z_{t}^{*}(z))\). \(E_{z^{*i}}(\eta _{O_{z^{*i}}})\) denotes the expected number of times the process visits \(O_{z^{*i}}\) starting from \(z^{*i}\) (see expression 75 below).
Or equivalently, if and only if, \(L(z^{*i},O_{z^{*i}})\) for all neighborhoods \(O_{z^{*i}}\) of \(z^{*i}\).
The collection,
$$\begin{aligned} \left\{ u(\omega ,v,\cdot ):v\in \mathcal {L}_{Y}^{\infty }\right\} , \end{aligned}$$is uniformly equicontinuous if for any \(\varepsilon >0\) there is a \(\delta >0 \) such that for any G and \(G^{\prime }\) in \(\Phi (\omega )\) with \( h_{K}(G,G^{\prime })<\delta \),
$$\begin{aligned} \rho _{Y}(u(\omega ,v,G),u(\omega ,v,G^{\prime }))<\varepsilon , \end{aligned}$$for all \(v\in \mathcal {L}_{Y}^{\infty }\).
References
Aliprantis CD, Border KC (2006) Infinite dimensional analysis: a Hitchhiker’s guide, 3rd edn. Springer, Berlin
Arnold T, Wooders M (2015) Dynamic club formation with coordination. J Dyn Games 2:341–361
Ash R (1972) Probability and real analysis. Wiley, New York
Balder EJ (1997) Remarks on games with additively coupled payoffs. Econ Theory 9:161–167
Balder EJ (2000) Lectures on Young measure theory and its application in economics. Rend Instit Mat Univ Trieste XXXI Suppl 1:1–69
Bertsekas DP, Shreve SE (1978) Stochastic optimal control: the discrete time case. Academic Press, New York
Billingsley P (1968) Convergence of probability measures. Wiley, New York
Billingsley P (1986) Probability and measure, 2nd edn. Wiley, New York
Blackwell D (1965) Discounted Dynamic Programming. Ann Math Stat 36:226–235
Costa OLV, Dufour F (2005) On the Ergodic Decomposition for a Class of Markov Chains. Stoch Process Appl 115:401–415
Delbaen F (1974) Continuity of expected utility. In: Dreze J, Mirrlees J, Malinvaud E (eds) Allocation under uncertainty: equilibrium and optimality. Macmillan, London
Duggan J (2012) Noisy Stochastic Games. Econometrica 80:2017–2045
Dynkin EB, Evstigneev IV (1976) Regular conditional expectations of correspondences. Theory Probab Appl 21:325–338
Elliott RJ, Kalton NJ, Markus L (1973) Saddle points for linear differential games. SIAM J Control 11:100–112
Fu J, Page F (2022) Discounted stochastic games, the 3M property and stationary Markov perfect equilibria. Discussion Paper 119. Systemic Risk Centre, London School of Economics
He W, Sun Y (2017) Stationary Markov perfect equilibria in discounted stochastic games. J Econ Theory 169:35–61
Hernández-Lerma O, Lasserre JB (2003) Markov chains and invariant probabilities. Birkhäuser Verlag, Basel
Himmelberg CJ (1975) Measurable relations. Fundam Math 87(1):53–72
Himmelberg CJ, Parthasarathy T, VanVleck FS (1976) Optimal plans for dynamic programming problems. Math Oper Res 1:390–394
Komlos J (1967) A generalization of a problem of Steinhaus. Acta Math Acad Sci Hung 18:217–229
Kucia A, Nowak A (2000) Approximate caratheodory selections. Bull Polish Acad Sci Math 48:81–87
Levy Y (2013) A discounted stochastic game with no stationary Nash equilibrium: two examples. Econometrica 81:1973–2007
Levy Y, McLennan A (2015) Corrigendum to: “Discounted stochastic games with no stationary Nash equilibrium: two examples’’. Econometrica 83:1237–1252
Meyn SP, Tweedie RL (2009) Markov chains and stochastic stability, 2nd edn. Springer, London
Nadler SB Jr (1992) Continuum theory: an introduction. Marcel Dekker, New York
Page F (1992) Mechanism design for general screening problems with moral hazard. Econ Theory 2:265–281
Page F (2015) A fixed point theorem for measurable-selection-valued correspondences arising in game theory. Discussion Paper 43. Systemic Risk Centre, London School of Economics
Page F (2016) Stationary Markov perfect equilibria for approximable discounted stochastic games. Discussion Paper 60. Systemic Risk Centre, London School of Economics
Page F, Wooders M, Kamat S (2005) Networks and farsighted stability. J Econ Theory 120:257–269
Page F, Wooders M (2007) Networks and clubs. J Econ Behav Organ 64:406–425
Page F, Wooders M (2009) Strategic basins of attraction, the path dominance core, and network formation games. Games Econ Behav 66:462–487
Page F, Wooders M (2010) Club networks with multiple memberships and noncooperative stability. Games Econ Behav 66:462–487
Solan E (1998) Discounted stochastic games. Math Oper Res 23:1010–1021
Tuominen P, Tweedie RL (1979) Markov chains with continuous components. Proc Lond Math Soc 38:89–114
Tweedie RL (2001) Drift conditions and invariant measures for Markov chains. Stoch Process Appl 92:345–354
Wooders M, Page F (2008) Coalitions. In: Durlauf Steven, Blume Lawrence (eds) The new Palgrave dictionary of economics, 2nd edn. Macmillan, London
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article is part of the topical collection “Group Formation and Farsightedness” edited by Francis Bloch, Ana Mauleon and Vincent Vannetelbosch.
The original online version of this article was revised: The textual corrections have been updated.
Jing Fu thanks JSPS KAKENHI for financial support under grant number 19K13662. Frank Page acknowledges financial support from the Systemic Risk Centre (under ESRC grant numbers ES/K002309/1 and ES/R009724/1).
Appendices
Appendix 1: The Hausdorff Metric on the Hyperspace of Layered Networks
To begin, equip the set of c-connections, \(K_{c}:=\cup _{d}K_{dc}:=\cup _{d}A(d,c)\times (\{d\}\times \{c\})\), with the sum metric,
Thus, the distance between c-connections, (a, (d, c)) and \((a^{\prime },(d^{\prime },c))\), is
We will equip each hyperspace of c-layers, \(2^{K_{c}}\), with the Hausdorff metric induced by the metric, \(\rho _{K_{c}}\), on the set of c -connections. In defining the Hausdorff metric \(h_{K_{c}}\) on \(2^{K_{c}}\), we must allow for empty c-layers. For nonempty c-layer \(G_{c}^{\prime }\in 2^{K_{c}}\) and connection \((a,(d,c))\in K_{c}\), we define the distance from (a, (d, c)) to the nonempty c-layer \(G_{c}^{\prime }\) to be
and for c-layers \(G_{c}\) \(\ne \varnothing ,\) \(G_{c}^{\prime }\) \(\ne \varnothing \), we define the excess of \(G_{c}\) over \(G_{c}^{\prime }\) to be
The Hausdorff distance between nonempty c-layers, \(G_{c}\) and \( G_{c}^{\prime }\) is given by
while
The diameter, \(diam(K_{c})\), of the set of c-connection \(K_{c},\)is given by
Thus, the Hausdorff metric on the hyperspace of c-layers, \(2^{K_{c}}\), is given by
Given that the basic building block of a club network array is the hyperspace \(\mathbb {K}_{dc}\) of feasible c-connections belonging to player d, with an underlying set of connections given by \(K_{dc}:=A(d,c)\times (\{d\}\times \{c\})\), and given that each player can take, at most one action in each club, we see that the Hausdorff metric \(h_{K_{dc}}\) on \( \mathbb {K}_{dc}\) reduces to
Recall that \(G_{dc}\in \mathbb {K}_{dc}\) if and only if
The Hausdorff metric on the hyperspace of feasible club networks, \(\mathbb {K} =(\mathbb {K}_{dc})_{dc}\), is given by
for \(G:=(G_{c})_{c\in C}\) and \(G^{\prime }:=(G_{c}^{\prime })_{c\in C}\) in \( \prod \limits _{c\in C}\mathbb {K}_{c}\). Because \((\mathbb {K}_{c},\rho _{K_{c}})\) is a compact metric space, we have by Proposition C.2 in Bertsekas and Shreve [6] that \((2^{K_{c}},h_{K_{c}})\) is a compact metric space of c-layers, and because \(\mathbb {K}_{dc}\) is an \(h_{K_{dc}}\)-closed subset of the \( h_{K_{dc}}\)-compact subset of \(2^{K_{dc}}\), \(\mathbb {K}_{dc}\) is \(h_{K_{dc}}\)-compact—implying that \((\mathbb {K},h_{K})\), given by
is a compact metric space.
Appendix 2: Technical Notes on Assumptions [A-1]
1.1 Weak Star Convergence of Random Player Club Networks
Under the metric, \(\rho _{w_{ca}^{*d}}\), compatible with the relative \( w_{ca}^{*d}\)-topology on \(\Delta (\mathbb {K}^{d})\), we have
where \(\mathcal {C}(\mathbb {K}^{d})\) is the Banach space of continuous functions defined on the compact metric space, \(\mathbb {K}^{d}\), with the sup norm. In fact, there exists a countable subcollection of continuous functions, \(\mathcal {C}^{0}(\mathbb {K}^{d})\), such that
for all \(c_{i}(\cdot )\in \mathcal {C}^{0}(\mathbb {K}^{d})\) (e.g., see Aliprantis and Border [1, Chapter 15]).
By Theorem 3.2 in [7], we know that
Thus, a sequence of behavioral action profiles \(w_{ca}^{*}\)-converges to a particular behavioral action profile if and only if each player’s sequence of behavioral actions \(w_{ca}^{*d}\)-converges to a particular behavioral action such that the \(w_{ca}^{*}\)-limit of behavioral action profiles is equal to the product of the \(w_{ca}^{*d}\)-limits of players’ behavioral action sequences. Unfortunately, the mapping \((\sigma _{d}(\cdot ),\sigma _{-d}(\cdot ))\longrightarrow \pi ((\sigma _{d}(\cdot ),\sigma _{-d}(\cdot )))\) is not jointly \(\rho _{W_{ca}^{*d}}\)-continuous. A good example of the failure of joint \(\rho _{W_{ca}^{*d}}\)-continuity can be found in [14], Example 3.16.
1.2 Weak Star and K-Convergence of Value Functions
A sequence, \(\{v^{n}\}_{n}\subset \) \(\mathcal {L}_{Y}^{\infty }\), converges weak star to \(v^{*}=(v_{1}^{*}(\cdot ),\ldots ,v_{m}^{*}(\cdot ))\in \mathcal {L}_{Y}^{\infty }\), denoted by \(v^{n}\underset{\rho _{w^{*}}}{\longrightarrow }v^{*}\), if and only if
for all \(l(\cdot )\in \mathcal {L}_{R^{m}}^{1}\).
A sequence, \(\{v^{n}\}_{n}\subset \) \(\mathcal {L}_{Y}^{\infty }\), K -converges (i.e., Komlos convergence—Komlos [20]) to \(\widehat{v}\in \mathcal {L}_{Y}^{\infty }\), denoted by \(v^{n}\underset{K}{\longrightarrow } \widehat{v}\), if and only if every subsequence, \(\{v^{n_{k}}(\cdot )\}_{k}\), \(of\ \{v^{n}(\cdot )\}_{n}\ \)has an arithmetic mean sequence,\(\ \{\widehat{v }^{n_{k}}(\cdot )\}_{k}\), where
such that
The relationship between \(w^{*}\)-convergence and K-convergence is summarized via the following results from Balder [5]: For every sequence of value functions, \(\{v^{n}\}_{n}\subset \mathcal {L}_{Y}^{\infty }\), and \( \widehat{v}\in \mathcal {L}_{Y}^{\infty }\) the following statements are true:
For any sequence of value function profiles, \(\{v^{n}\}_{n},\) in \( \mathcal {L}_{Y}^{\infty }\) it is automatic that
Thus, by the classical Komlos Theorem [20], any such sequence, \( \{v^{n}\}_{n}\), has a subsequence, \(\{v^{n_{k}}\}_{k}\) that K-converges to some K-limit, \(\widehat{v}\in \mathcal {L}_{Y}^{\infty }\).
1.3 Strong Stochastic Continuity of the Law of Motion
Under the stochastic continuity assumptions made above, [A-1](14), we have by Scheffee’s Theorem (see [8, Theorem 16.11]) that for each \( \omega \in \Omega \),
for any sequence of networks \(\{G^{n}\}_{n}\) in \(\Phi (\omega )\) converging to network \(G^{*}\in \Phi (\omega )\) (i.e., for each \(\omega \in \Omega \) the conditional density mapping, \(G\longrightarrow h(\cdot |\omega ,G)\), is continuous in \(L_{1}\) norm with respect to G). Thus, by Scheffee’s Theorem the \(L_{1}\) norm continuity of \(G\longrightarrow h(\cdot |\omega ,G)\) with respect to network G in each state \(\omega \) is equivalent to the continuity of \(G\longrightarrow q(E|\omega ,G)\) in each state \(\omega \) with respect to network G uniformly in \(E\in B_{\Omega }\) (i.e., for each \( \omega \in \Omega \), \(q(E|\omega ,\cdot )\) is continuous in G, uniformly with respect to \(E\in B_{\Omega }\)).
1.4 Convergence and Continuity
Under assumptions [A-1], for each \((\omega ,v)\) each player’s payoff function,
is jointly continuous in \(G=(G_{1},\ldots ,G_{m})\), and for any sequence of value function-network pairs, \(\{(v^{n},G^{n})\}_{n}\), if \(v^{n}\underset{ \rho _{w^{*}}}{\longrightarrow }v^{*}\) and \(G^{n}\underset{h_{K}}{ \longrightarrow }G^{*}\) then for each \(\omega \),
(i.e., \(u(\omega ,\cdot ,\cdot )\) is jointly continuous in (v, G)). Moreover, for each \(\omega \), the v-parameterized collection of m-tuples of integrands given by
where (see expression 68),
is uniformly equicontinuous (see Solan [33, Lemma 3.6]).Footnote 13 Thus, the Y-valued players’ payoff function, \(u(\cdot ,\cdot ,\cdot )\), is a Caratheodory function: \( \rho _{w^{*}\times h_{K}}\)-continuous in (v, G) for each \(\omega \), and \((B_{\Omega },B_{Y})\)-measurable in \(\omega \) on \(\Omega \) for each (v, G).
Appendix 3: Elementary Properties of Markov Transitions
Let \(\{W_{t}\}\) be the Markov process governed by the transition kernel, \( p(\mathrm{d}\omega ^{\prime }|\omega )\), where for each \(t=0,1,2,3,\ldots \), we have for each \(\omega \in \Omega \), \(W_{t}(\omega )=\omega _{t}\). Recall that
1.1 Hitting and Return Times
The number of visits of the process, \(W_{t}(\cdot )\), to the set of states B is given by \(\eta _{B}:=\sum _{t=1}^{\infty }I_{B}(W_{t}(\omega ))\). Let \( \varphi \) be a nontrivial, \(\sigma \)-finite measure on \(B_{\Omega }\).
We say that the process, \(W_{t}(\cdot )\), is \(\varphi \)-irreducible if for all \(B\in B_{\Omega }\) such that \(\varphi (B)>0\),
If the process, \(W_{t}(\cdot )\), is \(\varphi \)-irreducible, then we know that there exists a (maximal) dominating measure, \(\psi \), such that \( W_{t}(\cdot )\) is also \(\psi \)-irreducible (and see Proposition 4.2.2 in [24]).
Let
Return time probabilities and recurrent time probabilities are given by
The set \(B\in B_{\Omega }\) is Harris recurrent if \(P_{\omega }(\eta _{B}=\infty )=1\) for all \(\omega \in B\). The Markov process, \(W_{t}(\cdot )\) , is Harris recurrent if it is \(\psi \)-irreducible and every \(B\in B_{\Omega }\) such that \(\psi (B)>0\) is Harris recurrent. Thus a set B is Harris recurrent if when the Markov process \(W_{t}(\cdot )\) starts at \(\omega \in B\) , it returns to B infinitely many times, except when the process starts at any state contained in a set of initial states having probability zero—thus, \(P_{\omega }(\cdot )\)-almost surely. In fact, for any Markov process \( W_{t}(\cdot )\) that is Harris recurrent, \(P_{\omega }(\eta _{B}=\infty )=1\) for all \(\omega \in \Omega \) and for all \(B\in B_{\Omega }\) such that \(\psi (B)>0.\)
Summarizing for all \(B\in B_{\Omega }\) such that \(\psi (B)>0\)
1.2 Occupation Times
Given Markov process, \(W_{t}(\cdot )\), with Markov transition, \(p(\cdot |\cdot )\), the n-step occupation measure is given by
the pathwise occupation measure is given by
Thus,
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Fu, J., Page, F. & Zigrand, JP. Layered Networks, Equilibrium Dynamics, and Stable Coalitions. Dyn Games Appl 13, 636–668 (2023). https://doi.org/10.1007/s13235-022-00478-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13235-022-00478-4
Keywords
- Club networks
- Stable coalition structures
- Harris recurrent sets
- Topological Harris recurrent sets
- Basins of attraction
- Discounted stochastic games
- Stationary Markov perfect equilibria