Layered Networks, Equilibrium Dynamics, and Stable Coalitions

An important aspect of network dynamics that has been missing from our understanding of network dynamics in various applied settings is the influence of strategic behavior in determining equilibrium network dynamics. Our main objective hear to say what we can regarding the emergence of stable club networks—and therefore, stable coalition structures—based on the stability properties of strategically determined equilibrium network formation dynamics. Because club networks are layered networks, our work here can be thought of as a first work on the strategic dynamics of layered networks. In addition to constructing a discounted stochastic game model (i.e., a DSG model) of club network formation, (1) we show that our DSG of network formation possesses a stationary Markov perfect equilibrium in players’ membership-action strategies; (2) we identify the assumptions on primitives which ensure that the induced equilibrium Markov process of layered club network formation satisfies the Tweedie Stability Conditions (Tweedie in Stoch Process Appl 92:345–354, 2001) and (3) we show that, as a consequence, the equilibrium Markov network formation process generates a unique decomposition of the set of state-network pairs into a transient set together with finitely many basins of attraction. Moreover, we show that if there is a basin containing a vio set (a visited infinitely often set) of club networks sufficiently close together, then the coalition structures across club networks in the vio set will be the same (i.e., closeness across networks in a vio set leads to invariance in coalition structure across networks in a vio set).

strategies are bang-bang. Thus, rather than diversifying their club time across several clubs, each player in each state spends all club time in one and only one club. Moreover, we show that all that is required for the equilibrium time allocation process induced by players' bangbang time allocation strategies to satisfy the Tweedie conditions is for the state space to be compact and the conditional densities over coming states to be continuous in the current state and time allocation network for almost all coming states (i.e., except potential coming states having probability zero of occurring).

Layered Club Networks
We begin with a formal definition of club networks and a discussion of their properties. The discussion here is based, in part, on prior work by the second author with M. Wooders (see [29][30][31][32]36]). 4 Multiple membership club networks, as defined in [32], are examples of layered networks in which connections between layers is brought about by overlapping club memberships. In a club network where each player is the member of one and only one club, the induced club membership structure partitions the set of active players, making each club layer isolated-having no connections, via overlapping memberships, to other layers in the network.
In the club network model, we construct here the feasible action sets available to the players who are active in a particular club layer are subsets of a compact metric space of actions specific to that club layer-and these club specific action spaces can differ across layers. In [32], the underlying metric space of player actions-whose subsets form the various player feasible sets-is the same across club layers. Here, the heterogeneity of player action sets across club layers makes defining a metric to measure the distance between club networks a much more delicate task-but we do accomplish this, thereby providing us with a compact metric hyperspace of club networks in which to carry out our game theoretic analysis of the emergence of equilibrium layered club network dynamics.
Finally, here unlike in [29][30][31][32], the game of network formation is dynamic with the equilibrium network dynamics being determined by the law of motion and the stationary Markov perfect equilibrium in behavioral network formation strategies which emerge from the game of network formation.
We begin by defining connections, layers and networks.

Connections
[N-1] Assume the following: (1) D is a finite set of n players equipped with the discrete metric η D , having typical element d. 5 (2) C is a finite set of m clubs equipped with discrete metric η C having typical element c.
(3) A c is a compact metric space of arc types, equipped with metric ρ c , having typical element a. 4 It is important to note that in [29][30][31][32]36], the network formation games being considered are not dynamic. 5 Under the discrete metric the distance between two nodes d and d in D is given by specifying for each club, the set of players who can join club c by indicating a nonempty feasible set of arcs club c will allow player d to use in connecting to club c.

(4) 2 A c is the collection of all closed subsets of A c (including the empty set). (5) (d, c) −→ A(d, c) ∈ 2 A c is the feasible arc correspondence, with A(d, c) ⊂ A c for all c , with c-domain given by
A connection is an ordered pair, (a, (d, c)), consisting of an arc label, a, indicating player d s action (arc) choice and a player-club pair, (d, c), indicating player d s club membership choice c. The action choice made by a particular player d must be feasible for the club player d has chosen, a requirement satisfied provided a ∈ A(d, c) ⊂ A c .
We will refer to a player-club pair, (d, c), as a preconnection. A preconnection, (d, c), acquires the status of a connection only when player d has chosen a feasible arc and (a, (d, c)) becomes part of a network. For preconnection, (d, c), the set of all feasible connections is given by The collection, {K dc : (d, c) ∈ D×C}, contains the basic building block of layered networks. Let 2 K dc denote the collection of all closed subsets of K dc (including the empty set), and let K dc ⊂ 2 K dc be the collection of subsets of K dc of size at most 1.
[N-2] Assume the following: (1) Each player, in considering whether or not to connect to a particular club either does not connect or connects in one and only one way. (2) Each player or group of players can move freely and unilaterally from one club to another via feasible connections. Thus, a player can drop his membership in any given club and, if allowed by the feasible arc correspondence, join any other club without bargaining and without seeking the permission of any player or group of players. 6

Layered Networks
If G cd ∈ K dc , then The feasible set of club networks, K := (K dc ) dc , is given by an n × m array of feasible player-club connections (see, 1), where K c := (K dc ) d is the feasible set of all possible clayers (or club c layers), and where K d := (K dc ) c is the feasible set of all possible d-layers (or player d layers). In array form, we have Formally, we have the following definition of a feasible club network, G ∈ K := (K dc ) dc , as an n × m array of feasible player-club connections.

Definition 1 (Feasible Club Networks, c-Layers, d-Layers, and Connection Arrays)
A feasible club network G ∈ K is an n × m array of feasible player-club connections given by where for each player-club pair, (d, c), is player d s part of c-layer, G c ∈ K c , given by the cth column in the club network, G, above, and where G dc ∈ K dc if and only if If we agree to the notational convention that G dc := a dc then club networks in K = (K dc ) dc can be given a reduced form array representation-as an array of arc types (without loss of information)-as follows: A club network, G ∈ K implicitly determines an arc selection, with domain given by

Coalition Structure
Each feasible club network, with its implied arc selection determines a coalition structure. In particular, given arc selection G(·, ·) determined by club network, G ∈ K, each club c has a membership coalition given by the domain of the mapping d −→ G(d, c) for fixed c. In particular, club c has membership coalition in network G, denoted by, S cG , and given by where 2 D is the collection of all subsets of D, including the empty set. Thus, the coalition structure determined by club network G ∈ K is given by In a club network, connections between layers are made through overlapping club memberships. Without this, each club layer is isolated. For example, in layered club network, G ∈ K, if S cG ∩ S c G = ∅, then each player d ∈ S cG ∩ S c G is a member of club c as well as a member of club c . In this way, club layers G c and G c are connected in network G. Note that if in club network, G, players are members of one and only one club then if the network has multiple nonempty layers, there are no connections between these layers-each layer is isolated precisely because there are no overlapping club memberships. In this case, the coalition structure induced by club network G given by is a partition of the active club members-and players are siloed by their club memberships.

Measuring the Distance Between Club Networks
In order to analyze the co-evolution of strategic behavior, club network structure and equilibrium dynamics, we require a topology for the space of club networks that is simultaneously coarse enough to guarantee compactness of the set of networks and fine enough to discriminate between differences across networks that are due to differences in the ways nodes are connected (via differing arc types) versus differences across networks that are due to the complete absence of connections. We resolve this topological dilemma by equipping the space club networks, K, with the Hausdorff metric h K -making the space of feasible club network connection arrays, a compact metric space (see Appendix 1 on the Hausdorff metric on the hyperspace of layered networks). It is easy to show that if the Hausdorff distance between any pair club networks G and G is less than ε ∈ (0, 1), then the networks can differ only in the ways a given set of player-club pairs are connected-and not in the set of player-club pairs that are connected. In particular, if for networks G and G , h K (G, G ) < ε < 1, then (a, (d, c)) ∈ G if and only if (a , (d, c)) ∈ G for arcs a and a with ρ A c (a, a ) < ε. Thus, if two club networks are at h K -distance ε < 1, then both club networks G and G have the same coalition structures, i.e., Such closeness will often occur and can only persist in network vio sets (sets of networks visited infinitely often by the equilibrium stochastic process of network formation) i.e., sets belonging to basins of attraction generated by the equilibrium dynamics governing the movements of club networks-all discussed in detail below. We will equip the hyperspace of feasible club network connection arrays, K = (K dc ) dc , with the Hausdorff metric given by if d is not a member of club c, and where, due to the single arc connection rule, But now we add a further condition given by the time allocation constraint. We will require that the d-layers, where D(G(d, ·)) is the club domain of player d-the clubs to which d belongs. The feasible set of club networks is, Thus, in a time allocation network, players freely and noncooperatively choose which clubs they want to join as well as their levels of participation in each club they join. Their levels of participation in each club are then expressed as fractions of their total club time-and these fractions must sum to one. Following the notational convention that the connections array in reduced form for network G in Again following the notational convention (8) and writing out the connections long hand, the connections in the d-layers are given by while the connections in the c-layers are given by We note that the time allocation club networks in K T A and in general the club networks in K satisfy assumptions [N-1]. We will assume for the remainder of the paper that [N-2] holds. We will return to the example above later in the paper.

Discounted Stochastic Games of Club Network Formation
In order to address the questions of whether or not and under what conditions the strategic formation of club networks will lead to the emergence of dynamically stable coalition structures, we will show that our discounted stochastic game (DSG) of club network formation possesses Nash equilibria in stationary Markov perfect behavioral club network formation strategies. It is the players' equilibrium behavioral network formation strategies which determine the equilibrium dynamics of club network formation. By identifying conditions under which stationary Markov perfect equilibria (SMPE) exist in such behavioral strategies and by showing that the resulting equilibrium state and club network dynamics are stable, we will be able to establish the conditions under which stable coalition structures will emerge and persist in the form of stable club networks. In this section, we will construct a DSG model of club network formation and show that our model possesses SMPE in behavioral club network formation strategies. The SMPE existence problem in the setting considered here (with uncountable states and compact metric action spaces) is quite difficult and its solution-and counterexamples-are of independent interest (for details see [15,22,23,27,28]).
An m-player, nonzero sum, discounted stochastic game, DSG, over the product space of probability measures over player club networks (i.e., the convex, compact metric space of behavioral actions), is given by the following primitives: where is the state space, B is the Borel σ -field of events, and μ is a probability measure. For each player d, K d is the set of all possible player club networks available to player d, while ( d (ω)) is the convex, compact feasible set of behavioral action available to player d in state ω . A feasible behavioral action available to player d in state ω is a probability is player d s payoff function in state ω given valuations (or prices) v d , and q(·|ω, ·) is the law of motion in state ω. If players holding value function profile v = (v 1 . . . , v n ) choose feasible profile of behavioral actions, in state ω, then the next state ω is chosen in accordance with probability measure q(·|ω, G) ∈ ( ) and player d s stage payoff is given by is the product probability measure representing the random club network determined by the n-tuple of random player club networks, (σ 1 (dG 1 ), ..., σ n (dG n )), chosen by the players.
We will denote by,

Primitives and Assumptions
We will maintain the following assumptions throughout. Label these (i.e., assumptions (1) is player d s club network and where K d := K d1 × · · · × K dc × · · · × K dm is a compact metric space of feasible player club networks with typical element, G d , equipped with metric, h K d := c h K dc , and K is equipped with the sum metric,  7 We will equip (K d ) with a metric, ρ w * d ca , compatible with the relative w * d ca -topology on (K d ) inherited from ca(K d ) and we will refer to σ d as player d s random player club network. 7 Recall that the support of (a regular Borel) probability measure, σ d ∈ (K d ), is the unique closed subset, , the space of player behavioral action profiles, σ := (σ 1 , . . . , σ n ), equipped with the sum metric, ρ w * ca := d ρ w * d ca , a metric compatible with the relative w * ca -product topology on [μ] , and the set of all μ-equivalence classes of measurable profiles (selection profiles), σ (·) = (σ 1 (·), . . . , σ n (·)), defined on such that in σ (ω) ∈ ( (ω)) : The space of μ-equivalence classes of functions L ∞ R is the separable norm dual of the space of μ -equivalence classes of μ-integrable functions, L 1 R . Because the Borel σ -field B is countably generated, L 1 R , is separable. As a consequence, the subset of value function μ-equivalence classes, L ∞ Y d , is a compact, convex, and metrizable subset of L ∞ R for the weak star topology.
is absolutely continuous with respect to μ(denoted Q( × K) << μ), (ii) for each E ∈ B , q(E|·, ·) is measurable on × K, (iii) the collection of probability density functions, of q(·|ω, G) with respect to μ is such that for each state ω, the function is h K -continuous in G a.e. [μ] in ω .
We note that the assumptions are above are the usual assumptions underlying discounted stochastic game models (see the Appendix 2 for technical notes on these assumptions).
There exists a pair,

a Stationary Markov Perfect Equilibrium (SMPE) in behavioral club network formation strategies supported by Bellman prices
For a formal proof, we refer the reader to [15]. Informally, the proof proceeds along the following lines: Let S ∞ (P(·, v)) be the set of all μ-equivalence classes of measurable functions, u ∈ L ∞ Y , such that u(ω) ∈ P(ω, v) a.e.
[μ]. It follows from Blackwell's Theorem [9], extended to DSGs, that our discounted stochastic game of club network formation (10) will have stationary Markov perfect equilibria in network formation strategies if and only if there exists a value function profile or equivalently if and only if the Nash payoff selection correspondence, has fixed points, i.e., has at least one value function profile, v * ∈ L ∞ Y , such that v * ∈ S ∞ (P v * ). This is a very difficult fixed point problem because the measurable selection valued correspondence, S ∞ (P (·) ), is neither convex valued nor closed value-nor of course is it upper semicontinuous. Until [15], no results were available to establish the existence of a fixed point for Nash payoff selection correspondences. Essentially what [15] show is that while the Nash payoff selection correspondence, v −→ S ∞ (P v ), is badly behaved, under assumptions [A-1], its underlying upper Caratheodory Nash payoff correspondence, P(·, ·), always contains an upper Caratheodory sub-correspondence, p(·, ·), that has εapproximate Caratheodory selections for all ε > 0, implying that the Nash payoff selection sub-correspondence, v −→ S ∞ ( p v ), has fixed points. Thus, [15] show that, in general, there exists a Nash payoff sub-correspondence, S ∞ ( p (·) ), having fixed points, i.e., that there exists . More fundamentally, [15] are able to show that the upper Caratheodory Nash payoff sub-correspondence, p(·, ·), has ε-approximate Caratheodory selections for all ε > 0, because under assumption [A-1] the underlying upper Caratheodory Nash correspondence, N (·, ·), always contains an upper Caratheodory Nash sub-correspondence, η(·, ·), that takes closed connected values. Thus, because the continuity of U d (ω, v d , ·) in behavioral actions σ for each player d = 1, 2, . . . , n together with the closed connectedness of η(ω, v) implies that is upper Caratheodory and interval valued for each player d = 1, 2, . . . , n. It then follows from Corollary 4.3 in [21] that each player's Nash payoff sub-correspondence, p d (·, ·), is Caratheodory approximable and this together with the unusual properties of Komlos convergence [20] of value functions allow us to show that there exists a value function profile, , In order to complete our informal argument for existence we need only note that by implicit measurable selection (e.g., see Theorem 7.1 in Himmelberg [18]), there exists a profile, σ * (·) = (σ * 1 (·), . . . , σ * n (·)), of a.e. measurable selections of ω −→ η(ω, v * ), such that for each player d = 1, 2, . . . , n, Thus, for each player d, the state-contingent prices given by value function, v * d (·) ∈ L ∞ Y d , incentivizes the continued choice by each player d, of behavioral strategy, σ * d (·), and we have for the value function-strategy profile pair, Thus, for value function-behavioral strategy profile pair, (v * , σ * (·)), we have for each player d = 1, 2, . . . , n and for ω a.e.
Thus here we have argued informally-and shown formally in [15]-that under the usual assumptions specifying a discounted stochastic game (in this case a club network formation DSG), while the DSG s Nash payoff selection is badly behaved, nonetheless, it naturally possesses (without additional assumptions) an underlying Nash payoff correspondence, P(·, ·), containing sub-correspondences, p(·, ·), which are Caratheodory approximable implying that its induced selection correspondence, v −→ S ∞ ( p v * ) has fixed points. [16], by making an additional assumption (that the DSG is G -nonatomic or has a coarser transition kernel) guarantee that DSG s Nash payoff selection correspondence has a convex valued sub-correspondence-and therefore an approximable sub-correspondence. 9 Moreover, [16] show that [12] accomplishes the same thing by assuming that the DSG has a noisy state. In the negative direction, [22,23] construct counterexamples showing that not all uncountablefinite DSGs have stationary Markov perfect equilibria. They accomplish this by constructing counterexamples in which the Nash correspondences are not approximable-which follows from the fact that in their counterexamples, there is an absence of fixed points. Because the club network formation DSG we analyze here is approximable, we avoid the Levy-McLennan counterexamples.

Example: DSGs over Time Allocation Networks
The one-shot game for a DSG satisfying assumptions [A-1] over time allocation networks, K T A , is given by In a (ω, v)-game each player d has preferences over the d-layer networks in K d given by, where G d = (G d1 , . . . , G dm ) and G dc = {(a dc , (d, c))} for c ∈ D(G(d, ·)), and G dc = ∅ for c / ∈ D(G(d, ·)). Rewriting these preferences as preferences over d-layer networks in reduced from, we have where with notational convention, a dc k ∈ (0, 1] ⇐⇒ G dc k = {(a dc k , (d, c k ))} and a dc k = 0 ⇐⇒ G dc k = ∅. We will assume that for each player d and for each (ω, v) ∈ × L ∞ Y , u d (ω, v d , ·) is affine on K T A . Thus, for time allocation networks, G and G in K T A , we have for any λ ∈ [0, 1] , where We will also assume that players' one-shot payoff functions, are measurable in ω for each (v d , a d , a −d ) and jointly continuous in (v d , a d , a −d ) for each ω.
By Theorem 1 (also see [15]), there exists a value function profile, v * ∈ L ∞ Y and an n-tuple of measurable function where by notational convention, and where, In fact, given the assumptions that players' one-shot payoff functions are affine over the convex set of time allocation networks (see 28 has a reduced form n × m array, (a * dc (ω)) dc , consisting of 0 s and 1 s. Moreover, in equilibrium, the state-contingent coalition of members of any club c k determined by the equilibrium club network G * (·) is given by

Equilibrium State-Network Dynamics and Stable Coalitions
Returning to our general model, under the profile of stationary Markov perfect equilibrium behavioral strategies σ * (·) := (σ * 1 (·), . . . , σ * n (·)), the equilibrium state and network formation process is given by, with underlying probability space such that in each state z = (ω, G) ∈ × K, and where for any measurable set E ∈ B × B K , E ω := {G ∈ K : (ω, G) ∈ E} (see 2.6.2, the Product Measure Theorem, in [3]). The movements of the process, Z * t ∞ t=0 , are governed by the equilibrium Markov transition kernel, For each (ω, G) ∈ × K, π * (d(ω , G )|ω, G) is given by the product probability measure, Thus, if the current state-club network is (ω, G) = (ω, G 1 , . . . , G n ), with d-layers (G d ) n d=1 , then the probability that the coming state and coming club network lie in the set, R × R ∈ B × B K , is given by Whether or not a stable club network emerges depends on the stability properties of equilibrium state-network dynamics underlying club network formation. Our main objective is to say what we can regarding the emergence of stable club structures-and hence stable coalition structures-based on the stability properties of the equilibrium state-network formation dynamics. We would argue that this is one of the main aspects of network dynamics that has been missing from our understanding of network dynamics in various applied settings-the influence of strategic behavior on network dynamics. Here, we present a first attempt. We will proceed as follows: First we state the Tweedie (Stability) Conditions [35] guaranteeing Markov stability. Second, we show that if we slightly strengthen assumptions [A-1] (2) and (15) in our discounted stochastic game model, then the stationary Markov perfect equilibrium (SMPE) of our discounted stochastic game of network formation gives rise to a Markov stable equilibrium state-network formation process satisfying the Tweedie Conditions. Finally, we summarize some of the main implications of the Tweedie Conditions and the resulting Markov stability properties of the equilibrium state-network formation process-such as the existence of finitely many basins of attraction and ergodic probabilities, and the implications of Markov stability for the emergence of stable coalition structures.
We begin by stating the Tweedie Conditions, and after strengthening assumptions [A-1] (2) and (15)(iii) in our discounted stochastic game model, showing that our strengthened discounted stochastic game model, in general, gives rise to an equilibrium state-network process satisfying the Tweedie conditions.

The Tweedie Conditions and Discounted Stochastic Games of Club Network Formation
We say that the Markov transition, π * (·|·), satisfies the Tweedie conditions, [T], provided, (2) (Uniform Countable Additivity) For any sequence of measurable sets, The intuition behind the Tweedie conditions can be described as follows: Rewriting the first condition (36)-the drift condition-we have for each z ∈ × K is the drift operator evaluated at z = (ω, G) ∈ × K. The drift operator measures the expected drift (or movement) of the process away from states in C starting from state z. Drift is measured by the value taken by the drift operator, V (·), at z. We see from the state-contingent bound on the right-hand side of (38) that if we are measuring expected drift from a state z in C and if b > 1, so that b − 1 > 0, then condition (36) will tolerate some drift away from C. However, if we are measuring expected drift from a state z not in C, then condition (36) requires that the expected drift be back toward C. The first Tweedie condition requires that the equilibrium process be such that there exist such a function, V (·) : × K −→ [0, +∞], finite valued at least at one point, a set C, and a bounding constant, b , such (36) is satisfied. The second Tweedie condition requires that the collection of probability measures, π * ( × K) := {π * (dz |z) : z ∈ × K}, determined by discounted stochastic game's strengthened law of motion and the equilibrium behavioral strategies of the players be uniformly countably additive (37)-guaranteeing therefore that the expected drift of the process is a continuous function of state-network pairs (i.e., that there are no discontinuous jumps in the expected magnitude of the drift).
Up until now, we have assumed that the set of states, , is Polish (complete, separable, metric). Our strengthening of [A-1](2) is to assume that [A-1](2) * In the probability space of states, ( , B , μ), , is a compact metric space.
Because the hyperspace of club networks, K, is compact, with the strengthening of [A-1](2) making compact, we have for any sequence, {(ω n , G n )} n , in × K that there is a convergent subsequence {(ω n k , G n k )} k with limit (ω * , G * ) ∈ × K. Given the arguments and observations immediately above this means that the set of probability measures, is weak star compact-because Z := ×K is compact, any sequence {z n } n has a subsequence converging to some z * ∈ Z , implying that any sequence of probability measures, {π * (·|z n )} n , has a subsequence, {π * (·|z n k )} k , setwise converging and therefore weak star converging to π * (·|z * ). Moreover, by Proposition 1.4.2(a)(ii) in Hernandez-Lerma and Laserre [17], because := {π * (·|z) : z ∈ ×K} is weak star compact, the equilibrium Markov transition kernel, z −→ π * (dz |z), is uniformly countably additive on any closed subset C of × K. Thus, the equilibrium Markov kernel, π * (·|·), governing the state-network process, Z t , is such that for any closed subset C of × K, for any sequence of states {S n } ⊂ B × B K with S n ↓ ∅. We will denote by [A-1] * our list of assumptions [A-1] but with [A-1](2) and [A-1](15)(iii) strengthened as above. Our next result, which follows directly from the arguments above, states that any equilibrium state-network process determined by the stationary Markov perfect equilibrium of a DSG of club network formation satisfying assumptions [A-1] * satisfies the Tweedie conditions.
determined by a stationary Markov perfect behavioral club network formation strategy, σ * (·) := (σ * 1 (·), . . . , σ * n (·)), of a discounted stochastic game satisfying assumptions [A-1] * satisfies the Tweedie conditions. In particular, for V (z) = 1 for all z ∈ × K, b = 2, and C = × K we have In our DSG over time allocation networks discussed above, if we strengthen the stochastic continuity assumptions [A-1](15)(iii) to [A-1](15)(iii) * and if we assume that the state space, , is compact, then the equilibrium state-network process induced by players' bang-bang strategies will satisfy the Tweedie conditions. Moreover, if is compact and we assume that the set of conditional densities, of q(·|ω, G) with respect to μ is such that the function is ρ ×K -continuous (ρ ×K := ρ + h K ) in ω and G a.e. [μ] in ω , then the Tweedie conditions will be satisfied in our DSG over time allocation networks.

Strategically Stable Club Networks and Coalition Structures
By Theorem 2, the equilibrium state-network transition, π * (·|·), governing the movements of the state-network process, {Z * t } t , through the space, × K, satisfies the Tweedie conditions. As a consequence, as shown by [35], the space × K can be decomposed in a unique way into a transient set of state-network pairs and a finite collection of largest absorbing sets (i.e., basins of attraction given by Harris sets) consisting of state-network pairs that might persist in [10]). By Theorem 9.3.6, in [24], each Harris set H i in expression (44) is positive Harris recurrent and by Theorem 9.3.5, in [24] can be further decomposed as, where R i is the set of topological Harris recurrent state-network pairs and E i is topologically transient (i.e., z * i ∈ E i if and only if E z * i (η O z * i ) < ∞). 11 Therefore, a state-network pair , if and only if the process visits each state-network pair contained in any neighborhoods of z * i ∈ R i ⊂ H i infinitely often). 12 Essentially, when the process, Z * t , enters the maximal Harris set H i it stays there for all future periods and becomes a λ i (·) -irreducible, T -process, visiting the topological Harris recurrent state-network pairs, z * i ∈ R i , infinitely oftenpassing through state-network pairs in E i on its way to state-network pairs in R i . Thus, a refinement of the decomposition in Theorem 3 is given by If the process begins at some state-network pair in E, then in finite time with probability 1 the process will leave E and enter one of the basins, H i = R i ∪ E i , where it will remain, visiting each state-network pair in R i infinitely often with probability 1, and if the process visits a state-network pair in E i then with probability 1 it will leave that state-network pair in finite time never to return, perhaps visiting a different state-network pair in E i and with probability 1 leaving that state-network pair in finite time never to return. Because the equilibrium Markov transition, π * (·|·), governing the movements of the statenetwork process, Z * t , is strongly stochastically continuous (i.e., because z n converging to z * implies that π * (·|z n ) converges setwise to π * (·|z * )), π * (·|·) has the strong Feller property (e.g., see [24, Section 6.1]). Moreover, because, Z * i t restricted to H i is a λ i (·)-irreducible, Tprocess, governed by the Markov kernel, π * i (·|·), having the strong Feller property, it follows from Theorem 7.1(iii) in [34] that because the state-network space is compact (see [A-2] * ), the set of topologically Harris recurrent (THR) state-network pairs, R i is closed (and hence compact) and E i is open.
Each set of topological Harris recurrent state-network pairs, R i ⊂ Z := ×K, determines a correspondence, with range given by For club network G ∈ R i , there is at least one state ω (and possibly many states) such that covering R i such that any club network G ∈ R i is contained in an open ball of networks, B h K (ε, G * i εh ), with each network in B h K (ε, G * i εh ) being at h K -distance less than ε ∈ (0, 1) from network G * i εh for some h = 1, 2, . . . , N i ε . Given our prior observation concerning the properties of the Hausdorff metric on the hyperspace of club networks, we know that for ε ∈ (0, 1), all club networks contained in the open ball B h K (ε, G * i εh ) have the same coalition structure as club network G * i εh . Thus, for club networks G and G contained in B h K (ε, G * i εh ), for ε ∈ (0, 1) What conditions are sufficient to guarantee that the finite set of networks, underlying the finite ε-covering, have the same coalition structure? Consider the following definition.

Definition 3 (Chainable THR Sets)
Let R i be a compact topologically Harris recurrent (THR) set of state-network pairs with network part R i . We say that R i is chainable if there is an ε ∈ (0, 1) and a finite ε -covering with underlying set of networks, G * i ε := {G * i εh : h = 1, 2, . . . , N i ε } ⊂ R i , such that for any two club network, G and G in R i there is a subset of the finite ε-covering, Thus, for any two club networks in R i , there is a ε-open ball path from G to G consisting of pairwise intersecting open balls. Because consecutive open balls are intersecting, that is, because, all the networks contained in ∪ k B h K (ε, G * i εh k ) have the same coalition structure. Because R i is chainable such an ε-open ball path with intersecting consecutive open balls can be found between any two club networks in R i , implying that all the club networks in have the same coalition structure. We can distill the arguments and conclusions above in the following Theorem: Theorem 4 (Topological Harris Recurrence and Coalitional Homogeneity) Let {Z * t } t be an equilibrium state-network Markov process governed by the transition kernel, π * (dz |z), satisfying [T] induced by the stationary Markov perfect equilibrium behavioral strategy profile, σ * (·), of a discounted stochastic game of club network formation satisfying assumptions, all the networks contained in ∪ k B h K (ε, G * i εh k ) have the same coalition structure. Because R i is chainable such an ε-open ball path with intersecting consecutive open balls can be found between any two club networks in R i , implying that all the club networks in have the same coalition structure. Q.E.D.

Summary and Conclusions
If we strengthen the assumptions underlying our discounted stochastic game of club network formation (from [A-1] to [A-1] * ), then because the equilibrium state-network transition kernel, π * (dz |z), induced by the SM P E strategy profile, σ * (·), of the DSG satisfies the Tweedie conditions [T], we know by Theorem 2 in [35] and further results by [24] which build on [10,34,35] Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
For any sequence of value function profiles, {v n } n , in L ∞ Y it is automatic that sup n v n (ω) R m dμ(ω) < +∞.
Thus, by the classical Komlos Theorem [20], any such sequence, {v n } n , has a subsequence, {v n k } k that K -converges to some K -limit, v ∈ L ∞ Y .

Strong Stochastic Continuity of the Law of Motion
Under the stochastic continuity assumptions made above, [A-1](14), we have by Scheffee's Theorem (see [8,Theorem 16.11]) that for each ω ∈ , for any sequence of networks {G n } n in (ω) converging to network G * ∈ (ω) (i.e., for each ω ∈ the conditional density mapping, G −→ h(·|ω, G), is continuous in L 1 norm with respect to G). Thus, by Scheffee's Theorem the L 1 norm continuity of G −→ h(·|ω, G) with respect to network G in each state ω is equivalent to the continuity of G −→ q(E|ω, G) in each state ω with respect to network G uniformly in E ∈ B (i.e., for each ω ∈ , q(E|ω, ·) is continuous in G, uniformly with respect to E ∈ B ).