Some remarks on Linear-quadratic closed-loop games with many players

We identify structural assumptions which provide solvability of the Nash system arising from a linear-quadratic closed-loop game, with stable properties with respect to the number of players. In a setting of interactions governed by a sparse graph, both short-time and long-time existence of a classical solution for the Nash system set in infinitely many dimensions are addressed, as well as convergence to the solution to the respective ergodic problem as the time horizon goes to infinity; in addition, equilibria for the infinite-dimensional game are shown to provide $\epsilon$-Nash closed-loop equilibria for the $N$-player game. In a setting of generalized mean-field type (where the number of interactions is large but not necessarily symmetric), directly from the $N$-player Nash system estimates on the value functions are deduced on an arbitrary large time horizon, which should pave the way for a convergence result as $N$ goes to infinity.


Introduction
Consider a stochastic differential game with N players, indexed by i ∈ [[N ]] := {0, . . ., N − 1}, where the state X i of the i-th player evolves according to the R d -valued SDE His/her cost, in the fixed time horizon [0, T ], is given by (1.1) for some F i = f i ⊗ I d and G i = g i ⊗ I d , with f i ∈ C 0 ([0, T ]; S (N )) and g i ∈ S (N ).The B i 's are independent R d -valued Brownian motions, and the α i 's are closed-loop controls in feedback form, that is α i = α i (t, X t ).
It is known (see for instance [12,Section 2.1.4],or [21]) that the value functions of the players, u i = u i (t, x) with i ∈ [[N ]], t ∈ [0, T ] and x = (x 0 , . . ., x N −1 ) ∈ (R d ) N solve the so-called Nash system of Hamilton-Jacobi PDEs (1.2) ].The equilibrium feedbacks are then given by α i = −D i u i .Since the dynamic of each player is linear, and the costs are quadratic in the state and control variables, it is well-known that the previous system can be recast into a system of ODEs of Riccati type, by making the ansatz that u i are quadratic functions of the states.Here, we look for conditions on the solvability of such system, and we focus in particular on properties that are stable as the number of players goes to infinity, with the aim of addressing the limit problem with infinitely many players, whenever possible.The Laplacian appears in (1.2) by the presence of the independent noises B i , but it will not actually play any relevant role in our analysis.We keep it since the purely deterministic system is not known to be well posed, beyond the linear quadratic setting.
The study of differential games with many players has enjoyed a rapid development in the last two decades, since the introduction of the theory of Mean Field Games (MFGs) independently by Lasry and Lions [30] and Huang, Caines and Malhamé [24].MFG theory provides an effective paradigm for studying dynamic games with many players that are both symmetric (that is, indistinguishable) and negligible.In such framework, one has a limit model, involving the equilibrium between a typical player versus a mass of agents, which is decentralised and symmetric: in fact, it is realised by a feedback of the state of the player only, it is identical for all the players, and it can be computed just by observing the population of players at the initial time.We refer to the book [12] and the lecture notes [11] for a recent account of the theory of MFGs.
If the MFG assumption is not fullfilled, that is, F i and G i do not depend just on the empirical measure of the other players, one enters into the broader framework of network games.There has been in the last few years an increasing interest in the understanding of the large population limits of equilibria among players whose interactions are described by graphs.Roughly speaking, when the number of interactions is "large", the MFG description, or a suitable generalisation based on the notion of graphon effectively characterizes large population limits: see for instance [1,5,8,29] and references therein.See also [15] for the analysis of a model on Erdős-Rényi graphs.
On the other hand, when the underlying network structure is sparse, very few results are available in the literature.In the dense regime, one expects all the players to have an individual negligible influence on a given player, because their running costs involve cumulative functions of many variables, while in sparse regimes, most of the players should have a small impact just because they are "far" with respect to the graph distance.This mechanism of independency between players that are far in the graph has been first observed in [28], by means of correlation estimates.
The framework addressed in [28] is somehow similar to ours.The linear-quadratic setting is considered, and under some symmetry assumptions on the underlying graph, Nash equilibria are computed explicitly (exploiting also the running costs F i to be identically zero, and G i to have a specific structure).Then, a probabilistic information on the covariance of any two players' equilibrium state processes is derived.A main goal in our work is to derive an analogous information in more analytic terms; that is, we wish to quantify the influence of the j-th player on the i-th one by estimating D j u i , with the perspective of developing some ideas that could be applied also beyond the linear-quadratic framework.
We now describe more in details our results.The first part of the paper is devoted to analyse a sort of sparse regime with a special structure, namely shift-invariance, where basically f i and g i coincide with f i−1 and g i−1 , respectively, after the permutation of variables x i → x i−1 .Most importantly, we assume players to have nearsighted interactions, that means where (β k ) k is a suitable sequence in ℓ 1 (Z).Since β k−i decays as |k − i| → ∞, this means that f i hk , which quantifies the influence of the h-th and k-th player on the i-th one, decreases as |h − i| and |k − i| increase.A prototypical example is given by the bi-directed chain, where f i hk is a sparse matrix with zero entries except for |h − i| ≤ 1 and |k − i| ≤ 1, so that the i-th player cost depends only on the (i + 1)-th and the (i − 1)-th state.See also [19] for results on models that build upon a similar structure.
The first statement we get, which resumes Theorems 2.11 and 2.12, is the following.
Theorem.There exists T * > 0 such that if T T * then for any N ∈ N ∪ {∞} there exists a smooth solution to system (1.2) such that, for any i, j and m ∈ N, Note in particular that we get existence of a smooth classical solution to the infinite-dimensional Nash system (1.2) with N = ∞, i ∈ Z.We stress that the key issue in the analysis of our problem is that, despite the cost functions f i , g i may depend on very few variables, the system itself is strongly coupled by the transport terms j =i D j u j D j u i , which become in fact series when N = ∞.The closed-loop structure of equilibria forces the equilibrium feedbacks α j = −D j u j to be strongly "nonlocal", that is, they depend on the full vector state x.Hence, decay estimates on D j u i which are stable as N increases are crucial to pass to the limit N → ∞.These are obtained here by a careful choice of β: below, we will use the terminology self-controlled for the discrete convolution, or briefly c-self-controlled (where the "c" stands for "convolution") , that fits well with the structure of cyclic discrete convolution appearing in our problem.From the game perspective, the equilibrium feedbacks α j turn out to be almost "local", in the sense that the influence of "far" players is still negligible in the sense explained above.In other words, despite the strong coupling given by the full information structure of closed-loop equilibria, the property of "unimportance of distance players" is observed (see Remark 2.13 for further discussions).
While the shift-invariance condition can be actually dropped (see Section 2.5), one of the main restrictions of the previous result is that it guarantees short-time existence only.Note that, even with a finite number of players N , Riccati systems may fail to have long time solutions in general; therefore we look for further conditions on f i and g i such that existence holds for any time horizon T , and independently on N .To achieve this goal, we strengthen the previous assumption on nearsighted interactions, that now become of strongly gathering type, and require further directionality conditions.Section 2.6 contains precise definition of this notion and examples.The main existence result is stated in Theorem 2.22.Here, we exploit the possibility to relate a solution to system (1.2) to a flow of generating functions, which works well when N = ∞ but has no clear adaption to the finite N setting.
Within the special setting of systems with cost of strongly gathering type and directionality, we are able to push further our analysis and study the long time limit T → ∞, that is, we show that the value functions u i converge to solutions of the ergodic problem as the time horizon goes to infinity.To complete this program, estimates on solutions of the Riccati system are obtained at the level of the game with N = ∞ players, uniformly with respect to T .The main result of this part of the work is stated by Theorem 2.38.
We conclude this second part by showing that equilibria of the infinite-dimensional game provide ε-Nash closed-loop equilibria of suitable N player games, see Section 2.7.
In the last part of the work, we come back to the dense regime, and work without any symmetry assumption (like shift-invariance).Our goal is again to deduce some estimates on the Riccati system that do not depend on the number of players, under a mean-field-like condition and the same for g i .It is crucial to observe that in this dense setting there is no hope to have a limit system of the same form.Indeed, in the sparse case we had an ℓ 1 control of coefficients independent of N (given by β), so that the series j =i D j u j D j u i carries to the limit by dominated convergence, while now a dominated convergence cannot performed.In fact, at least in the symmetric case, where the costs are of the form ), the correct limiting object is the so-called Master Equation, which has been the object of the seminal work [10].The convergence of u i to a limit function defined on the space of probability measures has been shown under monotonicity assumptions of f i , and its regularity is obtained as a consequence of the stability properties of the MFG system, which characterizes the limit model.
Here, we wish to deduce the estimates on u i that allow for a passage to the limit N → ∞ directly on the Riccati system (that is, on the Nash system).For short time horizon, one can basically reproduce the same approach of the previous section (cf.Theorem 2.15).To achieve stability for arbitrary values of T , we impose a bound from below on the matrices (f i ij ) ij and (g i ij ) ij : If κ is small enough, then the existence of a solution to the Nash system is guaranteed for large T , by a mechanism that produces an analogous bound from below on the matrix (D ij u i ) ij .Recalling that the equilibrium feedbacks are given by α j = −D j u j , this is equivalent to the one-sided Lipschitz condition Our main estimate, which pave the way for a convergence result in the N → ∞ limit, is contained in Theorem 3.1.Interestingly, the monotonicity (or mild non-monotonicity) condition (1.3) generates the nice structural property (1.4) of the equilibrium drift vector (−D j u j ) j ; it is worth observing that, when κ = 0 and in the symmetric MFG case, (1.3) is equivalent to the displacement monotonicity condition used in [22] to get well-posedness of the Master Equation.
We now discuss some some further related references.In [3,18], the convergence problem of open-loop equilibria in symmetric N -person linear-quadratic games is addressed, while [16] deals with the selection problem for models without uniqueness.See also [4] for a convergence result in a finite state model.Linear-quadratic MFGs are studied in several works, see for instance [6,7,23,31,32] and references therein.Further contributions on the convergence problem are contained in [9,17,20,26,27], while recent results on the structure of many players cooperative equilibria can be found in [13,14,25].
As a final comment, despite trying to pursue the full generality of the linear-quadratic differential game framework, we carefully analysed here a few case studies, with the aim of highlighting some properties that could be investigated beyond the linear-quadratic setting, and directly on the Nash system of PDEs.With this purpose, we tried to avoid as much as possible the use of explicit formulas.Despite these were quite crucial to study the long time behaviour in the sparse regime, there are a few takeaways here that can be starting points for further investigations.On one hand, we are trying to address the short-time existence of the infinite-dimensional Nash system with non-quadratic cost functions, by estimating the derivatives of u i by means of the coefficients β constructed here.On the other hand, we wish to derive analogous bounds on u i in the mean-fieldlike setting, again by working directly on the Nash system, and without making use of limit objects such as the Master Equation.
Acknowledgements.The authors are partially supported by the Gruppo Nazionale per l'Analisi Matematica, la Probabilità e le loro Applicazioni (GNAMPA) of the Istituto Nazionale di Alta Matematica (INdAM), Italy.Cirant partially supported by the King Abdullah University of Science and Technology (KAUST) project CRG2021-4674 "Mean-Field Games: models, theory, and computational aspects", and by the project funded by the EuropeanUnion-NextGenerationEU under the National Recovery and Resilience Plan (NRRP), Mission   ] such that r = a mod b.The entries of a N × N matrix will be indexed over [[N ]].We will denote by L (N ) ∈ S (N ) the lower shift matrix mod N , defined by where δ is the Kronecker symbol.
Our main assumption in this section, on the structure of the running cost, is that f i and g i are shift-invariant, in the sense of the following definition.
Note that this is equivalent to requiring that for all i ∈ [[N ]] and x ∈ R N , Example 2.2.A very basic shift-invariant case is that with g 0 = 0 and (2.1) When ℓ = N , the cost is actually of Mean-Field type, that is, it penalizes the deviation of the private state X i from the mean of the vector X.
Here q 0 induces an underlying directed circulant graph structure G ℓ to the problem; indeed, by assumption (SI) ] − I N can be considered as the asymmetric and circulant adjacency matrix of G ℓ , so that (2.2) reads The same is true in the more general case when (2.4) w = (1, −w 1 , . . ., −w ℓ−1 ), with w j 0 ∀ j, here the w j 's are regarded as normalized weights, and we have which generalizes (2.3).Note finally that a sort of "directionality" is encoded in the above examples, that is, each player i is affected by the "following" ones j > i in the chain.This is not yet important at the current stage, namely we may allow for It is only from Section 2.6 that m = 0 will be required.

2.2.
The evolutive infinite-dimensional Nash system.The main object of our study is the Nash system When N = ∞, we need to be a bit careful about the notion of solution.In this case, x ∈ X = ℓ ∞ (Z; R d ).Then, we mean that (2.6) admits a classical solution in the following sense.Definition 2.3.A sequence of R-valued functions (u i ) i∈Z defined on [0, T ]×X is a classical solution to the Nash system (2.6) on [0, T ] × X if the following hold: (S1) each u i is of class C 1 with respect to t ∈ (0, T ) and C 2 with respect to x ∈ X , in the Fréchet sense; (S2) for each i ∈ N, the Laplacian series ∆u i = j ∆ j u i and the series j =i D j u j D j u i uniformly converge on all bounded subsets of [0, T ] × X ; (S3) system (2.6) is satisfied pointwise for all (t, x) ∈ (0, T ) × X ; As it is customary in linear-quadratic N -player games, we look for solutions of the form for some functions , where e j = e j ⊗ I d , {e j } N −1 j=0 being the canonical basis of R N .Hence, from (2.6) we obtain that is,

and
(2.7) c j e j e j T c i + c i e j e j T c j = f i , as x T Ax = 0 for all x if and only if A + A T = 0 for any square matrix A. Now, given the shift invariance of f i and g i , one expects a solution to (2.7) to enjoy the same property, hence we look for a solution such that (2.8) for some c : [0, T ] → S (N ).Clearly, this makes η i independent of i, as we will have η i = η := T • tr c.By plugging (2.8) into (2.7) and letting i = 0 one obtains the following system of ODEs for the entries of c: where f := f 0 and g := g 0 .
As we are interested in the limit problem of infinitely many players, which we expect to be indexed by Z since we have an undirected structure, it is convenient to shift the indices in such a way that i = 0 "stays in the middle"; that is, we let i = −N ′ , . . ., N ′′ , instead of i = 0, . . ., N , where, for example, Therefore, we rewrite system (2.9) as (2.10) where all indices are understood mod N and between −N ′ and N ′′ .Now it is immediate to identify a convenient limit system by letting N → ∞.
2.3.Self-controlled sequences for the discrete convolution.We can recognise a structure of a cyclic discrete convolution in the sums in (2.9); that is, (2.11) We wish to exploit this fact in order to prove existence of the solution to system (2.10) for small T , for any (possibly infinite) N .
Our main tool will be the following.
for some constant C > 0 independent of i.
Clearly, any nonnegative compactly supported β (as a sequence indexed by Z) is c-self-controlled.Nevertheless, we will also be interested in positive c-self-controlled sequences in ℓ 1 (Z), which indeed exist, as shown by the following result.Lemma 2.5.For any ε > 0, there exists a positive sequence β ∈ ℓ 1 (Z), with β 2 < ε and such that β ⋆ β 2β.In particular, one can choose β of the form satisfy the following property: if f, g ∈ L 2 ((−π, π)), then f g j = ( f ⋆ ĝ) j for all j ∈ Z.Let now be f α := e −α|•| , for any α > 0. It is elementary to compute, for each j ∈ Z, , the last inequality being straightforward to check using the explicit expression (2.13).Also, by Parseval's identity ) → 0 as α → +∞.so that β = f α has the desired properties for any choice of α = α(ε) sufficiently large.
Remark 2.6 (Variations on a c-self-controlled sequence).Clearly any positive multiple of a c-selfcontrolled sequence stays self-controlled.This allows to have self-controlled sequences of arbitrarily large ℓ ∞ -norm, although with a larger constant C in (2.12).On the other hand, one can also build c-self-controlled sequences which decay exponentially faster; indeed, if β is c-self-controlled, then for any γ > 0 so is the sequence defined by setting βi := β i e −γ|i| , with the same implied constant C.This is easily proven as follows.Suppose that i 0; then The case i 0 is analogous.
The next result will be useful to deal with convolution of the form (2.11).It essentially states that c-self-controllability is preserved by suitable perturbations, which include all perturbations with compact support.Lemma 2.7.Let β be c-self-controlled and let θ = (θ hk ) h,k∈Z be a nonnegative sequence such that θ β ⊗ β. 1 Let d be the sequence given by d := β ⊗ β + θ.Then Remark 2.8.We will exploit a straightforward implication of the above inequality, namely that 2.4.Short-time well-posedness for nearsighted interactions.We are now ready for our first existence and uniqueness results (Theorems 2.11 and 2.12 below), which is a direct consequence of the following proposition.
Let J N : K N → C N be the map given, for each i, j = −N, . . ., N , by Then there exist T * > 0, depending on β but independent of N , such that where the last estimate comes from Lemma 2.7.As d ∈ c 0 (Z 2 ), sup d ij with an implied constant which does not depend on N .The same holds for −N i < 0 by a symmetrical argument.Analogously, ( where the constant C depends only on β.It follows that for T > 0 small enough, depending on β, one has J N (c) ij ∞ < 2d ij for all i, j = −N, . . ., N .
Remark 2.10.We stated and proved Proposition 2.9 for an odd number of players.This is just a matter of having expressions that look more symmetrical, yet there is no preference about the parity of the number of players, so that the above result holds, mutatis mutandis, also if the number of players is even.It is also clear that, with a very much analogous proof, the thesis of Proposition 2.9 also holds for N = ∞, where one defines, in a natural way, Theorem 2.11.Under the hypotheses of Proposition 2.9, there exists T * > 0 such that if T T * then for any N ∈ N ∪ {∞} there exists a unique smooth solution to system (2.10) such that, for any i, j ∈ −N ′ , . . ., N ′′ and m ∈ N, where the implied constants depend only on T * , f , g and m.
Proof.A fixed point of the map J N defined in (2.15) is a solution.We deal with the case of J N with N = ∞, as the case with N ∈ N can be included therein.Note that K ∞ can be considered as a closed ball of the Banach space ℓ ∞ d (Z 2 ; C 0 ([0, T ])); that is, the space of functions from We prove that the map J ∞ is a contraction on K ∞ , provided that T is sufficiently small with respect to d.The conclusion will follow from Proposition 2.9, Remark 2.10 and the contraction mapping theorem; then, once one have a continuous solution, by the structure of equations (2.10), one bootstraps its regularity up to C ∞ , while estimate (2.17) for m > 1 follows by induction differentiating (2.10) and estimating as in the proof of Proposition 2.9.Let now c, c ∈ K ∞ .We have, for i, j fixed, (2.18) We have Theorem 2.12.Suppose that f i and g i are shift-invariant and there exists a positive c-self- There exists T * > 0, depending only on C and β, such that if T T * then there exists a smooth classical solution to the infinitedimensional Nash system (2.6) with i ∈ Z on [0, T ] × X .
Proof.Let c be the solution given by Theorem 2.11.For where we denoted by and continuous in t, because the series normally converge thanks to estimate (2.17); for the same reason, also are well-defined and continuous.Finally, for h ∈ ℓ ∞ (Z; R d ), note that (omitting the dependence on t) where (σx) j := x j−1 for j ∈ Z.We have hence j∈Z D j u j D j u i locally uniformly converges by estimate (2.17).Hence, by construction, u solves (2.6) in the desired sense.
Remark 2.13 (Unimportance of distance players).What essentially allows the Nash system to be well-posed in infinite dimensions is what we call an unimportance of distant players, in the sense that the farther a player is from a given one (say the 0-th player) the smaller the impact is has on it is.This is seen in the fact that, on any bounded B ⊂ X , 2.5.Beyond shift-invariance.We have made the shift-invariance hypothesis to reduce our system of infinitely many equations for c to one equation.Nevertheless, the reader should be aware that the above results can be adapted to a more general setting.
One can suppose that we only have a shift-invariant control on the data; that is In this case, the most natural limit of (2.7) is indexed over N and it suffices to replace K ∞ with Then, for instance, one obtains the following result.
Theorem 2.14.Assume (2. 19).There exists T * > 0 such that if T T * then for any N ∈ N∪{∞} there exists a unique smooth solution to system (2.7) such that, for any i, h, k ∈ where the implied constants depend only on T * , f , g and m.
Also, one can fix the dimension N 2 and consider β N ∈ (R + ) 2N +1 given by In this case one can look for a solution to (2.7) starting with assumption (2.19) with β = β N ; that is, (2.20) What we get is the following statement.
Theorem 2.15.Let N ∈ N, N 2 and assume (2.20).There exists T * > 0 such that if T T * then there exists a unique smooth solution to system (2.7) such that, for any where the implied constants depend only on T * , f , g and m.
Notice that this can be regarded as a result in a non-symmetric mean-field-like setting (see Remark 3.2 for further comments on this terminology), as assumption (2.20) is consistent with the fact that we expect the j-th derivative of a mean-field-like cost for the i-th player to scale by a factor of N −1 whenever j = i.
2.6.Long-time well-posedness for shift-invariant directed strongly gathering interactions.It is clear that the previous construction follows the standard Cauchy-Lipschitz local (in time) existence argument, and the existence (and uniqueness) of a solution can be as usual continued up to a maximal time T * , that is when the quantity max i,j β −1 i β −1 j |c 0 ij | blows up.So far, we cannot exclude in general that such blow up time T * is finite.
Upcoming Definition 2.16 introduces an additional assumption on the running and terminal costs which will allow us to prove long-time well-posedness for the infinite-dimensional Nash system.Given m n, we will identify Remark 2.17.The term "directed" is related to the fact that the i-th player's cost is affected by the states of the "following" players j > i in the chain, and this fact might not be immediately clear from the previous definition, which just requires the matrices M (N ) to be extensions of a fixed matrix, not depending on N .One may then have a look to the matrices of the form M = w ⊗ w in (the end of) Example 2.2.In particular, w = w (N ) in (2.5) gives rise to a directed family only when m = 0.Moreover, in the situations described in Example 2.2, the associated sequence of matrices as the dimension N diverges is not strongly gathering.Indeed, even though ℓ stays bounded, one has that µ is a polynomial with µ(1, 0) = 0.In fact, those situations can be seen as limit settings corresponding to taking ̺ = 1; we will comment on this case later on in Section 2.10.The validity of the strong gathering assumption as it is, can be achieved in different ways; two basic settings are given in the following examples.
Example 2.18.If we want to stick to a matrix f 0 of the form (2.1), in order to have (f 0 (N ) ) N ∈N (where N is the dimension) directed with strongly gathering limit one can require that ℓ is independent of N large and this means that we are considering an underlying graph where each node is directly connected with itself as well, and such link has a negative weight.As in this model a positive weight is associated to the tendency, in order to reduce their cost, of each player to get closer to their neighbors, a negative connection with themself is to be interpreted as a drift towards self-annihilation.More prosaically, this means that the state of each player will also tend to the common position 0 ∈ R d .
Regarding this example, it is also worth pointing out that the common attractive position 0 ∈ R d cannot be any other arbitrary point, in the sense that the structure of problem is not invariant under translation of the coordinates.This is due to the fact that we are considering a graph whose nodes have outdegree different from 1.
Example 2.19.Another setting with directedness and strong gathering is that of where w can be given by (2.4) with ℓ independent of N , ν > 0 and E 0 x = x 0 for all x = (x 0 , . . ., x N −1 ) ∈ (R d ) N .Here and with a translation of the coordinates we can lead to this situation also costs like for any given y ∈ R d .This example, more genuinely than the previous one, shows that the strong gathering assumption entails that we are giving some sort of preference about where the players should aggregate.This will strengthen the attractive structure yielded by a graph satisfying only assumption (SI) (Example 2.2), thus providing more stability to our game.
From this section on in this part, the following assumptions will be in force, declined in suitable manners which will be specified.
Assumptions (⋆).The matrices f i and g i are shift-invariant and f 0 = f 0 (N ) and g 0 = g 0 (N ) are directed with limits f, g ∈ S (ℓ) for some ℓ ∈ N. The matrix f is ̺-strongly gathering for some ̺ > 0 and g is compatible on I with f for some interval I ⊂ R, in the following sense: given Remark 2.20.Note that for any t ∈ R, one has that ψ tanh(tξ) + ξ is bounded on D, as ξ(z)t can be a singular point only if ξ(z) 2 = ϕ(z, 0) < 0, which contradicts that z ∈ D. Also note that condition (2.21) holds with I = R if, e.g., g = 0 or |ψ| |ξ|.
Considering system (2.10), we see that for any N sufficiently large with respect to ℓ we have f hk = 0 = g hk if either h or k is negative; hence the limit system as N → ∞ will be given by where f hk and g hk extend to h, k ∈ Z 2 by letting 4 Given this system, another reduction is possible.Since f hk = 0 = g hk whenever h ∧ k < 0, the assumption that c hk = 0 if h ∧ k < 0 is not a priori incompatible with the structure of system (2.23).Therefore we will look for a solution with this additional property; in this way, the coefficients which are not a priori null, (c hk ) h,k∈N , will define in a natural way a symmetric operator c(t) ⊗ I d on X which vanishes on ℓ ∞ (Z <0 ; R d ), so that it can also be seen as a trivial extension to X of a symmetric operator on ℓ ∞ (N; R d ).This reduces the system of ODEs in (2.22) to (2.23) which will be the object of our following study, and will eventually provide a solution with the particular form presented in the upcoming definition.
The following lemmata, in whose statement the hypotheses of Theorem 2.22 will be implied, provide the steps of our proof of Theorem 2.22.
All the other ĉhk 's satisfy first-order ODEs with coefficient which are second-order polynomials depending only on f hk and ĉh ′ k ′ with (h ′ , k ′ ) ≺ (h, k), where ≺ denotes the strict Pareto preference. 6Therefore, existence and uniqueness of the solution to the infinite system may be proved by induction.Indeed, suppose (ĉ 0k ′ ) 0 k ′ <k ⊂ C ∞ (R) are given, and note that then ĉ0k is unique and smooth as well.This proves the existence and uniqueness of ĉ0k for all k ∈ N; then, looking at this argument as the base step for a new induction over h, one proves analogously the existence and uniqueness of ĉhk for all h ∈ N and any k ∈ N. Finally, c hk = c kh since equations (2.9) are invariant with respect to the swap (h, k) → (k, h).
The arguments below show that the coefficients c hk can be thought as derivatives of a generating function Ξ, which will play a fundamental role in the long-time analysis.
Lemma 2.24.The solution to (2.23) on [0, T ] is given by ) is given by ı(x) i = x i if i ∈ N and ı(x) i = 0 otherwise. 6That is, (h ′ , k ′ ) ≺ (h, k) if and only if h ′ h and k ′ k with at least one strict inequality.
Lemma 2.27.Let c be the solution to (2.23) on [0, T ].The functions u i given by (2.24) are welldefined for (t, x) ∈ [0, T ] × X , differentiable with respect to t, and twice Fréchet-differentiable with respect to x.
Proof.Since a suitable shift of coordinates in X transforms u i (t, •) into u 0 (t, •), it is sufficient to prove the result for u 0 .The well-posedness follows from Lemma 2.26, and the differentiability with respect to t follows from the same lemma and the fundamental theorem of calculus.The differentiability with respect to x follows again from Lemma 2.26, as it is trivial to see that for any h ∈ X small one has where, with an obvious notation, (c(t Lemma 2.28.Let u i be given by (2.24), where c is the solution to where K = K(r, T ) is the constant appearing in Lemma 2.26.
8 And for all higher order derivatives.
Proof.We have D j u i (t, x) = 0 if j < i, and D j u i (t, x) = h∈N c j−i,h (t)x h if j i.Then, for x ∈ B R , and r ∈ (1, ̺) fixed, estimate (2.31) yields for all t ∈ [0, T ].The thesis now follows by computing j>i r i−j .
Since, by construction, choosing u i as in (2.24) satisfying (S1) and (S2) in Definition 2.3 yields a classical solution, the proof of Theorem 2.22 is complete.
2.7.Almost-optimal controls for the N -player game.We did not prove long-time wellposedness of the Nash system for the N -player game, with N > ℓ finite; nevertheless, in Corollary 2.32 below we show that on the horizon [0, T ] the infinite-dimensional optimal control for the i-th player, given by α * i (t, X t ) := j i c 0,j−i (t)X j t , if suitably "projected" onto (R d ) N provides an ε-Nash equilibrium for the N -player game, with ε → 0 as N → ∞.In claiming so, we consider classes of admissible controls in the sense of the following definition.
Remark 2.30.For such controls, it is known (cf., e.g., [2, Theorems 9.1 and 9.2]) that (2.33) has a unique solution, satisfying E sup [0,T ] |X| 2 C where C is a locally bounded function of R, L and T , directly proportional to 1 Theorem 2.31.Consider the N -player game on [0, T ] with time evolution of the state of the players given by (2.33) and costs given by (1.1) with f 0 , g 0 0. Let Assumptions (⋆) be in force with [0, T ] ⊆ I. Let c solve (2.23) and define the control α * by Then, for any R 0 and admissible L, for any i ∈ [[N ]] and any (α where ℓ is the dimension appearing in Assumptions (⋆) and the constant Ĉ is a locally bounded function of R, L, T and δ, directly proportional to 1 + |x 0 | 2 .
Proof.Since f i and g i are shift-invariant and α * is linear in the state variable with Dα * circulant, without loss of generality we can prove the thesis for i = 0. We denote by X * the solution to (2.33) when α = α * and by X the solution to (2.33) when α = (ψ, α * 1 , . . ., α * ,N −1 ) =: α * .We wish to estimate from above the quantity (2.34) where Note that we can replace ℓ with any M ℓ because c jk (T ) = g jk (T ) = 0 if k > ℓ.As c solves (2.23) we obtain where (Z t ) 0 t T is a martingale starting from 0. Straightforward computations show that, omitting the dependence on t, where we have set Note now that by the convexity of 1 2 | • | 2 , omitting the dependence on t, and by the convexity of F Using (2.35), (2.36) and (2.37) in (2.34) we obatin As, whenever {Y, Z} ⊆ {X * , X}, ) is the constant appearing in Remark 2.30, we have so that by Lemma 2.26, for any δ ∈ (̺ −1 , 1), there exists K > 0 such that (2.39) On the other hand, where we used again that α * , α * ∈ A R,L .Therefore, from (2.38), (2.39), (2.40) and Lemma 2.26 we obtain This concludes the proof.
Corollary 2.32.Let Ω ⊂ R d bounded, and assume the same as in Theorem 2.31, but with x 0 ∈ Ω N .Let R 0 and L be admissible (in the sense of Definition 2.29); consider as admissible those controls belonging to A R,L .Then, for any ε > 0 there exists N 0 = N 0 (ε, R, L, ̺, ℓ, Ω) such that if N N 0 then the control α * provides an ε-Nash equilibrium of the game.
Proof.It suffices to require that where, for instance, one sets δ = 1 2 (1 + ̺ −1 ).Choose M = M (N ) such that M → ∞ and M = o(N ) as N → ∞; for example, M = ⌊ √ N ⌋ for all N > ℓ 2 .Since x 0 ∈ Ω N there exists a constant Ĉ′ , independent of N , such that Ĉ N Ĉ′ .Then the conclusion follows from the fact that the left-hand side of (2.41) goes to 0 as N → ∞.
2.8.The ergodic Nash system.Consider now the related ergodic problem, with costs where the dynamics and the assumptions on F i are the same as before.The corresponding Nash system reads (2.42) where Y is either (R d ) N , for the N -player game, or X , for the limit game with infinitely many players.In the latter case we give notions of classical solution and QSD solution which are analogous as those in the previous section.
Definition 2.33.A sequence of pairs ((λ i , v i )) i∈Z of real numbers λ i and R-valued functions v i defined on X is a classical solution to the ergodic Nash system (2.42) on X if the following hold: (E1) each v i is of class C 2 with respect to x ∈ X , in the Fréchet sense; (E2) for each i ∈ N, the series j =i D j v j D j v i uniformly converges on all bounded subsets of X ; (E3) system (2.42) is satisfied pointwise for all x ∈ X ; A QSD ergodic (QSDE) solution will be a classical solution to (2.42) with (2.43) for some c ∈ ℓ 1 (N 2 ).
Remark 2.34.By the structure of (2.42), it is clear that if ((λ i , v i )) i∈Z is a classical solution, then so will be ((λ i , v i + µ i )) i∈Z for any choice of real numbers µ i .We will prove that there exists a special choice of µ ∈ R, and of c ∈ ℓ 1 (N 2 ), such that the solution ((λ, v i + µ)) i∈Z , with λ and v i given by (2.43), is in a precise sense the limit of the QSD solution as T → +∞ (see Theorem 2.38).
Arguing as in the previous section, the coefficients c hk of a QSDE solution are given by the solutions of the following system: (2.44) It is immediate to see that if c solves (2.44), then so does −c, hence we cannot have a unique solution to this limit system.
Lemma 2.35.There are exactly two sequences (c ± hk ) h,k∈N which solve (2.44).Such sequences are one the opposite of the other; that is, c − = −c + .
Proof.We have c 2 00 = f 00 > 0, hence c 00 ∈ {± √ f 00 }.Once the sign of c 00 is chosen, all other c hk can be uniquely determined by induction on h, k.
Both solutions can be represented also in this case by a generating function.
Note that the real part of the denominator in (2.46) can vanish only if |z|, |w| = ̺, hence Ξ is well-defined and analytic for (z, w) ∈ ̺D 2 .This is sufficient to prove the existence of exactly two opposite QSDE solutions, and thus that the limiting ergodic Nash system is well-posed in X .We state this as a theorem.
Theorem 2.37.There exist exactly two QSDE solutions to (2.42) on X .Such solutions are one the opposite of the other.
Proof.Argue as in the first part of the proof of Lemma 2.26 to say that c + ∈ ℓ 1 (N 2 ), then argue as in the proof of Lemma 2.27 to build the QSDE solution determined by choosing c = c + .Finally note that the solution determined by the choice c = c − is the opposite function.
2.9.Long-time asymptotics.As expected by KAM theory and ergodic control, we are going to prove now that, up to a constant, the QSDE solution corresponding to c + (that is, the solution with c 00 > 0) describes the long-time asymptotics of the QSD solution as T → +∞, while the opposite solution should be regarded as the result of considering the limit T → −∞ instead.To highlight the dependence of the QSD solution on T , we will write it as u i T ; on the other hand, since by the shift-invariance property it will suffice to show the convergence of u 0 T to the QSDE solution v 0 , we will omit the superscript 0.
Theorem 2.38.Let Assumptions (⋆) be in force with [0, +∞) ⊆ I. Let u T be the value function of the 0-th player corresponding to the QSD solution to the Nash system on [0, T ] × X ; let v be the value function of the 0-th player corresponding to the QSDE solution on X determined by c = c + .Let λ := tr c.Then, for any t 0, as t < T → +∞, the following limits hold, locally uniformly in both x ∈ X and t: and there exists a constant µ ∈ R such that The proof is based on the following result, which is strictly related to a refinement of Lemma 2.26 (cf.Remark 2.40 below) and is due to the possibility of explicitly compute the integral in formula (2.28) as showed in Remark 2.25.
Proof.By the continuity of ξ, given r ′ ∈ (r, ̺), , we obtain, also using (2.21), (2.51) where the implied constant depends only on r, f and g.At this point, it is easy to see that the desired conclusion follows.
Remark 2.40.This proof shows that if [0, +∞) ⊆ I, then the constant K appearing in Lemma 2.26 is in fact independent of T , since sup R×Qr | Ξ| is finite.In particular, c ∈ C 0 ([0, T ]; ℓ 1 (N 2 )) along with its derivatives, and their the norms are bounded uniformly with respect to T > 0.
The implied constant can be computed quite explicitly, by retracing the proofs above; we confine ourselves to pointing out that it depends only on L, r and f , and that it explodes as L → ∞ or r → 1.
2.10.Digression on a delicate limit case.We have noted in Example 2.2 that the case of a cost designed according to an underlying directed circulant graph structure is limit among those satisfying our assumptions, in the sense that Definition 2.16 holds with ̺ = 1.
Results like Lemmata 2.23, 2.24, 2.26, 2.35 and 2.36 continue to hold, but the other methods used in the previous sections are not refined enough to successfully prove all the previous theorems for those limit cases, even though, along with Remark 2.40, they are sufficient in order to establish ℓ ∞ -stability at the level of the system for c; that is, convergence in ℓ ∞ (N 2 ) of the solution to (2.23) to a solution of (2.44).
On the other hand, having a closer look at the simplest limit case, which is the directed chain given by the choice g = 0, f 0 00 = f 0 11 = 1 = −f 0 01 = −f 0 10 and f 0 hk = 0 for all other h, k, we note that we are also able to compute QSDE solutions quite easily, thanks to formula (2.46).In this case we have φ(z, 0) = 1 − z, and we find the expansion As by Stirling's formula |c ± hk | ≍ (h + k) δ 0,hk − 5 2 , one easily sees that these coefficients well-define a QSDE solution, hence the limit ergodic Nash system is well-posed.
One can also note that the coefficients enjoy the property that (2.57) where c is either c + or c − ; this can be seen from (2.56) or proved by induction using system (2.44).
In fact, the same can be done for system (2.23), so that property (2.57) also holds for the 12 solution of (2.23) on [0, T ], for any fixed T > 0. Therefore, the information about the coefficients of a prospective QSD or QSDE solution is encoded in the functions of one complex variable Ξ 0 (t, •) := Ξ(t, •, 0) and Ξ 0 = Ξ(•, 0), namely This peculiar fact is specific of the directed chain.It could be useful as it allows to conclude wellposedness of the infinite-dimensional evolutive Nash system provided that the sequence of functions (c 0k ) k∈N is monotone,13 yet this would not still be enough to deal with convergence to an ergodic solution.
Another property is instead shared by all problems having f of the form (2.1): the polynomial φ factors as φ(z, w) = ξ 2 (z)ξ 2 (w); this makes Ξ and Ξ functions of (ξ(z), ξ(w)), possibly helping in the analysis of the aforementioned limit cases.

Mean-field-like games
In this part we consider the Nash system (2.7) without the shift-invariant assumption.Instead, we make a mean-field-like assumption.Given (a i jk ) i,j,k ∈ S (N ) N we will use the notation B(a) for the matrix B(a) hk = a h hk .Given the i-th vector e i of the canonical basis of R N , we will write E i = e i (e i ) T .The indices will range over [[N ]].
Note that system (2.7) can be rewritten in a forward form as (3.1) where c i (t), f i (t), g i ∈ S (N ) for all i ∈ [[N ]] and t ∈ [0, T ].
Theorem 3.1.Let T > 0 and let f ∈ L 1 ((0, T ); S (N ) N ) and g ∈ S (N ) N satisfy Suppose that B(f ) −K f I and B(g) −K g I for some constants K f and K g such that Then there exists N 0 ∈ N such that if N > N 0 there exists a unique absolutely continuous solution c to (3.1) on [0, T ), which satisfies for some constant C which is independent of N .
Note that (3.3) is automatically satisfied when K f , K g 0 for any T > 0. Otherwise, for fixed K f and K g , it poses a restriction on the size of T (or, similarly, a restriction on the size of (K f ) + and (K g ) + once T is fixed).Back to the value functions u i , the previous estimate reads as follows: in particular, sup i j: j =i Remark 3.2.We use the terminology mean-field-like since any f i such that sup 2), with κ f depending on C (and not on N ).In turn, the previous inequality is satisfied when f i (x) = V i (x i , (N − 1) −1 j =i δ x j ), where V i is a smooth enough function defined over R d × P(R d ) (see for instance [10,Proposition 6.1.1]).
The proof of the theorem is based on the following lemmata. We

2 . 1 .
Shift-invariant games 2.General setting.For a ∈ Z and b ∈ Z + , we will use the notation [a] b to identify the unique natural number r ∈ [[b]

Definition 2 . 21 .
A quadratic shift-invariant directed (QSD) solution to (2.6) is a classical solution of the form(2.24)
Proof of Theorem 2.38.Fix r ∈ (1, ̺).By comparing expressions (2.30) and (2.46) one sees that where the implied constants depend only on r and f .As γ is integrable on [0, +∞), by (2.55) so is tr ĉ − λ, where we use the notation ĉ = c(T − •) introduced in the proof of Lemma 2.23; thus by the dominated convergence theorem there exists µ ∈ R such that Along with (2.54), this proves (2.47) and (2.48).Remark 2.41.The argument of the previous proof also applies to the case when t = sT , with s ∈ [0, 1].In this case, we can give the following estimate of the rate of convergence of (2.47): for any r ∈ (1, ̺), |z|,|w| r |Ξ(t, z, w) − Ξ(z, w)| γ(T − t),where γ is the function given in Lemma 2.39, and the implied constant depends only on the L ∞norms of φ and ξ on D r and Q r , respectively.By Lemma 2.24 and Cauchy's theorem on derivatives,T (t) − λ | γ(T − t), T t tr c T − (T − t)λ → µ as T → +∞,locally uniformly in t.