Weak KAM Approach to First-Order Mean Field Games with State Constraints

We study the asymptotic behavior of solutions to the constrained MFG system as the time horizon T goes to infinity. For this purpose, we analyze first Hamilton–Jacobi equations with state constraints from the viewpoint of weak KAM theory, constructing a Mather measure for the associated variational problem. Using these results, we show that a solution to the constrained ergodic mean field games system exists and the ergodic constant is unique. Finally, we prove that any solution of the first-order constrained MFG problem on [0, T] converges to the solution of the ergodic system as T goes to infinity.


INTRODUCTION
The theory of Mean Field Games (MFG) was introduced independently by Lasry and Lions [1,2,3] and Huang, Malhamé and Caines [4,5] to study Nash equilibria for games with a very large number of players.Without entering technical details, let us recall that such an approach aims to describe the optimal value u and distribution m of players at a Nash equilibrium by a system of partial differential equations.Stochastic games are associated with a second order PDE system while deterministic games lead to the analysis of the first order system (1.1) where Ω is an open domain in the Euclidean space or on a manifold.Following the above seminal works, this subject grew very fast producing an enormous literature.Here, for space reasons, we only refer to [6,7,8,9] and the references therein.However, most of the papers on this subject assumed the configuration space Ω to be the torus T d or the whole Euclidean space R d .
In this paper we investigate the long time behavior of the solution to (1.1) where Ω is a bounded domain of R d and the state of the system is constrained in Ω.
The constrained MFG system with finite horizon T was analyzed in [10,11,12].In particular, Cannarsa and Capuani in [10] introduced the notion of constrained equilibria and mild solutions (u T , m T ) of the constrained MFG system (1.1) with finite horizon on Ω and proved an existence and uniqueness result for such a system.In [11,12], Cannarsa, Capuani and Cardaliaguet studied the regularity of mild solutions of the constrained MFG system and used such results to give a precise interpretation of (1.1).
At this point, it is natural to raise the question of the asymptotic behavior of solutions as T → +∞.In the absence of state constraints, results describing the asymptotic behavior of solutions of the MFG system were obtained in [13,14], for second order systems on T d , and in [15,16], for first order systems on T d and R d , respectively.Recently, Cardaliaguet and Porretta studied the long time behavior of solutions for the so-called Master equation associated with a second order MFG system, see [17].As is well known, the introduction of state constraints creates serious obstructions to most techniques which can be used in the unconstrained case.New methods and ideas become necessary.
In order to understand the specific features of constrained problems, it is useful to recall the main available results for constrained Hamilton-Jacobi equations.The dynamic programming approach to constrained optimal control problems has a long history going back to Soner [18], who introduced the notion of constrained viscosity solutions as subsolutions in the interior of the domain and supersolutions up to the boundary.Several results followed the above seminal paper, for which we refer to [19,20] and the references therein.
As for the asymptotic behavior of constrained viscosity solutions of ∂ t u(t, x) + H(x, Du(t, x)) = 0, (t, x) ∈ [0, +∞) × Ω we recall the paper [21] by Mitake, where the solution u(t, x) is shown to converge as t → +∞ to a viscosity solution, ū, of the ergodic Hamilton-Jacobi equation (1.2) H(x, Du(x)) = c, x ∈ Ω for a unique constant c ∈ R. In the absence of state constraints, it is well known that the constant c can be characterized by using Mather measures, that is, invariant measures with respect to the Lagrangian flow which minimize the Lagrangian action, see for instance [22].On the contrary, such an analysis is missing for constrained optimal control problems and the results in [21] are obtained by pure PDE methods without constructing a Mather measure.
On the other hand, as proved in [15,16], the role of Mather measures is crucial in the analysis of the asymptotic behavior of solutions to the MFG system on T d or R d .For instance, on T d the limit behavior of u T is described by a solution (c, ū, m) of the ergodic MFG system where m is given by a Mather measure.Then, the fact that ū is differentiable on the support of the Mather measure, allows to give a precise interpretation of the continuity equation in the above system.Motivated by the above considerations, in this paper, we study the ergodic Hamilton-Jacobi equation (1.2) from the point of view of weak KAM theory, aiming at constructing a Mather measure.For this purpose, we need to recover a fractional semiconcavity result for the value function of a constrained optimal control problem, which is inspired by a similar property derived in [12].Indeed, such a regularity is needed to prove the differentiability of a constrained viscosity solution of (1.2) along calibrated curves and, eventually, construct the Mather set.
With the above analysis at our disposal, we address the existence and uniqueness of solutions to the ergodic constrained MFG system As for existence, we construct a triple (c, ū, m) ∈ R ×C(Ω) ×P(Ω) such that ū is a constrained viscosity solution of the first equation in (1.3) for c = c, Dū exists for m-a.e.x ∈ Ω, and m satisfies the second equation of system (1.3) in the sense of distributions (for the precise definition see Definition 4.1).Moreover, under an extra monotonicity assumption for F , we show that c is the unique constant for which the system (1.3) has a solution and F (•, m) is unique.
Then, using energy estimates for the MFG system, we prove our main result concerning the convergence of u T /T : there exists a constant C ≥ such that .
Even for the distribution of players m T we obtain an asymptotic estimate of the form: for some constant C ≥ 0.
We conclude this introduction recalling that asymptotic results for second-order MFG systems on T d have been applied in [13,14] to recover a Turnpike property with an exponential rate of convergence.A similar property, possibly at a lower rate, may then be expected for first-order MFG systems as well.We believe that the results of this paper could be used to the purpose.
The rest of this paper is organized as follows.In Section 2, we introduce the notation and some preliminaries.In Section 3, we provide some weak KAM type results for Hamilton-Jacobi equations with state constraints.Section 4 is devoted to the existence of solutions of (4.1).We show the convergence result of (2.2) in Section 5. • Let f be a real-valued function on R d .The set Denote f ∞,R d by f ∞ and f p,R d by f p , for brevity.
• C b (R d ) stands for the function space of bounded uniformly continuous functions on R d .C 2 b (R d ) stands for the space of bounded functions on R d with bounded uniformly continuous first and second derivatives.C k (R d ) (k ∈ N) stands for the function space of k-times continuously differentiable functions on R d , and We say that a sequence For p ∈ [1, +∞), the Wasserstein space of order p is defined as where x 0 ∈ R d is arbitrary.Given any two measures m and m ′ in P p (R n ), define The Wasserstein distance of order p between m and m ′ is defined by The distance d 1 is also commonly called the Kantorovich-Rubinstein distance and can be characterized by a useful duality formula (see, for instance, [23]) as follows We recall some definitions and results for the constrained MFG system where For any x ∈ Ω, define For any t ∈ [0, T ], denote by e t : Γ → Ω the evaluation map, defined by e t (γ) = γ(t).
For any t ∈ [0, T ] and any η ∈ P(Γ), we define m η t = e t ♯η ∈ P(Ω) where e t ♯η stands for the image measure (or push-forward) of η by e t .Thus, for any ϕ ∈ C(Ω) For any fixed m 0 ∈ P(Ω), denote by P m 0 the set of all Borel probability measures η ∈ P(Γ) such that m η 0 = m 0 .For any η ∈ P m 0 define the following functional where Assume that L ∈ C 1 (Ω ×R d ) is convex with respect to the second argument and satisfies: there are C i > 0, i = 1, 2, 3, 4, such that for all (x, v) ∈ Ω ×R d , there hold Under these assumptions on L, assuming that F : Ω ×P(Ω) → R and u f : Ω → R are continuous functions it has been proved in [10,Theorem 3.1] that there exists at least one constrained MFG equilibrium.

Definition 2.2 (Mild solutions of constrained MFG system)
We say that (u T , m T ) ∈ C([0, T ] × Ω) × C([0, T ]; P(Ω)) is a mild solution of the constrained MFG problem in Ω, if there is a constrained MFG equilibrium η ∈ P m 0 (Γ) such that (i) m T (t) = e t ♯η for all t ∈ [0, T ]; (ii) u T is given by for all (t, x) ∈ [0, T ] × Ω.The existence of a mild solution (u T , m T ) is of constraint MFG system on [0, T ] is a direct consequence of the existence of constrained MFG equilibrium.
In addition, assume that F is strictly monotone, i.e., ) for all x ∈ Ω. Cannarsa and Capuani [10] proved that if Moreover, they also provided examples of coupling functions F for which also the distribution m T is unique under the monotonicity assumption.

2.3.
Weak KAM theory on Euclidean space.In this part we recall some definitions and results in the weak KAM theory on the Euclidean space.Most of the results are due to Fathi [24] and Contreras [25].
• Tonelli Lagrangians and Hamiltonians.Let L : R n × R n → R be a C 2 Lagrangian.Definition 2.3 (Strict Tonelli Lagrangians) L is called a strict Tonelli Lagrangian if there exist positive constants Remark 2.4.Let L be a strict Tonelli Lagrangian.It is easy to check that there are two positive constants α, β depending only on

Define the Hamiltonian
It is straightforward to check that if L is a strict Tonelli Lagrangian, then H satisfies (a), (b), and (c) in Definition 2.3.Such a function H is called a strict Tonelli Hamiltonian.

We always work with Tonelli Lagrangians and Hamiltonians, if not stated otherwise.
• Invariant measures and holonomic measures.The Euler-Lagrange equation associated with It should be noted that, for any Tonelli Lagrangian L, the flow We recall that a Borel probability measure We denote by M L the set of all φ L t -invariant Borel probability measures on endowed with the topology induced by the uniform convergence on compact subsets.Denote by (C 0 l ) ′ the dual of C 0 l .Let γ : [0, T ] → R d be a closed absolutely continuous curve for some T > 0. Define a probability measure µ γ on the Borel σ-algebra of ) denote the set of such µ γ 's.We call K(R d ) the set of holonomic measures, where K(R d ) denotes the closure of K(R d ) with respect to the topology induced by the weak convergence on (C 0 l ) ′ .By [26, 2-4.1 Theorem], we have that absolutely continuous curve, we define its L action as The critical value of the Lagrangian L, which was introduced by Mañé in [27], is defined as follows: (2.7) c L := sup{k ∈ R : A L+k (γ) < 0 for some closed absolutely continuous curve γ}.
Since R d can be seen as a covering of the torus T d , Mañé's critical value has the following representation formula [28, Theorem A]: (2.8) By [26, 2-5.2 Theorem], c L can be also characterized in the following way: (2.9) where the action B L is defined as If L is a reversible Tonelli Lagrangian, and which, together with (2.6) and (2.9), implies that In view of (2.10), it is straightforward to see that • Weak KAM theorem.Let us recall definitions of weak KAM solutions and viscosity solutions of the Hamilton-Jacobi equation where c is a real number.Definition 2.5 (Weak KAM solutions)

is called a backward weak KAM solution of equation (HJ) with c = c L , if it satisfies the following two conditions: (i) for each continuous and piecewise
It is not difficult to check that if u satisfies condition (i) in Definition 2.5, then the curves appeared in condition (ii) in Definition 2.5 are necessarily (u, L, c L )-calibrated curves.
(iii) u is a viscosity solution of equation (HJ) on R d if it is both a viscosity subsolution and a viscosity supersolution on R d .In [24] Fathi and Maderna got the existence of backward weak KAM solutions (or, equivalently, viscosity solutions) for c = c L .

HAMILTON-JACOBI EQUATIONS WITH STATE CONSTRAINTS
3.1.Constrained viscosity solutions.Let us recall the notion of constrained viscosity solutions of equation (HJ) on Ω, see for instance [18].Definition 3.1 (Constrained viscosity solutions) u ∈ C(Ω) is said to be a constrained viscosity solution of (HJ) on Ω if it is a subsolution on Ω and a supersolution on Ω.
Consider the state constraint problem for equation (HJ) on Ω: Mitake [21] showed that there exists a unique constant, denoted by c H , such that problem (3.1)-(3.2) admits solutions.Moreover, c H can be characterized by Furthermore, by standard comparison principle for viscosity solutions it is easy to prove that following representation formula for constrained viscosity solutions holds true.
Hence, we get that By exchanging the roles of x and y, we get that where (i) For any x ∈ Ū and C > 0, we say that y ∈ Ū is a (x, C)-reachable in U, if there exists a curve γ ∈ AC([0, τ (x, y)]; Ū) for some τ (x, y) > 0 such that | γ(t)| ≤ 1 a.e. in [0, τ (x, y)], γ(0) = x, γ(τ (x, y)) = y and τ (x, y) ≤ C|x − y|.We denote by R C (x, y) the set of all (x, C)reachable points from x ∈ U. We say that U is C-quasiconvex if for any x ∈ Ū we have that R C (x, U) = U.
(ii) U is called a Lipschitz domain, if ∂U is locally Lipschitz, i.e., ∂U can be locally represented as the graph of a Lipschitz function defined on some open ball of R d−1 .
(iii) U is C-quasiconvex for some C > 0 if U is a bounded Lipschitz domain (see, for instance, Sections 2.5.1 and 2.5.2 in [30]).Since Ω is a bounded domain with C 2 boundary, then it is C-quasiconvex for some C > 0. Recalling the characterization (3.3), since ϕ is differentiable almost everywhere and H is a reversible Tonelli Hamiltonian we have that

Consider the assumption
Therefore, by (A1) and the fact that where the last equality holds by (2.11).
From now on, we assume that L is a reversible Tonelli Lagrangian and denote by c the common value of c L and c H . Remark 3.7.Comparing to the classical weak KAM solutions, see Definition 2.5, one can call u : Ω → R a constrained weak KAM solution if for any t 1 < t 2 and any absolutely continuous curve γ and, for any x ∈ Ω there exists a One can easily see that a constrained weak KAM solution must be a constrained viscosity solution by definition and Proposition 3.2.In order to prove the opposite relation we need to go back to the C 1,1 regularity of solutions of the Hamiltonian system associated with a state constraint control problem.This will be the subject of a forthcoming paper.

Semiconcavity estimates of constrained viscosity solutions.
Here, we give a semiconcavity estimate for constrained viscosity solutions of (1.2).Note that a similar result has been obtained in [12,Corollary 3.2] for a general calculus of variation problem under state constraints with a Tonelli Lagrangian and a regular terminal cost.Such regularity of the data allowed the authors to prove the semiconcavity result using the maximum principle which is not possible in our context: indeed, by the representation formula (3.4), i.e.
one can immediately observe that the terminal cost is not regular enough to apply the maximum principle in [11,Theorem 3.1].For these reasons, we decided to prove semiconcavity using a dynamical approach based on the properties of calibrated curves.
Let Γ t x,y (Ω) be the set of all absolutely continuous curve γ : [0, t] → Ω such that γ(0) = x and γ(t) = y.For each x, y ∈ Ω, t > 0, let We recall that, since the boundary of Ω is of class C 2 , there exists ρ 0 > 0 such that Proposition 3.9.For any x * ∈ Ω, the functions A Ω,L t (•, x * ) and A Ω,L t (x * , •) both are locally semiconcave with fractional modulus.More precisely, there exists C > 0 such that, if h ∈ R d with x ± h ∈ Ω, then we have In particular, for such an h, we also have where u is the constrained viscosity solution defined by (1.2).
Proof.Here we only study the semi-concavity of the function We divide the proof into several steps.
x,x * (Ω) be a minimizer for A t (x, x * ), and ε > 0. For r ∈ (0, ε/2], define This implies d Ω (γ ± (s)) ≤ ρ 0 for all s ∈ [0, t].Denote by γ ± the projection of γ ± onto Ω, that is From our construction of γ ± , it is easy to see that γ ± (0) = x±h and for all s ∈ [0, t], Moreover, in view of Lemma 3.8, we conclude that for almost all s ∈ [0, r] To proceed with the proof we need estimates for The first one is easily obtained by (3.6).That is To obtain the second estimate, notice that and It is enough to give the estimate of We will only give the estimate for ds since the other is similar.Recalling Lemma 3.8 we conclude that for s ∈ [0, r], In view of the fact that D 2 b Ω (x), Db Ω (x) = 0 for all x ∈ Σ ρ 0 , we have that Db Ω (γ + (s)), γ+ (s) 2 1 Recall γ ∈ C 1,1 ([0, T, Ω]), then | γ(s)| is uniformly bounded by a constant independent of h and r.It follows there exists C 1 > 0 such that In view of Lemma 3.8, we obtain that We observe that the set {s ∈ [0, r] : Integrating by parts we conclude that Therefore Combing the two estimates on I 1 and I 2 , we have that III. Fractional semiconcavity estimate of A t (x, x * ).From the previous estimates we have that Now, let γ ∈ Γ t x,x * (Ω) be a minimizer for A t (x, x * ).Thus, Owing to (3.6), we have On the other hand Therefore, taking r = |h| . IV. Fractional semiconcavity estimate for u defined in (3.4).By using the fundamental solution and fix t = 1, we have Thus the required semiconcavity estimate follow by the relation where the infimum above achieves at y = y * .3.3.Differentiability of constrained viscosity solutions.Let u be a constrained viscosity solution of (HJ) on Ω. Recall that, we call γ . Proposition 3.10 (Differentiability property I).Let u be a constrained viscosity solution of (HJ) on Ω.Let x ∈ Ω be such that there exists a (u, L, c)-calibrated curve γ : [−τ, τ ] → Ω such that γ(0) = x, for some τ > 0.Then, u is differentiable at x. Proposition 3.11 (Differentiability property II).Let u be a constrained viscosity solution of (HJ) on Ω.Let x ∈ ∂ Ω be such that there exists a (u, L, c)-calibrated curve γ : [−τ, τ ] → Ω such that γ(0) = x, for some τ > 0. For any direction y tangential to Ω at x, the directional derivative of u at x in the direction y exists.
We omit the proof of Proposition 3.10 here since it follows by standard arguments, as for the case without the constraint Ω, and for this we refer to [22,Theorem 4.11.5].Moreover, we prove Proposition 3.11 in Appendix A, for the reader convenience, since it is given by a combination of the arguments in [22,Theorem 4.11.5] and the so-called projection method.
Let x ∈ ∂ Ω be such that there exists a (u, L, c)-calibrated curve γ : [−τ, τ ] → Ω with γ(0) = x for some τ > 0. From Proposition 3.11, for any y tangential to Ω at x, we have Thus, one can define the tangential gradient of u at x by Given x ∈ ∂ Ω, each p ∈ D + u(x) can be written as where p ν = p, ν(x) ν(x), and p τ is the tangential component of p, i.e., p τ , ν(x) = 0.
By similar arguments to the one in [12, Proposition 2.5, Proposition 4.3 and Theorem 4.3] and by Proposition 3.11 it is easy to prove the following result: Proposition 3.12, Corollary 3.13 and Proposition 3.14.Proposition 3.12.Let x ∈ ∂ Ω and u : Ω → R be a Lipschitz continuous and semiconcave function.Then, and denotes the one-sided derivative of u at x in direction −ν.
By the properties of the Legendre transform, the above equality yields In order to prove (ii), proceeding as above by Proposition 3.11 and Proposition 3.14 we get Hence, we obtain that This completes the proof.
It is clear that the set MΩ is nonempty under assumption (A1) and we call MΩ the Mather set associated with the Tonelli Lagrangian L. Note that since L is reversible.Hence, it is straightforward to check that the constant curve at x is a minimizing curve for the action A L (•), where x ∈ M Ω := π 1 MΩ .We call M Ω the projected Mather set.Definition 3.16 (Mather measures) Let µ ∈ P(Ω ×R d ).We say that µ is a Mather measure for a reversible Tonelli Lagrangian L, if μ ∈ M L and spt(µ) ⊂ MΩ , where μ is defined by μ(B) : Remark 3.17.Let x ∈ M Ω .Let u be a constrained viscosity solution of (HJ) on Ω.
(ii) Let γ(t) ≡ x, t ∈ R. Note that u(γ(t ′ )) − u(γ(t)) = 0 for all t ≤ t ′ and that where the last equality comes from (2.11).Hence, the curve γ is a (u, L, c)-calibrated curve.(iii) By Theorem 3.15, we have that Proposition 3.18.Let u be a constrained viscosity solution of (HJ) on Ω.The function is Lipschitz with a Lipschitz constant depending only on H and Ω, where Proof.In view of (3.10) and the properties of the Legendre Transform, for any x, y ∈ M Ω we have where K 2 > 0 is a constant depending only on H and Ω.

ERGODIC MFG WITH STATE CONSTRAINTS
By the results proved so far the good candidate limit system for the MFG system (2.2) is the following and λ + is defined in Proposition 3.12.
4.1.Assumptions.From now on, we suppose that L is a reversible strict Tonelli Lagrangian on R d .Let F : R d × P(Ω) → R be a function, satisfying the following assumptions: (F1) for every measure m ∈ P(Ω) the function (F3) there is a constant C F > 0 such that for every m 1 , m 2 ∈ P(Ω), Note that assumption (A2) is MFG counterpart of assumption (A1) which guarantees, as we will see, that for any measure m the Mather set associated with the Lagrangian L(x, v) + F (x, m) is non-empty.(iii) m is a projected minimizing measure, i.e., there is a minimizing measure η m for L m such that m = π 1 ♯η m; (iv) m satisfies the second equation of system (4.1) in the sense of distributions, that is, where the vector field V is related to ū in the following way: if x ∈ Ω ∩ spt( m), then Dū(x) exists and We denote by S the set of solutions of system (4.1) and Theorem 4.4 below guarantees the nonemptiness of such set.
By assumption (F1), it is clear that for any given m ∈ P(Ω), L m (resp.H m ) is a reversible strict Tonelli Lagrangian (resp.Hamiltonian).So, in view of (A2), all the results recalled and proved in Section 3 still hold for L m and H m .
In view of Proposition 3.6, for any given m ∈ P(Ω), we have c Hm = c Lm .Denote the common value of c Hm and c Lm by λ(m).
(ii) Assume, in addition, (F3).Let (c H m1 , ū1 , m1 ), (c H m2 , ū2 , m2 ) ∈ S.Then, Proof of Theorem 4.4.The existence result (i) follows by the application of the Kakutani fixed point theorem.Indeed, by the arguments in Section 3, for any m ∈ P(Ω), there is a minimizing measure η m associated with L m .Thus we can define a set-valued map as follows where Ψ(m) := {π 1 ♯η m : η m is a minimizing measure for L m } .
Then, a fixed point m of Ψ is a solution in the sense of distributions of the stationary continuity equation and there exists a constrained viscosity solution associated with H m by [21].For more details see for instance [15,Theorem 3].
, ū2 , m2 ) ∈ S. Given any T > 0, define the following sets of curves: One can define Borel probability measures on Γ by By definition, it is direct to see that spt(µ i ) ⊂ M mi and Given any x 0 ∈ spt( m1 ), let γ 1 denote the constant curve t → x 0 , then for any t > 0 we have that By integrating the above inequality on Γ with respect to µ 1 , we get that In view of Fubini Theorem and (4.4), we deduce that Exchanging the roles of m1 and m2 , we obtain that Hence, we get that Recalling assumption (F3), we deduce that F (x, m2 ) = F (x, m1 ) for all x ∈ Ω and thus c H m1 = c H m2 .
Remark 4.6.Note that even though the uniqueness result is a consequence of the classical Lasry-Lions monotonicity condition for MFG system, our proof here differs from the one in [15] and in [16]: indeed, in our setting the stationary continuity equation has different vector fields depending on the mass of the measure in Ω and the mass on ∂ Ω.This is why we addressed the problem representing the Mather measures associated with the system through measures supported on the set of calibrated curves.

CONVERGENCE OF MILD SOLUTIONS OF THE CONSTRAINED MFG PROBLEM
This section is devoted to the long-time behavior of first-order constrained MFG system (2.2).We will assume (F1), (F2), (F3), (A2), and the following additional conditions: 2 , where g : P(Ω) → R is Lipschitz continuous with respect to the d 1 distance and f : Ω → R is such that f −1 is Lipschitz continuous.Thus the minimum is reached at f (x) = −g(m) and by the assumptions on f and g such minimum has a Lipschitz depends with respect to m ∈ P(Ω).

5.1.
Convergence of mild solutions.In order to get the convergence result of mild solutions of system (2.2), we prove two preliminary results first.

Lemma 5.3 (Energy estimate).
There exists a constant κ ≥ 0 such that for any mild solution (u T , m η t ) of constrained MFG system (2.2) associated with a constrained MFG equilibrium η ∈ P m 0 (Γ), and any solution (ū, λ, m) of constrained ergodic MFG system (4.1),there holds where Γ * η is as in Definition 2.1.First, we consider term A: Since η is a constrained MFG equilibrium associated with m 0 , then any curve γ ∈ spt(η) satisfies the following equality In view of (3.4) with L = L m, one can deduce that Hence, we have that By Proposition 3.4 we estimate the second term of the right-hand side of the above inequality as follows: where In view of Lemma 4.3 and the compactness of P(Ω), K 2 is well-defined and depends only on L, F and Ω.For the first term, since u f is bounded on Ω, we only need to estimate u T (0, γ(0)) where γ ∈ spt(η).
Part I: We denote by B + the set B r (x 0 ) ∩ {x ∈ R d : x d ≥ 0} and by B − the complement of B + .Let f denote the following extension of f in B r (x 0 ): Let χ r denote a cut-off function such that χ r (x) = 1 for x ∈ B r (x 0 ), χ r (x) = 0 for x ∈ R d \B 2r (x 0 ) and 0 ≤ χ r (x) ≤ 1 for x ∈ B 2r (x 0 )\B r (x 0 ) and let fr be the extension of f on R d , i.e. fr (x) := f • χ r (x).Moreover, for any δ > 0 we consider a cover of B r (x 0 ) through cubes of length δ denoted by Q δ .Then, by construction we have that for any cube for some constant C(δ) ≥ 0. Therefore, applying Lemma 4 in [15] we get 2,B + .Thus, recalling that by construction B + ≡ Ω we obtain that by (5.4) 2,Ω .Part II: Let x 0 ∈ ∂ Ω be such that Ω is not flat in a neighborhood of x 0 , that is we are in case I.Then, we can find a C 1 mapping Φ, with inverse given by Ψ such that changing the coordinate system according to the map Φ we obtain that Ω ′ := Φ(Ω) is flat in a neighborhood of x 0 .Proceeding similarly as in Part I, we define Thus, if we set y = Φ(x), we have that x = Ψ(y), and if we define f ′ (y) = f (Ψ(y)) then by Parti I we get 2,Ω ′ which implies, returning to the original coordinates, that 2,Ω for a general domain Ω not necessarily flat in a neighborhood of x 0 ∈ ∂ Ω.
Since Ω is compact, there exists a finitely many points x 0 i ∈ ∂ Ω, neighborhood W i is x 0 i and functions f ′ i defined as before for i = 1, . . ., N, such that, fixed W 0 ⊂ Ω, we have Ω ⊂ N i=1 W i .Furthermore, let {ζ i } i=1,...,N be a partition of unit associated with {W i } i=1,...,N and define f (x) = N i=1 ζ i f ′ i (x).Then, by (5.5) applied to f we get the conclusion.Theorem 5.5 (Convergence of mild solutions of (2.2)).For each T > 1, let (u T , m η t ) be a mild solution of (2.2).Let ( λ, ū, m) ∈ S.Then, there exists a positive constant C ′ such that (5.6) sup , where C ′ depends only on L, F , u f and Ω.
Proof.Let v(x) = ū(x) − ū(0) and define Since ( λ, ū, m) is a solution of (4.1), one can deduce that w is a constrained viscosity solution of the Cauchy problem in Ω .

1 .
Notation.We write below a list of symbols used throughout this paper.• Denote by N the set of positive integers, by R d the d-dimensional real Euclidean space, by •, • the Euclidean scalar product, by | • | the usual norm in R d , and by B R the open ball with center 0 and radius R. • Let Ω ⊂ R d be a bounded open set with C 2 boundary.Ω stands for its closure, ∂ Ω for its boundary and Ω c = R d \ Ω for the complement.For x ∈ ∂ Ω, denote by ν(x) the outward unit normal vector to ∂ Ω at x. • The distance function from Ω is the function d Ω : R d → [0, +∞) defined by d Ω (x) := inf y∈Ω |x − y|.Define the oriented boundary distance from ∂ Ω by b Ω (x) := d Ω (x) − d Ω c (x).Since the boundary of Ω is of class C 2 , then b Ω (•) is of class C 2 in a neighborhood of ∂ Ω. • Denote by π 1 : Ω ×R d → Ω the canonical projection.• Let Λ be a real n × n matrix.Define the norm of Λ by Λ = sup |x|=1, x∈R d |Λx|.

Proposition 3 . 2 (
Representation formula for constrained viscosity solutions).u ∈ C(Ω) is a constrained viscosity solution of (HJ) on Ω for c = c H if and only if H depending only on H and Ω. Remark 3.5.Let U ⊆ R d be a connected open set.

Proposition 3 . 6 .
Let H be a reversible Tonelli Hamiltonian.Assume (A1).Then, c H = c L .Proof.By [24, Theorem 1.1], there exists a global viscosity solution u d of equation (HJ) with c = c L .Since Ω is an open subset of R d , by definition u d Ω is solution of (3.1) for c = c L .Thus, c H ≤ c L .

Definition 4 . 2 (
Mean field Lagrangians and Hamiltonians)Let H be the reversible strict Tonelli Hamiltonian associated with L. For any m ∈ P(Ω), define the mean field Lagrangian and Hamiltonian associated with m by

Lemma 4 . 3 (
Lipschitz continuity of the critical value).The function m → λ(m) is Lipschitz continuous on P(Ω) with respect to the metric d 1 , where the Lipschitz constant depends on F only.Since the characterization (3.3) holds true, the proof of this result is an adaptation of [15, Lemma 1].

Remark 4 . 5 .
By (ii)  inTheorem 4.4, it is clear that each element of S has the form ( λ, ū, m), where m is a projected minimizing measure and λ denotes the common Mañé critical value of H m.
the Borel σ-algebra on Ω, by P(Ω) the set of Borel probability measures on Ω, by P(Ω ×R d ) the set of Borel probability measures on Ω ×R d .P(Ω) and P(Ω ×R d ) are endowed with the weak- * topology.One can define a metric on P(Ω) by (2.1) below, which induces the weak- * topology.
2.2.Measure theory and MFG with state constraints.Denote by B(R d ) the Borel σ-algebra on R d and by P(R d ) the space of Borel probability measures on R d .The support of a measure µ ∈ P(R n ), denoted by spt(µ), is the closed set defined by Remark 3.3.If u is a constrained viscosity solution of (HJ) on Ω for c = c H , then by Proposition 3.2 and [21, Theorem 5.2] one can deduce that u ∈ W 1,∞ (Ω).Thus, u is Lipschitz in Ω, since Ω is open and bounded with ∂Ω of class C 2 (see, for instance, [29, Chapter 5]).Proposition 3.4 (Equi-Lipschitz continuity of constrained viscosity solutions).Let u be a constrained viscosity solution of (HJ) on Ω for c = c H .Then, u is Lipschitz continuous on Ω with a Lipschitz constant K 1 > 0 depending only on H.