Comparison Principle for Hamilton-Jacobi-Bellman Equations via a Bootstrapping Procedure

. We study the well-posedness of Hamilton–Jacobi–Bellman equations on subsets of R d in a context without boundary conditions. The Hamiltonian is given as the supremum over two parts: an internal Hamiltonian depending on an external control variable and a cost functional penalizing the control. The key feature in this paper is that the control function can be unbounded and discontinuous. This way we can treat functionals that appear e.g. in the Donsker–Varadhan theory of large deviations for occupation-time measures. To allow for this ﬂexibility, we assume that the internal Hamiltonian and cost functional have controlled growth, and that they satisfy an equi-continuity estimate uniformly over compact sets in the space of controls. In addition to establishing the comparison principle for the Hamilton–Jacobi–Bellman equation, we also prove existence, the viscosity solution being the value function with exponentially discounted running costs. As an application, we verify the conditions on the internal Hamiltonian and cost functional in two examples.


Introduction and aim of this note
The main purpose of this note is to establish well-posedness for first-order nonlinear partial differential equations of Hamilton-Jacobi-Bellman type on subsets E of R d , in the context without boundary conditions and where the Hamiltonian flow generated by H remains inside E. In (HJB), λ > 0 is a scalar and h is a 0123456789().: V,-vol R. C. Kraaij  where θ ∈ Θ plays the role of a control variable. For fixed θ, the function Λ can be interpreted as an Hamiltonian itself. We call it the internal Hamiltonian. The function I can be interpreted as the cost of applying the control θ.
The main result of this paper is the comparison principle for (HJB) in order to establish uniqueness of viscosity solutions. The standard assumption in the literature that allows one to obtain the comparison principle in the context of optimal control problems (e.g. [2] for the first order case and [10] for the second order case) is that either there is a modulus of continuity ω such that |H(x, p) − H(y, p)| ≤ ω (|x − y|(1 + |p|)) , (1.2) or that H is uniformly coercive: More generally, the two estimates (1.2) and (1.3) can be combined in a single estimate, called pseudo-coercivity, see [4, (H4), Page 34], that uses the fact that the sub-and supersolution properties roughly imply that the estimate (1.2) only needs to hold for appropriately chosen x, y and p such that H is finite uniformly over these chosen x, y, p.
The pseudo-coercivity property is harder to translate as in this way the control on H does not necessarily imply the same control on Λ, in particular in the case when I is unbounded. We return on this issue below.
The estimates (I) and (II) are not satisfied for Hamiltonians arising from natural examples in the theory of large deviations [12,13] for Markov processes with two scales (see e.g. [6,18,27,29] for PDE's arising from large deviations with two scales, see [3,16,17,20,21] for other works connection PDE's with large deviations). Indeed, in [6] the authors mention that well-posedness of the Hamilton-Jacobi-Bellman equation for examples arising from large deviation theory is an open problem. Recent generalizations of the coercivity condition, see e.g. [9], also do not cover these examples.
In the large deviation context, however, we typically know that we have the comparison principle for the Hamilton-Jacobi equation in terms of Λ. In addition, even though I might be discontinuous, we do have other types of regularity for the functional I, see e.g. [32]. Thus, we aim to prove a comparison principle for (HJB) on the basis of the assumption that we have the following natural relaxations of (or the pseudo-coercive version of) (I) and (II).
(i) For θ ∈ Θ, define the Hamiltonian H θ (x, p) := Λ(x, p, θ). We have an estimate on H θ that is uniform over θ in compact sets K ⊆ Θ. This estimate, for one fixed θ, is in spirit similar to the pseudo-coercivity estimate of [4] and is morally equivalent to the comparison principle for H θ . The uniformity is made rigorous as the continuity estimate in Assumption 2.14 (Λ5) below. (ii) The cost functional I(x, θ) satisfies an equi-continuity estimate of the type |I(x, θ) − I(y, θ)| ≤ ω I,C (|x − y|) on sublevel sets {I ≤ C} which we assume to be compact. This estimate is made rigorous in Assumption 2.15 (I5) below.
To work with these relaxations, we introduce a procedure that allows us to restrict our analysis to compact sets in the space of controls. In the proof of the comparison principle, the sub-and supersolution properties give boundedness of H when evaluated in optimizing points. We then translate this boundedness to boundedness of I, which implies that the controls lie in a compact set.
The transfer of control builds upon (i) for Λ(x, p, θ 0 x ) when we use a control θ 0 x that satisfies I(x, θ 0 x ) = 0. This we call the bootstrap procedure: we use the comparison principle for the Hamilton-Jacobi equation in terms of Λ(x, p, θ 0 x ) to shift the control on H to control on Λ and I for general θ. That R. C. Kraaij and M. C. Schlottke NoDEA this sketch, we refrain from performing localization arguments that are needed for non-compact E.
Thus, to summarize, we use the growth conditions posed on Λ and I and the pseudo-coercivity estimate for Λ to transfer the control on the full Hamiltonian H to the functions Λ and the cost function I. Then the control on Λ and I allows us to apply the estimates (i) and (ii) to obtain the comparison principle.
Next to our main result, we also state for completeness an existence result in Theorem 2.8. The viscosity solution will be given in terms of a discounted control problem as is typical in the literature, see e.g. [2,Chapter 3]. Minor difficulties arise from working with H that arise from irregular I.
Finally, we show that the conditions (i) to (vi) are satisfied in two examples that arise from large deviation theory for two-scale processes. In our companion paper [26], we will use existence and uniqueness for (HJB) for these examples to obtain large deviation principles.

Illustration in the context of an example
As an illustrating example, we consider a Hamilton-Jacobi-Bellman equation that arises from the large deviations of the empirical measure-flux pair of weakly coupled Markov jump processes that are coupled to fast Brownian motion on the torus. We skip the probabilistic background of this problem (See [26]), and come to the set-up relevant for this paper.
Let G := {1, . . . , q} be some finite set, and let is the set of probability measures on G. Let F = P(S 1 ) be the set of probability measures on the one-dimensional torus. We introduce Λ and I.
• Let r : G × G × P(E) × P(S 1 ) → [0, ∞) be some function that codes the P(E) × P(S 1 ) dependent jump rate of the Markov jump process over each bond (a, b) ∈ Γ. The internal Hamiltonian Λ is given by • Let σ 2 : S 1 × P(G) → (0, ∞) be a bounded and strictly positive function. The cost function I : E × Θ → [0, ∞] is given by Aiming for the comparison principle, we note that classical methods do not apply. The functionals Λ are not coercive and do not satisfy (I). We show in "Appendix E" that they are also not pseudo-coercive as defined in [4]. The functional I is neither continuous nor bounded. Once can check e.g. that if θ is a finite combination of Dirac measures, then I(μ, θ) = ∞. We show in Sect. 5, however, that (i) to (vi) hold, implying the comparison principle for the Hamilton-Jacobi-Bellman equations. The verification of these properties is based in part on results from [23,32].

Summary and overview of the paper
To summarize, our novel bootstrap procedure allows to treat Hamilton-Jacobi-Bellman equations where: • We assume that the cost function I satisfies some regularity conditions on its sub-levelsets, but allow I to be possibly unbounded and discontinuous. • We assume that Λ satisfies the continuity estimate uniformly for controls in compact sets, which in spirit extends the pseudo-coercivity estimate of [4]. This implies that Λ can be possibly non-coercive, non-pseudo-coercive and non-Lipschitz as exhibited in our example above.
In particular, allowing discontinuity in I allows us to treat the comparison principle for examples like the one we considered above, which so far has been out of reach. We believe that the bootstrap procedure we introduce in this note has the potential to also apply to second order equations or equations in infinite dimensions. Of interest would be, for example, an extension of the results of [10] who work with continuous I. For clarity of the exposition, and the already numerous applications for this setting, we stick to the finite-dimensional firstorder case. We think that the key arguments that are used in the proof in Sect. 3 do not depend in a crucial way on this assumption.
The paper is organized as follows. The main results are formulated in Sect. 2. In Sect. 3 we establish the comparison principle. In Sect. 4 we establish that a resolvent operator R(λ) in terms of an exponentially discounted control problem gives rise to viscosity solutions of the Hamilton-Jacobi-Bellman equation (HJB). Finally, in Sect. 5 we treat two examples including the one mentioned in the introduction.

Main results
In this section, we start with preliminaries in Sect. 2.1, which includes the definition of viscosity solutions and that of the comparison principle.
We proceed in Sect. 2.2 with the main results: a comparison principle for the Hamilton-Jacobi-Bellman equation (HJB) based on variational Hamiltonians of the form (1.1), and the existence of viscosity solutions. In Sect. 2.3 we collect all assumptions that are needed for the main results.

Preliminaries
For a Polish space X we denote by C(X ) and C b (X ) the spaces of continuous and bounded continuous functions respectively. If X ⊆ R d then we denote by C ∞ c (X ) the space of smooth functions that vanish outside a compact set. We denote by C ∞ cc (X ) the set of smooth functions that are constant outside of a compact set in X , and by P(X ) the space of probability measures on X . We equip P(X ) with the weak topology induced by convergence of integrals against bounded continuous functions.
Throughout the paper, E will be the set on which we base our Hamilton-Jacobi equations. We assume that E is a subset of R d that is a Polish space which is contained in the R d closure of its R d interior. This ensures that gradients of functions are determined by their values on E. Note that we do not necessarily assume that E is open. We assume that the space of controls Θ is Polish.
We next introduce viscosity solutions for the Hamilton-Jacobi equation with Hamiltonians like H(x, p) of our introduction.

Definition 2.1. (Viscosity solutions and comparison principle) Let
Consider the Hamilton-Jacobi equation We say that u is a (viscosity) subsolution of equation ( We say that v is a (viscosity) supersolution of Eq. (2.1) if v is bounded from below, lower semi-continuous and if, for every f ∈ D(A)there exists a sequence We say that u is a (viscosity) solution of Eq.
A similar simplification holds in the case of supersolutions.

Remark 2.4.
For an explanatory text on the notion of viscosity solutions and fields of applications, we refer to [8].
Remark 2.5. At present, we refrain from working with unbounded viscosity solutions as we use the upper bound on subsolutions and the lower bound on supersolutions in the proof of Theorem 2.6. We can, however, imagine that the methods presented in this paper can be generalized if u and v grow slower than the containment function Υ that will be defined below in Definition 2.13.

Main results: comparison and existence
Then R(λ)h is the unique viscosity solution to f − λHf = h. Remark 2.9. The form of the solution is typical, see for example Section III.2 in [2]. It is the value function obtained by an optimization problem with exponentially discounted cost. The difficulty of the proof of Theorem 2.8 lies in treating the irregular form of H.

Assumptions
In this section, we formulate and comment on the assumptions imposed on the Hamiltonians defined in the previous sections. The key assumptions were already mentioned in the sketch of the bootstrap method in the introduction. To these, we add minor additional assumptions on the regularity of Λ and I in Assumptions 2.14 and 2.15. Finally, Assumption 2.17 will imply that even if E has a boundary, no boundary conditions are necessary for the construction of the viscosity solution.
We start with the continuity estimate for Λ, which was briefly discussed in (i) in the introduction. To that end, we first introduce a function that is used in the typical argument that doubles the number of variables. Definition 2.10. (Penalization function) We say that Ψ : and if x = y if and only if Ψ(x, y) = 0.
We will apply the definition below for G = Λ. Definition 2.11. (Continuity estimate) Let Ψ be a penalization function and let G : Suppose that for each ε > 0, there is a sequence of positive real numbers α → ∞. For sake of readability, we suppress the dependence on ε in our notation.
(2.6) Remark 2.12. In "Appendix C", we state a slightly more general continuity estimate on the basis of two penalization functions. A proof of a comparison principle on the basis of two penalization functions was given in [23].
The continuity estimate is indeed exactly the estimate that one would perform when proving the comparison principle for the Hamilton-Jacobi equation in terms of the internal Hamiltonian (disregarding the control θ). Typically, the control on (x ε,α , y ε,α ) that is assumed in (C1) and (C2) is obtained from choosing (x ε,α , y ε,α ) as optimizers in the doubling of variables procedure (see Lemma 3.5), and the control that is assumed in (C3) is obtained by using the viscosity sub-and supersolution properties in the proof of the comparison principle. The required restriction to compact sets in Lemma 3.5 is obtained by including in the test functions a containment function. • For every c ≥ 0, the set {x | Υ(x) ≤ c} is compact; To conclude, our assumption on Λ contains the continuity estimate, the controlled growth, the existence of a containment function and two regularity properties.
Assumption 2.14. The function Λ : E × R d × Θ → R in the Hamiltonian (2.2) satisfies the following.
For any x ∈ E and θ ∈ Θ, the map p → Λ(x, p, θ) is convex. We have Λ(x, 0, θ) = 0 for all x ∈ E and all θ ∈ Θ. (Λ3) There exists a containment function Υ : E → [0, ∞) for Λ in the sense of Definition 2.13. (Λ4) For every compact set K ⊆ E, there exist constants M, C 1 , C 2 ≥ 0 such that for all x ∈ K, p ∈ R d and all θ 1 , θ 2 ∈ Θ, we have (I5) For every compact set K ⊆ E and each M ≥ 0 the collection of functions {I(·, θ)} θ∈ΩK,M is equicontinuous. That is: for all ε > 0, there is a δ > 0 such that for all θ ∈ Ω K,M and x, y ∈ K such that d(x, y) ≤ δ we have To establish the existence of viscosity solutions, we will impose one additional assumption. For a general convex functional p → Φ(p) we denote Assumption 2.17. The set E is closed and convex. The map Λ is such that In Lemma 4.1 we will show that the assumption implies that ∂ p H(x, p) ⊆ T E (x), which in turn implies that the solutions of the differential inclusion in terms of ∂ p H(x, p) remain inside E. Motivated by our examples, we work with closed convex domains E. While in this context we can apply results from e.g. Deimling [11], we believe that similar results can be obtained in different contexts.
is intuitively implied by the comparison principle for H and therefore, we expect it to hold in any setting for which Theorem 2.6 holds. Here, we argue in a simple case why this is to be expected. First of all, note that the comparison principle for H builds upon the maximum principle.
. As x = 0 is a boundary point, we conclude that f (0) ≤ g (0). If indeed the maximum principle holds, we must have

The comparison principle
In this section, we establish Theorem 2.6. To establish the comparison principle for f − λHf = h we use the bootstrap method explained in the introduction. We start by a classical localization argument.
We carry out the localization argument by absorbing the containment function Υ from Assumption 2.14 (Λ3) into the test functions. This leads to two new operators, H † and H ‡ that serve as an upper bound and a lower bound for the true H. We will then show the comparison principle for the Hamilton-Jacobi equation in terms of these two new operators. We therefore have to extend our notion of Hamilton-Jacobi equations and the comparison principle. This extension of the definition is standard, but we included it for completeness in the appendix as Definition A.1.
This procedure allows us to clearly separate the reduction to compact sets on one hand, and the proof of the comparison principle on the basis of the bootstrap procedure on the other. Schematically, we will establish the following diagram: In this diagram, an arrow connecting an operator A with operator B with subscript 'sub' means that viscosity subsolutions of f − λAf = h are also viscosity subsolutions of f − λBf = h. Similarly for arrows with a subscript 'super'.
We introduce the operators H † and H ‡ in Sect. 3.1. The arrows will be established in Sect. 3.2. Finally, we will establish the comparison principle for H † and H ‡ in Sect. 3.3. Combined these two results imply the comparison principle for H.
Proof of Theorem 2.6. We start with the proof of (a). Let f ∈ D(H). Then Hf is continuous since by Proposition B.3 in "Appendix B", the Hamiltonian H is continuous.
We proceed with the proof of (b). Fix h 1 , h 2 ∈ C b (E) and λ > 0. Let u 1 , u 2 be a viscosity sub-and supersolution to f − λHf = h 1 and f − λHf = h 2 respectively. By Lemma 3.3 proven in Sect. 3.2, u 1 and u 2 are a sub-and supersolution to

Definition of auxiliary operators
In this section, we repeat the definition of H, and introduce the operators H † and H ‡ .
We proceed by introducing H † and H ‡ . Recall Assumption (Λ3) and the constant C Υ := sup θ sup x Λ(x, ∇Υ(x), θ) therein. Denote by C ∞ (E) the set of smooth functions on E that have a lower bound and by C ∞ u (E) the set of smooth functions on E that have an upper bound.

Definition 3.2. (The operators H † and H
and set

Preliminary results
The operator H is related to H † , H ‡ by the following Lemma.
We only prove (a) of Lemma 3.3, as (b) can be carried out analogously.
As the function [u − (1 − ε)f ] is bounded from above and εΥ has compact sublevel-sets, the sequence x n along which the first limit is attained can be assumed to lie in the compact set Denote by f ε the function on E defined by By construction f ε is smooth and constant outside of a compact set and thus lies in establishing (3.2). This concludes the proof.

The comparison principle
In this section, we prove the comparison principle for the operators H † and H ‡ .
The proof uses a variant of a classical estimate that was proven e.g. in [8,Proposition 3.7] or in the present form in Proposition A.11 of [7]. Fix Additionally, for every ε > 0 we have that Let u 1 be a viscosity subsolution and u 2 be a viscosity supersolution of f − λH † f = h 1 and f − λH ‡ f = h 2 respectively. We prove Theorem 3.4 in five steps of which the first two are classical.
We sketch the steps, before giving full proofs.
Step 1: We prove that for ε > 0 and α > 0, there exist points x ε,α , y ε,α ∈ E satisfying the properties listed in Lemma 3.5 and momenta p 1 This step is solely based on the sub-and supersolution properties of u 1 , u 2 , the continuous differentiability of the penalization function Ψ(x, y), the containment function Υ, and convexity of p → H(x, p). We conclude it suffices to establish for each ε > 0 that Step 2 : We will show that there are controls θ ε,α such that As a consequence we have For establishing (3.7), it is sufficient to bound the differences in (3.9) by using Assumptions 2.14 (Λ5) and 2.15 (I5).
Step 3: We verify the conditions to apply the continuity estimate, Assumption 2.14 (Λ5).
Step 4 : We verify the conditions to apply the estimate on I, Assumption 2.15 (I5). R. C. Kraaij and M. C. Schlottke NoDEA Step 5 : Using the outcomes of Steps 3 and 4, we can apply the continuity estimate of Assumption 2.14 (Λ4) and the equi-continuity of Assumption 2.15 (I5) to estimate (3.9) for any ε: which establishes (3.7) and thus also the comparison principle.
We proceed with the proofs of the first four steps, as the fifth step is immediate.
Proof of Step 1: The proof of this first step is classical. We include it for completeness. For any ε > 0 and any α > 0, define the map Φ ε,α : Let ε > 0. By Lemma 3.5, there is a compact set K ε ⊆ E and there exist points and lim α→∞ αΨ(x ε,α , y ε,α ) = 0. (3.12) As in the proof of Proposition A.11 of [23], it follows that At this point, we want to use the sub-and supersolution properties of u 1 and u 2 . Define the test functions ϕ ε,α Using (3.11), we find that u 1 − ϕ ε,α 1 attains its supremum at x = x ε,α , and thus Denote p 1 ε,α := α∇ x Ψ(x ε,α , y ε,α ). By our addition of the penalization (x − x ε,α ) 2 to the test function, the point x ε,α is in fact the unique optimizer, and we obtain from the subsolution inequality that With a similar argument for u 2 and ϕ ε,α 2 , we obtain by the supersolution inequality that With that, estimating further in (3.13) leads to Thus, (3.6) in Step 1 follows.

Proof of Step 3, (i) and (ii) :
We first establish (i). By the subsolution inequality (3.14), 21) and the lower bound (3.18) follows. We next establish (ii). By the supersolution inequality (3.15), we can estimate To perform this estimate, we first write To estimate the second term, we aim to apply the continuity estimate for the controls θ 0 ε,α . To do so, must establish that (x ε,α , y ε,α , θ 0 ε,α ) is fundamental for Λ with respect to Ψ. By Assumption 2.15 (I3), for each ε the set of controls θ 0 ε,α is relatively compact. Thus it suffices to establish inf These two estimates follow by Assumption 2.14 (Λ4) and (3.18) and (3.19).
The continuity estimate of Assumption 2.14 (Λ5) yields that This means that there exists a subsequence, also denoted by α such that Thus, we can estimate (3.24) by (3.27) and (3.26). This implies that (3.22) holds for the chosen subsequences α and that for these the collection (x ε,α , y ε,α , θ ε,α ) is fundamental for Λ with respect to Ψ establishing Step 3.

Existence of viscosity solutions
In this section, we will prove Theorem 2.8. In other words, we show that for h ∈ C b (E) and λ > 0, the function R(λ)h given by is indeed a viscosity solution to f −λHf = h. To do so, we will use the methods of Chapter 8 of [19]. For this strategy, one needs to check three properties of R(λ): The operator R(λ) is a pseudo-resolvent: for all h ∈ C b (E) and 0 < α < β we have Thus, if R(λ) serves as a classical left-inverse to 1 − λH and is also a pseudoresolvent, then it is a viscosity right-inverse of (1 − λH). For a second proof of this statement, outside of the control theory context, see Proposition 3.4 of [24]. Establishing (c) is straightforward. The proof of (a) and (b) stems from two main properties of exponential random variable. Let τ λ be the measure on R + corresponding to the exponential random variable with mean λ −1 .
• (a) is related to integration by parts: for bounded measurable functions z on R + , we have • (b) is related to a more involved integral property of exponential random variables. For 0 < α < β, we have Establishing  [7]. Since we use the argument further below, we briefly recall it here. We need to show that for any compact set K ⊆ E, any finite time T > 0 and finite bound M ≥ 0, there exists a compact set K = K (K, T, M ) ⊆ E such that for any absolutely continuous path γ : for any 0 ≤ τ ≤ T , so that the compact set K := {z ∈ E : Υ(z) ≤ C} satisfies the claim. We proceed with the verification of Conditions 8.10 and 8.11 of [19]. By Proposition B.1, we have H(x, 0) = 0 and hence the application of H to constant 1 function 1 satisfies H1 = 0. Thus, Condition 8.10 is implied by Condition 8.11 (see Remark 8.12 (e) in [19]).
We establish that Condition 8.11 is satisfied: for any function f ∈ D(H) = C ∞ cc (E) and x 0 ∈ E, there exists an absolutely continuous path x : [0, ∞) → E such that x(0) = x 0 and for any t ≥ 0, To do so, we solve the differential inclusioṅ where the subdifferential of H was defined in (2.9) on page 10.
Since the addition of a constant to f does not change the gradient, we may assume without loss of generality that f has compact support. A general method to establish existence of differential inclusionsẋ ∈ F (x) is given by Lemma 5.1 of Deimling [11]. We have included this result as Lemma D.5, and corresponding preliminary definitions in "Appendix D". We use this result for F (x) := ∂ p H(x, ∇f (x)). To apply Lemma D.5, we need to verify that: (F1) F is upper hemi-continuous and F (x) is non-empty, closed, and convex for all x ∈ E.
(For the definition of T E , see Definition 2.16 on page 10). Part (F1) follows from the properties of subdifferential sets of convex and continuous functionals. H is continuous in (x, p) and convex in p by Proposition B.1. Part (F3) is a consequence of Lemma 4.1, which yields that F (x) ⊆ T E (x). Part (F2) is in general not satisfied. To circumvent this problem, we use properties of H to establish a-priori bounds on the range of solutions.
Step 1: Let T > 0, and assume that x(t) solves (4.4). We establish that there is some M such that (4.1) is satisfied. By (4.4) we obtain for all p ∈ R d , and as a consequencė

x(t)∇f (x(t)) − H(x(t), ∇f (x(t))) ≥ L(x(t),ẋ(t)).
Since f has compact support and H(y, 0) = 0 for any y ∈ E, we estimate H(y, ∇f (y)). R. C. Kraaij and M. C. Schlottke NoDEA By continuity of H the field F is bounded on compact sets, so the first term can be bounded by Therefore, for any T > 0, we obtain that the integral over the Lagrangian is bounded from above by M = M (T ), with H(y, ∇f (y)).
From the first part of the, see the argument concluding after (4.2), we find that the solution x(t) remains in the compact set for all t ∈ [0, T ].
Step 2 : We prove that there exists a solution x(t) of (4.4) on [0, T ]. Using F , we define a new multi-valued vector-field F (z) that equals F (z) = ∂ p H(z, ∇f (z)) inside K , but equals {0} outside a neighborhood of K. This can e.g. be achieved by multiplying with a smooth cut-off function g K : E → [0, 1] that is equal to one on K and zero outside of a neighborhood of K .
By the estimate established in step 1 and the fact that Υ(γ(t)) ≤ C for any 0 ≤ t ≤ T , it follows from the argument as shown above in (4.2) that the solution y stays in K up to time T . Since on K , we have F = F , this implies that setting x = y| [0,T ] , we obtain a solution x(t) of (4.4) on the time interval [0, T ].
Proof. Fix x ∈ E and p 0 ∈ R d . We aim to prove that The result will follow from the following claim, where ch denotes the convex hull. Having established this claim, the result follows from Assumption 2.17 and the fact that T E (x) is a convex set by Lemma D.4. We start with the proof of (4.7). For this we will use [22,Theorem D.4.4.2]. To study the subdifferential of the function ∂ p H(x, p 0 ), it suffices to restrict the domain of the map p → H(x, p) to the closed ball B 1 (p 0 ) around p 0 with radius 1.
To apply [22,Theorem D.4.4.2] for this restricted map, first recall that Λ is continuous by Assumption 2.14 (Λ1) and that I is lower semi-continuous by Assumption 2.15 (I1). Secondly, we need to find a compact set Ω ⊆ Θ such that we can restrict the supremum (for any p ∈ B 1 (p 0 )) in (4.6) to Ω: In particular, we show that we can take for Ω a sublevelset of I(x, ·) which is compact by Assumption 2.15 (I3).
Let θ 0 x be the control such that I(x, θ 0 x ) = 0, which exists due to Assumption 2.15 (I2). Let M * be such that (with the constants M, C 1 , C 2 as in Assumption 2.14 (Λ4)) Note that M * is finite as p → Λ(x, p, θ 0 x ) is continuous on the closed unit ball B 1 (p 0 ). Then we find, due to Assumption 2.14 (Λ4), that if θ satisfies I(x, θ) > M * , then for any p ∈ B 1 (p 0 ) we have where ch denotes the convex hull. Now (4.7) follows by noting that I(x, θ) does not depend on p.

Examples of Hamiltonians
In The purpose of this section is to showcase that the method introduced in this paper is versatile enough to capture interesting examples that could not be treated before.
First, we consider in Proposition 5.1 Hamiltonians that one encounters in the large deviation analysis of two-scale systems as studied in [6] and [27] when considering a diffusion process coupled to a fast jump process. Second, we consider in Proposition 5.7 the example treated in our introduction that arises from models of mean-field interacting particles that are coupled to fast external variables. This example will be further analyzed in [26].
with non-negative rates r : Suppose that the cost function I satisfies the assumptions of Proposition 5.9 below and the function Λ satisfies the assumptions of Proposition 5.11 below. Then Theorems 2.6 and 2.8 apply to the Hamiltonian (5.1).
Proof. To apply Theorems 2.6 and 2.8, we need to verify Assumptions 2.14, 2.15 and 2.17. Assumption 2.14 follows from Proposition 5.11, Assumption 2.15 follows from Proposition 5.9 and Assumption 2.17 is satisfied as E = R d .

Remark 5.2.
We assume uniform ellipticity of a, which we use to establish (Λ4). This leaves our comparison principle slightly lacking to prove a large deviation principle as general as in [5]. In contrast, we do not need a Lipschitz condition on r in terms of x.
While we believe that the conditions on a can be relaxed by performing a finer analysis of the estimates in terms of a, we do not pursue this relaxation here.

Remark 5.3.
The cost function is the large deviation rate function for the occupation time measures of a jump process taking values in a finite set {1, . . . , J}, see e.g. [13,14].

Remark 5.4.
In the context with a = 0 and I as general as Assumption 2.15, we improve upon the results of Chapter III of [2] by allowing a more general class of functionals I, that are e.g. discontinuous as for example in Proposition 5.7 below.
In [10] the authors consider a second order Hamilton-Jacobi-Bellman equation, with the quadratic part replaced by a second order part. They work, however, with continuous cost functional I. An extension of [10] that allows for a similar flexibility in the choice of I would therefore be of interest.

Remark 5.5.
Under irreducibility conditions on the rates, as we shall assume below in Proposition 5.9, by [15] the Hamiltonian H(x, p) is the principal eigenvalue of the matrix A x,p ∈ Mat J×J (R) given by a(x, 1)p, p + b(x, 1), p , . . . , a(x, J) Next we consider Hamiltonians arising in the context of weakly interacting jump processes on a collection of states {1, . . . , q} as described in our introduction. We analyze and motivate this example in more detail in our companion paper [26]. We give the terminology as needed for the results in this paper.
The empirical measure of the interacting processes takes its values in the set of measures P ({1, . . . , q}). The dynamics arises from mass moving over the bonds (a, b) ∈ Γ = (i, j) ∈ {1, . . . , q} 2 | i = j . As the number of processes is send to infinity, there is a type of limiting result for the total mass moving over the bonds.
We will denote by v(a, b, μ, θ) the total mass that moves from a to b if the empirical measure equals μ and the control is given by θ. We will make the following assumption on the kernel v. (a, b, μ, θ) is either identically equal to zero or satisfies the following two properties:  exists a decomposition v(a, b, μ, θ) = v  † (a, b, μ(a))v ‡ (a, b, μ, θ) such that v † is increasing in the third coordinate and such that v ‡ (a, b, ·, ·) is continuous and satisfies v ‡ (a, b, μ, θ) > 0.
A typical example of a proper kernel is given by   v(a, b, μ, θ) = μ(a)r(a, b, θ) where L x is a second-order elliptic operator locally of the form Proof. To apply Theorems 2.6 and 2.8, we need to verify Assumptions 2.14, 2.15 and 2.17. Assumption 2.14 follows from Proposition 5.13 and Assumption 2.15 follows from Proposition 5.10. We verify Assumption 2.17 in Proposition 5.19.
Remark 5.8. The cost function stems from occupation-time large deviations of a drift-diffusion process on a compact manifold, see e.g. [15,32]. We expect Proposition 5.7 to extend also to non-compact spaces F , but we feel this technical extension is better suited for a separate paper. Suppose that the rates r : {1, . . . , J} 2 × E → R + are continuous as a function on E and moreover satisfy the following: each pair (i, j), we either have r(i, j, ·) ≡ 0 or for each compact set K ⊆ E, it holds that

Verifying assumptions for cost functions I
Then the Donsker-Varadhan functional I : E × Θ → R + defined by  Hence I is uniformly bounded on K × Θ, and (I4) follows with U x the interior of K. R. C. Kraaij and M. C. Schlottke NoDEA (I5) : Let d be some metric that metrizes the topology of E. We will prove that for any compact set K ⊆ E and ε > 0 there is some δ > 0 such that for all x, y ∈ K with d(x, y) ≤ δ and for all θ ∈ P(F ), we have Let x, y ∈ K. By continuity of the rates the I(x, ·) are uniformly bounded for x ∈ K: For any n ∈ N, there exists w n ∈ R J such that By reorganizing, we find for all bonds (a, b) the bound r(a, b, x). Thereby, evaluating in I(y, θ) the same vector w n to estimate the supremum, We take n → ∞ and use that the rates x → r(a, b, x) are continuous, and hence uniformly continuous on compact sets, to obtain (5.3).

the second-order elliptic operator that in local coordinates is given by
where a x is a positive definite matrix and b x is a vector field having smooth entries a ij x and b i x on F . Suppose that for all i, j the maps is the minimizer of I(x, ·), that is I(x, θ 0 x ) = 0. This follows by considering the Hille-Yosida approximation L ε x of L x and using the same argument (using w = log u) as in Proposition 5.9 for these approximations. For any u > 0 and ε > 0, Sending ε → 0 and then using (5.5) gives (I2). (I3): Since Θ = P(F ) is compact, any closed subset of Θ is compact. Hence any union of sub-level sets of I(x, ·) is relatively compact in Θ. (I4): Fix x ∈ E and M ≥ 0. Let θ ∈ Θ {x},M . As I(x, θ) ≤ M , we find by [31] that the density dθ dz exists, where dz denotes the Riemannian volume measure. As the dependence is continuous in y, we can find a open set U ⊆ E of x such that there are constants c 1 , c 2 , c 3 , c 4 , c 1 , c 3 being positive, that do not depend on θ, such that for any y ∈ U : From (5.7), (I4) immediately follows. (I5): Since the coefficients a x and b x of the operator L x depend continuously on x, assumption (I5) follows from Theorem 2 of [32].

Verifying assumptions for functions Λ
Furthermore, there exists a constant L > 0 such that for all x, y ∈ E and z ∈ F , and suppose that the functions b are one-sided Lipschitz. Then Assumption 2.14 holds.
Remark 5.12. The above proposition is slightly more general than what we consider in Proposition 5.1, as there we assume that F = {1, . . . , J} is a finite set. Proof.
Continuity of Λ is a consequence of the fact that is the pairing of a continuous and bounded function V (x, p, ·) with the measure θ ∈ P(F ).
(Λ2): Let x ∈ E and θ ∈ P(F ). Convexity of p → Λ(x, p, θ) follows since a(x, z) is positive definite by assumption. If p 0 = 0, then evidently Λ(x, p 0 , θ) = 0. (Λ3): We show that the map Υ : E → R defined by Υ(x) := 1 2 log 1 + |x| 2 is a containment function for Λ. For any x ∈ E and θ ∈ P(F ), we have , and the boundedness condition follows with the constant (Λ4): Let K ⊆ E be compact. We have to show that there exist constants M, C 1 , C 2 ≥ 0 such that for all x ∈ K, p ∈ R d and all θ 1 , θ 2 ∈ P(F ), we have Fix θ 1 , θ 2 ∈ P(F ). We have for x ∈ K a(x, z)p, p dθ 1 (z) ≤ a K,max a K,min a(x, z)p, p dθ 2 (z) In addition, as a K,min > 0 and b K,max < ∞ we have for any C > 0 and sufficiently large |p| that Thus, for sufficiently large |p| (depending on C) we have . We proceed with the example in which Λ depends on p through exponential functions (Proposition 5.7). Let q ∈ N be an integer and Γ : = (a, b) a, b ∈ {1, . . . , q}, a = b be the set of oriented edges in {1, . . . , q}. Proposition 5.13. (Exponential function Λ) Let E ⊆ R d be the embedding of E = P({1, . . . , q}) × (R + ) |Γ| and Θ be a topological space. Suppose that Λ is given by where v is a proper kernel in the sense of Definition 5.6. Suppose in addition that there is a constant C > 0 such that for all (a, b) ∈ Γ such that v(a, b, ·, ·) = 0 we have Then Λ satisfies Assumption 2.14.

Verifying the continuity estimate
With the exception of the verification of the continuity estimate in Assumption 2.14 the verification in Sect. 5.2 is straightforward. On the other hand, the continuity estimate is an extension of the comparison principle, and is therefore more complex. We verify the continuity estimate in three contexts, which illustrates that the continuity estimate follows from essentially the same arguments as the standard comparison principle. We will do this for: • Coercive Hamiltonians This list is not meant to be an exhaustive list, but to illustrate that the continuity estimate is a sensible extension of the comparison principle, which is satisfied in a wide range of contexts. In what follows, E ⊆ R d is a Polish subset and Θ a topological space. Then the continuity estimate holds for Λ with respect to any penalization function Ψ.
For the empirical measure of a collection of independent processes one obtains maps Λ that are neither uniformly coercive nor Lipschitz. Also in this context one can establish the continuity estimate. We treat a simple 1d case and then state a more general version for which we refer to [23].
We have Now note that y ε,α − x ε,α is positive if and only if e pε,α − 1 is negative so that the first term is bounded above by 0. With a similar argument the second term is bounded above by 0. Thus the continuity estimate is satisfied.
where v is a proper kernel. Then the continuity estimate holds for Λ with respect to penalization functions (see Sect. C) Here we denote r + = r ∨ 0 for r ∈ R.
In this context, one can use coercivity like in Proposition 5.15 in combination with directional properties used in the proof of Proposition 5.17 above.
To be more specific: the proof of this proposition can be carried out exactly as the proof of Theorem 3.8 of [23]: namely at any point a converging subsequence is constructed, the variables α need to be chosen such that we also get convergence of the measures θ ε,α in P(F ).
Open Access. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons. org/licenses/by/4.0/.
Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

A. Viscosity solutions
In Sect. 3 we work with a pair of Hamilton-Jacobi equations instead of a single Hamilton-Jacobi equation. To this end, we need to extend the notion of a viscosity solution and that of the comparison principle of Sect. 2.1.
We say that u is a (viscosity) subsolution of Eq. (A.1) if u is bounded, upper semi-continuous and if for all (f, g) ∈ A 1 there exists a sequence x n ∈ E such that We say that v is a (viscosity) supersolution of Eq. (A.2) if v is bounded, lower semi-continuous and if for all (f, g) ∈ A 2 there exists a sequence x n ∈ E such that As before, if test functions have compact levelsets, the existence of a sequences can be replaced by the existence of a point.

B. Regularity of the Hamiltonian
In this section, we establish continuity, convexity and the existence of a containment function for the Hamiltonian H of (2.2). We repeat its definition for convenience: To prove that H is continuous, we use Assumption 2.15. What we truly need, however, is that I Gamma converges as a function of x. We establish this result first.  Before we start with the proof, we give a remark on the generality of its statement and on the assumption that Θ is Polish.
Remark B.4. The proof of upper semi-continuity of H works in general, using continuity properties of Λ, lower semi-continuity of (x, θ) → I(x, θ) and the compact sublevel sets of I(x, ·). To establish lower semi-continuity, we need that the functionals I Gamma converge as a function of x. This was established in Proposition B.2.
Remark B.5. In the lemma we use a sequential characterization of upper hemicontinuity which holds if Θ is Polish. This is inspired by the natural formulation of Gamma convergence in terms of sequences. An extension of our results to spaces Θ beyond the Polish context should be possible to Hausdorff Θ that are k-spaces in which all compact sets are metrizable.
φ(x, p) is non-empty as θ 0 x ∈ φ(x, p) and it is compact due to Assumption 2.15 (I3). We are left to show that φ is upper hemi-continuous.
We proceed with proving lower semi-continuity of H. Suppose that (x n , p n ) → (x, p), we prove that lim inf n H(x n , p n ) ≥ H(x, p).
Let θ be the measure such that H(x, p) = Λ(x, p, θ) − I(x, θ). We have • By Proposition B.2 there are θ n such that θ n → θ and lim sup n I(x n , θ n ) ≤ I(x, θ). • Λ(x n , p n , θ n ) converges to Λ(x, p, θ) by Assumption (Λ1). establishing that H is lower semi-continuous. The Lagrangian L is obtained as the supremum over continuous functions. This implies L is lower semi-continuous.

C. A more general continuity estimate
In classical literature, the comparison principle for the Hamilton-Jacobi equation f − λHf = h is often proven using a squared distance as a penalization function. This often works well due to the quadratic structure of the Hamiltonian. In different contexts, e.g. for the Hamiltonians arising from the large deviations of jump processes, this is not natural, see the issues arising in the proofs in [16,23]. In absence of a general method to solve these issues, ad-hoc procedures can be introduced. One such ad-hoc procedure introduced in [23] is to work with multiple penalization functions (in that context {Ψ 1 , Ψ 2 }) that explore different parts of the state-space.
Any argument that has been carried out in the main text can be carried out with the generalization of the continuity estimate below.
In other words, the operator G evaluated in the proper momenta is eventually bounded from above and from below.

D. Differential inclusions
To establish that Condition 8.11 of [19] is satisfied in the proof of Theorem 2.8, we need to solve a differential inclusion. The following appendix is based on [11,28] and is a copy of the one in [25]. We state it for completeness. Let D ⊆ R d be a non-empty set. A multi-valued mapping F : D → 2 R d \ {∅} is a map that assigns to every x ∈ D a set F (x) ⊆ R d , F (x) = ∅.
If we assume sufficient regularity on the multi-valued mapping F , we can ensure the existence of a solution to differential inclusions that remain inside D.
Definition D.2. Let D ⊆ R d be a non-empty set and let F : D → 2 R d \ {∅} be a multi-valued mapping.
(i) We say that F is closed, compact or convex valued if each set F (x), x ∈ D is closed, compact or convex, respectively. Then the differential inclusionγ ∈ F (γ) has a solution on R + for every starting point x ∈ D.
Note that the comparison principle for f −λΛf = h does in fact hold. This is due to the fact that one only needs to establish that lim inf α→∞ Λ(x α , p α ) − Λ(y α , p α ) ≤ 0 for appropriately chosen x α , y α , p α without absolute value signs, see Proposition 5.17. The removal of absolute value signs is essential: the comparison principle for f − Λf = h for Λ(x, p) = x [e p − 1] fails. This is related to the statement that an associated large deviation principle fails, see Example E of [33].