Comparison Principle for Hamilton-Jacobi-Bellman Equations via a Bootstrapping Procedure

We study the well-posedness of Hamilton–Jacobi–Bellman equations on subsets of Rd\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb {R}}^d$$\end{document} in a context without boundary conditions. The Hamiltonian is given as the supremum over two parts: an internal Hamiltonian depending on an external control variable and a cost functional penalizing the control. The key feature in this paper is that the control function can be unbounded and discontinuous. This way we can treat functionals that appear e.g. in the Donsker–Varadhan theory of large deviations for occupation-time measures. To allow for this flexibility, we assume that the internal Hamiltonian and cost functional have controlled growth, and that they satisfy an equi-continuity estimate uniformly over compact sets in the space of controls. In addition to establishing the comparison principle for the Hamilton–Jacobi–Bellman equation, we also prove existence, the viscosity solution being the value function with exponentially discounted running costs. As an application, we verify the conditions on the internal Hamiltonian and cost functional in two examples.


Introduction and aim of this note
The main purpose of this note is to establish well-posedness for first-order nonlinear partial differential equations of Hamilton-Jacobi-Bellman type on subsets E of R d , in the context without boundary conditions and where the Hamiltonian flow generated by H remains inside E. In (HJB), λ > 0 is a scalar and h is a where θ ∈ Θ plays the role of a control variable. For fixed θ, the function Λ can be interpreted as an Hamiltonian itself. We call it the internal Hamiltonian. The function I can be interpreted as the cost of applying the control θ.
The main result of this paper is the comparison principle for (HJB) in order to establish uniqueness of viscosity solutions. The standard assumption in the literature that allows one to obtain the comparison principle in the context of optimal control problems (e.g. [2] for the first order case and [10] for the second order case) is that either there is a modulus of continuity ω such that |H(x, p) − H(y, p)| ≤ ω (|x − y|(1 + |p|)) , (1.2) or that H is uniformly coercive: More generally, the two estimates (1.2) and (1.3) can be combined in a single estimate, called pseudo-coercivity, see [4, (H4), Page 34], that uses the fact that the sub-and supersolution properties roughly imply that the estimate (1.2) only needs to hold for appropriately chosen x, y and p such that H is finite uniformly over these chosen x, y, p.
The pseudo-coercivity property is harder to translate as in this way the control on H does not necessarily imply the same control on Λ, in particular in the case when I is unbounded. We return on this issue below.
The estimates (I) and (II) are not satisfied for Hamiltonians arising from natural examples in the theory of large deviations [12,13] for Markov processes with two scales (see e.g. [6,18,27,29] for PDE's arising from large deviations with two scales, see [3,16,17,20,21] for other works connection PDE's with large deviations). Indeed, in [6] the authors mention that well-posedness of the Hamilton-Jacobi-Bellman equation for examples arising from large deviation theory is an open problem. Recent generalizations of the coercivity condition, see e.g. [9], also do not cover these examples.
Thus, to summarize, we use the growth conditions posed on Λ and I and the pseudo-coercivity estimate for Λ to transfer the control on the full Hamiltonian H to the functions Λ and the cost function I. Then the control on Λ and I allows us to apply the estimates (i) and (ii) to obtain the comparison principle.
Next to our main result, we also state for completeness an existence result in Theorem 2.8. The viscosity solution will be given in terms of a discounted control problem as is typical in the literature, see e.g. [2,Chapter 3]. Minor difficulties arise from working with H that arise from irregular I.
Finally, we show that the conditions (i) to (vi) are satisfied in two examples that arise from large deviation theory for two-scale processes. In our companion paper [26], we will use existence and uniqueness for (HJB) for these examples to obtain large deviation principles.

Illustration in the context of an example
As an illustrating example, we consider a Hamilton-Jacobi-Bellman equation that arises from the large deviations of the empirical measure-flux pair of weakly coupled Markov jump processes that are coupled to fast Brownian motion on the torus. We skip the probabilistic background of this problem (See [26]), and come to the set-up relevant for this paper.
Let G := {1, . . . , q} be some finite set, and let Γ = {(a, b) ∈ G 2 | a = b} be the set of directed bonds. Let E := P(G)×[0, ∞) Γ , where P(G) is the set of probability measures on G. Let F = P(S 1 ) be the set of probability measures on the one-dimensional torus. We introduce Λ and I.
• Let r : G × G × P(E) × P(S 1 ) → [0, ∞) be some function that codes the P(E) × P(S 1 ) dependent jump rate of the Markov jump process over each bond (a, b) ∈ Γ. The internal Hamiltonian Λ is given by • Let σ 2 : S 1 × P(G) → (0, ∞) be a bounded and strictly positive function. The cost function I : E × Θ → [0, ∞] is given by Aiming for the comparison principle, we note that classical methods do not apply. The functionals Λ are not coercive and do not satisfy (I). We show in "Appendix E" that they are also not pseudo-coercive as defined in [4]. The functional I is neither continuous nor bounded. Once can check e.g. that if θ is a finite combination of Dirac measures, then I(μ, θ) = ∞. We show in Sect. 5, however, that (i) to (vi) hold, implying the comparison principle for the Hamilton-Jacobi-Bellman equations. The verification of these properties is based in part on results from [23,32].

Summary and overview of the paper
To summarize, our novel bootstrap procedure allows to treat Hamilton-Jacobi-Bellman equations where: • We assume that the cost function I satisfies some regularity conditions on its sub-levelsets, but allow I to be possibly unbounded and discontinuous. • We assume that Λ satisfies the continuity estimate uniformly for controls in compact sets, which in spirit extends the pseudo-coercivity estimate of [4]. This implies that Λ can be possibly non-coercive, non-pseudo-coercive and non-Lipschitz as exhibited in our example above.
In particular, allowing discontinuity in I allows us to treat the comparison principle for examples like the one we considered above, which so far has been out of reach. We believe that the bootstrap procedure we introduce in this note has the potential to also apply to second order equations or equations in infinite dimensions. Of interest would be, for example, an extension of the results of [10] who work with continuous I. For clarity of the exposition, and the already numerous applications for this setting, we stick to the finite-dimensional firstorder case. We think that the key arguments that are used in the proof in Sect. 3 do not depend in a crucial way on this assumption.
The paper is organized as follows. The main results are formulated in Sect. 2. In Sect. 3 we establish the comparison principle. In Sect. 4 we establish that a resolvent operator R(λ) in terms of an exponentially discounted control problem gives rise to viscosity solutions of the Hamilton-Jacobi-Bellman equation (HJB). Finally, in Sect. 5 we treat two examples including the one mentioned in the introduction.

Main results
In this section, we start with preliminaries in Sect. 2.1, which includes the definition of viscosity solutions and that of the comparison principle.
We proceed in Sect. 2.2 with the main results: a comparison principle for the Hamilton-Jacobi-Bellman equation (HJB) based on variational Hamiltonians of the form (1.1), and the existence of viscosity solutions. In Sect. 2.3 we collect all assumptions that are needed for the main results.

Preliminaries
For a Polish space X we denote by C(X ) and C b (X ) the spaces of continuous and bounded continuous functions respectively. If X ⊆ R d then we denote by C ∞ c (X ) the space of smooth functions that vanish outside a compact set. We denote by C ∞ cc (X ) the set of smooth functions that are constant outside of a compact set in X , and by P(X ) the space of probability measures on X . We equip P(X ) with the weak topology induced by convergence of integrals against bounded continuous functions.
Throughout the paper, E will be the set on which we base our Hamilton-Jacobi equations. We assume that E is a subset of R d that is a Polish space Consider the Hamilton-Jacobi equation We say that u is a (viscosity) subsolution of equation ( We say that v is a (viscosity) supersolution of Eq. (2.1) if v is bounded from below, lower semi-continuous and if, for every f ∈ D(A)there exists a sequence We say that u is a (viscosity) solution of Eq.
A similar simplification holds in the case of supersolutions.

Remark 2.4.
For an explanatory text on the notion of viscosity solutions and fields of applications, we refer to [8].
Remark 2.5. At present, we refrain from working with unbounded viscosity solutions as we use the upper bound on subsolutions and the lower bound on supersolutions in the proof of Theorem 2.6. We can, however, imagine that the methods presented in this paper can be generalized if u and v grow slower than the containment function Υ that will be defined below in Definition 2.13.
Then R(λ)h is the unique viscosity solution to f − λHf = h. Remark 2.9. The form of the solution is typical, see for example Section III.2 in [2]. It is the value function obtained by an optimization problem with exponentially discounted cost. The difficulty of the proof of Theorem 2.8 lies in treating the irregular form of H.

Assumptions
In this section, we formulate and comment on the assumptions imposed on the Hamiltonians defined in the previous sections. The key assumptions were already mentioned in the sketch of the bootstrap method in the introduction. To these, we add minor additional assumptions on the regularity of Λ and I in Assumptions 2.14 and 2.15. Finally, Assumption 2.17 will imply that even if E has a boundary, no boundary conditions are necessary for the construction of the viscosity solution.
We start with the continuity estimate for Λ, which was briefly discussed in (i) in the introduction. To that end, we first introduce a function that is used in the typical argument that doubles the number of variables. Definition 2.10. (Penalization function) We say that Ψ : and if x = y if and only if Ψ(x, y) = 0.
We will apply the definition below for G = Λ. Definition 2.11. (Continuity estimate) Let Ψ be a penalization function and let G : Suppose that for each ε > 0, there is a sequence of positive real numbers α → ∞. For sake of readability, we suppress the dependence on ε in our notation.
(2.6) Remark 2.12. In "Appendix C", we state a slightly more general continuity estimate on the basis of two penalization functions. A proof of a comparison principle on the basis of two penalization functions was given in [23].
The continuity estimate is indeed exactly the estimate that one would perform when proving the comparison principle for the Hamilton-Jacobi equation in terms of the internal Hamiltonian (disregarding the control θ). Typically, the control on (x ε,α , y ε,α ) that is assumed in (C1) and (C2) is obtained from choosing (x ε,α , y ε,α ) as optimizers in the doubling of variables procedure (see Lemma 3.5), and the control that is assumed in (C3) is obtained by using the viscosity sub-and supersolution properties in the proof of the comparison principle. The required restriction to compact sets in Lemma 3.5 is obtained by including in the test functions a containment function. • For every c ≥ 0, the set {x | Υ(x) ≤ c} is compact; To conclude, our assumption on Λ contains the continuity estimate, the controlled growth, the existence of a containment function and two regularity properties.
Assumption 2.14. The function Λ : (Λ5) The function Λ satisfies the continuity estimate in the sense of Definition 2.11, or in the extended sense of Definition C.2.
Our second main assumption is on the properties of I. For a compact set K ⊆ E and a constant M ≥ 0, write and

there exists an open neighbourhood
U ⊆ E of x and constants M , C 1 , C 2 ≥ 0 such that for all y 1 , y 2 ∈ U and θ ∈ Θ {x},M we have To establish the existence of viscosity solutions, we will impose one additional assumption. For a general convex functional p → Φ(p) we denote Assumption 2.17. The set E is closed and convex. The map Λ is such that In Lemma 4.1 we will show that the assumption implies that ∂ p H(x, p) ⊆ T E (x), which in turn implies that the solutions of the differential inclusion in terms of ∂ p H(x, p) remain inside E. Motivated by our examples, we work with closed convex domains E. While in this context we can apply results from e.g. Deimling [11], we believe that similar results can be obtained in different contexts.
is intuitively implied by the comparison principle for H and therefore, we expect it to hold in any setting for which Theorem 2.6 holds. Here, we argue in a simple case why this is to be expected. First of all, note that the comparison principle for H builds upon the maximum principle.
. As x = 0 is a boundary point, we conclude that f (0) ≤ g (0). If indeed the maximum principle holds, we must have

The comparison principle
In this section, we establish Theorem 2.6. To establish the comparison principle for f − λHf = h we use the bootstrap method explained in the introduction. We start by a classical localization argument.
We carry out the localization argument by absorbing the containment function Υ from Assumption 2.14 (Λ3) into the test functions. This leads to two new operators, H † and H ‡ that serve as an upper bound and a lower bound for the true H. We will then show the comparison principle for the Hamilton-Jacobi equation in terms of these two new operators. We therefore have to extend our notion of Hamilton-Jacobi equations and the comparison principle. This extension of the definition is standard, but we included it for completeness in the appendix as Definition A.1.
This procedure allows us to clearly separate the reduction to compact sets on one hand, and the proof of the comparison principle on the basis of the bootstrap procedure on the other. Schematically, we will establish the following diagram: In this diagram, an arrow connecting an operator A with operator B with subscript 'sub' means that viscosity subsolutions of f − λAf = h are also viscosity subsolutions of f − λBf = h. Similarly for arrows with a subscript 'super'.
We introduce the operators H † and H ‡ in Sect. 3.1. The arrows will be established in Sect. 3.2. Finally, we will establish the comparison principle for H † and H ‡ in Sect. 3.3. Combined these two results imply the comparison principle for H.
Proof of Theorem 2.6. We start with the proof of (a). Let f ∈ D(H). Then Hf is continuous since by Proposition B.3 in "Appendix B", the Hamiltonian H is continuous.
We proceed with the proof of (b). Fix h 1 , h 2 ∈ C b (E) and λ > 0. Let u 1 , u 2 be a viscosity sub-and supersolution to f − λHf = h 1 and f − λHf = h 2 respectively. By Lemma 3.3 proven in Sect. 3.2, u 1 and u 2 are a sub-and supersolution to

Definition of auxiliary operators
In this section, we repeat the definition of H, and introduce the operators H † and H ‡ .
We proceed by introducing H † and H ‡ . Recall Assumption (Λ3) and the constant C Υ := sup θ sup x Λ(x, ∇Υ(x), θ) therein. Denote by C ∞ (E) the set of smooth functions on E that have a lower bound and by C ∞ u (E) the set of smooth functions on E that have an upper bound. and set

Preliminary results
The operator H is related to H † , H ‡ by the following Lemma.
We only prove (a) of Lemma 3.3, as (b) can be carried out analogously.
As the function [u − (1 − ε)f ] is bounded from above and εΥ has compact sublevel-sets, the sequence x n along which the first limit is attained can be assumed to lie in the compact set Denote by f ε the function on E defined by By construction f ε is smooth and constant outside of a compact set and thus lies in establishing (3.2). This concludes the proof.

The comparison principle
In this section, we prove the comparison principle for the operators H † and H ‡ .
The proof uses a variant of a classical estimate that was proven e.g. in [8,Proposition 3.7] or in the present form in Proposition A.11 of [7]. Fix Additionally, for every ε > 0 we have that Let u 1 be a viscosity subsolution and u 2 be a viscosity supersolution of f − λH † f = h 1 and f − λH ‡ f = h 2 respectively. We prove Theorem 3.4 in five steps of which the first two are classical.
We sketch the steps, before giving full proofs.
Step 1: We prove that for ε > 0 and α > 0, there exist points x ε,α , y ε,α ∈ E satisfying the properties listed in Lemma 3.5 and momenta p 1 This step is solely based on the sub-and supersolution properties of u 1 , u 2 , the continuous differentiability of the penalization function Ψ(x, y), the containment function Υ, and convexity of p → H(x, p). We conclude it suffices to establish for each ε > 0 that Step 2 : We will show that there are controls θ ε,α such that As a consequence we have For establishing (3.7), it is sufficient to bound the differences in (3.9) by using Assumptions 2.14 (Λ5) and 2.15 (I5).
Step 3: We verify the conditions to apply the continuity estimate, Assumption 2.14 (Λ5).
Step 4 : We verify the conditions to apply the estimate on I, Assumption 2.15 (I5). Step 5 : Using the outcomes of Steps 3 and 4, we can apply the continuity estimate of Assumption 2.14 (Λ4) and the equi-continuity of Assumption 2.15 (I5) to estimate (3.9) for any ε: which establishes (3.7) and thus also the comparison principle. We proceed with the proofs of the first four steps, as the fifth step is immediate.
Proof of Step 1: The proof of this first step is classical. We include it for completeness. For any ε > 0 and any α > 0, define the map Φ ε,α : Let ε > 0. By Lemma 3.5, there is a compact set K ε ⊆ E and there exist points and lim α→∞ αΨ(x ε,α , y ε,α ) = 0. (3.12) As in the proof of Proposition A.11 of [23], it follows that At this point, we want to use the sub-and supersolution properties of u 1 and u 2 . Define the test functions ϕ ε,α Using (3.11), we find that u 1 − ϕ ε,α 1 attains its supremum at x = x ε,α , and thus Denote p 1 ε,α := α∇ x Ψ(x ε,α , y ε,α ). By our addition of the penalization (x − x ε,α ) 2 to the test function, the point x ε,α is in fact the unique optimizer, and we obtain from the subsolution inequality that With a similar argument for u 2 and ϕ ε,α 2 , we obtain by the supersolution inequality that where p 2 ε,α := −α∇ y Ψ(x ε,α , y ε,α ). With that, estimating further in (3.13) leads to Thus, (3.6) in Step 1 follows.

Proof of
Step 3: We will construct for each ε > 0 a sequence α = α(ε) → ∞ such that the collection (x ε,α , y ε,α , θ ε,α ) is fundamental for Λ with respect to Ψ in the sense of Definition 2.11. We thus need to verify for each ε > 0 iii) The set of controls θ ε,α is relatively compact. To prove (i), (ii) and (iii), we introduce auxiliary controls θ 0 ε,α , obtained by (I2), satisfying We will first establish (i) and (ii) for all α. Then, for the proof of (iii), we will construct for each ε > 0 a suitable subsequence α → ∞.

Proof of Step 3, (i) and (ii) :
We first establish (i). By the subsolution inequality (3.14), 21) and the lower bound (3.18) follows. We next establish (ii). By the supersolution inequality (3.15), we can estimate To perform this estimate, we first write To estimate the second term, we aim to apply the continuity estimate for the controls θ 0 ε,α . To do so, must establish that (x ε,α , y ε,α , θ 0 ε,α ) is fundamental for These two estimates follow by Assumption 2.14 (Λ4) and (3.18) and (3.19). The continuity estimate of Assumption 2.14 (Λ5) yields that This means that there exists a subsequence, also denoted by α such that Thus, we can estimate (3.24) by (3.27) and (3.26). This implies that (3.22) holds for the chosen subsequences α and that for these the collection (x ε,α , y ε,α , θ ε,α ) is fundamental for Λ with respect to Ψ establishing Step 3.

Existence of viscosity solutions
In this section, we will prove Theorem 2.8. In other words, we show that for h ∈ C b (E) and λ > 0, the function R(λ)h given by is indeed a viscosity solution to f −λHf = h. To do so, we will use the methods of Chapter 8 of [19]. For this strategy, one needs to check three properties of R(λ): The operator R(λ) is a pseudo-resolvent: for all h ∈ C b (E) and 0 < α < β we have Thus, if R(λ) serves as a classical left-inverse to 1 − λH and is also a pseudoresolvent, then it is a viscosity right-inverse of (1 − λH). For a second proof of this statement, outside of the control theory context, see Proposition 3.4 of [24]. Establishing (c) is straightforward. The proof of (a) and (b) stems from two main properties of exponential random variable. Let τ λ be the measure on R + corresponding to the exponential random variable with mean λ −1 .
• (a) is related to integration by parts: for bounded measurable functions z on R + , we have • (b) is related to a more involved integral property of exponential random variables. For 0 < α < β, we have Establishing  [7]. Since we use the argument further below, we briefly recall it here. We need to show that for any compact set K ⊆ E, any finite time T > 0 and finite bound M ≥ 0, there exists a compact set K = K (K, T, M ) ⊆ E such that for any absolutely continuous path γ : then γ(t) ∈ K for any 0 ≤ t ≤ T . For K ⊆ E, T > 0, M ≥ 0 and γ as above, this follows by noting that Υ(γ(τ )) = Υ(γ(0)) + τ 0
We establish that Condition 8.11 is satisfied: for any function f ∈ D(H) = C ∞ cc (E) and x 0 ∈ E, there exists an absolutely continuous path x : [0, ∞) → E such that x(0) = x 0 and for any t ≥ 0, To do so, we solve the differential inclusioṅ where the subdifferential of H was defined in (2.9) on page 10.
Since the addition of a constant to f does not change the gradient, we may assume without loss of generality that f has compact support. A general method to establish existence of differential inclusionsẋ ∈ F (x) is given by Lemma 5.1 of Deimling [11]. We have included this result as Lemma D.5, and corresponding preliminary definitions in "Appendix D". We use this result for F (x) := ∂ p H(x, ∇f (x)). To apply Lemma D.5, we need to verify that: (F1) F is upper hemi-continuous and F (x) is non-empty, closed, and convex for all x ∈ E. Step 1: Let T > 0, and assume that x(t) solves (4.4). We establish that there is some M such that (4.1) is satisfied. By (4.4) we obtain for all p ∈ R d , and as a consequencė ẋ(t)).
Since f has compact support and H(y, 0) = 0 for any y ∈ E, we estimate H(y, ∇f (y)).
Therefore, for any T > 0, we obtain that the integral over the Lagrangian is bounded from above by M = M (T ), with From the first part of the, see the argument concluding after (4.2), we find that the solution x(t) remains in the compact set for all t ∈ [0, T ].
Step 2 : We prove that there exists a solution x(t) of (4.4) on [0, T ]. Using F , we define a new multi-valued vector-field F (z) that equals F (z) = ∂ p H(z, ∇f (z)) inside K , but equals {0} outside a neighborhood of K. This can e.g. be achieved by multiplying with a smooth cut-off function g K : E → [0, 1] that is equal to one on K and zero outside of a neighborhood of K .
By the estimate established in step 1 and the fact that Υ(γ(t)) ≤ C for any 0 ≤ t ≤ T , it follows from the argument as shown above in (4.2) that the solution y stays in K up to time T . Since on K , we have F = F , this implies that setting x = y| [0,T ] , we obtain a solution x(t) of (4.4) on the time interval [0, T ].  The result will follow from the following claim, where ch denotes the convex hull. Having established this claim, the result follows from Assumption 2.17 and the fact that T E (x) is a convex set by Lemma D.4. We start with the proof of (4.7). For this we will use [22,Theorem D.4.4.2]. To study the subdifferential of the function ∂ p H(x, p 0 ), it suffices to restrict the domain of the map p → H(x, p) to the closed ball B 1 (p 0 ) around p 0 with radius 1.
To apply [22,Theorem D.4.4.2] for this restricted map, first recall that Λ is continuous by Assumption 2.14 (Λ1) and that I is lower semi-continuous by Assumption 2.15 (I1). Secondly, we need to find a compact set Ω ⊆ Θ such that we can restrict the supremum (for any p ∈ B 1 (p 0 )) in (4.6) to Ω: In particular, we show that we can take for Ω a sublevelset of I(x, ·) which is compact by Assumption 2.15 (I3).
Let θ 0 x be the control such that I(x, θ 0 x ) = 0, which exists due to Assumption 2.15 (I2). Let M * be such that (with the constants M, C 1 , C 2 as in Assumption 2.14 (Λ4)) Note that M * is finite as p → Λ(x, p, θ 0 x ) is continuous on the closed unit ball B 1 (p 0 ). Then we find, due to Assumption 2.14 (Λ4), that if θ satisfies I(x, θ) > M * , then for any p ∈ B 1 (p 0 ) we have . We obtain that if p ∈ B 1 (p 0 ), then we can restrict our supremum in (4.6) to the compact set Ω := Θ {x},M * , see Assumption 2.15 (I3).

.4.2] that
where ch denotes the convex hull. Now (4.7) follows by noting that I(x, θ) does not depend on p.

Examples of Hamiltonians
In The purpose of this section is to showcase that the method introduced in this paper is versatile enough to capture interesting examples that could not be treated before. First, we consider in Proposition 5.1 Hamiltonians that one encounters in the large deviation analysis of two-scale systems as studied in [6] and [27] when considering a diffusion process coupled to a fast jump process. Second, we consider in Proposition 5.7 the example treated in our introduction that arises from models of mean-field interacting particles that are coupled to fast external variables. This example will be further analyzed in [26].
with non-negative rates r : Suppose that the cost function I satisfies the assumptions of Proposition 5.9 below and the function Λ satisfies the assumptions of Proposition 5.11 below. Then Theorems 2.6 and 2.8 apply to the Hamiltonian (5.1).
Proof. To apply Theorems 2.6 and 2.8, we need to verify Assumptions 2.14, 2.15 and 2.17. Assumption 2.14 follows from Proposition 5.11, Assumption 2.15 follows from Proposition 5.9 and Assumption 2.17 is satisfied as E = R d .

Remark 5.2.
We assume uniform ellipticity of a, which we use to establish (Λ4). This leaves our comparison principle slightly lacking to prove a large deviation principle as general as in [5]. In contrast, we do not need a Lipschitz condition on r in terms of x.
While we believe that the conditions on a can be relaxed by performing a finer analysis of the estimates in terms of a, we do not pursue this relaxation here. In [10] the authors consider a second order Hamilton-Jacobi-Bellman equation, with the quadratic part replaced by a second order part. They work, however, with continuous cost functional I. An extension of [10] that allows for a similar flexibility in the choice of I would therefore be of interest.
Remark 5.5. Under irreducibility conditions on the rates, as we shall assume below in Proposition 5.9, by [15] the Hamiltonian H(x, p) is the principal eigenvalue of the matrix A x,p ∈ Mat J×J (R) given by a(x, 1)p, p + b(x, 1), p , . . . , a(x, J) Next we consider Hamiltonians arising in the context of weakly interacting jump processes on a collection of states {1, . . . , q} as described in our introduction. We analyze and motivate this example in more detail in our companion paper [26]. We give the terminology as needed for the results in this paper.
The empirical measure of the interacting processes takes its values in the set of measures P ({1, . . . , q}). The dynamics arises from mass moving over the bonds (a, b) ∈ Γ = (i, j) ∈ {1, . . . , q} 2 | i = j . As the number of processes is send to infinity, there is a type of limiting result for the total mass moving over the bonds.
We will denote by v(a, b, μ, θ) the total mass that moves from a to b if the empirical measure equals μ and the control is given by θ. We will make the following assumption on the kernel v.  v(a, b, μ, θ) is either identically equal to zero or satisfies the following two properties:  exists a decomposition v(a, b, μ, θ) = v  † (a, b, μ(a))v ‡ (a, b, μ, θ) such that v † is increasing in the third coordinate and such that v ‡ (a, b, ·, ·) is continuous and satisfies v ‡ (a, b, μ, θ) > 0.
A typical example of a proper kernel is given by   v(a, b, μ, θ) = μ(a)r(a, b, θ) (i) The set of control variables Θ equals P(F ).
(ii) The function Λ is given by with a proper kernel v in the sense of Definition 5.6. (iii) The cost function I : E × Θ → [0, ∞] is given by where L x is a second-order elliptic operator locally of the form Proof. To apply Theorems 2.6 and 2.8, we need to verify Assumptions 2.14, 2.15 and 2.17. Assumption 2.14 follows from Proposition 5.13 and Assumption 2.15 follows from Proposition 5.10. We verify Assumption 2.17 in Proposition 5.19.
Remark 5.8. The cost function stems from occupation-time large deviations of a drift-diffusion process on a compact manifold, see e.g. [15,32]. We expect Proposition 5.7 to extend also to non-compact spaces F , but we feel this technical extension is better suited for a separate paper.

Verifying assumptions for cost functions I
Then the Donsker-Varadhan functional I : E × Θ → R + defined by  Hence I is uniformly bounded on K × Θ, and (I4) follows with U x the interior of K. Let x, y ∈ K. By continuity of the rates the I(x, ·) are uniformly bounded for x ∈ K: For any n ∈ N, there exists w n ∈ R J such that By reorganizing, we find for all bonds (a, b) the bound r(a, b, x). Thereby, evaluating in I(y, θ) the same vector w n to estimate the supremum, We take n → ∞ and use that the rates x → r(a, b, x) are continuous, and hence uniformly continuous on compact sets, to obtain (5.3).

the second-order elliptic operator that in local coordinates is given by
where a x is a positive definite matrix and b x is a vector field having smooth entries a ij x and b i x on F . Suppose that for all i, j the maps is the minimizer of I(x, ·), that is I(x, θ 0 x ) = 0. This follows by considering the Hille-Yosida approximation L ε x of L x and using the same argument (using w = log u) as in Proposition 5.9 for these approximations. For any u > 0 and ε > 0, Sending ε → 0 and then using (5.5) gives (I2). (I3): Since Θ = P(F ) is compact, any closed subset of Θ is compact. Hence any union of sub-level sets of I(x, ·) is relatively compact in Θ. (I4): Fix x ∈ E and M ≥ 0. Let θ ∈ Θ {x},M . As I(x, θ) ≤ M , we find by [31] that the density dθ dz exists, where dz denotes the Riemannian volume measure.
From (5.7), (I4) immediately follows. (I5): Since the coefficients a x and b x of the operator L x depend continuously on x, assumption (I5) follows from Theorem 2 of [32]. is a containment function for Λ. For any x ∈ E and θ ∈ P(F ), we have

Verifying assumptions for functions
, and the boundedness condition follows with the constant (Λ4): Let K ⊆ E be compact. We have to show that there exist constants M, C 1 , C 2 ≥ 0 such that for all x ∈ K, p ∈ R d and all θ 1 , θ 2 ∈ P(F ), we have Fix θ 1 , θ 2 ∈ P(F ). We have for x ∈ K a(x, z)p, p dθ 1 (z) ≤ a K,max a K,min a(x, z)p, p dθ 2 (z) In addition, as a K,min > 0 and b K,max < ∞ we have for any C > 0 and sufficiently large |p| that b(x, z), p dθ 1 (z) − (C + 1) b(x, z), p dθ 2 (z) ≤ C a(x, z)p, p dθ 2 (z) Thus, for sufficiently large |p| (depending on C) we have We proceed with the example in which Λ depends on p through exponential functions (Proposition 5.7). Let q ∈ N be an integer and Γ : = (a, b) a, b ∈ {1, . . . , q}, a = where v is a proper kernel in the sense of Definition 5.6. Suppose in addition that there is a constant C > 0 such that for all (a, b) ∈ Γ such that v(a, b, ·, ·) = 0 we have Then Λ satisfies Assumption 2.14.
Remark 5.14. Similar to the previous proposition, the assumptions on Λ are satisfied when Θ = P(F ) for some Polish space F , we have v(a, b, μ, θ) = μ(a) r(a, b, μ, z)θ(dz), and there are constants 0 < r min ≤ r max < ∞ such that for all (a, b) ∈ Γ such that sup μ,z r(a, b, μ, z) > 0, we have Regarding (5.9), for (a, b) ∈ Γ for which v(a, b, ·, ·) is non-trivial, we have Proof is a containment function for Λ. For a verification, see [23].

Verifying the continuity estimate
With the exception of the verification of the continuity estimate in Assumption 2.14 the verification in Sect. 5.2 is straightforward. On the other hand, the continuity estimate is an extension of the comparison principle, and is therefore more complex. We verify the continuity estimate in three contexts, which illustrates that the continuity estimate follows from essentially the same arguments as the standard comparison principle. We will do this for: • Coercive Hamiltonians • One-sided Lipschitz Hamiltonians • Hamiltonians arising from large deviations of empirical measures.
This list is not meant to be an exhaustive list, but to illustrate that the continuity estimate is a sensible extension of the comparison principle, which is satisfied in a wide range of contexts. In what follows, E ⊆ R d is a Polish subset and Θ a topological space. Then the continuity estimate holds for Λ with respect to any penalization function Ψ.
For the empirical measure of a collection of independent processes one obtains maps Λ that are neither uniformly coercive nor Lipschitz. Also in this context one can establish the continuity estimate. We treat a simple 1d case and then state a more general version for which we refer to [23]. Proof. Let Ψ(x, y) = 1 2 (x−y) 2 . Let (x α,ε , y α,ε , θ ε,α ) be fundamental for Λ with respect to Ψ. Set p α,ε = α(x ε,α − y ε,α ).
In this context, one can use coercivity like in Proposition 5.15 in combination with directional properties used in the proof of Proposition 5.17 above.
To be more specific: the proof of this proposition can be carried out exactly as the proof of Theorem 3.8 of [23]: namely at any point a converging subsequence is constructed, the variables α need to be chosen such that we also get convergence of the measures θ ε,α in P(F ). Then we have ∂ p Λ((μ, x), p) ⊆ T E (μ, w).