Representation of Hamilton–Jacobi Equation in Optimal Control Theory with Unbounded Control Set

In this paper, we study the existence of sufficiently regular representations of Hamilton–Jacobi equations in the optimal control theory with unbounded control set. We use a new method to construct representations for a wide class of Hamiltonians. This class is wider than any constructed before, because we do not require Legendre–Fenchel conjugates of Hamiltonians to be bounded. However, in this case we obtain representations with unbounded control set. We apply the obtained results to study regularities of value functions and correlations between variational and optimal control problems.


Introduction
The Hamilton-Jacobi equation (6) with a convex Hamiltonian H in the gradient variable can be studied with connection to calculus of variations problems, namely the value function of the calculus of variations problem given by (7) is the unique viscosity solution, see, e.g., [1][2][3][4][5][6][7]. The Hamilton-Jacobi equation (6) can be also studied with connection to optimal control problems. It is possible provided that there exists a sufficiently regular triple (A, f , l) satisfying the equality (4). In particular, the value function of the optimal control problem given by (8) is the unique viscosity solution, see, e.g., [1,3,4,8,9]. The triple (A, f , l), which satisfies the equality (4), is called a representation of the Hamiltonian H . In general, if a representation of H exists, then infinitely many other representations exist. There are also irregular representations among them. The triple (A, f , l), which satisfies the equality (4) and inherits Lipschitz-type properties of H , is called a faithful representation of H .
In this paper, we provide further developments of representation theorems from [10]. Misztela [10] studied faithful representations of Hamiltonians with the compact control set. A necessary condition for the existence of such representations is boundedness of Legendre-Fenchel conjugates of Hamiltonians on effective domains, see [10,Theorem 3.1]. However, in many cases Hamiltonians do not have bounded Legendre-Fenchel conjugates on effective domains. In Sect. 4, we see that for this type of Hamiltonians there exist faithful representations with the unbounded control set. We used a new method to construct a faithful representation. Our representation (A, f , l) of H is an epigraphical representation introduced in [10]. The construction of this representation is as follows: First, using Steiner selection we parameterize the set-valued map obtained via epigraph of the Legendre-Fenchel conjugate of H . Steiner selection guarantees that this parametrization e with the parameter set A is locally Lipschitz continuous with respect to the state variable. Next, we define the functions f and l as components of the function e, i.e., e = ( f , l). In view of [10,Proposition 5.7], any triple (A, f , l) obtained in such a way is a representation of H . Earlier, Frankowska-Sedrakyan [11] and Rampazzo [12] used a graphical representation to construct a faithful representation. In a graphical representation, the function l, without additional assumptions, may be discontinuous with respect to the statecontrol variable, see Sect. 3. Another differences between graphical and epigraphical representations can be found in [10]. Earlier, Ishii [13] proposed a representation involving continuous functions f and l with the infinite-dimensional control set A. The lack of local Lipschitz continuity of f and l with respect to the state variable and finite-dimensional control set A in Ishii [13] paper causes troubles in applications.
We present differences between representations with unbounded and compact control sets. The fact that a control set is not compact makes significant problems in applications which we discuss below. Therefore, compactness of a control set must be replaced by another property that is convenient in practice. The property (A3) from Theorem 4.1 which is a consequence of our construction of a faithful representation plays a role of such extra-property. Our extra-property is apparently new. In the literature, one usually requires coercivity of the function l(t, x, ·), see, e.g., [9,Condition (A 4 )]. However, the function l(t, x, ·) from our faithful representation (A, f , l) does not have this property. Coercivity of the function l(t, x, ·) enables us to study not only measurability of controls but also their integrability. In this paper, the extra-property plays a similar role, see Remarks 4.1 and 4.2. It is well-known that in applications one requires at least integrability of controls. In the case when the control set is compact the above problem does not occur, because every measurable control with values in the compact control set is integrable.
In general, the value functions (7) and (8) are not equal. However, in our case these value functions are identical due to the extra-property, see Corollary 4.1. Moreover, we obtain a fundamental relation between variational and optimal control problems, see Theorem 4.4. More precisely, we consider a variational problem associated with the given Lagrangian L. We define Hamiltonian H as the Legendre-Fenchel conjugate of L in its velocity variable. Applying our result to Hamiltonian H , we obtain its faithful representation (A, f , l). Then the variational problem associated with Lagrangian L is equivalent to the optimal control problem associated with the triple (A, f , l). Earlier, Olech [14] and Rockafeller [15,16] investigated the opposite problem to ours.
Our faithful representations are stable, see Theorems 4.2 and 4.3. This fact is used in the proof of stability of value functions, see Sect. 6. The method of this proof is not standard, because properties of a faithful representation are nonstandard. These nonstandard properties are: an unbounded control set, the extra-property and the sublinear growth of l with respect to the control variable. In this case, one cannot apply methods from Sedrakyan [17] to prove stability of value functions. Indeed, this method uses compactness of the control set and boundedness of l independent of the control variable. We also prove that the value function is locally Lipschitz continuous, provided that the final cost function is locally Lipschitz continuous. In the proof of this fact, nonstandard boundedness of the function l plays a significant role.
The outline of the paper is as follows. Section 2 contains hypotheses and background material. In Sect. 3, we show differences between graphical and epigraphical representations with the unbounded control set. In Sect. 4, we gathered our main results. Sections 5 and 6 contain proofs.

Hypotheses and Background Material
We will need hypotheses and results similar to those in [10, Sect. 2].
(H1) H : [0, T ] × R n × R n → R is Lebesgue measurable in t for any x, p ∈ R n ; (H2) H (t, x, p) is continuous with respect to (x, p) for every t ∈ [0, T ]; (H3) H (t, x, p) is convex with respect to p for every (t, x) ∈ [0, T ] × R n ; (H4) There exists a measurable map c : Let ϕ be an extended-real-valued function from R m to R∪{±∞}. The sets: dom ϕ = {z ∈ R m : ϕ(z) = ±∞}, gph ϕ = {(z, r ) ∈ R m × R : ϕ(z) = r } and epi ϕ = {(z, r ) ∈ R m × R : ϕ(z) ≤ r } are called the effective domain, the graph and the epigraph of ϕ, respectively. We say that ϕ is proper if it never takes the value −∞ and it is not identically equal to +∞.
Let H * (t, x, ·) denotes the Legendre-Fenchel conjugate of H (t, x, ·): Using properties of the conjugate from [18], we can prove the following.

Proposition 2.1 Assume that H satisfies (H1)-(H3). Then
is convex and proper for every (t, x) ∈ [0, T ] × R n ; Additionally, if H is continuous, then H * is lower semicontinuous and Let K be a nonempty subset of R m . We put K := sup ξ ∈K |ξ |. The distance from y ∈ R m to K is defined by d(y, Let us define the set-valued map E H * : [0, T ] × R n R n × R by the formula In view of Proposition 2.1 and Results in [18,Chap. 14], we obtain

Corollary 2.1 Assume that H satisfies (H1)-(H3). Then
Additionally, if H is continuous, then (E6) (t, x) → E H * (t, x) has a closed graph and is lower semicontinuous. Now we present Hausdorff continuity of a set-valued map E H * in Hamiltonian and its conjugate terms. Let B(x, R) denote the closed ball in R n of centerx and radius R ≥ 0. We put B R := B(0, R) and B := B(0, 1). For nonempty subsets K and D of R m , the extended Hausdorff distance between K and D is defined by the formula By Theorem 2.1 (ELC), we obtain the following corollary.

Graphical and Epigraphical Representations of the Hamiltonian
x, ·). We show differences between graphical and epigraphical representations of the Hamiltonian, whose conjugate is unbounded on the effective domain.
Let us define the Hamiltonian H H H : This Hamiltonian satisfies the conditions (H1)-(H4) and (HLC). Its conjugate H H H * : Since H H H satisfies (H1)-(H4) and (HLC), we can construct an epigraphical representation (R × R, f , l) of H H H like in Theorem 4.1. However, the method of constructing a graphical representation given in [11,12] cannot be applied to H H H , since the parametrization theorem of set-valued maps involves closed-valued maps.
is not a closed-valued map. Therefore, we cannot utilize this approach to parametrize x → dom H H H * (x, ·). Nevertheless, to parametrize . Let l l l by given by Of course, (R × R, f , l l l) is a graphical representation of H H H . However, the function l l l at the point (0, 0, r ) is discontinuous for r > 0. Indeed, let a 1 = r |x|/(1 +r ) and a 2 = r with x ∈ R, r > 0. We observe that (a 1 , a 2 ) ∈ epi H H H * (x, ·). By the extra-property (A3) of Theorem 4.1, we have Let a 1i = r |x i |/(1 + r ) and a 2i = r with This contradicts the fact that r > 0. The lack of continuity of the function l l l is not surprising, because the function l l l is a composition of discontinuous and continuous functions. Such compositions usually are not continuous. However, it is not a rule. We observe that the function In .
Passing to the limit as i → ∞ we get 1 ≤ 0, a contradiction.

Main Results
In this section, we describe the main results of the paper that concern faithful representations (A, f , l) with the unbounded control set A := R n × R. Then there exist f : and . Moreover, we have the following.  , l), and, by the extra-property, epi H * (t, x, ·) = e(t, x, epi H * (t, x, ·)). Hence, e(t, x, R n+1 ) = epi H * (t, x, ·). Thus, the extra-property implies that (R n+1 , f , l) is an epigraphical representation of H . It turns out that the extra-property is much stronger than e(t, x, R n+1 ) = epi H * (t, x, ·). Indeed, we consider the absolutely continuous

Correlation Between Variational and Optimal Control Problems
In this subsection, we consider a special kind of variational and optimal control problems describing solutions of the Hamilton-Jacobi equation with Hamiltonian H , which satisfies (H1)-(H4) and (HLC). These problems are theoretical in nature. Nevertheless, they can be useful in investigating practical problems. For instance, using these variational and optimal control problems we prove stability of value functions and local Lipschitz continuity of the value function. We consider the following variational problem and the following optimal control problem minimize where A([t 0 , T ], R n ) denotes the space of absolutely continuous functions. Besides, ifx(·) is the optimal arc of (P v ) such thatx(·) ∈ dom , then (x,ā)(·) is the optimal arc of (P c ) withā(·) = (ẋ(·), H * (·,x(·),ẋ(·))) such that (x,ā)(·) ∈ dom . Conversely, if (x,ā)(·) is the optimal arc of (P c ), thenx(·) is the optimal arc of (P v ).
The indicator function ψ K (·) of the set K has value 0 on this set and +∞ outside. Let S f (t 0 , x 0 ) denotes the set of all trajectory-control pairs (x, a)(·) of the control system: , we obtain:

Remark 4.3
Proofs of stability of value functions in paper [10] were omitted, because they base on to simple methods. However, standard tools cannot be applied to Theorem 4.5, because we consider the optimal arc ( Then one can prove that the sequence {x i (t)} is bounded in R n . However, the sequence {a i (t)} is, in general, not bounded in R n+1 . This means that to the sequence {(x i (t), a i (t))} one cannot apply Theorem 4.3, because this theorem works only on compact subsets of the set R n × R n+1 . Therefore, we decided to strengthen Theorem 4.3 to work on sets of the type B R × R n+1 . It can be done by assuming significantly stronger convergence of Hamiltonians than the one considered in this paper, see [19,Theorem 3.14]. However, the strengthened Theorem 4.3 turned out to be needless, because introducing the nonstandard method of the proof overcame the above problem, see Sect. 6.
One can prove that the family {x π (·) : π ∈ } is equi-bounded by a constant function. However, the family {a π (·) : π ∈ } is not bounded in general. Whereas, if g is a locally Lipschitz function, then there exists an integrable function that equi-bounds the family {a π (·) : π ∈ }, see [20,Theorem 4.7]. Thus, the family {l(·, x π (·), a π (·)) : π ∈ } can be bounded by an integrable function. It turns out that proceeding adequately in the proof of Theorem 4.6 we can omit equi-boundedness of optimal controls, see Sect. 6. In the literature, the above problem is solved assuming boundedness of the function l independent of a, see [8].

Let g be a continuous/lower semicontinuous function. Then the value function associated with (R n+1 , f , l, g) is continuous/lower semicontinuous on [0, T ] × R n .
Remark 4.5 Theorem 4.7 is a direct consequence of Theorems 6.1 and 6.3.

Representation, Optimality and Stability Theorems
The support function σ (K , ·) : R m → R of a nonempty, convex, compact set K ⊂ R m is a convex real-valued function defined by Let m−1 denote the unit sphere in R m and let μ be the measure on m−1 proportional to the Lebesgue measure and satisfying μ( m−1 ) = 1.  2d(a, E(t, x))).
We notice that the set-valued map is defined as in the proof of Theorem 5.6 from [10] with ω ≡ 1. Next, we define the single-valued map e by e(t, x, a) := s m ( (t, x, a)), where s m in the Steiner selection. Since is defined as in the proof of Theorem 5.6 from [10], so the single-valued map e is well-defined. Moreover, e(·, x, a) is measurable for every (x, a) ∈ R n × R m and e(t, ·, ·) is continuous for every t ∈ [0, T ]. If we assume that ω ≡ 1, then by [10,Theorem 5.6] and [10, Lemma 5.1] we obtain (a 4 ) and (a 5 ). It remains to prove (a 1 )-(a 3 ).
Because of Corollaries 2.1 and 2.2, the function E satisfies assumptions of Theorem 5.1. Therefore, there exists a map e : [0, T ] × R n × R n+1 → R n+1 such that e(·, x, a) is measurable for every (x, a) ∈ R n × R n+1 and e(t, ·, ·) is continuous for every t ∈ [0, T ]. Moreover, the map e satisfies (a 1 )-(a 5 ) from Theorem 5.1. By Theorem 5.1 (a 4 ) and Corollary 2.2, we have for all t ∈ [0, T ], x, y ∈ B R , a, b ∈ R n+1 , R > 0. It means that (e 4 ) is satisfied. Moreover, if we assume that H is continuous, then by Corollary 2.1 we get that (E6) is verified. Thus, by Theorem 5.1 (a 5 ), we obtain that the map e is continuous. We observe that (e 1 ) and (e 2 ) follow from definition of E and the properties (a 1 ) and (a 2 ) in Theorem 5.1. It remains to prove (e 3 ). E(t, x). The latter, together with Theorem 5.1 (a 3 ), implies This completes the proof of the theorem. e(t, x, a)) and l(t, x, a) := π 2 (e(t, x, a)), where π 1 (v, η) = v and π 2 (v, η) = η for all v ∈ R n and η ∈ R. Then for all t ∈ [0, T ], x ∈ R n , a ∈ R n+1 the equality e(t, x, a) = ( f (t, x, a), l(t, x, a)) holds. Thus, for all t ∈ [0, T ], x, y ∈ R n , a, b ∈ R n+1 , we get |l(t, x, a)| ≤ |e(t, x, a)|, From the above inequalities, it follows that the properties of the function e are inherited by functions f and l. It is not difficult to show that Theorem 4.1 follows from [10, Proposition 5.7], Theorem 5.2 and Corollary 2.1 (E5).

The Stability Theorems
The proofs of Theorems 4.2 and 4.3 are consequences of [10, Theorem 6.6 and Remark 6.7], if we assume that ω i ≡ 1 for all i ∈ N ∪ {0} and H 0 = H . In [10], one assumed that for all i ∈ N ∪ {0} the function ω i is given by where c i is coefficient in (H4) and λ i is upper-boundedness of H * i . In [10, Theorem 6.6], convergence ω i to ω 0 is required. For this reason, in [10, Theorems 3.8 and 3.9] one assumes convergence H i to H 0 as well as convergence λ i to λ 0 and c i to c 0 . Since, in our case ω i ≡ 1, so Theorems 4.2 and 4.3 do not need convergence c i to c 0 .

Upper Semicontinuity of Value Functions
Theorem 6.1 Let (R n+1 , f i , l i ) and (R n+1 , f , l) be as in Theorem 4.5. Assume that g i and g are continuous functions and g i converge to g uniformly on compacts in R n . Let V i and V be the value functions associated with (R n+1 , f i , l i , g i ) and (R n+1 , f , l, g), respectively. Then, for every (t 0 , x 0 ) ∈ [0, T ] × R n , we have Without loss of generality, we may assume V (t 0 , x 0 ) < +∞ since otherwise there is nothing to prove. Then by Corollary 4.1, there exists the optimal arc (x,ā)(·) of V (t 0 , x 0 ) defined on [t 0 , T ]. We extendā(·) from [t 0 , T ] to [0, T ] by settingā(t) = 0 for t ∈ [0, t 0 ]. Next, because of the sublinear x i0 ). Then, our assumptions and Gronwall's Lemma imply where ω R [·] := 2μ(·) + (10(n We notice that ) converge to f (t, ·,ā(t)) uniformly on compacts in R n for all t ∈ [0, T ], we have lim i→∞ i (t) = 0 for all t ∈ [0, T ]. Therefore, by virtue of Lebesgue's theorem and (16), we obtain Observe that (15), together with (17), implies lim i→∞ x i −x = 0. Since l i (t, ·,ā(t)) and l(t, ·,ā(t)) are continuous, l i (t, ·,ā(t)) converge to l(t, ·,ā(t)) uniformly on compacts in R n and x i (t) →x(t) for all t ∈ [0, T ], we have l i (t, x i (t),ā(t)) → l(t,x(t),ā(t)) for all t ∈ [0, T ]. By Theorem 4.1 (A2) we get |l i (t, Again by our assumptions and Gronwall's Lemma, we obtain where l i [x i ](·) := l i (·, x i (·),ā(·)) and l[x](·) := l(·,x(·),ā(·)). Since g i and g are continuous functions, g i converge to g uniformly on compacts in R n , and x i (T ) → x(T ), we see that g i (x i (T )) → g(x(T )). The latter, together with (18) and (19), imply that lim sup i→∞ V i (t i0 , x i0 ) ≤ V (t 0 , x 0 ). (R n+1 , f , l) be as in Theorem 4.5. Assume that g i and g are proper, lower semicontinuous and e-lim i→∞ g i = g. Let V i and V be the value functions associated with (R n+1 , f i , l i , g i ) and (R n+1 , f , l, g), respectively. Then, for every (t 0 , x 0 ) ∈ [0, T ] × R n , we have Without loss of generality, we may assume V (t 0 , x 0 ) < +∞ since otherwise there is nothing to prove. Then, by Corollary 4.1, there exists the optimal arc (x,ā) , similarly as (14) and (15), we get z i ∨ x ≤ R, Similarly to (17) we show that lim i→∞ f i [z i ] − f [z i ] L 1 = 0. The latter and (20), together with z i0 →x(T ), imply that lim i→∞ z i −x = 0. Hence we obtain z i (t 0 ) → x(t 0 ) = x 0 . Moreover, similarly to (18), we can also show that lim Passing to the limit in the above inequality, we get lim

Lower Semicontinuity of Value Functions
We consider the set-valued maps: Assume that u i (T ) ≥ M for all i ∈ N and some constant M. Then there exist a function Moreover, there exist a subsequence ( The proof of Lemma 6.1 is similar to the proof of Lower Closure theorem from the monograph of Cesari [22], so we omit it (see [20,Sect. 8] for more details).
We show that V (t 0 , x 0 ) ≤ . Without loss of generality, we may assume t i0 < T and < +∞. By definition of , there exists a subsequence (we do not relabel) such that V i (t i0 , x i0 ) → . Hence V i (t i0 , x i0 ) < +∞ for all large i ∈ N. By Corollary 4.1 there exist Since V i (t i0 , x i0 ) < +∞ for all large i ∈ N, we have H * i (t, x i (t),ẋ i (t)) < +∞ for a.e. t ∈ [t i0 , T ] and all large i ∈ N. The latter, together with (C5), implies that |ẋ i (t)| ≤ c(t)(1 + |x i (t)|) for a.e. t ∈ [t i0 , T ] and all large i ∈ N. Thus, because of Gronwall's Lemma, for all large i ∈ N, x i (·) ≤ sup i∈N |x i0 | + c L 1 exp c L 1 =: R.

Remarks
In the proof of stability of value functions, we used the formula (8) on the value function as well as the formula (7). We do that, because formulas (7) and (8) have advantages and drawbacks.
The advantages of the formula (8) are regularities of functions f and l such that: a sublinear growth of the function f with respect to the state variable, a sublinear growth of the function l with respect to the control variable and local Lipschitz continuity with respect to the state variable for both functions f and l. These regularities of functions f and l together with the extra-property allow us to prove upper semicontinuity of value functions. On the other hand, the problems appear in the proof of lower semicontinuity of value functions. They can be overcome using convexity and coercivity of the function l with respect to the control variable, see [23,24]. However, in our case the function l does not possess these properties and it is a drawback of the formula (8).
Lower semicontinuity of value functions is proven using the formula (7). It is possible due to convexity and coercivity of the conjugate H * (t, x, ·). These properties of the conjugate H * are advantages of the formula (7). The example of the Hamiltonian H in Sect. 3 shows that the conjugate H * is an extended-real-valued function and discontinuous on the effective domain dom H * . These properties of the conjugate H * are drawbacks of the formula (7).