Representation of Hamilton-Jacobi equation in optimal control theory with unbounded control set

In this paper we study the existence of sufficiently regular representations of Hamilton-Jacobi equations in the optimal control theory with unbounded control set. We use a new method to construct representations for a wide class of Hamiltonians. This class is wider than any constructed before, because we do not require Legendre-Fenchel conjugates of Hamiltonians to be bounded. However, in this case we obtain representations with unbounded control set. We apply the obtained results to study regularities of value functions and correlations between variational and optimal control problems.


INTRODUCTION
The Hamilton-Jacobi equation with a convex Hamiltonian H in the gradient variable can be studied with connection to calculus of variations problems. Let H * be the Legendre-Fenchel conjugate of H in its gradient variable: Then the value function of the calculus of variations problem defined by
The Hamilton-Jacobi equation (1.1) can be also studied with connection to optimal control problems. It is possible, provided that there exists a sufficiently regular triple (A, f, l) satisfying the following equality Then the value function of the optimal control problem defined by
While working with optimal control problems we require f and l to be locally Lipschitz continuous functions with respect to x. In addition to this f is to have the sublinear growth with respect to x and l is to have the sublinear growth with respect to a. This guarantees that to every integrable control a(·) on [t 0 , T ] with values in a closed subset A of R m there corresponds the unique solution x(·) of (1.5) defined on [t 0 , T ] and l(·, x(·), a(·)) is an integrable function on [t 0 , T ].
The triple (A, f, l), which satisfies the equality (1.3), is called a representation of H. In general, if a representation of H exists, then infinitely many other representations exist. There are also irregular representations among them. The triple (A, f, l), which satisfies the equality (1.3) and inherits Lipschitz-type properties of the Hamiltonian H, is called a faithful representation of the Hamiltonian H In this paper we provide further developments of representation theorems from [13]. Misztela [13] studied faithful representations of Hamiltonians with the compact control set. A necessary condition for the existence of such representations is boundedness of Legendre-Fenchel conjugates of Hamiltonians on effective domains; see [13,Thm. 3.1]. However, in many cases Hamiltonians do not have bounded Legendre-Fenchel conjugates on effective domains. In Section 4 we see that for this type of Hamiltonians there exist faithful representations with the unbounded control set. We used a new method to construct a faithful representation. Our representation (A, f, l) of H is an epigraphical representation, i.e. a triple (A, f, l) which satisfies the following condition (1.6) gph H * (t, x, ·) ⊂ ( f (t, x, A), l(t, x, A)) ⊂ epi H * (t, x, ·), where ( f (t, x, A), l(t, x, A)) denotes the set {( f (t, x, a), l(t, x, a)) | a ∈ A}. The construction of this representation is as follows: first, using the Steiner Selection we parametrize the set epi H * (t, x, ·) in such a way that e(t, x, A) = epi H * (t, x, ·). The Steiner Selection guarantees that e is local Lipschitz continuous with respect to x. Next, we define the functions f , l as components of the function e, i.e. e = ( f, l). From the equality e(t, x, A) = epi H * (t, x, ·) it follows that (1.6) holds. In view of [10,Prop. 5.7] any triple (A, f, l) satisfying (1.6) is a representation of H. Earlier, Frankowska-Sedrakyan [9] and Rampazzo [17] used a graphical representation to construct a faithful representation. The representation (A, f, l) of H is a graphical representation, if a triple (A, f, l) satisfies ( f (t, x, A), l(t, x, A)) = gph H * (t, x, ·).
In a graphical representation the function l, without additional assumptions on H * , may be discontinuous with respect to (x, a); see Section 3. Another differences between graphical and epigraphical representations can be found in [13]. Earlier, Ishii [10] proposed a representation involving continuous functions f , l with the infinite-dimensional control set A.
The lack of local Lipschitz continuity of f and l with respect to x and finite-dimensional control set A in Ishii [10] paper causes troubles in applications.
We present differences between representations with unbounded and compact control sets. The fact that a control set is not compact makes significant problems in applications which we discuss below. Therefore, compactness of a control set must be replaced by another property that is convenient in practice. The following property which is a consequence of our construction of a faithful representation plays a role of such extra-property: Our extra-property is apparently new. In literature one usually requires coercivity of the function l(t, x, ·); see, e.g. [18,Condition (A 4 )]. However, the function l(t, x, ·) from our faithful representation (A, f, l) does not have this property. Coercivity of the function l(t, x, ·) enables us to study not only measurability of controls but also its integrability. In this paper the extra-property plays a similar role; see Remarks 4.2 and 4.7. It is wellknown that in applications one requires at least integrability of controls. In the case when the control set is compact the above problem does not occur, because every measurable control with values in the compact control set is integrable.
In general, the value functions (1.2) and (1.4) are not equal. However, in our case these value functions are identical due to the extra-property; see Corollary 4.6. Moreover, we obtain a fundamental relation between variational and optimal control problems; see Theorem 4.5. More precisely, we consider a variational problem associated with the given Lagrangian L. We define Hamiltonian H as the Legendre-Fenchel transform of L in its velocity variable. Applying our result to Hamiltonian H we obtain its faithful representation (A, f, l). Then the variational problem associated with Lagrangian L is equivalent to the optimal control problem associated with the triple (A, f, l). Earlier, Olech [14] and Rockafeller [19,20] investigated the opposite problem. They considered the optimal control problem associated with the given triple (A, f, l). Using this triple they defined Lagrangian L in such a way that the optimal control problem associated with the triple (A, f, l) is equivalent to the variational problem associated with Lagrangian L.
Our faithful representations are stable; see Theorems 4.8 and 4.10. This fact is used in the proof of stability of value functions; see Section 6. The method of this proof is not standard, because properties of a faithful representation are nonstandard. These nonstandard properties are unbounded control set, the extra-property and the sublinear growth of l with respect to a. In this case one cannot apply methods from Sedrakyan [22] to prove stability of value functions. Indeed, this method uses compactness of the control set and boundedness of l independent of a. We also prove that the value function V is locally Lipschitz continuous, provided that g is locally Lipschitz continuous. In the proof of this fact nonstandard boundedness of the function l plays significant role.
The outline of the paper is as follows. Section 2 contains hypotheses and preliminary results. In Section 3 we show differences between graphical and epigraphical representations with the unbounded control set. In Section 4 we gathered our main results. Sections 5 and 6 contain proofs of results from Section 4. Section 7 contains concluding remarks.

HYPOTHESES AND BACKGROUND MATERIAL
We will need hypotheses and results similar to those in [13,Sect. 2].
Let ϕ be an extended-real-valued function. The sets: are called the effective domain, the graph and the epigraph of ϕ, respectively. We say that ϕ is proper if it never takes the value −∞ and it is not identically equal to +∞. Using properties of the Legendre-Fenchel conjugate from [21] we can prove the following proposition.

Additionally, if H is continuous, then H * is lower semicontinuous and
Let K be a nonempty subset of R m . We put K := sup ξ∈K |ξ|. The distance from y ∈ R m to K is defined by d(y, K) := inf ξ∈K |y−ξ|. Let us define the set-valued map E H * : [0, T ] × R n ⊸ R n × R by the formula From Proposition 2.1 and Results in [21,Chap. 14] we deduce the following corollary.
x) has a closed graph and is lower semicontinuous. Now we present Hausdorff continuity of a set-valued map E H * in Hamiltonian and its conjugate terms. Let IB(x, R) denote the closed ball in R n of centerx and radius R 0. We put IB R := IB(0, R) and IB := IB(0, 1).
(CLC) For any R > 0 there exists a measurable map k R : (ELC) For any R > 0 there exists a measurable map k R : Equivalences hold for the same map k R (·).
For nonempty subsets K, D of R m , the extended Hausdorff distance between K and D is defined by the formula By Theorem 2.3 (ELC) we obtain the following corollary.

GRAPHICAL AND EPIGRAPHICAL REPRESENTATIONS OF THE HAMILTONIAN
In this section we show differences between graphical and epigraphical representations of the Hamiltonian whose conjugate is unbounded on the effective domain.
Since IH satisfies (H1)-(H4) and (HLC) we can construct an epigraphical representation (R × R, f, l) of IH like in Theorem 4.1 below. However, the method of constructing a graphical representation given in [9,17] cannot be applied to IH, since the parametrization theorem of set-valued maps involves closed-valued maps. However, x → dom IH * (x, ·) is not a closed-valued map. Therefore we cannot utilize this approach to parametrize x → dom IH * (x, ·). Nevertheless, to parametrize x → dom IH * (x, ·) we can use an epigraphical Of course, (R × R, f, ll) is a graphical representation of IH. However, the function ll at the point (0, 0, r) is discontinuous for r > 0. Indeed, let a 1 = r|x|/(1 + r) and a 2 = r with x ∈ R, r > 0. We observe that (a 1 , a 2 ) ∈ epi IH * (x, ·). By the extra-property (A3) of Theorem 4.1, This contradicts the fact that r > 0.
The lack of continuity of the function ll is not suprising, because the function ll is a composition of discontinuous and continuous functions. Such compositions usually are not continuous. However, it is not a rule. We observe that the function ff(x, a) = a|x| 2 /(1 + |a| |x|) is parametrization of dom IH * (x, ·) such that ff(x, R) = dom IH * (x, ·). Then, the function ll(x, a) = IH * (x, ff(x, a)) = |a| |x| is continuous. Therefore (R, ff, ll) is a continuous graphical representation of IH. In general, it is difficult to indicate the class of Hamiltonians with discontinuous conjugates for which continuous graphical representations with the unbounded control set exist.
The Hamiltonian IH does not have a graphical representation (A, ff, ll) such that In particular, the Hamiltonian IH does not have a graphical representation (A, ff, ll) which satisfies (A1) from Theorem 4.1. Let us assume by contradiction that the Hamiltonian IH has a graphical representation (A, ff, ll) satisfying (3.3).
Passing to the limit as i → ∞ we obtain 1 0, a contradiction.

MAIN RESULTS
In this section we describe the main results of the paper that concern faithful representations (A, f, l) with the unbounded control set A := R n × R.

Correlation between variational and optimal control problems.
In this subsection we consider a special kind of variational and optimal control problems describing solutions of the Hamilton-Jacobi equation with Hamiltonian which satisfies (H1)-(H4) and (HLC). These problems are theoretical in nature. Nevertheless, they can be useful in investigating practical problems. For instance, using these variational and optimal control problems we prove stability of value functions and local Lipschitz continuity of the value function. We consider the following variational problem and the following optimal control problem Besides, ifx(·) is the optimal arc of (P v ) such thatx(·) ∈ dom Γ, then (x,ā)(·) is the optimal arc of (P c ) withā(·) = (ẋ(·), H * (·,x(·),ẋ(·))) such that (x,ā)(·) ∈ dom Λ. Conversely, if (x,ā)(·) is the optimal arc of (P c ), thenx(·) is the optimal arc of (P v ).
The indicator function ψ K (·) of the set K has value 0 on this set and +∞ outside. Applying Theorem 4.5 to φ(z, x) := ψ {x 0 } (z) + g(x), we obtain the following corollary.
Remark 4.7. Observe that the considered optimal control problem (P c ) has integrable controls. Investigating integrable controls is possible due to argumentation contained in Remark 4.2; see Subsect. 5.1. In addition, correlation between the optimal controlā(·) and the optimal trajectoryx(·) can be expressed by the simple formula (ẋ(·), H * (·,x(·),ẋ(·))) = a(·). Of course, this formula does not make sense in the case of the optimal control problem with the compact control set considered in paper [13]. Indeed, on the one hanḋ x(·) and H * (·,x(·),ẋ(·))) need not be be bounded functions. On the other hand, the control a(·) has values in a compact control set.  Remark 4.11. Proofs of stability of value functions in paper [13] were omitted, because they relate to simple methods. However, standard tools cannot be applied to Theorem 4.8 and Theorem 4.10, because we consider the optimal arc (x i , a i )(·) of V i (x i0 , a i0 ) for all i ∈ N. Fix t ∈ [0, T ]. Then one can prove that the sequence {x i (t)} is bounded in R n .
However, the sequence {a i (t)} is, in general, not bounded in R n+1 . This means that to the sequence {(x i (t), a i (t))} one cannot apply Theorem 4.4, because this theorem works only on compact subsets of the set R n × R n+1 . Therefore, we decided to strengthen Theorem 4.4 to work on sets of the type IB R × R n+1 . It can be done by assuming significantly stronger convergence of Hamiltonians than the one considered in this paper; see [arXiv:1507.01424v1, Theorem 3.14]. However, the strengthened Theorem 4.4 turned out to be needless, because introducing the nonstandard method of the proof overcame the above problem; see Section 6.
One can prove that the family { x π (·) | π ∈ Π } is equi-bounded by a constant function. However, the family { a π (·) | π ∈ Π } is not bounded in general. Whereas, if g is a locally Lipschitz function, then there exists an integrable function that equi-bounds the family { a π (·) | π ∈ Π }; see [arXiv:1807.03640v1, Theorem 4.7]. So, the family { l(·, x π (·), a π (·)) | π ∈ Π } can be bounded by an integrable function. It turns out that proceeding adequately in the proof of Theorem 4.12 we can omit equi-boundedness of optimal controls; see Section 6. In the literature the above problem is solved assuming boundedness of the function l independent of a; see [2].

REPRESENTATION, OPTIMALITY AND STABILITY THEOREMS
The support function σ(K, ·) : R m → R of a nonempty, convex, compact set K ⊂ R m is a convex real-valued function defined by Let ∑m−1 denotes the unit sphere in R m and let µ be the measure on ∑m−1 proportional to the Lebesgue measure and satisfying µ(∑m−1) = 1. (a 1 ) We consider the set-valued map defined by 2d(a, E(t, x))).
We observe that the set-valued map Φ is defined as in the proof of [13, Theorem 5.6], if we assume that ω ≡ 1. We define the single-valued map e from [0, T ] × R n × R m to R m by e(t, x, a) := s m (Φ(t, x, a)), where s m in the Steiner selection. Since Φ is defined as in the proof of [13, Theorem 5.6], so the single-valued map e is well-defined. Moreover, e(·, x, a) is measurable for every (x, a) ∈ R n × R m and e(t, ·, ·) is continuous for every t ∈ [0, T ]. If we assume that ω ≡ 1, then by [13, Theorem 5.6] and [13, Lemma 5.1] we obtain (a 4 ) and (a 5 ). It remains to prove (a 1 )-(a 3 ).

5.2.
The stability theorems. The proofs of Theorems 4.3 and 4.4 are consequences of [13, Theorem 6.6 and Remark 6.7], if we assume that ω i ≡ 1 for all i ∈ N ∪ {0} and H 0 = H. In [13] one assumed that for all i ∈ N ∪ {0} the function ω i is given by where c i is coefficient in (H4) and λ i is upper boundedness of H * i . In [13, Theorem 6.6] convergence ω i to ω 0 is required. For this reason, in [13, Theorems 3.8 and 3.9] one assumes convergence H i to H 0 as well as convergence λ i to λ 0 and c i to c 0 . Since, in our case ω i ≡ 1, so Theorems 4.3 and 4.4 do not need convergence c i to c 0 .
Moreover, similarly to (6.5), we can also show that lim Passing to the limit in the above inequality, we get lim Remark 6.3. Let (R n+1 , f, l) be as above. Assume that g is a continuous function. Let V be the value function associated with (R n+1 , f, l, g). Applying Theorem 6.1 to f i := f, l i := l, g i := g, we obtain that the value function V is upper semicontinuous.
Additionally, we assume that H i (t, ·, ·) converge uniformly on compacts to H(t, ·, ·) for all t ∈ [0, T ]. This assumptions imply that the set-valued maps x, ·) } for all t ∈ [0, T ], x ∈ R n have the following property (cf. Cesari [5,Sections 8.5 and 10.5]) Assume that u i (T ) M for every i ∈ N and some constant M. Then there exist a function (x, v)(·) ∈ A([t 0 , T ], R n × R) and a real number v 0 u 0 such that Proof. Let (t i0 , x i0 , u i0 ) → (t 0 , x 0 , u 0 ). We consider a sequence of functions (x i , u i )(·) ∈ A([t i0 , T ], R n × R) which satisfy (6.9), i.e. (x i , u i )(t i0 ) = (x i0 , u i0 ) and By our assumptions we can find a sequence t i ∈ [t i0 , T ] such that t i → t + 0 . Therefore, H * i (t,x i (t),ẋ i (t)) < +∞ for a.e. t ∈ [t i0 , T ] and all i ∈ N. The latter, together with (C5), implies that |ẋ i (t)| c(t)(1 + |x i (t)|) for a.e. t ∈ [t i0 , T ] and all i ∈ N. Therefore, because of Gronwall's Lemma, for every i ∈ N, Hence , T ] and all i ∈ N. We observe that The latter, together with . We notice that the family {x i (·)} i∈N of such extended functions is equi-bounded. Moreover, the family {ẋ i (·)} i∈N of derivatives is equi-absolutely integrable. Therefore, in view of Arzelà-Ascoli and Dunford-Pettis Theorems, there exists a subsequence (we do not relabel) such that x i (·) converges uniformely to an absolutely continuous function x : [t 0 , T ] → R n andẋ i (·) converges weakly in L 1 ([t 0 , T ], R n ) toẋ(·). We observe that x(t 0 ) = x 0 . By our assumptions we conclude that , T ] and every i ∈ N. The latter, together with (6.12), implies thatu i (t) µ R (t) for a.e. t ∈ [t i0 , T ] and every i ∈ N. Moreover, we know that u i (T ) M for every i ∈ N. Therefore, for all t ∈ [t i0 , T ] and i ∈ N. Since {u i (t i )} i∈N is bounded, we conclude that there exists a subsequence (we do not relabel) such that u i (t i ) → v 0 . We observe that for all i ∈ N The latter, together with u i0 → u 0 , implies that v 0 u 0 . Furthermore, we observe that . We notice that the family {u i (·)} i∈N of such extended functions is equi-bounded. Moreover, the variations of functions u i (·) on [t 0 , T ] are equi-bounded. Therefore, in view of Helly Theorem, there exists a subsequence (we do not relabel) such that u i (·) converges pointwise (everywhere) to a bounded variation function u : [t 0 , T ] → R.
Theorem 6.5. Let H i , H, i ∈ N be as above. Assume that g i and g are proper, lower semicontinuous functions and e-lim i→∞ g i = g. Let V i and V be the value functions associated with (H * i , g i ) and (H * , g), respectively. Then for every (t 0 , x 0 ) ∈ [0, T ] × R n lim inf i → ∞ V i (t i0 , x i0 ) V(t 0 , x 0 ) for every sequence (t i0 , x i0 ) → (t 0 , x 0 ).
Without loss of generality we may assume t i0 < T and ∆ < +∞. Because of the definition ∆, there exists a subsequence (we do not relabel) such that V i (t i0 , x i0 ) → ∆. Hence for all large i ∈ N we have V i (t i0 , x i0 ) < +∞. In view of Corollary 4.6 there exist a sequence of functions x i (·) ∈ A([t i0 , T ], R n ) such that x i (t i0 ) = x i0 and Since V i (t i0 , x i0 ) < +∞ for all large i ∈ N, we have H * i (t, x i (t),ẋ i (t)) < +∞ for a.e. t ∈ [t i0 , T ] and all large i ∈ N. The latter, together with (C5), implies that |ẋ i (t)| c(t)(1 + |x i (t)|) for a.e. t ∈ [t i0 , T ] and all large i ∈ N. Thus, because of Gronwall's Lemma, for all large i ∈ N, sup i∈N |x i0 | + Hence |ẋ i (t)| (1 + R) c(t) for a.e. t ∈ [t i0 , T ] and all large i ∈ N. By our assumptions we conclude that H * i (t, x i (t),ẋ i (t)) −µ R (t) for a.e. t ∈ [t i0 , T ] and all large i ∈ N. Because of e-lim i→∞ g i = g, there exists a constant M such that g i (x) M for all x ∈ IB R and all large 6.3. Remarks. In the proof of stability of value functions we used the formula (1.4) on the value function as well as the formula (1.2). We do that, because formulas (1.2) and (1.4) have advantages and drawbacks.
The advantages of the formula (1.4) are regularities of functions f and l such that: a sublinear growth of the function f with respect to the state variable, a sublinear growth of the function l with respect to the control variable and local Lipschitz continuity with respect to the state variable for both functions f and l. These regularities of functions f and l together with the extra-property allow us to prove upper semicontinuity of value functions. On the other hand, the problems appear in the proof of lower semicontinuity of value functions. They can be overcome using convexity and coercivity of the function l with respect to the control variable; see [3,15]. However, in our case the function l does not possess these properties and it is a drawback of the formula (1.4).
Lower semicontinuity of value functions is proven using the formula (1.2). It is possible due to convexity and coercivity of the conjugate H * (t, x, ·). These properties of the conjugate H * are advantages of the formula (1.2). The example of the Hamiltonian H in Section 3 shows that the conjugate H * is an extended-real-valued function and discontinuous on the effective domain dom H * . These properties of the conjugate H * are drawbacks of the formula (1.2).  where ω R [·] = 2|H(·, 0, 0)| + (10 (n + 1) k R (·) + 2c(·))(1 + R) and D R denotes the Lipschitz constant of g on B R . Let us consider the following function Proposition 6.7. Consider (R n+1 , f, l) as above. Assume that g is a real-valued lower semicontinuous function. If V is the value function associated with (R n+1 , f, l, g), then V is a real-valued function on [0, T ] × R n .