# On Nonuniqueness of Solutions of Hamilton–Jacobi–Bellman Equations

- 356 Downloads

## Abstract

An example of a nonunique solution of the Cauchy problem of Hamilton–Jacobi–Bellman (HJB) equation with surprisingly regular Hamiltonian is presented. The Hamiltonian *H*(*t*, *x*, *p*) is locally Lipschitz continuous with respect to all variables, convex in *p* and with linear growth with respect to *p* and *x*. The HJB equation possesses two distinct lower semicontinuous solutions with the same final conditions; moreover, one of them is the value function of the corresponding Bolza problem. The definition of lower semicontinuous solution was proposed by Frankowska (SIAM J. Control Optim. 31:257–272, 1993) and Barron and Jensen (Commun. Partial Differ. Equ. 15(12):1713–1742, 1990). Using the example an analysis and comparison of assumptions in some uniqueness results in HJB equations is provided.

## Keywords

Hamilton–Jacobi–Bellman equation Optimal control theory Nonsmooth analysis Viscosity solution## Mathematics Subject Classification

35Q93 49L25 49J52## 1 Introduction

*H*is convex in the gradient variable, then there are relations between solutions of HJB and optimization problems involving a function dual to

*H*. This function, called the Lagrangian and denoted by

*L*, is the Legendre–Fenchel transform of

*H*in its gradient variable:

*H*and

*g*,

*V*is the unique solution of HJB. Since we use the nonsmooth analysis we need a notion of a subgradient. For a vector \(v\in {\mathbb {R}}^n\) and a function \(f:{\mathbb {R}}^n\rightarrow {\mathbb {R}}\cup \{+\infty \}\),

*v*is a subgradient of

*f*at \(x\in {\mathrm {dom}}\,f\), written \(v\in \partial f(x)\), if

*lower semicontinuous solutions*.

## Definition 1.1

- (i)
*U*is lower semicontinuous function and \(U(T,x)=g(x)\); - (ii)For every \((t,x)\in {\mathrm {dom}}\,U\), every \((p_{t},p_{x})\in \partial U(t,x)\) the following holds:$$\begin{aligned} \left\{ \begin{array}{lll} -p_{t}+H(t,x,-p_{x})\geqslant 0 &{} \,{if}\, &{} t\in [0,T[, \\ -p_{t}+H(t,x,-p_{x})\leqslant 0 &{} \,{if}\, &{} t\in \, ]0,T]. \end{array}\right. \end{aligned}$$(1.4)

*classical assumptions*i.e. firstly, it is convex with respect to

*p*, secondly, it increases linearly in

*p*and

*x*, i.e. \(|H(t,x,p)|\leqslant 2|p|\) for any \(t\in [0,T]\), \(x,p\in {\mathbb {R}}\), thirdly, it is locally Lipschitz continuous, i.e.

Frankowska [7] proved that, the value function is the unique lower semicontinuous solution of the HJB equation if the Hamiltonian meets the classical assumptions and it is positively homogeneous in *p*, i.e. \(\forall _{r>0}\,H(t,x,rp)=rH(t,x,p)\). Actually, the result of Frankowska does not require local Lipschitz continuity with respect to the triple (*t*, *x*, *p*). It is enough to assume it is satisfied with respect to state variable *x* only. The example of nonuniqueness of solution of HJB introduced in the current paper, does not contradict the result of Frankowska as the Hamiltonian in our example, fulfills the classical assumptions, but it is not positively homogeneous in *p*.

*p*and

*x*of Hamiltonian, the following condition

*p*and possesses a linear growth of the form \(H(t,x,p)\leqslant 2(1+|x|)|p|\), then condition (1.5) holds with constant \(C=2\). Therefore the Hamiltonian from our example of nonuniqueness also satisfies (1.5). Next, the results from papers [11, Theorem 2.5] and [5, Theorem VI.1] require the Lipschitz-type condition for the Hamiltonian:

*p*, but the uniqueness of HJB solutions is obtained in the class of continuous functions. Using these results, Bardi and Capuzzo-Dolcetta [2, Chapter V, Theorem 5.16] showed the uniqueness of HJB solutions in the class of lower semicontinuous functions assuming additionally the convexity of the Hamiltonian in

*p*. Because the Hamiltonian in our example of nonuniqueness satisfies the classical assumptions, also the condition (1.5) is satisfied. It means that in the uniqueness results, the key point is (SLC), at least in the case of lower semicontinuous solutions of the HJB equation.

In order to understand better the reason for nonuniqueness of the solution of the HJB equation given in our example, we need to recall the Loewen–Rockafellar condition from [12] that is more general version of (SLC). This condition was used by Loewen and Rockafellar [13, 14] to study necessary conditions satisfied by optimal solutions of Bolza problem. Galbraith [9] proved that, the value function is the unique lower semicontinuous solution of HJB assuming the Loewen–Rockafellar condition [12]. This uniqueness result has, in general, the same nature as the uniqueness result of Dal Maso and Frankowska [6]. The Hamiltonian from our example of nonuniqueness is constructed using the function \(\varphi \). In Sect. 6 we show that if \(\varphi \) is replaced by a function that is sufficiently regular, then we obtain the Hamiltonian that does not satisfy the condition (SLC), while it satisfies the Loewen–Rockafellar condition [12]. Therefore, by virtue of classical results, we are not able to say if after the mentioned change, we get the uniqueness of the solution of HJB or not. However, using the Galbraith result [9], we know that HJB with this Hamiltonian has the unique solution. Thus, the HJB equation can have the unique solution even if the Hamiltonian does not satisfy the condition (SLC). Moreover, the Hamiltonian from our example of nonuniqueness shows the difference between (LLC), (SLC) and Loewen–Rockafellar condition [12]. We know that the condition (LLC) does not guarantee uniqueness of the solution to the HJB equation, so the natural question can be stated—what extra conditions the Hamiltonian should satisfy in order the HJB equation has the unique solution? The answer to this question is obtained when analysing conditions (LLC), (SLC) and Loewen–Rockafellar condition [12]. Namely, some extra conditions in the interdependence between space and subgradient for large values of the subgradient are mandatory (see Sect. 2).

*x*. It means that these conditions play important roles in the uniqueness of lower semicontinuous solutions of HJB. Notice that Hamiltonians from the above two examples of nonuniqueness do not satisfy the classical assumptions, but the Hamiltonian from our example of nonuniqueness does. In the transport equation we cannot find the example of nonuniqueness as ours. Because on one hand the HJB equation with the Hamiltonian satisfying the classical assumptions and positively homogeneous in

*p*has the unique solution (see [7]). On the other hand conditions (LLC), (SLC) and Loewen–Rockafellar condition [12] are equivalent, if the Hamiltonian is convex in

*p*, positively homogeneous in

*p*and continuous. Therefore, we cannot find differences between them while considering Hamiltonians from the transport equation.

Summarizing, the above examples of nonuniqueness show that the key role in the uniqueness of viscosity solutions of HJB is played by the Lipschitz-type conditions originated in optimization instead of local Lipschitz continuity originated in differential equations theory. Therefore our result is proper for optimization problems.

We have found the example of nonuniqueness presented here while generalizing a result of Plaskacz and Quincampoix [17]. This generalization was published in [16].

## 2 Existence and Uniqueness Theorem

- (H1)
\(H:[0,T]\times {\mathbb {R}}^{n}\times {\mathbb {R}}^{n}\rightarrow {\mathbb {R}}\) is continuous;

- (H2)
*H*(*t*,*x*,*p*) is convex with respect to*p*for every \((t,x)\in [0,T]\times {\mathbb {R}}^n\); - (H3)
there exists a constant \(C>0\) such that for all \((t,x,p)\in [0,T]\times {\mathbb {R}}^n\times {\mathbb {R}}^n\) the following inequality is satisfied \(H(t,x,p)\leqslant C(1+|x|)|p|\).

*a linear growth in p and x of the Hamiltonian*. Basing on (H1)–(H3) we can state the following theorem on existence and uniqueness:

## Theorem 2.1

We suppose that *H* satisfies (H1)–(H3) and (SLC). Let *g* be lsc and bounded from below, and *V* be the value function associated with *g* and *L*, where *L* is given by (1.1). Then the value function *V* is bounded from below lower semicontinuous solution of HJB. Moreover, if *U* is bounded from below lower semicontinuous solution of HJB, then \(U=V\) on \([0,T]\times {\mathbb {R}}^n\).

*L*we extend it in the following way: \(L(t,x,v):=L(0,x,v)\) for \(t<0\) and \(L(t,x,v):=L(T,x,v)\) for \(t>T\).

*p*, a mild growth of Hamiltonian with respect to

*p*(see (A1) from [9]) and slightly more general Lipschitz-type condition than (2.2) (see (A2) from [9]). Thus, if we replace the condition (SLC) by (2.2) in Theorem 2.1, then the claim of Theorem 2.1 still holds. In [9, 10, 12, 13, 14] the authors use a limiting subgradient that is larger than the regular subgradient we use. We know that replacing the regular subgradient by the limiting subgradient one the condition (2.2) is unchanged if we assume that Hamiltonian is continuous. We also know that the condition (2.2) implies (LLC) (see [9], Proposition 2.4). Therefore, assuming conditions (H1)–(H3) we obtain

## 3 Example of Hamiltonian

*H*given by (3.2) is continuous on \([0,T]\times {\mathbb {R}}\times {\mathbb {R}}\), convex with respect to

*p*for each \((t,x)\in [0,T]\times {\mathbb {R}}\) and has a linear growth in

*p*and

*x*, i.e. \(|H(t,x,p)|\leqslant 2|p|\) for all \(t\in [0,T]\), \(x,p\in {\mathbb {R}}\), so it satisfies (H1)–(H3).

## Theorem 3.1

*H*given by (3.2) satisfies locally the Lipschitz continuity, i.e. for each \((t_0,x_0,p_0)\in [0,T]\times {\mathbb {R}}\times {\mathbb {R}}\) there exist numbers \(r,k>0\) such that

## Proof

For \(\theta \geqslant 0\) we define an auxiliary function *f* by the formula \(f(\theta )=\sqrt{\theta }\exp (2\sqrt{\theta })\). We notice that the function *f* is increasing and \(f(|t-x|)=\varphi (t,x)\) for each \(t\in [0,T]\), \(x\in {\mathbb {R}}\). Furthermore, the function *f* on [*a*, *b*] satisfies Lipschitz continuity if \(0<a<b\). We fix \(t_0\in [0,T]\), \(x_0\in {\mathbb {R}}\) and \(p_0\in {\mathbb {R}}\), and consider two cases.

*Case 1*Let \(p_0\in {\mathbb {R}}\) and \(t_0=x_0\). We define \(r:=1/[2\exp (4)(1+2|p_0|)^2]\) and \(k:=1\). We notice that \(|p|\leqslant r+|p_0|\) for each \(p\in \mathbb {B}(p_0,r)\) and \(|t-x|\leqslant 2r\) for each \(t\in \mathbb {B}(t_0,r)\), \(x\in \mathbb {B}(x_0,r)\), with \(r\leqslant 1/2\). Therefore, for \(t\not =x\) we obtain

*Case 2* Let \(p_0\in {\mathbb {R}}\) and \(t_0\not =x_0\). We define *r* by the formula \(r:=|t_0-x_0|/3\), then \(r\leqslant |t-x|\leqslant 5r\) for each \(t\in \mathbb {B}(t_0,r)\), \(x\in \mathbb {B}(x_0,r)\). Let *l* be the Lipschitz constant of the function *f* on [*r*, 5*r*]. We define *k* by \(k:=2+l/f^2(r)\).

- (a)
For \(2|p|\leqslant 1/\varphi (t,x)\) and \(2|q|\leqslant 1/\varphi (s,y)\) we have \({LS}(3.3)= 0 \leqslant {RS}(3.3)\).

- (b)For \(2|p|\geqslant 1/\varphi (t,x)\) and \(2|q|\geqslant 1/\varphi (s,y)\) we have inequalities$$\begin{aligned} {LS}({3.3})\leqslant & {} 2|p-q|+\frac{1}{f(|t-x|)f(|s-y|)}|f(|t-x|)-f(|s-y|)|\\\leqslant & {} 2|p-q|+\frac{l}{f^2(r)}\left( |t-s|+|x-y|\right) \,\,\leqslant \,\, {RS}({3.3}). \end{aligned}$$
- (c)For \(2|p|\geqslant 1/\varphi (t,x)\) and \(2|q|\leqslant 1/\varphi (s,y)\) we have inequalities$$\begin{aligned} {LS}({3.3}) \,\leqslant \, 2|p|-\frac{1}{\varphi (t,x)}+\frac{1}{\varphi (s,y)} -2|q|\,\leqslant \, {RS}({3.3}). \end{aligned}$$

## Proof

*L*be given by the formula (3.4). Then for \(t>x\), \(v\in \,]0,2[\) and \((w_1,w_2,p)\in \partial L(t,x,v)\) we have

## 4 An Example of Nonuniqueness

*H*given by (3.2) and the terminal condition \(g:{\mathbb {R}}\rightarrow {\mathbb {R}}\) given by the formula

*U*(

*t*,

*x*) from \([0,T]\times {\mathbb {R}}\) to entire space by setting \(+\infty \) for \((t,x)\notin [0,T]\times {\mathbb {R}}\).

### 4.1 First Solution

## Theorem 4.1

The function *U* given by (4.2) is bounded lower semicontinuous solution of the HJB equation with the Hamiltonian (3.2) and the terminal condition (4.1).

## Proof

It is not difficult to notice that the function *U* is lower semicontinuous, bounded and \(U(T\!,x)\!\!=\!\!g(x)\). We will prove that the function *U* satisfies conditions (1.4). We consider five cases.

*Case 1* Let \(x>t\) and \(t\in [0,T[\). Then \(-p_t\geqslant -1/\varphi (t,x)\) and \(-p_x =1/\varphi (t,x)\) for each \((p_t,p_x)\in \partial U(t,x)\). By definition of *H*, we have \(-p_t+H(t,x,-p_x)\geqslant 0\).

*Case 2* Let \(x>t\) and \(t\in \,]0,T]\). Then \(-p_t\leqslant -1/\varphi (t,x)\) and \(-p_x =1/\varphi (t,x)\) for each \((p_t,p_x)\in \partial U(t,x)\). By definition of *H*, we have \(-p_t+H(t,x,-p_x)\leqslant 0\).

*Case 3* Let \(x=t\) and \(t\in [0,T]\). Then \(\partial U(t,x)=\emptyset \).

*Case 4* Let \(x<t\) and \(t\in [0,T[\). Then \(-p_t\geqslant 0\) and \(-p_x =0\) for each \((p_t,p_x)\in \partial U(t,x)\). By definition of *H*, we have \(-p_t+H(t,x,-p_x)\geqslant 0\).

*Case 5* Let \(x<t\) and \(t\in \,]0,T]\). Then \(-p_t\leqslant 0\) and \(-p_x =0\) for each \((p_t,p_x)\in \partial U(t,x)\). By definition of *H*, we have \(-p_t+H(t,x,-p_x)\leqslant 0\).

That finishes the proof. \(\square \)

### 4.2 Second Solution

## Theorem 4.2

The function *V* given by (4.3) is bounded lower semicontinuous solution of the HJB equation with the Hamiltonian (3.2) and the terminal condition (4.1).

## Proof

It is not difficult to notice that the function *V* is lower semicontinuous, bounded and \(V(T,x)=g(x)\). We prove that the function *V* satisfies conditions (1.4). Since \(\partial V(t,x)=\emptyset \) for \(x=t\) and \(V(t,x)=U(t,x)\) for \(x\geqslant t\) and \(x<2t-T\), then by the Theorem 4.1 it is sufficient to show that *V* satisfies conditions (1.4), when \(2t-T\leqslant x< t\). To do it we consider two cases.

*Case 1* Let \(2t-T\leqslant x<t\) and \(t\in \,]0,T[\). Then \(-p_x\geqslant 1/\varphi (t,x)\) and \(1/\varphi (t,x)= -p_t-2p_x\) for all \((p_t,p_x)\in \partial V(t,x)\). By definition of *H*, we have \(-p_t+H(t,x,-p_x)=0\).

*Case 2* Let \(2t-T\leqslant x<t\) and \(t=0\). Then \(-p_x\geqslant 1/\varphi (t,x)\) and \(-p_t-2p_x\geqslant 1/\varphi (t,x)\) for all \((p_t,p_x)\in \partial V(t,x)\). By definition of *H*, we have \(-p_t+H(t,x,-p_x)\geqslant 0\). \(\square \)

## 5 The Value Function

In this section we show that the function *V* given by (4.3) is the value function, in the sense of definition (1.2), corresponding to the Lagrangian (3.4) and the terminal condition (4.1). To this purpose, we use the methods of [15], whose scheme is as follows: We construct a sequence of Hamiltonians \((H_n)_{n\in {\mathbb {N}}}\) such that \(H_n\) satisfy (H1)–(H3) together with (SLC) and \(H_n\searrow H\). Then, by [15, Corollary 3.5], the value functions \(V_n\) corresponding to HJB with Hamiltonians \(H_n\) and terminal conditions \(g_n=g\) converge to the value function *V* (\(V_n\nearrow V\)). Since Hamiltonians \(H_n\) additionally, satisfy the condition (SLC), then by Theorem 2.1, the value functions \(V_n\) are unique lower semicontinuous solutions. Therefore, to find the value function *V*, one needs to find solutions \(U_n\) of equations HJB with Hamiltonians \(H_n\) and the terminal condition *g* and then take the limit \(U_n=V_n\rightarrow V\).

### 5.1 Approximation of the Hamiltonian

*p*and have linear growth in

*p*and

*x*, so they satisfy conditions (H1)–(H3). Moreover \(H_n\searrow H\), because \(\varphi _n\searrow \varphi \). Similarly, for the proof of Case 2 in Theorem 3.1 we can prove that Hamiltonians (5.1) satisfy the condition (SLC).

### 5.2 Solutions \(\pmb {U_n}\) of HJB with \(\pmb {H_n}\)

Let \(U_{n}:[0,T]\times {\mathbb {R}}\rightarrow {\mathbb {R}}\) be given by formula:

- (a)if \(|t-x|\leqslant 1/n\) and \(T-2t+x\geqslant 1/n\), then$$\begin{aligned} \,\,\,\,U_n(t,x)= & {} (t-x)\sqrt{n}\exp (-2/\sqrt{n}\,)+(1+1/\sqrt{n}\,) \exp (-2/\sqrt{n}\,)-1, \end{aligned}$$
- (b)if \(|t-x|\leqslant 1/n\) and \(T-2t+x\leqslant 1/n\), then$$\begin{aligned} \!\!U_n(t,x)= & {} \exp (-2\sqrt{T-2t+x}\,)+(T-t)\sqrt{n}\exp (-2/\sqrt{n}\,)-1, \end{aligned}$$
- (c)if \(t-x\geqslant 1/n\) and \(T-2t+x\geqslant 1/n\), then$$\begin{aligned} U_n(t,x)= & {} 2(1+1/\sqrt{n}\,)\exp (-2/\sqrt{n}\,)-\exp (-2\sqrt{t-x}\,)-1, \end{aligned}$$
- (d)if \(t-x\geqslant 1/n\) and \(T-2t+x\leqslant 1/n\), then$$\begin{aligned} U_n(t,x)=&[1+\sqrt{n}(T-2t+x)+1/\sqrt{n}\,]\exp (-2/\sqrt{n}\,)\\&\, +\exp (-2\sqrt{T-2t+x}\,)-\exp (-2\sqrt{t-x}\,)-1, \end{aligned}$$
- (e)
if \(1/n\leqslant x-t\), then \(U_n(t,x)=\exp (-2\sqrt{x-t}\,)-1\). For \(x< 2t-T\) we have \(U_n(t,x)=1\).

## Proposition 5.1

Let Hamiltonians \(H_n\) be given by (5.1), the terminal condition *g* by (4.1), and the function *V* by (4.3). Then functions \(U_n\) given by the above formula are bounded lower semicontinuous solutions of equations HJB with Hamiltonians \(H_n\) and the terminal condition *g*, moreover \(U_n\rightarrow V\).

## Proof

It is not difficult to notice that functions \(U_n\) are bounded ( i.e. \(-1\leqslant U_n(\,\cdot \,,\,\cdot \,)\leqslant 1\) ) and \(U_n(T,x)=g(x)\). We can prove that functions \(U_n\) are lower semicontinuous on \([0,T]\times {\mathbb {R}}\). Furthermore, functions \(U_n\) are differentiable on \(A=\{(t,x)\in \,]0,T[\times {\mathbb {R}}: x> 2t-T\}\) and \(B=\{(t,x)\in \,]0,T[\times {\mathbb {R}}: x< 2t-T\}\), moreover \(\partial U_n(t,x)=\emptyset \) for \(x=2t-T\) and \(t\in [0,T]\), in addition \(U_n\) satisfy conditions (1.4) with Hamiltonians \(H_n\) and \(U_n\rightarrow V\). \(\square \)

## 6 Regularity of Hamiltonian

Let \({\mathcal {C}}\) be a family of continuous functions \(\varphi :[0,T]\times {\mathbb {R}}\rightarrow [0,+\infty [\) that satisfy the condition \(\varphi (t,x)=0\,\Leftrightarrow \, t=x\). Then it is easy to prove that the Hamiltonian *H* given by (3.2) with \(\varphi \in {\mathcal {C}}\) is well-defined and satisfies (H1)–(H3). Next, by \({\mathcal {L}}\) we denote a subfamily of \({\mathcal {C}}\) that contains locally Lipschitz functions. Notice that \(\varphi \) given by the formula (3.1) belongs to \({\mathcal {C}}\), but does not belong to \({\mathcal {L}}\). The example of a function, that is contained in \({\mathcal {L}},\) is \(\varphi (t,x)=|t-x|\).

In this section we prove that the Lagrangian *L* given by (3.4) with \(\varphi \in {\mathcal {L}}\) satisfies the Loewen–Rockafellar condition (2.2). However, it can be shown easily that its Hamiltonian does not satisfy the Lipschitz-type condition (SLC). It means that we cannot reverse the first implication in (2.3). Moreover, by the result of Galbraith [9] it follows that HJB equation with *H* given by (3.2) with \(\varphi \in {\mathcal {L}}\) has the unique solution.

So, it could be said that the nonuniqueness of HJB is due to a particular choice of \(\varphi \in {\mathcal {C}}\) and that choosing a different finction \(\varphi \in {\mathcal {L}}\subset {\mathcal {C}}\) one can get the unique solution of HJB.

Let the function \(\varphi \) belongs to \({\mathcal {C}}\) and \(w(\,\cdot \,,r)\) be modulus of continuity of the \(\varphi \) on the set \([0,T]\times r\mathbb {B}\). Then the following proposition holds.

## Proposition 6.1

*L*be given by (3.4) with the function \(\varphi \) belongs to \({\mathcal {C}}\). Moreover, let \(w(\,\cdot \,,r)\) be modulus of continuity of \(\varphi \) . Then for every \(t,s\in [0,T]\) and \(x,y\in r\mathbb {B}\), every \(v\in {\mathrm {dom}}\,L(t,x,\cdot )\) there exists \(\nu \in {\mathrm {dom}}\,L(s,y,\cdot )\) such that

- (i)
\(|\nu -v|\leqslant 2(1+|v|+|L(t,x,v)|)\,w(|s-t|+|y-x|,r)\);

- (ii)
\(L(s,y,\nu )\leqslant L(t,x,v)+2(1+|v|+|L(t,x,v)|)\, w(|s-t|+|y-x|,r)\).

## Proof

To prove the proposition we consider 3 cases.

*Case 1* Let \(t\not =x\) and \(\varphi (s,y)/\varphi (t,x)\leqslant 1\). Then for \(v\in {\mathrm {dom}}\,L(t,x,\cdot )\) we put \(\nu =v\,\varphi (s,y)/\varphi (t,x)\). We notice that \(\nu \in {\mathrm {dom}}\,L(s,y,\cdot )\). Moreover \({LS(ii)}\leqslant L(t,x,v)\leqslant {RS(ii)}\) and \({LS(i)}=2L(t,x,v)|\varphi (s,y)-\varphi (t,x)|\leqslant {RS(i)}\).

*Case 2* Let \(t\not =x\) and \(\varphi (s,y)/\varphi (t,x)> 1\). Then for \(v\in {\mathrm {dom}}\,L(t,x,\cdot )\) we put \(\nu =v\). We notice that \(\nu \in {\mathrm {dom}}\,L(s,y,\cdot )\). Moreover \({LS(i)}=0\leqslant {RS(i)}\) and \({LS(ii)}\leqslant L(t,x,v)\leqslant {RS(ii)}\).

*Case 3* Let \(t=x\). If \(v\in {\mathrm {dom}}\,L(t,x,\cdot )\), then \(v=0\). Put \(\nu =0\). We notice that \(\nu \in {\mathrm {dom}}\,L(s,y,\cdot )\). Moreover \({LS(i)}=0\leqslant {RS(i)}\) and \({LS(ii)}=0\leqslant {RS(ii)}\).

Therefore, the proposition is proven. \(\square \)

## Proposition 6.2

- (A)For every \(r>0\) there exists \(k>0\) such that for every \(t,s\in [0,T]\) and \(x,y\in r\mathbb {B}\), every \(v\in {\mathrm {dom}}\,L(t,x,\cdot )\) there exists \(\nu \in {\mathrm {dom}}\,L(s,y,\cdot )\) such that
- (i)
\(|\nu -v|\leqslant k(1+|v|+|L(t,x,v)|)(|s-t|+|y-x|)\);

- (ii)
\(L(s,y,\nu )\leqslant L(t,x,v)+k(1+|v|+|L(t,x,v)|)(|s-t|+|y-x|)\).

- (i)

## Proof

*L*in the following way: \(L(t,x,v):=L(0,x,v)\) for \(t<0\) and \(L(t,x,v):=L(T,x,v)\) for \(t>T\). Fix \(r>0\) and choose \(k>0\) for \(1+r\) in such a way that the condition (A) holds. Let \(t\in [0,T]\), \(x\in r\mathbb {B}\), \(v\in {\mathbb {R}}^n\) and \((w_1,w_2,p)\in \partial L(t,x,v)\). Without loss of generality we can assume that \((w_1,w_2)\not =0\). Let \((t_n,x_n):=(t,x)+(w_1,w_2)/[n|(w_1,w_2)|]\), then \(x_n\in (1+r)\mathbb {B}\,\). Since \(v\in {\mathrm {dom}}\,L(t,x,\cdot )\), then there exist \(v_n\in {\mathrm {dom}}\,L(t_n,x_n,\cdot )\) such that

- (i)
\(|v_n-v|\leqslant 2k(1+|v|+|L(t,x,v)|)\,|(t_n,x_n)-(t,x)|\),

- (ii)
\(L(t_n,x_n,v_n)\leqslant L(t,x,v)+2k(1+|v|+|L(t,x,v)|)\, |(t_n,x_n)-(t,x)|\).

## References

- 1.Aubin, J.-P., Frankowska, H.: Set-Valued Analysis. Birkhäuser, 1990 (Modern Birkhäuser Classics, reprint 2008)Google Scholar
- 2.Bardi, M., Capuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser, Boston (1997)CrossRefMATHGoogle Scholar
- 3.Barron, E.N., Jensen, R.: Semicontinuous viscosity solutions for Hamilton-Jacobi equations with convex Hamiltonians. Commun. Partial Differ. Equ.
**15**(12), 1713–1742 (1990)MathSciNetCrossRefMATHGoogle Scholar - 4.Crandall, M.G., Lions, P.-L.: Viscosity solutions of Hamilton-Jacobi equations. Trans. Am. Math. Soc.
**277**, 1–42 (1983)MathSciNetCrossRefMATHGoogle Scholar - 5.Crandall, M.G., Lions, P.-L.: Remarks on the existence and uniqueness of unbounded viscosity solutions of Hamilton-Jacobi equations. Ill. J. Math.
**31**, 665–688 (1987)MathSciNetMATHGoogle Scholar - 6.Dal Maso, G., Frankowska, H.: Value functions for Bolza problems with discontinuous Lagrangians and Hamilton-Jacobi inequalities. ESAIM Control Optim. Calc. Var.
**5**, 369–393 (2000)MathSciNetCrossRefMATHGoogle Scholar - 7.Frankowska, H.: Lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim.
**31**, 257–272 (1993)MathSciNetCrossRefMATHGoogle Scholar - 8.Frankowska, H., Sedrakyan, H.: Stable representation of convex Hamiltonians. Nonlinear Anal.
**100**, 30–42 (2014)MathSciNetCrossRefMATHGoogle Scholar - 9.Galbraith, G.H.: Extended Hamilton-Jacobi characterization of value functions in optimal control. SIAM J. Control Optim.
**39**(1), 281–305 (2000)MathSciNetCrossRefMATHGoogle Scholar - 10.Galbraith, G.H.: Cosmically Lipschitz set-valued mappings. Set-Valued Anal.
**10**, 331–360 (2002)MathSciNetCrossRefMATHGoogle Scholar - 11.Ishii, H.: Uniqueness of unbounded solutions of Hamilton-Jacobi equations. Indiana Univ. Math. J.
**33**, 721–748 (1984)MathSciNetCrossRefMATHGoogle Scholar - 12.Loewen, P.D., Rockafellar, R.T.: Optimal control of unbounded differential inclusions. SIAM J. Control Optim.
**32**(2), 442–470 (1994)MathSciNetCrossRefMATHGoogle Scholar - 13.Loewen, P.D., Rockafellar, R.T.: New necessary conditions for the generalized problem of Bolza. SIAM J. Control Optim.
**34**(5), 1496–1511 (1996)MathSciNetCrossRefMATHGoogle Scholar - 14.Loewen, P.D., Rockafellar, R.T.: Bolza problems with general time constraints. SIAM J. Control Optim.
**35**(6), 2050–2069 (1997)MathSciNetCrossRefMATHGoogle Scholar - 15.Misztela, A.: Optimal control problems with upper semicontinuous Hamiltonians. Discuss. Math. Differ. Incl. Control Optim.
**30**, 71–99 (2010)MathSciNetCrossRefMATHGoogle Scholar - 16.Misztela, A.: The value function representing Hamilton-Jacobi equation with Hamiltonian depending on value of solution. ESAIM Control Optim. Calc. Var.
**20**(3), 771–802 (2014)MathSciNetCrossRefMATHGoogle Scholar - 17.Plaskacz, S., Quincampoix, M.: On representation formulas for Hamilton Jacobi’s equations related to calculus of variations problems. Topol. Methods Nonlinear Anal.
**20**, 85–118 (2002)MathSciNetCrossRefMATHGoogle Scholar - 18.Rampazzo, F.: Faithful representations for convex Hamilton-Jacobi equations. SIAM J. Control Optim.
**44**(3), 867–884 (2005)MathSciNetCrossRefMATHGoogle Scholar - 19.Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (1998)CrossRefMATHGoogle Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.