Abstract
We consider an optimal control problem involving a nonlinear ODE with control, an integral cost functional, and a control constraint. Our main assumptions include a coercivity condition and the condition that the optimal control is an isolated solution of the variational inequality appearing in the first-order optimality condition. We show that the optimal open-loop control is Lipschitz continuous in time; moreover, we identify the dependence of the Lipschitz constant of the optimal control on the data of the problem. Then, we establish the existence of a Lipschitz continuous optimal feedback control. As an application, we study regularity properties of the optimal value function. A main tool for obtaining these results is the property of uniform strong metric regularity.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
In this paper, we consider an optimal control problem for a time-dependent nonlinear control system over a fixed time interval [0, T] with an integral cost functional. The set of feasible controls consists of all functions in L∞ (the space of measurable and essentially bounded functions over [0, T]) with values in a given convex and closed set in \(\mathbb {R}^{m}\). We assume twice differentiability with respect to the state and the control of the functions involved in the problem and local Lipschitz continuity of these functions together with all their derivatives with respect to all arguments. We also assume the existence of a reference optimal solution. Since the reference optimal control is a function in L∞, its values can be changed in a subset of [0, T] with Lebesgue measure zero without violating the optimality. In fact, the optimal control is a class of functions that differ from each other on a set of measure zero.
Our first task is to prove that, under an integral coercivity condition at the reference solution, we can select from the class of optimal controls a function which satisfies the first-order optimality condition for all t ∈ [0, T], instead of for almost every (a.e.) t ∈ [0, T]. Then, we show that under a coercivity condition this representative of the optimal controls is Lipschitz continuous with respect to time t ∈ [0, T] provided that it is an isolated solution of the Hamiltonian variational inequality in the first-order optimality condition. Moreover, we establish that the Lipschitz constant of the optimal control depends only on two constants: the coercivity constant and the Lipschitz constant of all functions defining the problem and their first and second derivatives over a bounded set in the space of variables (time, state, control).
The integral coercivity condition is a rather standard assumption in optimal control; the specific condition we use here goes back to the work of Hager [7]. In contrast, the isolatedness condition was introduced only recently in [2, Definition 3.6] in the context of the so-called differential variational inequalities, with the aim to prevent different solution curves from crossing each other. The isolatedness assumption is automatically satisfied when the Hamiltonian has a unique minimizer for each t ∈ [0, T], e.g., when the Hamiltonian is strictly convex. In [2, Theorem 4.1], it was established that if an optimal control \(\bar u\) is an isolated solution of the Hamiltonian variational inequality and for each t ∈ [0, T] the mapping defining this variational inequality is strongly metrically regular at \(\bar u\) for 0; then, the optimal control \(\bar u\) is Lipschitz continuous on [0, T]. We also mention the earlier work [4] in that direction for an optimal control problem with linear dynamics and a strongly convex cost for which strong regularity holds automatically; in fact, only continuity of the optimal control is claimed there but the Lipschitz continuity can be gleaned from the proof. We note that the coercivity condition implies strong metric regularity in the respective function spaces, see [2, Theorem 4.2].
Our next task is to prove the existence of a Lipschitz continuous optimal feedback control. We show that under the coercivity and isolatedness conditions for the optimal control, there exists an optimal feedback control (τ, ξ)↦u∗(τ, ξ) which is a Lipschitz continuous function; here (τ, ξ) represents the parametrizing pair initial time–initial condition.
Our third and last task is to show that the existence of a Lipschitz continuous optimal feedback control implies that the optimal value function (τ, ξ)↦V (τ, ξ) is differentiable with respect to ξ and its derivative is Lipschitz continuous.
An outline of the paper follows. In Section 2, we introduce the optimal control problem considered and set the stage for the further developments. Section 3 contains preliminary material showing in particular that the optimal control can be redefined on a set of measure zero so that the first-order optimality system holds for all t ∈ [0, T]. Section 4 gives conditions for Lipschitz continuity in time of the optimal open-loop control while Section 5 is devoted to the existence of a Lipschitz continuous optimal feedback control. The last Section 6 applies the latter result to show Lipschitz differentiability of the value function.
2 The Optimal Control Problem
We consider the following optimal control problem:
subject to
where the state \(x(t) \in \mathbb {R}^{n}\), the set U of feasible control values is a closed and convex subset of \(\mathbb {R}^{m}\), and the functions \(g :\mathbb {R}^{n} \to \mathbb {R}\), \(h : [0,T] \times \mathbb {R}^{n} \times \mathbb {R}^{m} \to \mathbb {R}\) and \(f : [0,T] \times \mathbb {R}^{n} \times \mathbb {R}^{m} \to \mathbb {R}^{n}\). The final time T and the initial state x0 are fixed.
Throughout we assume that the function g is twice differentiable and its second derivative is locally Lipschitz continuous, the functions h(t,⋅,⋅) and f(t,⋅,⋅) are two times continuously differentiable (with respect to (x, u)), and these functions, together with all their derivatives, are locally Lipschitz continuous (with respect to (t, x, u)).
We also assume that problem (1)–(2) has a locally optimal solution \((\bar x, \bar u)\). The local optimality is understood in the following way: there exists a number e0 > 0 such that for every \(u \in \mathcal {U}\) with \(\|u - \bar u\|_{\infty } \leq e_{0}\) either there is no solution of (2) over [0, T] or such a solution exists and \(J(u) \geq J(\bar u)\).
In this paper, we employ the standard function spaces L∞, L2, W1, ∞, W1,2, all over [0, T]. Specifically, the space of controls u is L∞, the space of measurable and essentially bounded functions. The state trajectory x is in W1, ∞, the space of Lipschitz continuous functions. For the controls we also use the space L2 of measurable square integrable functions, and for the state trajectory x the space W1,2 such that both x and its derivative \(\dot x\) are in L2. Furthermore, for an element x of a metric space we denote by IBa(x) (respectively \({\overset {\circ }{I\!\!B}}_{a}(y)\)) the closed (respectively open) ball centered at x with radius a.
Clearly, any feasible control u is actually a class of functions which differ from each other on a set of Lebesgue measure zero. We call any particular function from this class a representative and denote it in the same way, by u.
Introducing the Hamiltonian H(t, x, u, λ) = h(t, x, u) + λ⊤f(t, x, u), where ⊤ means transposition, we employ the standard first-order necessary optimality condition (a consequence of the Pontryagin maximum principle) in the form used, e.g., in [7], according to which there exists a Lipschitz continuous function \(\bar \lambda : [0,T] \to \mathbb {R}^{n}\) such that the triple \((\bar x, \bar u,\bar \lambda )\) satisfies for a.e. t ∈ [0, T] the following optimality system:
where Hx denotes the derivative of H with respect to x, etc., and NU is the normal cone mapping to the set U defined as
In further lines, we give the following long but important remark, which summarizes various observations that will be used later on.
Remark 1
It is a standard fact that under our assumptions there exist positive reals d0 and d such that for every \(\tilde u \in \mathcal {U}\) with \(\| \tilde u - \bar u\|_{\infty } \leq d\) and for every \(\xi \in {I\!\!B}_{d_{0}}(x_{0})\) there exists a unique solution \(\tilde x\) of the differential equation
which satisfies \(\|\tilde x - \bar x\|_{W^{1,\infty }} \leq 1\). Moreover, making d0 and d smaller if necessary, we obtain that the (unique) solution \(\tilde \lambda \) of the linear adjoint equation
satisfies \(\|\tilde \lambda - \bar \lambda \|_{W^{1,\infty }} \leq 1\). Without loss of generality, we assume that d ≤ 1 and d ≤ e0, where e0 appears in the definition of local optimality given in the beginning of this section.
Since \(\bar u \in L^{\infty }\), there exists a compact set \(\bar U\) such that \(\bar u(t) \in \bar U\) for a.e. t ∈ [0, T]. Define the set
Denote by L the Lipschitz constant on Ω of each of the functions f, g, h, fx, fu, hx, hu, fxx, fxu, fuu, hxx, hxu, huu, as well as of the functions H, Hx, Hu, Hxx, Hxu, Huu. Since f and Hx are bounded in Ω, then \(\dot {\tilde x}\) and \(\dot {\tilde \lambda }\) are also bounded. Make L larger if needed so that for every \(\tilde x\) and \(\tilde \lambda \) that satisfy (4) and (5), respectively, the functions \((t,v) \mapsto H_{u}(t, \tilde x(t), v, \tilde \lambda (t))\) and \((t,v) \mapsto H_{uu}(t, \tilde x(t), v, \tilde \lambda (t))\) are Lipschitz continuous with constant L in the set \(\{ (t,v) ~:~ t \in [0,T],~\text {dist}(v,\bar U) \leq 1\}\). This concludes Remark 1.
To shorten the notations, we skip arguments with “bar”, shifting the “bar” to the functions, e.g., \(\bar H(t) := H(t,\bar x(t),\bar u(t),\bar \lambda (t))\), \(\bar H(t,u) := H(t,\bar x(t),u,\bar \lambda (t))\), \(\bar f(t) := f(t,\bar x(t),\bar u(t))\), \(\bar g_{xx} := g_{xx}(\bar x(T))\), etc. Define the matrices
Our first main assumption is the following:
-
Coercivity: there exists a constant ρ > 0 such that
$$ \begin{array}{@{}rcl@{}} &&y(T)^{\top} \bar g_{xx} y(T) + {{\int}_{0}^{T}} \left( y(t)^{\top} Q(t)y(t) + w(t)^{\top} R(t)w(t) + 2y(t)^{\top} S(t)w(t)\right) \mathrm{d} t \\ &&\qquad\geq \rho {{\int}_{0}^{T}} |w(t)|^{2} \mathrm{d} t \end{array} $$(6)for all w ∈ L2, y ∈ W1,2 such that w(t) ∈ U − U, \(\dot y(t) = A(t)y(t)+B(t)w(t)\) for a.e. t ∈ [0, T], and y(0) = 0.
The coercivity condition was first used in [7] to show convergence of the multiplier method and later in [5] to establish Lipschitz stability as well as convergence of discrete approximations in optimal control. It can be viewed as a strong second-order sufficient condition in optimal control. Checking this condition would very much depend on the specific problem at hand; sometimes it is enforced numerically by adding penalty terms to the cost. The coercivity condition has also been used for a posteriori numerical verification of optimality after an approximate solution is found.
In the following section, we present some preparatory material. In particular, we show that the coercivity condition implies a pointwise in time coercivity property which plays an important role in further analysis.
3 Preliminaries
Denote by meas(E) the Lebesgue measure of a set E. Let Δ ⊂ [0, T] be a measurable set with meas(Δ) > 0, and let \(v: {\Delta } \to \mathbb {R}^{m}\) be a measurable and bounded function. For t ∈Δ denote by VΔ(v;t) the set of points \(w \in \mathbb {R}^{m}\) with the following property: there is a sequence of measurable sets Ek ⊂Δ such that
We denote V[0, T](v;t) simply by V (v;t).
A point t ∈Δ is said to be essentially non-isolated if for every ε > 0 the set [t − ε, t + ε] ∩Δ is of positive measure.
Lemma 1
Let Δ ⊂ [0, T] be a measurableset and let\(v: {\Delta } \to \mathbb {R}^{m}\)bea measurable and bounded function. Then, for anyt ∈Δ, thefollowing statements are equivalent:
-
(i)
VΔ(v;t) ≠ ∅;
-
(ii)
t is essentially non-isolated point ofΔ.
Proof
If (i) holds, then the very definition of VΔ(v;t) implies that t is essentially non-isolated.
Let us pick an essentially non-isolated point t of Δ. Let \(K \subset \mathbb {R}^{m}\) be a compact set such that v(s) ∈ K for every s ∈Δ. Take an arbitrary w ∈ K. If for every ε > 0 and every natural number k there exists Ek ⊂ [t − 1/k, t + 1/k] ∩Δ such that meas(Ek) > 0 and \(\sup _{s \in E_{k}} |v(s) - w | < \varepsilon \), then w ∈ VΔ(v;t). If this is not the case, then there exist ε(w) > 0 and a natural number k(w) such that |v(s) − w|≥ ε(w) for a.e. s ∈ [t − 1/k(w), t + 1/k(w)] ∩Δ; that is, \(v(s) \not \in {\overset {\circ }{I\!\!B}}_{\varepsilon (w)}(w)\) for a.e. s ∈ [t − 1/k(w), t + 1/k(w)] ∩Δ. If w∉VΔ(v;t) for every w ∈ K, then, due to the compactness of K, there exist w1,…, wr ∈ K such that \(K \subset \cup _{i=1}^{r} {\overset {\circ }{I\!\!B}}_{\varepsilon (w_{i})}(w_{i})\). Denote \(\bar k := \max \{k(w_{1}), \dots , k(w_{r}) \}\); then, \(v(s) \not \in \cup _{i=1}^{r} {\overset {\circ }{I\!\!B}}_{\varepsilon (w_{i})}(w_{i})\) for a.e. \(s \in [t - 1/\bar k, t + 1/\bar k]\cap {\Delta }\). This contradicts the essential non-isolatedness of t, since \(K \subset \cup _{i=1}^{r} {\overset {\circ }{I\!\!B}}_{\varepsilon (w_{i})}(w_{i})\) and meas\(([t - 1/\bar k, t + 1/\bar k]\cap {\Delta }) > 0\). Hence VΔ(v;t) ≠ ∅ and the proof is complete. □
Taking Δ = [0, T], we obtain that V (v;t) is non-empty for every t ∈ [0, T].
Lemma 2
Letu and\(\tilde u\)betwo measurable and bounded functions acting from[0, T] to\(\mathbb {R}^{m}\), and letu(t) ∈ V (u;t) for everyt ∈ [0, T]. Then, the function\(\tilde u\)canbe redefined on a set of measure zero in such a waythat\(\tilde u(t) \in V(\tilde u;t)\)and\(|\tilde u(t) - u(t)| \leq \|\tilde u - u\|_{\infty }\)foreveryt ∈ [0, T].
Proof
Take an arbitrary t ∈ [0, T]. Consider first the case where both functions u and \(\tilde u\) are approximately continuous at t. We recall that u is approximately continuous at t ∈ (0, T) if there exists a measurable set E ⊂ [0, T] containing t such that
and the restriction of u to E is continuous. Let \(\tilde E\) be the set in the definition of approximate continuity of \(\tilde u\) at t ∈ (0, T). Then, the set \({E}_{k}^{\prime } := E \cap \tilde E \cap [t - 1/k, t + 1/k]\) satisfies \(\lim _{k \to \infty } 2k \text {meas}({E}_{k}^{\prime }) = 1\). In particular, meas\(({E}_{k}^{\prime }) >0\) for all sufficiently large k. Due to the continuity of u and \(\tilde u\) on \(E \cap \tilde E\), we have
Moreover, since the sets Ek in the definition of V can be replaced by \({E}_{k}^{\prime }\), we conclude that \(\tilde u(t) \in V(\tilde u;t)\).
Now, let t ∈ [0, T] be such that u or \(\tilde u\) is not approximately continuous at t, or t equals 0 or T.
We will now redefine \(\tilde u(t)\) to fit the claim. It is well known (see, e.g., [8, Theorem 7.54]) that almost all t ∈ [0, T] are points of approximate continuity of both u and \(\tilde u\); therefore we need to redefine \(\tilde u\) only on a set of measure zero. Note that the sets \(V(\tilde u;t)\) are invariant with respect to changes of \(\tilde u\) on a set of measure zero.
Denote w := u(t) ∈ V (u;t). Let Ek be the sets in the definition of V. In particular, \(\varepsilon _{k} := \sup _{s \in E_{k}} |u(s) - w| \stackrel {k}{\longrightarrow } 0\). Since Ek is of positive measure, it contains an essentially non-isolated point tk ∈ Ek. According to Lemma 1, there exists \(\tilde w_{k} \in V_{E_{k}}(\tilde u; t_{k})\); hence, there exists a sequence \(\{{{E}_{k}^{i}}\}_{i}\), \({{E}_{k}^{i}} \subset E_{k}\), such that
Let \(\tilde w\) be a cluster point of the sequence \(\{\tilde w_{k}\}\). To show that \(\tilde w \in V(\tilde u; t)\), we employ the following argument involving choosing a diagonal sequence. For an arbitrary natural number j, choose k = kj so large that
Then, choose i = ij such that
We have
and
Taking also into account that meas\((\tilde E_{j}) > 0\), the last two relations imply that \(\tilde w \in V(\tilde u; t)\).
For every k and i, we have \({{E}_{k}^{i}} \subset E_{k}\),
and
Hence,
Passing to the limit with i and then with k, we obtain \(|\tilde w - w| \leq \| \tilde u - u \|_{\infty }\). Then, we redefine \(\tilde u(t)\) as \(\tilde u(t) = \tilde w\). This completes the proof. □
Corollary 1
Every\(v \in \mathcal {U}\)canbe redefined on a set of measure zero in such a way thatv(t) ∈ V (v;t) for everyt ∈ [0, T].
For a proof, apply Lemma 2 with \(\tilde u = v\) and the constant function u(t) = u for all t ∈ [0, T].
Remark 2
From now on, the element \(\bar u \in L^{\infty }\) will be identified with a function (denoted again by \(\bar u\)) satisfying \(\bar u(t) \in V(\bar u;t)\) for every t ∈ [0, T].
Observe that the coercivity condition (6) does not depend on the particular representative of \(\bar u\).
Lemma 3
Let the coercivity condition (6) hold, where\(\bar u\)isidentified as in Remark2. Then,
Proof
Fix an arbitrary t ∈ [0, T]. Since \(\bar u(t) \in V(\bar u;t)\), there exists a sequence Ek ⊂ [0, T] such that
For an arbitrary w ∈ U − U, we define a function wk as
Using the Cauchy formula for the equation
we obtain that yk(s) = 0 for s ∈ [0, t − 1/k] and |yk(s)|≤ c1meas(Ek) for s ∈ (t − 1/k, T], where here and further c1, c2,… are positive reals independent of k. Then, for the terms involved in (6), we have
Since \(R(s) = \bar H_{uu}(s,\bar u(s))\), using (8), we obtain (see Remark 1) that for s ∈ Ek one has
Using the above estimated in (6) and the above five displayed formulas, we obtain
Dividing by meas(Ek) (here we use the first inequality in (8)) and passing to the limit with k, we obtain (7). □
4 Lipschitz Continuity of the Optimal Control
Let us recall the optimality system (3):
Lemma 4
Let the coercivity condition hold. Then, the optimalcontrol\(\bar u \in L^{\infty }\)hasa representative\(\bar u\)suchthat the matrix\(R(t) = \bar H_{uu}(t,\bar u(t))\)satisfies(7) and\((\bar x(t), \bar u(t),\bar \lambda (t))\)satisfies(9) for allt ∈ [0, T]. In fact, any representative of the optimal control thatsatisfies\(\bar u(t) \in V(\bar u; t)\)forallt ∈ [0, T] has this property.
Proof
Let us redefine \(\bar u\) so that \(\bar u(t) \in V(\bar u;t)\) for all t ∈ [0, T] (see Corollary 1 and Remark 2). Then, according to Lemma 3, the pointwise coercivity condition (7) holds for every t ∈ [0, T].
Fix an arbitrary t ∈ [0, T]. Since \(\bar u(t) \in V(\bar u;t)\), there exists a sequence {Ek} of measurable subsets of [0, T] such that (8) holds. Since meas(Ek) > 0 and (9) is satisfied by \((\bar x(t), \bar u(t),\bar \lambda (t))\) almost everywhere, there exists tk ∈ Ek such that (9) holds for tk. From (8), we obtain that tk → t and \(\bar u(t_{k}) \to \bar u(t)\). Then, due to the continuity of the function \((t,u) \mapsto H_{u}(t,\bar x(t),u,\bar \lambda (t))\) and the upper semi-continuity of the mapping u↦NU(u), (9) holds for t as well. □
We recall next the property of strong metric regularity of a general set-valued mapping \(\mathcal {F} : \mathcal {Y} \rightrightarrows \mathcal {Z}\), where \(\mathcal {Y}\) and \(\mathcal {Z}\) are Banach spaces (for more on that, see, e.g., [6, Section 3.7]). A mapping \(\mathcal {F}\) is said to be strongly metrically regular at \(\hat y\) for \(\hat z\) if there exist constants κ ≥ 0, a > 0 and b > 0 such that the truncated inverse mapping
is single-valued (a function) and Lipschitz continuous on \({I\!\!B}_{b}(\hat z)\). Here \(\mathcal {F}^{-1}(z) = \{y \mid z \in \mathcal {F}(y)\}\).
Our further analysis is based on the following version of Robinson’s implicit function theorem. It was first stated as [6, Theorem 5G.3]Footnote 1 and then in corrected form as Theorem 3.2 in [2] (see also [3, Theorem 2.3] for a slight extension):
Theorem 1
Leta,b, andκbe positive scalars and let amapping\(\mathcal {F}:\mathcal {Y} \rightrightarrows \mathcal {Z}\)be strongly metrically regularat\(\hat y\)for\(\hat z\)withneighborhoods\({I\!\!B}_{a}({\hat y})\)and\({I\!\!B}_{b}({\hat z})\)andconstantκ. Letμ > 0 be suchthatκμ < 1 and letκ′ > κ/(1 − κμ). Then, forevery positiveαandβsuchthat
and for every function\(g:\mathcal {Y} \to \mathcal {Z}\)satisfying
the mapping\(z \mapsto (g+\mathcal {F})^{-1}(z)\cap {I\!\!B}_{\alpha }(\hat y)\)is a Lipschitz continuous function on\({I\!\!B}_{\beta }(\hat z)\)with Lipschitz constantκ′.
Compared with the standard Robinson’s implicit function theorem, see [6, Theorem 2B.1], Theorem 1 exhibits the fact that everything hinges on the constants involved; that is, the constants of metric regularity of the perturbed mapping \(g+\mathcal {F}\) do not depend on the actual perturbations but only on \(\|g({\hat y})\|\), the Lipschitz constant of g and the constants of the strong regularity of \(\mathcal {F}\). In that sense, Theorem 1 shows strong metric regularity which is uniform with respect to perturbations.
Let us get back to the optimal control problem at hand. If \((t,u) \in \text {cl} \text {gph}(\bar u),\) then there exists a sequence tk → t such that \(\bar u(t_{k}) \to u\). According to (7), we have
Passing to the limit, we obtain that
for every \( (t,u) \in \text {cl} \text {gph}(\bar u)\) and every w ∈ U − U. It is well known that the property (10) implies that for every \((t,u) \in \text {cl} \text {gph}(\bar u)\) the mapping
is strongly metrically regular at u for 0 with constants κ′ = 1/ρ, a′ = b′ = +∞ (that is, with any positive a′ and b′), see, e.g., [7, Lemma 1]. Note that these constants are independent of t.
Next, we reformulate, adapted to our notations and needs, a simplified version of Theorem 3.5 in [2], which in turn is a corollary of Theorem 1.
Theorem 2
Assume that for every\((t,u) \in \text {cl} \text {gph}(\bar u)\)themapping in (11) is strongly metrically regular atu for0 with constantsκ′, a′, b′that are independent of (t, u). Then, for everyt ∈ [0, T], the mapping\(u \mapsto \bar H_{u}(t,u) + N_{U}(u)\)isstrongly metrically regular at\(\bar u(t)\)for0 with any constantsκ, a, b satisfying the inequalities
whereL′is a Lipschitz constant of the mapping\(u \mapsto \bar H_{uu} (t,u)\) on \({I\!\!B}_{a^{\prime }}(\bar u(t))\), for every t ∈ [0, T].
The conditions (12) are not stated in Theorem 3.5 in [2], but are explicitly written in the beginning of its proof there.
Continuing the analysis of (11), we apply Theorem 2 with a′ = 1, b′ = +∞ and κ′ = 1/ρ, having the inequalities (12) reduced to
where now L is the constant from Remark 1.
Remark 3
The important consequence of (13) is that the constants κ, a, b of strong regularity of \(u \mapsto \bar H_{u}(t,u) + N_{U}(u)\) at \(\bar u(t)\) for 0 can be chosen to depend only on the constant ρ in the coercivity condition (6) and the constant L in Remark 1.
We introduce next our second main assumption:
-
Isolatedness: The function \(\bar u\) (represented as in Lemma 4) is an isolated solution of the inclusion \(\bar H_{u}(t,u) + N_{U}(u) \ni 0\) for all t ∈ [0, T], meaning that there exists a (relatively) open set \(\mathcal {O} \subset [0,T] \times \mathbb {R}^{m}\) such that
$$ \{ (t,u) \in [0,T] \times \mathbb{R}^{m}~:~ \bar H_{u}(t,u) + N_{U}(u) \ni 0 \} \cap \mathcal{O} = \text{gph}(\bar u). $$(14)
For example, the isolatedness assumption holds if for every t ∈ [0, T] the inclusion \(\bar H_{u}(t,u) + N_{U}(u) \ni 0\) has a unique solution (which has to be \(\bar u(t)\)). In this case, one can verify the isolatedness condition taking any (relatively) open set \(\mathcal {O} \subset [0,T] \times \mathbb {R}^{m}\) containing \(\text {gph}(\bar u)\).
Theorem 3
Suppose that the isolatedness assumption (14) and condition (7) hold. Then, the optimalcontrol\(\bar u\)isLipschitz continuous on [0, T]. Moreover, the Lipschitz constant of\(\bar u\)dependsonly on the numberρin (7) and the constant L in Remark1.
Proof
The proof is somewhat parallel to the proof of Theorem 3.7 in [2]. Here we use Theorem 2 and (13) instead of the more general Theorem 3.5 in [2] (used in the proof of Theorem 3.7 in [2]), which does not imply the second claim of Theorem 3.
As mentioned around (10), condition (7) implies that for every \((t,u) \in \text {cl} \text {gph}(\bar u)\) the mapping in (11) is strongly metrically regular at u for 0. Then, we can apply Theorem 2. Let the numbers a, b, κ be chosen to satisfy conditions (13), so that for every t ∈ [0, T] the mapping \(u \mapsto \bar H_{u}(t,u) + N_{U}(u)\) is strongly metrically regular at \(\bar u(t)\) for 0 (see Theorem 2). Let L be the constant in Remark 1; then, the mappings \((t,u) \mapsto \bar H_{u}(t,u)\) and \((t,u) \mapsto \bar H_{uu}(t,u)\) are Lipschitz continuous with constant L on the set \(\{ (t,u) ~:~ t \in [0,T],~u \in {I\!\!B}_{a}(\bar u(t))\}\). Without loss of generality we consider \(\bar u\) as taking values in the set \(\bar U\) in Remark 1; we also recall that a ≤ 1.
Take an arbitrary t ∈ [0, T]. Then, pick αt < a/2 and then γt ∈ (0,1) such that \((\tau ,v) \in \mathcal {O}\) for every τ ∈ [t − γt, t + γt] ∩ [0, T] and \(v \in {I\!\!B}_{\alpha _{t}}(\bar u(t))\), and also
For an arbitrary τ ∈ [t − γt, t + γt] ∩ [0, T] define the mapping \(g_{\tau ,t} : U \to \mathbb {R}^{m}\) as
Then, we have that
and, for any \(u,~u^{\prime } \in {I\!\!B}_{a}(\bar u(t))\),
Set
According to the inequality αt < a/2, the inequalities in (15), and the definitions of \(\kappa ^{\prime }_{t}\), αt, βt, the following inequalities are fulfilled (for convenience we skip the subscripts t for a moment):
Now, we apply Theorem 1. For short, denote \(G_{t}(u): = \bar H_{u}(t,u) + N_{U}(u)\). In our context all assumptions of the last theorem are satisfied with g = gτ, t. Thus we obtain that the mapping
is Lipschitz continuous with Lipschitz constant \(\kappa ^{\prime }_{t} = 2 \kappa /(1 - \kappa L \gamma _{t}) \leq 4\kappa \) (see the first inequality in (15)). In particular, there exists a unique \(v \in {I\!\!B}_{\alpha _{t}}(\bar u(t))\) such that 0 ∈ Gτ(v). Since τ ∈ [t − γt, t + γt] ∩ [0, T] and \(v \in {I\!\!B}_{\alpha _{t}}(\bar u(t))\), we also have that \((\tau ,v) \in \mathcal {O}\). Due to isolatedness condition, we obtain that \(v = \bar u(\tau )\). From (16), we obtain that \(g_{\tau ,t}(\bar u(t)) \in {I\!\!B}_{\beta _{t}}(0)\). Thus
Since \(\bar u(\tau ) = (g_{\tau ,t} + G_{t})^{-1} (0) \cap {I\!\!B}_{\alpha }(\bar u(t))\), using (16), we get that
Summarizing, we obtain that for every t ∈ [0, T] there exists a neighborhood (t − γt, t + γt) ∩ [0, T] in [0, T] in which \(\bar u\) is Lipschitz continuous with the same constant 4κL. This implies that \(\bar u\) is Lipschitz continuous with the same constant in the whole interval [0, T].
The second claim of the theorem follows from Remark 3 concerning κ. □
The example displayed in Remark 9 in [5] demonstrates that the isolatedness assumption (14) is essential for the Lipschitz continuity of the optimal control shown in Theorem 3. In this example h = (u2 − 1)2, g = 0, f = 0, \(U = \mathbb {R}\), T = 1. Here, for each measurable set Ω ⊂ [0,1] the function defined as u(t) = − 1 for t ∈Ω and u(t) = 1 for t ∈ [0,1] ∖Ω is an optimal control, and the coercivity condition is satisfied. However, the isolatedness condition is satisfied only if the measure of Ω is either zero or 1. In these two cases the optimal control is Lipschitz continuous.
5 Lipschitz Continuous Optimal Feedback Control
In this section, we prove the existence of a Lipschitz continuous locally optimal feedback control for problem (1)–(2). For this purpose we embed the problem into a family of problems by replacing the initial time 0 with any τ ∈ [0, T] and the initial condition x(0) = x0 with \(x(\tau ) = \xi \in \mathbb {R}^{n}\). Denote this new family of problems by P(τ, ξ), so that P(0, x0) is (1)–(2). Also, denote by J(τ, ξ;u) the value of the objective function of P(τ, ξ) for a control \(u \in \mathcal {U}\) being defined as
where x is the solution of the initial-value problem
To set the stage, we give first the following definition which recasts the usual way a locally optimal feedback control is understood. Recall that \((\bar x, \bar u)\) is a locally unique solution of problem (1)–(2).
Definition 1
The function \(u^{\ast }: [0,T] \times \mathbb {R}^{n} \to U\) is said to be a locally optimal feedback control around the reference solution pair \((\bar x, \bar u)\) if there exist positive numbers ε0 and \(\bar a\), and a set \({\Gamma } \subset [0,T] \times \mathbb {R}^{n}\) such that
-
(i)
\(\text {gph}(\bar x) + \{0\}\times {I\!\!B}_{\varepsilon _{0}}(0) \subset {\Gamma }\);
-
(ii)
for every (τ, ξ) ∈Γ the equation
$$ \dot x(t) = f(t,x(t),u^{\ast}(t,x(t))), \quad x(\tau) = \xi, $$(19)has a unique absolutely continuous solution \(\hat x[\tau ,\xi ]\) on [τ, T] which satisfies \(\text {gph}(\hat x[\tau ,\xi ]) \subset {\Gamma }\);
-
(iii)
the function \(\hat u[\tau ,\xi ](\cdot ) := u^{\ast }(\cdot ,\hat x[\tau ,\xi ](\cdot ))\) is measurable, bounded, and satisfies \(\|\hat u[\tau ,\xi ] - \bar u\|_{\infty } \leq \bar a\), and \(J(\tau ,\xi ;\hat u[\tau ,\xi ]) \leq J(\tau ,\xi ;u)\), where u is any admissible control on [τ, T] with \(\| u - \bar u\|_{\infty } \leq \bar a\) for which the corresponding solution x of (19) exists on [τ, T] and satisfies gph(x) ⊂Γ;
-
(iv)
\(u^{\ast }(\cdot ,\bar x(\cdot )) = \bar u(\cdot )\).
The main result of this section follows.
Theorem 4
Let the coercivity condition (6) and the isolatedness condition(14) hold. Then, there exists a locally optimal feedbackcontrol\(u^{\ast }: [0,T] \times \mathbb {R}^{n} \to U\)around\((\bar x, \bar u)\)whichis Lipschitz continuous on the set Γ appearing (together with the positive numbersε0and\(\bar a\))in Definition1.
Let us first sketch the idea of the proof. First, we prove that for ξ close to \(\bar x(\tau )\) a unique solution \((\bar x[\tau ,\xi ],\bar u[\tau ,\xi ])\) exists and it is close to the restriction of \((\bar x, \bar u)\) to [τ, T]; moreover, \(\bar u[\tau ,\xi ]\) depends in a Lipschitz way on ξ (in the space L∞). Then, we show that \(\bar u[\tau ,\xi ]\) is Lipschitz continuous.
For any τ ∈ [0, T), we define the spaces
where the time interval for these functional spaces is [τ, T]. It is convenient to define the norm in Yτ as ∥(x, u, λ)∥ := max{∥x∥1, ∞, ∥u∥∞, ∥λ∥1, ∞}. For any fixed τ ∈ [0, T), any (locally) optimal solution-multiplier triple y := (x, u, λ) ∈ Yτ for P(\(\tau ,\bar x(\tau )\)) satisfies the inclusion
where Fτ : Yτ → Zτ and \(G_{\tau }: Y_{\tau } \rightrightarrows Z_{\tau }\) are defined as
Here
By using the superscript ∞ in the notation of the latter set we emphasize that the cone \(N_{\mathcal {U}}^{\infty } (u)\) includes only a part of the normal cone \(N_{\mathcal {U}}(u)\) which is a subset of the dual space of L∞; note that the dependence on τ is not indicated.
Proposition 1
Let the coercivity condition (6) hold. Then, the mappingFτ + Gτis strongly metrically regular at the restrictionof\(\bar y:= (\bar x,\bar u,\bar \lambda )\)to[τ, T] (denoted in the same way) for 0. Moreover, the constants of strong regularity, callthem\(\bar \kappa \), \(\bar a\), \(\bar b\), can be chosen independent ofτ.
Proof
The strong metric regularity of the mapping Fτ follows from [5, Theorem 5], with the only difference that in [5] there is no terminal term in the cost functional and the functions h and f do not depend on time t. As is well known, under the smoothness conditions imposed the problem with a terminal cost can be transformed into an equivalent problem without a terminal cost. In addition, the time-dependent problem is handled in exactly the same way as the time-invariant; thus, the difference is basically formal. For reader’s convenience, below we outline the proof by highlighting the main steps and utilizing Theorem 1 as a shortcut.
First, observe that the coercivity condition (6) is fulfilled for problem P(\(\tau ,\bar x(\tau )\)) with the same constant ρ for all τ. To show this, it is enough to take w(t) = 0 on [0, τ) in (6). The next step is to linearize the generalized (20) at \(\bar y = (\bar x, \bar u, \bar \lambda )\), obtaining
where
The strong regularity of the mapping appearing in the linearization (21), say with constants κ, a, b independent of τ is established in [7, Lemma 3] (with the caveat concerning the terminal cost and the dependence on t). Consider the function
Then, \(g_{\tau }(\bar y) = 0\). Since \(\mathcal {A}_{\tau }\) is the strict derivative (in L∞) of Fτ at \(\bar y\), the Lipschitz modulus of gτ at \(\bar y\) is zero. Thus, in the notation of Theorem 1, taking α sufficiently small one can make μ arbitrarily close to zero; furthermore, κ′ and β could be chosen accordingly to satisfy (17). It remains to put \(\bar k = k^{\prime }\), \(\bar a = \alpha \), \(\bar b = \beta \) and to observe that these constants are independent of τ. □
As a consequence of the last proposition, for any \(\xi \in {I\!\!B}_{\bar b}(\bar x(\tau ))\) the inclusion Fτ(y) + Gτ(y) + ζ ∋ 0 with ζ = (0,−ξ,0,0,0) has a unique solution \((\bar x[\tau ,\xi ],\bar u[\tau ,\xi ], \bar \lambda [\tau ,\xi ])\) in \({I\!\!B}_{\bar a}((\bar x,\bar u,\bar \lambda ))\) and it is Lipschitz continuous with respect to \(\xi \in {I\!\!B}_{\bar b}(\bar x(\tau ))\) with Lipschitz constant \(\bar \kappa \) in the norms of \(\mathbb {R}^{n}\) and Yτ.
Clearly, the constant \(\bar b\) can be decreased, if necessary, without affecting the strong regularity property. Then, we may assume that \(\bar b > 0\) is chosen so small that the coercivity assumption (6) adapted to problem P(τ, ξ) with \(\xi \in {I\!\!B}_{\bar b}(\bar x(\tau ))\) holds with a constant ρ/2 (instead of ρ). Here and further “adapted” means that the matrices A, B, Q, R, S are calculated along \((\bar x[\tau ,\xi ],\bar u[\tau ,\xi ],\bar \lambda [\tau ,\xi ])\) instead of \((\bar x,\bar u,\bar \lambda )\) and the integration in (6) is on [τ, T]. Since under the coercivity assumption, the necessary optimality condition (3) is also sufficient (for local optimality), we obtain the following proposition.
Proposition 2
Let the coercivity condition and isolatedness condition hold. Then, forany\(\xi \in {I\!\!B}_{\bar b}(\bar x(\tau ))\), the pair\((\bar x[\tau ,\xi ],\bar u[\tau ,\xi ])\)definedin the second to last paragraph is the unique locally optimal solution of problemP(τ, ξ),in the set\({I\!\!B}_{\bar a}((\bar x,\bar u))\). Moreover, the function\({I\!\!B}_{\bar b}(\bar x(\tau )) \ni \xi \mapsto \bar u[\tau ,\xi ]\)isLipschitz continuous in the norm ofL∞with Lipschitz constant\(\bar \kappa \).
It is important to note that assuming \(\bar b\) small enough we may guarantee that Remark 1 is still valid with e0, d0 and d replaced with e0/2, d0/2 and d/2, respectively, and for the interval [τ, T] and the function \(\bar u[\tau ,\xi ]\), \(\xi \in {I\!\!B}_{\bar b}(\bar x(\tau ))\), instead of [0, T] and \(\bar u\). The constant L remains the same.
As already mentioned, the coercivity assumption, adapted to problem P(τ, ξ), would hold (with ρ/2 instead of ρ) provided that \(\xi \in {I\!\!B}_{\bar b}(\bar x(\tau ))\). According to Lemma 4, an arbitrary redefinition of \(\bar u[\tau ,\xi ]\) on a set of measure zero which satisfies \(\bar u[\tau ,\xi ](t) \in V_{[\tau ,T]}(\bar u[\tau ,\xi ];t)\) for every t ∈ [τ, T] (and such exists due to Corollary 1) fulfills the conditions that the matrix R(t) satisfies (7), and \(\bar y[\tau ,\xi ]\) satisfies (9) for every t ∈ [τ, T], all adapted to problem P(τ, ξ). Moreover, according to Lemma 2, \(\bar u[\tau ,\xi ]\) can be assumed to satisfy \(|\bar u[\tau ,\xi ](t) - \bar u(t)| \leq \|\bar u[\tau ,\xi ] - \bar u\|_{\infty }\) for every t ∈ [τ, T].
Lemma 5
Let the coercivity condition and the isolatedness condition hold. Then, thereexists a number\(\varepsilon \in (0, \bar b]\)suchthat for everyτ ∈ [0, T) and\(\xi \in {I\!\!B}_{\varepsilon }(\bar x(\tau ))\)thecontrol\(\bar u[\tau ,\xi ]\)satisfiesthe isolatedness condition (on [τ, T]),namely there exists a (relatively) open set\(\mathcal {O} \subset [\tau ,T] \times \mathbb {R}^{m}\)suchthat
Proof
Let us take ε > 0 so small that
where \(\bar L\) is the Lipschitz constant of \(\bar u\) (see Theorem 3). For arbitrarily fixed τ ∈ [0, T) and \(\xi \in {I\!\!B}_{\varepsilon }(\bar x(\tau ))\) denote (for short) \(\tilde x(t) = \bar x[\tau ,\xi ](t)\), \(\tilde u = \bar u[\tau ,\xi ]\) redefined as described before the statement of the lemma, \(\tilde \lambda (t) = \bar \lambda [\tau ,\xi ](t)\) and \(\tilde y = (\tilde x, \tilde u, \tilde \lambda )\). Also denote \(\tilde R(t) = H_{uu}(t,\tilde y(t))\), \(\tilde H_{u}(t,u) = H_{u}(t,\tilde x(t),u,\tilde \lambda (t))\). Then, due to the first inequality in (23) and the redefinition of \(\tilde u\), we know that for every t ∈ [τ, T]
Let us define
which is relatively open in \([\tau ,T] \times \mathbb {R}^{m}\). We shall prove that the claim of the lemma holds with this set \(\mathcal {O}\).
Note that the right side of (22) is contained in the left side; thus, it is sufficient to prove the opposite inclusion. Targeting a contradiction, let us assume that there exists a point
which is not in \(\text {gph}(\tilde u)\). Then, \(\tilde u(t_{0}) \not =u_{0}\). From (26) and the second relation in (24), we have
From here
which implies
Then, using (25) (notice that u0 ∈ U, since otherwise NU(u0) = ∅), we obtain
Hence,
Due to the inclusion \((t_{0},u_{0}) \in \mathcal {O}\), there exists \((t_{1},u_{1}) \in \text {gph}(\tilde u)\) such that |t1 − t0|≤ ε, |u1 − u0|≤ ε. Then, continuing the inequality (26), we obtain
This last inequality contradicts (23). Hence, (22) holds. □
Having proved that the isolatedness condition is also fulfilled for problem P(τ, ξ), we can apply Theorem 3 to this problem and obtain that the (locally) optimal control \(\bar u[\tau ,\xi ]\) is Lipschitz continuous. The Lipschitz constant, \(\bar L\), depends on the problem only through the constant ρ (now ρ/2) and the constant L, therefore can be chosen independent of τ and ξ, provided that \(|\xi - \bar x(\tau )| \leq \varepsilon \), where ε > 0 is sufficiently small (independent of τ).
Proof
of Theorem 4 Let the number \(\varepsilon \in (0, \bar b]\) be the one from Lemma 5. There exists a number ε0 ∈ (0, ε] such that for every τ ∈ [0, T] and \(\xi \in {I\!\!B}_{\varepsilon _{0}}(\bar x(\tau ))\) the corresponding \((\bar x[\tau ,\xi ], \bar u[\tau ,\xi ])\) exists on [τ, T] and satisfies, \(| \bar x[\tau ,\xi ](t) - \bar x(t) | \leq \varepsilon \) for every t ∈ [τ, T], and \(\|\bar u[\tau ,\xi ] - \bar u \|_{\infty } < \bar a\). Such an ε0 exists because the mapping \({I\!\!B}_{\bar b} \ni \xi \mapsto (\bar x[\tau ,\xi ],\bar u[\tau ,\xi )]\) is Lipschitz continuous in W1, ∞× L∞ with constant \(\bar \kappa \).
Define the set
Clearly, \(\text {gph}(\bar x) + \{0\} \times {I\!\!B}_{\varepsilon _{0}}(0) \subset {\Gamma }\). For τ ∈ [0, T] denote
Since for every τ ∈ [0, T] we have \({\Gamma }_{\tau } \subset B_{\varepsilon }(\bar x(\tau ))\), the function \({\Gamma }_{\tau } \ni \xi \mapsto \bar u[\tau ,\xi ]\) is Lipschitz continuous with Lipschitz constant \(\bar \kappa \). Moreover, each of these functions \(\bar u[\tau ,\xi ]\) is Lipschitz continuous on [τ, T] with Lipschitz constant \(\bar L\). In addition, according to Proposition 2, for every τ ∈ [0, T) and ξ ∈Γτ, the function \(\bar u[\tau ,\xi ]\) is the unique locally optimal control for the corresponding problem P(τ, ξ) in the set \({I\!\!B}_{\bar a}(\bar u)\).
Now, define feedback control mapping x↦u∗(⋅, x) as
The values \(\bar u[t,x](t)\) are well defined since \(\bar u[t,x]\) is a (Lipschitz) continuous function. Clearly, all requirements of Definition 1 are satisfied; the last one following from the identity \(\bar u[t,\bar x(t)] = \bar u(t)\).
Let us consider two arbitrary pairs (τ, ξ),(s, η) ∈Γ. Due to the Dynamic Programming Principle, for every s ≥ τ, τ, s ∈ [0, T) and every t ∈ [s, T], we have
Then,
where M is an upper bound of |f(t, x, u)| in the set Ω defined in Remark 1. This completes the proof of Theorem 4.
□
Remark 4
The last part of the proof and the uniqueness claim in Proposition 2 imply that the function \(\hat u[\tau ,\xi ]\) appearing in Definition 1 is the unique locally optimal control for in problem P(τ, ξ) in the set \({I\!\!B}_{\bar a}(\bar u)\).
6 Regularity of the Value Function
In this section, we show that the existence of a Lipschitz continuous optimal feedback established in Theorem 4 implies certain smoothness properties of the value function. In the preceding sections we assume only local optimality at the reference point, see Definition 1. In line with that assumption, we introduce the following definition:
Definition 2
The function \(V: [0,T] \times \mathbb {R}^{n} \to \mathbb {R}\) is said to be a local value function of problem (1)–(2) around a reference admissible pair \((\bar x, \bar u)\) if there exist positive numbers ε0 and \(\bar a\), and a set \({\Gamma } \subset [0,T] \times \mathbb {R}^{n}\) such that \(\text {gph}(\bar x) + \{0\} \times {I\!\!B}_{\varepsilon _{0}}(0) \subset {\Gamma }\) and for every (τ, ξ) ∈Γ one has
where the infimum of the objective function J(τ, ξ;u) in problem P(τ, ξ) is taken over all admissible pairs (x, u) for which \(\|u - \bar u\|_{\infty } \leq \bar a\), (18) has a unique solution x on [τ, T], and gph(x) ⊂Γ.
By this definition, the local value function, with a set Γ and a neighborhood \({I\!\!B}_{\bar a}(\bar u)\), is finite if for every (τ, ξ) ∈Γ there exists at least one admissible pair (x, u) satisfying \(\| u - \bar u \|_{\infty } \leq \bar a\) and gph(x) ∈Γ. Clearly, in that case \((\bar x, \bar u)\) is a locally optimal solution.
As in Section 5, we denote Γτ := {ξ : (τ, ξ) ∈Γ}. Thus, the condition \(\text {gph}(\bar x) + \{0\} \times {I\!\!B}_{\varepsilon _{0}}(0) \subset {\Gamma }\) in Definition 2 means that \({I\!\!B}_{\varepsilon _{0}}(\bar x(t)) \subset {\Gamma }_{t}\) for every t ∈ [0, T]. We also denote \(\overset {\circ }{\Gamma } = \{(\tau ,\xi ) ~:~ \tau \in [0,T],~\xi \in \text {int}({\Gamma }_{\tau })\}\).
Theorem 5
Let the coercivity condition and the isolatedness condition hold.Then, problem (1)–(2) has a (finite) local value function V around\((\bar x, \bar u)\)(witha set Γ and parametersε0and\(\bar a\));moreoverV (τ,⋅) is differentiable with respect toξwhenever\((\tau ,\xi ) \in \overset {\circ }{\Gamma }\)andthe derivativeVξis Lipschitz continuous on\(\overset {\circ }{\Gamma }\).
Proof
The proof is routine, in principle, but we present it in full, because we deal here with a local value function, which requires some attention to detail. We will prove the theorem with Γ, ε0 and \(\bar a\) as in Theorem 4. Then, there is a locally optimal Lipschitz continuous feedback control u∗ in the sense of Definition 1, with the corresponding pairs \((\hat x[\tau ,\xi ], \hat u(\tau ,\xi ])\). According to this definition, we have
First, we prove the following claim.
Claim A: for every \((\tau ,\xi _{0}) \in \overset {\circ }{\Gamma }\) there exists a number δ > 0 such that for every ξ, ξ′∈ IBδ(ξ0) one has \(\|\hat u[\tau ,\xi ] - \bar u\|_{\infty } \leq \bar a\) and the initial value problem
has a unique solution \(x^{\xi ,\xi ^{\prime }}\) on [τ, T] and \(\text {gph}(x^{\xi ,\xi ^{\prime }}) \subset {\Gamma }\). Note that \(x^{\xi ,\xi } = \hat x[\tau ,\xi ]\).
We recall (see Definition 1) that \(\hat x[\tau ,\xi ]\) is the solution of (18) with control \(u^{\ast }(t,\hat x[\tau ,\xi ](t))\), where u∗ is defined in (28). Since x↦u∗(⋅, x) is Lipschitz continuous, we obtain by a standard argument that for any \((\tau ,\xi _{0}) \in \overset {\circ }{\Gamma }\) there is a constant \(\delta _{0} \in (0, \bar a/\bar L]\) such that \({I\!\!B}_{\delta _{0}}(\hat x[\tau ,\xi _{0}](t)) \subset {\Gamma }_{t}\) for every t ∈ [τ, T]. Indeed, due to the Lipschitz continuity of the right-hand side of (18) every solution starting backwards from a point in a sufficiently small neighborhood of \(\hat x[\tau ,\xi _{0}](t)\) at time t > τ takes values only in \({I\!\!B}_{\varepsilon _{0}}(\xi _{0})\) at time τ. From the definition of Γ in (27), the graph of each such trajectory in contained in Γ and \(\|\hat u[\tau ,\xi ] - \bar u\|_{\infty } \leq \bar a\). Then, Claim A follows from (30) thanks to the Lipschitz continuity of \(\hat u[\tau ,\xi ]\) in ξ (Proposition 2). The proof of that last assertion, e.g., by contradiction, is straightforward
In addition, we have the representation
where Φ[τ, ξ](t, s) is the fundamental matrix solution of the linearization of the differential equation in (30) normalized at t = s, that is,
In particular, there exists a constant C such that for every ξ, ξ′∈ IBδ(ξ0) one has
From Definition 2 and Claim A, we have that
Utilizing this last inequality in (29), we obtain
where \(\tilde x^{T} \in \text {co}\left \{x^{\xi ,\xi ^{\prime }}(T), \hat x[\tau ,\xi ](T)\right \}\) and \(\tilde x(\cdot )\) is a measurable selection of the set-valued map \(s \mapsto \text {co} \{x^{\xi ,\xi ^{\prime }}(s), \hat x[\tau ,\xi ](s)\}\). Due to the Lipschitz continuity of gx and hx, and (31), we obtain
It is well-known that the expression in the brackets equals \((\hat \lambda [\tau ,\xi ](\tau ))^{\top }\), where, as before, \(\hat \lambda [\tau ,\xi ]\) is the solution of the adjoint equation (9) for the reference pair \((\hat x[\tau ,\xi ], \hat u[\tau ,\xi ])\) and end-point condition \(\hat \lambda (T) = g_{x}(\hat x[\tau ,\xi ](T))\). Hence,
Using this inequality with ξ′ = ξ0 and taking into account the Lipschitz continuity of \(\hat \lambda [\tau ,\xi ]\) with respect to ξ, we obtain
Using (32) with ξ′ := ξ and ξ = ξ0, we obtain
Combining the last two inequalities, we obtain that V is differentiable with respect to ξ at ξ0; furthermore, \(V_{\xi }(\tau ,\xi _{0}) = \hat \lambda [\tau ,\xi _{0}](\tau )\). The Lipschitz continuity of Vξ follows from the last expression and the Lipschitz continuity of the function \((\tau ,\xi ) \mapsto \hat \lambda [\tau ,\xi _{0}](\tau )\). □
Observe that Γ, ε0 and \(\bar a\) in this theorem can be taken to be those in the proof of Theorem 4. Also, observe that at the end of the last proof we obtained the equality \(V_{\xi }(\tau ,\xi _{0}) = \hat \lambda [\tau ,\xi _{0}](\tau )\), which, as is well known, holds under various sets of assumptions. Moreover, based on Theorem 5, one can verify that if \(\bar u\) is a globally optimal solution, then the value function V is a classical solution of the corresponding Hamilton-Jacobi-Bellman equation (see, e.g., [1, Chapter III.3]).
Notes
See Errata and Addenda at https://sites.google.com/site/adontchev/
References
Bardi, M., Capuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser, Boston (1997)
Cibulka, R., Dontchev, A. L., Krastanov, M., Veliov, V. M.: Metrically regular differential generalized equations. SIAM J. Control Optim. 56, 316–342 (2018)
Cibulka, R., Preninger, J., Roubal, T.: On Uniform Regularity and Strong Regularity. Research Report, 2018–01, ORCOS, TU Wien (2018)
Dontchev, A.L.: Efficient estimates of the solutions of perturbed control problems. J. Optim. Theory Appl. 35, 85–109 (1981)
Dontchev, A.L., Hager, W.W.: Lipschitzian stability in nonlinear control and optimization. SIAM J. Control Optim. 31, 569–603 (1993)
Dontchev, A.L., Rockafellar, R.T.: Implicit Functions and Solution Mappings, 2nd edn. Springer, New York (2014)
Hager, W.W.: Multiplier methods for nonlinear optimal control. SIAM J. Numer. Anal. 27, 1061–1080 (1990)
Ziemer, W.P.: Modern Real Analysis, 2nd edn. Springer, Berlin (2017)
Acknowledgements
Open access funding provided by TU Wien (TUW). A.L. Dontchev is supported by the National Science Foundation Award CMMI 1562209 and by the Australian Research Council (ARC) Project DP160100854. M.I. Krastanov is supported by the Sofia University “St. Kliment Ohridski” under contract no. 80-10-133/25.04.2018. V.M. Veliov is supported by the Austrian Science Foundation (FWF) under grant no. P31400-N32.
Author information
Authors and Affiliations
Corresponding author
Additional information
Dedicated to Alexander Ioffe.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Dontchev, A.L., Krastanov, M.I. & Veliov, V.M. On the Existence of Lipschitz Continuous Optimal Feedback Control. Vietnam J. Math. 47, 579–597 (2019). https://doi.org/10.1007/s10013-019-00347-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10013-019-00347-5