Mixed Use of Pontryagin’s Principle and the Hamilton-Jacobi-Bellman Equation in Infinite- and Finite-Horizon Constrained Optimal Control

Abstract

This paper proposes a framework for solving a class of nonlinear infinite- and finite-horizon optimal control problems with constraints. Establishment of existence and uniqueness of solutions to the Hamilton-Jacobi-Bellman (HJB) equation plays a crucial role in verifying well-posedness of a given problem and in streamlining numerical solutions. The proposed framework revolves around infinite-horizon Bolza-type cost functions with running costs exponentially decaying in time. We show $\varGamma$-convergence of solutions with such cost functions to the solutions of initial constrained (in)finite-horizon problems (that is, without running costs exponentially decaying in time). Basically, we demonstrate how to approximate solutions of (in)finite-horizon constrained optimal problems using our framework. Employing a solver based on the Pontryagin’s Principle, we efficiently obtain optimal solutions for finite- and infinite-horizon problems. Efficiency of the proposed framework is demonstrated in simulation by solving a 3D path planing problem with obstacles for a full nonlinear model of an autonomous underwater vehicle (AUV).

Ivana Palunko: All authors are with LARIAT—Laboratory of intelligent and autonomous systems.

Appendix

In order to make this paper as self-contained as possible and for the reader’s convenience, we establish some additional notation needed for this section. Let $B_{r}(x)$ be the closed ball in $\mathbb {R}^{k}$ of radius $r > 0$ centered at $x\in \mathbb {R}^{k}$. For a nonempty set $C \subset \mathbb {R}^{k}$, we denote its convex hull by co(C).The negative polar cone of C is defined as $C^{-} := \{p \in \mathbb {R}^{k} : \langle p,c\rangle \le 0 \text { for all } c\in C\}$. Denote the set distance from $x\in \mathbb {R}^{k}$ to C by $d_{C}(x) := \inf \{|x-y| :\; y\in C\}$.

For a nonempty set $C \subset \mathbb {R}^{k}$, collection of nonempty subsets $\{C_{w}\}_{w\in C_{2}}$, and $w_{0}\in C_{2}$, let

$$\text {Lim}\sup \limits _{w\rightarrow w_{0}}C_{w} = \{v\in \mathbb {R}^{k}:\; \liminf \limits _{w\rightarrow w_{0}}\;d_{C_{w}}(v)=0\}$$

and

$$\text {Lim} \inf \limits _{w\rightarrow w_{0}}C_{w} = \{v\in \mathbb {R}^{k}:\; \limsup \limits _{w\rightarrow w_{0}}\;d_{C_{w}}(v)=0\}.$$

For $x \in \overline{C}$ define $T_{C}(x) := \{v\in \mathbb {R}^{k}: \liminf \limits _{h\rightarrow 0^{+}}\;d_{A}(x+hv)/h = 0 \}$ and $N_{C}(x) := \text {Lim} \sup \limits _{y\rightarrow C^{x}}T_{C}(y)^{-}$. Finally, denote the epigraph of a function $\varphi$ as epi $\varphi$.

Below we state the necessary assumptions and proposition required for the proof of Theorem 1.

• A1 For all $x \in \mathbb {R}^{n}$, the mappings $\tilde{f}(\cdot , x, \cdot )$ and $\tilde{F}(\cdot , x, \cdot )$ are Lebesgue-Borel measurable and there exists $\phi \in L^{1}([0,\infty );\mathbb {R})$ such that $\tilde{F}(t,x,u)\ge \phi (t)$ for a.e. $t \ge 0$ and all $(x,u) \in \mathbb {R}^{n}\times \mathbb {R}^{m}$.

• A2 There exists a function $c \in L^{1}_{loc}([0,\infty ); \mathbb {R}^{+})$ such that for a.e. $t \ge 0$ and for all $x \in \mathbb {R}^{n},\; u \in U(t)$

  $$|\tilde{f}(t,x,u)| + |\tilde{F}(t,x,u)| \le c(t)(1+|x|).$$

• A3 For a.e. $t \ge 0$ and all $x \in \mathbb {R}^{n}$, the set-valued map

  $$y \rightarrow \{(\tilde{f}(t,y,u), \tilde{F}(t,y,u)): u\in U(t)\}$$

  is continuous with closed images, and the set

  $$\{(\tilde{f}(t,x,u), \tilde{F}(t,x,u)+r): u \in U(t),\; r\ge 0 \}$$

  is convex.

• A4 There exists a function $k \in L^{1}_{loc}([0,\infty ); \mathbb {R}^{+})$ such that for a.e. $t\ge 0$ and for all $x,y\in \mathbb {R}^{n},\; u \in U(t)$: $|\tilde{f}(t,x,u) - \tilde{f}(t,y,u)| + |\tilde{F}(t,x,u)-\tilde{F}(t,y,u)| \le k(t)|x-y|.$

• A5 $k\in \mathscr {L}_{loc}$ and $\limsup _{t\rightarrow \infty } \frac{1}{t}\int ^{t}_{0}(c(s)+k(s))ds < \infty .$

• A6 There exists a function $q\in \mathscr {L}_{loc}$ such that for a.e. $t \ge 0$

  $$\sup \limits _{u\in U(t)}(|\tilde{f}(t,x,u)|+|\tilde{F}(t,x,u)|) \le q(t),\; \text {for all } x\in \partial C$$

  where $C \subset \mathbb {R}^{n}$ is closed.

• A7 Let $V: [0,\infty )\times C \rightarrow \mathbb {R}\cup \{\pm \infty \}$ be the value function of (8)–(9). Then assume Dom $V \ne \varnothing$ and there exist $T > 0$ and $\psi \in L^{1}([T,\infty ); \mathbb {R}^{+})$ such that for all $(t_{0}, x_{0}) \in \text {Dom }V \cap ([T,\infty )\times \mathbb {R}^{n})$ and any feasible trajectory-control pair $(x(\cdot ), u(\cdot ))$ on $[t_{0}, \infty ),$ with $x(t_{0}) = x_{0},$

  $$|\tilde{F}(t, x(t),u(t))| \le \psi (t)\text { for a.e. } t\ge t_{0}.$$

• A8 There exist $\eta ,\; r > 0$ and $M \ge 0$ such that for a.e. $t > 0$ and any $y\in \partial C + \eta B_{1}({\textbf {0}}),$ and any $v \in \tilde{f}(t,y,U(t)),$ with $\inf \limits _{n\in N^{1}_{y,\eta }}\langle n,v \rangle \le 0,$ we can find $w \in \tilde{f}(t,y,U(t))\cap B_{M}(v)$ satisfying

  $$\inf _{n\in N^{1}_{y,\eta }}\{\langle n,w\rangle ,\; \langle n, w-v \rangle \}\ge r,$$

  where $N^{1}_{y,\eta } := \{n \in \partial B_{1}({\textbf {0}}): n \in \overline{co}N_{C}(x),\; x \in \partial C\cap B_{\eta }(y)\}.$

Proposition 2

(Theorem 3.3, [2]) Let $W:[0,\infty ) \times C \rightarrow \mathbb {R}\cup \{+\infty \}$ be a lower semicontinuous function such that $\text {Dom }V(t,\cdot )\subset \text {Dom }W(t,\cdot ) \ne \emptyset$ for all large $t > 0$ and

$$\begin{aligned} \lim _{t\rightarrow \infty }\sup \limits _{y\in \text {Dom }W(t,\cdot )}|W(t,y)| = 0. \end{aligned}$$

(25)

Then the following statements are equivalent:

• 1. W = V;

• 2. W is a weak (or viscosity) solution of HJB equation on $(0,\infty )\times C$ and $t \rightarrow \text {epi }W(t,\cdot )$ is locally absolutely continuous.

In addition, V is the unique weak solution satisfying (25) with locally absolutely continuous $t \rightarrow \text {epi }V(t,\cdot )$.