Infinite Horizon Sparse Optimal Control

Abstract

A class of infinite horizon optimal control problems involving nonsmooth cost functionals is discussed. The existence of optimal controls is studied for both the convex case and the nonconvex case, and the sparsity structure of the optimal controls promoted by the nonsmooth penalties is analyzed. A dynamic programming approach is proposed to numerically approximate the corresponding sparse optimal controllers.

Appendix

The proof of Theorem 3.1 is given as follows.

Proof

Let $(T_k)_{k\in {\mathbb {N}}}$ be an arbitrary sequence of positive numbers with

$$\begin{aligned} T_k<T_{k+1},\quad \forall \,k\in {\mathbb {N}}\ \text {and}\ T_k\rightarrow \infty \ \text {as}\ k\rightarrow \infty . \end{aligned}$$

For each $k\in {\mathbb {N}}$, consider the following optimal control problems $(P_k)$ defined on the finite time interval $[0,T_k]$:

$$\begin{aligned} \inf _{u\in L^{\infty }(0,T_k;U)} J_k(x,u), \end{aligned}$$

(27)

where

$$\begin{aligned} J_k(x,u)=\int ^{T_k}_0 e^{-\lambda s}\ell (y(s),u(s))\hbox {d}s \end{aligned}$$

and $(y(\cdot ),u(\cdot ))$ satisfies the dynamical system (2). Due to Lemma 3.2, there exists an optimal control $u_k$ for (27). Denote by $y_k$ the optimal trajectory corresponding to $u_k$ and

$$\begin{aligned} \eta _k(s):=\int ^s_0 e^{-\lambda (s'-s)}\ell (y_k(s'),u_k(s'))\hbox {d}s',\quad \forall \,s\in [0,T_k]. \end{aligned}$$

Then $\eta _k$ is optimal for (8) with the final time $T_k$. The aim is to construct an admissible control ${\bar{u}}$ defined on the infinite time interval $[0,\infty [$. The corresponding extended state will be denoted by $({\bar{y}},{\bar{\eta }})$. The construction is described step by step as follows.

Consider at first the sequence $(y_k,\eta _k)_{k\in {\mathbb {N}}}$ on the time interval $[0,T_1]$. By the same arguments as in the proof of Lemma 3.1, there exists a subsequence $(y_{k,1},\eta _{k,1})$ of $(y_k,\eta _k)$ such that

$$\begin{aligned}&(y_{k,1},\eta _{k,1})\rightarrow ({\bar{y}},{\bar{\eta }}) \text { uniformly in } [0,T_1],\ \text {as}\ k\rightarrow \infty ;\\&(\dot{y}_{k,1},\dot{\eta }_{k,1})\rightarrow (\dot{{\bar{y}}},\dot{{\bar{\eta }}}) \text { weakly in } L^1(0,T_1;{\mathbb {R}}^{d+1}),\ \text {as}\ k\rightarrow \infty , \end{aligned}$$

for some $({\bar{y}},{\bar{\eta }})$ satisfying (9) on $[0,T_1]$. Note that each control $u_{k,1}$ is an optimal control for a corresponding problem $(P_{n(k,1)})$ of the form (27) for some integer $n(k,1)\ge 1$ on the time interval $[0,T_{n(k,1)}]$, and each $\eta _{k,1}$ is optimal for the problem (8) with $T_{n(k,1)}$.

Now for the second step, consider the sequence $(y_{k,1},\eta _{k,1})_{k\in {\mathbb {N}}}$ on the time interval $[0,T_2]$ for $k\ge 2$. Analogously to the previous step, there exists $(y_{k,2},\eta _{k,2})$ of $(y_{k,1},\eta _{k,1})$ such that

$$\begin{aligned}&(y_{k,2},\eta _{k,2})\rightarrow ({\bar{y}},{\bar{\eta }}) \text { uniformly in } [0,T_2],\ \text {as}\ k\rightarrow \infty ;\\&(\dot{y}_{k,2},\dot{\eta }_{k,2})\rightarrow (\dot{{\bar{y}}},\dot{{\bar{\eta }}}) \text { weakly in } L^1(0,T_2;{\mathbb {R}}^{d+1}),\ \text {as}\ k\rightarrow \infty , \end{aligned}$$

for $({\bar{y}},{\bar{\eta }})$ satisfying (9) on $[0,T_2]$. Here $({\bar{y}},{\bar{\eta }})$ coincides with the one constructed in the previous step on $[0,T_1]$, and it is denoted again by the same symbol. Each control $u_{k,2}$ is an optimal control for a corresponding problem $(P_{n(k,2)})$ of the form (27) for some integer $n(k,2)\ge 2$ on the time interval $[0,T_{n(k,2)}]$, and each $\eta _{k,2}$ is optimal for the problem (8) with $T_{n(k,2)}$.

By repeating this procedure, we construct $({\bar{y}},{\bar{\eta }})$ satisfying (9) on the infinite time interval $[0,\infty )$. Simultaneously, we obtain a countable family of $(y_{k,i},\eta _{k,i})$ for $i,k\in {\mathbb {N}}$, $k\ge i$. Each $u_{k,i}$ is an optimal control for problem $(P_{n(k,i)})$ of the form (27) for some integer $n(k,i)\ge i$ on the time interval $[0,T_{n(k,i)}]$, and each $\eta _{k,i}$ is optimal for the problem (8) with $T_{n(k,i)}$. Moreover, for all $i\in {\mathbb {N}}$,

$$\begin{aligned}&(y_{k,i},\eta _{k,i})\rightarrow ({\bar{y}},{\bar{\eta }}) \text { uniformly in } [0,T_i],\ \text {as}\ k\rightarrow \infty ;\\&(\dot{y}_{k,i},\dot{\eta }_{k,i})\rightarrow (\dot{{\bar{y}}},\dot{{\bar{\eta }}}) \text { weakly in } L^1(0,T_i;{\mathbb {R}}^{d+1}),\ \text {as}\ k\rightarrow \infty , \end{aligned}$$

Let us take the diagonal sequence $(u_{k,k})_{k\in {\mathbb {N}}}$ and denote

$$\begin{aligned} {\tilde{u}}_k=u_{k,k},\ {\tilde{y}}_k=y_{k,k},\ {\tilde{\eta }}_k=\eta _{k,k},\ \text {and}\ n_k=n(k,k),\quad \forall \,k\in {\mathbb {N}}. \end{aligned}$$

Then the following properties hold:

• (i) $\forall \,k\in {\mathbb {N}}$, the control ${\tilde{u}}_k$ is defined on the time interval $[0,T_{n_k}]$ with $n_k\ge k$, ${\tilde{u}}_k$ is an optimal control for the problem $(P_{n_k})$ of the form (27), and ${\tilde{\eta }}_k$ is optimal for the problem (8) with $T_{n_k}$.

• (ii) $\forall \,i\in {\mathbb {N}}$, we have

  $$\begin{aligned}&({\tilde{y}}_k,{\tilde{\eta }}_k) \rightarrow ({\bar{y}},{\bar{\eta }}) \text { uniformly in } [0,T_i]\ \text {as}\ k\rightarrow \infty ,\\&(\dot{{\tilde{y}}}_k,\dot{{\tilde{\eta }}}_k) \rightarrow (\dot{{\bar{y}}},\dot{{\bar{\eta }}})\ \text {weakly in}\ L^1(0,T_i;{\mathbb {R}}^{d+1})\ \text {as}\ k\rightarrow \infty . \end{aligned}$$

• (iii) There exists ${\bar{u}}\in L^{\infty }(0,\infty ;U)$ such that

  $$\begin{aligned} \dot{{\bar{y}}}(s)=f({\bar{y}}(s),{\bar{u}}(s))\ \text {and}\ \dot{{\bar{\eta }}}(s)\ge \lambda {\bar{\eta }}(s)+\ell ({\bar{y}}(s),{\bar{u}}(s)),\quad \forall \,s\in ]0,\infty [. \end{aligned}$$

We proceed with proving that ${\bar{u}}$ is an optimal control for the problem (1). Arguing by contradiction, if ${\bar{u}}$ is not optimal for (1), there exists $\varepsilon >0$ and $({\tilde{y}},{\tilde{u}})$ satisfying (2) such that

$$\begin{aligned} J(x,{\tilde{u}})+\varepsilon < J(x,{\bar{u}}). \end{aligned}$$

(28)

By the assumption (3), there exists $(y^*,u^*)$ satisfying (2) such that

$$\begin{aligned} J(x,u^*)<+\infty . \end{aligned}$$

Since ${\tilde{u}}_k$ is optimal for ($P_{n_k}$), we have for any $k\in {\mathbb {N}}$ that

$$\begin{aligned} J_{n_k}(x,{\tilde{u}}_k)\le J_{n_k}(x,u^*)\le J(x,u^*). \end{aligned}$$

Then for any $N>0$ and any $n_k$ with $T_{n_k}\ge N$,

$$\begin{aligned} \int ^N_0 e^{-\lambda s}\ell ({\tilde{y}}_k(s),{\tilde{u}}_k(s))\hbox {d}s\le J_{n_k}(x,{\tilde{u}}_k)\le J(x,u^*), \end{aligned}$$

and thus

$$\begin{aligned} e^{-\lambda N}{\tilde{\eta }}_k(N)\le J(x,u^*). \end{aligned}$$

By taking $k\rightarrow \infty$, it holds that

$$\begin{aligned} e^{-\lambda N}{\bar{\eta }}(N)\le J(x,u^*). \end{aligned}$$

We thus obtain,

$$\begin{aligned} \int ^N_0 e^{-\lambda s}\ell ({\bar{y}}(s),{\bar{u}}(s))\hbox {d}s\le e^{-\lambda N}{\bar{\eta }}(N)\le J(x,u^*),\quad \forall \,N>0, \end{aligned}$$

and therefore

$$\begin{aligned} \int ^{\infty }_0 e^{-\lambda s}\ell ({\bar{y}}(s),{\bar{u}}(s))\hbox {d}s\le J(x,u^*). \end{aligned}$$

There exists $k_1\in {\mathbb {N}}$ such that for any $k\ge k_1$

$$\begin{aligned} \int ^{\infty }_{T_{k}}e^{-\lambda s}\ell ({\bar{y}}(s),{\bar{u}}(s))\hbox {d}s<\frac{\varepsilon }{2}. \end{aligned}$$

(29)

Due to the fact that ${\tilde{\eta }}_k\rightarrow {\bar{\eta }}$ uniformly, there exists $k_2\ge k_1$ such that for all $k\ge k_2$,

$$\begin{aligned} e^{-\lambda T_{n_{k_1}}}{\bar{\eta }}\left( T_{n_{k_1}}\right) \le e^{-\lambda T_{n_{k_1}}}{\tilde{\eta }}_k\left( T_{n_{k_1}}\right) +\frac{\varepsilon }{2}, \end{aligned}$$

which implies that

$$\begin{aligned} J_{n_{k_1}}(x,{\bar{u}})\le J_{n_{k_1}}(x,{\tilde{u}}_k)+\frac{\varepsilon }{2}. \end{aligned}$$

(30)

Since ${\tilde{u}}_{k_2}$ is optimal for $(P_{n_{k_2}})$,

$$\begin{aligned} J_{n_{k_2}}(x,{\tilde{u}}_{k_2})\le J_{n_{k_2}}(x,{\tilde{u}})\le J(x,{\tilde{u}}). \end{aligned}$$

(31)

Note that $n_{k_1}\le n_{k_2}$, and thus together with (30) and (31), we have

$$\begin{aligned} J_{n_{k_1}}(x,{\bar{u}})\le & {} J_{n_{k_1}}(x,{\tilde{u}}_{k_2})+\frac{\varepsilon }{2} \le J_{n_{k_2}}(x,{\tilde{u}}_{k_2})+\frac{\varepsilon }{2} \le J(x,{\tilde{u}}) + \frac{\varepsilon }{2}. \end{aligned}$$

Finally, by (29) we deduce that

$$\begin{aligned} J(x,{\bar{u}})= & {} J_{n_{k_1}}(x,{\bar{u}})+ \int ^{\infty }_{T_{n_{k_1}}}e^{-\lambda s}\ell ({\bar{y}}(s),{\bar{u}}(s))\hbox {d}s \\\le & {} J_{n_{k_1}}(x,{\bar{u}})+ \frac{\varepsilon }{2} \le J(x,{\tilde{u}}) + \varepsilon , \end{aligned}$$

which contradicts (28). Hence, ${\bar{u}}$ is an optimal control for (1), which ends the proof. $\square$