A Uniform Tauberian Theorem in Optimal Control

Abstract

In an optimal control framework, we consider the value V _T (x) of the problem starting from state x with finite horizon T, as well as the value W _λ(x) of the λ-discounted problem starting from x. We prove that uniform convergence (on the set of states) of the values V _T ( ⋅) as T tends to infinity is equivalent to uniform convergence of the values W _λ( ⋅) as λ tends to 0, and that the limits are identical. An example is also provided to show that the result does not hold for pointwise convergence. This work is an extension, using similar techniques, of a related result by Lehrer and Sorin in a discrete-time framework.

Appendix

We give here another proof⁴ of Theorem 10.5 by using the analoguous result in discrete time [24] as well as an argument of equivalence between discrete and continuous dynamic.

Consider a deterministic dynamic programming problem in continuous time as defined in Sect. 10.2.1, with a state space Ω, a payoff g and a dynamic Γ. Recall that, for any ω ∈ Ω, Γ(ω) is the non empty set of feasible trajectories, starting from ω. We construct an associated deterministic dynamic programming problem in discrete time as follows.

Let $\widetilde{\Omega } = \Omega \times [0,1]$ be the new state space and let $\widetilde{g}$ be the new cost function, given by $\widetilde{g}(\omega ,x) = x$. We define a multivalued-function with nonempty values $\widetilde{\Gamma } :\widetilde{ \Omega } \rightrightarrows \widetilde{ \Omega }$ by

$$(\omega ,x) \in \widetilde{ \Gamma }(\omega \prime,x\prime)\Longleftrightarrow\exists X \in \Gamma (\omega \prime),\quad \text{ with }X(1) = \omega \quad \text{ and }\quad {\int \nolimits }_{0}^{1}g(X(t))\,\mathrm{d}t = x.$$

Following [24], we define, for any initial state $\widetilde{\omega } = (\omega ,x)$

$$\begin{array}{rcl} {v}_{n}(\widetilde{\omega })& =& \inf \frac{1} {n}\sum\limits_{i=1}^{n}\widetilde{g}(\widetilde{{\omega }}_{ i}) \\ {w}_{\lambda }(\widetilde{\omega })& =& \inf \lambda \sum\limits_{i=1}^{+\infty }{(1 - \lambda )}^{i-1}\widetilde{g}(\widetilde{{\omega }}_{ i}) \\ \end{array}$$

where the infima are taken over the set of sequences $\{\widetilde{{\omega }{}_{i}\}}_{i\in \mathbb{N}}$ such that $\widetilde{{\omega }}_{0} =\widetilde{ \omega }$ and $\widetilde{{\omega }}_{i+1} \in \widetilde{ \Gamma }(\widetilde{{\omega }}_{i})$ for every i ≥ 0.

Theorem 10.5 is then the consequence of the following three facts.

Firstly, the main theorem of Lehrer and Sorin in [24], which states that uniform convergence (on $\widetilde{\Omega }$) of v _n to some v is equivalent to uniform convergence of w _λ to the same v.

Secondly, the concatenation hypothesis (10.4) on Γ implies that for any $(\omega ,x)\in \widetilde{\Omega }$

$$\begin{array}{rcl}{ v}_{n}(\omega ,x) = {V }_{n}(\omega )& & \\ \end{array}$$

where ${V }_{t}(\omega ) {=\inf }_{X\in \Gamma (\omega )}\frac{1} {t} { \int \nolimits }_{0}^{n}g(X(s))\,\mathrm{d}s$, as defined in equation (10.7). Consequently, because of the bound on g, for any t ∈ ℝ ₊ we have

$$\vert {V }_{t}(\omega ) - {v}_{\lfloor t\rfloor }(\omega ,x)\vert \leq \frac{2} {\lfloor t\rfloor }$$

where ⌊t⌋ stands for the integer part of t.

Finally, again because of hypothesis (10.4), for any λ ∈ ]0, 1],

$${w}_{\lambda }(\omega ,x) {=\inf }_{X\in \Gamma (\omega )}\lambda {\int \nolimits }_{0}^{+\infty }{(1 - \lambda )}^{\lfloor t\rfloor }g(X(t))\,\mathrm{d}t.$$

Hence, by equation (10.8) and the bound on the cost function, for any λ ∈ ]0, 1],

$$\vert {W}_{\lambda }(\omega ) - {w}_{\lambda }(\omega ,x)\vert \leq \lambda {\int \nolimits }_{0}^{+\infty }\left\vert {(1 - \lambda )}^{\lfloor t\rfloor }-\mathrm{ {e}}^{-\lambda t}\right\vert \mathrm{d}t$$

which tends uniformly (with respect to x and ω) to 0 as λ goes to 0 by virtue of the following lemma.

Lemma 10.6.

The function

$$\lambda \mapsto \lambda {\int \nolimits }_{0}^{+\infty }\left\vert {(1 - \lambda )}^{\lfloor t\rfloor }-\mathrm{ {e}}^{-\lambda t}\right\vert \mathrm{d}t$$

converges to 0 as λ tends to 0.

Proof.

Since λ ∫₀ ^{+ ∞}(1 − λ)^⌊t⌋ = λ ∫₀ ^{+ ∞}e^− λtdt = 1, for any λ > 0, the lemma is equivalent to the convergence to 0 of

$$E(\lambda ) := \lambda {\int \nolimits }_{0}^{+\infty }{\left[{(1 - \lambda )}^{\lfloor t\rfloor }-\mathrm{ {e}}^{-\lambda t}\right]}_{ +}\mathrm{d}t$$

where [x]₊ denotes the positive part of x. Now, from the relation 1 − λ ≤ e^− λ, true for any λ, one can easily deduce that, for any λ > 0,  t ≥ 0, the relation (1 − λ)^⌊t⌋e^λt ≤ e^λ holds. Hence,

$$\begin{array}{rcl} E(\lambda )& =& \lambda {\int \nolimits }_{0}^{+\infty }\mathrm{{e}}^{-\lambda t}{\left[{(1 - \lambda )}^{\lfloor t\rfloor }\mathrm{{e}}^{\lambda t} - 1\right]}_{ +}\mathrm{d}t \\ & \leq & \lambda {\int \nolimits }_{0}^{+\infty }\mathrm{{e}}^{-\lambda t}(\mathrm{{e}}^{\lambda } - 1)\,\mathrm{d}t \\ & =& \mathrm{{e}}^{\lambda } - \end{array}$$

(1)

which converges to 0 as λ tends to 0. □