On the turnpike property with interior decay for optimal control problems

In this paper the turnpike phenomenon is studied for problems of optimal control where both pointwise-in-time state and control constraints can appear. We assume that in the objective function, a tracking term appears that is given as an integral over the time-interval [0,T]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$[0,\, T]$$\end{document} and measures the distance to a desired stationary state. In the optimal control problem, both the initial and the desired terminal state are prescribed. We assume that the system is exactly controllable in an abstract sense if the time horizon is long enough. We show that that the corresponding optimal control problems on the time intervals [0,T]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$[0, \, T]$$\end{document} give rise to a turnpike structure in the sense that for natural numbers n if T is sufficiently large, the contribution of the objective function from subintervals of [0, T] of the form [t-t/2n,t+(T-t)/2n]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} {[}t - t/2^n,\; t + (T-t)/2^n] \end{aligned}$$\end{document}is of the order 1/min{tn,(T-t)n}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1/\min \{t^n, (T-t)^n\}$$\end{document}. We also show that a similar result holds for ϵ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon $$\end{document}-optimal solutions of the optimal control problems if ϵ>0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon >0$$\end{document} is chosen sufficiently small. At the end of the paper we present both systems that are governed by ordinary differential equations and systems governed by partial differential equations where the results can be applied.


Introduction
Since the turnpike property has been discussed by P. A. Samuelson in mathematical economics in 1949 (see [2]), the turnpike phenomenon for optimization problems has been analyzed in various contexts, see for example [1,21,22]. For optimal control problems with partial differential equations see [15] or [19], where distributed control is considered for linear-quadratic optimal control problems. Problems of optimal boundary control are studied in [7]. In [17], both integral-and measure-turnpike properties are considered. The turnpike phenomenon for linear quadratic optimal control problems with time-discrete systems is studied in [5].
In this paper, we consider exponential integral-turnpike properties in an abstract framework where the system is governed by an abstract nonlinear semigroup. For our turnpike result we assume that the governing system is exactly controllable for sufficiently large time horizons. For systems that are governed by hyperbolic partial differential equations, this assumption is often satisfied. We consider a process on a finite time interval [0, T ] where an initial state is prescribed at the initial time zero and a terminal state is prescribed at the terminal time T . The objective function is given by an integral on the time interval [0, T ] and contains the control cost and a penalization of the distance to a desired stationary state. The solution of the corresponding static problem with this objective function defines a desired steady state. We show that the resulting optimal controls have a turnpike structure, where for sufficiently large T for any natural number n there exists a neighborhood of t ∈ (0, T ) of the form where the contribution to the objective function from this neighborhood decays with order 1/ min{t n , (T − t) n }. This result is shown in a general framework that allows for pointwise-in-time state constraints and pointwise-in-time control constraints. One option for the numerical treatment of problems with terminal constraints is the exact penalization of the terminal constraints that has been considered in [11]. The control constraints that we consider include the case of switching constraints, that only allow for a discrete set of control values (see [13]).
Our turnpike result is given for ε-optimal controls and states. For the case of finitedimensional systems, turnpike results for slightly suboptimal control-state pairs have already been stated in [4]. They imply that the members of a minimizing sequence for a fixed time horizon T approach the corresponding exponential turnpike structure. Since we consider ε-optimal control-state pairs, our turnpike results are also applicable to optimal control problems where the existence of an optimal control is not clear a priori. These turnpike result are also useful for numerical computations since they show that for sufficiently large time horizons T , sufficiently accurate approximations of the optimal control-state pairs must be close to the optimal steady state in a neighborhood of T /2. This paper has the following structure. First we introduce some notation and define the dynamic optimal control problem. Then we motivate our analysis by pointing out that it completes the characterization of the turnpike situation as an addition to the measure turnpike property that has been studied in [17]. Our analysis is more specific in showing that close to interior points of the time interval the distance between the static and the dynamic optimum becomes very small. In order to clarify this, we introduce the measure turnpike property with interior decay. Then in Sect. 3 we state the turnpike result. For the proof, we need several auxiliary results. At the end of the paper, we present some examples where our results can be applied.

Notation and definition of optimal control problems
Let a time interval [0, T ], a Banach space Y with the norm · Y and a Banach space X with the norm · X be given. Let an initial state y 0 ∈ Y and a desired stationary state y d ∈ Y be given. We state the relation between the state and the control in the form y = (a, y 0 , u, t), where for a ∈ [0, ∞) and t > a the mapping maps We assume the semigroup property that for any subinterval (a 1 , t 1 ) of (a, t) we have As an example, think of a strongly continuous semigroup of contractions, see [20]. Let a static state y (σ ) ∈ Y corresponding to a static control u (σ ) ∈ X be given, that is the constant function y (σ ) in C([a, t], Y ) and u (σ ) as a constant control in L 2 ((a, t), X ) satisfy the equation Let real numbers a and b with a < b be given. Let f 0 : R × X × Y → [0, ∞) be a continuous function. For states y ∈ L 2 ((a, b), Y ) and controls in u ∈ L 2 ((a, b), X ) for any subinterval (a 1 , b 1 ) of (a, b), we have the inequality For states y ∈ L 2 ((a, b), Y ) and controls in u ∈ L 2 ((a, b), X ) define the objective function where y is the system state that is generated by the control function u. This type of objective functions has also been considered in [17]. For y 0 and y d ∈ Y , we consider a dynamic optimal control problem with the initial condition y(a) = y 0 (1.5) and the terminal condition We consider the state constraint where F is a nonempty closed subset of Y such that y (σ ) ∈ F. We also consider the control constraint where U is a nonempty closed subset of X such that u (σ ) ∈ U . For real numbers a, b > a, an initial state y 0 ∈ Y and a terminal state y d ∈ Y consider the parametric optimization problem Let v(a, b, y 0 , y d ) denote the optimal value for P(a, b, y 0 , y d ).

The measure turnpike property with interior decay
In [17] a certain integral turnpike property and a stronger measure turnpike property are studied. In Definition 2 in [17] (see also [3,22]) the measure turnpike property is defined as follows: The problem P(a, b, y 0 , y d ) enjoys the measure turnpike property at (y (σ ) , u (σ ) ) ∈ F × U if for every ε > 0 there exists a real number (ε) > 0 such that for all b > a we have the inequality where μ denotes the Lebesgue measure of the set.
In [17], the measure turnpike property is shown under the strict dissipativity assumption. Similarly as in Definition 1 in [17], we define: The problem P(a, b, y 0 , y d ) is strictly dissipative at (y (σ ) , u (σ ) ) ∈ F × U if f 0 is time-independent and for all (y, u) ∈ F × U for the supply rate function there exists a storage function S : F → R that is locally bounded and bounded from below and a continuous and strictly increasing function α : [0, ∞) → [0, ∞) with α(0) = 0 such that for all b > a the dissipation inequality holds, that is for any admissible pair (y(·), u(·)) and for all τ ∈ [a, b] we have Let M S denote an upper bound for |S(y)| for y ∈ F.

Remark 2.1
The relation between strict dissipativity and turnpike properties is discussed in [4], see also [3].

Example 2.3
In [17], an example for a strictly dissipative optimal control problem with distributed control of the heat equation is given.
The measure turnpike property defined above can be satisfied if there exist real numbers M > 0, ϒ 0 > 0 and λ ∈ (0, 1) such that for all b sufficiently large such that holds. However, such a situation contradicts the intuition about the turnpike phenomenon that in the interior of the time interval close to the point a + λ (b − a), the distance of the dynamic optimum to the static optimum becomes very small, which is for example the case if an exponential turnpike property holds. In order to exclude such a situation, we say that the problem P(a, b, y 0 , y d ) enjoys the measure turnpike property with interior decay at (y (σ ) , u (σ ) ) ∈ F × U if problem P(a, b, y 0 , y d ) enjoys the measure turnpike property at (y (σ ) , u (σ ) ) ∈ F × U and in addition there exist C 1 > 0 and λ 1 ∈ (0, 1) such that for all λ ∈ (0, 1) and all b sufficiently large we have the inequality .
2) is valid, the measure turnpike property with interior decay cannot hold. This can be seen as follows. If (2.3) holds we have

Hence (2.4) contradicts (2.5).
So we see that the measure turnpike property with interior decay provides a more detailed picture of the turnpike phenomenon than the classical measure turnpike.

Remark 2.4
Note that also an exponential turnpike property, where there exist C 1 > 0 and μ > 0 such that for all T > 0 for all t ∈ [0, T ] we have implies the measure turnpike property with interior decay.
For finite-dimensional systems, the exponential turnpike property has been studied in [18]. Exponential sensitivity and turnpike analysis for linear quadratic optimal control of general evolution equations have been studied in [6]. The results that we present here allow for general nonlinear infinite-dimensional dynamcis that satisfy (1.1).

A general turnpike result
Throughout this section we assume that for all (a, b) ⊂ (0, T ) and all z 0 , z 1 ∈ F the problems P(a, b, z 0 , z 1 ) are strictly dissipative at (y (σ ) , u (σ ) ) ∈ F × U . For the subsequent analysis, we replace the objective function in (1.4) by where we assume that for all y ∈ F and u ∈ U we have ω(y, u) ≥ 0. Note that this is equivalent to subtracting the number (b − a) f 0 (y (σ ) , u (σ ) ) from the objective function. We show that under the strict dissipativity and an abstract exact controllability assumption that we define in (3.7) below, problem P(a, b, y 0 , y d ) has the measure turnpike property with interior decay. This is stated in Theorem 3.6. The exact controllability assumption requires that if the control time is greater than or equal to a minimal time t min , the optimal values of the control problem are uniformly bounded with a bound that depends only on the distance between the initial state and y (σ ) and the distance between the terminal state and y (σ ) .
For our analysis we need the following lemma.
Then for any -optimal stateŷ (a 1 , a 2 , y 0 , This implies the inequality where the last step follows from the definition (1.9) of an -optimal solution and the definition (3.1) of the objective function. Define Then we have the inequality

Integration yields
This is a contradiction.
Now we consider the optimal control Problem P(0, T , y 0 , y d ).

Consider the intervals
Then we have I λ + ⊂ (0, T /2) and I λ − ⊂ (T /2, T ). For the length of I λ + we have (3.6) Now we state our abstract exact controllability assumption: Assume that there exist a constant μ 0 > 0 and a time t min > 0 such that for all initial times t init and all initial states z init ∈ F and for all terminal times t term and all terminal states z term ∈ F if t term − t init ≥ t min we have the inequality With (3.5) and (3.6), inequality (3.7) implies that if T ≥ t min we have (3.7) is that for t 0 ≥ t min there exists a control function u 1 ∈ L 2 (a, t 0 ) such that u 1 (t) ∈ U for t ∈ (a, t 0 ) almost everywhere,

Remark 3.2 A sufficient condition for
This is the case if for t > t min , we have y 1 (t) = y (σ ) and u 1 (t) = u (σ ) .
As an example, think of a system that can be steered exactly to y (σ ) at the time t min with the control function u 1 | (a,t min ) and where the state remains equal to y

With (1.12) and (3.7) this yields
With the upper bounds for X that we have derived above this yields where M S denotes an upper bound for |S(y (0, T , y 0 , y d )(·))|.
Thus we obtain an upper bound for the part of the objective function (3.1) that comes from the subinterval (λ T /2, (1 + λ)T /2). Note that the point λ T is contained in this interval. More generally, by an inductive procedure where for n ∈ {2, 3, 4, ...} we consider the nested sequence of subintervals of [0, T ] that all contain the point λ T . With the special choice λ = 1 2 the intervals have the mindpoint T /2. We obtain an upper bound for the part of the objective function that comes from these subinterval. For T → ∞, upper bound decays as 1/T n−1 . This is our main turnpike result that is stated in the following theorem.
Then we have the inequality Note that for all k ≥ 2 we have g k → 0 for T → ∞.

Remark 3.5
For ε = 0, the assumptions of Theorem 3.3 imply that the measure turnpike property with interior decay defined in Sect. 2 holds. In fact we have the following more concise theorem: Theorem 3.6 As throughout the section, assume that for all (a, b) ⊂ (0, T ) and all z 0 , z 1 ∈ F the problems P(a, b, z 0 , z 1 ) are strictly dissipative at (y (σ ) , u (σ ) ) ∈ F × U and that for the storage function S there exists a constant L s such that for all y ∈ F with α( y − y σ Y ) sufficiently small we have Assume that the abstract exact controllability assumption (3.7) holds for all t 0 > a with t 0 − a ≥ t min and all initial states z 0 ∈ Y and all terminal states z d ∈ Y . Then problem P(0, T , y 0 , y d ) enjoys the measure turnpike property with interior decay at (y (σ ) , u (σ ) ) ∈ F × U .
Proof For the proof of Theorem 3.6, we first point out that in the setting of [17] it has been shown that strict dissipativity implies the measure turnpike property. In our framework, the measure turnpike properties follows using the exact controllability to show the uniform boundedness (with respect to T ) of the objective function (3.1). Moreover, in the definition of the measure turnpike property with interior decay in (2.3), with a derivation similar as for (3.8), we can choose λ 1 = 1 2 and

Remark 3.7
If M S = 0, Theorem 3.3 implies that for a minimizing sequence, that is a sequence of ε k -optimal solutions with lim k→∞ ε k = 0, the members of the sequence come closer and closer to an exponential turnpike structure.

Remark 3.8
It is also possible to state a variant of Theorem 3.3 for the case that no terminal constraint (1.6) is present. In this case, in the bound on the right-hand side λ 0 can be replaced with λ. Actually, the proof is easier in this case since as an upper bound for the subintervals that appear in the integral we can always take T .

Remark 3.9
In order to discuss Theorem 3.3, we consider again the nested intervals I n (λ, T ) defined in (3.9). For all n ∈ {1, 2, 3, ...} we have λT ∈ I n (λ, T ) and I n+1 (λ, T ) ⊂ I n (λ, T ). The intervals I n (λ, T ) have the length l(I n (λ, T )) = T /2 n−1 and This means that the average values of the integrand in the objective functions on the intervals I n (λ, T ) decays for T → ∞ with the order For ε > 0, the last term is decisive for the speed of convergence which is only of the order 1/T . For ε = 0, only we obtain faster convergence of the order 1/T 2 .
If also M S = 0, only the first term remains. If λ is close to zero or close to 1, the convergence to zero in the first term becomes slower than for λ close to 1/2. Hence in our turnpike result we have the typical situation that sufficiently close to the middle T /2 of the time interval [0, T ] the convergence to zero is faster than at the boundaries.

Example 3.10
From the inequality in Theorem 3.3, for n = 3 and λ = 1/2, if T ≥ 8 t min , for = 0 we obtain For n = 4 if T ≥ 16 t min we obtain 9 16 T This illustrates the typical turnpike behavior: For every natural number n ≥ 2 if T is sufficiently large the contribution to the objective function that comes from the interval is less than a constant that is independent of T multiplied by 1/T n−1 plus the term with M S /T .

Examples
In this section we present examples of optimal control problems where Theorem 3.3 is applicable.
Example 4. 1 We start with a system similar to the motivating example in [11] that is governed by an ordinary differential equation. Define Here the turnpike is zero, that is y (σ ) = 0 and u (σ ) = 0. First we show that the feasible set is nonempty. Define t 0 = 1 andû(t) = e − e t ≥ 0 for t ∈ (0, t 0 ) and u(t) = 0 for t ≥ t 0 . Then for t ∈ (0, t 0 ) we havê and for t ≥ t 0 we haveŷ(t) = 0.
All the feasible controls can be characterized by the moment equation Hence for all feasible controls u ≥ 0 where y ≤ 0 integration by parts yields For T > 1, due to the L 1 -norm that appears in the objective function, the solution has an extreme turnpike structure where the system is steered to zero in the finite time t 0 = 1 that is independent of T and remains there for t ∈ (T 0 , T ). This can be seen as follows. Let u(t) =û(t) + δ(t) with T 0 δ(τ ) dτ = 0 and δ(t) ≥ 0 for t ≥ t 0 . Then we have Thusû is the optimal control. Example 4.2 Let us consider another system that is governed by an ordinary differential equation. Let y 0 , y 1 be vectors in R n and f : R n → R n , g : R n → R n×m be C 2 maps with f (0) = 0. Let C ∈ R n×n be regular and define the Hilbert space Y with the norm In [16], the following problem is considered: Here the turnpike is zero, that is y (σ ) = 0 and u (σ ) = 0. In this case, the exact controllability assumption (3.7) requires that there is a constant μ 0 > 0 and a time t If the system is linear, that is of the form y = A y + Bu, the exact controllability can be checked by Kalman's rank condition.
Our results show that also with additional constraints, for example with the solution of (OCP) T has a turnpike structure as described in Theorem 3.3.
Consider the optimal control problem For an initial state y 0 = (y p , y v ) ∈ Y and a control u ∈ L 2 ((0, T ), X ) the map (a, y 0 , u, t 0 ) is given by the solution y of the initial boundary value problem on the time interval (a, t 0 ) Similar as in Theorem 1.1 in [8] it can be shown that maps to the function space C([a, T ], Y ). In fact, corresponds to a strongly continuous semigroup of contractions, see [20], hence (1.2) holds. The exact controllability of the system with controls u ∈ L 2 (0, T ), Y ) is shown for example in [14]. In this case we have t min = 2. Inequality (3.7) can be shown using the observability inequality (65) from [14]. Theorem 3.6 shows that the problem enjoys the measure turnpike property with interior decay. Moreover, since Theorem 3.3 is applicable with S M = 0, the solution has a turnpike structure where the contribution of the objective function from time intervals of the form where M > 0 is a given upper bound, we obtain the problem with state constraints from [8]. In this case it is requires more work to verify (3.7), since it requires to solve a problem of constrained exact controllability that respects the state constraints. We expect this property to hold for sufficiently regular initial and terminal states-However, it is out of the scope of this presentation.
The situation is similar if additional control constraints of the form |u| ≤ M 1 with a given bound M 1 > 0 are present. Problems of this type with pure homogeneous Neumann boundary conditions are considered in [12]. Here the verification of (3.7) requires to show control-constrained exact controllability. As shown in [12], this requires additional regularity of the initial state and the desired state.
We have Y = L 2 (0, 1) and y (σ ) = 0, u (σ ) = 0. Note that also the desired terminal state is zero, so in this case both the objective function and the terminal state drive the system to the zero state. In this case, the exact controllability assumption (3.7) requires that there is a constant μ 0 > 0 and a time t min > 0 such that if t 0 − a ≥ t min for all z 0 , z d ∈ Y we have the inequality v(a, t 0 , z 0 , Note that the Euler-Bernoulli beam is exactly controllable in arbitrarily short times (see [20], Example 11.2.8), so in this case t min > 0 can be chosen arbitrarily small. However, for t min decreasing to zero the constant μ 0 has to be chosen larger.

Conclusion
We have shown a turnpike theorem for a problem of optimal control that is stated in a general framework and allows for state constraints and control constraints. The control-to-state map can be nonlinear. In the optimal control problem, both the initial state and the terminal state are prescribed but the results are also applicable in the case where only initial conditions are prescribed. We have shown that for strictly dissipative systems that are exactly controllable the optimal control problems enjoy the measure turnpike property with interior decay which is a stronger property than the classical measure turnpike property. The turnpike result shows that regardless of the initial state, for a sufficiently large time horizon T for the optimal controls the contribution of the objective function that comes from a sufficiently small neighborhood of the middle of the time interval decays faster than 1/T . For -optimal state-control pairs a similar estimate holds where in the upper bound an additional constant that is multiplied by appears.