Abstract
In this paper the turnpike phenomenon is studied for problems of optimal control where both pointwise-in-time state and control constraints can appear. We assume that in the objective function, a tracking term appears that is given as an integral over the time-interval \([0,\, T]\) and measures the distance to a desired stationary state. In the optimal control problem, both the initial and the desired terminal state are prescribed. We assume that the system is exactly controllable in an abstract sense if the time horizon is long enough. We show that that the corresponding optimal control problems on the time intervals \([0, \, T]\) give rise to a turnpike structure in the sense that for natural numbers n if T is sufficiently large, the contribution of the objective function from subintervals of [0, T] of the form
is of the order \(1/\min \{t^n, (T-t)^n\}\). We also show that a similar result holds for \(\epsilon \)-optimal solutions of the optimal control problems if \(\epsilon >0\) is chosen sufficiently small. At the end of the paper we present both systems that are governed by ordinary differential equations and systems governed by partial differential equations where the results can be applied.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Since the turnpike property has been discussed by P. A. Samuelson in mathematical economics in 1949 (see [2]), the turnpike phenomenon for optimization problems has been analyzed in various contexts, see for example [1, 21, 22]. For optimal control problems with partial differential equations see [15] or [19], where distributed control is considered for linear–quadratic optimal control problems. Problems of optimal boundary control are studied in [7]. In [17], both integral- and measure–turnpike properties are considered. The turnpike phenomenon for linear quadratic optimal control problems with time-discrete systems is studied in [5].
In this paper, we consider exponential integral–turnpike properties in an abstract framework where the system is governed by an abstract nonlinear semigroup. For our turnpike result we assume that the governing system is exactly controllable for sufficiently large time horizons. For systems that are governed by hyperbolic partial differential equations, this assumption is often satisfied. We consider a process on a finite time interval [0, T] where an initial state is prescribed at the initial time zero and a terminal state is prescribed at the terminal time T. The objective function is given by an integral on the time interval \([0, \, T]\) and contains the control cost and a penalization of the distance to a desired stationary state. The solution of the corresponding static problem with this objective function defines a desired steady state. We show that the resulting optimal controls have a turnpike structure, where for sufficiently large T for any natural number n there exists a neighborhood of \(t\in (0,\, T)\) of the form
where the contribution to the objective function from this neighborhood decays with order \(1/ \min \{t^n, (T-t)^n\} \). This result is shown in a general framework that allows for pointwise-in-time state constraints and pointwise-in-time control constraints. One option for the numerical treatment of problems with terminal constraints is the exact penalization of the terminal constraints that has been considered in [11]. The control constraints that we consider include the case of switching constraints, that only allow for a discrete set of control values (see [13]).
Our turnpike result is given for \(\varepsilon \)-optimal controls and states. For the case of finite-dimensional systems, turnpike results for slightly suboptimal control-state pairs have already been stated in [4]. They imply that the members of a minimizing sequence for a fixed time horizon T approach the corresponding exponential turnpike structure. Since we consider \(\varepsilon \)-optimal control–state pairs, our turnpike results are also applicable to optimal control problems where the existence of an optimal control is not clear a priori. These turnpike result are also useful for numerical computations since they show that for sufficiently large time horizons T, sufficiently accurate approximations of the optimal control–state pairs must be close to the optimal steady state in a neighborhood of T/2.
This paper has the following structure. First we introduce some notation and define the dynamic optimal control problem. Then we motivate our analysis by pointing out that it completes the characterization of the turnpike situation as an addition to the measure turnpike property that has been studied in [17]. Our analysis is more specific in showing that close to interior points of the time interval the distance between the static and the dynamic optimum becomes very small. In order to clarify this, we introduce the measure turnpike property with interior decay. Then in Sect. 3 we state the turnpike result. For the proof, we need several auxiliary results. At the end of the paper, we present some examples where our results can be applied.
1.1 Notation and definition of optimal control problems
Let a time interval [0, T], a Banach space Y with the norm \(\Vert \cdot \Vert _Y\) and a Banach space X with the norm \(\Vert \cdot \Vert _X\) be given. Let an initial state \(y_0 \in Y\) and a desired stationary state \(y_d\in Y\) be given. We state the relation between the state and the control in the form
where for \(a \in [0,\infty )\) and \(t > a\) the mapping \(\Phi \) maps \(\{a\} \times Y \times L^2((a,\,t),\, X) \times \{t\}\) to \(C([a,\,t],\, Y)\) with \( \Phi (a,\, y_0, u, t)(a) = y_0\). We assume the semigroup property that for any subinterval \((a_1,\, t_1)\) of \((a,\, t)\) we have
As an example, think of a strongly continuous semigroup of contractions, see [20].
Let a static state \(y^{(\sigma )}\in Y\) corresponding to a static control \(u^{(\sigma )}\in X\) be given, that is the constant function \(y^{(\sigma )}\) in \(C([a,\,t],\, Y)\) and \(u^{(\sigma )}\) as a constant control in \(L^2((a,\,t),\, X)\) satisfy the equation
Let real numbers a and b with \(a<b\) be given. Let \(f_0: {\mathbb {R}} \times X \times Y \rightarrow [0,\,\infty )\) be a continuous function. For states \(y\in L^2((a,b),\, Y)\) and controls in \(u\in L^2((a,b),\, X)\) for any subinterval \((a_1,\, b_1)\) of \((a,\, b)\), we have the inequality
For states \(y\in L^2((a,b),\, Y)\) and controls in \(u\in L^2((a,b),\, X)\) define the objective function
where y is the system state that is generated by the control function u. This type of objective functions has also been considered in [17].
For \(y_0\) and \(y_d\in Y\), we consider a dynamic optimal control problem with the initial condition
and the terminal condition
We consider the state constraint
where F is a nonempty closed subset of Y such that \(y^{(\sigma )}\in F\). We also consider the control constraint
where U is a nonempty closed subset of X such that \(u^{(\sigma )}\in U\).
For real numbers a, \(b>a\), an initial state \(y_0\in Y\) and a terminal state \(y_d\in Y\) consider the parametric optimization problem
subject to (1.1), (1.5), (1.6), (1.7) and (1.8).
Let \(v(a,\,b,\,y_0,\, y_d)\) denote the optimal value for \(P(a,b,y_0, y_d)\).
For \(\epsilon \ge 0\), let \(\hat{y}_\epsilon (a,b,y_0,\, y_d)\) denote an \(\epsilon \)–optimal state and \(\hat{u}_\epsilon (a,b,y_0,\, y_d)\) denote an \(\epsilon \)–optimal control for \(P(a,b,y_0, y_d)\) in the sense that the constraints (1.1), (1.5), (1.6), (1.7) and (1.8) are satisfied and
that is
Note that for any subinterval \((a_1,\, t_1)\) of \((a,\, b)\), assumptions (1.2) and (1.3) imply the inequality
Moreover, due to (1.2), for all \(y_1\in Y\), \(y_2 \in Y\) we have the inequality
This implies the following lemma:
Lemma 1.1
For any subinterval \((a_1,\, t_1)\) of \((a,\, b)\), we have
that is \(( \hat{u}_\epsilon (a,\, b, y_0, y_d),\; \hat{y}_\epsilon (a,\, b, y_0, y_d))|_{(a_1,\, t_1)}\) yield an \(\epsilon \)–optimal control–state pair for the optimal control problem
Proof
We have
Moreover we have
With (1.11) this yields
Thus we have shown (1.12) and Lemma 1.1 is proved. \(\square \)
2 The measure turnpike property with interior decay
In [17] a certain integral turnpike property and a stronger measure turnpike property are studied. In Definition 2 in [17] (see also [3, 22]) the measure turnpike property is defined as follows:
The problem \(P(a,\, b, \, y_0,\, y_d)\) enjoys the measure turnpike property at \((y^{(\sigma )}, \, u^{(\sigma )} ) \in F \times U\) if for every \(\varepsilon >0\) there exists a real number \(\Lambda (\varepsilon )>0\) such that for all \(b>a\) we have the inequality
where \(\mu \) denotes the Lebesgue measure of the set.
In [17], the measure turnpike property is shown under the strict dissipativity assumption. Similarly as in Definition 1 in [17], we define:
The problem \(P(a,\, b, \, y_0,\, y_d)\) is strictly dissipative at \((y^{(\sigma )}, \, u^{(\sigma )} ) \in F \times U\) if \(f_0\) is time-independent and for all \((y, u) \in F \times U\) for the supply rate function
there exists a storage function \(S: F \rightarrow \mathbb {R}\) that is locally bounded and bounded from below and a continuous and strictly increasing function \(\alpha : [0, \infty ) \rightarrow [0, \, \infty )\) with \(\alpha (0)=0\) such that for all \(b>a\) the dissipation inequality holds, that is for any admissible pair \((y(\cdot ), u(\cdot ))\) and for all \(\tau \in [a,\, b]\) we have
Let \(M_S\) denote an upper bound for |S(y)| for \(y\in F\).
Remark 2.1
The relation between strict dissipativity and turnpike properties is discussed in [4], see also [3].
Example 2.2
Let \(\gamma \in (0, 1]\) be given. For
the problem \(P(a,\, b, \, y_0,\, y_d)\) is strictly dissipative at \((y^{(\sigma )}, \, u^{(\sigma )} ) \in F \times U\) with \(M_S=0\) and \(\alpha (z) = \frac{\gamma }{2} |z|^2\).
Example 2.3
In [17], an example for a strictly dissipative optimal control problem with distributed control of the heat equation is given.
The measure turnpike property defined above can be satisfied if there exist real numbers \(M>0\), \(\Upsilon _0>0\) and \(\lambda \in (0,\, 1)\) such that for all b sufficiently large such that
for all \(t \in [ a + \lambda \,(b-a) -M,\; a + \lambda \,(b-a) +M ]\) the inequality
holds. However, such a situation contradicts the intuition about the turnpike phenomenon that in the interior of the time interval close to the point \(a + \lambda \,(b-a)\), the distance of the dynamic optimum to the static optimum becomes very small, which is for example the case if an exponential turnpike property holds.
In order to exclude such a situation, we say that the problem \(P(a,\, b, \, y_0,\, y_d)\) enjoys the measure turnpike property with interior decay at \((y^{(\sigma )}, \, u^{(\sigma )} ) \in F \times U\) if problem \(P(a,\, b, \, y_0,\, y_d)\) enjoys the measure turnpike property at \((y^{(\sigma )}, \, u^{(\sigma )} ) \in F \times U\) and in addition there exist \(C_1>0\) and \(\lambda _1\in (0,\, 1)\) such that for all \(\lambda \in (0, 1)\) and all b sufficiently large we have the inequality
If (2.2) is valid, the measure turnpike property with interior decay cannot hold. This can be seen as follows. If (2.3) holds we have
If inequality (2.2) holds, we have for b sufficiently large
For b sufficiently large, we have
Hence (2.4) contradicts (2.5).
So we see that the measure turnpike property with interior decay provides a more detailed picture of the turnpike phenomenon than the classical measure turnpike.
Remark 2.4
Note that also an exponential turnpike property, where there exist \(C_1>0\) and \(\mu >0\) such that for all \(T>0\) for all \(t\in [0, T]\) we have
implies the measure turnpike property with interior decay.
For finite-dimensional systems, the exponential turnpike property has been studied in [18]. Exponential sensitivity and turnpike analysis for linear quadratic optimal control of general evolution equations have been studied in [6]. The results that we present here allow for general nonlinear infinite-dimensional dynamcis that satisfy (1.1).
3 A general turnpike result
Throughout this section we assume that for all \((a,\, b) \subset (0, T)\) and all \(z_0\), \(z_1\in F\) the problems \(P(a,\, b, \, z_0,\, z_1)\) are strictly dissipative at \((y^{(\sigma )}, \, u^{(\sigma )} ) \in F \times U\). For the subsequent analysis, we replace the objective function in (1.4) by
where we assume that for all \(y\in F\) and \(u\in U\) we have \(\omega (y,u) \ge 0\). Note that this is equivalent to subtracting the number \((b - a) f_0( y^{(\sigma )}, \, u^{(\sigma )} ) \) from the objective function. We show that under the strict dissipativity and an abstract exact controllability assumption that we define in (3.7) below, problem \(P(a,\, b , \, y_0, \, y_d)\) has the measure turnpike property with interior decay. This is stated in Theorem 3.6.
The exact controllability assumption requires that if the control time is greater than or equal to a minimal time \(t_{\min }\), the optimal values of the control problem are uniformly bounded with a bound that depends only on the distance between the initial state and \(y^{(\sigma )}\) and the distance between the terminal state and \(y^{(\sigma )}\).
For our analysis we need the following lemma.
Lemma 3.1
Let \(\epsilon \ge 0\) and a nonempty subinterval \([b_1,\, b_2]\subset [a_1,\, a_2]\) be given, that is we have \(a_1 \le b_1 < b_2 \le a_2\). Then for any \(\epsilon \)-optimal state \( \hat{y}_\epsilon (a_1, a_2,\, y_0,\, y_d)\) of \(P(a_1,\, a_2,\, y_0,\, y_d)\) there exists a number \(t_1 \in (b_1,\, b_2)\) such that
This implies the inequality
Proof
The dissipation inequality (2.1) yields
where the last step follows from the definition (1.9) of an \(\epsilon \)-optimal solution and the definition (3.1) of the objective function. Define \( I_1(b_1,\, b_2) \)
Then we have the inequality
Suppose that for all \(t\in (b_1, b_2)\) we have
Integration yields
With (3.4) this implies
This is a contradiction. \(\square \)
Now we consider the optimal control Problem \(P(0,T,y_0,y_d)\).
Let \(\lambda \in (0, \,1)\) be given. Define
Consider the intervals
Then we have \(I_+^\lambda \subset (0, T/2)\) and \(I_-^\lambda \subset (T/2, \, T)\). For the length of \(I_+^\lambda \) we have \({l}(I_+^\lambda ) = \lambda T/2 \ge \lambda _0 \, T/2\). Moreover \({l}(I_-^\lambda ) = ( 1 -\lambda ) T/2 \ge \lambda _0 \, T/2\).
Let \(\epsilon \ge 0\) be given. Lemma 3.1 implies that there exists a point \(t_1^+ \in I_+^\lambda \) such that
Moreover there exists a point \(t_1^- \in I_-^\lambda \) such that
Now we state our abstract exact controllability assumption:
Assume that there exist a constant \(\mu _0>0\) and a time \(t_{\min }>0\) such that for all initial times \(t_{init}\) and all initial states \(z_{init}\in F\) and for all terminal times \(t_{term}\) and all terminal states \(z_{term}\in F\) if \(t_{term} - t_{init} \ge t_{\min }\) we have the inequality
With (3.5) and (3.6), inequality (3.7) implies that if \(T \ge t_{\min }\) we have
and
Remark 3.2
A sufficient condition for (3.7) is that for \(t_0 \ge t_{\min }\) there exists a control function \(u_1\in L^2(a, \, t_0)\) such that \(u_1(t) \in U\) for \(t \in (a,\, t_0)\) almost everywhere, \(y_1= \Phi (a,\, z_0,\, u_1\, t_0)\) satisfies \(y_1(a)=z_0\), \(y_1(b)=z_d\) and \(y_1(t) \in F\) for \(t \in (a,\, t_0)\) and
This is the case if for \(t >t_{\min }\), we have \(y_1(t) = y^{(\sigma )}\) and \(u_1(t) = u^{(\sigma )} \).
As an example, think of a system that can be steered exactly to \(y^{(\sigma )}\) at the time \(t_{\min }\) with the control function \(u_1|_{(a, t_{\min })}\) and where the state remains equal to \(y^{(\sigma )}=y_d\) if for \(t \ge t_{\min }\) a constant control \(u^{(\sigma )}\) is chosen. (For examples, see Sect. 4).
Since \(t_1^+ \in I_+^\lambda \) and \(t_1^- \in I_-^\lambda \) due to (1.3) we have the inequality
With (1.12) and (3.7) this yields
With the upper bounds for
that we have derived above this yields
where \(M_S\) denotes an upper bound for \(|S(y_\epsilon (0, T,\, y_0,\, y_d)(\cdot ))|\).
Thus we obtain an upper bound for the part of the objective function (3.1) that comes from the subinterval \((\lambda \,T/2 ,\, (1 + \lambda )T/2 )\). Note that the point \(\lambda \, T\) is contained in this interval. More generally, by an inductive procedure where for \(n \in \{ 2,\, 3,\, 4,...\}\) we consider the nested sequence of subintervals
of \([0,\, T]\) that all contain the point \(\lambda \, T\). With the special choice \(\lambda = \frac{1}{2}\) the intervals have the mindpoint T/2. We obtain an upper bound for the part of the objective function that comes from these subinterval. For \(T\rightarrow \infty \), upper bound decays as \(1/T^{n-1}\). This is our main turnpike result that is stated in the following theorem.
Theorem 3.3
Assume that the abstract exact controllability assumption (3.7) holds for all \(t_0 >a\) with \( t_0- a \ge t_{\min }\) and all initial states \(z_0\in Y\) and all terminal states \(z_d \in Y\). Let \(\lambda \in (0, \,1)\) be given. Define \( \lambda _0 = \min \{\lambda , \, (1 - \lambda )\}\). Let a natural number \(n\ge 2\) and \(\epsilon \ge 0\) be given.
For all real numbers T such that
define \(g_2 = \tfrac{4\, \mu _0}{\lambda _0 \, T} \) and for \( k \in \{2,3,4...\}\) let
Then we have the inequality
Remark 3.4
We have \(g_3 = 2^5 \, \left( \tfrac{\mu _0}{\lambda _0 \,T}\right) ^2 + 2^4 \, \tfrac{\mu _0}{\lambda _0\, T}\) and
Note that for all \(k\ge 2\) we have \(g_k\rightarrow 0\) for \(T\rightarrow \infty \).
Remark 3.5
For \(\varepsilon =0\), the assumptions of Theorem 3.3 imply that the measure turnpike property with interior decay defined in Sect. 2 holds. In fact we have the following more concise theorem:
Theorem 3.6
As throughout the section, assume that for all \((a,\, b) \subset (0, T)\) and all \(z_0\), \(z_1\in F\) the problems \(P(a,\, b, \, z_0,\, z_1)\) are strictly dissipative at \((y^{(\sigma )}, \, u^{(\sigma )} ) \in F \times U\) and that for the storage function S there exists a constant \(L_s\) such that for all \(y\in F\) with \(\alpha ( \Vert y - y^{\sigma }\Vert _Y)\) sufficiently small we have
Assume that the abstract exact controllability assumption (3.7) holds for all \(t_0 >a\) with \( t_0- a \ge t_{\min }\) and all initial states \(z_0\in Y\) and all terminal states \(z_d \in Y\).
Then problem \(P(0,\, T, \, y_0,\, y_d)\) enjoys the measure turnpike property with interior decay at \((y^{(\sigma )}, \, u^{(\sigma )} ) \in F \times U\).
Proof
For the proof of Theorem 3.6, we first point out that in the setting of [17] it has been shown that strict dissipativity implies the measure turnpike property. In our framework, the measure turnpike properties follows using the exact controllability to show the uniform boundedness (with respect to T) of the objective function (3.1). Moreover, in the definition of the measure turnpike property with interior decay in (2.3), with a derivation similar as for (3.8), we can choose \(\lambda _1 = \frac{1}{2}\) and
\(\square \)
Remark 3.7
If \(M_S=0\), Theorem 3.3 implies that for a minimizing sequence, that is a sequence of \(\varepsilon _k\)–optimal solutions with \( \lim _{k\rightarrow \infty } \varepsilon _k=0\), the members of the sequence come closer and closer to an exponential turnpike structure.
Remark 3.8
It is also possible to state a variant of Theorem 3.3 for the case that no terminal constraint (1.6) is present. In this case, in the bound on the right–hand side \(\lambda _0\) can be replaced with \(\lambda \). Actually, the proof is easier in this case since as an upper bound for the subintervals that appear in the integral we can always take T.
Proof of Theorem 3.3
For \(n=2\) we have already shown the assertion in the inequality (3.8) with \(I_0\) on the left-hand side. More precisely we have shown that for \(n=2\) there exist
and
such that
Moreover, Lemma 1.1 implies that the restriction of the control–state pair \((\hat{u}_\epsilon (0, T,\, y_0, y_d), \, \hat{y}_\epsilon (0, T,\, y_0,\, y_d))\) to the interval \((t_{n-1}^+, \, t_{n-1}^-)\) is an \(\epsilon \)–optimal control–state pair for the optimal control problem
Now we proceed inductively. Assume that for some \(n\ge 2\) there exist \(t_{n-1}^+ \), \(t_{n-1}^-\) such that (3.10), (3.11) and the chain of inequalities (3.12) hold and \((\hat{u}_\epsilon (0, T,\, y_0,\, y_d),\, \hat{y}_\epsilon (0, T,\, y_0,\, y_d))\) is \(\epsilon \)-optimal for
Due to Lemma 3.1 there exist points
and
such that
Then \(t_n^+ \ge t_{n-1}^+\) and \(t_n^- \le t_{n-1}^-\).
Our assumption on (3.13) and Lemma 1.1 imply that the control–state pair \((\hat{u}_\epsilon (0, T,\, y_0,\, y_d),\, \hat{y}_\epsilon (0, T,\, y_0,\, y_d))\) is \(\epsilon \)-optimal for
Then due to (3.7) and \( \lambda _0 \, T/2^n \ge t_{\min }\) we have
By our induction assumption this implies
This shows the assertion. \(\square \)
Remark 3.9
In order to discuss Theorem 3.3, we consider again the nested intervals \(I_n(\lambda ,\, T)\) defined in (3.9). For all \(n\in \{1,2,3,...\}\) we have \(\lambda T \in I_n(\lambda ,\, T)\) and \(I_{n+1}(\lambda ,\, T) \subset I_n(\lambda ,\, T)\). The intervals \(I_n(\lambda ,\, T)\) have the length \( {l}( I_n(\lambda ,\, T) )= T/2^{n-1}\) and
Theorem 3.3 gives an upper bound for the average values of the integrand in the objective functions on the intervals \(I_n(\lambda ,\, T)\), namely
This means that the average values of the integrand in the objective functions on the intervals \(I_n(\lambda ,\, T)\) decays for \(T\rightarrow \infty \) with the order
For \(\varepsilon >0\), the last term is decisive for the speed of convergence which is only of the order 1/T. For \(\varepsilon =0\), only we obtain faster convergence of the order \(1/T^2\).
If also \(M_S=0\), only the first term remains. If \(\lambda \) is close to zero or close to 1, the convergence to zero in the first term becomes slower than for \(\lambda \) close to 1/2. Hence in our turnpike result we have the typical situation that sufficiently close to the middle T/2 of the time interval \([0,\, T]\) the convergence to zero is faster than at the boundaries.
Example 3.10
From the inequality in Theorem 3.3, for \(n=3\) and \(\lambda = 1/2\), if \(T \ge 8 \, t_{\min }\), for \(\epsilon =0\) we obtain
For \(n=4\) if \(T \ge 16 \, t_{\min }\) we obtain
This illustrates the typical turnpike behavior:
For every natural number \(n\ge 2\) if T is sufficiently large the contribution to the objective function that comes from the interval
is less than a constant that is independent of T multiplied by \(1/T^{n-1}\) plus the term with \(M_S/T\).
4 Examples
In this section we present examples of optimal control problems where Theorem 3.3 is applicable.
Example 4.1
We start with a system similar to the motivating example in [11] that is governed by an ordinary differential equation. Define \(F= (-\infty ,\, 0]\) and \(U=[0,\infty )\). For \(T\ge 1\) we consider the problem
Here the turnpike is zero, that is \(y^{(\sigma )}=0\) and \(u^{(\sigma )}=0\). First we show that the feasible set is nonempty. Define \(t_0=1\) and \(\hat{u}(t) = \mathrm{e} - \mathrm{e}^t\ge 0 \) for \(t\in (0,\, t_0)\) and \(u(t)=0\) for \(t\ge t_0\). Then for \(t\in (0, t_0)\) we have
and for \(t\ge t_0\) we have \(\hat{y}(t) = 0\).
All the feasible controls can be characterized by the moment equation
Hence for all feasible controls \(u\ge 0\) where \(y\le 0\) integration by parts yields
For \(T>1\), due to the \(L^1\)-norm that appears in the objective function, the solution has an extreme turnpike structure where the system is steered to zero in the finite time \(t_0=1\) that is independent of T and remains there for \(t\in (T_0,\, T)\). This can be seen as follows. Let \(u(t) = \hat{u}(t) + \delta (t)\) with \(\int _0^T \delta (\tau ) \, \mathrm{d}\tau =0\) and \(\delta (t) \ge 0\) for \(t\ge t_0\). Then we have
Thus \(\hat{u}\) is the optimal control.
Example 4.2
Let us consider another system that is governed by an ordinary differential equation. Let \(y_0\), \(y_1\) be vectors in \({\mathbb {R}}^n\) and \(f: {\mathbb {R}}^n \rightarrow {\mathbb {R}}^n\), \(g: {\mathbb {R}}^n \rightarrow {\mathbb {R}}^{n\times m}\) be \(C^2\) maps with \(f(0)=0\). Let \(C\in {\mathbb {R}}^{n\times n}\) be regular and define the Hilbert space Y with the norm
In [16], the following problem is considered:
Here the turnpike is zero, that is \(y^{(\sigma )}=0\) and \(u^{(\sigma )}=0\). In this case, the exact controllability assumption (3.7) requires that there is a constant \(\mu _0>0\) and a time \(t_{\min }>0\) such that if \(t_0 -a \ge t_{\min }\) for all \(z_0\), \(z_d\in Y\) we have the inequality
If the system is linear, that is of the form \(y' = A \, y+ B u\), the exact controllability can be checked by Kalman’s rank condition.
Our results show that also with additional constraints, for example with
the solution of \(\mathbf{(OCP)}_T\) has a turnpike structure as described in Theorem 3.3.
Example 4.3
Consider the problem of distributed optimal control of the wave equation. Define \(Q=(0, \, T) \times (0,1)\). Here we have \(Y=H^1_0(0,1) \times L^2(0,1)\), \(X = L^2(0,\, 1)\). Let \(F=Y\) and \(U = X\). Let \(y_d \in F\) and \( u_d \in U\) be given.
Consider the optimal control problem
For an initial state \(y_0 = (y_p,\, y_v) \in Y\) and a control \(u\in L^2((0,\,T), X)\) the map \(\Phi (a, y_0, u, t_0)\) is given by the solution y of the initial boundary value problem on the time interval \((a, \, t_0)\)
Similar as in Theorem 1.1 in [8] it can be shown that \(\Phi \) maps to the function space \(C([a,\, T],\; Y)\). In fact, \(\Phi \) corresponds to a strongly continuous semigroup of contractions, see [20], hence (1.2) holds.
The exact controllability of the system with controls \(u\in L^2(0, \, T),\, Y)\) is shown for example in [14]. In this case we have \(t_{\min }=2\). Inequality (3.7) can be shown using the observability inequality (65) from [14].
Theorem 3.6 shows that the problem enjoys the measure turnpike property with interior decay. Moreover, since Theorem 3.3 is applicable with \(S_M=0\), the solution has a turnpike structure where the contribution of the objective function from time intervals of the form
is of the order \(1/\min \{t^n, (T-t)^n\}\) if T is sufficiently large.
If F is defined as \(F= \{ f \in Y: \, |f|\le M\, \text{ almost } \text{ everywhere } \}\) where \(M>0\) is a given upper bound, we obtain the problem with state constraints from [8]. In this case it is requires more work to verify (3.7), since it requires to solve a problem of constrained exact controllability that respects the state constraints. We expect this property to hold for sufficiently regular initial and terminal states- However, it is out of the scope of this presentation.
The situation is similar if additional control constraints of the form \(|u| \le M_1\) with a given bound \(M_1>0\) are present. Problems of this type with pure homogeneous Neumann boundary conditions are considered in [12]. Here the verification of (3.7) requires to show control-constrained exact controllability. As shown in [12], this requires additional regularity of the initial state and the desired state.
Example 4.4
Now we consider a problem or optimal torque control for an Euler–Bernoulli beam. Let \(y_0 \in H^2(0, 1)\) and \(y_1\in H^1(0, 1)\) be given. We study the following optimal control problem:
We have \(Y= L^2(0,1)\) and \(y^{(\sigma )}=0\), \(u^{(\sigma )}=0\). Note that also the desired terminal state is zero, so in this case both the objective function and the terminal state drive the system to the zero state. In this case, the exact controllability assumption (3.7) requires that there is a constant \(\mu _0>0\) and a time \(t_{\min }>0\) such that if \(t_0 -a \ge t_{\min }\) for all \(z_0\), \(z_d\in Y\) we have the inequality
Note that the Euler–Bernoulli beam is exactly controllable in arbitrarily short times (see [20], Example 11.2.8), so in this case \( t_{\min }>0\) can be chosen arbitrarily small. However, for \( t_{\min }\) decreasing to zero the constant \(\mu _0 \) has to be chosen larger.
5 Conclusion
We have shown a turnpike theorem for a problem of optimal control that is stated in a general framework and allows for state constraints and control constraints. The control–to–state map can be nonlinear. In the optimal control problem, both the initial state and the terminal state are prescribed but the results are also applicable in the case where only initial conditions are prescribed. We have shown that for strictly dissipative systems that are exactly controllable the optimal control problems enjoy the measure turnpike property with interior decay which is a stronger property than the classical measure turnpike property.
The turnpike result shows that regardless of the initial state, for a sufficiently large time horizon T for the optimal controls the contribution of the objective function that comes from a sufficiently small neighborhood of the middle of the time interval decays faster than 1/T. For \(\epsilon \)–optimal state–control pairs a similar estimate holds where in the upper bound an additional constant that is multiplied by \(\epsilon \) appears.
References
Damm T, Grüne L, Stieler M, Worthmann K (2014) An exponential turnpike theorem for dissipative discrete time optimal control problems. SIAM J Control Optim 52:1935–1957
Dorfman R, Samuelson PA, Solow RM (1958) Linear programming and economic analysis. McGraw-Hill, New York
Faulwasser T, Korda M, Jones CN, Bonvin D (2017) On turnpike and dissipativity properties of continuous-time optimal control problems. Automatica 81:297–304
Grüne L, Müller MA (2016) On the relation between strict dissipativity and turnpike properties. Syst Control Lett 90:45–53
Grüne L, Guglielmi R (2018) Turnpike properties and strict dissipativity for discrete time linear quadratic optimal control problems. SIAM J Control Optim 56:1282–1302
Grüne L, Schaller M, Schiela A (2020) Exponential sensitivity and turnpike analysis for linear quadratic optimal control of general evolution equations. Journal of Differential Equations 268:7311–7341
Gugat M, Hante F (2019) On the turnpike phenomenon for optimal boundary control problems with hyperbolic systems. SIAM J Control Optim 57:264–289
Gugat M, Keimer A, Leugering G (2009) Optimal distributed control of the wave equation subject to state constraints. ZAMM 89:420–444
Gugat M, Leugering G, Sklyar G (2005) \(L^p\)-optimal boundary control for the wave equation. SIAM J Control Optim 44:49–74
Gugat M, Trélat E, Zuazua E (2016) Optimal Neumann control for the 1D wave equation: finite horizon, infinite horizon, boundary tracking terms and the turnpike property. Syst Control Lett 90:61–70
Gugat M, Zuazua E (2016) Exact penalization of terminal constraints for optimal control problems. Optim Control Appl Methods 37:1329–1354
Gugat M (2009) Lavrentiev-prox-regularization for optimal control of PDEs with state constraints. Comput Appl Math 28:231–257
Liberzon D (2003) Switching in systems and control. Birkhäuser, Basel
Micu S, Zuazua E (2004) An introduction to the controllability of partial differential equations. In: Sari T (ed) Quelques questions de théorie du contrôle. Collection Travaux en Cours, Hermann, pp 69–157
Porretta A, Zuazua E (2013) Long time versus steady state optimal control. SIAM J Control Optim 51:4242–4273
Sakamoto N, Pighin D, Zuazua E (2019) The turnpike property in nonlinear optimal control—a geometric approach. Preprint
Trélat E, Zhang C (2018) Integral and measure-turnpike properties for infinite-dimensional optimal control systems. Math Control Signals Syst 30:3
Trélat E, Zuazua E (2015) The turnpike property in finite-dimensional nonlinear optimal control. J Differ Equ 258:81–114
Trélat E, Zhang C, Zuazua E (2018) Steady-state and periodic exponential turnpike property for optimal control problems in Hilbert spaces. SIAM J Control Optim 56:1222–1252
Tucsnak M, Weiss G (2009) Observation and control for operator semigroups. Birkhäuser Advanced Texts, Basel
Zaslavski A (2000) Existence and structure of optimal solutions of infinite dimensional control problems. Appl Math Opt 42:291–313
Zaslavski A (2006) Turnpike properties in the calculus of variations and optimal control. Springer, New York
Acknowledgements
This work was supported by the DFG Grant CRC/Transregio 154, project C03 and project C05.
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Gugat, M. On the turnpike property with interior decay for optimal control problems. Math. Control Signals Syst. 33, 237–258 (2021). https://doi.org/10.1007/s00498-021-00280-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00498-021-00280-4
Keywords
- Optimal control
- Boundary control
- Turnpike
- Hyperbolic system
- Exact controllability
- State constraint
- Control constraint