Manifold turnpikes of nonlinear port-Hamiltonian descriptor systems under minimal energy supply

Turnpike phenomena of nonlinear port-Hamiltonian descriptor systems under minimal energy supply are studied. Under assumptions on the smoothness of the system nonlinearities, it is shown that the optimal control problem is dissipative with respect to a manifold. Then, under controllability assumptions, it is shown that the optimal control problem exhibits a manifold turnpike property.


Introduction
This paper is concerned with turnpike phenomena.These phenomena were first noticed in the context of economics [5,17] and have since been observed in many different situations, see, e.g., [3,28,36] and the references therein.Usually, turnpike phenomena are studied in optimal control problems, where the goal is the minimization of a cost functional C(u).
Here, the function u acts as the control of a system of interest.In many cases, it can be observed that an optimal control u * will, for a majority of the time horizon, steer the associated state trajectory x * to a point [23,24,30], a set [25,28,36] or, as in our case, a manifold [7].In other words, optimal solutions depend mainly on the underlying system and the optimization objective and are more or less independent on the choice of the time horizon and other data, such as initial or final values.The behaviour is reminiscent of an observation from daily life: when traveling a long distance by car, it is usually faster to take a detour via a turnpike than to choose a more direct way on slower streets.Also, the chosen turnpike usually does not heavily depend on the start and end of the route.If one would start the journey a few blocks away, then the fastest path would remain more or less the same.
Here, we consider a special class of systems called port-Hamiltonian (pH) systems.Parts of the origins of port-Hamiltonian systems date back to the late 1950s [21], and the interested reader is referred to [31] for an overview on the origins of this system class.Despite their long history, they continue to be the focus of active research [2,10,14,18,19,32].Arguably, the key feature of pH systems is their modeling perspective: they focus on taking energy as the lingua franca between subsystems.As a consequence, the class of pH systems is a promising class for modeling real world processes [18].Benefits of port-Hamiltonian models include inherent stability and passivity, the invariance under Galerkin projection and congruence transformation, and the possibility to interconnect multiple pH systems in a structure-preserving manner.
When pH systems are considered in an optimal control setting, the objective of minimizing the supplied energy is quite natural.This results in a cost term of the form T 0 y(t) T u(t) dt, where y is a collocated observation of the system, and renders the corresponding optimal control problem singular.In [9,26], the authors have considered this objective for linear time invariant port-Hamiltonian (descriptor) systems.They have shown that the optimal control problem has a measure turnpike property with respect to the dissipative part of the state space, given by the kernel of the matrix corresponding to the non-conservative system dynamics.The infinite-dimensional linear case was discussed in [22].In this paper, we are concerned with the finite-dimensional nonlinear case.We show that, under smoothness assumptions on the nonlinearities and controllability assumptions on the system, nonlinear pH descriptor systems admit a turnpike phenomenon with respect to a submanifold of R n .This submanifold corresponds, as in the linear case, to the energy dissipating part of the state space.
The structure of this paper is as follows.In Section 2, we recall the definition of port-Hamiltonian systems and precisely state the optimal control problem that is considered.After that, a short repetition of results on submanifolds of R n follows in Section 3. In Section 4, we define manifold dissipativity and manifold turnpikes following [7] and recall that, under weak assumptions, manifold dissipativity implies a manifold turnpike property.Section 5 contains the main results of this work, where the previously established results are applied to finite-dimensional nonlinear port-Hamiltonian descriptor systems.The theoretical results are then illustrated by a numerical example in Section 6.Finally, in Section 7 a conclusion is drawn and an outlook on future research is given.

Notation
For a set Z ⊆ R n we define Z • as the interior of Z.We denote the Euclidean norm by • and define the distance of a point x ∈ R n to the set M ⊆ R n as dist(x, M) := inf p∈M x − p .We denote the set of all k-times continuously differentiable functions from U to V by C k (U,V ) and define C(U,V ) := C 0 (U,V ).When the spaces U and V are clear from context, we say that , where • T denotes the transpose.The kernel and range of the matrix A are denoted by ker(A) and ran(A), respectively.The non-negative square-root of a positive semidefinite matrix A is denoted by A 1/2 .Often, we surpress the time dependency of functions and write z instead of z(t).

Preliminaries and problem setting
Following the definiton of [18], we consider nonlinear port-Hamiltonian descriptor systems of the form Here, x, u and y are the state, input and output of the system, respectively.We restrict our analysis to pH systems without feedthrough but note that the discussion can be extended to systems with a feedthrough term as introduced in [18].We consider the state space X = R n and a set of admissible controls U ad and require that Further, the functions J and R have to satisfy J(x) = −J(x) T and R(x) = R(x) T 0 for all x ∈ X .We assume that the system (pH) is associated with a Hamiltonian H ∈ C 1 (X , R) which is bounded from below along any solution of (pH) and satisfies for each x ∈ X .Without loss of generality, as in [18] we may assume that H is nonnegative along any solution of (pH).
Remark 1.We only consider pH systems which do not explicitly depend on time.The definition given here can be generalized to include explicit time dependence, but these systems can easily be made autonomous [18,Remark 4.2].
When it comes to the optimal control of port-Hamiltonian systems, the cost functional should take into account that pH systems stem from using energy as the lingua franca.Hence, choosing the supplied energy as the optimization objective is quite natural.For this, usually the impedance supply y T u is considered, which is related to the scattering supply u 2 − y 2 via the Cayley transform [27].We focus on the former and thus consider the optimal control problem min Here and in the following, we assume x 0 , x T ∈ K, where K ⊆ R n is a compact set.Further, as we are interested in the properties of optimal solutions to (pH OCP T ), throughout the paper we assume that an optimal solution u * and a corresponding trajectory x * exist.This assumption is quite restrictive and may be violated.The existence and uniqueness of optimal controls for nonlinear differential-algebraic equations is studied in, e.g., [13].
It can be shown [18] that the power-balance equation holds along any solution x of (pH).This allows us to rewrite the cost functional C pH,T (u) as This equation is called the energy-balance equation, and we can interpret each of the terms physically [20].The term H(x T ) − H(x 0 ) measures the conserved energy, while the integral term corresponds to the dissipated energy.By rearranging and plugging in the definiton of C pH,T , we see that Note that this implies dissipativity in the sense of Willems [34], and as a consequence shows the aforementioned passivity of pH systems.We will use both of these equations in Section 5.

Submanifolds of R n and the orthogonal projection
This section repeats mostly well-known results regarding submanifolds of R n , with a focus on manifolds defined as the zero locus of some smooth function.The main result of the section is Lemma 5, which provides an upper bound for the distance of a point to such a manifold.We begin with recalling the classical defintion of submanifolds of R n .Following [6,15], we distinguish manifolds whose tangent spaces locally satisfy a Lipschitz condition.
• Let M be a subset of R n .We call M an s-dimensional C k manifold if for each p ∈ M there exists an open neighborhood U of p and a C k diffeomorphism φ : The function φ is called a local coordinate system of M at p. • Let M ⊆ R n be an s-dimensional manifold, p ∈ M and let φ : U → R n be a local coordinate system of M at p. We define the tangent space at p relative to M as The space N p M := T p M ⊥ is called the normal space at p relative to M.
ifold and for all p ∈ M there exists a set V ⊆ M that is open relative to M and a positive constant L > 0 such that p ∈ V and for all p ∈ V it holds that Here, d H denotes the Hausdorff distance defined by where S := {z ∈ R n | z = 1} is the unit sphere.
Next, we recall the definition of the orthogonal projection on a manifold from [15].For this, consider a manifold M ⊆ R n and define the set of points with the unique nearest points property as Clearly, for each x ∈ unpp(M) there exists a unique p(x) ∈ M with the property Definition 3 (orthogonal projection on manifold, [15]).Let M ⊆ R n be a manifold.The function p : unpp(M) → M, x → p(x) is called the orthogonal projection on M.
Often, we write p x instead of p(x).The maximal open set on which the orthogonal projection is defined plays a special role in [15] and also in our setting.We will refer to this set as E(M) := unpp(M) • .
The next proposition collects selected results on submanifolds of R n defined in a particular manner and will be useful in Section 5, where it will allow us to study the optimal control problem (pH OCP T ).
is nonempty and assume that there exists an open neighborhood G ⊆ R n of M such that for all x ∈ G it holds that dim(ker(D f x )) = s, where the constant s is independent of x and satisfies 0 < s < n.Then (i) the set M is an s-dimensional C 2 submanifold of R n , (ii) the tangent space at p ∈ M is given by T p M = ker(D f p ), (iii) the manifold M is a C 2,1 manifold, and (iv) it holds that M ⊆ E(M), and if x ∈ E(M) \ M then x − p x ⊥ T p x M. Proof.
(i) Let p ∈ M. It can be shown [33] that there exist open neighborhoods for all y ∈ φ (U 1 ).Thus, for each v ∈ U 1 we have (ii) Suppose that p ∈ M and that φ : we have The claim then follows from the fact that ker(D f p ) has dimension s. (iii) In [6,Equations (3.3) and (3.6)], it is shown that a C k manifold with k ≥ 2 is also a C k,1 manifold, from which the claim follows.(iv) This claim was proven in [15].
We finish this section with Lemma 5 and a corresponding remark.The lemma establishes an upper bound on the distance to the manifold M defined in Proposition 4 in terms of the function f .This result will be the key in our application to port-Hamiltonian systems, as it will allow us to deduce a dissipativity property for (pH OCP T ).
is nonempty and assume that there exists an open neighborhood G ⊆ R n of M such that for all x ∈ G it holds that dim(ker(D f x )) = s, where the constant s is independent of x and satisfies 0 < s < n.Further, assume that for each x ∈ G the smallest nonzero singular value of D f x is bounded from below by c > 0. Then M is a C 2 manifold and there exists an open set V ⊆ R n and a constant c > 0 with M ⊆ V ⊆ E(M) and for all x ∈ V .
Proof.We will first show that (2) is true locally.Fix a point p ∈ M and notice that due to f ∈ C 2 , for all x ∈ R n we have where the remainder g p satisfies To establish (2) locally, our first goal is to show that there exists an open set U p ⊆ E(M) such that for all x ∈ V p := U p ∩ N p M we have For the sake of simplicity, let us set A p := D f T p D f p .Then, by the definition of M and Proposition 4, we have ker p is symmetric positive definite, for any x ∈ N p M we obtain Now, using ( 4) and ( 6), we obtain where we set c p := λ min (A p ) 1/2 > 0. In particular, we have Thus, choosing x ∈ N p M sufficiently close to p we obtain the estimate ( 5), and we can deduce that an open set U p with the sought-after properties has to exist.Now, since (3) and p ∈ M implies To finish the local argument, notice that by (6) we have which together with (7) shows that (2) holds for all x ∈ V p .
To construct the set V , first notice that the differentiability of f implies that an expression of the form (3) is possible on the set E(M).In other words, there exists a function g • (•) such that for all x ∈ E(M).Since the orthogonal projection x → p x is differentiable [15] and f ∈ C 2 , the map x → g p x (x − p x ) is continuous on E(M).Define the function Then h is continuous and hence the preimage of the open set 2 ) ⊆ E(M).Then for each p ∈ M we have The previous arguments can then be used to show that for c := c 2 and finishing the proof.Remark 6.The estimate (2) is related to the Łojasiewicz inequality [12,16], which states that for a real analytic function g : U → R defined on an open set U ⊆ R n and a compact set K ⊆ U , the distance of x ∈ K to the zero locus Z := {z ∈ U | g(z) = 0} of g may be estimated by where α and C are positive constants.

Manifold dissipativity and manifold turnpikes
In this section, we recall the definition of dissipativity with respect to a manifold and the definition of manifold turnpikes as introduced in [7].Further, a theorem relating the two properties is stated.Here and in the following, the set K is defined as α is continuous and strictly increasing}.
We consider the general optimal control problem As before, we assume x 0 , x T ∈ K, where K ⊆ R n is a compact set.Here, the function g defines the dynamics of the system and the function h corresponds to possible algebraic constraints.We refrain from further specification of these functions as (OCP T ) is only used for general definitions.Throughout this section, we assume that an optimal control u * of (OCP T ) and an associated trajectory x * exist.We begin with the definition of manifold dissipativity.The definition is related to Willems' notion of dissipativity [34] and is also found in [7].
Definition 7 (manifold dissipativity).Consider the optimal control problem (OCP T ) together with the manifold M ⊆ R n .We say that (OCP T ) is dissipative with respect to the manifold M if there exists a function S : R n → [0, ∞) that is bounded on compact sets and a function α ∈ K such that all optimal controls u * and associated trajectories x * satisfy the dissipation inequality for all T > 0.
The function S from Definition 7 is also called storage function.Note that we require the dissipation inequality (8) only to hold along optimal solutions of (OCP T ).This is not a severe restriction when turnpike phenomona are studied, as we are only interested in properties of optimal solutions.
Next, we define a manifold turnpike property, again following [7].The property is essentially a notion of measure turnpikes, see, e.g., [3,8,28,36].Definition 8 (manifold turnpike).Consider the optimal control problem (OCP T ) together with the manifold M ⊆ R n .We say that (OCP T ) has the manifold turnpike property with respect to the manifold M if for all compact sets K ⊆ R n and all ε > 0 there exists a constant C K,ε > 0 such that for all T > 0 all optimal trajectories x * of (OCP T ) satisfy for all x 0 , x T ∈ K. Here, λ denotes the Lebesgue-measure.
The next theorem can be found similarly in [7,8,11].The theorem shows that manifold dissipativity implies a manifold turnpike property.
Theorem 9 (manifold dissipativity implies manifold turnpike).Consider the optimal control problem (OCP T ) together with a submanifold M ⊆ R n and assume that (i) there exists a constant C ℓ (K) > 0 such that for all optimal controls u * of (OCP T ) and the associated trajectories x * we have for all T > 0, and (ii) the optimal control problem is dissipative with respect to the manifold M. Then the optimal control problem (OCP T ) has the manifold turnpike property.
Remark 10.In [7], Theorem 9 is stated with a stronger assumption in the place of (i), which can be interpreted as a controllability property.For the sake of simplicity, we do not consider this case.

Application to port-Hamiltonian systems
Finally, we are ready to apply the previous results to port-Hamiltonian systems and the optimal control problem (pH OCP T ).First, recall that we can rewrite the cost functional C pH,T (u) as and that rearranging gives Equation ( 9) hints that any optimal trajectory will have to spend most of the time close to the set and that H can be used as a storage function to derive dissipativity notions with respect to M. Our aim will be to formalize these ideas.The first step will be to ensure that M has the necessary manifold structure.For that, we make the following assumptions.
(A1) The map f : The set M is nonempty and there exists an open neighborhood G ⊆ R n of M such that for all x ∈ G it holds that dim(ker(D f x )) = s, where the constant s is independent of x and satisfies 0 < s < n. (A3) For each x ∈ G, the smallest nonzero singular value of D f x is bounded from below by a positive constant c > 0. (A4) Let V be the open set from Lemma 5. Any optimal trajectory x * of (pH OCP T ) remains in V for all times.
With assumptions (A1) and (A2), Proposition 4 ensures that M is an s-dimensional C 2,1 submanifold of R n and that M ⊆ E(M).
The next step is to show that the problem is dissipative.As we will see shortly, this is ensured by assumptions (A3) and (A4), which allow us to use Lemma 5 to conclude that the optimal control problem (pH OCP T ) is dissipative with respect to the manifold M. Notice that (A4) implies x 0 , x T ∈ V .
Theorem 12 ((pH OCP T ) is dissipative).Under Assumption 11, the optimal control problem (pH OCP T ) is dissipative with respect to the manifold M with storage function H.
Proof.The proof is essentially an application of Lemma 5. Since all assumptions of Lemma 5 are satisfied under Assumption 11, there exists an open set V ⊆ R n and constant c > 0 such that M ⊆ V ⊆ E(M) and holds for all x ∈ V .In particular, assumption (A4) ensures that the estimate holds along any optimal trajectory x * of (pH OCP T ).With this and (10), we see that for any optimal control u * and the associated trajectory x * and output y * , we have where α : s → c 2 s 2 ∈ K. Finally, note that the Hamiltonian H, acting as a storage function here, is bounded on compact sets since it is differentiable.
Remark 13.Let us emphasize that the estimate (2) from Lemma 5 was the key to conclude the dissipativity property of (pH OCP T ).As we have mentioned in Remark 6, the estimate is related to Łojasiewicz' inequality [12,16].In fact, if the map g : x → f (x) 2 is real analytic, then we may use the Łojasiewicz inequality to derive dissipativity without Lemma 5, as long as any optimal trajectory stays in some compact set D ⊆ R n .
Now, an application of Theorem 9 yields the following result, showing that the optimal control problem (pH OCP T ) has the manifold turnpike property with respect to M. Theorem 14 ((pH OCP T ) has a manifold turnpike).In addition to Assumption 11, assume (A5) there exists a control u 1 ∈ U ad that steers the associated trajectory x 1 from x 0 onto the manifold M in time T 1 ≥ 0, and (A6) there exists a control u 2 ∈ U ad that steers the associated trajectory x 2 from the manifold M to x T in time T 2 ≥ 0. Then the optimal control problem (pH OCP T ) has a manifold turnpike at the manifold M.
Proof.Since the port-Hamiltonian system (pH) is autonomous and there is no control cost on the manifold M, the conditions (A5) and (A6) ensure that the total cost of the optimal control u * is bounded by Notice that the constants C H (K), C 1 (K) and C 2 (K) are independent of the final time T .Thus, using the results of Theorem 12, we may apply Theorem 9 to conclude that the optimal control problem (pH OCP T ) has the manifold turnpike property at M.
In Theorem 14, in order to show that a turnpike property holds true, we needed to make the controllability assumptions (A5) and (A6).This is a common pattern in turnpike results, similar controllability assumptions are made in [3,7] and [26].
Remark 15.In [26], the authors considered linear pH systems of the form ẋ = (J − R)Qx + Bu, y = B T Qx, where J = −J T , R = R T 0 and Q = Q T ≻ 0. They have shown that in this case the optimal control problem (pH OCP T ) admits a subspace turnpike property with respect to ker(R 1/2 Q).We can interpret Theorem 14 as a generalization of this result to the nonlinear case.In the linear case, assumption (A1) is immediately satisfied.Further, the set M is the kernel of R 1/2 Q, and if the dimension of ker(R 1/2 Q) is 0 < s < n, then assumptions (A2) and (A3) are also satisfied.Since the distance estimate (11) can be shown to hold true globally [26,Lemma 13], the set V from Lemma 5 is V = R n and thus assumption (A4) is also satisfied.
We finish this section with a simple example illustrating Theorem 14.
Example 16.Consider the functions E, J, R, η and B defined by , which together with the Hamiltonian x T 2 0 0 1 x form the port-Hamiltonian system E(x) ẋ = J(x) − R(x) η(x) + B(x)u, y = B(x) T η(x). (pH-1) For the system (pH-1), the function f reads as where we take x = x 1 x 2 T .Thus, the derivative D f reads as and the subspace ker(D f (x)) is one-dimensional for all x ∈ R 2 .A simple calculation shows that the nonzero singular value of D f (x) is given by The zero locus M of f is given by Hence, assumptions (A1), (A2) and (A3) are satisfied for (pH-1).As M is a linear subspace, the orthogonal projection on M is well defined globally and we have E(M) = R 2 .Further, for the set V from Lemma 5 it holds that for all x ∈ R 2 .Thus, also assumption (A4) is satisfied for (pH-1).Now, for ξ = ξ 1 ξ 2 ξ 3 T ∈ R 3 , let us define the functions E, J, R, η and B by It is easy to see that also the system satisfies assumptions (A1) -(A4).The zero locus of the map f :

Numerical example
As an example, we consider the optimal control problem (pH OCP T ) together with the pH systems (pH-1) and (pH-2) from Example 16.For the implementation, we use the open-source software package CasADi [1].
In order to use CasADi, the optimal control problem (pH OCP T ) is formulated as a minimization problem of the form min w J(w) subject to w lb ≤ w ≤ w ub and G(w) = 0. ( We follow a similar procedure as [1, Section 5.4].In our implementation, w contains the values x(t i ) for the discretization points t i ∈ [0, T ], and the values u(t i ) for the discretization points t i ∈ [0, T ], i = 0.The initial condition and possible control constraints are incorporated in w lb and w ub .The function G is used to enforce the final condition and a continuity condition on x by using an integrator scheme to determine the value x(t i+1 ) given the values x(t i ) and u(t i ).This integrator scheme is also used to calculate the cost J via the quad option in CasADi's integrator function.For the solution of the nonlinear optimization problem (12), Ipopt [35] is used.
In Figure 1, the solution of the optimal control problem (pH OCP T ) with the system (pH-1) under the control constraint −50 ≤ u(t) ≤ 50 is shown.The turnpike behaviour is clearly visible; the first component x 1 of the optimal trajectory x * approaches the manifold M = {x ∈ R 2 | x 1 = 0} very quickly and remains there for the majority of the time horizon.The same observation can be made for larger time horizons, which is not shown in Figure 1.
In Figure 2, the solution of the optimal control problem (pH OCP T ) with the system (pH-2) under the control constraint −200 ≤ u(t) ≤ 200 is shown.Again, the turnpike phenomenon can be observed.

Conclusion
In this paper, we have considered the optimal control of port-Hamiltonian systems under minimal energy supply with fixed initial and final values.We have seen that a map f , corresponding to the energy dissipating portion of the right hand side, and its zero locus M = {x | f (x) = 0}, which corresponds to the dissipative part of the state space, play an important role.It was shown that under smoothness assumptions on f , the set M forms a C 2 submanifold of R n .In particular, using results from [15], we observed that the orthogonal projection onto M is well-defined in an open set E(M).Further, we have shown that under these assumptions the distance of a point x to M can essentially be bounded by f (x) from above.This fact allowed us to deduce that the considered optimal control problem is dissipative with respect to the manifold M. Our main result was a consequence of this dissipativity property.Under additional controllability assumptions, we have seen that the problem has a manifold turnpike property with respect to M. This theoretical observation was confirmed in a simple numerical example.
An open question from a theoretical perspective is the existence of optimal controls of (pH OCP T ).Here, similar to [9,22,26], the particular structure of pH systems should ] and x T = [ 1  1 ] are chosen.The considered time horizon is [0, T ] with final time T = 10, and the control is constrained via −50 ≤ u(t) ≤ 50.We used 100 discretization steps.be exploited.Another open topic is the study of stronger turnpike properties, such as exponential turnpikes [11,29].Applications of the theoretical results to specific port-Hamiltonian systems such as gas networks [4] will be studied in future works.