Multi-objective variational curves

Riemannian cubics in tension are critical points of the linear combination of two objective functionals, namely the squared norms of the velocity and acceleration of a curve on a Riemannian manifold. We view this variational problem of finding a curve as a multi-objective optimization problem and construct the Pareto fronts for some given instances where the manifold is a sphere and where the manifold is a torus. The Pareto front for the curves on the torus turns out to be particularly interesting: the front is disconnected and it reveals two distinct Riemannian cubics with the same boundary data, which is the first known nontrivial instance of this kind. We also discuss some convexity conditions involving the Pareto fronts for curves on general Riemannian manifolds.


Introduction
Variational curves are curves which minimize an objective functional such as the length or some other quantity obtained from its velocity or acceleration, and so on.Very often more than just one of these (competing) functionals are to be minimized simultaneously, giving rise to a multi-objective optimization problem.A typical example is cubic curves in tension, or simply cubics in tension, where a critical point of a linear combination of the squared L 2 -norms of the velocity and the acceleration is found [18,19,21].Although, clearly, two objective functionals are involved here, this problem is not treated as a multi-objective optimization problem in the literature in that it is only posed as a (single-objective) weightedsum scalarization of the two objective functionals.
The aim of a multi-objective optimization problem is to find the set of all trade-off solutions.A trade-off solution, or a Pareto solution, is a solution which cannot be improved any further to make an objective functional value better (smaller), without making some of the other objective functional values worse (larger).Such a solution set is referred to as the Pareto set, or the efficient set, and in the objective value space, the Pareto front, or the efficient front, respectively; see for example [9,16].
It is of great interest to construct the Pareto front for inspection, so that a desirable solution can be selected by the practitioner.The most popular approach in constructing a Pareto front is first to scalarize the problem via parameters, or weights, so as to obtain a continuum of single-objective optimization problems.These parametric problems are then solved to get a solution in the Pareto front for each parameter value [8,9,16].
To be able to construct the whole Pareto front the parametric scalarization is required to be a surjection from the space of weights to the set of Pareto solutions.If the problem is convex, i.e., the objective functionals and the constraint set are convex, then the weightedsum scalarization suffices to furnish a surjection [3].If, on the other hand, the problem is nonconvex, the weighted-sum scalarization is not guaranteed to be surjective anymore.
It is well-known that variational problems can often be cast as optimal control problems; see for example [14].In [12, Theorem 1], it is stated that the so-called Chebyshev scalarization (the weighted max-norm of the objective functionals) is surjective, guaranteeing construction of the whole Pareto front.
In this paper, we consider variational curves with two objective functionals to minimize, motivated by cubics in tension.We state a similar result to that in [12,Theorem 1].We construct the Pareto fronts of certain instances of two Riemannian manifolds, namely a sphere and a torus.To the best knowledge of the authors, this is the first time multi-objective variational curves on Riemannian manifolds are studied.
We observe that, on the torus, the Pareto front generated by using the Chebychev scalarization is disconnected and Riemannian cubics with the same boundary conditions are not unique.Non-uniqueness of Riemannian cubics under given nontrivial boundary conditions as presented in this paper has not been encountered in the literature before.
The paper is organized as follows.In Section 2, we introduce the multi-objective optimization problem motivated by cubics in tension.Then we sketch the scalarization techniques, namely the Chebychev and weighted-sum scalarizations.In Section 3, we present some necessary and sufficient conditions for the epigraph of Pareto fronts to be convex on general Riemannian manifolds.Numerical experiments in Section 4 illustrate the Pareto fronts and Pareto curves on a sphere and torus endowed with the standard Euclidean metric.Finally, we conclude the paper in Section 5 and present some future directions.

Problem Statement and Preliminaries
We are motivated by the problem of finding Riemannian cubics in tension.Let M be a pathconnected Riemannian manifold with a Riemannian metric •, • and a Levi-Civita connection ∇, T M the tangent bundle of M .Given two points x 0 , x 1 and their associated velocities where t 0 and t f are fixed.Riemannian cubics in tension are defined to be critical points of the functional where x ∈ X , τ > 0 is a constant.Equivalently, Riemannian cubics in tension are solutions of the following Euler-Lagrange equations [18,19,21] over X , where R is the Riemannian curvature tensor associated with the Levi-Civita connection We pose the problem of finding a critical point of the above functional in a more general form as a multi-objective optimization problem: Define the vector of objective functionals, F (x) := (F 1 (x), F 2 (x)).The point x * ∈ X is said to be a Pareto minimum if there exists no x ∈ X such that F (x) = F (x * ) and On the other hand, x * ∈ X is said to be a weak Pareto minimum if there exists no x ∈ X such that The set of all vectors of objective functional values at the weak Pareto minima is said to be the Pareto front (or efficient front) of Problem (MOP) in the 2-dimensional objective value space.Note that the coordinates of a point in the Pareto front are simply The Pareto front is usually a curve (connected or disconnected).

Scalarization
For computing a solution of the nonconvex multi-objective problem (MOP), we will consider the following single-objective problem, referred to as scalarization.While this scalarization is often considered for nonconvex multi-objective finite-dimensional optimization problems [16], it has also been considered for infinite-dimensional (optimal control) problems in [12,13].
(MOP w ) min where w 1 and w 2 are referred to as weights.Problem (MOP w ) is referred to as the weighted Chebychev problem (or Chebychev scalarization) 1 .It is required that w 1 , w 2 ≥ 0. For brevity in notation, we set w 1 = w and w 2 = 1 − w with w ∈ [0, 1].
The following theorem is a direct consequence of [11,Corollary 5.35].It can also be concluded by generalizing the proofs given in a finite-dimensional setting in [16,Theorems 3.4.2 and 3.4.5] to the infinite-dimensional setting in a straightforward manner.

Theorem 1
The point x * is a weak Pareto minimum of (MOP) if, and only if, x * is a solution of (MOP w ) for some w 1 , w 2 > 0, w 1 + w 2 = 1.An ideal cost F * i , i = 1, 2, associated with Problem (MOP w ) is the optimal value of the optimal control problem, Remark 1 Since F i (x) for x ∈ X is bounded below, the problem (P i ) is well-defined.Note that for the optimization problem min x∈X F 1 (x) it is customary to specify two endpoints, but problems (P 1 ) and (P 2 ) share the same set of inputs.For the optimization problem min x∈X F 2 (x), we usually specify two endpoints and their associated end-velocities which usually determines a Riemannian cubic curve.The difficulty is that F 1 usually does not have a minimizer when the curves are restricted in this way, which is why we ask for an infimum instead of a minimizer.✷ Let x be a minimizer of the single-objective problem (P i ) above.Then we also define F j := F j (x), for j = i and j = 1, 2.
In general, it is useful to add a positive number to each objective in order to obtain an even spread of the Pareto points approximating the Pareto front-see, for example, [8], for further discussion and geometric illustration.For this purpose, we define next the so-called utopian objective values.
A utopian objective vector β * associated with Problem (MOP w ) consists of the components β * i given as Problem (MOP w ) can be equivalently written as min We note that although the objective functionals in Problem (MOP) are differentiable in their arguments, Problem (MOP w ), or its equivalent above, is still a non-smooth optimization problem, because of the max operator in the objective.However, it is well-known that Problem (MOP w ) can be re-formulated in the following (smooth) form (cf. the formulation in [14]) by introducing a new variable α ≥ 0.
2 ) ≤ α .Problem (SP w ) is referred to as goal attainment method [16], as well as Pascoletti-Serafini scalarization [9].We will solve Problem (SP w ) for the two examples we want to study.
Note that the popular weighted-sum scalarization, given below, fails to generate the nonconvex parts of a Pareto front.
(MOPws) min We will illustrate this with the torus problem in Section 4.2.We note that the above weightedsum scalarization is equivalent to the minimization of (1) (or the Riemannian cubics in tension problem) with τ = w/(1 − w), w ∈ [0, 1).

Essential interval of weights
With the Chebyshev scalarization, it would usually be enough for the weight w to take values over a (smaller) subinterval [w 0 , w f ] ⊂ [0, 1], with w 0 > 0 and w f < 1, for the generation of the whole front.Figure 1 illustrates the geometry to compute the subinterval end-points, w 0 and w f .In the illustration, the points (F * 1 , F 2 ) and (F 1 , F * 2 ) represent the boundary of the Pareto front.The equations of the rays which emanate from the utopia point (β * 1 , β * 2 ) and pass through the boundary points are also shown.By substituting the boundary values of the Pareto curve into the respective equations, and solving each equation for w 0 and w f one simply gets and From the geometry depicted in Figure 1, as also discussed in [12], one can deduce that with every w ∈ [0, w 0 ] the solution of (MOP w ) will yield the same boundary point (F 1 , F * 2 ) on the Pareto front.Likewise with every w ∈ [w f , 1] the same boundary point (F * 1 , F 2 ) is generated.This observation justifies the avoidance of the weights w ∈ [0, w 0 ) ∪ (w f , 1] in order not to keep getting the boundary points of the Pareto front, as otherwise one would end up wasting valuable computational effort and time.

Pareto front
Figure 1: Determination of the essential subinterval of weights

Convexity Conditions of Pareto Fronts
The main purpose of this section is to present some necessary and sufficient conditions for Pareto Fronts resulting from the problem (MOP) to be convex.Throughout this paper, a Pareto Front is called convex when its epigraph is convex in the standard sense.In detail, if the Pareto Front P F can be written as F 2 = P F (F 1 ), then P F is convex if and only if P F (sF 1 + (1 − s) F1 ) ≤ sP F (F 1 ) + (1 − s)P F ( F1 ) for any F 1 , F1 on the front and s ∈ [0, 1] such that sF 1 + (1 − s) F1 is also on the front.
Suppose the optimal curve of the problem (MOP) with respect to some scalarization (such as weighted-sum, Chebychev) is x = x(t, w), where 0 < w < 1 is the weight parameter.Denote F 1 (w) := t f t 0 ẋ(t, w), ẋ(t, w) dt, F 2 (w) := t f t 0 ∇ t ẋ(t, w), ∇ t ẋ(t, w) dt and their derivatives with respect to w by F ′ 1 and F ′ 2 .
Theorem 2 Suppose the Pareto front P F of the problem (MOP) with respect to some weight w can be denoted by F 2 (w) = P F (F 1 (w)) and F 1 , F 2 , P F are all twice-differentiable.Then, P F is convex if and only if Proof.This proof can be seen from the fact that P F is convex if and only if P F ′′ ≥ 0. By straightforward calculations, we have Then substituting the first identity into the second one, we get the relation (4).✷ If F ′ 1 = 0, then we find Now we consider the convexity conditions of Pareto Fronts generated by the problem (MOP) with respect to the weighted-sum scalarization.
Theorem 3 The Pareto front PF of the problem (MOP) generated with respect to the weightedsum scalarization (M OP ws ) is convex if and only if where x satisfies the Euler-Lagrange equation (2) with τ = w 1−w (w = 1) and x ′ is the derivative with respect to the weight w.
Proof.By integration by parts, we have where we have used the Euler-Lagrange equation ( 2) in the second last identity above.Then, condition (4) implies which completes this proof.✷ Frankly speaking, the condition ( 5) is not easy to verify in practice.Partly because it is difficult to solve the the Euler-Lagrange equation (2) to get x(t, w) and partly because evaluating the derivative of x(t, w) with respect to w and its associated integration brings a lot of difficulties.Now we apply Theorem 3 to the case where the Riemannian manifold M is the standard Euclidean space, i.e., the Euler-Lagrange equation (2) can be solved explicitly.
Remark 2 When M is specified as the Euclidean space E n , the Pareto front PF of the problem (MOP) generated with respect to the weighted-sum scalarization (M OP ws ) is convex for the weight w on some interval (w 0 , 1), where 0 < w 0 < 1. Refer to Appendix A for more detailed discussions.
The following example gives a visualisation of the Pareto front of the multi-objective optimisation problem in the Euclidean space and verifies Theorem 3.
we can get the analytical form of cubics in tension in E 3 (see expressions in Appendix A). Figure 2 shows the Pareto front of the problem (MOP) with respect to the scalarization (MOP ws ) and plots the convexity condition (5).

Numerical Experiments
For numerical experiments, we consider variational curves on two two-dimensional manifolds, namely on sphere and torus.To construct an approximation of the Pareto front of a given problem we implement Algorithm 1 in [12].
We employ the scalarize-discretize-then-optimize approach that was previously used in [12].Under this approach, one first scalarizes the multi-objective problem in the infinite-dimensional In the subproblems of the algorithm in [12], a direct discretization of Problem (SP w ) employing the trapezoidal rule is solved by using Knitro, version 13.0.1.Knitro is a popular optimization software; see [4].We use AMPL [10] as an optimization modelling language, which employs Knitro as a solver.We set the Knitro parameters feastol=1e-8 and opttol=1e-8.We also set the number of discretization points to be 2,000 for the sphere and 5,000 for the torus.

Sphere
Let S 2 := {x = (x 1 , x 2 , x 3 ) ∈ E 3 | x = 1} be the unit-sphere endowed with the standard Euclidean metric/norm.With respect to the induced metric, the covariant derivative ∇ t ẋ(t) on S 2 can be written as where we have used the relation of the twice differentiation of the constraint x(t), x(t) = 1.
Then the squared norm of the covariant acceleration on S 2 is given by Now the general multi-objective optimization problem (MOP) on S 2 can be specified as follows, (S) We have Then the smooth version of the Chebyshev scalarization of the sphere problem (S) can simply be stated as where w ∈ [0, 1) is fixed and we have taken β * 1 = β * 2 = 0.In Figure 3, "speed on sphere" is simply ẋ(t) and the "acceleration on sphere" is ẍ(t) 2 − ẋ(t) 4 .
In generating the Pareto solutions and an approximation of the Pareto front, we have solved (S w ) in the subproblems of Algorithm 1 of [12], with 0 ≤ w < 1.
Numerical observations suggest that the solution represented by red is asymptotically given by (F 1 , F 2 ) = (9.88,∞), with F 1 correct to three decimal places, as w → 1 − .For practical (illustration) purposes we depict F 2 < ∞ in Figure 3; otherwise the Pareto curve extends to infinity to the left.
The curves on the sphere appear as if they have different end-velocities, which is not the case except for the red curve.The end-accelerations of the solution coded in red seem to be impulsive, resulting in a jump in the end-speeds.This can be observed as instantaneous rotation of the end-velocities of the curves on the sphere.
The next example illustrates that not only the Pareto front can be disconnected but also can contain two distinct Riemannian cubics.

Torus
Torus T 2 is a geometric object described by with the induced standard Euclidean metric, where the constants a and c are respectively the smaller and greater radii of the torus.
By straightforward calculations, we get the derivatives and the first fundamental form which implies the Riemannian metric g and its inverse g −1 as follows, Further, the non-zero Christoffel symbols are given by Therefore, the covariant derivative ∇ t ẋ(t) = ẍi (t) + Γ i jk ẋj (t) ẋk (t) on the torus T 2 can be denoted by where The problem of interest is the simultaneous minimization of the objective functionals In other words, we are interested in solving the problem (T) Recall that a certain trade-off solution set of Problem (T) is referred to as the Pareto set.
The Chebyshev scalarization of Problem (T) is given by Take an example instance: Let 4 for the Pareto front constructed by using the scalarization (T w ), after weeding out the dominated points found as local minima of (T w ).The essential interval of the weights is found to be [0.593,1).The right-most (boundary) point in the front represents the Riemannian cubic found with w = w 0 = 0.5931 as (F 1 , F 2 ) = (170.143,248.049), indicated with the colour green.As can be seen the Pareto front is disconnected with a large gap.Moreover, the right-hand segment's epigraph is nonconvex, prohibiting the use of the classical weighted sum as scalarization.
Numerical observations suggest that the left-hand part of the Pareto front is asymptotic to (F 1 , F 2 ) = (80.2,∞), correct to three significant figures, as w → 1 − .The related curves are shown in the colour red.Moreover, the Pareto solution (F 1 , F 2 ) = (95.15,457.2), found with w = 0.8259 and indicated with the colour blue, represents another Riemannian cubic.To the best knowledge of the authors this is the first nontrivial example of multiple Riemannian cubics with the same boundary data.
The remaining boundary point is indicated with the colour light blue, computed as (F 1 , F 2 ) = (138.8,456.3) with w = 0.7667.
The weighted-sum scalarization of Problem (T) (or the classical expression for the Riemannian cubics in tension) is given by (Tws) See Figure 5 for the Pareto front constructed by using the scalarization (Tws).When compared with the Pareto front in Figure 4 the classical expression for the Riemannian cubics in tension clearly misses an important segment of compromise/Pareto solutions.
We note that the Euler-Lagrange equations as optimality conditions of Problem (Tws) is provided in Appendix B.

Conclusion
Measurements from variational curves, for example, kinetic energy and squared norm of higher-order derivatives, are seldom minimized at the same time in the literature.Motivated by the problem of finding Riemannian cubics in tension, we have formulated a multi-objective optimisation problem, that is, we have posed the minimization of the vector of total kinetic energy and squared norm of acceleration of a curve on a Riemannian manifold.To obtain a sequence of single-objective optimization problems, we adopted the Chebychev and weightedsum scalarizations in this paper.Some numerical experiments presented on a sphere and a torus illustrated disconnected Pareto fronts and non-uniqueness of Riemannian cubics.To the authors' best knowledge, these are non-trivial observations that may bring the optimization and differential geometry communities' attention.Furthermore, we derived some convexity conditions for the Pareto fronts on general Riemannian manifolds.
Future research may include more theoretical studies on the phenomenon of non-unique Riemannian cubics on a torus and the disconnected Pareto fronts that they belong to.We have investigated the local optimality of the two Riemannian cubics by utilizing the test looking for bounded solutions of certain Riccati equations presented in [15].However, it was not possible for us to obtain bounded Riccati solutions deeming the test inconclusive.Further investigation in search for bounded Riccati solutions might also be a direction interesting to pursue.

A Discussion on Remark 2
Cubics in tension are solutions of the following Euler-Lagrange equation which can be analytically written as where c 1 and c 2 satisfy To simplify notations, we denote by c Then, the integration It is a bit messy to write down the closed forms of the eigenvalues of B 3 , instead, we plot the eigenvalues for straightforward visualization.In Figure 6, we choose T = 1, from which we can see that eigenvalues of B 3 are positive at least on (0.2, 1).Therefore, the convexity condition 5 is verified at least on (0.2, 1).

B Riemannian cubics in tension on torus
In this section, we consider Riemannian cubics in tension on torus using variational methods.To short notations, we denote by p := 2a sin v c+a cos v , q := sin v c a + cos v , r := c + a cos v and