Optimization over the Pareto front of nonconvex multi-objective optimal control problems

Abstract

Simultaneous optimization of multiple objective functions results in a set of trade-off, or Pareto, solutions. Choosing a, in some sense, best solution in this set is in general a challenging task: In the case of three or more objectives the Pareto front is usually difficult to view, if not impossible, and even in the case of just two objectives constructing the whole Pareto front so as to visually inspect it might be very costly. Therefore, optimization over the Pareto (or efficient) set has been an active area of research. Although there is a wealth of literature involving finite dimensional optimization problems in this area, there is a lack of problem formulation and numerical methods for optimal control problems, except for the convex case. In this paper, we formulate the problem of optimizing over the Pareto front of nonconvex constrained and time-delayed optimal control problems as a bi-level optimization problem. Motivated by existing solution differentiability results, we propose an algorithm incorporating (i) the Chebyshev scalarization, (ii) a concept of the essential interval of weights, and (iii) the simple but effective bisection method, for optimal control problems with two objectives. We illustrate the working of the algorithm on two example problems involving an electric circuit and treatment of tuberculosis and discuss future lines of research for new computational methods.

1 Introduction

We consider optimal control problems where one wishes to minimize simultaneously a number of conflicting objective functionals. These problems are referred to as multi-objective optimal control problems and can be expressed in the following concise form:

$$\begin{aligned} \hbox {(P)}\, \min _{(x,u,t_f)\in X}\ \left( \varphi _1(x(t_f),t_f),\ldots ,\varphi _r(x(t_f),t_f)\right) . \end{aligned}$$

The constraint or the feasible set X in Problem (P) involves a system of differential equations (DEs) in the state and control variables $x(\cdot )$ and $u(\cdot )$, respectively, over a time horizon $[0,t_f]$. The set X also typically involves point and path equality and inequality constraints. The DEs and constraints in X might even include time delays in the variables $x(\cdot )$ and $u(\cdot )$. It is worth noting that although each of the objective functionals $\varphi _i(x(t_f),t_f)$, $i = 1,\ldots ,r$, in (P) above constitutes the so-called Mayer form, other forms (Bolza and Lagrange) can easily be converted into this form conveniently. Therefore, the general model in (P) covers a wide range of conflicting objectives; for instance, minimization of the energy, the terminal time, the deviations from a reference state trajectory, or the uncertainty in measurements, to name just a few. We note that Problem (P) is an infinite dimensional optimization problem.

Broadly speaking, the simultaneous or Pareto minimization in Problem (P) is the process of finding a compromise solution, referred to as a Pareto minimum, where the value of any cost cannot be improved (i.e., reduced) further, without making the value of some other cost worse (i.e., higher). One typical example is the case when one wants to minimize simultaneously the fuel consumption of an airplane travelling from one given city to another and the time the airplane takes for this travel: A shorter travel time often requires a higher fuel consumption. The set of all such compromise or trade-off solutions form the Pareto set in the optimization space, and their function values the Pareto front in the value space. Pareto set and Pareto front are also commonly referred to as the efficient set and the efficient front, respectively¹.

The aim of this paper is to study the problem of minimizing an additional (master) objective functional over the Pareto front, for example minimization of the distance of the trade-off solutions to the origin. While a reformulation of the problem will be carried out with an arbitrary number of objective functionals, a bisection algorithm will be proposed to solve the problem for two objective functionals only. This aim will be motivated and elaborated further in what follows.

The authors of this paper have studied in [28] the problem of constructing the Pareto front of Problem (P) involving ODEs and constraints of general form. They discussed and demonstrated that for the nonconvex optimal control problems like the one in Problem (P), it is better to use the so-called weighted Chebyshev-norm scalarization (or just Chebyshev scalarization) to guarantee that the whole Pareto front can be constructed, instead of using the traditional weighted-sum scalarization, i.e., a convex combination of the objective functionals. They discretized the scalarized problem directly and utilized large-scale optimization software (the AMPL–Ipopt suite [23, 47]) to find the Pareto fronts of two constrained optimal control problems as examples, one involving tumour anti-angiogenesis and the other a fed-batch bioreactor, by means of what they called a scalarize–discretize–then–optimize approach. This approach is in contrast with the other existing discretize–scalarize–then–optimize approach (see e.g. [29,30,31, 40]) which scalarizes the discretized problem rather than the original (continuous-time) problem. With the first approach, as opposed to the second, the theory of discretization (see e.g. [17,18,19,20]) can be checked as well as the maximum principle can numerically be verified.

An additional benefit of the Chebyshev scalarization is also reported and illustrated in [28]: One can compute the whole Pareto front by using only those weights of the objective functionals within what they name as the essential subinterval of weights, instead of the whole interval. Having to compute fewer Pareto solutions over a smaller number of grid points in a subinterval is obviously a computational advantage. For further details and an extensive list of references on multi-objective optimal control the reader is referred to [28]. Other relevant studies on the topic in more recent years have appeared in [13, 16].

Apart from certain trivial or special cases, the Pareto front consists of infinitely many solutions to choose from. When a discrete approximation of the front is found the number of solutions to choose from is still relatively large since the approximate front is required to be accurate enough. Making a decision as to which Pareto solution in the front is the most suitable (to the needs of a practitioner) is often very hard for the following reasons.

• In the case of three or more objectives, the Pareto front might be difficult (if not impossible) to view and to carry out a visual inspection (or “eyeballing”) for a decision.

• Even with two objectives, a visual inspection alone may not be enough to choose a desirable solution.

• Construction of a sufficiently accurate representation of the Pareto front might just be too costly a thing to do numerically.

Motivated by these drawbacks, minimization of an additional (single) objective function over the Pareto front has been of great interest to many researchers over the past decades—see, for example, [2, 3, 5, 6, 14, 15, 25, 26, 32, 42, 48]. Despite this rich collection of works, to the knowledge of the authors, it was not before the reference [7] that optimization over the Pareto front was studied and a numerical method proposed for convex multi-objective optimal control problems. In the current paper, we extend the works in [7, 28] to nonconvex multi-objective optimal control problems and propose a numerical method for carrying out optimization over the Pareto front.

We set the optimal control problem as a bi-level optimization problem as in [7]: One has to minimize a master objective functional subject to the minimization of a scalarization of Problem (P). The lower level problem uses the Chebyshev scalarization as in [28], as opposed to the weighted-sum scalarization in [7]. The problems we consider are in much more general form in this paper: We consider nonconvex instead of convex problems compared to [7] and we consider problems with time-delay instead of those without time delay compared to [28]. Just to re-iterate, [28] only proposes a technique to construct the Pareto front, otherwise it does not carry out optimization over the Pareto front.

As the optimization technique over the Pareto front, we propose the simplest possible technique, namely the bisection method to find a root of the derivative of the master objective functional, over the set of weights for the bi-objective problem, which are the parameters of the lower level optimal control problem. Even in this simplest case, it is necessary to obtain derivatives with respect to the weight, for which we employ difference approximations. However, is it guaranteed that these derivatives exist? This question is answered by [33, 34, 37, 38] which studied the differentiability of a solution of a parametric optimal control problem with respect to the parameters. We add a discussion concerning these studies in the paper.

The main algorithm first finds the essential interval of weights over which the first step of the bisection method is taken to find a new subinterval. Then the subsequent steps of the bisection method are carried out until the stopping criterion is met.

The algorithm is illustrated on two challenging numerical examples: the Rayleigh problem, which comes from an electric circuit, and a compartmental optimal control model for tuberculosis. In the first problem there are constraints on the control variables, and the second problem not only has constraints on the two control variables but also time delays on both the control and state variables.

The paper is organized as follows. In Sect. 2, we introduce the multi-objective optimal control problem, discuss scalarization, introduce the problem of optimization over the Pareto front, and elaborate on solution differentiability. In Sect. 3, we first define and explain the essential interval of weights, and then introduce the bisection method for our problem and provide the detailed algorithm. In Sect. 4, we illustrate the algorithm on two example optimal control problems. Finally, in Sect. 5, we provide concluding remarks.

2 Problem statement and preliminaries

2.1 Multi-objective optimal control problem

We consider the following general multi-objective optimal control problem (similar to that in [28] but in slightly more general form here) to underlie our study on minimization over its Pareto front. The ensuing notation and definitions can also be found in [28] but are given here for completeness as well as convenience.

$$\begin{aligned} \hbox {(OCP) }\left\{ \begin{array}{rl} \displaystyle \min _{(x,u,t_f)} &{} \ \ \displaystyle (\varphi _1(x(t_f),t_f),\ldots ,\varphi _r(x(t_f),t_f)) \\ \hbox {subject to} &{} \ \ \dot{x}(t) = f(x(t),u(t),t), \ \ \hbox {for a.e. } t\in [0,t_f], \\ {} &{} \ \ \theta (x(0),x(t_f),t_f) = 0, \\ {} &{} \ \ {\widetilde{\theta }}(x(0),x(t_f),t_f) \le 0, \\ {} &{} \ \ C(x(t),u(t),t) \le 0, \ \ \hbox {for a.e. } t\in [0,t_f], \\ {} &{} \ \ S(x(t),t) \le 0, \ \ \hbox {for all } t\in [0,t_f], \end{array} \right. \end{aligned}$$

where $r\in \{2,3,4,\ldots \}$ is fixed, the state variable $x\in W^{1,\infty }(0,t_f;\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n)$, ${\dot{x}}:= dx/dt$, and the control variable $u\in L^{\infty }(0,t_f;\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^m)$, with $x(t):= (x_1(t),\ldots ,x_n(t))\in \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n$ and $u(t):= (u_1(t),\ldots ,u_m(t))\in \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^m$. The functions $\varphi _i:\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}_+\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}$, $f:\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^m\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}_+\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n$, $\theta :\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}_+\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^{p_1}$, ${\widetilde{\theta }}:\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}_+\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^{p_2}$, $C:\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^m\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}_+\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^{p_3}$, and $S:\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}_+\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^{p_4}$, are continuous in their arguments. In this problem, $t_f$ is either fixed or free. Here, $L^{\infty }(0,t_f;\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^m)$ denotes the space of essentially bounded, measurable functions equipped with the essential supremum norm. Furthermore, $W^{1,\infty }(0,t_f;{\mathbb {R}}^n)$ is the Sobolev space consisting of functions $x:[0,t_f]\rightarrow {\mathbb {R}}^n$ whose first derivatives lie in $L^{\infty }$.

Assume that $\varphi _i(x(t_f),t_f)\ge 0$, for all $i=1,\ldots ,r$. Note that this assumption can easily be met by adding a large enough positive number to each objective functional.

Note that Problem (OCP) is in general a nonsmooth problem, because it does not require differentiability of the objective functionals or the constraints. Moreover, although we have stated Problem (OCP) in very broad terms, it can further be generalized, for example by adding multi-point constraints, partial differential equations, time delays, etc. In other words, although Problem (OCP) is already in a more general form than what one usually encounters in applications, it can be further made look more general.

Of the possible extensions mentioned above, time delays in the state and control variables, for instance, can be incorporated into Problem (OCP) by replacing the ODEs in Problem (OCP) with

$$\begin{aligned}{} & {} \dot{x}(t) = f(x(t),x(t-d_x),u(t),u(t-d_u),t), \ \ \hbox {for a.e. } t\in [0,t_f], \end{aligned}$$

(1a)

$$\begin{aligned}{} & {} x(t) = x_0(t), \ \ \hbox {for all } t\in [-d_x,0), \end{aligned}$$

(1b)

$$\begin{aligned}{} & {} u(t) = u_0(t), \ \ \hbox {for all } t\in [-d_u,0), \end{aligned}$$

(1c)

where $d_x,d_u > 0$ are the time delays in the state and control variables, respectively.

For technical convenience, let $t_f\le t_f^{\max }$, where $t_f^{\max }>0$ is some constant. Next, we define the feasible set, $X\subset W^{1,\infty }(0,t_f;\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n)\times L^\infty (0,t_f;\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^m)\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}_+$, such that

$$\begin{aligned} X&{:}{=}\{(x,u,t_f)\,:\,\dot{x}(t) = f(x(t),x(t-d_x),u(t),u(t-d_u),t)\,, \ \ \hbox {for a.e. } t\in [0,t_f]\,; \\ {}&\quad x(t) = x_0(t)\,,\ \ \hbox {for all } t\in [-d_x,0];\ \ u(t) = u_0(t)\,,\ \ \hbox {for all } t\in [-d_u,0)\,; \\&\quad \theta (x(0),x(t_f),t_f) = 0\,;\ {\widetilde{\theta }}(x(0),x(t_f),t_f) \le 0\,; \\ {}&\quad C(x(t),u(t),t) \le 0\,,\hbox { for a.e. } t\in [0,t_f];\ S(x(t),t) \le 0,\hbox { for all } t\in [0,t_f]\}\,. \end{aligned}$$

Note that, for the case of time delays in the state and control variables, we have included Eqs. (1a)–(1c) instead of the ODEs ${\dot{x}}(t) = f(x(t),u(t),t)$ in the set X.

Define the vector of objective functionals, $\varphi (x(t_f),t_f):= (\varphi _1(x(t_f),t_f),\ldots ,\varphi _r(x(t_f),t_f))$. The triplet $(x^*,u^*,t_f^*)\in X$ is said to be a Pareto minimum if there exists no $(x,u,t_f)\in X$ such that $\varphi (x(t_f),t_f) \ne \varphi (x^*(t_f^*),t_f^*)$ and

$$\begin{aligned} \varphi _i(x(t_f),t_f) \le \varphi _i(x^*(t_f^*),t_f^*),\quad \hbox {for all } i=1,\ldots ,r. \end{aligned}$$

On the other hand, $(x^*,u^*,t_f^*)\in X$ is said to be a weak Pareto minimum if there exists no $(x,u,t_f)\in X$ such that

$$\begin{aligned} \varphi _i(x(t_f),t_f) < \varphi _i(x^*(t_f^*),t_f^*),\quad \hbox {for all } i=1,\ldots ,r. \end{aligned}$$

The set of all the weak Pareto minima is said to be the Pareto set. On the other hand, the set of all vectors of objective functional values at the Pareto and weak Pareto minima is said to be the Pareto front (or the efficient front) of Problem (OCP) in the r-dimensionalobjective value, or outcome, space. Note that the coordinates of a point in the Pareto front are simply $\varphi _i(x^*(t_f^*),t_f^*)$, $i=1,\ldots ,r$. Obviously, when $r=2$ the Pareto front is in general a curve; and when $r=3$ the Pareto front is in general a surface.

2.2 Scalarization

An ideal cost $\varphi ^*_i$, $i=1,\ldots ,r$, associated with Problem (OCP) is the optimal value of the optimal control problem,

$$\begin{aligned} \min _{(x,u,t_f)\in X}\ \varphi _i(x(t_f),t_f). \end{aligned}$$

(2)

Let $({\overline{x}},{\overline{u}},{\overline{t}}_f)$ be a minimizer of the single-objective problem in (2). Then $\varphi ^*_i:= \varphi _i({\overline{x}}({\overline{t}}_f),{\overline{t}}_f)$ and we also define $\overline{\varphi }_j:= \varphi _j({\overline{x}}({\overline{t}}_f),{\overline{t}}_f)$, for $j\ne i$ and $j=1,\ldots ,r$.

In general, it is useful to add a positive number to each objective in order to obtain an even spread of the Pareto points approximating the Pareto front – see for example [21] for further discussion and geometric illustration. To serve this purpose, it is common practice to define the so-called utopian objective values.

A utopian objective vector associated with Problem (OCP) is given as $\beta ^*:= (\beta _1^*,\ldots ,\beta _r^*)$, with $\beta ^*_i:= \varphi _i^* - \eta _i$ and $\eta _i > 0$ for all $i = 1, \ldots , r$. For example choices of the utopian objective vector, see [21, 28].

In [28], to compute a solution of Problem (OCP), the following single-objective problem (P$_w$), i.e., scalarization, was employed.

$$\begin{aligned} ({\hbox {P}}_w)\qquad \min _{(x,u,t_f)\in X}\ \max \{w_1\,(\varphi _1(x(t_f),t_f) - \beta _1^*),\ldots ,w_r\,(\varphi _r(x(t_f),t_f) - \beta _r^*)\}, \end{aligned}$$

where $w_i$, $i=1,\ldots ,r$, are referred to as weights, with the vector of weights w defined as $w:= (w_1,\ldots ,w_r)\in \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^r$, such that $\sum _{i=1}^r w_i = 1$, $w_i \ge 0$, $i = 1,\ldots ,r$. Problem (P$_w$) is referred to as the weighted Chebyshev problem (or Chebyshev scalarization) because of the weighted Chebyshev norm, $\max _i |w_i\,(\varphi _i(x(t_f),t_f) - \beta _i^*)|=\max _i w_i\,(\varphi _i(x(t_f),t_f) - \beta _i^*)$, appearing in the objective. This type of scalarization is typically used for nonconvex multi-objective finite-dimensional optimization problems, as opposed to the weighted sum scalarization which is effective for convex problems but not necessarily the nonconvex ones—see, for example, [39].

Define the set of weights

$$\begin{aligned} Y:= \left\{ w\in \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^r\ |\ \sum _{i=1}^r w_i = 1,\ w_i \ge 0,\ i = 1,\ldots ,r\right\} . \end{aligned}$$

The following theorem was originally presented in [28, Theorem 1] for the case when there was no delay in the state and control variables. It still holds with the set X modified with the delayed state equations. The theorem is a direct consequence of [27, Corollary 5.35]. It can also be concluded by generalizing the proofs given in a finite-dimensional setting in [39, Theorems 3.4.2 and 3.4.5] to the infinite dimensional setting in a straightforward manner.

Theorem 1

(Solutions of scalarization and Pareto minima [28]) The triplet $(x^*,u^*,t_f^*)$ is a weak Pareto minimum of (OCP) if, and only if, $(x^*,u^*,t_f^*)$ is a solution of (P$_w$) for some $w_1,\ldots ,w_r\ge 0$.

Remark 1

Suppose that $Z\subset X$ denotes the Pareto set, namely the set of all Pareto minima of (OCP). By solving (P$_w$) for all $w\in Y$, one can obtain the whole Pareto set Z and in turn get the Pareto front. With numerical computations on the other hand, one would of course carry out some discretization of the weight space Y and typically get a discrete approximation of the Pareto front. $\square$

In the case when the objective functionals and the constraints in Problem (OCP) are differentiable in their arguments, it is worth reformulating Problem (P$_w$) using a standard technique from mathematical programming in the following (smooth) form.

$$\begin{aligned} (\hbox {OCP}_w) \left\{ \begin{array}{rl} \displaystyle \min {\begin{array}{c} \alpha \ge 0 \\ (x,u,t_f)\in X \end{array}} &{} \ \ \alpha \\ \hbox {subject to} &{} \ \ w_1\,(\varphi _1(x(t_f),t_f) - \beta _1^*) \le \alpha , \\ &{}\qquad \qquad \,\vdots \\ &{} \ \ w_r\,(\varphi _r(x(t_f),t_f) - \beta _r^*) \le \alpha . \end{array} \right. \end{aligned}$$

Problem (OCP$_w$) is referred to as Pascoletti-Serafini scalarization [22]. We will solve Problem (OCP$_w$) in an algorithm we present in the next section, for the two examples we want to study.

We re-iterate that the “popular” weighted-sum scalarization, given below, fails to generate the “nonconvex parts” of a Pareto front.

$$\begin{aligned} (\hbox {P}_{ws})\qquad \min _{(x,u,t_f)\in X}\ \sum _{i=1}^{r} w_i\,\varphi _i(x(t_f),t_f). \end{aligned}$$

This deficiency is illustrated with a multi-objective optimal control problem, for example, in the fed-batch bioreactor problem in [28].

2.3 Optimization over the pareto front

The main task in this paper is to devise a numerical algorithm for solving the problem of decision making as to which Pareto point should be chosen. This obviously depends on the criterion a decision maker uses in making his/her choice. As pointed in Remark 1, the whole Pareto front can be parameterized in terms of the vector of weights w. Therefore, Problem (P$_w$), or equivalently (OCP$_w$), can be regarded as a parametric optimal control problem, and it also makes sense to express the decision maker’s objective as the minimization of a function of w.

Before going ahead with the statement of this problem, we re-write the variables of the optimal control problem, with a slight abuse of notation, as $x^w(t):= x(t,w)$, $u^w(t):= u(t,w)$, and $t_f^w:= t_f(w)$ to emphasize their dependence on the vector of weights w.

We call the decision maker’s objective function the master objective function, expressed by $\varphi _0(x^w, u^w, t_f^w)$. With the weight vector w of the scalarization treated now as a variable, the problem of optimization over the Pareto front reduces to the problem of finding an optimal weight $w^*$. Then the corresponding Pareto minimum is a solution of Problem (OCP$_{w^*}$).

The problem of optimizing a master objective function over the Pareto front of (OCP) with $r\ge 2$ objectives is nothing but a bilevel programming problem and can be written as

$$\begin{aligned} \hbox {(OPF) }\left\{ \begin{array}{rll} \displaystyle \min _{w\in Y} &{} \ \ \varphi _0(x^w, u^w, t_f^w) &{} \\ \ \ \hbox {subject to} &{} \ \ \displaystyle \min _{\begin{array}{c} \alpha \ge 0 \\ (x,u,t_f)\in X \end{array}} &{} \alpha \\ {} &{} \ \ \hbox {subject to} &{} \, w_1\,(\varphi _1(x(t_f,w),t_f) - \beta _1^*) \le \alpha , \\ &{} \ \ {} &{} \,\qquad \qquad \vdots \\ &{} \ \ {} &{} \, w_r\,(\varphi _r(x(t_f,w),t_f) - \beta _r^*) \le \alpha . \end{array} \right. \end{aligned}$$

Remark 2

The lower-level problem in (OPF) for some given w is simply Problem (OCP$_w$). A solution of (OCP$_w$) is nothing but a point in the Pareto set Z of (OCP) and is described by the triplet $Z_w:= (x^*(t,w), u^*(t,w), t_f^*(w))$. Then the (whole) Pareto set can be expressed as $Z = \cup _{w\in Y} Z_w$. Now Problem (OPF) can equivalently be written as

$$\begin{aligned} \left\{ \begin{array}{rll} \displaystyle \min _{w\in Y} &{} \ \ \varphi _0(x^w, u^w, t_f^w) &{} \\ \ \ \hbox {subject to} &{} \ \ (x^w, u^w, t_f^w)\in Z. \end{array} \right. \end{aligned}$$

We note that the optimization variable of the upper-level problem is the “unknown” parameter w. If the solution $(x^*(t,w), u^*(t,w), t_f^*(w))$ of Problem (OCP$_w$) is differentiable in the parameter w, then powerful differentiable optimization techniques can be employed in solving Problem (OPF) (or in a more concise form the above problem). This is what was done in [7] for convex multi-objective optimal control problems. Recall that in [7], weighted-sum scalarization was implemented. In this paper, we are extending the work in [7] to the nonconvex setting by also incorporating the Chebyshev scalarization and the concept of essential interval of weights given in [28]. $\square$

2.4 Solution differentiability

We briefly review results on solution differentiability or $C^1$-sensitivity of solutions to the following parametric optimal control problems depending on a parameter $p \in P$, where P is a Banach space:

$$\begin{aligned} (\hbox {OCP}(p)) \left\{ \begin{array}{rl} \min _{\,x,u,p} &{} g(x(t_f),t_f,p) \\ \hbox {subject to} &{} \dot{x}(t) = {\widetilde{f}}(x(t),u(t),p), \ \ \hbox {for a.e. } t\in [0,t_f], \\ {} &{} \psi (x(0),x(t_f),t_f,p) = 0 ,\quad \widetilde{\psi }(x(0),x(t_f),t_f,p) \le 0 , \\ {} &{} {\widetilde{C}}(x(t),u(t),p) \le 0, \ \ \hbox {for a.e. } \; t\in [0,t_f], \\ {} &{} {\widetilde{S}}(x(t),p) \le 0, \ \ \hbox {for a.e. } \; t\in [0,t_f], \end{array} \right. \end{aligned}$$

where the functions $g:\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}_+\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^r\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}$, ${\widetilde{f}}:\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^m\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^r\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n$, $\psi :\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}_+\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^r\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^{p_1}$, ${\widetilde{\psi }}:\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}_+\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^r\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^{p_2}$, ${\widetilde{C}}:\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^m\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^r\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^{p_3}$, and ${\widetilde{S}}:\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^n\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^r\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^{p_4}$, are continuously differentiable in their arguments.

We note that problem (OCP$_w$) is a special case of the parametric problem (OCP(p)) by simply taking the parameter as the weight, $p=w$, which then appears only in the terminal inequality constraints. The problem (OCP($p_0$)) corresponding to a reference parameter $p_0$ is considered as the nominal or unperturbed problem. It is assumed that a local solution $(x_0,u_0)$ of the reference solution exists. Let p be a parameter in a neighbourhood of the nominal parameter $p_0$ and denote the solution to (OCP(p)) by (x(t, p), u(t, p)). Dontchev and Hager [17] gave conditions under which the mapping $p\mapsto (x(\cdot ,p), u(\cdot ,p))$ is Lipschitz. Malanowski and Maurer [33, 34] and Maurer and Pesch [37, 38] investigated the solution differentiability or $C^1$-sensitivity of the optimal solution. The authors derived conditions such that an optimal solution $(x(\cdot ,p),u(\cdot ,p))$ of the perturbed control problem (OCP(p)) exists for all parameters p in a neighborhood of $p_0$ and, moreover, the solution (x(t, p), u(t, p)) is a $C^1$ function with respect to both arguments (t, p). In broad descriptions, these conditions include certain smoothness of the functions in Problem (OCP(p)), satisfaction of the strict Legendre–Clebsch condition, uniqueness of the optimal control minimizing the Hamiltonian, nonsingularity of the Jacobian of an associated boundary-value problem, and boundedness of the symmetric solution of an associated Riccati ODE.

Fixing an increment $d \in P$, the differentials

$$\begin{aligned} z_d(t,p_0) = \frac{\partial x}{\partial p}(t,p_0)d, \quad v_d(t,p_0) = \frac{\partial u}{\partial p}(t,p_0)d, \end{aligned}$$

satisfy a linear boundary value problem that contains only information obtained in the process of computing the unperturbed solution. The computations of these sensitivity differentials can also be performed by discretization methods applied to the parametric optimal control problem; see Büskens [11] and Büskens and Maurer [12]. The sensitivity differentials can be conveniently used in the minimization of a master function defined on the Pareto front; see Sect. 2.3.

The above mentioned conditions for showing solution differentiability exclude optimal control problems with control appearing linearly, since for this class of problems the strict Legendre–Clebsch condition does not hold. Here, optimal controls are combinations of bang–bang and singular arcs. In case of finitely many switching times and junction times with the boundary of a mixed control-state constraint or a pure state constraint, one can set up a finite-dimensional optimization problem, the Induced Optimization Problem, where the switching and junction times are optimized directly; see Maurer et al. [35] and Osmolovskii and Maurer [41]. If second-order sufficient conditions hold for the Induced Optimization Problem (see [41]), one immediately obtains the result that the switching and junction times locally are differentiable functions of the parameter p.

To our knowledge extensions of these results on solution differentiability to optimal control problems with control and state delays can not be found in the literature.

3 An algorithm for optimization over the pareto front

As discussed in Sect. 2.4, the results [37, Theorem 3.1] and [38, Theorem 5.1] lay the ground for devising and implementing numerical methods for solving Problem (OPF). Bonnel and Kaya propose in [7] a barrier method for convex bi-objective optimal control problems with pure control constraints. Their method relies on twice continuous differentiability of the solution (class $C^2$) in the weight w, using the result in [37, Theorem 3.1].

In this paper, we propose a bisection method also for the case of two objectives, which relies on the solution of Problem (OCP$_w$) being of class $C^1$ w.r.t. the weight w, and thus taking the result in [38, Theorem 5.1] as a basis. Although a mathematical justification of the applicability of our proposed method, i.e., solution differentiability, is given only for Problem (OCP(p)), the working of the method will also be illustrated on problems of more general class as in Problem (OCP$_w$).

In the scalarized problem (OCP$_w$) with two objectives ($r = 2$), by choosing $w_1 = w$, and $w_2=1 - w$, where $w\in [0,1]$, one can simply consider the single parameter w.

3.1 Essential interval of weights

With the Chebyshev scalarization, it would usually be enough for the weight w to take values over a (smaller) subinterval $[w_0,w_f]\subset [0,1]$, with $w_0 > 0$ and $w_f<1$, for the generation of the whole front. Figure 1 illustrates the geometry to compute the subinterval end-points, $w_0$ and $w_f$. In the illustration, the points $(\varphi _1^*,\overline{\varphi }_2)$ and $(\overline{\varphi }_1,\varphi _2^*)$ represent the boundary of the Pareto front. The equations of the “rays” which emanate from the utopia point $(\beta _1^*,\beta _2^*)$ and pass through the boundary points are also shown. By substituting the boundary values of the Pareto curve into the respective equations, and solving each equation for $w_0$ and $w_f$ one simply gets

$$\begin{aligned} w_0 = \frac{(\varphi _2^* - \beta _2^*)}{(\overline{\varphi }_1 - \beta _1^*) + (\varphi _2^* - \beta _2^*)} \qquad \hbox {and}\qquad w_f = \frac{(\overline{\varphi }_2 - \beta _2^*)}{(\varphi _1^* - \beta _1^*) + (\overline{\varphi }_2 - \beta _2^*)}. \end{aligned}$$

(3)

We note that for finding $w_0$ and $w_f$ as in (3) it is essential that the single objective problems yielding $\varphi _1^*$ and $\varphi _2^*$ are well-posed. In the case a problem is not well-posed, one might resort to regularizing the objective functional.

Fig. 1

Determination of the essential subinterval of weights $[w_0,w_f]$ [28]

From the geometry depicted in Fig. 1, as also discussed in [28], one can deduce that with every $w\in [0,w_0]$ the solution of (OCP$_w$) will yield the same boundary point $(\overline{\varphi }_1,\varphi _2^*)$ on the Pareto front. Likewise with every $w\in [w_f,1]$ the same boundary point $(\varphi _1^*,\overline{\varphi }_2)$ is generated. This observation justifies the avoidance of the weights $w\in [0,w_0)\cup (w_f,1]$ in order not to keep getting the boundary points of the Pareto front, as otherwise one would end up wasting valuable computational effort and time.

As a result of the above argument, the bisection method, implemented in the algorithm described in the next section, starts with the essential interval $[w_0,w_f]$ rather than [0, 1]. It is worth re-iterating that our main concern here, unlike in [28], is not really to construct the Pareto front, but rather do a search (in this case using the bisection method) over the Pareto front, at the same time avoiding the task of constructing the front, so as to find in some sense the best solution point in the Pareto front.

3.2 Bisection method for solving problem (OPF)

The problem of finding a best point in the Pareto front/set has now been transformed into a problem of finding best w, by virtue of the surjection from the set of weights to the set of Pareto minima furnished by Theorem 1. This has resulted in Problem (OPF) and its concise form: Find some weight $w\in [w_0,w_f]$ such that the master objective function $\varphi _0(x^w, u^w, t_f^w)$ is minimized, where $(x^w, u^w, t_f^w)$ is found by solving (OCP$_w$) for that w. For a simpler setting, it is helpful to define a function $F:[0,1]\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}_+$ representing the function we want to minimize over the Pareto front:

$$\begin{aligned} F(w):= \varphi _0(x^w, u^w, t_f^w), \end{aligned}$$

(4)

such that $(x^w, u^w, t_f^w)$ solves (OCP$_w$). In other words, an evaluation of the function $F(\cdot )$ at w requires the solution of Problem (OCP$_w$) with that w.

Problem (OPF) can now be re-written in an even more concise form as

$$\begin{aligned} \min _{w\in [w_0,w_f]}\ F(w), \end{aligned}$$

(5)

where $F(\cdot )$ is evaluated as in (4). In [7], a log-barrier method is proposed and implemented to solve (5), with an underlying convex and smooth optimal control problem with no state constraints for which the solution can be assumed to be of class $C^2$, and so Newton-like methods are used with heuristic barrier parameter updates. For the general form we have in Problem (OCP), which is nonconvex and has state constraints, we assume that the solution is of class $C^1$. As elaborated in Sect. 2.4, under certain regularity conditions which can in many cases be checked, this assumption is guaranteed to hold. Therefore we apply the bisection method [10] as an effective and simple approach to solving (5) in the case of this paper.

Albeit elementary and standard, a statement of the optimality conditions in the fact below will be useful in formulating a computational algorithm later in this section.

Fact 1

Consider the minimization problem in (5) with $F(\cdot )$ of class $C^1$.

• (a) The interior point $w^*\in (w_0,w_f)$ is a strict local minimizer of $F(\cdot )$ if, and only if,

  $$\begin{aligned} F'(w^*) = 0, \end{aligned}$$

  (6)

  and, for arbitrarily small $\varepsilon >0$,

  $$\begin{aligned} F'(w^* - \varepsilon ) < 0\quad \hbox {and}\quad F'(w^* + \varepsilon ) > 0. \end{aligned}$$

  (7)

• (b) The end point $w_0$ (resp. $w_f$) is a strict local minimizer of $F(\cdot )$ if, and only if, either

  • (i) $F'(w_0) > 0$ (resp. $F'(w_f) < 0$) or

  • (ii) $F'(w_0) = 0$ (resp. $F'(w_f) = 0$) and, for arbitrarily small $\varepsilon >0$, $F'(w_0 + \varepsilon ) > 0$ (resp. $F'(w_f - \varepsilon ) < 0$).

Remark 3

(Three Cases for the End Points of $\varvec{[w_0,w_f)}$) We will apply the bisection method starting with the essential interval $[w_0,w_f]$. Before introducing the pertaining algorithm, we consider below the cases for the end points of this interval.

• Case I. $F'(w_0)F'(w_f) < 0$ : Since $F'(\cdot )$ is assumed to be continuous the bisection method is guaranteed to find a numerical solution to (6) by the intermediate value theorem. Condition (7) needs to be checked to see if $w^*$ is a strict local minimizer.

• Case II. $F'(w_0)F'(w_f) > 0$ : By the conditions in Fact 1(b)(i), at least one of $w_0$ and $w_f$ is a strict local minimizer.

• Case III. $F'(w_0)F'(w_f) = 0$ : If one of the inequalities in Fact 1(b)(ii) is satisfied, then $w_0$ or $w_f$ is a strict local minimizer. It is possible that both $w_0$ and $w_f$ are, or only one or neither is, a local minimizer.

In Case I, the bisection method starts with the interval $[w_0,w_f]$ and terminates with an approximate solution in the interior of the interval. In Case II, a local minimum is found immediately, and so in principle there is no need to do a further search. In Case III, however, the conclusion might be that neither $w_0$ nor $w_f$ is a strict local minimizer, in which case it would be necessary to start the bisection method with a subinterval of $[w_0,w_f]$, and consider Cases I–III again. $\square$

Remark 4

In any of the scenarios elaborated in Remark 3, consideration of another subinterval of $[w_0,w_f]$ might as well yield a better (lower-value) solution, since the problem is nonconvex and we can only hope to get a locally optimal solution. In our approach here, however, we do not endeavour to obtain a global minimum. As a result of our discussion in Remark 3, we will consider only Case I, which clearly prompts us to use the bisection method directly. As suggested above, in the event of Case III not yielding a solution, the new subinterval could be chosen in such a way that one would fall into Case I. $\square$

The derivative of $F(\cdot )$ is defined at the end points of the interval $[w_0,w_f]$ as one-sided limits,

$$\begin{aligned} F'(w_0):= \lim _{\delta \rightarrow 0^+} \frac{F(w_0+\delta ) - F(w_0)}{\delta } \quad \hbox {and}\quad F'(w_f):= \lim _{\delta \rightarrow 0^-} \frac{F(w_f+\delta ) - F(w_f)}{\delta }, \end{aligned}$$

and in the interior, i.e., for $w\in (w_0,w_f)$, as

$$\begin{aligned} F'(w):= \lim _{\delta \rightarrow 0} \frac{F(w+\delta ) - F(w)}{\delta }, \end{aligned}$$

where $F(\cdot )$ is evaluated as in (4). In computations, we will use the forward, and backward, finite difference approximations of $F'(\cdot )$. Namely, for some small $\delta > 0$, we will set

$$\begin{aligned} F'(w) \approx \left\{ \begin{array}{rl} \dfrac{F(w+\delta ) - F(w)}{\delta }, &{}\ \hbox { if } w\in [w_0,w_f-\delta ), \\ \dfrac{F(w) - F(w-\delta )}{\delta }, &{}\ \hbox { if } w\in [w_f-\delta ,w_f]. \end{array} \right. \end{aligned}$$

(8)

The step $\delta$ in the difference approximation formula (8) is small for an accurate estimation of the derivative but not too small in order not to divide one very small number by another and cause numerical instabilities.

In what follows we provide an algorithm to solve Problem (OPF). The algorithm first finds the essential interval $[w_0,w_f]$, computes the signs of $F'(w_0)$ and $F'(w_f)$ and checks the cases I–III in Remark 3, and then if $F'(w_0)F'(w_f) < 0$ it uses the bisection method, to find a numerical solution to Problem (OPF). We note that in Steps 0.4 and k.2 we solve a discretization of Problem (OCP$_w$), which is elaborated further in Remark 5 in the next section. We also note that the given algorithm is for bi-objective problems, as it employs the bisection method. Possible extensions of the algorithm to problems with more than two objectives are discussed in the Conclusion.

Algorithm 1

$\hbox {Step}\,\, {{0.0}}$:

(Initialization) Choose utopia parameters, $\eta _1,\eta _2>0$, a small numerical differentiation step $\delta > 0$, a stopping tolerance $\epsilon > 0$, and a maximum number of iterations $k_{\max }$ . Set $k:=1$.

$\hbox {Step} \,\,{{0.1}}$:

(Boundary points of the front) Solve (2) to get $({\overline{x}}^i,{\overline{u}}^i,{\overline{t}}_f^i)$, $i=1,2$. Set $(\varphi _1^*,\overline{\varphi }_2):= (\varphi _1({\overline{x}}^1({\overline{t}}_f^1),{\overline{t}}_f^1), \varphi _2({\overline{x}}^1({\overline{t}}_f^1),{\overline{t}}_f^1))$, $(\overline{\varphi }_1,\varphi _2^*):= (\varphi _1({\overline{x}}^2({\overline{t}}_f^2),{\overline{t}}_f^2), \varphi _2({\overline{x}}^2({\overline{t}}_f^2),{\overline{t}}_f^2))$ .

$\hbox {Step}\,\, {{0.2}}$:

(Utopia point) Set $\beta ^*:= (\beta _1^*,\beta _2^*)$ with $\beta ^*_i:= \varphi _i^* - \eta _i$, $i=1,2$.

$\hbox {Step}\,\, {{0.3}}$:

(Essential interval) Determine the subinterval $[w_0,w_f]\subset [0,1]$ using (3).

$\hbox {Step} \,\,{{0.4}}$:

(Signs at end points) Compute $F'(w_0)$ and $F'(w_f)$ using (8), with $F(\cdot )$ evaluated as in (4).

• If Fact 1(b)(i) or (ii) is satisfied then $w^* = w_0$ or $w^* = w_f$ accordingly; STOP.

• If $F'(w_0)F'(w_f) = 0$ and neither of the inequalities in Fact 1(b)(ii) is satisfied then declare “Algorithm failed. Change the interval $[w_0,w_f]$.” and STOP.

Let $a:= w_0$ and $b:= w_f$.

$\hbox {Step}\,\, {{k.1}}$:

(Bisection) Find the midpoint $c:= a + (b-a)/2$ of the interval [a, b].

$\hbox {Step}\,\, {{k.2}}$:

(Stopping criterion) Compute $F'(c)$ using (8), with $F(\cdot )$ evaluated as in (4).

• If $F'(c) = 0$ or $(b-a)/2 < \epsilon$ then set $w^* = c$ and STOP.

• If $k = k_{\max }$ then declare “Maximum number of iterations exceeded.” and STOP.

$\hbox {Step} {{k.3}}$:

(New subinterval) Set $k:=k+1$ . If $F'(a)F'(c) > 0$ then update the subinterval as $[a,b]:= [c,b]$; otherwise, set $[a,b]:= [a,c]$. GO TO Step k.1.

4 Numerical examples

In this section, we illustrate the working of Algorithm 1 on two optimal control problems, one involving an electric circuit in Sect. 4.1 and the other a tuberculosis (TB) epidemic in Sect. 4.2.

In computations, we use direct discretization of optimal control problems for which convergence theory has been an active topic of research in the literature (see for example [1, 4, 18,19,20, 43], and see [28] for additional references and discussion).

We employ the scalarize–discretize–then–optimize approach that was previously used in [28]. Under this approach, one first scalarizes the multi-objective problem in the infinite-dimensional space, and then discretizes the scalarized problem directly and applies a usually large-scale finite-dimensional optimization method to find a discrete approximate solution of the scalarized problem. By the existing theory of discretization mentioned above, under certain assumptions, the discrete approximate solution converges to a solution of the continuous-time scalarization of the original problem, yielding a Pareto minimum of the original problem. When possible, we will also check a posteriori to see if the necessary optimality conditions are satisfied by an accurate-enough numerical solution.

Remark 5

In Step 0.4 of Algorithm 1, a direct discretization of Problem (OCP$_w$), for example employing a Runge–Kutta scheme, such as Euler’s method or the Trapezoidal rule, is solved by using Ipopt, version 3.12.13, four times. In Step k.2, Problem (OCP$_w$) is solved in a similar way two times. Ipopt is a popular optimization software based on an interior point method; see [47]. We use AMPL [23] as an optimization modelling language, which employs Ipopt as a solver. $\square$

4.1 Example: tunnel-diode oscillator (rayleigh problem)

The tunnel-diode oscillator problem, also referred to as the Rayleigh problem in the literature, involves dynamics represented by the following differential equations.

$$\begin{aligned}{} & {} {\dot{x}}_1(t) = x_2(t), \\ {}{} & {} {\dot{x}}_2(t) = -x_1(t) + x_2(t)\,(1.4 - 0.14\,x_2^2(t)) + 4\,u(t), \ \ \hbox {for a.e. } t\in [0,t_f], \end{aligned}$$

where the state variable $x_1(t)$ denotes electric current, and the control variable u(t) stands for a suitable transformation of the voltage at a generator, both at time $t\in [0,t_f]$—see [36] for a detailed exposition of the problem. In this particular instance of the problem, the initial and terminal values of the state variables are specified as

$$\begin{aligned} (x_1(0), x_2(0)) = (-5, -5)\quad \hbox {and}\quad (x_1(t_f), x_2(t_f)) = (0,0), \end{aligned}$$

and the dynamics are subject to constraints on the control variable such that

$$\begin{aligned} -1\le u(t)\le 1, \ \ \hbox {for a.e. } t\in [0,t_f]. \end{aligned}$$

The optimal control problem is posed as a bi-objective problem with

$$\begin{aligned} \min \ \left[ t_f,\ \ \int _0^{t_f} \left( x_1^2(t) + u^2(t)\right) dt\right] , \end{aligned}$$

where the competing objectives are the minimization of the final time $t_f$ and the minimization of the sum of the square $L^2$-norms, or in some sense the magnitudes, of the current and the generator voltage. Define a new state variable $x_3$ such that

$$\begin{aligned} {\dot{x}}_3(t) = x_1^2(t) + u^2(t), \ \ \hbox {for a.e. } t\in [0,t_f],\ \ x_3(0) = 0. \end{aligned}$$

Then the two objective functionals as in Problem (OCP), or Problem (OCP$_w$), can be expressed as

$$\begin{aligned} \varphi _1(x(t_f),t_f) = t_f\quad \hbox {and}\quad \varphi _2(x(t_f),t_f) = x_3(t_f). \end{aligned}$$

As we have stated above, the bi-objective Rayleigh problem is in the same form as Problem (OCP) and, in particular, Problem (OCP$_w$). The decision maker’s objective for this problem will be to minimize a weighted distance to the origin of the value space. We choose

$$\begin{aligned} \varphi _0(x^w,u^w,t_f^w):= 100\,\varphi _1^2(x^w(t_f),t_f^w) + \varphi _2^2(x^w(t_f),t_f^w), \end{aligned}$$

where the scaling multiplier 100 is used to make the orders of magnitudes of $\varphi _1$ and $\varphi _2$ the same. We aim to solve Problem (OPF), to determine a scalar $w\in (0,1)$ with $w_1:= w$ and $w_2:= 1-w$ that results in the best Pareto solution in the sense that $\varphi _0(\cdot ,\cdot ,\cdot )$ is minimized, subject to the solution of Problem (OCP$_w$).

In [36], Maurer and Oberle numerically illustrate that an optimal solution does not exist for the single objective problem minimizing the quadratic functional $\varphi _2(x(t_f),t_f)$, in that $t_f$ tends to infinity. They carry out a numerical test for checking the second-order sufficient conditions (SSC) of optimality and show that the test fails to confirm the SSC. Therefore, we will impose a bound on the terminal time, namely set $t_f \le 5$. On the other hand, they illustrate also in [36] that for certain instances of the weighted-sum problem, the SSC of optimality are satisfied.

Problem (OCP$_w$) can now explicitly be written for the Rayleigh problem as

$$\begin{aligned} \left\{ \begin{array}{rll} \displaystyle \min _{\begin{array}{c} \alpha \ge 0 \\ x(\cdot ),u(\cdot ),t_f \end{array}} &{} \ \ \alpha &{} \\ \hbox {subject to} &{} \ \ {\dot{x}}_1(t) = x_2(t), &{} x_1(0) = -5,\ x_1(t_f) = 0, \\ {} &{} \ \ {\dot{x}}_2(t) = -x_1(t) + x_2(t)\,(1.4 - 0.14\,x_2^2(t)) + 4\,u(t), &{} x_2(0) = -5,\ x_2(t_f) = 0, \\ &{} \ \ {\dot{x}}_3(t) = x_1^2(t) + u^2(t), &{} x_3(0) = 0, \\ {} &{} \ \ -1\le u(t)\le 1, \ \ \hbox {for a.e. } t\in [0,t_f],\ \ t_f \le 5, &{} \\ {} &{} \ \ w\,(t_f - \beta _1^*) \le \alpha , &{} \\ {} &{} \ \ (1-w)\,(x_3(t_f) - \beta _2^*) \le \alpha . &{} \end{array} \right. \end{aligned}$$

The Hamiltonian $H:\mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^3\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}\times \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^3\rightarrow \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}$ for this problem simply is

$$\begin{aligned} H(x,u,\lambda ):= \lambda _1 x_2 + \lambda _2\left[ (-x_1 + x_2\,(1.4 - 0.14\,x_2^2) + 4 u\right] + \lambda _3 (x_1^2 + u^2), \end{aligned}$$

where $\lambda (t):= (\lambda _1(t),\lambda _2(t),\lambda _3(t))\in \mathrm{{I} \hspace{-5.39993pt}\textrm{ R}}^3$ is referred to as the adjoint variable vector. Using the convenient notation $H[t]:= H(x(t),u(t),\lambda (t))$, suppose that

$$\begin{aligned}{} & {} {\dot{\lambda }}_1(t):= -\frac{\partial H}{\partial x_1}[t] = \lambda _2(t) - 2\lambda _3(t) x_1(t), \end{aligned}$$

(9a)

$$\begin{aligned}{} & {} {\dot{\lambda }}_2(t):= -\frac{\partial H}{\partial x_2}[t] = -\lambda _1(t) - \lambda _2(t) (1.4 - 0.42 x_2^2(t)), \end{aligned}$$

(9b)

$$\begin{aligned}{} & {} {\dot{\lambda }}_3(t):= -\frac{\partial H}{\partial x_3}[t] = 0, \end{aligned}$$

(9c)

for all $t\in [0,t_f]$, with certain transversality conditions as required by the maximum principle. We will not go into the details of these (boundary) conditions here. However we note that $\lambda _3(t) = \overline{\lambda }_3$, a constant, for all $t\in [0,t_f]$. Then the maximum principle states that if $(x,u,t_f)$ is an optimal solution triplet then there exists a continuous function $\lambda (\cdot )$ satisfying (9a)–(9c), along with certain transversality conditions, such that $\lambda (t) \ne 0$, for all $t\in [0,t_f]$, and

$$\begin{aligned} u(t) = {{\,\textrm{argmin}\,}}_{v\in [-1,1]} H(x(t),v,\lambda (t)) = {{\,\textrm{argmin}\,}}_{v\in [-1,1]} \left( 4\lambda _2(t) v + \lambda _3(t) v^2\right) ,\nonumber \\ \end{aligned}$$

(10)

for a.e. $t\in [0,t_f]$. If $w = 1$, then the problem is a single-objective one, referred to as a time-optimal control problem, and the condition (10) reduces to

$$\begin{aligned} u(t) = {{\,\textrm{argmin}\,}}_{v\in [-1,1]} \lambda _2(t) v, \end{aligned}$$

resulting in

$$\begin{aligned} u^w(t) = \left\{ \begin{array}{rl} 1, &{}\ \ \hbox {if}\\ \lambda ^w_2(t) < 0, \\ -1, &{}\ \ \hbox {if}\\ \lambda ^w_2(t) > 0, \\ \hbox {undetermined}, &{}\ \ \hbox {if}\\ \lambda ^w_2(t) = 0, \end{array} \right. \end{aligned}$$

(11)

for a.e. $t\in [0,t_f]$. By the discussion given in Sect. 3.1 (also see [28]), $u^w(t)$ given in (11) is the same for all $w\in [w_f, 1]$. Recall that if one does not have $\lambda ^w_2(t) = 0$ for all $[t',t'']\subset [0,t_f]$, where $t' < t''$, then $u^w(t)$ in (11) is referred to as optimal control of bang–bang type. We assume (and therefore will numerically double-check) that the optimal control for the particular instance of the problem is of bang–bang type.

The optimality condition (10) can be shown to yield, for any given $w\in [w_0,w_f)$,

$$\begin{aligned} u^w(t) = \left\{ \begin{array}{rl} 1, &{}\ \ \hbox {if}\\ 2\lambda ^w_2(t) < -\overline{\lambda }^w_3, \\ -2\lambda ^w_2(t)/\overline{\lambda }^w_3, &{}\ \ \hbox {if}\\ -\overline{\lambda }^w_3 \le 2\lambda ^w_2(t) \le \overline{\lambda }^w_3, \\ -1, &{}\ \ \hbox {if}\\ 2\lambda ^w_2(t) > \overline{\lambda }^w_3, \\ \end{array} \right. \end{aligned}$$

(12)

for all $t\in [0,t_f]$, provided $\overline{\lambda }^w_3 \ne 0$. Again by virtue of the discussion in Sect. 3.1, $u^w(t)$ in (12) is the same for all $w\in [0,w_0]$. We define the switching function as

$$\begin{aligned} \sigma ^w(t):= \left\{ \begin{array}{rl} 2\,\lambda ^w_2(t)/\overline{\lambda }^w_3, &{}\ \ \hbox {if}\\ 0\le w < w_f, \\ 16\,\lambda ^w_2(t), &{}\ \ \hbox {if}\\ w_f\le w \le 1. \end{array} \right. \end{aligned}$$

(13)

The constant coefficients 2 and 16 above are used for scaling purposes, so that the graphs in Fig. 2b can be viewed more easily. Now, using (13), we can summarize and combine the expressions for the optimal control in (11) and (12) as follows.

$$\begin{aligned} u^w(t) = \left\{ \begin{array}{rl} \left\{ \begin{array}{rll} 1, &{}\ \ \hbox {if} &{} \sigma ^w(t)< -1\, \\ -2\lambda ^w_2(t)/\overline{\lambda }^w_3, &{} \ \ \hbox {if} &{} -1 \le \sigma ^w(t) \le 1, \\ -1, &{}\ \ \hbox {if} &{} \sigma ^w(t)> 1. \\ \end{array} \right\} , &{}\ \ \hbox {if}\quad 0\le w< w_f, \\ \left\{ \begin{array}{rll} 1, &{}\ \ \hbox {if} &{}\quad \sigma ^w(t) < 0, \\ -1, &{}\ \ \hbox {if} &{}\quad \sigma ^w(t) > 0. \\ \end{array} \right\} ,&\ \ \hbox {if}\quad w_f\le w \le 1. \end{array} \right. \end{aligned}$$

(14)

As to why $\sigma ^w(\cdot )$ is referred to as the switching function should now be more clear from (14): the value of $\sigma ^w(\cdot )$ determines when to switch from one case of the control function $u^w(\cdot )$ to another.

For Problem (OCP$_w$) written for the Rayleigh problem above, we have chosen the utopia vector as $(\beta _1^*, \beta _2^*) = (0, 0)$, since $\varphi _i(x(t_f),t_f) > 0$, for $i=1,2$. Figure 2a depicts the Pareto front (obtained by the algorithm in [28]) for the instance of the multi-objective Rayleigh problem we consider here. It also displays the iterations of Algorithm 1. The Rayleigh problem is discretized using the trapezoidal rule, the number of grid points is set to be $N = 5000$, and the Ipopt’s tolerance to $10^{-10}$, so as to get solutions for w accurate at least up to four decimal places (dp).

The essential interval is found to be $[w_0,w_f] = [0.8994, 0.9269]$, with

$$\begin{aligned} (\varphi _1^{w_0},\varphi _2^{w_0}) = (5.000, 44.71)\quad \hbox {and}\quad (\varphi _1^{w_f},\varphi _2^{w_f}) = (3.668, 46.50), \end{aligned}$$

correct to four significant digits, where $\varphi _i^{w}:= \varphi _i(x^{w}(t_f),t_f^{w})$, $i = 1, 2$, with $w = w_0$ or $w_f$, or as will be the case below, $w = w^*$. Optimization over the Pareto front results in $w^* = 0.9247$, after 14 iterations of Algorithm 1, yielding

$$\begin{aligned} \varphi _0^{w^*} = 58.71 \quad \hbox {and}\quad (\varphi _1^{w^*},\varphi _2^{w^*}) = (3.709,45.51). \end{aligned}$$

If there is a need to save the computational resources further, the algorithm can be asked to yield a less accurate result, say correct to three dp, which then yields $w^* = 0.925$ in eight iterations with $(\varphi _1^{w^*},\varphi _2^{w^*}) = (3.71,45.5)$. In Fig. 2a only five iterations are displayed (labels 1–5 appearing to the right of each iteration) for clarity in viewing. The Pareto (master) solution with $w = w^*$ is represented by a square.

Fig. 2

Rayleigh problem—Boundary Pareto solutions, corresponding to $w_0 = 0.8994$ and $w_f = 0.9269$, are shown with (blue) solid curves and (green) dashed curves, respectively. Master Pareto solution, corresponding to $w^* = 0.9247$, is shown with dashed-and-dotted (red) curves

The numerical Pareto-optimal state and control variable solutions are presented in Figs. 2(c)–(d) for $w = w_0, w^*, w_f$. One of the boundary Pareto-optimal solutions is shown using solid (blue) curves for $w = w_0$, which is the same solution for all $w\in [0,w_0]$, as previously discussed in Sect. 3.1. On the other hand, the other boundary Pareto-optimal solution for $w = w_f$, which holds for all $w\in [w_f,1]$, is shown using dashed (green) curves. The latter is nothing but a time-optimal control solution for the Rayleigh problem (a solution with the smallest $t_f$), resulting in a bang–bang type function with the sequence of values $\{1,-1,1\}$, namely with two switchings. The master Pareto solution is given for $w = w^*$ using dashed-and-dotted (red) curves.

The switching function $\sigma ^w(\cdot )$ that is plotted in Fig. 2b by using (13) (recall that discrete approximations of $\lambda ^w_2(t)$ and $\lambda ^w_3(t)$ can readily be obtained from AMPL) furnishes the means to verify the optimality condition for $u^w(\cdot )$ expressed in (14). It is evident from the dashed (green) plot of the switching function that, for $w\in [w_f,1]$, when $\sigma ^w(\cdot )$ crosses the time axis there is a jump (from 1 to $-1$ or vice versa) in the value of the corresponding $u^w(\cdot )$ plot. Likewise, for $w\in [0,w_0]$ and for $w = w^*\in [w_0,w_f)$, whenever $\sigma ^w(\cdot )$ crosses one of the lines $\sigma ^w(t) = 1$ and $\sigma ^w(t) = -1$ (shown by two black lines in Fig. 2b for convenience) the expression for the control function $u^w(\cdot )$ switches from one case in (14) to another, as required.

Remark 6

Remark 3 establishes convergence of the bisection method under the assumption that the function $F(\cdot )$ is of class $C^1$. We can verify this assumption for the case of the Rayleigh problem. Namely, for all weights $w_0 \le w \le w_f$ the SSC in [36] are satisfied: the strict Legendre–Clebsch condition holds and the associated Riccati equation has a bounded solution. This in turn implies two important facts:

• The Euler discretized solution converges to the true solution for stepsize $h \rightarrow 0$.

• Solution differentiability holds which implies that $F(\cdot )$ is of class $C^1$ over $w_0 \le w \le w_f$. $\square$

4.2 Example: compartmental model for tuberculosis

In 2020 and 2021, tuberculosis (TB) was the second leading cause of death from an infectious disease worldwide after COVID-19 [45]. Active TB refers to disease that occurs in someone infected with Mycobacterium tuberculosis. It is characterized by signs or symptoms of active disease, or both, and is distinct from latent tuberculosis infection, which occurs without signs or symptoms of active disease. Only individuals with active TB can transmit the infection. Many people with active TB do not experience typical TB symptoms in the early stages of the disease. These individuals are unlikely to seek care early, and may not be properly diagnosed when seeking care. Delays to diagnosis of active TB present a major obstacle to the control of a TB epidemic, it may worsen the disease, increase the risk of death and enhance tuberculosis transmission to the community. Both patient and the health system may be responsible for the diagnosis delay.

We study the control model with control and state delays presented in Silva et al. [44]. In this model, reinfection and post-exposure interventions for tuberculosis are considered. The population is divided into five categories (compartments) (i.e., the control system has five state variables):

S::

susceptible individuals,

$L_1$::

early latent individuals, recently infected (less than two years),

I::

infectious individuals, who have active TB,

$L_2$::

persistent latent individuals,

R::

recovered individuals,

N::

total population $N = S + L_1 + I + L_2 + R$, assumed constant.

  The model has two control variables and three delays:

$u_1$::

effort on early detection and treatment of recently infected individuals $L_1$,

$d_{u_1}$::

delay on the diagnosis of latent TB, and commencement of latent TB treatment,

$u_2$::

chemotherapy or post-exposure vaccine to persistent latent individuals $L_2$,

$d_{u_2}$::

delay in the prophylactic treatment of persistent latent $L_2$,

$d_I$::

delay in I, i.e., delay in diagnosis.

The dynamical system is given by

$$\begin{aligned} \left\{ \begin{aligned} {\dot{S}}(t)&= \mu N - \frac{\beta }{N} I(t) S(t) - \mu S(t), \\ {\dot{L}}_1(t)&= \frac{\beta }{N} I(t)\left( S(t) + \sigma L_2(t) + \sigma _R R(t)\right) - \left( \delta + \tau _1 + \epsilon _1 u_1(t-d_{u_1}) + \mu \right) L_1(t), \\ {\dot{I}}(t)&= \phi \,\delta \, L_1(t) + \omega L_2(t) + \omega _R R(t) - \tau _0 I(t-d_I)\ + \mu I(t), \\ {\dot{L}}_2 (t)&= (1 - \phi ) \delta L_1(t) - \sigma \frac{\beta }{N} I(t) L_2(t) - (\omega + \epsilon _2 u_2(t-d_{u_2}) + \tau _2 + \mu )L_2(t). \end{aligned} \right. \end{aligned}$$

(15)

The recovered population is defined by

$$\begin{aligned} R(t):= N - S(t) - L_1(t) - I(t) - L_2(t), \end{aligned}$$

(16)

with $N=30000$. The system and delay parameters in the model (15) along with their values are listed in Table 1. In view of the delays the initial conditions and functions are:

$$\begin{aligned} \begin{array}{l} S(0) = 76\,N/120, \ L_1(0) = 36\,N/120, \ L_2(0) = 2\,N/120, \ R(0) = N /120, \\ I(t) = 5\,N/120 \quad \hbox {for} \ -\!d_I \le t \le 0, \quad u_k(t) = 0 \quad \hbox {for} \ -\!d_{u_k} \le t < 0, \quad (k=1,2). \end{array} \end{aligned}$$

(17)

Table 1 Parameter values for the TB control model

The control constraints are given by

$$\begin{aligned} 0 \le u_k(t) \le 1, \quad \forall t\in [0, t_f] ,\quad (k=1,2). \end{aligned}$$

(18)

We consider the following parametric objective functional with control weights $a_1, a_2 \ge 0$:

$$\begin{aligned} \int \limits _0^{t_f} (I(t) + L_2(t) + a_1 u_1(t) + a_2 u_2(t)) \, dt . \end{aligned}$$

(19)

Depending on the priorities, the weights $a_1, a_2$ can be chosen in different ways (for example, both can be chosen to be very small or very large) giving rise to competing objectives. Namely,

$$\begin{aligned} \begin{array}{l} x_5(t_f):= \displaystyle \int \limits _0^{t_f} \Big (I(t) + L_2(t) + a_{11} \,u_1(t) + a_{12} \,u_2(t)\Big )\, dt\,, \\ x_6(t_f):= \displaystyle \int \limits _0^{t_f} \Big (I(t) + L_2(t) + a_{21} \,u_1(t) + a_{22} \,u_2(t)\Big )\, dt. \end{array} \end{aligned}$$

(20)

with control weights $a_{11}, a_{12}, a_{21}, a_{22} \ge 0$, constitute two competing objective functionals. Both functionals are given in Lagrange form. The standard method to obtain an optimal control problem of Bolza type is to introduce additional state variables $x_5$ and $x_6$ defined by

$$\begin{aligned} \begin{array}{l} \dot{x}_5(t) = I(t) + L_2(t) + a_{11}\, u_1(t) + a_{12}\, u_2(t),\ \ x_5(0) = 0, \\ \dot{x}_6(t) = I(t) + L_2(t) + a_{21}\, u_1(t) + a_{22}\, u_2(t),\ \ x_6(0) = 0. \end{array} \end{aligned}$$

(21)

Denoting the (augmented) state vector by $x(t) = (S(t),L_1(t),I(t),L_2(t),x_5(t),x_6(t)) \in {\mathbb {R}}^6$ and the control vector $u(t):= (u_1(t),u_2(t)) \in {\mathbb {R}}^2$, the two competing objectives in the general problem (P) are given by

$$\begin{aligned} \varphi _1 (x(t_f),t_f) = x_5(t_f) =: F_1(x,u)\quad \hbox {and}\quad \varphi _2 (x(t_f),t_f) = x_6(t_f) =: F_2(x,u), \end{aligned}$$

where $F_1(x,u)$ and $F_2(x,u)$ denote the two functionals in Lagrange form.

The bi-objective TB problem is now in the same form as Problem (OCP). The decision maker’s objective for this problem will be to minimize the distance to the origin of the value space. We therefore choose

$$\begin{aligned} \varphi _0\left( x^w,u^w,t_f^w\right) := \varphi _1^2\left( x^w(t_f),t_f^w\right) +\varphi _2^2\left( x^w(t_f),t_f^w\right) , \end{aligned}$$

Our aim is to solve Problem (OPF), to determine a scalar $w\in (0,1)$ with $w_1:= w$ and $w_2:= 1-w$ that results in the best Pareto solution in the sense that $\varphi _0(\cdot ,\cdot ,\cdot )$ is minimized, subject to the solution of Problem (OCP$_w$).

Next we focus on the solution of Problem (OCP$_w$): We aim to find a pair of functions $(x,u) \in W^{1,\infty }([0,t_f],{\mathbb {R}}^6) \times L^\infty ([0,t_f],{\mathbb {R}}^2)$ that minimizes the parameter $\alpha$ subject to the time-delayed dynamics (15) and the auxiliary dynamics (21), initial conditions (17), control constraints (18) and auxiliary weighted inequalities involving $\varphi _1$ and $\varphi _2$.

We consider the necessary optimality conditions for the time-delayed optimal control problem (OCP$_w$); see Göllmann and Maurer [24], Vinter [46]. For this purpose we introduce the delayed state variable $y_3(t) = x_3(t-d_I) = I(t-d_I)$ and delayed control variables $v_k(t) = u_k(t-d_{u_k})$, $k=1,2$. Denoting the adjoint variable vector by $\lambda (t):= (\lambda _S(t),\lambda _{L_1}(t),\lambda _I(t),\lambda _{L_2}(t),\lambda _5(t),\lambda _6(t)) \in {\mathbb {R}}^6$ the Hamiltonian or Pontryagin function is given by

$$\begin{aligned} \begin{array}{rl} H(x,y_3,\lambda ,u_1,v_1,u_2,v_2) = &{} \lambda _S\, (\mu N - \frac{\beta }{N} I S - \mu S) \\ &{}\ \ \, +\ \lambda _{L_1}\, (\frac{\beta }{N} I\left( S + \sigma L_2 + \sigma _R R\right) \\ &{}\ \ \,- \left( \delta + \tau _1 + \epsilon _1 v_1 + \mu \right) L_1) \\ {} &{}\ \ \, +\ \lambda _I\, (\,\phi \,\delta L_1 + \omega L_2 + \omega _R\, R - \tau _0 y_3 + \mu I ) \\ {} &{}\ \ \, +\ \lambda _{L_2}\, ((1 - \phi ) \delta L_1 - \sigma \frac{\beta }{N} I L_2 \\ {} &{}\ \ \,- (\omega + \epsilon _2 v_2 + \tau _2 + \mu )L_2 ) \\ {} &{}\ \ \, +\ \lambda _5\,(I + L_2 + a_{11} u_1 + a_{12} u_2) \\ {} &{}\ \ \, +\ \lambda _6\,(I + L_2 + a_{21} u_1 + a_{22} u_2), \end{array} \end{aligned}$$

(22)

where R is given as in (16). The Minimum Principle [24, 46] yields the adjoint equations

$$\begin{aligned} {\dot{\lambda }}_S(t) = -\frac{\partial H}{\partial S}[t], \quad {\dot{\lambda }}_{L_1}(t) = -\frac{\partial H}{\partial L_1}[t], \quad {\dot{\lambda }}_{L_2}(t) = -\frac{\partial H}{\partial L_2}[t], \\ {\dot{\lambda }}_{x_5}(t) = -\frac{\partial H}{\partial x_5}[t] = 0, \quad {\dot{\lambda }}_{x_6}(t) = -\frac{\partial H}{\partial x_6}[t] = 0, \end{aligned}$$

and the advanced adjoint equation

$$\begin{aligned} {\dot{\lambda }}_I(t) = -\frac{\partial H}{\partial I}[t] - \chi _{[0,t_f-d_I]}(t) \frac{\partial H}{\partial I}[t+d_I], \end{aligned}$$

where the argument [t] stands for evaluating all arguments at time t. We note that $\lambda _5^w(t) = \overline{\lambda }_5^w$ and $\lambda _6^w(t) = \overline{\lambda }_5^w$, constants, for any fixed $w\in [0,1]$. In the last equation, the term $\chi _{[0,t_f-d_I]}(t)$ denotes the characteristic function of the interval $[0,t_f-d_I]$ at time t. The minimization of the Hamiltonian with respect to the controls $u_1,u_2$ and delayed controls $v_1,v_2$ involves the switching functions $\sigma _k(t)$ for $k=1,2$:

$$\begin{aligned} \begin{array}{rcl} \sigma _k^w(t) &{}=&{} \displaystyle \frac{\partial H}{\partial u_k}[t] + \chi _{[0,t_f-d_{u_k}]}(t) \frac{\partial H}{\partial v_k}[t+d_{u_k}] \\ &{}=&{} \left\{ \begin{array}{lcl} a_{1k}\overline{\lambda }_5^w + a_{2k}\overline{\lambda }_6^w -\ \epsilon _k \lambda _{L_k}^w(t+d_{u_k}) L_k^w(t+d_{u_k}), &{} \hbox {if} &{} 0 \le t \le t_f-d_{u_k},\\ a_{1k}\overline{\lambda }_5^w + a_{2k}\overline{\lambda }_6^w, &{} \hbox {if} &{} t_f-d_{u_k} \le t \le t_f. \end{array} \right. \end{array}\nonumber \\ \end{aligned}$$

(23)

As in the Rayleigh problem, the superscript “w” above denotes dependence on the scalarization parameter/weight w. Then the controls minimizing the Hamiltonian are characterized by the switching conditions (control law)

$$\begin{aligned} u_k^w(t) = \left\{ \begin{array}{rcl} 0, &{} \hbox { if} &{} \sigma _k^w(t) > 0, \\ 1, &{} \hbox { if} &{} \sigma _k^w(t) < 0, \end{array} \right. \qquad k=1,2. \end{aligned}$$

(24)

for all $w\in [0,1]$. In particular, for positive weights $a_1 >0$, $a_2 >0$, the switching functions (23) and the control law (24) imply

$$\begin{aligned} u_k^w(t) = 0 \quad \forall \; t_f-d_{u_k} \le t \le t_f , \end{aligned}$$

for all $w\in [0,1]$.

In what follows we choose the control weights as $a_{11}=a_{12} = 10$ (small) and $a_{21}=a_{22} = 1000$ (large) in the objective functionals $\varphi _1$ and $\varphi _2$.

For Problem (OCP$_w$) written for the TB problem, we have chosen the utopia vector as $(\beta _1^*, \beta _2^*) = (0, 0)$. Figure 3 depicts the Pareto front for the TB problem we consider here. The plot also displays the iterations of Algorithm 1. The TB problem is discretized using the trapezoidal rule, the number of grid points is set to be $N = 5000$, and the Ipopt’s tolerance to $10^{-10}$, so as to get solutions for w accurate at least up to four decimal places (dp).

The essential interval in this case is found to be $[w_0,w_f] = [0.5251, 0.5709]$, with

$$\begin{aligned} (\varphi _1^{w_0},\varphi _2^{w_0}) = (28155, 31133)\quad \hbox {and}\quad (\varphi _1^{w_f},\varphi _2^{w_f}) = (26459, 35205), \end{aligned}$$

where $\varphi _i^{w}:= \varphi _i(x^{w}(t_f),t_f^{w})$, $i = 1, 2$, with $w = w_0$ or $w_f$, or as will be the case below, $w = w^*$. Optimization over the Pareto front results in $w^* = 0.5358$, after 10 iterations of Algorithm 1, yielding

$$\begin{aligned} \varphi _0^{w^*} = 41621 \quad \hbox {and}\quad (\varphi _1^{w^*},\varphi _2^{w^*}) = (27255, 31455). \end{aligned}$$

In Fig. 3 only five iterations are displayed (labelled 1–5) for clarity in viewing. The Pareto (master) solution with $w = w^*$ is represented by a square.

Fig. 3

TB problem—Pareto front, and iterations of Algorithm 1: Master solution is depicted by a (red) square and iterates by (light blue) circles

Fig. 4

TB problem—Boundary Pareto solutions, corresponding to $w_0 = 0.5251$ and $w_f = 0.5709$, are shown with (blue) solid curves and (green) dashed curves, respectively. Master Pareto solution, corresponding to $w^* = 0.5358$,is shown with dashed-and-dotted (red) curves

The numerical Pareto-optimal control variable solutions $u_1^w(\cdot )$ and $u_2^w(\cdot )$ are presented in Figs. 4a–b for $w = w_0, w^*, w_f$. As with Rayleigh, one of the boundary Pareto-optimal solutions is shown using solid (blue) curves for $w = w_0$, the same solution for all $w\in [0,w_0]$. The other boundary Pareto-optimal solution for $w = w_f$, which holds for all $w\in [w_f,1]$, is shown using dashed (green) curves. Both of the control solutions are of bang–bang type (as required by (24)), with one switching (the number of switchings not dictated by (24) alone). The master Pareto solution is given for $w = w^*$ using dashed-and-dotted (red) curves, in which the controls are also of bang–bang type with one switching.

Table 2 TB problem

The switching functions for each control and case, $\sigma _k^w(\cdot )$, $k = 1,2$, scaled as indicated, are plotted with (black) dotted curves and superposed with the control plots in Fig. 4a and b. We remind that, by using (23) (recall that discrete approximations of $\lambda ^w_{L_k}(t)$, $k = 1,2$, $\lambda ^w_5(t)$ and $\lambda ^w_6(t)$ can readily be obtained as constraint multipliers from AMPL), one verifies the optimality condition in (24).

In each strategy, the two control efforts are “on” until the times $t_{s_k}^w$, $k = 1,2$, at which the respective $u_k^w(\cdot )$ is switched “off” (down to zero). These types of bang–bang controls are also referred to as on–off controls. In Table 2 the switching times for the boundary as well as the optimal weights are listed. Under these controls, the resulting terminal values of the state variables are also listed in Table 2. The plots of these variables are not provided as they are difficult to distinguish at earlier times (as expected) and that they become distinguishable/comparable only near the terminal time.

Under the controls minimizing $x_5(t_f)$ (with $w = w_f = 0.5709$ and minimum $x_5^{w_f}(t_f) = 26459$) the number of persistent latent individuals $L_2(t_f)$ turns out to be about 419 (in a population of 30000). This number is more than doubled to 864 if $x_6(t_f)$ is minimized (with $w = w_0 = 0.5709$ and minimum $x_6^{w_0}(t_f) = 31133$). The optimal Pareto solution minimizing the distance in value space to the origin yields with $w = w^* = 0.5358$ the optimal $L_2(t_f)$ as 748.

5 Conclusion

We have proposed an algorithm to solve the problem of optimization over the Pareto front. The algorithm employs bisection method which starts with an essential interval of weights of the Chebyshev scalarization. It is applicable to a wide range of optimal control problems, including state- and control-constrained problems with time delay. Numerical solution of two challenging optimal control problems has demonstrated the effectiveness of the algorithm.

In both of the example applications, the Pareto front turns out to be the graph of a convex functional; so one might ask why one could not use the weighted-sum scalarization? There are two reasons. The first reason is that each problem is nonconvex, therefore it is necessary to use the Chebyshev scalarization in order not to miss any potential “nonconvex part” of the Pareto front. The second reason is that with Chebyshev one can construct the essential interval $[w_0,w_f]$ which in general is smaller than the whole interval [0, 1], in turn saving computational time. The weighted-sum scalarization does not have this advantage.

The main motive behind the algorithm we have proposed is that one can find the optimal solution minimizing a master objective functional without having to construct the Pareto front. The algorithm solves the challenging optimal control problem (OCP$_w$) a relatively smaller number of times than the case of constructing the Pareto front. In the examples we have studied the algorithm had to solve (OCP$_w$) 20 to 30 times. On the other hand, without the algorithm we propose, it is necessary to construct the Pareto front by solving (OCP$_w$) thousands of times in order to obtain the same solution with the same computational accuracy.

The proposed algorithm can be improved/modified in various ways. For example, scalarization techniques other than Chebyshev might be employed; see for example [8, 9] and the references therein. Bisection method might be replaced by methods with higher convergence rates, for example regula falsi and secant methods (see [10]), at the expense of approximating higher order derivatives of course, although the latter would make the algorithm applicable to problems with more than just two objective functionals.