ROM-Based Multiobjective Optimization of Elliptic PDEs via Numerical Continuation

Abstract

Multiobjective optimization plays an increasingly important role in modern applications, where several objectives are often of equal importance. The task in multiobjective optimization and multiobjective optimal control is therefore to compute the set of optimal compromises (the Pareto set) between the conflicting objectives. Since the Pareto set generally consists of an infinite number of solutions, the computational effort can quickly become challenging which is particularly problematic when the objectives are costly to evaluate as is the case for models governed by partial differential equations (PDEs). To decrease the numerical effort to an affordable amount, surrogate models can be used to replace the expensive PDE evaluations. Existing multiobjective optimization methods using model reduction are limited either to low parameter dimensions or to few (ideally two) objectives. In this chapter, we present a combination of the reduced basis model reduction method with a continuation approach using inexact gradients. The resulting approach can handle an arbitrary number of objectives while yielding a significant reduction in computing time.

Appendix A: Proof of Theorem 2.10

To prove Theorem 2.10, we first have to investigate some of the properties of the optimization problem (2.6). This problem is quadratic with linear equality and inequality constraints. We will first investigate the uniqueness of the solution in the following lemma.

Lemma A.1

Let u ∈ φ⁻¹(0) and let α¹ and α² be two solutions of (2.6) with α¹ ≠ α² . Then ω(α) = 0 for all α ∈ span({α¹, α²}) and

$$\displaystyle \begin{aligned} span( \{ \alpha^1, \alpha^2 \} ) \cap ker(DJ(u)^\top) \neq \emptyset. \end{aligned} $$

(A.1)

Proof

For $c_1, c_2 \in \mathbb {R} \setminus \{ 0 \}$ we have

$$\displaystyle \begin{aligned} &\omega(c_1 \alpha^1 + c_2 \alpha^2) \\ &= (c_1 \alpha^1 + c_2 \alpha^2)^\top (DJ^r(u) DJ^r(u)^\top - \epsilon \epsilon^\top) (c_1 \alpha^1 + c_2 \alpha^2) \\ &= c_1^2 \omega(\alpha^1) + 2 ({c_1 \alpha^1}^\top DJ^r(u) DJ^r(u)^\top c_2 \alpha^2 - {c_1 \alpha^1}^\top \epsilon \epsilon^\top c_2 \alpha^2) + c_2^2 \omega(\alpha^2) \\ &= 2 ({c_1 \alpha^1}^\top DJ^r(u) DJ^r(u)^\top c_2 \alpha^2 - {c_1 \alpha^1}^\top \epsilon \epsilon^\top c_2 \alpha^2) \\ &= 2 c_1 c_2 ( (DJ^r(u)^\top \alpha^1)^\top (DJ^r(u)^\top \alpha^2) - (\epsilon^\top \alpha^1) (\epsilon^\top \alpha^2) ). \end{aligned} $$

From ω(α¹) = ω(α²) = 0 it follows that 𝜖^⊤ α¹ = ∥DJ^r(u)^⊤ α¹∥ and 𝜖^⊤ α² = ∥DJ^r(u)^⊤ α²∥. Let $\sphericalangle$ be the angle between DJ^r(u)^⊤ α¹ and DJ^r(u)^⊤ α². Then

$$\displaystyle \begin{aligned} &\omega(c_1 \alpha^1 + c_2 \alpha^2) \\ &= 2 c_1 c_2 (cos(\sphericalangle) \| DJ^r(u)^\top \alpha^1 \| \| DJ^r(u)^\top \alpha^2 \| - \| DJ^r(u)^\top \alpha^1 \| \| DJ^r(u)^\top \alpha^2 \|) \\ &= 2 c_1 c_2 (cos(\sphericalangle) - 1) \| DJ^r(u)^\top \alpha^1 \| \| DJ^r(u)^\top \alpha^2 \|. \end{aligned} $$

(A.2)

Assume $cos(\sphericalangle ) \neq 1$ (i.e., $cos(\sphericalangle ) - 1 < 0$), ∥DJ^r(u)^⊤ α¹∥≠ 0, and ∥DJ^r(u)^⊤ α²∥≠ 0. If we choose c ₁ = t and c ₂ = 1 − t for t ∈ (0, 1), then tα¹ + (1 − t)α² ∈ Δ_k and ω(tα¹ + (1 − t)α²) < 0, which contradicts u ∈ φ⁻¹(0). If ∥DJ^r(u)^⊤ α¹∥ = 0 or ∥DJ^r(u)^⊤ α²∥ = 0, then (A.1) holds for or , respectively. If $cos(\sphericalangle ) - 1 = 0$, then DJ^r(u)^⊤ α¹ and DJ^r(u)^⊤ α² are linearly dependent, so there are , such that for . In particular, in any case we must have ω(α) = 0 for all α ∈ span({α¹, α²}). □

The previous lemma implies that for k = 2, the solution of (2.6) for u ∈ φ⁻¹(0) is non-unique iff DJ^r(u)DJ^r(u)^⊤− 𝜖𝜖^⊤ = 0. For k > 2, we can only have non-uniqueness if (A.1) holds. If we consider the dimensions of the spaces in (A.1), we see that in the generic case, it can only hold if

$$\displaystyle \begin{aligned} & dim(span( \{ \alpha^1, \alpha^2 \} ) \cap ker(DJ(u)^\top)) \geq 1 \\ \Leftrightarrow \ & 2 + k - rk(DJ(u)^\top) - k \geq 1 \\ \Leftrightarrow \ & rk(DJ(u)^\top) \leq 1, \end{aligned} $$

i.e., if all gradients of the objectives are linearly dependent in u. This motivates us to assume that in general, the solution of (2.6) is unique for almost all u ∈ φ⁻¹(0).

We will now investigate the differentiability of φ. Our strategy is to apply the implicit function theorem to the KKT conditions of (2.6) to obtain a differentiable function ϕ that maps a point $u \in \mathbb {R}^n$ onto the solution of (2.6) in u. This would imply the differentiability of φ via concatenation with ω. An obvious problem here is the fact that (2.6) has inequality constraints which, when activated or deactivated under variation of u, lead to non-differentiabilities in ϕ. Note that an inequality constraint being active means that one component of α is zero, i.e., one of the objective functions has no impact on the current problem. Thus, for our theoretical purposes, if there is an active inequality constraint in (2.6) we will just ignore the corresponding objective function. This approach is strongly related to the hierarchical decomposition of the Pareto critical set (cf. [10]).

For the reasons mentioned above, we will now consider the case where the solution of (2.6) is strictly positive in each component. The following lemma shows a technical result that will be used in a later proof.

Lemma A.2

Let u ∈ φ⁻¹(0) and let be a solution of (2.6) with α _i > 0 ∀i ∈{1, …, k}. Then is unique if and only if there is no $\beta \in \mathbb {R}^k \setminus \{ 0 \}$ with ω(β) = 0 and $\sum _{i = 1}^k \beta _i = 0$.

Proof

We will show that α is non-unique if and only if there is some $\beta \in \mathbb {R}^k$ with ω(β) = 0 and $\sum _{i = 1}^k \beta _i = 0$.

: Let $\tilde {\alpha }$ be another solution of (2.6). Then, as in the proof of Lemma A.1, we must have for all $c_1, c_2 \in \mathbb {R}$. This means we can choose .

⇐: Let $\beta \in \mathbb {R}^k$ with ω(β) = 0 and $\sum _{i = 1}^k \beta _i = 0$. Let s > 0 be small enough such that . Then, as in (A.2), we have

Since by assumption φ(u) = 0 we must have , so is another solution of (2.6). □

To be able to use the KKT conditions of (2.6) to obtain its solution, we have to make sure that these conditions are sufficient. Since (2.6) is a quadratic problem, this means we have to show that the matrix in the objective ω is positive semidefinite.

Lemma A.3

Let u ∈ φ⁻¹(0) and let be the unique solution of (2.6) with ∀i ∈{1, …, k}. Then ω(β) ≥ 0 for all $\beta \in \mathbb {R}^k$ . In particular, DJ(u)DJ(u)^⊤− 𝜖𝜖^⊤ is positive semidefinite.

Proof

Assume there is some $\beta \in \mathbb {R}^k$ with ω(β) < 0, i.e., 𝜖^⊤ β > ∥DJ^r(u)^⊤ β∥. We distinguish between two cases:

Case 1::

$\sum _{i = 1}^k \beta _i = 0$: Similar to the proof of Lemma A.1 we get

for all s > 0. In particular, since is positive, there is some such that with , which is a contradiction.

Case 2::

$\sum _{i = 1}^k \beta _i \neq 0$. W.l.o.g. assume that $\sum _{i = 1}^k \beta _i = 1$. Consider

Then and . By assumption, we must have for all s such that . By continuity of there must be some s^∗ with . Let . Using (A.2) we get

for all t ∈ (0, 1), which is a contradiction. □

The previous results now allow us to prove Theorem 2.10.

Theorem 2.10

Let such that (2.6) has a unique solution with for all i ∈{1, …, k}. Let (2.6) be uniquely solvable in a neighborhood of . Then there is an open set $U \subseteq \mathbb {R}^n$ with such that φ|_U is continuously differentiable.

Proof

The KKT conditions for (2.6) are

$$\displaystyle \begin{aligned} (DJ(u) DJ(u)^\top - \epsilon \epsilon^\top) \alpha - \begin{pmatrix} \lambda + \mu_1 \\ \vdots \\ \lambda + \mu_k \end{pmatrix} &= 0, \\ \sum_{i = 1}^k \alpha_i - 1 &= 0, \\ \alpha_i &\geq 0 \ \forall i \in \{1,\ldots,k\}, \\ \mu_i &\geq 0 \ \forall i \in \{1,\ldots,k\}, \\ \mu_i \alpha_i &= 0 \ \forall i \in \{1,\ldots,k\}. \end{aligned} $$

(A.3)

for $\lambda \in \mathbb {R}$ and $\mu \in \mathbb {R}^k$. By Lemma A.3 these conditions are sufficient for optimality. By our assumption there is an open set U′ with such that the solution of (2.6) is unique and positive. Thus, on U′, (A.3) is equivalent to

$$\displaystyle \begin{aligned} (DJ(u) DJ(u)^\top - \epsilon \epsilon^\top) \alpha - \begin{pmatrix} \lambda \\ \vdots \\ \lambda \end{pmatrix} &= 0, \\ \sum_{i = 1}^k \alpha_i - 1 &= 0, \end{aligned} $$

for some $\lambda \in \mathbb {R}$. This system can be rewritten as G(u, (α, λ)) = 0 for

Derivating G with respect to (α, λ) yields

$$\displaystyle \begin{aligned} D_{(\alpha,\lambda)} G(u,(\alpha,\lambda)) = \begin{pmatrix} (DJ(u) DJ(u)^\top - \epsilon \epsilon^\top) & (-1,\ldots,-1)^\top \\ (1,\ldots,1) & 0 \end{pmatrix} \in \mathbb{R}^{(k+1) \times (k+1)}. \end{aligned}$$

Let such that . (Note that uniqueness of implies uniqueness of here.) For to be singular, there would have to be some $v = (v^1,v^2) \in \mathbb {R}^{k+1}$ with

and thus

By Lemma A.2, this is a contradiction to the assumption that is a unique solution of (2.6). So has to be regular. This means we can apply the implicit function theorem to obtain open sets $U \subseteq U' \subseteq \mathbb {R}^n$, $V \subseteq \mathbb {R}^{k+1}$ with , and a continuously differentiable function ϕ = (ϕ _α, ϕ _λ) : U → V  with

$$\displaystyle \begin{aligned} G(u,(\alpha,\lambda)) = 0 \ \Leftrightarrow \ (\alpha,\lambda) = \phi(u) \quad \forall u \in U,(\alpha,\lambda) \in V. \end{aligned}$$

In particular,

(A.4)

so φ|_U is continuously differentiable. □

Remark A.4

From the proof of Theorem 2.10 we can even derive an explicit formula for the derivative of φ|_U in : First of all, the derivative of the implicit function ϕ is given by

By applying the chain rule to (A.4), we obtain

(A.5)

$\blacksquare$