A Survey on Proximal Point Type Algorithms for Solving Vector Optimization Problems

Abstract

In this survey paper we present the existing generalizations of the proximal point method from scalar to vector optimization problems, discussing some of their advantages and drawbacks, respectively, presenting some open challenges and sketching some possible directions for future research.

Dedicated to the memory of J.M. Borwein

Appendix: Proof of Theorem 11.17

In the following we provide an example of a convergence proof for a proximal point algorithm for determininig weakly efficient solutions to a vector optimization problem. It originates from an earlier version of [19] and incorporates some ideas from the proofs of [24, Theorem 3.1] and [3, Theorem 2.1 and Proposition 2.1]. Before formulating it, we recall the celebrated Opial’s Lemma (cf. [65]).

Lemma 11.2

Let (x _n)_n ⊆ X a sequence such that there exists a nonempty set S ⊆ X such that

• (a) lim_n→+∞∥x _n − x∥ exists for every x ∈ S;

• (b) if $x_{n_j} \rightharpoonup \hat x$ for a subsequence n _j → +∞, then $\hat x\in S$.

Then, there exists an $\bar x \in S$ such that $x_k \rightharpoonup \bar x$ when k → +∞.

Theorem 11.17

Let F be C-convex and positively C-lower semicontinuous and F(X) ∩ (F(x ₁) − C) be C-complete. Then any sequence (x _n)_n generated by Algorithm 17 converges weakly towards a weakly efficient solution to (V P).

Proof

We show first that the algorithm is well-defined. Assuming that we have obtained an x _n, where n ≥ 1, we have to secure the existence of x _n+1. Take a $z^*_n\in C^*\setminus \{0\}$ and without loss of generality assume that $\|z^*_n\|=1$ for all n ≥ 1. Then $\langle z^*_n, e_n\rangle > 0$ and the function

$$\displaystyle \begin{aligned}x\mapsto \langle z^*_n, \lambda_n F(x) + \frac{\alpha_n}{2}\|x-x_n-\beta_n(x_n-x_{n-1})\|{}^2 e_n \rangle + \delta_{\varOmega_n} (x)\end{aligned}$$

is lower semicontinuous, being a sum of continuous and lower semicontinuous functions, respectively, and strongly convex, as the sum of some convex functions and a squared norm, having thus exactly one minimum. By Lemma 11.1 this minimum is a weakly efficient solution to the vector optimization problem in Step 3 of Algorithm 17 and we denote it by x _n+1.

The next step is to show the Fejér monotonicity of the sequence (x _n)_n with respect to the set Ω = {x ∈ X : F(x) ≦_C F(x _k) ∀k ≥ 0}, that is nonempty because of the C-completeness hypothesis. Let n ≥ 1. The function $x\mapsto \langle z^*_n, \lambda _n F(x) + ({\alpha _n}/{2})\|x-x_n-\beta _n(x_n-x_{n-1})\|{ }^2 e_n \rangle + \delta _{\varOmega _n} (x)$ attains its only minimum at x _n+1, and this fact can be equivalently written as

$$\displaystyle \begin{aligned}0\in \partial \big(\langle z^*_n, \lambda_n F(\cdot) + \frac{\alpha_n}{2}\|\cdot-x_n-\beta_n (x_n-x_{n-1})\|{}^2 e_n \rangle+ \delta_{\varOmega_n} (\cdot)\big)(x_{n+1}).\end{aligned}$$

Using the continuity of the norm, this yields (e.g. via [21, Theorem 3.5.6]) $0\in \partial \big (\langle z^*_n, \lambda _n F(\cdot ) \rangle + \delta _{\varOmega _n} (\cdot )\big ) (x_{n+1}) + \partial \big (({\alpha _n}/{2})\langle z^*_n, e_n\rangle \|\cdot -x_n-\beta _n (x_n-x_{n-1})\|{ }^2 \big )(x_{n+1}) = \partial \big (\langle z^*_n, \lambda _n F(\cdot ) \rangle + \delta _{\varOmega _n} (\cdot )\big ) (x_{n+1})$ $+ \alpha _n \langle z^*_n, e_n\rangle (x_{n+1}-x_n-\beta _n (x_n-x_{n-1}))$. Then, since x _n+1 ∈ Ω _n, for any x ∈ Ω _n it holds

$$\displaystyle \begin{aligned} \lambda_n \langle z^*_n, F(x)- F(x_{n+1})\rangle \geq \alpha_n \langle z^*_n, e_n\rangle \langle x_{n+1}-x_n-\beta_n (x_n-x_{n-1}), x_{n+1}-x\rangle. \end{aligned} $$

(11.2)

Let us take an element $\tilde x\in \varOmega$. By construction $\tilde x\in \varOmega _n$, thus (11.2) yields, after taking into consideration that $F(\tilde x)\leqq _C F(x_{n+1})$, λ _n > 0 and $z^*_n\in C^*\setminus \{0\}$, that $\alpha _n \langle z^*_n, e_n\rangle \langle x_{n+1}-x_n-\beta _n (x_n-x_{n-1}), \tilde x-x_{n+1}\rangle \geq 0$.

For each k ≥ 0 denote $\varphi _k=(1/2)\|x_k-\tilde x\|{ }^2$. The previous inequality, after dividing with the positive number $\alpha _n \langle z^*_n, e_n\rangle$, can be rewritten as

$$\displaystyle \begin{aligned}\varphi_{n+1}-\varphi_n + \frac 12 \|x_{n+1}-x_n\|{}^2 - \beta_n \langle x_n-x_{n-1}, x_{n+1}-\tilde x\rangle \leq 0, \end{aligned}$$

and, since $\langle x_n-x_{n-1}, x_{n+1}-\tilde x\rangle = \varphi _n-\varphi _{n-1} + (1/2)\|x_n-x_{n-1}\|{ }^2+ \langle x_n-x_{n-1}, x_{n+1}-x_n\rangle$, it turns into

$$\displaystyle \begin{aligned}\varphi_{n+1}-\varphi_n - \beta_n (\varphi_n-\varphi_{n-1}) \leq \frac{\beta_n}{2} \|x_n-x_{n-1}\|{}^2 +\end{aligned}$$

$$\displaystyle \begin{aligned} \beta_n \langle x_n-x_{n-1}, x_{n+1}-x_n\rangle -\frac 12 \|x_{n+1}-x_n\|{}^2. \end{aligned} $$

(11.3)

Since the right-hand side of (11.3) is less than or equal to ((β _n − 1)∕2)∥x _n+1 − x _n∥² + β _n∥x _n − x _n−1∥², denoting μ _k = φ _k − β _k φ _k−1 + β _k∥x _k − x _k−1∥², k ≥ 1, it follows that

$$\displaystyle \begin{aligned} \mu_{n+1}-\mu_n \leq \frac {3\beta - 1}{2}\|x_{n+1}-x_n\|{}^2 \leq 0, \end{aligned} $$

(11.4)

thus the sequence (μ _k)_k is nonincreasing, as n ≥ 1 was arbitrarily chosen. Then φ _n ≤ β ⁿ φ ₀ + μ ₁∕(1 − β) and one also gets ∥x _n+1 − x _n∥² ≤ (2∕(1 − 3β))(μ _n − μ _n+1). Employing (11.4), one obtains then

$$\displaystyle \begin{aligned}\sum_{k=1}^n \|x_{k+1}-x_k\|{}^2 \leq \frac {2}{1-3\beta}(\mu_1- \mu_{n+1}) \leq \frac {2}{1-3\beta} \left( \beta^{n+1}\varphi_0 + \frac{\mu_1}{1-\beta}\right) < +\infty, \end{aligned}$$

in particular

$$\displaystyle \begin{aligned} \sum_{k=1}^{+\infty} \|x_{k+1}-x_k\|{}^2 \leq \frac{2\mu_1}{(1-\beta)(1-3\beta)} < +\infty. \end{aligned} $$

(11.5)

The right-hand side of (11.3) can be rewritten as (1∕2)(β _n(β _n + 1)∥x _n − x _n−1∥² −∥x _n+1 − x _n − β _n(x _n − x _n−1)∥²). Denoting τ _k+1 = x _k+1 − x _k − β _k(x _k − x _k−1), θ _k = φ _k − φ _k−1 and δ _k = β _k∥x _k − x _k−1∥² for k ≥ 0 and taking into consideration that β _n ∈ [0, 1∕3), (11.3) yields

$$\displaystyle \begin{aligned} \theta_{n+1}-\beta_n\theta_n \leq \delta_n-\frac 12 \|\tau_{n+1}\|{}^2. \end{aligned} $$

(11.6)

Then [θ _n+1]₊ ≤ (1∕3)[θ _n]₊ + δ _n, followed by $[\theta _{n+1}]_+ \leq (1/3^n) [\theta _1]_+ + \sum _{k=0}^{n-1}\delta _{n-k}/3^k$. Hence $\sum _{k=0}^{+\infty }[\theta _{k+1}]_+ \leq 3/2 ( [\theta _1]_+ + \sum _{k=0}^{+\infty }\delta _k)$ and, as the right-hand side of this inequality is finite due to (11.5), so is $\sum _{k=1}^{+\infty }[\theta _k]_+$, too. This yields that the sequence (w _k)_k defined as $w_k=\varphi _k - \sum _{j=1}^k [\theta _j]_+$, k ≥ 0, is bounded. Moreover, $w_{k+1}-w_k = \varphi _{k+1}-\varphi _k - [\varphi _{k+1}-\varphi _k]_+ = \varphi _{k+1}-\varphi _k + \min \{0, \varphi _k - \varphi _{k+1}\} \leq 0$ for all k ≥ 1, thus (w _k)_k is convergent. Consequently, $\lim _{k \rightarrow +\infty } \varphi _k = \lim _{k \rightarrow +\infty } w_k + \sum _{j=1}^{+\infty }[\theta _{j+1}]_+$, therefore (φ _k)_k is convergent. Finally, $(\|x_k-\tilde x\|{ }^2)_k$ is convergent, too, i.e. (a) in Lemma 11.2 with S = Ω is fulfilled.

We show now that (x _k)_k is weakly convergent. The convergence of (φ _k)_k implies that (x _k)_k is bounded, so it has weak cluster points. Let $\hat x\in X$ be one of them and $(x_{k_j})_j$ the subsequence that converges towards it. Then, as F is positively C-lower semicontinuous and C-convex, it follows that for any z ^∗∈ C ^∗ the function 〈z ^∗, F(⋅)〉 is lower semicontinuous and convex, thus

$$\displaystyle \begin{aligned} \langle z^*, F(\hat x)\rangle \leq \lim_{j\rightarrow +\infty}\langle z^*, F(x_{k_j})\rangle = \inf_{k\geq 0} \langle z^*, F(x_k)\rangle, \end{aligned} $$

(11.7)

with the last equality following from the fact that the sequence (F(x _k))_k is by construction nonincreasing. Assuming that there exists a k ≥ 0 such that $F(\hat x)\nleqq _C F(x_k)$, there exists a $\tilde z\in C^*\setminus \{0\}$ such that $\langle \tilde z, F(\hat x) - F(x_k)\rangle > 0$, which contradicts (11.7), consequently $F(\hat x)\leqq _C F(x_k)$ for all k ≥ 0, i.e. $\hat x\in \varOmega$, therefore one can employ Lemma 11.2 with S = Ω since its hypothesis (b) is fulfilled as well. This guarantees then the weak convergence of (x _k)_k to a point $\bar x\in \varOmega$.

The last step is to show that $\bar x \in \mathcal {W}\mathcal {E}(VP)$. Assuming that $\bar x\notin \mathcal {W}\mathcal {E} (VP)$, there exists an x′∈ X such that $F(x')< _C F(\bar x)$. This yields x′∈ Ω. As $\|z^*_k\|=1$ for all k ≥ 0, the sequence $(z_k^*)_k$ has a weak^∗ cluster point, say $\bar z^*$, that is the limit of a subsequence $(z^*_{k_j})_j$. Because $z^*_k\in C^*$ for all k ≥ 0 and C ^∗ is weakly^∗ closed, it follows that $\bar z^*\in C^*$. Moreover, $\bar z^*\neq 0$, since it can be shown via [23, Lemma 2.2] that $\langle \bar z^*, c\rangle > 0$ for any $c\in \operatorname *{\mathrm {int}} C$. Consequently, $\langle \bar z^*, F(x') - F(\bar x)\rangle < 0$. For any j ≥ 0 it holds by (11.2)

$$\displaystyle \begin{aligned}\lambda_{k_j}\langle z^*_{k_j}, F(x') - F(x_{k_j+1})\rangle \geq -\langle \alpha_{k_j}\langle z^*_{k_j}, e_{k_j}\rangle (x_{k_j+1}- x_{k_j} - \beta_{k_j}(x_{k_j}-x_{k_j-1}), x' -\end{aligned}$$

$$\displaystyle \begin{aligned} x_{k_j+1}\rangle \geq - \alpha_{k_j}\langle z^*_{k_j}, e_{k_j}\rangle \|x' - x_{k_j+1}\| \big(\|x_{k_j+1}- x_{k_j}\| + \beta_{k_j}\|x_{k_j}-x_{k_j-1}\|\big). \end{aligned} $$

(11.8)

Because of (11.5), (∥x _k − x _k−1∥)_k converges towards 0 for k → +∞, therefore so does the last expression in the inequality chain (11.8) when j → +∞ as well. Letting j converge towards + ∞, (11.8) yields $\langle \bar z^*, F(x') - F(\bar x)\rangle \geq 0$, contradicting the inequality obtained above. Consequently, $\bar x \in \mathcal {W}\mathcal {E}(VP)$. □

Remark 11.43

In order to guarantee the lower semicontinuity of the functions $\delta _{\varOmega _n}$, n ≥ 1, it is enough to have the vector function F only C-level closed (i.e. the set {x ∈ X : F(x) ≦_C y} is closed for any y ∈ Y ), a hypotheses weaker than the positive C-lower semicontinuity imposed on F in Theorem 11.17 and Theorem 11.18. However, the latter is also necessary in the proofs of these statements in order to guarantee the lower semicontinuity of the functions $(z^*_nF)$, n ≥ 1.