A strong sequential optimality condition for cardinality-constrained optimization problems

Abstract

In this paper, we consider the continuous relaxation reformulation of cardinality-constrained optimization problems that has become more popular in recent years and propose a new sequential optimality condition (approximate stationarity) for cardinality-constrained optimization problems, which is proved to be a genuine necessary optimality condition that does not require any constraint qualification to hold. We compare this condition with the rest of the sequential optimality conditions and prove that our condition is stronger and closer to the local minimizer. A problem-tailored regularity condition is proposed, and we show that this regularity condition ensures that the approximate stationary point proposed in this paper is the exact stationary point and is the weakest constraint qualification with this property. Finally, we apply the results of this paper to safeguarded augmented Lagrangian method and prove that the algorithm converges to the approximate stationary point proposed in this paper under mild assumptions, the existing theoretical results of this algorithm are further enhanced.

Appendix: A. Some auxiliary results

In this section, we show that CC-PCAM-stationarity is a necessary optimality condition for TNLP(x^∗).

Theorem 17

Let x^∗ be a local minimizer of TNLP(x^∗), then x^∗ is CC-PCAM-stationary.

Proof

We define the local problem for TNLP(x^∗) as

$$\left\{\begin{array}{cl} \min& f(x)+\frac{1}{2}\|x-x^{*}\|^{2}\\ s.t. &g(x)\leq 0,\quad h(x)=0,\quad \\ &x\in B(x^{*},\epsilon),\quad x_{\imath}=0~~~\forall\imath\in I_{0}(x^{*}), \end{array}\right.$$

(A1)

where 𝜖 > 0 is such that x^∗ is the only global minimizer of the problem (A1). We consider the partial penalty problem for TNLP(x^∗)

$$\left\{\begin{array}{cl} \min &f(x)+\rho_{k} F(x)+\frac{1}{2}\|x-x^{*}\|^{2}\\ s.t. &x\in B(x^{*},\epsilon), \end{array}\right.$$

(A2)

where ρ_k > 0 and

$$F(x)=\frac{1}{2}\left[\left\|g(x)_{+}\right\|^{2}+\left\|h(x)\right\|^{2}+\sum\limits_{\imath\in I_{0}(x^{*})}x_{\imath}^{2}\right].$$

The problem (A2) has a continuous objective function and a compact feasible set, then there exists a global minimizer, denoted as x^k. Obviously, {x^k} is bounded, and for simplicity, we set $x^{k}\rightarrow \bar {x}\in B(x^{*},\epsilon )$ and let $\rho _{k}\rightarrow \infty$. Then, for any k there is

$$f(x^{k})+\rho_{k}F(x^{k})+\frac{1}{2}\|x^{k}-x^{*}\|^{2}\leq f(x^{*}).$$

(A3)

Dividing both sides of (A3) by ρ_k and taking the limit yields $F(\bar {x})=0$, which combined with $\bar {x}\in B(x^{*},\epsilon )$ shows that $\bar {x}$ is feasible for the problem (A1). Since x^∗ is the global minimizer of the problem (A1), then $f(x^{*})\leq f(\bar {x})$, and (A3) shows that

$$f(x^{k})+\frac{1}{2}\|x^{k}-x^{*}\|^{2}\leq f(x^{*}),$$

taking the limit yields $f(\bar {x})+\frac{1}{2}\|\bar {x}-x^{*}\|^{2}\leq f(x^{*})$. Thus, $\bar {x}=x^{*}$, i.e., $x^{k}\rightarrow x^{*}$. For simplicity, we set $\{x^{k}\}\subseteq int~B(x^{*},\epsilon )$.

From the necessary optimality condition for the problem (A2) we have

$$\nabla f(x^{k})+\nabla g(x^{k})(\rho_{k}g(x^{k})_{+})+\nabla h(x^{k})(\rho_{k} h(x^{k}))+\sum\limits_{\imath\in I_{0}(x^{*})}(\rho_{k}x_{\imath}^{k})+x^{k}-x^{*}=0.$$

Let $\lambda ^{k}=\rho _{k}g(x^{k})_{+},~\mu ^{k}=\rho _{k} h(x^{k}),~\gamma _{\imath }^{k}=\rho _{k}x_{\imath }^{k}~(\imath \in I_{0}(x^{*})),~\gamma _{\imath }^{k}=0~(\imath \notin I_{0}(x^{*}))$. Since $x^{k}\rightarrow x^{*}$, we get

$$\nabla f(x^{k})+\nabla g(x^{k})\lambda^{k}+\nabla h(x^{k})\mu^{k}+\gamma^{k}\rightarrow 0,$$

that is, the condition (a) of Definition 5 holds. Moreover, from (A3) and $x^{k}\rightarrow x^{*}$, we know that $\lim _{k\rightarrow \infty }\rho _{k} F(x^{k})=0$, then

$$\lim\limits_{k\rightarrow\infty} \left|\lambda^{k}\circ g(x^{k})_{+}\right|+\left|\mu^{k}\circ h(x^{k})\right|+\left|\gamma^{k}\circ x^{k}\right|=0.$$

(A4)

If i∉I_g(x^∗), then g_i(x^k) < 0 when k is sufficiently large, so ${\lambda _{i}^{k}}=0$. Conversely, if i ∈ I_g(x^∗), then $g_{i}(x^{k})\rightarrow 0$, similar to Theorem 3, only two cases of g_i(x^k) ↑ 0 and g_i(x^k) ↓ 0 need to be analyzed. When g_i(x^k) ↑ 0, we have g_i(x^k) < 0, so ${\lambda _{i}^{k}}=0$. If g_i(x^k) ↓ 0, then g_i(x^k)₊ = g_i(x^k) and

$$0\leq \frac{1}{2}{\lambda_{i}^{k}} g_{i}(x^{k})=\frac{1}{2}\rho_{k}(g_{i}(x^{k})_{+})^{2}\leq\rho_{k} F(x^{k},y^{k})\rightarrow 0.$$

Therefore, ${\lambda _{i}^{k}}g_{i}(x^{k})\rightarrow 0~(\forall i\in I_{g}(x^{*}))$, then $\vert \lambda ^{k}\circ g(x^{k})\vert \rightarrow 0$. Combining (A4), we have

$$\lim\limits_{k\rightarrow\infty} \left|\lambda^{k}\circ g(x^{k})\right|+\left|\mu^{k}\circ h(x^{k})\right|+\left|\gamma^{k}\circ x^{k}\right|=0,$$

i. e., the condition (b) holds. By the definition of (λ^k,μ^k,γ^k), it is easy to verify that the conditions (c)-(e) holds using the method of Theorem 3. In summary, we prove that x^∗ is CC-PCAM-stationary. □