Abstract
In this paper, we consider the continuous relaxation reformulation of cardinality-constrained optimization problems that has become more popular in recent years and propose a new sequential optimality condition (approximate stationarity) for cardinality-constrained optimization problems, which is proved to be a genuine necessary optimality condition that does not require any constraint qualification to hold. We compare this condition with the rest of the sequential optimality conditions and prove that our condition is stronger and closer to the local minimizer. A problem-tailored regularity condition is proposed, and we show that this regularity condition ensures that the approximate stationary point proposed in this paper is the exact stationary point and is the weakest constraint qualification with this property. Finally, we apply the results of this paper to safeguarded augmented Lagrangian method and prove that the algorithm converges to the approximate stationary point proposed in this paper under mild assumptions, the existing theoretical results of this algorithm are further enhanced.
Similar content being viewed by others
Data availability
Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.
Notes
Consider the complementarity constraint G(z) ≥ 0, H(z) ≥ 0, G(z) \(\circ\) H(z) = 0, we say z∗ satisfies the strict complementarity condition, if the point z∗ satisfies this complementarity constraint and there is no index i such that Gi(z∗) = Hi(z∗) = 0.
See [16] for the definition of o-minimal structure.
o-minimal structures include semi-algebraic sets, global subanalytic sets, log-exp structures, etc., and it is powerfully stable in that finite sums of functions definable over \(\mathcal {O}\), composites of functions, the inverse of a function is still definable in \(\mathcal {O}\).
References
Andreani, R., Fazzio, N.S., Schuverdt, M.L., Secchin, L.D.: A sequential optimality condition related to the quasi-normality constraint qualification and its algorithmic consequences. SIAM J. Optim. 29(1), 743–766 (2019). https://doi.org/10.1137/17m1147330
Andreani, R., Haeser, G., Martínez, J. M.: On sequential optimality conditions for smooth constrained optimization. Optimization 60(5), 627–641 (2011). https://doi.org/10.1080/02331930903578700
Andreani, R., Haeser, G., Schuverdt, M.L., Silva, P.J.S.: A relaxed constant positive linear dependence constraint qualification and applications. Math. Program. 135(1-2), 255–273 (2012). https://doi.org/10.1007/s10107-011-0456-0
Andreani, R., Haeser, G., Secchin, L.D., Silva, P.J.S.: New sequential optimality conditions for mathematical programs with complementarity constraints and algorithmic consequences. SIAM J. Optim. 29(4), 3201–3230 (2019). https://doi.org/10.1137/18M121040X
Andreani, R., Martínez, J. M., Svaiter, B. F.: A new sequential optimality condition for constrained optimization and algorithmic consequences. SIAM J. Optim. 20(6), 3533–3554 (2010). https://doi.org/10.1137/090777189
Andreani, R., Ramos, A., Ribeiro, A.A., Secchin, L.D., Velazco, A.R.: On the convergence of augmented Lagrangian strategies for nonlinear programming. Ima. J. Numer. Anal. (2021). https://doi.org/10.1093/imanum/drab021
Berk, L., Bertsimas, D.: Certifiably optimal sparse principal component analysis. Math. Program. Comput. 11(3), 381–420 (2019). https://doi.org/10.1007/s12532-018-0153-6
Bertsimas, D., Shioda, R.: Algorithm for cardinality-constrained quadratic optimization. Comput. Optim. Appl. 43(1), 1–22 (2009). https://doi.org/10.1007/S10589-007-9126-9
Bienstock, D.: Computational study of a family of mixed-integer quadratic programming problems. Math. Program. 74(2), 121–140 (1996). https://doi.org/10.1007/BF02592208
Birgin, E.G., Martínez, J.M.: Practical augmented Lagrangian methods for constrained optimization. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (2014). https://doi.org/10.1137/1.9781611973365
Branda, M., Bucher, M., Červinka, M., Schwartz, A.: Convergence of a scholtes-type regularization method for cardinality-constrained optimization problems with an application in sparse robust portfolio optimization. Comput. Optim. Appl. 70(2), 503–530 (2018). https://doi.org/10.1007/s10589-018-9985-2
Bucher, M., Schwartz, A.: Second-order optimality conditions and improved convergence results for regularization methods for cardinality-constrained optimization problems. J. Optim. Theory Appl. 178(2), 383–410 (2018). https://doi.org/10.1007/s10957-018-1320-7
Burdakov, O.P., Kanzow, C., Schwartz, A.: Mathematical programs with cardinality constraints: Reformulation by complementarity-type conditions and a regularization method. SIAM. J. Optim. 26(1), 397–425 (2016). https://doi.org/10.1137/140978077
Candes, E.J., Wakin, M.B.: An introduction to compressive sampling. IEEE Signal. Process. Mag. 25(2), 21–30 (2008). https://doi.org/10.1109/MSP.2007.914731
Dong, H., Ahn, M., Pang, J.-S.: Structural properties of affine sparsity constraints. Math. Program. 176(1), 95–135 (2019). https://doi.org/10.1007/S10107-018-1283-3
Dries, L.P.D.v.d.: Tame topology and O-minimal structures. London Mathematical Society Lecture Note Series, Cambridge University Press (1998). https://doi.org/10.1017/CBO9780511525919
Kanzow, C., Raharja, A.B., Schwartz, A.: An augmented Lagrangian method for cardinality-constrained optimization problems. J. Optim. Theory Appl. 189(3), 793–813 (2021). https://doi.org/10.1007/s10957-021-01854-7
Kanzow, C., Raharja, A.B., Schwartz, A.: Sequential optimality conditions for cardinality-constrained optimization problems with applications. Comput. Optim. Appl. 80(1), 185–211 (2021). https://doi.org/10.1007/s10589-021-00298-z
Ribeiro, A.A., Sachine, M., Krulikovski, E.H.M.: A comparative study of sequential optimality conditions for mathematical programs with cardinality constraints. J. Optim. Theory Appl. (2022). https://doi.org/10.1007/s10957-022-02007-0
Krulikovski, E.H.M., Ribeiro, A.A., Sachine, M.: On the weak stationarity conditions for mathematical programs with cardinality constraints: a unified approach. Appl Math Optim 84(3), 3451–3473 (2021). https://doi.org/10.1007/s00245-021-09752-0
Mangasarian, O.L.: Nonlinear programming. McGraw-Hill, New York (1969)
Miroforidis, J.: Bounds on efficient outcomes for large-scale cardinality-constrained markowitz problems. J. Glob. Optim. 80(3), 617–634 (2021). https://doi.org/10.1007/s10898-021-01022-1
Pang, L., Xue, M., Xu, N.: A new sequential optimality condition of cardinality-constrained optimization problem and application. (2021). https://arxiv.org/abs/2110.01220
Raharja, A.B.: Optimisation problems with sparsity terms: theory and algorithms. Thesis, University of Würzburg (2021). https://doi.org/10.25972/OPUS-24195
Rockafellar, R.T., Wets, R.J.-B.: Variational analysis. Springer (1998). https://doi.org/10.1007/978-3-642-02431-3
Sun, C., Dai, R., Mesbahi, M.: Weighted network design with cardinality constraints via alternating direction method of multipliers. IEEE Trans. Control. Netw. Syst. 5(4), 2073–2084 (2018). https://doi.org/10.1109/tcns.2018.2789726
Červinka, M., Kanzow, C., Schwartz, A.: Constraint qualifications and optimality conditions for optimization problems with cardinality constraints. Math. Program. 160(1–2), 353–377 (2016). https://doi.org/10.1007/s10107-016-0986-6
Acknowledgements
The authors would like to thank two anonymous reviewers for their suggestions on this paper, which were significant in improving the quality of this paper.
Funding
The research were partially supported by the Natural Science Foundation of Shandong Province with No. ZR2019BA014 and the Key Research and Development Projects of Shandong Province with No.2019GGX104089.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. The first draft of the manuscript was written by Menglong Xue and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix: A. Some auxiliary results
Appendix: A. Some auxiliary results
In this section, we show that CC-PCAM-stationarity is a necessary optimality condition for TNLP(x∗).
Theorem 17
Let x∗ be a local minimizer of TNLP(x∗), then x∗ is CC-PCAM-stationary.
Proof
We define the local problem for TNLP(x∗) as
where 𝜖 > 0 is such that x∗ is the only global minimizer of the problem (A1). We consider the partial penalty problem for TNLP(x∗)
where ρk > 0 and
The problem (A2) has a continuous objective function and a compact feasible set, then there exists a global minimizer, denoted as xk. Obviously, {xk} is bounded, and for simplicity, we set \(x^{k}\rightarrow \bar {x}\in B(x^{*},\epsilon )\) and let \(\rho _{k}\rightarrow \infty\). Then, for any k there is
Dividing both sides of (A3) by ρk and taking the limit yields \(F(\bar {x})=0\), which combined with \(\bar {x}\in B(x^{*},\epsilon )\) shows that \(\bar {x}\) is feasible for the problem (A1). Since x∗ is the global minimizer of the problem (A1), then \(f(x^{*})\leq f(\bar {x})\), and (A3) shows that
taking the limit yields \(f(\bar {x})+\frac{1}{2}\|\bar {x}-x^{*}\|^{2}\leq f(x^{*})\). Thus, \(\bar {x}=x^{*}\), i.e., \(x^{k}\rightarrow x^{*}\). For simplicity, we set \(\{x^{k}\}\subseteq int~B(x^{*},\epsilon )\).
From the necessary optimality condition for the problem (A2) we have
Let \(\lambda ^{k}=\rho _{k}g(x^{k})_{+},~\mu ^{k}=\rho _{k} h(x^{k}),~\gamma _{\imath }^{k}=\rho _{k}x_{\imath }^{k}~(\imath \in I_{0}(x^{*})),~\gamma _{\imath }^{k}=0~(\imath \notin I_{0}(x^{*}))\). Since \(x^{k}\rightarrow x^{*}\), we get
that is, the condition (a) of Definition 5 holds. Moreover, from (A3) and \(x^{k}\rightarrow x^{*}\), we know that \(\lim _{k\rightarrow \infty }\rho _{k} F(x^{k})=0\), then
If i∉Ig(x∗), then gi(xk) < 0 when k is sufficiently large, so \({\lambda _{i}^{k}}=0\). Conversely, if i ∈ Ig(x∗), then \(g_{i}(x^{k})\rightarrow 0\), similar to Theorem 3, only two cases of gi(xk) ↑ 0 and gi(xk) ↓ 0 need to be analyzed. When gi(xk) ↑ 0, we have gi(xk) < 0, so \({\lambda _{i}^{k}}=0\). If gi(xk) ↓ 0, then gi(xk)+ = gi(xk) and
Therefore, \({\lambda _{i}^{k}}g_{i}(x^{k})\rightarrow 0~(\forall i\in I_{g}(x^{*}))\), then \(\vert \lambda ^{k}\circ g(x^{k})\vert \rightarrow 0\). Combining (A4), we have
i. e., the condition (b) holds. By the definition of (λk,μk,γk), it is easy to verify that the conditions (c)-(e) holds using the method of Theorem 3. In summary, we prove that x∗ is CC-PCAM-stationary. □
Rights and permissions
About this article
Cite this article
Xue, M., Pang, L. A strong sequential optimality condition for cardinality-constrained optimization problems. Numer Algor 92, 1875–1904 (2023). https://doi.org/10.1007/s11075-022-01371-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-022-01371-2
Keywords
- Sequential optimality condition
- Cardinality constraints
- Constraint qualification
- Safeguarded augmented Lagrangian method