Abstract
In this work, we consider unconstrained nonlinear optimization problems where the objective function presents a penalty term on the cardinality of a subset of the variables vector; specifically, we prove that an alternate minimization scheme has global asymptotic convergence guarantees towards points satisfying first-order optimality conditions, even when the optimization step with respect to one of the blocks of variables is inexact and without introducing proximal terms. This result, supported by numerical evidence, justifies the use of pure alternate minimization in applications, even in absence of convexity assumptions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
No datasets were generated or analyzed during the current study.
References
Blumensath T, Davies ME (2009) Iterative hard thresholding for compressed sensing. Appl Comput Harmon Anal 27(3):265–274
Foucart S, Rauhut H (2013) An invitation to compressive sensing. In: A Mathematical Introduction to Compressive Sensing. Springer, pp 1–39
Bach F, Jenatton R, Mairal J, Obozinski G (2011) Optimization with sparsity-inducing penalties. Preprint at http://arxiv.org/abs/1108.0775
Weston J, Elisseeff A, Schölkopf B, Tipping M (2003) Use of the zero norm with linear models and kernel methods. J Mach Learn Res 3:1439–1461
Bertsimas D, King A, Mazumder R et al (2016) Best subset selection via a modern optimization lens. Ann Stat 44(2):813–852
Civitelli E, Lapucci M, Schoen F, Sortino A (2021) An effective procedure for feature subset selection in logistic regression based on information criteria. Comput Optim Appl 1–32
Di Gangi L, Lapucci M, Schoen F, Sortino A (2019) An efficient optimization approach for best subset selection in linear regression, with application to model selection and fitting in autoregressive time-series. Comput Optim Appl 74(3):919–948
Tillmann AM, Bienstock D, Lodi A, Schwartz A (2021) Cardinality minimization, constraints, and regularization: a survey. Preprint at http://arxiv.org/abs/2106.09606
Bertsekas D, Tsitsiklis J (2015) Parallel and distributed computation: Numerical methods. Athena Scientific
Grippo L, Sciandrone M (1999) Globally convergent block-coordinate techniques for unconstrained optimization. Optimization methods and software 10(4):587–637
Lu Z, Zhang Y (2013) Sparse approximation via penalty decomposition methods. SIAM J Optim 23(4):2448–2478
Bolte J, Sabach S, Teboulle M (2014) Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math Program 146(1):459–494
Bertsekas DP, Hager W, Mangasarian O (1999) Nonlinear programming. Athena Scientific Belmont: Massachusets, USA
Bertsekas DP (1997) Nonlinear programming. J Oper Res Soc 48(3):334–334
Kanzow C, Lapucci M (2023) Inexact penalty decomposition methods for optimization problems with geometric constraints. Comput Optim Appl 1–35
Lapucci M, Levato T, Sciandrone M (2021) Convergent inexact penalty decomposition methods for cardinality-constrained problems. J Optim Theory Appl 188(2):473–496
Bishop CM et al (1995) Neural networks for pattern recognition. Oxford University Press
Fasshauer GE (2007) Meshfree approximation methods with MATLAB, vol. 6. World Scientific
Poggio T, Girosi F (1990) Networks for approximation and learning. Proc IEEE 78(9):1481–1497
Wendland H (2004) Scattered data approximation, vol. 17. Cambridge University Press
Buzzi C, Grippo L, Sciandrone M (2001) Convergent decomposition techniques for training RBF neural networks. Neural Comput 13(8):1891–1920
Gould NI, Orban D, Toint PL (2015) Cutest: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput Optim Appl 60:545–557
Liu DC, Nocedal J (1989) On the limited memory bfgs method for large scale optimization. Math Program 45(1):503–528
Author information
Authors and Affiliations
Contributions
M.L. devised the paper concept and carried out the theoretical analysis; both authors carried out the literature review, designed the algorithmic scheme, identified applications, carried out numerical experiments, and wrote the manuscript.
Corresponding author
Ethics declarations
Competing Interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Lapucci, M., Sortino, A. On the Convergence of Inexact Alternate Minimization in Problems with \(\ell _0\) Penalties. Oper. Res. Forum 5, 41 (2024). https://doi.org/10.1007/s43069-024-00323-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s43069-024-00323-x