A superlinearly convergent R-regularized Newton scheme for variational models with concave sparsity-promoting priors
- 278 Downloads
A general class of variational models with concave priors is considered for obtaining certain sparse solutions, for which nonsmoothness and non-Lipschitz continuity of the objective functions pose significant challenges from an analytical as well as numerical point of view. For computing a stationary point of the underlying variational problem, a Newton-type scheme with provable convergence properties is proposed. The possible non-positive definiteness of the generalized Hessian is handled by a tailored regularization technique, which is motivated by reweighting as well as the classical trust-region method. Our numerical experiments demonstrate selected applications in image processing, support vector machines, and optimal control of partial differential equations.
KeywordsSparsity Concave priors Nonconvex minimization Semismooth Newton method Superlinear convergence
This research was supported by the Austrian Science Fund (FWF) through START project Y305 “Interfaces and Free Boundaries” and through SFB project F3204 “Mathematical Optimization and Applications in Biomedical Sciences”. The authors would like to thank T. Pock (TU Graz) for communication on the problem concerning the overcomplete dictionary. We also thank the referee for the pointer to a recent thesis work , which contains an extensive algorithmic study of nonconvex-regularization based sparse optimization.
- 1.ℓ_1-magic. http://users.ece.gatech.edu/~justin/l1magic/
- 2.Attouch, H., Buttazzo, G., Michaille, G.: Variational Analysis in Sobolev and BV Spaces: Applications to PDEs and Optimization. SIAM, Philadelphia (2006) Google Scholar
- 7.Candès, E.J., Tao, T.: Near optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inf. Theory 52, 5406–5425 (2006) Google Scholar
- 10.Charbonnier, P., Blanc-Féraud, L., Aubert, G., Barlaud, M.: Deterministic edge-preserving regularization in computed imaging. IEEE Trans. Image Process. 6, 298–311 (1997) Google Scholar
- 11.Chartrand, R., Yin, W.: Iteratively reweighted algorithms for compressive sensing. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3869–3872 (2008) Google Scholar
- 21.Geman, D., Reynolds, G.: Constrained restoration and the recovery of discontinuities. IEEE Trans. Pattern Anal. Mach. Intell. 14, 367–383 (1992) Google Scholar
- 22.Hintermüller, M., Wu, T.: Nonconvex TVq-models in image restoration: analysis and a trust-region regularization based superlinearly convergent solver. SIAM J. Imaging Sci. (to appear) Google Scholar
- 25.Huber, P.J.: Robust estimation of a location parameter. Ann. Math. Stat. 53, 73–101 (1964) Google Scholar
- 28.Lin, Q.: Sparsity and nonconvex nonsmooth optimization. Ph.D. thesis, University of Washington (2009) Google Scholar
- 38.Weston, J., Mukherjee, S., Chapelle, O., Pontil, M., Poggio, T., Vapnik, V.: Feature selection for SVMs. Adv. Neural Inf. Process. Syst. 13, 668–674 (2000) Google Scholar