Nonconvex Sorted \(\ell _1\) Minimization for Sparse Approximation
The \(\ell _1\) norm is the tight convex relaxation for the \(\ell _0\) norm and has been successfully applied for recovering sparse signals. However, for problems with fewer samples than required for accurate \(\ell _1\) recovery, one needs to apply nonconvex penalties such as \(\ell _p\) norm. As one method for solving \(\ell _p\) minimization problems, iteratively reweighted \(\ell _1\) minimization updates the weight for each component based on the value of the same component at the previous iteration. It assigns large weights on small components in magnitude and small weights on large components in magnitude. The set of the weights is not fixed, and it makes the analysis of this method difficult. In this paper, we consider a weighted \(\ell _1\) penalty with the set of the weights fixed, and the weights are assigned based on the sort of all the components in magnitude. The smallest weight is assigned to the largest component in magnitude. This new penalty is called nonconvex sorted \(\ell _1\). Then we propose two methods for solving nonconvex sorted \(\ell _1\) minimization problems: iteratively reweighted \(\ell _1\) minimization and iterative sorted thresholding, and prove that both methods will converge to a local minimizer of the nonconvex sorted \(\ell _1\) minimization problems. We also show that both methods are generalizations of iterative support detection and iterative hard thresholding, respectively. The numerical experiments demonstrate the better performance of assigning weights by sort compared to assigning by value.
KeywordsIteratively reweighted \(\ell _1\) minimization Iterative sorted thresholding Local minimizer Nonconvex optimization Sparse approximation
Mathematics Subject Classification49M37 65K10 90C26 90C52
The authors are grateful to the anonymous reviewers for their helpful comments.
- 3.Bogdan, M., van den Berg, E., Su, W., Candes, E.: Statistical estimation and testing via the sorted L1 norm. arXiv:1310.1969 [e-prints] (2013)
- 9.Chartrand, R.: Generalized shrinkage and penalty functions. In: IEEE Global Conference on Signal and Information Processing (2013)Google Scholar
- 10.Chartrand, R.: Shrinkage mappings and their induced penalty functions. In: IEEE International Conference on Acoustics, Speech and Signal Processing, Florence, 1026–1029 (2014)Google Scholar
- 11.Chartrand, R., Yin, W.: Iteratively reweighted algorithms for compressive sensing. In: IEEE International Conference on Acoustics, Speech and Signal Processing, Washington D.C, pp 3869–3872 (2008)Google Scholar
- 14.Chen, X., Zhou, W.: Convergence of the reweighted \(\ell _1\) minimization algorithm for \(\ell _2\)-\(\ell _p\) minimization. Comput. Optim. Appl. 1–2(59), 47–61 (2013)Google Scholar
- 21.Foucart, S., Lai, M.J.: Sparsest solutions of underdetermined linear systems via \(\ell _q\)-minimization for \(0 < q \leqslant 1\). Appl. Comput. Harmon. Anal. 26(3), 395–407 (2009)Google Scholar
- 24.Goldberger, A.L., Amaral, L.A., Glass, L., Hausdorff, J.M., Ivanov, P.C., Mark, R.G., Mietus, J.E., Moody, G.B., Peng, C.K., Stanley, H.E.: Physiobank, physiotoolkit, and physionet components of a new research resource for complex physiologic signals. Circulation 101(23), e215–e220 (2000)CrossRefGoogle Scholar
- 36.Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, T., Yan, S.: Sparse representation for computer vision and pattern recognition. Proc. IEEE 98(6), 1031–1044 (2010)Google Scholar
- 39.Yin, P., Lou, Y., He, Q., Xin, J.: Minimizaiton of \(\ell _1-\ell _2\) for compressed sensing. CAM-report 14–01, UCLA (2014)Google Scholar