Skip to main content
SpringerLink
Log in

Iterative reweighted methods for \(\ell _1-\ell _p\) minimization

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

In this paper, we focus on the \(\ell _1-\ell _p\) minimization problem with \(0<p<1\), which is challenging due to the \(\ell _p\) norm being non-Lipschizian. In theory, we derive computable lower bounds for nonzero entries of the generalized first-order stationary points of \(\ell _1-\ell _p\) minimization, and hence of its local minimizers. In algorithms, based on three locally Lipschitz continuous \(\epsilon \)-approximation to \(\ell _p\) norm, we design several iterative reweighted \(\ell _1\) and \(\ell _2\) methods to solve those approximation problems. Furthermore, we show that any accumulation point of the sequence generated by these methods is a generalized first-order stationary point of \(\ell _1-\ell _p\) minimization. This result, in particular, applies to the iterative reweighted \(\ell _1\) methods based on the new Lipschitz continuous \(\epsilon \)-approximation introduced by Lu (Math Program 147(1–2):277–307, 2014), provided that the approximation parameter \(\epsilon \) is below a threshold value. Numerical results are also reported to demonstrate the efficiency of the proposed methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  2. Belloni, A., Chernozhukov, V.: L1-penalized quantile regression in high-dimensional sparse models. Ann. Stat. 39, 82–130 (2011)

    Article  MATH  Google Scholar 

  3. Bradic, J., Fan, J., Wang, W.: Penalized composite quasi-likelihood for ultrahigh-dimensional variable selection. J. R. Stat. Soc. Ser. B 73, 325–349 (2011)

    Article  MathSciNet  Google Scholar 

  4. Bruckstein, A., Donoho, D., Elad, A.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51, 34–81 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  5. Candès, E., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  6. Candès, E., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  7. Candès, E., Walkin, M., Boyd, S.: Enhancing sparsity by reweighted \(l_2\) minimization. J. Fourier Anal. Appl. 14(5–6), 877–905 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  8. Chen, X.: Smoothing methods for nonsmooth, nonconvex minimization. Math. Program. 134(1), 71–99 (2012)

  9. Chen, X., Ge, D., Wang, Z., Ye, Y.: Complexity of unconstrained \(L_2{-}L_p\) minimization. Math. Program. 143(1–2), 371–383 (2014)

  10. Chen, X., Xu, F., Ye, Y.: Lower bound theory of nonzero entries in solutions of \(\ell _2{-}\ell _p\) minimization. SIAM J. Sci. Comput. 32, 2832–2852 (2010)

  11. Chen, X., Ng, M., Zhang, C.: Nonconvex \(\ell _p\) regularization and box constrained model for image restoration. IEEE Trans. Image Process. 21, 4709–4721 (2012)

  12. Chen, X., Zhou, W.: Convergence of Reweighted \(\ell _1\) Minimization Algorithms and Unique Solution of Truncated \(\ell _p\) Minimization. The Hong Kong Polytechnic University, Department of Applied Mathematics (2010)

  13. Chen, X., Zhou, W.: Convergence of reweighted \(\ell _1\) minimization algorithm for \(\ell _2{-}\ell _p\) minimization. Comput. Optim. Appl. 59, 47–61 (2014)

  14. Daubechies, I., DeVore, R., Fornasier, M., Gunturk, C.: Iteratively reweighted least squares minimization for sparse recovery. Commun. Pure Appl. Math. 63, 1–38 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  15. Donoho, D.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  16. Eldar, Y., Kutyniok, G.: Compressed Sensing: Theory and Applications. Cambridge University Press, Cambridge (2012)

    Book  Google Scholar 

  17. Fan, Y., Fan, J., Barut, E.: Adaptive robust variable selection. Ann. Stat. 42(1), 324–351 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  18. Foucart, S., Lai, M.: Sparsest solutions of underdetermined linear systems via \(\ell _p\)-minimization for \(0 < q \le 1\). Appl. Comput. Harmon. Anal. 26, 395–407 (2009)

  19. Ge, D., Jiang, X., Ye, Y.: A note on the complexity of \(\ell _p\) minimization. Math. Program. 21, 1721–1739 (2011)

  20. Lai, M., Wang, J.: An unconstrained \(l_q\) minimization with \(0 < q \le 1\) for sparse solution of underdetermined linear systems. SIAM J. Optim. 21, 82–101 (2010)

  21. Lai, M., Xu, Y., Yin, W.: Improved iteratively reweighted least squares for unconstrained smoothed \(l_q\) minimization. SIAM J. Numer. Anal. 51(2), 927–957 (2013)

  22. Lu, Z.: Iterative reweighted minimization methods for \(\ell _p\) regularized unconstrained nonlinear programming. Math. Program. 147(1–2), 277–307 (2014)

  23. Mordukhovich, B., Nam, N.: An easy path to convex analysis and applications. Synth. Lect. Math. Stat. 6(2), 1–218 (2013)

    Article  Google Scholar 

  24. Wang, L.: L1 penalized LAD estimator for high dimensional linear regression. J. Multi. Anal. 120, 135–151 (2013)

    Article  MATH  Google Scholar 

  25. Wang, L., Wu, Y., Li, R.: Quantile regression for analyzing heterogeneity in ultrahigh dimension. J. Am. Stat. Assoc. 107, 214–222 (2012)

    Article  MATH  Google Scholar 

  26. Wright, S., Nowak, R., Figueiredo, M.: Sparse reconstruction by separable approximation. IEEE Trans. Image Process. 57, 2479–2493 (2009)

    Article  MathSciNet  Google Scholar 

  27. Xiu, X., Kong, L.: Rank-min-one and sparse tensor decomposition for surveillance video. Pac. J. Optim. 11(2), 403–418 (2015)

    MathSciNet  MATH  Google Scholar 

  28. Yin, W., Osher, S., Goldfarb, D., Darbonet, J.: Bregman iterative algorithms for \(\ell _1\)-minimization with applications to compressed sensing. SIAM J. Imaging Sci. 1(1), 143–168 (2008)

  29. Zhang, L., Li, Y., Kong, L., Zhou, S.: A smoothing iterative method for quantile regression with nonconvex \(\ell _p\) penalty. J. Ind. Manag. Optim. 13(1), 93–112 (2017)

Download references

Acknowledgements

The authors would like to thank the anonymous referees and the associate editor for their numerous insightful comments and suggestions, which have greatly improved the paper. In particular, we would like to thank a referee for clarifying the chain rule for computing the subdifferential of the \(\ell _1\) norm composited with a linear mapping. The work was supported in part by the National Natural Science Foundation of China (11431002, 11671029), the 111 Project of China (B16002), the Ministry of Education of Humanities and Social Science Project (17YJC910005), the Fundamental Research Funds for the Central Universities in UIBE (15YB04) and the Fundamental Research Funds for the Central Universities in BJTU (2016YJS159).

Author information

Authors and Affiliations

  1. Department of Applied Mathematics, Beijing Jiaotong University, Beijing, 100044, People’s Republic of China

    Xianchao Xiu & Lingchen Kong

  2. School of Insurance and Economics, University of International Business and Economics, Beijing, 100029, People’s Republic of China

    Yan Li

  3. School of Mathematics, University of Southampton, Southampton, UK

    Houduo Qi

Authors
  1. Xianchao Xiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lingchen Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Houduo Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianchao Xiu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiu, X., Kong, L., Li, Y. et al. Iterative reweighted methods for \(\ell _1-\ell _p\) minimization. Comput Optim Appl 70, 201–219 (2018). https://doi.org/10.1007/s10589-017-9977-7

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-017-9977-7

Keywords

Search

Navigation