Iteratively weighted thresholding homotopy method for the sparse solution of underdetermined linear equations

Science China Mathematics volume 64, pages639–664(2021)Cite this article

Abstract

Recently, iteratively reweighted methods have attracted much interest in compressed sensing, outperforming their unweighted counterparts in most cases. In these methods, decision variables and weights are optimized alternatingly, or decision variables are optimized under heuristically chosen weights. In this paper, we present a novel weighted l1-norm minimization problem for the sparsest solution of underdetermined linear equations. We propose an iteratively weighted thresholding method for this problem, wherein decision variables and weights are optimized simultaneously. Furthermore, we prove that the iteration process will converge eventually. Using the homotopy technique, we enhance the performance of the iteratively weighted thresholding method. Finally, extensive computational experiments show that our method performs better in terms of both running time and recovery accuracy compared with some state-of-the-art methods.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

References

  1. 1

    Beck A, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci, 2009, 2: 183–202

    MathSciNet  Article  Google Scholar 

  2. 2

    Bi S, Liu X, Pan S. Exact penalty decomposition method for zero-norm minimization based on MPEC formulation. SIAM J Sci Comput, 2014, 36: 1451–1477

    MathSciNet  Article  Google Scholar 

  3. 3

    Blumensath T, Davies M E. Iterative thresholding for sparse approximations. J Fourier Anal Appl, 2008, 14: 629–654

    MathSciNet  Article  Google Scholar 

  4. 4

    Candès E J, Romberg J, Tao T. Stable signal recovery from incomplete and inaccurate measurements. Comm Pure Appl Math, 2006, 59: 1207–1223

    MathSciNet  Article  Google Scholar 

  5. 5

    Candès E J, Wakin M B, Boyd S P. Enhancing sparsity by reweighted l1 minimization. J Fourier Anal Appl, 2008, 14: 877–905

    MathSciNet  Article  Google Scholar 

  6. 6

    Chartrand R, Yin W. Iteratively reweighted algorithms for compressive sensing. In: IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2008: 3869–3872

    Google Scholar 

  7. 7

    Chen X, Zhou W. Convergence of reweighted l1 minimization algorithm for l2-lp minimization. Comput Optim Appl, 2014, 59: 47–61

    MathSciNet  Article  Google Scholar 

  8. 8

    Chen Y C, Wang M D. Worst-case hardness of approximation for sparse optimization with L0 norm. http://www.optimization-online.org/DBFILE/2016/02/5334.pdf, 2016

  9. 9

    Dai W, Milenkovic O. Subspace pursuit for compressive sensing signal reconstruction. IEEE Trans Inform Theory, 2009, 55: 2230–2249

    MathSciNet  Article  Google Scholar 

  10. 10

    Donoho D L, Tsaig Y, Drori I, et al. Sparse solution of underdetermined systems of linear equations by stagewise orthogonal matching pursuit. IEEE Trans Inform Theory, 2012, 58: 1094–1121

    MathSciNet  Article  Google Scholar 

  11. 11

    Foucart S, Lai M. Sparsest solutions of underdetermined linear systems via lq-minimization for 0 < q ⩾ 1. Appl Comput Harmon Anal, 2009, 26: 395–407

    MathSciNet  Article  Google Scholar 

  12. 12

    Foucat S, Rauhut H. A Mathematical Introduction to Compressive Sensing, Chapter 9: Sparse Recovery with Random Matrices. Basel: Birkhäuser, 2013

    Book  Google Scholar 

  13. 13

    Hale E T, Yin W, Zhang Y. Fixed-point continuation for l1-minimization: Methodology and convergence. SIAM J Optim, 2008, 19: 1107–1130

    MathSciNet  Article  Google Scholar 

  14. 14

    Jiao Y, Jin B, Lu X. A primal dual active set with continuation algorithm for the l0-regularized optimization problem. Appl Comput Harmon Anal, 2015, 39: 400–426

    MathSciNet  Article  Google Scholar 

  15. 15

    Khajehnejad M A, Xu W, Avestimehr A S, et al. Analyzing weighted l1-norm minimization for sparse recovery with nonuniform sparse models. IEEE Trans Signal Process, 2011, 59: 1985–2001

    MathSciNet  Article  Google Scholar 

  16. 16

    Lu Z. Iterative hard thresholding methods for l0 regularized convex cone programming. Math Program, 2014, 147: 125–154

    MathSciNet  Article  Google Scholar 

  17. 17

    Lu Z, Zhang Y. Sparse approximation via penalty decomposition methods. SIAM J Optim, 2012, 23: 2448–2478

    MathSciNet  Article  Google Scholar 

  18. 18

    Mallat S G, Zhang Z. Matching pursuits with time-frequency dictionaries. IEEE Trans Signal Process, 1993, 41: 3397–3415

    Article  Google Scholar 

  19. 19

    Natarajan B K. Sparse approximate solutions to linear systems. SIAM J Comput, 1995, 24: 227–234

    MathSciNet  Article  Google Scholar 

  20. 20

    Needell D, Tropp J A. CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. Appl Comput Harmon Anal, 2009, 26: 301–321

    MathSciNet  Article  Google Scholar 

  21. 21

    Needell D, Vershynin R. Uniform uncertainty principle and signal recovery via regularized orthogonal matching pursuit. Found Comput Math, 2009, 9: 317–334

    MathSciNet  Article  Google Scholar 

  22. 22

    Rao B D, Kreutz-Delgado K. An affine scaling methodology for best basis selection. IEEE Trans Signal Process, 1999, 47: 187–200

    MathSciNet  Article  Google Scholar 

  23. 23

    Tropp J A, Gilbert A C. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans Inform Theory, 2007, 53: 4655–4666

    MathSciNet  Article  Google Scholar 

  24. 24

    Wang Y, Yin W. Sparse signal reconstruction via iterative support detection. SIAM J Imaging Sci, 2010, 3: 462–491

    MathSciNet  Article  Google Scholar 

  25. 25

    Wen Z, Yin W, Goldfard D, et al. A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization, and continuation. SIAM J Sci Comput, 2010, 32: 1832–1857

    MathSciNet  Article  Google Scholar 

  26. 26

    Xiao L, Zhang T. A proximal-gradient homotopy method for the sparse least-squares problem. SIAM J Optim, 2013, 23: 1062–1091

    MathSciNet  Article  Google Scholar 

  27. 27

    Xu W, Khajehnejad M A, Avestimehr A S, et al. Breaking through the thresholds: An analysis for iterative reweighted l1 minimization via the Grassmann angle framework. In: IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2010, 1–5

    Google Scholar 

  28. 28

    Xu Z, Chang X, Xu F, et al. L1/2 regularization: A thresholding representation theory and a fast solver. IEEE Trans Neural Netw Learn Syst, 2012, 23: 1013–1027

    Article  Google Scholar 

  29. 29

    Yang J, Zhang Y. Alternating direction algorithms for l1-problems in compressive sensing. SIAM J Sci Comput, 2011, 33: 250–278

    MathSciNet  Article  Google Scholar 

  30. 30

    Zhao Y B, Li D. Reweighted l1-minimization for sparse solutions to underdetermined linear systems. SIAM J Optim, 2012, 22: 1065–1088

    MathSciNet  Article  Google Scholar 

  31. 31

    Zhao Y B, Kočvara M. A new computational method for the sparsest solutions to systems of linear equations. SIAM J Optim, 2015, 25: 1110–1134

    MathSciNet  Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant Nos. 61672005 and 11571074). The authors thank the anonymous reviewers for their helpful suggestions and comments, which helped improve the quality of this manuscript.

Author information

Affiliations

  1. Center for Discrete Mathematics and Theoretical Computer Science, Fuzhou University, Fuzhou, 350116, China

    Wenxing Zhu, Zilin Huang, Jianli Chen & Zheng Peng

Authors
  1. Wenxing Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zilin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianli Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zheng Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenxing Zhu.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhu, W., Huang, Z., Chen, J. et al. Iteratively weighted thresholding homotopy method for the sparse solution of underdetermined linear equations. Sci. China Math. 64, 639–664 (2021). https://doi.org/10.1007/s11425-018-9467-7

Download citation

Keywords

  • sparse optimization
  • weighted thresholding method
  • homotopy method

MSC(2010)

  • 15A06
  • 15A29
  • 65K05
  • 90C25
  • 90C26
  • 90C59