Entropy Function-Based Algorithms for Solving a Class of Nonconvex Minimization Problems

  • Yu-Fan Li
  • Zheng-Hai HuangEmail author
  • Min Zhang


Recently, the \(l_p\) minimization problem (\(p\in (0,\,1)\)) for sparse signal recovery has been studied a lot because of its efficiency. In this paper, we propose a general smoothing algorithmic framework based on the entropy function for solving a class of \(l_p\) minimization problems, which includes the well-known unconstrained \(l_2\)\(l_p\) problem as a special case. We show that any accumulation point of the sequence generated by the proposed algorithm is a stationary point of the \(l_p\) minimization problem, and derive a lower bound for the nonzero entries of the stationary point of the smoothing problem. We implement a specific version of the proposed algorithm which indicates that the entropy function-based algorithm is effective.


\(l_p\) minimization problem Entropy function Smoothing conjugate gradient method 

Mathematics Subject Classification

65K05 90C26 90C59 


  1. 1.
    Eldar, Y.C., Kutyniok, G.: Compressed Sensing: Theory and Applications. Cambridge University Press, Cambridge (2012)CrossRefGoogle Scholar
  2. 2.
    Chen, X., Ge, D., Wang, Z., Ye, Y.: Complexity of unconstrained \(L_2\)-\(L_p\) minimization. Math. Program. A 143, 371–383 (2014)zbMATHMathSciNetCrossRefGoogle Scholar
  3. 3.
    Ge, D., Jiang, X., Ye, Y.: A note on complexity of \(L_p\) minimization. Math. Program. 129, 285–299 (2011)zbMATHMathSciNetCrossRefGoogle Scholar
  4. 4.
    Chartrand, R.: Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Process. Lett. 14, 707–710 (2007)CrossRefGoogle Scholar
  5. 5.
    Chartrand, R., Staneva, V.: Restricted isometry properties and nonconvex compressive sensing. Inverse Probl. 24, 035020 (2008)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Foucart, S., Lai, M.: Sparsest solutions of underdetermined linear systems via \(l_q\) minimization for \(0 < q {\leqslant }\,1\). Appl. Comput. Harmon. Anal. 26, 26–407 (2009)Google Scholar
  7. 7.
    Sun, Q.: Recovery of sparsest signals via \(l_q\) minimization. Appl. Comput. Harmon. Anal. 32, 329–341 (2012)zbMATHMathSciNetCrossRefGoogle Scholar
  8. 8.
    Candès, E., Wakin, M., Boyd, S.: Enhancing sparsity by reweighted \(l_1\) minimization. J. Fourier Anal. Appl. 14, 877–905 (2008)zbMATHMathSciNetCrossRefGoogle Scholar
  9. 9.
    Gasso, G., Rakotomamonjy, A., Canu, S.: Recovering sparse signals with a certain family of nonconvex penalties and DC programming. IEEE Trans. Signal Process. 57, 4686–4698 (2009)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Zhao, Y., Li, D.: Reweighted \(l_1\)-minimization for sparse solutions to underdetermined linear systems. SIAM J. Optim. 22, 1065–1088 (2012)zbMATHMathSciNetCrossRefGoogle Scholar
  11. 11.
    Chartrand, R., Yin, W.: Iteratively reweighted algorithms for compressive sensing. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3869–3872 (2008)Google Scholar
  12. 12.
    Mourad, N., Reilly, J.P.: \(l_p\) minimization for sparse vector reconstruction. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 3345–3348 (2009)Google Scholar
  13. 13.
    Rao, B.D., Kreutz-Delgado, K.: An affine scaling methodology for best basis selection. IEEE Trans. Signal Process. 47, 187–200 (1999)zbMATHMathSciNetCrossRefGoogle Scholar
  14. 14.
    Xu, Z., Chang, X., Xu, F., Zhang, H.: \(L_{\frac{1}{2}}\) regularization: a thresholding representation theory and a fast solver. IEEE Trans. Neural Netw. Learn. Syst. 23, 1013–1027 (2012)CrossRefGoogle Scholar
  15. 15.
    Xu, Z.B., Zhang, H., Wang, Y., Chang, X.Y., Liang, Y.: \(L_{\frac{1}{2}}\) regularization. Sci. China F 53, 1159–1169 (2010)MathSciNetCrossRefGoogle Scholar
  16. 16.
    She, Y.: Thresholding-based iterative selection procedures for model selection and shrinkage. Electron. J. Stat. 3, 384–415 (2009)zbMATHMathSciNetCrossRefGoogle Scholar
  17. 17.
    She, Y.: An iterative algorithm for fitting nonconvex penalized generalized linear models with grouped predictors. Comput. Stat. Data Anal. 9, 2976–2990 (2012)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Chen, X., Zhou, W.: Convergence of reweighted \(l_1\) minimization algorithms and unique solution of truncated \(l_p\) minimization. Comput. Optim. Appl. 59, 47–61 (2014)zbMATHMathSciNetCrossRefGoogle Scholar
  19. 19.
    Lai, M.-J., Wang, J.: An unconstrained \(l_q\) minimization with \(0<q{\leqslant }\, 1\) for sparse solution of underdetermined linear systems. SIAM J. Optim. 21, 82–101 (2011)Google Scholar
  20. 20.
    Lai, M.-J., Xu, Y., Yin, W.: Improved iteratively reweighted least squares for unconstrained smoothed \(l_q\) minimization. SIAM J. Numer. Anal. 5, 927–957 (2013)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Chen, X., Xu, F., Ye, Y.: Lower bound theory of nonzero entries in solutions of \(l_2\)-\(l_p\) minimization. SIAM J. Sci. Comput. 32, 2832–2852 (2010)zbMATHMathSciNetCrossRefGoogle Scholar
  22. 22.
    Lu, Z.: Iterative reweighted minimization methods for \(l_p\) regularized unconstrained nonlinear programming. Math. Program. 147, 277–307 (2014)zbMATHMathSciNetCrossRefGoogle Scholar
  23. 23.
    Kort, B.W., Bertsekas, D.P.: A new penalty function for constrained minimization. In: Proceedings of the 1972 IEEE Conference on Decision and Control and 11th Symposium on Adaptive Processes, pp. 162–166 (1972)Google Scholar
  24. 24.
    Li, X.S.: An aggregate function method for nonlinear programming. Sci. China A 34, 1467–1473 (1991)zbMATHGoogle Scholar
  25. 25.
    Bertsekas, D.P.: Approximation procedures based on the method of multipliers. J. Optim. Theory Appl. 23, 487–510 (1977)zbMATHMathSciNetCrossRefGoogle Scholar
  26. 26.
    Chang, P.L.: A minimax approach to nonlinear programming. PhD Dissertation, Department of Mathematics, University of Washington, Seattle (1980)Google Scholar
  27. 27.
    Fang, S.-C., Han, J., Huang, Z.H., Ilker Birbil, S.: On the finite termination of an entropy function based non-interior continuation method for vertical linear complementarity problems. J. Glob. Optim. 33, 369–391 (2005)Google Scholar
  28. 28.
    Goldstein, A.A.: Chebyshev Approximation and Linear Inequalities Via Exponentials. Department of Mathematics, University of Washington, Technical Report. Seattle (1997)Google Scholar
  29. 29.
    Tseng, P., Bertsekas, D.P.: On the convergence of the exponential multiplier method for convex programming. Math. Program. 60, 1–19 (1993)zbMATHMathSciNetCrossRefGoogle Scholar
  30. 30.
    Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2, 183–202 (2009)zbMATHMathSciNetCrossRefGoogle Scholar
  31. 31.
    Chen, X., Zhou, W.: Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 3, 765–790 (2010)zbMATHMathSciNetCrossRefGoogle Scholar
  32. 32.
    Lyu, Q., Lin, Z., She, Y., Zhang, C.: A comparison of typical \(l_p\) minimization algorithms. Neurocomputing 119, 413–424 (2013)CrossRefGoogle Scholar

Copyright information

© Operations Research Society of China, Periodicals Agency of Shanghai University, Science Press, and Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Department of Mathematics, School of ScienceTianjin UniversityTianjinChina
  2. 2.Center for Applied Mathematics of Tianjin UniversityTianjinChina

Personalised recommendations