Mathematical Programming

, Volume 143, Issue 1–2, pp 371–383 | Cite as

Complexity of unconstrained \(L_2-L_p\) minimization

Short Communication Series A

Abstract

We consider the unconstrained \(L_q\)-\(L_p\) minimization: find a minimizer of \(\Vert Ax-b\Vert ^q_q+\lambda \Vert x\Vert ^p_p\) for given \(A \in R^{m\times n}\), \(b\in R^m\) and parameters \(\lambda >0\), \(p\in [0, 1)\) and \(q\ge 1\). This problem has been studied extensively in many areas. Especially, for the case when \(q=2\), this problem is known as the \(L_2-L_p\) minimization problem and has found its applications in variable selection problems and sparse least squares fitting for high dimensional data. Theoretical results show that the minimizers of the \(L_q\)-\(L_p\) problem have various attractive features due to the concavity and non-Lipschitzian property of the regularization function \(\Vert \cdot \Vert ^p_p\). In this paper, we show that the \(L_q\)-\(L_p\) minimization problem is strongly NP-hard for any \(p\in [0,1)\) and \(q\ge 1\), including its smoothed version. On the other hand, we show that, by choosing parameters \((p,\lambda )\) carefully, a minimizer, global or local, will have certain desired sparsity. We believe that these results provide new theoretical insights to the studies and applications of the concave regularized optimization problems.

Keywords

Nonsmooth optimization Nonconvex optimization Variable selection Sparse solution reconstruction Bridge estimator 

Mathematics Subject Classification (2010)

90C26 90C51 

References

  1. 1.
    Chartrand, R.: Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Process. Lett. 14, 707–710 (2007)CrossRefGoogle Scholar
  2. 2.
    Chartrand, R., Staneva, V.: Restricted isometry properties and nonconvex compressive sensing. Inverse Probl. 24, 1–14 (2008)CrossRefMathSciNetGoogle Scholar
  3. 3.
    Chen, X., Xu, F., Ye, Y.: Lower bound theory of nonzero entries in solutions of \(\ell _2-\ell _p\) minimization. SIAM J. Sci. Comput. 32, 2832–2852 (2010)CrossRefMATHMathSciNetGoogle Scholar
  4. 4.
    Chen, X., Zhou, W.: Convergence of reweighted \(l_1\) minimization algorithms and unique solution of truncated \(l_p\) minimization. Department of Applied Mathematics, The Hong Kong Polytechnic University, Preprint (2010)Google Scholar
  5. 5.
    Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Soc. 96, 1348–1360 (2001)CrossRefMATHMathSciNetGoogle Scholar
  6. 6.
    Foucart, S., Lai, M.J.: Sparsest solutions of under-determined linear systems via \(l_q\) minimization for \(0<q\le 1\). Appl. Comput. Harmon. Anal. 26, 395–407 (2009)CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Frank, I.E., Freidman, J.H.: A statistical view of some chemometrics regression tools (with discussion). Technometrics 35, 109–148 (1993)CrossRefMATHGoogle Scholar
  8. 8.
    Garey, M.R., Johnson, D.S.: “Strong” NP-Completeness results: motivation, examples, and implications. J. Assoc. Comput. Mach. 25, 499–508 (1978)CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness. W. H. Freeman, New York (1979)MATHGoogle Scholar
  10. 10.
    Ge, D., Jiang, X., Ye, Y.: A note on the complexity of \(L_p\) minimization. Math. Program. 129, 285–299 (2011)CrossRefMATHMathSciNetGoogle Scholar
  11. 11.
    Huang, J., Horowitz, J.L., Ma, S.: Asymptotic properties of bridge estimators in sparse high-dimensional regression models. Ann. Stat. 36, 587–613 (2008)CrossRefMATHMathSciNetGoogle Scholar
  12. 12.
    Knight, K., Fu, W.J.: Asymptotics for lasso-type estimators. Ann. Stat. 28, 1356–1378 (2000)CrossRefMATHMathSciNetGoogle Scholar
  13. 13.
    Lai, M., Wang, Y.: An unconstrained \(l_q\) minimization with \(0 < q< 1\) for sparse solution of under-determined linear systems. SIAM J. Optim. 21, 82–101 (2011)CrossRefMATHMathSciNetGoogle Scholar
  14. 14.
    Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24, 227–234 (1995)CrossRefMATHMathSciNetGoogle Scholar
  15. 15.
    Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)MATHGoogle Scholar
  16. 16.
    Tibshirani, R.: Regression shrinkage and selection via the Lasso. J R. Stat. Soc. B 58, 267–288 (1996)MATHMathSciNetGoogle Scholar
  17. 17.
    Vazirani, V.: Approximation Algorithms. Springer, Berlin (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2012

Authors and Affiliations

  • Xiaojun Chen
    • 1
  • Dongdong Ge
    • 2
  • Zizhuo Wang
    • 3
  • Yinyu Ye
    • 4
  1. 1.Department of Applied MathematicsThe Hong Kong Polytechnic UniversityHong KongChina
  2. 2.Antai School of Economics and ManagementShanghai Jiao Tong UniversityShanghaiChina
  3. 3.Department of Industrial and System EngineeringUniversity of MinnesotaMinneapolisUSA
  4. 4.Department of Management Science and EngineeringStanford UniversityStanfordUSA

Personalised recommendations