Complexity of unconstrained \(L_2-L_p\) minimization Short Communication Series A First Online: 30 November 2012 Received: 14 May 2011 Accepted: 14 November 2012 DOI :
10.1007/s10107-012-0613-0

Cite this article as: Chen, X., Ge, D., Wang, Z. et al. Math. Program. (2014) 143: 371. doi:10.1007/s10107-012-0613-0
21
Citations
930
Downloads
Abstract We consider the unconstrained \(L_q\) -\(L_p\) minimization: find a minimizer of \(\Vert Ax-b\Vert ^q_q+\lambda \Vert x\Vert ^p_p\) for given \(A \in R^{m\times n}\) , \(b\in R^m\) and parameters \(\lambda >0\) , \(p\in [0, 1)\) and \(q\ge 1\) . This problem has been studied extensively in many areas. Especially, for the case when \(q=2\) , this problem is known as the \(L_2-L_p\) minimization problem and has found its applications in variable selection problems and sparse least squares fitting for high dimensional data. Theoretical results show that the minimizers of the \(L_q\) -\(L_p\) problem have various attractive features due to the concavity and non-Lipschitzian property of the regularization function \(\Vert \cdot \Vert ^p_p\) . In this paper, we show that the \(L_q\) -\(L_p\) minimization problem is strongly NP-hard for any \(p\in [0,1)\) and \(q\ge 1\) , including its smoothed version. On the other hand, we show that, by choosing parameters \((p,\lambda )\) carefully, a minimizer, global or local, will have certain desired sparsity. We believe that these results provide new theoretical insights to the studies and applications of the concave regularized optimization problems.

Keywords Nonsmooth optimization Nonconvex optimization Variable selection Sparse solution reconstruction Bridge estimator

References 1.

Chartrand, R.: Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Process. Lett.

14 , 707–710 (2007)

CrossRef Google Scholar 2.

Chartrand, R., Staneva, V.: Restricted isometry properties and nonconvex compressive sensing. Inverse Probl.

24 , 1–14 (2008)

CrossRef MathSciNet Google Scholar 3.

Chen, X., Xu, F., Ye, Y.: Lower bound theory of nonzero entries in solutions of

\(\ell _2-\ell _p\) minimization. SIAM J. Sci. Comput.

32 , 2832–2852 (2010)

CrossRef MATH MathSciNet Google Scholar 4.

Chen, X., Zhou, W.: Convergence of reweighted \(l_1\) minimization algorithms and unique solution of truncated \(l_p\) minimization. Department of Applied Mathematics, The Hong Kong Polytechnic University, Preprint (2010)

5.

Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Soc.

96 , 1348–1360 (2001)

CrossRef MATH MathSciNet Google Scholar 6.

Foucart, S., Lai, M.J.: Sparsest solutions of under-determined linear systems via

\(l_q\) minimization for

\(0<q\le 1\) . Appl. Comput. Harmon. Anal.

26 , 395–407 (2009)

CrossRef MATH MathSciNet Google Scholar 7.

Frank, I.E., Freidman, J.H.: A statistical view of some chemometrics regression tools (with discussion). Technometrics

35 , 109–148 (1993)

CrossRef MATH Google Scholar 8.

Garey, M.R., Johnson, D.S.: “Strong” NP-Completeness results: motivation, examples, and implications. J. Assoc. Comput. Mach.

25 , 499–508 (1978)

CrossRef MATH MathSciNet Google Scholar 9.

Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness. W. H. Freeman, New York (1979)

MATH Google Scholar 10.

Ge, D., Jiang, X., Ye, Y.: A note on the complexity of

\(L_p\) minimization. Math. Program.

129 , 285–299 (2011)

CrossRef MATH MathSciNet Google Scholar 11.

Huang, J., Horowitz, J.L., Ma, S.: Asymptotic properties of bridge estimators in sparse high-dimensional regression models. Ann. Stat.

36 , 587–613 (2008)

CrossRef MATH MathSciNet Google Scholar 12.

Knight, K., Fu, W.J.: Asymptotics for lasso-type estimators. Ann. Stat.

28 , 1356–1378 (2000)

CrossRef MATH MathSciNet Google Scholar 13.

Lai, M., Wang, Y.: An unconstrained

\(l_q\) minimization with

\(0 < q< 1\) for sparse solution of under-determined linear systems. SIAM J. Optim.

21 , 82–101 (2011)

CrossRef MATH MathSciNet Google Scholar 14.

Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Comput.

24 , 227–234 (1995)

CrossRef MATH MathSciNet Google Scholar 15.

Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)

MATH Google Scholar 16.

Tibshirani, R.: Regression shrinkage and selection via the Lasso. J R. Stat. Soc. B

58 , 267–288 (1996)

MATH MathSciNet Google Scholar 17.

Vazirani, V.: Approximation Algorithms. Springer, Berlin (2003)

CrossRef Google Scholar © Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2012

Authors and Affiliations 1. Department of Applied Mathematics The Hong Kong Polytechnic University Hong Kong China 2. Antai School of Economics and Management Shanghai Jiao Tong University Shanghai China 3. Department of Industrial and System Engineering University of Minnesota Minneapolis USA 4. Department of Management Science and Engineering Stanford University Stanford USA