Nonnegative estimation and variable selection under minimax concave penalty for sparse high-dimensional linear regression models

  • Ning Li
  • Hu YangEmail author
Regular Article


As a promising alternative to the Lasso penalty, the minimax concave penalty (MCP) produces nearly unbiased shrinkage estimate. In this paper, we propose the nonnegative MCP estimator in the specific high-dimensional settings where the regression coefficients are imposed with the nonnegativity constraint, which is particularly relevant when modelling nonnegative data. We prove that the asymptotic theory for the nonnegative MCP estimator requires much weaker assumptions than those in the literature. In particular, we do not impose the restrictive distributional conditions on the random errors and do not assume the irrepresentable condition or its variants on the design matrix. For the nonnegative MCP estimator, despite its good asymptotic properties, the corresponding optimization problem is non-convex, and consequently much hard to solve. Thus, we develop an efficient computation algorithm, which is a coupling of the difference convex algorithm with the multiplicative updates algorithm, for the implementation of the nonnegative MCP estimator. Simulation studies are carried out to examine superior performance of the nonnegative MCP estimator against alternative methods and a real data analysis for index tracking is also presented for illustration.


High-dimensional variable selection Minimax concave penalty Nonnegativity constraints Oracle property 



This work was supported by the National Natural Science Foundation of China (Grant No. 11671059).


  1. An LTH, Tao PD (1997) Solving a class of linearly constrained indefinite quadratic problems by d.c. algorithms. J Global Optim 11:253–285MathSciNetCrossRefzbMATHGoogle Scholar
  2. Antoniadis A, Gijbels I, Lambert-Lacroix S (2014) Penalized estimation in additive varying coefficient models using grouped regularization. Stat Pap 55:727–750MathSciNetCrossRefzbMATHGoogle Scholar
  3. Breiman L (1995) Better subset regression using the nonnegative garrote. Technometrics 37:373–384MathSciNetCrossRefzbMATHGoogle Scholar
  4. Breheny P, Huang J (2011) Coordinate descent algorithms for nonconvex penalized regression, with application to biological feature selection. Ann Appl Stat 5:232–253MathSciNetCrossRefzbMATHGoogle Scholar
  5. Connor G, Leland H (1995) Cash management for index tracking. Financial Anal J 51:75–80CrossRefGoogle Scholar
  6. Fan J, Li R (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 96:1348–1360MathSciNetCrossRefzbMATHGoogle Scholar
  7. Franks E (1992) Targeting excess-of-benchmark returns. J Portf Manage 18:6–12CrossRefGoogle Scholar
  8. Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for non-orthogonal problems. Technometrics 12:55–67CrossRefzbMATHGoogle Scholar
  9. Huang J, Ma SG, Zhang CH (2008) Adaptive lasso for sparse high-dimensional regression models. Stat Sin 18:1603–1618MathSciNetzbMATHGoogle Scholar
  10. Jacobs B, Levy K (1996) Residual risk: how much is too much. J Portf Manage 22:10–15CrossRefGoogle Scholar
  11. Jobst N, Horniman M, Lucas C, Mitra G (2001) Computational aspects of alternative portfolio selection models in the presence of discrete asset choice constraints. Quant Finance 1:1–13MathSciNetCrossRefzbMATHGoogle Scholar
  12. Kim Y, Choi H, Oh H-S (2008) Smoothly clipped absolute deviation on high dimensions. J Am Stat Assoc 103:1665–1673MathSciNetCrossRefzbMATHGoogle Scholar
  13. Li Y, Yang H (2012) A new Liu-type estimator in linear regression model. Stat Pap 532:427–437MathSciNetCrossRefzbMATHGoogle Scholar
  14. Liu K (1993) A new class of biased estimate in linear regression. Commmun Stat Theory Methods 22:393–402MathSciNetCrossRefzbMATHGoogle Scholar
  15. Liu Y, Shen X, Doss H (2005) Multicategory \(\psi \)-learning and support vector machine: computational tools. J Comput Graph Stat 14:219–236MathSciNetCrossRefGoogle Scholar
  16. Meinshausen N (2013) Sign-constrained least squares estimation for high-dimensional regression. Electron J Stat 7:1607–1631MathSciNetCrossRefzbMATHGoogle Scholar
  17. Neykov NM, Filzmoser P, Neytchev PN (2014) Ultrahigh dimensional variable selection through the penalized maximum trimmed likelihood estimator. Stat Pap 55:187–207MathSciNetCrossRefzbMATHGoogle Scholar
  18. Roll R (1992) A mean/variance analysis of tracking error. J Portf Manage 18:13–22CrossRefGoogle Scholar
  19. Sha F, Lin Y (2007) Multiplicative updates for nonnegative quadratic programming. Neural Comput 19:2004–2031MathSciNetCrossRefzbMATHGoogle Scholar
  20. Slawski M, Hein M (2013) Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization. Electron J Stat 7:3004–3056MathSciNetCrossRefzbMATHGoogle Scholar
  21. Stein C (1956) Inadmissibility of the usual estimator for mean of multivariate normal distribution. In: Neyman J (ed) Proceedings of the 3rd Berkley symposium on mathematical and statistics probability, vol 1, pp 197–206Google Scholar
  22. Tibshirani R (1996) Regression shrinkage and selection via the LASSO. J R Stat Soc Ser B 58:267–288MathSciNetzbMATHGoogle Scholar
  23. Toy W, Zurack M (1989) Tracking the Euro-Pac index. J Portf Manage 15:55–58CrossRefGoogle Scholar
  24. Wu L, Yang YH (2014) Nonnegative elastic net and application in index tracking. Appl Math Comput 227:541–552MathSciNetzbMATHGoogle Scholar
  25. Wu L, Yang YH, Liu HZ (2014) Nonnegative-lasso and application in index tracking. Comput Stat Data Anal 70:116–126MathSciNetCrossRefzbMATHGoogle Scholar
  26. Wu Y, Liu Y (2007) Robust truncated-hinge-loss support vector machines. J Am Stat Assoc 102:974–983MathSciNetCrossRefzbMATHGoogle Scholar
  27. Wu Y, Liu Y (2009) Variable selection in quantile regression. Stat Sin 19:801–817MathSciNetzbMATHGoogle Scholar
  28. Yang H, Xu J (2009) An alternative stochastic restricted Liu estimator in linear regression. Stat Pap 50:639–647MathSciNetCrossRefzbMATHGoogle Scholar
  29. Yuan M, Lin Y (2007) On the non-negative garrotte estimator. J R Stat Soc Ser B 69:143–161CrossRefzbMATHGoogle Scholar
  30. Zhang CH (2010) Nearly unbiased variable selection under minimax concave penalty. Ann Stat 38:894–942MathSciNetCrossRefzbMATHGoogle Scholar
  31. Zhang CH, Zhang T (2012) A general theory of concave regularization for high-dimensional sparse estimation problems. Stat Sci 27:576–593MathSciNetCrossRefzbMATHGoogle Scholar
  32. Zhao P, Yu B (2006) On model selection consistency of lasso. J Mach Learn Res 7:2541–2563MathSciNetzbMATHGoogle Scholar
  33. Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc Ser B 67:301–320MathSciNetCrossRefzbMATHGoogle Scholar
  34. Zou H, Zhang H (2009) On the adaptive elastice-net with a diverging number of parameters. Ann Stat 37:1733–1751CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of Mathematics and StatisticsChongqing UniversityChongqingChina

Personalised recommendations