Skip to main content
SpringerLink
Log in

A group adaptive elastic-net approach for variable selection in high-dimensional linear regression

  • Articles
  • Published:
Science China Mathematics Aims and scope Submit manuscript

Abstract

In practice, predictors possess grouping structures spontaneously. Incorporation of such useful information can improve statistical modeling and inference. In addition, the high-dimensionality often leads to the collinearity problem. The elastic net is an ideal method which is inclined to reflect a grouping effect. In this paper, we consider the problem of group selection and estimation in the sparse linear regression model in which predictors can be grouped. We investigate a group adaptive elastic-net and derive oracle inequalities and model consistency for the cases where group number is larger than the sample size. Oracle property is addressed for the case of the fixed group number. We revise the locally approximated coordinate descent algorithm to make our computation. Simulation and real data studies indicate that the group adaptive elastic-net is an alternative and competitive method for model selection of high-dimensional problems for the cases of group number being larger than the sample size.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bickel P J, Ritov Y, Tsybakov A B. Simultaneous analysis of LASSO and Dantzig selector. Ann Statist, 2009, 37: 1705–1732

    Article  MathSciNet  MATH  Google Scholar 

  2. Breheny P, Huang J. Penalized methods for bi-level variable selection. Stat Interface, 2009, 2: 369–380

    Article  MathSciNet  MATH  Google Scholar 

  3. Candès E, Tao T. Decoding by linear programming. IEEE Trans Inform Theory, 2005, 51: 4203–4215

    Article  MathSciNet  MATH  Google Scholar 

  4. Candès E, Tao T. The Dantzig selector: Statistical estimation when p is much larger than n. Ann Statist, 2007, 35: 2313–2351

    Article  MathSciNet  MATH  Google Scholar 

  5. Fan J, Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. J Amer Statist Assoc, 2001, 96: 1348–1360

    Article  MathSciNet  MATH  Google Scholar 

  6. Friedman J, Hastie T, Tibshirani R. A note on the group LASSO and a sparse group LASSO. Techinical report. Stanford: Stanford University, 2010

    Google Scholar 

  7. Horvath S, Zhang B, Carlson M, et al. Analysis of oncogenic signaling networks in glioblastoma identifies aspm as a molecular target. Proc Natl Acad Sci USA, 2006, 103: 17402–17407

    Article  Google Scholar 

  8. Huang J, Breheny P, Ma S. A selective review of group selection in high-dimensional models. Statist Sci, 2012, 27: 481–499

    Article  MathSciNet  MATH  Google Scholar 

  9. Huang J, Ma S, Xie H, et al. A group bridge approach for variable selection. Biometrika, 2009, 96: 339–355

    Article  MathSciNet  MATH  Google Scholar 

  10. Jiang D, Huang J. Concave 1-norm group selection. Biostatistics, 2015, 16: 252–267

    Article  MathSciNet  Google Scholar 

  11. Knight K, Fu W. Asymptotics for LASSO-type estimators. Ann Statist, 2000, 28: 1356–1378

    Article  MathSciNet  MATH  Google Scholar 

  12. Li R, Zhong W, Zhu L. Feature screening via distance correlation learning. J Amer Statist Assoc, 2012, 107: 1129–1139

    Article  MathSciNet  MATH  Google Scholar 

  13. Lounici K, Pontil M, Van De Geer S, et al. Oracle inequalities and optimal inference under group sparsity. Ann Statist, 2011, 39: 2164–2204

    Article  MathSciNet  MATH  Google Scholar 

  14. Simon N, Friedman J, Hastie T, et al. A sparse-group LASSO. J Comput Graph Statist, 2013, 22: 231–245

    Article  MathSciNet  Google Scholar 

  15. Tibshirani R. Regression shrinkage and selection via the LASSO. J R Stat Soc Ser B Stat Methodol, 1996, 58: 267–288

    MathSciNet  MATH  Google Scholar 

  16. Vaart A V D. Asymptotic Statistics. Cambridge: Cambridge University Press, 1998

    Book  MATH  Google Scholar 

  17. Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. J R Stat Soc Ser B Stat Methodol, 2006, 68: 49–67

    Article  MathSciNet  MATH  Google Scholar 

  18. Zhang C-H. Nearly unbiased variable selection under minimax concave penalty. Ann Statist, 2010, 38: 894–942

    Article  MathSciNet  MATH  Google Scholar 

  19. Zou H. The adaptive LASSO and its oracle properties. J Amer Statist Assoc, 2006, 101: 1418–1429

    Article  MathSciNet  MATH  Google Scholar 

  20. Zou H, Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc Ser B Stat Methodol, 2005, 67: 301–320

    Article  MathSciNet  MATH  Google Scholar 

  21. Zou H, Zhang H H. On the adaptive elastic-net with a diverging number of parameters. Ann Statist, 2009, 37: 1733–1751

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant No. 11571219) and the Open Research Fund Program of Key Laboratory of Mathematical Economics (SUFE) (Grant No. 201309KF02), Ministry of Education, and Changjiang Scholars and Innovative Research Team in University (Grant No. IRT13077). The authors thank anonymous referees for their constructive comments that lead to the current improved version.

Author information

Authors and Affiliations

  1. School of Statistics and Management, Shanghai University of Finance and Economics, Shanghai, 200433, China

    Jianhua Hu & Feng Qiu

  2. Key Laboratory of Mathematical Economics (SUFE), Ministry of Education, Shanghai, 200433, China

    Jianhua Hu

  3. Department of Biostatistics, University of Iowa, Iowa City, IA, 52242, USA

    Jian Huang

  4. Science College, Zhejiang Agriculture and Forestry University, Lin’an, 311300, China

    Feng Qiu

Authors
  1. Jianhua Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianhua Hu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, J., Huang, J. & Qiu, F. A group adaptive elastic-net approach for variable selection in high-dimensional linear regression. Sci. China Math. 61, 173–188 (2018). https://doi.org/10.1007/s11425-016-0071-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11425-016-0071-x

Keywords

MSC(2010)

Search

Navigation