Advertisement

Statistical Papers

, Volume 59, Issue 3, pp 985–1008 | Cite as

A flexible shrinkage operator for fussy grouped variable selection

  • Xiaoli GaoEmail author
Regular Article

Abstract

Existing grouped variable selection methods rely heavily on prior group information, thus they may not be reliable if an incorrect group assignment is used. In this paper, we propose a family of shrinkage variable selection operators by controlling the k-th largest norm (KAN). The proposed KAN method exhibits some flexible group-wise variable selection naturally even though no correct prior group information is available. We also construct a group KAN shrinkage operator using a composite of KAN constraints. Neither ignoring nor relying completely on prior group information, the group KAN method has the flexibility of controlling within group strength and therefore can reduce the effect caused by incorrect group information. Finally, we investigate an unbiased estimator of the degrees of freedom for (group) KAN estimates in the framework of Stein’s unbiased risk estimation. Extensive simulation studies and real data analysis are performed to demonstrate the advantage of KAN and group KAN over the LASSO and group LASSO, respectively.

Keywords

Degrees of freedom Group shrinkage k-th largest norm Shrinkage estimator Variable selection 

Notes

Acknowledgments

The author wants to thank Sijian Wang and Yuan Wu for their valuable comments and Jonathan Rowell for his professional proofreading. She also would like to thank the reviewers for their helpful and constructive comments for improvement of the manuscript. The author gratefully acknowledges Simons Foundation (#359337) and UNC Greensboro (New Faculty Grant) for their support in this Project.

References

  1. Akaike H (1973) Maximum likelihood identification of Gaussian autoregressive moving average models. Biometrika 60:255–265MathSciNetCrossRefzbMATHGoogle Scholar
  2. Anderson PK, Gill RD (1982) Cox’s regression model for counting processes: a large sample study. Ann Stat 10:1100–1120MathSciNetCrossRefzbMATHGoogle Scholar
  3. Bogdan M, van den Berg E, Chiara S, Su W, Candes E (2015) SLOPE-adaptive variable selection via convex optimization. Ann ApplStat 9(3):1103–1140MathSciNetzbMATHGoogle Scholar
  4. Chen J, Chen Z (2008) Extended bayesian information criteria for model selection with large model spaces. Biometrika 95(3):759–771MathSciNetCrossRefzbMATHGoogle Scholar
  5. Chiang AP, Beck JS, Yen HJ, Tayeh MK, Scheetz TE, Swiderski R, Nishimura D, Braun TA, Kim K, Huang J, Elbedour K, Carmi R, Slusarski DC, Casavant TL, Stone EM, Sheffield VC (2006) Homozygosity mapping with SNP arrays identifies a novel gene for Bardet-Biedl syndrome (BBS10). Proc Natl Acad Sci 103:6287–6292CrossRefGoogle Scholar
  6. Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression. Ann Stat 32:407–499MathSciNetCrossRefzbMATHGoogle Scholar
  7. Frank I, Friedman J (1993) A statistical view of some chemometrics regression tools (with discussion). Technometrics 35:109–148CrossRefzbMATHGoogle Scholar
  8. Huang J, Ma SG, Zhang C (2008) Adaptive lasso for sparse high-dimensional regression models. Stat Sin 18:1603–1618MathSciNetzbMATHGoogle Scholar
  9. Kanehisa M, Goto S (2000) Kyoto encyclopedia of genes and genomes. Nucl Acids Res 28:27–30CrossRefGoogle Scholar
  10. Kato K (2009) On the degrees of freedom in shrinkage estimation. J Multivar Anal 100:1338–1352MathSciNetCrossRefzbMATHGoogle Scholar
  11. Kaufman L, Rousseeuw PJ (1990) Finding groups in data: an introduction to cluster analysis. Wiley, New YorkCrossRefzbMATHGoogle Scholar
  12. Kim Y, Kim J, Kim Y (2006) Blockwise sparse regression. Stat Sin 16:375–390MathSciNetzbMATHGoogle Scholar
  13. Knight K, Fu W (2000) Asymptotics for lasso-type estimators. Ann Stat 28:1356–1378MathSciNetCrossRefzbMATHGoogle Scholar
  14. Pollard D (1991) Asymptotics for least absolute deviation regression estimators. Econ Theory 7:186–199MathSciNetCrossRefGoogle Scholar
  15. Scheetz TE, Kim KYA, Swiderski RE, Philp AR, Braun TA, Knudtson KL, Dorrance AM, DiBona GF, Huang J, Casavant TL, Sheffield VC, Stone EM (2006) Regulation of gene expression in the mammalian eye and its relevance to eye disease. Proc Natl Acad Sci 103(39):14429–14434CrossRefGoogle Scholar
  16. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6:461–464MathSciNetCrossRefzbMATHGoogle Scholar
  17. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B 58:267–288MathSciNetzbMATHGoogle Scholar
  18. Wang L, You Y, Lian H (2015) Convergence and sparsity of lasso and group lasso in high-dimensional generalized linear models. Stat Pap 56:819–828MathSciNetCrossRefzbMATHGoogle Scholar
  19. Ye J (1998) On measuring and correcting the effects of data mining and model selection. J Am Stat Assoc 93:120–131MathSciNetCrossRefzbMATHGoogle Scholar
  20. Yuan M, Lin Y (2006) Model selection and estimation in regression with grouped variables. J R Stat Soc Ser B 68:49–67MathSciNetCrossRefzbMATHGoogle Scholar
  21. Zhao P, Rocha G, Yu B (2009) The composite absolute penalties family for grouped and hierarchical variable selection. Ann Stat 37(6A):3468–3497MathSciNetCrossRefzbMATHGoogle Scholar
  22. Zou H, Yuan M (2008) The f-infinity-norm support vector machine. Stat Sin 18:379–398zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.GreensboroUSA

Personalised recommendations