Skip to main content
SpringerLink
Log in

Some theoretical results on the Grouped Variables Lasso

  • Published:
Mathematical Methods of Statistics Aims and scope Submit manuscript

Abstract

We consider the linear regression model with Gaussian error. We estimate the unknown parameters by a procedure inspired by the Group Lasso estimator introduced in [22]. We show that this estimator satisfies a sparsity inequality, i.e., a bound in terms of the number of non-zero components of the oracle regression vector. We prove that this bound is better, in some cases, than the one achieved by the Lasso and the Dantzig selector.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. R. J. Adler, An Introduction to Continuity, Extrema, and Related Topics for General Gaussian Processes, in Lecture Notes—Monograph Series (IMS, Hayward, CA, 1990), Vol. 12.

    Google Scholar 

  2. P. J. Bickel, Ya. Ritov, and A. B. Tsybakov, Simultaneous Analysis of Lasso and Dantzig Selector, Ann. Statist. (in press).

  3. F. Bunea, A. B. Tsybakov, and M. H. Wegkamp, “Sparse Density Estimation with ℓ1 Penalties”, in Lecture Notes in Artificial Intelligence, vol. 4539: 20th Annual Conference on Learning Theory, COLT 2007 Proceedings, N. H. Bshouty and C. Gentile, eds. (Springer, 2007), pp. 530–543.

  4. F. Bunea, A. B. Tsybakov, and M. H. Wegkamp, “Aggregation for Gaussian Regression”, Ann. Statist. 35(4), 1674–1697 (2007).

    Article  MATH  MathSciNet  Google Scholar 

  5. F. Bunea, A. B. Tsybakov, and M. H. Wegkamp, “Sparsity Oracle Inequalities for the Lasso”, Electron. J. Statist. 1, 169–194 (2007).

    Article  MATH  MathSciNet  Google Scholar 

  6. T. Cai, “On Block Thresholding in Wavelet Regression: Adaptivity, Block Size and Threshold Level”, Statist. Sinica 12(4), 1241–1273 (2002).

    MATH  MathSciNet  Google Scholar 

  7. E. J. Candes and T. Tao, “The Dantzig Selector: Statistical Estimation when p is Much Larger than n”, Ann. Statist. 35, 2392–2404 (2007).

    Article  MathSciNet  Google Scholar 

  8. L. Cavalier and A. B. Tsybakov, “Penalized Blockwise Stein’s Method, Monotone Oracles and Sharp Adaptive Estimation, Math. Methods Statist. 10(3), 247–282 (2001).

    MATH  MathSciNet  Google Scholar 

  9. A. Dalalyan and A. B. Tsybakov, “Aggregation by Exponential Weighting and Sharp Oracle Inequalities”, in Lecture Notes in Artificial Intelligence, Vol. 4539: 20th Annual Conference on Learning Theory, COLT 2007 Proceedings, N. H. Bshouty and C. Gentile, eds. (Springer, 2007), pp. 97–111.

  10. D. L. Donoho, M. Elad, and V. N. Temlyakov, “Stable Recovery of Sparse Overcomplete Representations in the Presence of Noise”, IEEE Trans. Inform. Theory 52(1), 6–18 (2006).

    Article  MathSciNet  Google Scholar 

  11. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least Angle Regression”, Ann. Statist. 32(2), 407–499 (2004) (With discussion, and a rejoinder by the authors).

    Article  MATH  MathSciNet  Google Scholar 

  12. J. Fan and R. Li, “Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties”, J. Amer. Statist. Assoc. 96(456), 1348–1360 (2001).

    Article  MATH  MathSciNet  Google Scholar 

  13. Yu. Kim, J. Kim, and Yo. Kim, “Blockwise Sparse Regression”, Statist. Sinica 16(2), 375–390 (2006).

    MATH  MathSciNet  Google Scholar 

  14. K. Knight and W. Fu, “Asymptotics for Lasso-Type Estimators”, Ann. Statist. 28(5), 1356–1378 (2000).

    Article  MATH  MathSciNet  Google Scholar 

  15. V. Koltchinskii, Sparsity in Penalized Empirical Risk Minimization, Preprint (2005).

  16. L. Meier, S. van de Geer, and P. Bühlmann, “The Group Lasso for Logistic Regression”, J. Roy. Statist. Soc. Ser. B. 70(1), 53–71 (2008).

    Google Scholar 

  17. N. Meinshausen, “Lasso with Relaxation”, Computational Statist. and Data Analysis, 52(1), 374–393 (2007).

    Article  MathSciNet  Google Scholar 

  18. N. Meinshausen and P. Bühlmann, “High Dimensional Graphs and Variable Selection with the Lasso”, Ann. Statist. 34(3), 1436–1462 (2006).

    Article  MATH  MathSciNet  Google Scholar 

  19. R. Tibshirani, “Regression Shrinkage and Selection via the Lasso”, J. Roy. Statist. Soc. Ser. B 58(1), 267–288 (1996).

    MATH  MathSciNet  Google Scholar 

  20. R. Tibshirani, M. Saunders, S. Rosset, J. Zhu, and K. Knight, “Sparsity and Smoothness via the Fused Lasso”, J. Roy. Statist. Soc. Ser. B 67(1), 91–108 (2005).

    Article  MATH  MathSciNet  Google Scholar 

  21. S. van de Geer, “High-Dimensional Generalized Linear Models and the Lasso”, Ann. Statist. 36(2), 614–645 (2008).

    Article  MATH  MathSciNet  Google Scholar 

  22. M. Yuan and Y. Lin, “Model Selection and Estimation in Regression with Grouped Variables”, J. Roy. Statist. Soc. Ser. B 68(1), 49–67 (2006).

    Article  MATH  MathSciNet  Google Scholar 

  23. P. Zhao and B. Yu, “On Model Selection Consistency of Lasso”, J. Mach. Learn. Res. 7, 2541–2563 (2006).

    MathSciNet  Google Scholar 

  24. H. Zou, “The Adaptive Lasso and Its Oracle Properties”, J. Amer. Statist. Assoc. 101(476), 1418–1429 (2006).

    Article  MATH  MathSciNet  Google Scholar 

  25. H. Zou and T. Hastie, “Regularization and Variable Selection via the Elastic Net”, J. Roy. Statist. Soc. Ser. B 67(2), 301–320 (2005).

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Université de Caen, Caen, France

    Ch. Chesneau

  2. Université Paris VII, Paris, France

    M. Hebiri

Authors
  1. Ch. Chesneau
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Hebiri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ch. Chesneau.

About this article

Cite this article

Chesneau, C., Hebiri, M. Some theoretical results on the Grouped Variables Lasso. Math. Meth. Stat. 17, 317–326 (2008). https://doi.org/10.3103/S1066530708040030

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S1066530708040030

Key words

2000 Mathematics Subject Classification

Search

Navigation