Skip to main content
SpringerLink
Log in

The adaptive lasso in high-dimensional sparse heteroscedastic models

  • Published:
Mathematical Methods of Statistics Aims and scope Submit manuscript

Abstract

In this paper we study the asymptotic properties of the adaptive Lasso estimate in high-dimensional sparse linear regression models with heteroscedastic errors. It is demonstrated that model selection properties and asymptotic normality of the selected parameters remain valid but with a suboptimal asymptotic variance. A weighted adaptive Lasso estimate is introduced and investigated. In particular, it is shown that the new estimate performs consistent model selection and that linear combinations of the estimates corresponding to the non-vanishing components are asymptotically normally distributed with a smaller variance than those obtained by the “classical” adaptive Lasso. The results are illustrated in a data example and by means of a small simulation study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. R. Bhatia, Matrix Analysis (Springer, New York, 1997).

    Book  Google Scholar 

  2. P. Craven and G. Wahba, “Smoothing Noisy Data with Spline Function: Estimating the Correct Degree of Smoothing by the Method of Generalized Cross Validation”, Numerische Mathematik 31, 337–403 (1979).

    MathSciNet  Google Scholar 

  3. H. Dette and A. Munk, “Testing Heteroscedasticity in Nonparametric Regression”, J. Roy. Statist. Soc. Ser. B 60, 693–708 (1998).

    Article  MathSciNet  MATH  Google Scholar 

  4. B. Efron, T. Hastie, and R. Tibshirani, “Least Angle Regression (with Discussion)”, Ann. Statist. 32, 407–451 (2004).

    Article  MathSciNet  MATH  Google Scholar 

  5. J. Fan and R. Li, “Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties”, J. Amer. Statist. Assoc. 96, 1348–1360 (2001).

    Article  MathSciNet  MATH  Google Scholar 

  6. J. Fan and J. Lv, “A Selective Overview of Variable Selection in High-Dimensional Feature Space”, Statistica Sinica 20, 101–148 (2010).

    MathSciNet  MATH  Google Scholar 

  7. J. Fan and H. Peng, “Nonconcave Penalized Likelihood with a Diverging Number of Parameters”, Ann. Statist. 32, 928–961 (2004).

    Article  MathSciNet  MATH  Google Scholar 

  8. I. E. Frank and J. H. Friedman, “A Statistical View of Some Chemometrics Regression Tools (with Discussion)”, Technometrics 35, 109–148 (1993).

    Article  MATH  Google Scholar 

  9. A. Hörl and R. Kennard, “Ridge Regression: Biased Estimation for Nonorthogonal Problems”, Technometrics, 12, 55–67 (1970).

    Article  MATH  Google Scholar 

  10. J. Huang, J. L. Horowitz, and S. Ma, “Asymptotic Properties of Bridge Estimators in Sparse High Dimensional Regression Models”, Ann. Statist. 36, 587–613 (2008).

    Article  MathSciNet  MATH  Google Scholar 

  11. J. Huang, S. Ma, and C. Zhang, Adaptive Lasso for Sparse High-Dimensional Regression Models, Technical Report, Univ. of Iowa (2006).

    Google Scholar 

  12. J. Huang, S. Ma, and C. Zhang, “Adaptive Lasso for Sparse High-Dimensional Regression Models”, Statistica Sinica 18, 1603–1618 (2008).

    MathSciNet  MATH  Google Scholar 

  13. Y. Kim, H. Choi, and H.-S. Oh, “Smoothly Clipped Absolute Deviation on High Dimensions”, J. Amer. Statist. Assoc. 103, 1665–1673 (2008).

    Article  MathSciNet  Google Scholar 

  14. K. Knight and W. Fu, (2000). “Asymptotics for Lasso-type Estimators”, Ann. Statist, 28, 1356–1378 (2000).

    Article  MathSciNet  MATH  Google Scholar 

  15. H. Leeb and B. M. Pötscher, “Sparse Estimators and the Oracle Property, or the Return of Hodges’ Estimator”, J. Econometrics 142, 201–211 (2008).

    Article  MathSciNet  Google Scholar 

  16. B.M. Pötscher and U. Schneider, “Distributional Results for Thresholding Estimators in High-Dimensional Gaussian Regression Models”, Electronic J. Statist. 5, 1876–1934 (2011).

    Article  MATH  Google Scholar 

  17. R Development Core Team, R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, Vienna, Austria, 2009) ISBN 3-900051-07-0.

    Google Scholar 

  18. R. Tibshirani, “Regression Shrinkage and Selection via the Lasso”, J. Roy. Statist. Soc. B 58, 267–288 (1996).

    MathSciNet  MATH  Google Scholar 

  19. A.W. van der Vaart and J. A. Wellner, Weak Convergence and Empirical Processes, in Springer Series in Statist. (Springer, New York, 1996).

    Book  MATH  Google Scholar 

  20. J. Wagener and H. Dette, “Bridge Estimators and the Adaptive Lasso under Heteroscedasticity”, Math. Methods Statist. 21, 109–126 (2012).

    Article  MathSciNet  Google Scholar 

  21. M. J. Wainwright, “Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using l 1-Constrained Quadratic Programming (Lasso)”, IEEE Trans. Inform. Theory 55, 2183–2202 (2009).

    Article  MathSciNet  Google Scholar 

  22. H. Zou, “The Adaptive Lasso and Its Oracle Properties”, J. Amer. Statist. Assoc. 101, 1418–1429 (2006).

    Article  MathSciNet  MATH  Google Scholar 

  23. P. Zhao and B. Yu, “On Model Selection Consistency of Lasso”, J. Machine Learning Research 7, 2541–2563 (2006).

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Fakultät für Math., Ruhr-Universität Bochum, Bochum, Germany

    J. Wagener & H. Dette

Authors
  1. J. Wagener
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. H. Dette
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. Wagener.

About this article

Cite this article

Wagener, J., Dette, H. The adaptive lasso in high-dimensional sparse heteroscedastic models. Math. Meth. Stat. 22, 137–154 (2013). https://doi.org/10.3103/S106653071302004X

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S106653071302004X

Keywords

2000 Mathematics Subject Classification

Search

Navigation