Abstract
Bounds for the bias of the Lasso are derived. These bounds are based on so-called worst possible sub-directions or surrogate versions thereof. Both random design as well as fixed design is considered. In the fixed design case the bounds for groups of variables may be different than the ones for single variables due to a different choice of the surrogate inverse. An oracle inequality for subsets of the variables is presented, where it is assumed that the ℓ 1-operator norm of the worst possible sub-direction is small. It is shown that the latter corresponds to the irrepresentable condition. It is furthermore examined under what circumstances variables with small coefficients are de-selected by the Lasso. To explain the terminology “worst possible sub-direction”, a section on the semi-parametric lower bound is added.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
P. Bickel, C. Klaassen, Y. Ritov, J. Wellner, Efficient and Adaptive Estimation for Semiparametric Models (Johns Hopkins University, Press Baltimore, 1993)
P. Bühlmann, S. van de Geer, Statistics for High-Dimensional Data: Methods, Theory and Applications (Springer, Heidelberg, 2011)
K. Lounici, Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators. Electron. J. Stat. 2, 90–102 (2008)
S. van de Geer, P. Bühlmann, On the conditions used to prove oracle results for the Lasso. Electron. J. Stat. 3, 1360–1392 (2009)
S. van de Geer, J. Janková, Semi-parametric efficiency bounds and efficient estimation for high-dimensional models. ArXiv:1601.00815 (2016)
A. van der Vaart, Asymptotic Statistics, vol. 3 (Cambridge University Press, Cambridge, 2000)
G. Watson, Characterization of the subdifferential of some matrix norms. Linear Algebra Appl. 170, 33–45 (2015)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
van de Geer, S. (2016). The Bias of the Lasso and Worst Possible Sub-directions. In: Estimation and Testing Under Sparsity. Lecture Notes in Mathematics(), vol 2159. Springer, Cham. https://doi.org/10.1007/978-3-319-32774-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-32774-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-32773-0
Online ISBN: 978-3-319-32774-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)