Skip to main content

The Lasso

Part of the Lecture Notes in Mathematics book series (LNMECOLE,volume 2159)

Abstract

Sharp oracle inequalities for the prediction error and 1-error of the Lasso are given. We highlight the ingredients for establishing these. The latter is also for later reference where results are extended to other norms and other loss functions.

Keywords

  • Prediction Error
  • Tuning Parameter
  • Basis Pursuit
  • Smallish Coefficient
  • Oracle Inequality

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (Canada)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    A suitable notation that expresses the non-uniqueness is β 0 ∈ argmin{ ∥ β ∥ 1:  X β = f 0}. In our analysis, non-uniqueness is not a major concern.

  2. 2.

    If X 1, , X n are n elements of some space \(\mathcal{X}\) and \(f: \mathcal{X} \rightarrow \mathbb{R}\) is some real-valued function on \(\mathcal{X}\), one may view \(\sum _{i=1}^{n}f^{2}(X_{i})/n\) as the squared L 2(P n )-norm of f, with \(P_{n} =\sum _{ i=1}^{n}\delta _{X_{i}}/n\) being the measure that puts equal mass 1∕n at each X i (i = 1, , n). Let us denote the L 2(P n )-norm by \(\|\cdot \|_{2,P_{n}}\). We have abbreviated this to \(\|\cdot \|_{P_{n}}\) and then further abbreviated it to ∥ ⋅ ∥  n . Finally, we identified f with the vector \((\,f(X_{1}),\ldots,f(X_{n}))^{T} \in \mathbb{R}^{n}\).

  3. 3.

    Or non-sparsity actually.

  4. 4.

    The “argmin” argument takes the inequality: \(\|Y - X\hat{\beta }\|_{n}^{2} + 2\lambda \|\hat{\beta }\|_{1} \leq \vert Y - X\beta \|_{n}^{2} + 2\lambda \|\beta \|_{1}\ \forall \ \beta\), as starting point.

References

  • P. Bickel, Y. Ritov, A. Tsybakov, Simultaneous analysis of Lasso and Dantzig selector. Ann. Stat. 37, 1705–1732 (2009)

    CrossRef  MathSciNet  MATH  Google Scholar 

  • P. Bühlmann, S. van de Geer, Statistics for High-Dimensional Data: Methods, Theory and Applications (Springer, Heidelberg, 2011)

    CrossRef  MATH  Google Scholar 

  • G. Chen, M. Teboulle, Convergence analysis of a proximal-like minimization algorithm using Bregman functions. SIAM J. Optim. 3, 538–543 (1993)

    CrossRef  MathSciNet  MATH  Google Scholar 

  • S.S. Chen, D.L. Donoho, M.A. Saunders, Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20, 33–61 (1998)

    CrossRef  MathSciNet  MATH  Google Scholar 

  • O. Güler, On the convergence of the proximal point algorithm for convex minimization. SIAM J. Control Optim. 29, 403–419 (1991)

    CrossRef  MathSciNet  MATH  Google Scholar 

  • V. Koltchinskii, Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems: École d’Été de Probabilités de Saint-Flour XXXVIII-2008, vol. 38 (Springer, Heidelberg, 2011)

    CrossRef  MATH  Google Scholar 

  • V. Koltchinskii, K. Lounici, A. Tsybakov, Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion. Ann. Stat. 39, 2302–2329 (2011)

    CrossRef  MathSciNet  MATH  Google Scholar 

  • R. Tibshirani, Regression analysis and selection via the Lasso. J. R. Stat. Soc. Ser. B 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  • S. van de Geer, Least squares estimation with complexity penalties. Math. Methods Stat. 10, 355–374 (2001)

    MathSciNet  MATH  Google Scholar 

  • S. van de Geer, The deterministic Lasso, in JSM Proceedings, 2007, 140 (American Statistical Association, Alexandria, 2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and Permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

van de Geer, S. (2016). The Lasso. In: Estimation and Testing Under Sparsity. Lecture Notes in Mathematics(), vol 2159. Springer, Cham. https://doi.org/10.1007/978-3-319-32774-7_2

Download citation