Advertisement

On the Behavior of the Risk of a LASSO-Type Estimator

  • Silvelyn Zwanzig
  • M. Rauf AhmadEmail author
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 193)

Abstract

We introduce a LASSO-type estimator as a generalization of the classical LASSO estimator for non-orthogonal design. The generalization, named the SVD-LASSO, allows the design matrix to be of less than full rank. We assume fixed design matrix and normality but otherwise the properties of the SVD-LASSO do not necessarily rest on any strong conditions, particularly sparsity. We derive exact expressions for the risk of the SVD-LASSO and compare it with that of the corresponding ridge estimator.

Keywords

Shrinkage estimation High-dimensional inference Linear models SVD MSE 

Notes

Acknowledgements

The authors are thankful to the reviewer for several constructive comments which lead to an improved version of the article.

References

  1. 1.
    Tibshirani, R.J.: Regression shrinkage and selection via the LASSO. JRSS B 58(1), 267–288 (1996)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Gruber, M.H.J.: Improving Efficiency by Shrinkage: The James–Stein and Ridge Regression Estimators. Marcel Dekker Inc, New York (1998)zbMATHGoogle Scholar
  3. 3.
    Hoerl, A.E., Kennard, R.W.: Ridge Regression: Applications to Nonorthogonal Problems. Technometrics 12(1), 55–67 and 69–82 (1970)Google Scholar
  4. 4.
    Bühlmann, P., van de Geer, S.: Statistics for High-Dimensional Data: Models, Theory & Applications. Springer, New York (2011)CrossRefzbMATHGoogle Scholar
  5. 5.
    Hastie, T., Tibshirani, R.J., Waineright, M.: Statistical Learning with Sparsity: The LASSO and Generalizations. CRC Press, Boca Raton (2015)zbMATHGoogle Scholar
  6. 6.
    Gentle, J.E.: Matrix Algebra: Theory, Computations & Applications in Statistics. Springer, New York (2007)CrossRefzbMATHGoogle Scholar
  7. 7.
    Harville, D.A.: Matrix Anlgebra from a Statistician’s Perspective. Springer, NY (1997)CrossRefzbMATHGoogle Scholar
  8. 8.
    Dicker, L.H.: Ridge regression and asymptotic minimax estimation over spheres of growing dimension. Bernoulli 22(1), 1–37 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Gruber, Marvin H.J.: Liu and ridge estimator: a comparison. Commun. Stat.: Theory Methods 41, 3739–3749 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Johnson, N.L., Kotz, S., Balakrishnan, N.: Continuous Univariate Distributions, vol. 1. Wiley, New York (1994)zbMATHGoogle Scholar
  11. 11.
    Bertsekas, D.P.: Nonlinear programming, 2nd edn. Athena Scientific, MA (1999)zbMATHGoogle Scholar
  12. 12.
    Lange, K.: Optimization, 2nd edn. Springer, NY (2013)CrossRefzbMATHGoogle Scholar
  13. 13.
    Bickel, P.J., Doksum, K.A.: Mathematical Statistics: Basic Ideas and Selected Topics. Prentice Hall, NJ (2001)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of MathematicsUppsala UniversityUppsalaSweden
  2. 2.Department of StatisticsUppsala UniversityUppsalaSweden

Personalised recommendations