On the Behavior of the Risk of a LASSO-Type Estimator

  • Silvelyn Zwanzig
  • M. Rauf AhmadEmail author
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 193)


We introduce a LASSO-type estimator as a generalization of the classical LASSO estimator for non-orthogonal design. The generalization, named the SVD-LASSO, allows the design matrix to be of less than full rank. We assume fixed design matrix and normality but otherwise the properties of the SVD-LASSO do not necessarily rest on any strong conditions, particularly sparsity. We derive exact expressions for the risk of the SVD-LASSO and compare it with that of the corresponding ridge estimator.


Shrinkage estimation High-dimensional inference Linear models SVD MSE 



The authors are thankful to the reviewer for several constructive comments which lead to an improved version of the article.


  1. 1.
    Tibshirani, R.J.: Regression shrinkage and selection via the LASSO. JRSS B 58(1), 267–288 (1996)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Gruber, M.H.J.: Improving Efficiency by Shrinkage: The James–Stein and Ridge Regression Estimators. Marcel Dekker Inc, New York (1998)zbMATHGoogle Scholar
  3. 3.
    Hoerl, A.E., Kennard, R.W.: Ridge Regression: Applications to Nonorthogonal Problems. Technometrics 12(1), 55–67 and 69–82 (1970)Google Scholar
  4. 4.
    Bühlmann, P., van de Geer, S.: Statistics for High-Dimensional Data: Models, Theory & Applications. Springer, New York (2011)CrossRefzbMATHGoogle Scholar
  5. 5.
    Hastie, T., Tibshirani, R.J., Waineright, M.: Statistical Learning with Sparsity: The LASSO and Generalizations. CRC Press, Boca Raton (2015)zbMATHGoogle Scholar
  6. 6.
    Gentle, J.E.: Matrix Algebra: Theory, Computations & Applications in Statistics. Springer, New York (2007)CrossRefzbMATHGoogle Scholar
  7. 7.
    Harville, D.A.: Matrix Anlgebra from a Statistician’s Perspective. Springer, NY (1997)CrossRefzbMATHGoogle Scholar
  8. 8.
    Dicker, L.H.: Ridge regression and asymptotic minimax estimation over spheres of growing dimension. Bernoulli 22(1), 1–37 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Gruber, Marvin H.J.: Liu and ridge estimator: a comparison. Commun. Stat.: Theory Methods 41, 3739–3749 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Johnson, N.L., Kotz, S., Balakrishnan, N.: Continuous Univariate Distributions, vol. 1. Wiley, New York (1994)zbMATHGoogle Scholar
  11. 11.
    Bertsekas, D.P.: Nonlinear programming, 2nd edn. Athena Scientific, MA (1999)zbMATHGoogle Scholar
  12. 12.
    Lange, K.: Optimization, 2nd edn. Springer, NY (2013)CrossRefzbMATHGoogle Scholar
  13. 13.
    Bickel, P.J., Doksum, K.A.: Mathematical Statistics: Basic Ideas and Selected Topics. Prentice Hall, NJ (2001)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of MathematicsUppsala UniversityUppsalaSweden
  2. 2.Department of StatisticsUppsala UniversityUppsalaSweden

Personalised recommendations