Abstract
The term ‘high-dimensional’ refers to the case where the number of unknown parameters to be estimated, p, is of much larger order than the number of observations, n, that is p ≫ n. Since traditional statistical methods assume many observations and a few unknown variables, they can not cope up with the situations when p ≫ n. In this article, we study a statistical method, called the ‘Least Absolute Shrinkage and Selection Operator’ (LASSO), that has got much attention in solving high-dimensional problems. In particular, we consider the LASSO for high-dimensional linear regression models. We aim to provide an introduction of the LASSO method as a constrained quadratic programming problem, and we discuss the convex optimization based approach to solve the LASSO problem. We also illustrate applications of LASSO method using a simulated and a real data examples.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Suggested Reading
Simon Foucart and Holger Rauhut, A Mathematical Introduction to Compressive Sensing, New York: Springer, 2013.
Trevor Hastie, Robert Tibshirani, and Jerome Friedman, The Elements of Statistical Learning; Data Mining, Inference and Prediction, New York: Springer, 2001.
Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani, An Introduction to Statistical Learning with Applications in R, Springer, 2013.
Tibshirani R, Regression Analysis and Selection via the Lasso, Royal Statistical Society Series, 58:267288, 1996.
Stephen Boyd and Lieven Vandenberghe, Convex Optimization, Cambridge University Press, 2004.
Dimitri P Bertsekas, Convex Optimization Algorithms, Athena Scientific, 2015.
Tseng P, Coordinate Ascent for Maximizing Non-differentiable Concave Functions, Technical Report LIDS-P(MIT), 1988.
George A F Seber and Alan J Lee, Linear Regression Analysis, Wiley, 2003.
Edward I George, The Variable Selection Problem, Journal of the American Statistical Association, 95:13041308, 2000.
Dziak J, Li R, and Collins L, Critical Review and Comparison of Variable Selection Procedures for Linear Regression, Technical report, 2005.
Jianqing Fan and Jinchi Lv, A Selective Overview of Variable Selection in High-dimensional Feature Space, Statistica Sinica, 20(1):101148, 2010.
Alois Kneip and Pascal Sarda, Factor Models and Variable Selection in Highdimensional Regression Analysis, The Annals of Statistics, 39(5):24102447, 2011.
Trevor Hastie, Robert Tibshirani, and Martin Wainwright, Statistical Learning with Sparsity: The Lasso and Generalizations, CRC Press, 2015.
Bradley Efron, Trevor Hastie, Iain Johnstone, and Robert Tibshirani, Least Angle Regression, Ann. Statist., 32(2):407499, 2004.
B A Turlach, On Algorithms for Solving Least Squares Problems Under an l1 Penalty or an l1 Constraint, Proceedings of the American Statistical Association, Statistical Computing Section, pages 2572–2577, 2004.
M R Osborne, B Presnell, and B A Turlach, A New Approach to Variable Selection in Least Squares Problems, IMA Journal of Numerical Analysis, 20(3):389403, 2000.
Trevor Park and George Casella, The Bayesian Lasso, Journal of the American Statistical Association, 103(482):681686, 2008.
Yiyun Zhang Runze Li and Chih-Ling Tsai, Regularization Parameter selections via Generalized Information Criterion, Journal of the American Statistical Association, 105(489):312323, 2010.
Peter Bühlmann, Markus Kalisch, and Lukas Meier, High-dimensional Statistics with a View Towards Applications in Biology, Annual Review of Statistics and its Applications, 1:255278., 2014.
Author information
Authors and Affiliations
Corresponding author
Additional information
Niharika Gauraha is a PhD student at Indian Statistical Institute, Bangalore. Her research interests include statistical pattern recognition and machine learning. Currently she is working as a researcher at the Department of Pharmaceutical Biosciences, Uppsala University.
Rights and permissions
About this article
Cite this article
Gauraha, N. Introduction to the LASSO. Reson 23, 439–464 (2018). https://doi.org/10.1007/s12045-018-0635-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12045-018-0635-x