In statistics, the technique of least squares is used for estimating the unknown parameters in a linear regression model (see Linear Regression Models). This method minimizes the sum of squared distances between the observed responses in a set of data, and the fitted responses from the regression model. Suppose we observe a collection of data {y i , x i } n i = 1 on n units, where y i s are responses and x i = (x i1, x i2, …, x ip )T is a vector of predictors. It is convenient to write the model in matrix notation, as,
where y is n ×1 vector of responses, X is n ×p matrix, known as the design matrix, β = (β 1, β 2, …, β p )T is the unknown parameter vector and ε is the vector of random errors. In ordinary least squares (OLS) regression, we estimate β by minimizing the residual sum of squares, \(RSS = {(y -X\beta )}^{T}(y -X\beta ),\) giving \(\hat{{\beta }}_{\mathrm{OLS}} = {({X}^{T}\!X)}^{-1}{X}^{T}\!y.\)This estimator is simple and has some good...
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References and Further Reading
Ahmed SE, Doksum KA, Hossain S, You J (2007) Shrinkage, pretest and absolute penalty estimators in partially linear models. Aust NZ J Stat 49(4):435–454
Breiman L (1993) Better subset selection using the non-negative garotte. Technical report, University of California, Berkeley
Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression (with discussion). Ann Stat 32(2):407–499
Frank IE, Friedman JH (1993) A statistical view of some chemometrics regression tools. Technometrics 35:109–148
Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference, and prediction, 2nd edn. Springer, New York
Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12:55–67
Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc B 58:267–288
Tikhonov An (1963) Solution of incorrectly formulated problems and the regularization method. Soviet Math Dokl 4:1035–1038, English translation of Dokl Akad Nauk SSSR 151, 1963,501–504
Zou H, Hastie T (2005) Regularization and variable selction via the elastic net. J R Stat Soc B 67(2):301–320
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this entry
Cite this entry
Ahmed, E.S., Raheem, E., Hossain, S. (2011). Absolute Penalty Estimation. In: Lovric, M. (eds) International Encyclopedia of Statistical Science. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04898-2_102
Download citation
DOI: https://doi.org/10.1007/978-3-642-04898-2_102
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04897-5
Online ISBN: 978-3-642-04898-2
eBook Packages: Mathematics and StatisticsReference Module Computer Science and Engineering