Ridge regression is a method for the estimation of the parameters of a linear regression model (see Linear Regression Models) which is useful when the predictor variables are highly collinear, that is, when there is a strong linear relationship among the predictor variables. Hoerl (1959) named the method ridge regression because of its similarity to ridge analysis used in his earlier work to study second-order response surfaces in many variables. Some standard references for ridge regression are Hoerl and Kennard (1970, 1976), Belsley et al. (1980), and Chatterjee and Hadi (2006).
The standard linear regression model can be written as
where Y is an n ×1 vector of observations on the response variable, X = (X 1, …, X p ) is an n ×p matrix of n observations on p predictor variables, β is a p ×1 vector of regression coefficients, and ε is an n ×1 vector of random errors. It is usual to assume that E(ε) = 0, E(εεT) = σ2 I n , where σ2is...
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References and Further Reading
Belsley DA, Kuh E, Welsch RE (1980) Regression diagnostics: identifying influential data and sources of collinearity. Wiley, New York
Chatterjee S, Hadi AS (2006) Regression analysis by example, 4th edn. Wiley, New York
Dorugade AV, Kashid DN (2010) Alternative method for choosing ridge parameter for regression. Appl Math Sci 4:447–456
Golub GH, van Loan C (1989) Matrix computations. John Hopkins, Baltimore
Hoerl AE (1959) Optimum solution of many variables. Chem Eng Prog 55:69–78
Hoerl AE (1962) Application of ridge analysis to regression problems. Chem Eng Prog 58:54–59
Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12:55–68
Hoerl AE, Kennard RW (1976) Ridge regression: iterative estimation of the biasing parameter. Commun Stat, Theory Methods A5:77–88
Hoerl AE, Kennard RW (1981) Ridge regression – 1980: advances, algorithms, and applications. Am J Math Manag Sci 1:5–83
Hoerl AE, Kennard RW, Baldwin KF (1975) Ridge regression: some simulations. Commun Stat, Theory Methods 4:105–123
Jensen DR, Ramirez DE (2008) Anomalies in the foundations of ridge regression. Int Stat Rev 76:89–105
Khalaf G, Shukur G (2005) Choosing ridge parameter for regression problem. Commun Stat, Theory Methods 34:1177–1182
Lawless JF, Wang P (1976) A simulation study of ridge and other regression estimators. Commun Stat, Theory Methods 14:1589–1604
Masuo N (1988) On the almost unbiased ridge regression estimation. Commun Stat, Simul 17:729–743
Tikhonov AN (1943) On the stability problems. Dokl Akad Nauk SSSR 39:195–198
Wahba G, Golub GH, Health CG (1979) Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21:215–223
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this entry
Cite this entry
Hadi, A.S. (2011). Ridge and Surrogate Ridge Regressions. In: Lovric, M. (eds) International Encyclopedia of Statistical Science. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04898-2_493
Download citation
DOI: https://doi.org/10.1007/978-3-642-04898-2_493
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04897-5
Online ISBN: 978-3-642-04898-2
eBook Packages: Mathematics and StatisticsReference Module Computer Science and Engineering