Test

, Volume 15, Issue 2, pp 271–344

Regularization in statistics

  • Peter J. Bickel
  • Bo Li
  • Alexandre B. Tsybakov
  • Sara A. van de Geer
  • Bin Yu
  • Teófilo Valdés
  • Carlos Rivero
  • Jianqing Fan
  • Aad van der Vaart
Article

DOI: 10.1007/BF02607055

Cite this article as:
Bickel, P.J., Li, B., Tsybakov, A.B. et al. Test (2006) 15: 271. doi:10.1007/BF02607055

Abstract

This paper is a selective review of the regularization methods scattered in statistics literature. We introduce a general conceptual approach to regularization and fit most existing methods into it. We have tried to focus on the importance of regularization when dealing with today's high-dimensional objects: data and models. A wide range of examples are discussed, including nonparametric regression, boosting, covariance matrix estimation, principal component estimation, subsampling.

Key Words

Regularizationlinear regressionnonparametric regressionboostingcovariance matrixprincipal componentbootstrapsubsamplingmodel selection

AMS subject classification

Primary 62G08, 62H12Secondary 62F12, 62G20, 62H25

Copyright information

© Sociedad Española de Estadistica e Investigacion Operativa 2006

Authors and Affiliations

  • Peter J. Bickel
    • 1
  • Bo Li
    • 2
  • Alexandre B. Tsybakov
    • 3
  • Sara A. van de Geer
    • 4
  • Bin Yu
    • 1
  • Teófilo Valdés
    • 5
  • Carlos Rivero
    • 5
  • Jianqing Fan
    • 6
  • Aad van der Vaart
    • 7
  1. 1.Department of StatisticsUniversity of California at BerkeleyUSA
  2. 2.School of Economics and ManagementTsinghua UniversityChina
  3. 3.Laboratoire de Probabilités et Modèles AléatoiresUniversité Paris VIFrance
  4. 4.Seminar für StatistikETH ZürichSwitzerland
  5. 5.Department of Statistics and Operational ResearchComplutense University of MadridSpain
  6. 6.Department of Operations Research and Financial EngineeringPrinceton UniversityUSA
  7. 7.Department of MathematicsVrije Universiteit AmsterdamNetherlands