Skip to main content
SpringerLink
Log in

On Early Stopping in Gradient Descent Learning

  • Published:
Constructive Approximation Aims and scope

Abstract

In this paper we study a family of gradient descent algorithms to approximate the regression function from reproducing kernel Hilbert spaces (RKHSs), the family being characterized by a polynomial decreasing rate of step sizes (or learning rate). By solving a bias-variance trade-off we obtain an early stopping rule and some probabilistic upper bounds for the convergence of the algorithms. We also discuss the implication of these results in the context of classification where some fast convergence rates can be achieved for plug-in classifiers. Some connections are addressed with Boosting, Landweber iterations, and the online learning algorithms as stochastic approximations of the gradient descent method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Author information

Authors and Affiliations

  1. Department of Mathematics, University of California, Berkeley, CA 94720, USA

    Yuan Yao

  2. C.B.C.L., Massachusetts Institute of Technology, Bldg. E25-201, 45 Carleton St., Cambridge, MA 02142, USA

    Lorenzo Rosasco & Andrea Caponnetto

  3. DISI, Universita di Genova, Via Dodecaneso 35, 16146 Genova, Italy

    Lorenzo Rosasco & Andrea Caponnetto

Authors
  1. Yuan Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lorenzo Rosasco
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrea Caponnetto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yuan Yao, Lorenzo Rosasco or Andrea Caponnetto.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yao, Y., Rosasco, L. & Caponnetto, A. On Early Stopping in Gradient Descent Learning. Constr Approx 26, 289–315 (2007). https://doi.org/10.1007/s00365-006-0663-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00365-006-0663-2

Keywords

Search

Navigation