Skip to main content

Online Learning Algorithms


In this paper, we study an online learning algorithm in Reproducing Kernel Hilbert Spaces (RKHSs) and general Hilbert spaces. We present a general form of the stochastic gradient method to minimize a quadratic potential function by an independent identically distributed (i.i.d.) sample sequence, and show a probabilistic upper bound for its convergence.

This is a preview of subscription content, access via your institution.

Author information

Authors and Affiliations


Corresponding authors

Correspondence to Steve Smale or Yuan Yao.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Smale, S., Yao, Y. Online Learning Algorithms. Found Comput Math 6, 145–170 (2006).

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:

  • Online learning
  • Stochastic approximation
  • Regularization
  • Reproducing Kernel Hilbert Spaces