Skip to main content

Improving Confidence of Dual Averaging Stochastic Online Learning via Aggregation

  • Conference paper
  • 1071 Accesses

Part of the Lecture Notes in Computer Science book series (LNAI,volume 7526)

Abstract

Stochastic online learning algorithms typically exhibit slow convergence speed, but their solutions of moderate accuracy often suffice in practice. Since the outcomes of these algorithms are random variables, not only their accuracy but also their probability of achieving a certain accuracy, called confidence, is important. We show that a rather simple aggregation of outcomes from parallel dual averaging runs can provide a solution with improved confidence, and it can be controlled by the number of runs, independently of the length of learning processes.

Keywords

  • Empirical Risk
  • Subgradient Method
  • Dual Average
  • Simple Aggregation
  • Machine Learn Research

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-33347-7_20
  • Chapter length: 4 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   54.99
Price excludes VAT (USA)
  • ISBN: 978-3-642-33347-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   72.00
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Hoeffding, W.: Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association 58(301), 13–30 (1963)

    MathSciNet  MATH  CrossRef  Google Scholar 

  2. Lee, S., Wright, S.: Manifold identification of dual averaging methods for regularized stochastic online learning. In: Proceedings of the 28th International Conference on Machine Learning, pp. 1121–1128 (2011)

    Google Scholar 

  3. Nesterov, Y.: Primal-dual subgradient methods for convex problems. Mathematical Programming 120, 221–259 (2009)

    MathSciNet  MATH  CrossRef  Google Scholar 

  4. Rockafellar, R.T., Wets, R.J.B.: On the interchange of subdifferentiation and conditional expectation for convex functionals. Stochastics 7(3), 173–182 (1982)

    MathSciNet  MATH  CrossRef  Google Scholar 

  5. Xiao, L.: Dual averaging methods for regularized stochastic learning and online optimization. Journal of Machine Learning Research 11, 2543–2596 (2010)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Lee, S. (2012). Improving Confidence of Dual Averaging Stochastic Online Learning via Aggregation. In: Glimm, B., Krüger, A. (eds) KI 2012: Advances in Artificial Intelligence. KI 2012. Lecture Notes in Computer Science(), vol 7526. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33347-7_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-33347-7_20

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-33346-0

  • Online ISBN: 978-3-642-33347-7

  • eBook Packages: Computer ScienceComputer Science (R0)