Improving Confidence of Dual Averaging Stochastic Online Learning via Aggregation
Stochastic online learning algorithms typically exhibit slow convergence speed, but their solutions of moderate accuracy often suffice in practice. Since the outcomes of these algorithms are random variables, not only their accuracy but also their probability of achieving a certain accuracy, called confidence, is important. We show that a rather simple aggregation of outcomes from parallel dual averaging runs can provide a solution with improved confidence, and it can be controlled by the number of runs, independently of the length of learning processes.
KeywordsEmpirical Risk Subgradient Method Dual Average Simple Aggregation Machine Learn Research
Unable to display preview. Download preview PDF.
- 2.Lee, S., Wright, S.: Manifold identification of dual averaging methods for regularized stochastic online learning. In: Proceedings of the 28th International Conference on Machine Learning, pp. 1121–1128 (2011)Google Scholar