Joint European Conference on Machine Learning and Knowledge Discovery in Databases

ECML PKDD 2015: Machine Learning and Knowledge Discovery in Databases pp 645-658

Adaptive Stochastic Primal-Dual Coordinate Descent for Separable Saddle Point Problems

Conference paper

DOI: 10.1007/978-3-319-23528-8_40

Part of the Lecture Notes in Computer Science book series (LNCS, volume 9284)
Cite this paper as:
Zhu Z., Storkey A.J. (2015) Adaptive Stochastic Primal-Dual Coordinate Descent for Separable Saddle Point Problems. In: Appice A., Rodrigues P., Santos Costa V., Soares C., Gama J., Jorge A. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2015. Lecture Notes in Computer Science, vol 9284. Springer, Cham

Abstract

We consider a generic convex-concave saddle point problem with a separable structure, a form that covers a wide-ranged machine learning applications. Under this problem structure, we follow the framework of primal-dual updates for saddle point problems, and incorporate stochastic block coordinate descent with adaptive stepsizes into this framework. We theoretically show that our proposal of adaptive stepsizes potentially achieves a sharper linear convergence rate compared with the existing methods. Additionally, since we can select “mini-batch” of block coordinates to update, our method is also amenable to parallel processing for large-scale data. We apply the proposed method to regularized empirical risk minimization and show that it performs comparably or, more often, better than state-of-the-art methods on both synthetic and real-world data sets.

Keywords

Large-scale optimization Parallel optimization Stochastic coordinate descent Convex-concave saddle point problems 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Institute of Adaptive Neural Computation, School of InformaticsThe University of EdinburghEdinburghUK

Personalised recommendations