Abstract
The self-concordant-like property of a smooth convex function is a new analytical structure that generalizes the self-concordant notion. While a wide variety of important applications feature the self-concordant-like property, this concept has heretofore remained unexploited in convex optimization. To this end, we develop a variable metric framework of minimizing the sum of a “simple” convex function and a self-concordant-like function. We introduce a new analytic step-size selection procedure and prove that the basic gradient algorithm has improved convergence guarantees as compared to “fast” algorithms that rely on the Lipschitz gradient property. Our numerical tests with real-data sets show that the practice indeed follows the theory.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bach, F.: Self-concordant analysis for logistic regression. Electron. J. Statist. 4, 384–414 (2010)
Bach, F.: Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression (2013)
Beck, A., Teboulle, M.: A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems. SIAM J. Imaging Sciences 2(1), 183–202 (2009)
Becker, S., Candès, E.J., Grant, M.: Templates for convex cone problems with applications to sparse signal recovery. Mathematical Programming Computation 3(3), 165–218 (2011)
Becker, S., Fadili, M.J.: A quasi-Newton proximal splitting method. In: Adv. Neural Information Processing Systems (2012)
Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press (2004)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)
Lee, J.D., Sun, Y., Saunders, M.A.: Proximal Newton-type methods for convex optimization. SIAM J. Optim. 24(3), 1420–1443 (2014)
Mine, H., Fukushima, M.: A minimization method for the sum of a convex function and a continuously differentiable function. J. Optim. Theory Appl. 33, 9–23 (1981)
Nesterov, Y.: Introductory lectures on convex optimization: a basic course. Applied Optimization, vol. 87. Kluwer Academic Publishers (2004)
Nesterov, Y.: Gradient methods for minimizing composite objective function. Math. Program. 140(1), 125–161 (2013)
Nesterov, Y., Nemirovski, A.: Interior-point Polynomial Algorithms in Convex Programming. Society for Industrial Mathematics (1994)
Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer Series in Operations Research and Financial Engineering. Springer (2006)
Parikh, N., Boyd, S.: Proximal algorithms. Foundations and Trends in Optimization 1(3), 123–231 (2013)
Robinson, S.M.: Strongly Regular Generalized Equations. Mathematics of Operations Research 5(1), 43–62 (1980)
Rockafellar, R.T.: Convex Analysis. Princeton Mathematics Series, vol. 28. Princeton University Press (1970)
Schmidt, M., Roux, N.L., Bach, F.: Convergence rates of inexact proximal-gradient methods for convex optimization. In: NIPS, Granada, Spain (2011)
Tran-Dinh, Q., Kyrillidis, A., Cevher, V.: Composite self-concordant minimization. J. Mach. Learn. Res. 15, 1–54 (2014) (accepted)
Tran-Dinh, Q., Li, Y.-H., Cevher, V.: Composite convex minimization involving self-concordant-like cost functions. LIONS-Tech. Report., 1–19 (2015), http://arxiv.org/abs/1502.01068
Yuan, Y.X.: Recent advances in numerical methods for nonlinear equations and nonlinear least squares. Numerical Algebra, Control and Optimization 1(1), 15–34 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Tran-Dinh, Q., Li, YH., Cevher, V. (2015). Composite Convex Minimization Involving Self-concordant-Like Cost Functions. In: Le Thi, H., Pham Dinh, T., Nguyen, N. (eds) Modelling, Computation and Optimization in Information Systems and Management Sciences. Advances in Intelligent Systems and Computing, vol 359. Springer, Cham. https://doi.org/10.1007/978-3-319-18161-5_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-18161-5_14
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18160-8
Online ISBN: 978-3-319-18161-5
eBook Packages: EngineeringEngineering (R0)