iPiasco: Inertial Proximal Algorithm for Strongly Convex Optimization
- 498 Downloads
In this paper, we present a forward–backward splitting algorithm with additional inertial term for solving a strongly convex optimization problem of a certain type. The strongly convex objective function is assumed to be a sum of a non-smooth convex and a smooth convex function. This additional knowledge is used for deriving a worst-case convergence rate for the proposed algorithm. It is proved to be an optimal algorithm with linear rate of convergence. For certain problems this linear rate of convergence is better than the provably optimal worst-case rate of convergence for smooth strongly convex functions. We demonstrate the efficiency of the proposed algorithm in numerical experiments and examples from image processing.
KeywordsHeavy-ball method Strongly convex optimization Inertial proximal method Convergence analysis
Thomas Pock acknowledges support from the Austrian science fund (FWF) under the START project BIVISION, No. Y729. Peter Ochs and Thomas Brox acknowledge funding by the German Research Foundation (DFG Grant BR 3815/5-1).
- 4.Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. CMS Books in Mathematics, Springer (2011)Google Scholar
- 8.Chambolle, A., Pock, T.: On the ergodic convergence rates of a first-order primal-dual algorithm (2014). to appearGoogle Scholar
- 16.Hong, M., Luo, Z.-Q.: On the linear convergence of the alternating direction method of multipliers. ArXiv e-prints, August (2012)Google Scholar
- 17.Mainberger, M., Weickert, J.: Edge-based image compression with homogeneous diffusion. In: Jiang, Xiaoyi, Petkov, Nicolai (eds.) Computer Analysis of Images and Patterns. Lecture Notes in Computer Science, vol. 5702, pp. 476–483. Springer, Berlin (2009)Google Scholar
- 19.Nesterov, Y.: A method of solving a convex programming problem with convergence rate \(O(1/k^2)\). Soviet Math. Doklady 27, 372–376 (1983)Google Scholar
- 20.Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Applied Optimization, vol. 87. Kluwer Academic Publishers, Boston (2004)Google Scholar
- 25.Poljak, B.T.: Introduction to optimization. Optimization Software (1987)Google Scholar
- 29.Vegeta from Dragon Ball Z. http://1.bp.blogspot.com/-g3wpzDWE0QI/UbycqlJWjDI/AAAAAAAAAbU/ty0EZl0kqXw/s1600/VEGETA%2B1.jpg