Advertisement

EURO Journal on Computational Optimization

, Volume 4, Issue 1, pp 27–46 | Cite as

On the rate of convergence of the proximal alternating linearized minimization algorithm for convex problems

  • Ron Shefi
  • Marc Teboulle
Original Paper

Abstract

We analyze the proximal alternating linearized minimization algorithm (PALM) for solving non-smooth convex minimization problems where the objective function is a sum of a smooth convex function and block separable non-smooth extended real-valued convex functions. We prove a global non-asymptotic sublinear rate of convergence for PALM. When the number of blocks is two, and the smooth coupling function is quadratic we present a fast version of PALM which is proven to share a global sublinear rate efficiency estimate improved by a squared root factor. Some numerical examples illustrate the potential benefits of the proposed schemes.

Keywords

Non-smooth convex minimization Alternating proximal methods Coordinate descent Non-asymptotic rate of convergence 

Mathematics Subject Classification

90C25 49M27 65K05 

References

  1. Auslender A (1971) Méthodes numériques pour la décomposition et la minimisation de fonctions non différentiables. Numer Math 18(3):213–223CrossRefGoogle Scholar
  2. Auslender A (1976) Optimisation: méthodes numériques. Masson, ParisGoogle Scholar
  3. Beck A (2015) On the convergence of alternating minimization for convex programming with applications to iteratively reweighted least squares and decomposition schemes. SIAM J Optim 25(1):185–209CrossRefGoogle Scholar
  4. Beck A, Teboulle M (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci 2(1):183–202CrossRefGoogle Scholar
  5. Beck A, Teboulle M (2010) Gradient-based algorithms with applications in signal recovery problems. In: Palomar D, Eldar Y (eds) Convex optimization in signal processing and communications. Cambribge University Press, LondonGoogle Scholar
  6. Beck A, Tetruashvili L (2013) On the convergence of block coordinate descent type methods. SIAM J Optim 23(4):2037–2060CrossRefGoogle Scholar
  7. Bertsekas D (1999) Nonlinear programming, 2nd edn. Athena Scientific, BelmontGoogle Scholar
  8. Bertsekas D, Tsitsiklis J (1989) Parallel and distributed computation: numerical methods. Prentice Hall, New JerseyGoogle Scholar
  9. Bolte J, Sabach S, Teboulle M (2014) Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math Progr 146(1–2):459–494CrossRefGoogle Scholar
  10. Chambolle A, Pock T (2015) A remark on accelerated block coordinate descent for computing the proximity operators of sum of convex functions. Optimization Online: http://www.optimization-online.org/DB_FILE/2015/01/4719
  11. Grippo L, Sciandrone M (2000) On the convergence of the block nonlinear Gauss–Seidel method under convex constraints. Oper Res Lett 26(3):127–136CrossRefGoogle Scholar
  12. Moreau J (1965) Proximité et dualité dans un espace hilbertien. Bull Soc Math France 93(2):273–299Google Scholar
  13. Nesterov Y (2012) Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J Optim 22(2):341–362CrossRefGoogle Scholar
  14. Ortega JM, Rheinboldt WC (1970) Iterative solution of nonlinear equations in several variables, vol 30. SIAM, PhiladelphiaGoogle Scholar
  15. Palomar D, Eldar Y (2010) Convex optimization in signal processing and communications. Cambridge University Press, LondonGoogle Scholar
  16. Powell MJ (1973) On search directions for minimization algorithms. Math Progr 4(1):193–201CrossRefGoogle Scholar
  17. Sra S, Nowozin S, Wright SJ (2011) Optimization for machine learning. MIT Press, CambridgeGoogle Scholar
  18. Tseng P (2001) Convergence of a block coordinate descent method for nondifferentiable minimization. J Optim Theory Appl 109(3):475–494CrossRefGoogle Scholar
  19. Wright SJ (2015) Coordinate descent methods. Mathematical programming series B. Special issue, international symposium on MP, Pittsburg, vol 151, pp 3–34Google Scholar

Copyright information

© EURO - The Association of European Operational Research Societies 2015

Authors and Affiliations

  1. 1.School of Mathematical SciencesTel-Aviv UniversityRamat-AvivIsrael

Personalised recommendations