Journal of Mathematical Imaging and Vision

, Volume 59, Issue 3, pp 481–497 | Cite as

Accelerated Alternating Descent Methods for Dykstra-Like Problems

  • Antonin ChambolleEmail author
  • Pauline Tan
  • Samuel Vaiter


This paper extends recent results by the first author and T. Pock (ICG, TU Graz, Austria) on the acceleration of alternating minimization techniques for quadratic plus nonsmooth objectives depending on two variables. We discuss here the strongly convex situation, and how ‘fast’ methods can be derived by adapting the overrelaxation strategy of Nesterov for projected gradient descent. We also investigate slightly more general alternating descent methods, where several descent steps in each variable are alternatively performed.


Alternating minimizations Block descent algorithms Accelerated methods Total variation minimization 



This work is supported by the ANR via the international project ‘EANOI’ (Efficient Algorithms for Nonsmooth Optimization in Imaging), FWF No. I1148 / ANR-12-IS01-0003. A. Chambolle also benefits from support of the ‘Programme Gaspard Monge pour l’Optimisation et la Recherche Opérationnelle’ (PGMO), through the ‘MAORI’ group, as well as the ‘GdR MIA’ of the CNRS. He also warmly thanks Churchill College and DAMTP, Centre for Mathematical Sciences, University of Cambridge, for their kind hospitality during the completion of this work, thanks to a support of the French Embassy in the UK and the Cantab Capital Institute for Mathematics of Information.


  1. 1.
    Aujol, J.-F., Dossal, C.: Stability of over-relaxations for the forward-backward algorithm, application to FISTA. SIAM J. Optim. 25(4), 2408–2433 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Beck, A.: On the convergence of alternating minimization for convex programming with applications to iteratively reweighted least squares and decomposition schemes. SIAM J. Optim. 25(1), 185–209 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Beck, A., Teboulle, M.: A fast iterative shrinkage–thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Beck, A., Tetruashvili, L.: On the convergence of block coordinate descent type methods. SIAM J. Optim. 23(4), 2037–2060 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Boyle, J.P., Dykstra, R.L.: A method for finding projections onto the intersection of convex sets in Hilbert spaces. In: Dykstra, R., Robertson, T., Wright, F.T. (eds) Advances in Order Restricted Statistical Inference (Iowa City, Iowa, 1985), vol. 37 of Lecture Notes in Statistics, pp. 28–47. Springer, Berlin (1986)Google Scholar
  6. 6.
    Braides, A.: \(Gamma\)-Convergence for Beginners. Number 22 in Oxford Lecture Series in Mathematics and Its Applications. Oxford University Press, Oxford (2002)Google Scholar
  7. 7.
    Cai, J.-F., Dong, B., Osher, S., Shen, Z.: Image restoration: total variation, wavelet frames, and beyond. J. Am. Math. Soc. 25(4), 1033–1089 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Chambolle, A., Dossal, C.: On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”. J. Optim. Theory Appl. 166(3), 968–982 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Chambolle, A., Pock, T.: On the ergodic convergence rates of a first-order primal-dual algorithm. Math. Program. 159, 253–287 (2016)Google Scholar
  11. 11.
    Chambolle, A., Pock, T.: A remark on accelerated block coordinate descent for computing the proximity operators of a sum of convex functions. SMAI J. Comput. Math. 1, 29–54 (2015)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Chambolle, A., Pock, T.: An introduction to continuous optimization for imaging. Acta Numer. 25, 161–319, 5 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Combettes, P.L., Pesquet, J.-C.: Proximal splitting methods in signal processing. In: Bauschke, H.H., Burachik, R., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering, vol. 49 of Springer Optimization and Applications, pp. 185–212. Springer, New York (2011)Google Scholar
  14. 14.
    Dal Maso, G.: An Introduction to \(\Gamma \)-Convergence. Birkhäuser, Boston (1993)CrossRefzbMATHGoogle Scholar
  15. 15.
    Demengel, F., Temam, R.: Convex functions of a measure and applications. Indiana Univ. Math. J. 33(5), 673–709 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Deutsch, F., Hundal, H.: The rate of convergence of Dykstra’s cyclic projections algorithm: the polyhedral case. Numer. Funct. Anal. Optim. 15(5–6), 537–565 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Evans, L.C., Gariepy, R.F.: Measure Theory and Fine Properties of Functions. CRC Press, Boca Raton (1992)zbMATHGoogle Scholar
  18. 18.
    Nemirovski, A.S., Yudin, D.: Informational complexity of mathematical programming. Izv. Akad. Nauk SSSR Tekhn. Kibernet. 1, 88–117 (1983)MathSciNetGoogle Scholar
  19. 19.
    Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, vol. 87 of Applied Optimization. Kluwer, Boston (2004)CrossRefzbMATHGoogle Scholar
  20. 20.
    Shefi, R., Teboulle, M.: On the rate of convergence of the proximal alternating linearized minimization algorithm for convex problems. EURO J. Comput. Optim. 4(1), 27–46 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Villa, S., Salzo, S., Baldassarre, L., Verri, A.: Accelerated and inexact forward–backward algorithms. SIAM J. Optim. 23(3), 1607–1633 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Ziemer, W.P.: Weakly Differentiable Functions. Sobolev Spaces and Functions of Bounded Variation. Springer, New York (1989)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.CMAP, CNRSEcole PolytechniquePalaiseauFrance
  2. 2.IMB, CNRSUniversité de BourgogneDijonFrance

Personalised recommendations