Accelerated Alternating Descent Methods for Dykstra-Like Problems

Article

Abstract

This paper extends recent results by the first author and T. Pock (ICG, TU Graz, Austria) on the acceleration of alternating minimization techniques for quadratic plus nonsmooth objectives depending on two variables. We discuss here the strongly convex situation, and how ‘fast’ methods can be derived by adapting the overrelaxation strategy of Nesterov for projected gradient descent. We also investigate slightly more general alternating descent methods, where several descent steps in each variable are alternatively performed.

Keywords

Alternating minimizations Block descent algorithms Accelerated methods Total variation minimization 

References

  1. 1.
    Aujol, J.-F., Dossal, C.: Stability of over-relaxations for the forward-backward algorithm, application to FISTA. SIAM J. Optim. 25(4), 2408–2433 (2015)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Beck, A.: On the convergence of alternating minimization for convex programming with applications to iteratively reweighted least squares and decomposition schemes. SIAM J. Optim. 25(1), 185–209 (2015)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Beck, A., Teboulle, M.: A fast iterative shrinkage–thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Beck, A., Tetruashvili, L.: On the convergence of block coordinate descent type methods. SIAM J. Optim. 23(4), 2037–2060 (2013)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Boyle, J.P., Dykstra, R.L.: A method for finding projections onto the intersection of convex sets in Hilbert spaces. In: Dykstra, R., Robertson, T., Wright, F.T. (eds) Advances in Order Restricted Statistical Inference (Iowa City, Iowa, 1985), vol. 37 of Lecture Notes in Statistics, pp. 28–47. Springer, Berlin (1986)Google Scholar
  6. 6.
    Braides, A.: \(Gamma\)-Convergence for Beginners. Number 22 in Oxford Lecture Series in Mathematics and Its Applications. Oxford University Press, Oxford (2002)Google Scholar
  7. 7.
    Cai, J.-F., Dong, B., Osher, S., Shen, Z.: Image restoration: total variation, wavelet frames, and beyond. J. Am. Math. Soc. 25(4), 1033–1089 (2012)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Chambolle, A., Dossal, C.: On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”. J. Optim. Theory Appl. 166(3), 968–982 (2015)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Chambolle, A., Pock, T.: On the ergodic convergence rates of a first-order primal-dual algorithm. Math. Program. 159, 253–287 (2016)Google Scholar
  11. 11.
    Chambolle, A., Pock, T.: A remark on accelerated block coordinate descent for computing the proximity operators of a sum of convex functions. SMAI J. Comput. Math. 1, 29–54 (2015)CrossRefGoogle Scholar
  12. 12.
    Chambolle, A., Pock, T.: An introduction to continuous optimization for imaging. Acta Numer. 25, 161–319, 5 (2016)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Combettes, P.L., Pesquet, J.-C.: Proximal splitting methods in signal processing. In: Bauschke, H.H., Burachik, R., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering, vol. 49 of Springer Optimization and Applications, pp. 185–212. Springer, New York (2011)Google Scholar
  14. 14.
    Dal Maso, G.: An Introduction to \(\Gamma \)-Convergence. Birkhäuser, Boston (1993)CrossRefMATHGoogle Scholar
  15. 15.
    Demengel, F., Temam, R.: Convex functions of a measure and applications. Indiana Univ. Math. J. 33(5), 673–709 (1984)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Deutsch, F., Hundal, H.: The rate of convergence of Dykstra’s cyclic projections algorithm: the polyhedral case. Numer. Funct. Anal. Optim. 15(5–6), 537–565 (1994)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Evans, L.C., Gariepy, R.F.: Measure Theory and Fine Properties of Functions. CRC Press, Boca Raton (1992)MATHGoogle Scholar
  18. 18.
    Nemirovski, A.S., Yudin, D.: Informational complexity of mathematical programming. Izv. Akad. Nauk SSSR Tekhn. Kibernet. 1, 88–117 (1983)MathSciNetGoogle Scholar
  19. 19.
    Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, vol. 87 of Applied Optimization. Kluwer, Boston (2004)CrossRefMATHGoogle Scholar
  20. 20.
    Shefi, R., Teboulle, M.: On the rate of convergence of the proximal alternating linearized minimization algorithm for convex problems. EURO J. Comput. Optim. 4(1), 27–46 (2016)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Villa, S., Salzo, S., Baldassarre, L., Verri, A.: Accelerated and inexact forward–backward algorithms. SIAM J. Optim. 23(3), 1607–1633 (2013)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Ziemer, W.P.: Weakly Differentiable Functions. Sobolev Spaces and Functions of Bounded Variation. Springer, New York (1989)CrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.CMAP, CNRSEcole PolytechniquePalaiseauFrance
  2. 2.IMB, CNRSUniversité de BourgogneDijonFrance

Personalised recommendations