Advertisement

Mathematical Programming

, Volume 174, Issue 1–2, pp 433–451 | Cite as

Stochastic quasi-Fejér block-coordinate fixed point iterations with random sweeping II: mean-square and linear convergence

  • Patrick L. CombettesEmail author
  • Jean-Christophe Pesquet
Full Length Paper Series B

Abstract

Combettes and Pesquet (SIAM J Optim 25:1221–1248, 2015) investigated the almost sure weak convergence of block-coordinate fixed point algorithms and discussed their applications to nonlinear analysis and optimization. This algorithmic framework features random sweeping rules to select arbitrarily the blocks of variables that are activated over the course of the iterations and it allows for stochastic errors in the evaluation of the operators. The present paper establishes results on the mean-square and linear convergence of the iterates. Applications to monotone operator splitting and proximal optimization algorithms are presented.

Keywords

Block-coordinate algorithm Fixed-point algorithm Mean-square convergence Monotone operator splitting Linear convergence Stochastic algorithm 

Mathematics Subject Classification

47J25 46M10 65K10 90C15 

References

  1. 1.
    Acker, F., Prestel, M.A.: Convergence d’un schéma de minimisation alternée. Ann. Fac. Sci. Toulouse V. Sér. Math. 2, 1–9 (1980)CrossRefzbMATHGoogle Scholar
  2. 2.
    Atchadé, Y.F., Fort, G., Moulines, E.: On perturbed proximal gradient algorithms. J. Mach. Learn. Res. 18, 1–33 (2017)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Attouch, H., Briceño-Arias, L.M., Combettes, P.L.: A parallel splitting method for coupled monotone inclusions. SIAM J. Control Optim. 48, 3246–3270 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Baillon, J.-B., Combettes, P.L., Cominetti, R.: There is no variational characterization of the cycles in the method of periodic projections. J. Funct. Anal. 262, 400–408 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Bauschke, H.H., Bello Cruz, J.Y., Nghia, T.T.A., Phan, H.M., Wang, X.: Optimal rates of linear convergence of relaxed alternating projections and generalized Douglas–Rachford methods for two subspaces. Numer. Algorithms 73, 33–76 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edn. Springer, New York (2017)CrossRefzbMATHGoogle Scholar
  7. 7.
    Bauschke, H.H., Combettes, P.L., Reich, S.: The asymptotic behavior of the composition of two resolvents. Nonlinear Anal. 60, 283–301 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Bauschke, H.H., Moffat, S.M., Wang, X.: Firmly nonexpansive mappings and maximally monotone operators: correspondence and duality. Set Valued Var. Anal. 20, 131–153 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Boţ, R.I., Csetnek, E.R.: Convergence rates for forward–backward dynamical systems associated with strongly monotone inclusions. J. Math. Anal. Appl. 457, 1135–1152 (2018)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Boţ, R.I., Csetnek, E.R., Heinrich, A., Hendrich, C.: On the convergence rate improvement of a primal-dual splitting algorithm for solving monotone inclusion problems. Math. Program. 150, 251–279 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Combettes, P.L., Pesquet, J.-C.: Stochastic quasi-Fejér block-coordinate fixed point iterations with random sweeping. SIAM J. Optim. 25, 1221–1248 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Combettes, P.L., Pesquet, J.-C.: Stochastic approximations and perturbations in forward–backward splitting for monotone operators. Pure Appl. Funct. Anal. 1, 13–37 (2016)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Combettes, P.L., Vũ, B.C.: Variable metric forward–backward splitting with applications to monotone inclusions in duality. Optimization 63, 1289–1318 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Fortet, R.M.: Vecteurs, Fonctions et Distributions Aléatoires dans les Espaces de Hilbert. Hermès, Paris (1995)zbMATHGoogle Scholar
  15. 15.
    Ledoux, M., Talagrand, M.: Probability in Banach Spaces: Isoperimetry and Processes. Springer, New York (1991)CrossRefzbMATHGoogle Scholar
  16. 16.
    Loève, M.: Probability Theory II, 4th edn. Springer, New York (1978)CrossRefzbMATHGoogle Scholar
  17. 17.
    Moreau, J.J.: Fonctions convexes duales et points proximaux dans un espace hilbertien. C. R. Acad. Sci. Paris Sér. A Math. 255, 2897–2899 (1962)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Nesterov, Yu.: Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J. Optim. 22, 341–362 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Pesquet, J.-C., Pustelnik, N.: A parallel inertial proximal optimization method. Pac. J. Optim. 8, 273–305 (2012)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Richtárik, P., Takáč, M.: Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function. Math. Program. A144, 1–38 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Richtárik, P., Takáč, M.: On optimal probabilities in stochastic coordinate descent methods. Optim. Lett. 10, 1233–1243 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Rockafellar, R.T., Wets, R.J.B.: Variational Analysis, 3rd edn. Springer, New York (2009)zbMATHGoogle Scholar
  24. 24.
    Rosasco, L., Villa, S., Vũ, B.C.: Stochastic forward–backward splitting method for solving monotone inclusions in Hilbert spaces. J. Optim. Theory Appl. 169, 388–406 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Schwartz, L.: Analyse III—Calcul Intégral. Hermann, Paris (1993)zbMATHGoogle Scholar
  26. 26.
    Sibony, M.: Méthodes itératives pour les équations et inéquations aux dérivées partielles non linéaires de type monotone. Calcolo 7, 65–183 (1970)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2018

Authors and Affiliations

  • Patrick L. Combettes
    • 1
    Email author
  • Jean-Christophe Pesquet
    • 2
  1. 1.Department of MathematicsNorth Carolina State UniversityRaleighUSA
  2. 2.CentraleSupélec, Université Paris-Saclay, Center for Visual ComputingChâtenay-MalabryFrance

Personalised recommendations