Advertisement

Pointwise and Ergodic Convergence Rates of a Variable Metric Proximal Alternating Direction Method of Multipliers

  • Max L. N. Gonçalves
  • Maicon Marques Alves
  • Jefferson G. Melo
Article

Abstract

In this paper, we obtain global pointwise and ergodic convergence rates for a variable metric proximal alternating direction method of multipliers for solving linearly constrained convex optimization problems. We first propose and study nonasymptotic convergence rates of a variable metric hybrid proximal extragradient framework for solving monotone inclusions. Then, the convergence rates for the former method are obtained essentially by showing that it falls within the latter framework. To the best of our knowledge, this is the first time that global pointwise (resp. pointwise and ergodic) convergence rates are obtained for the variable metric proximal alternating direction method of multipliers (resp. variable metric hybrid proximal extragradient framework).

Keywords

Alternating direction method of multipliers Variable metric Pointwise and ergodic convergence rates Hybrid proximal extragradient method Convex program 

AMS subject classification

90C25 90C60 49M27 47H05 47J22 65K10 

Notes

Acknowledgements

The work of these authors was supported in part by CNPq Grants 406250/2013-8, 444134/2014-0, 309370/2014-0 and 406975/2016-7. We thank the reviewers for their careful reading and comments.

References

  1. 1.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)CrossRefzbMATHGoogle Scholar
  2. 2.
    Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2, 17–40 (1976)CrossRefzbMATHGoogle Scholar
  3. 3.
    Glowinski, R., Marroco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par penalisation-dualité, d’une classe de problèmes de Dirichlet non linéaires (1975)Google Scholar
  4. 4.
    Attouch, H., Soueycatt, M.: Augmented Lagrangian and proximal alternating direction methods of multipliers in Hilbert spaces. Applications to games, PDE’s and control. Pac. J. Optim. 5(1), 17–37 (2008)MathSciNetzbMATHGoogle Scholar
  5. 5.
    He, B., Liu, H., Wang, Z., Yuan, X.: A strictly contractive peaceman-rachford splitting method for convex programming. SIAM J. Optim. 24(3), 1011–1040 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Davis, D., Yin, W.: Convergence rate analysis of several splitting schemes. In: Glowinski, R., Osher, S., Yin, W. (eds.) Splitting Methods in Communication, Imaging, Science and Engineering, pp. 115–163. Springer, New York (2016)CrossRefGoogle Scholar
  8. 8.
    Deng, W., Yin, W.: On the global and linear convergence of the generalized alternating direction method of multipliers. J. Sci. Comput. 66(3), 889–916 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Eckstein, J.: Some saddle-function splitting methods for convex programming. Optim. Method Softw. 4(1), 75–83 (1994)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Fang, E.X., Bingsheng, H., Liu, H., Xiaoming, Y.: Generalized alternating direction method of multipliers: new theoretical insights and applications. Math. Prog. Comput. 7(2), 149–187 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Fazel, M., Pong, T.K., Sun, D., Tseng, P.: Hankel matrix rank minimization with applications to system identification and realization. SIAM J. Matrix Anal. Appl. 34(3), 946–977 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Gonçalves, M.L.N., Melo, J.G., Monteiro, R.D.C.: Improved pointwise iteration-complexity of a regularized ADMM and of a regularized non-euclidean HPE framework. SIAM J. Optim. 27(1), 379–407 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Hager, W.W., Yashtini, M., Zhang, H.: An \({O}(1/k)\) convergence rate for the variable stepsize Bregman operator splitting algorithm. SIAM J. Numer. Anal. 54(3), 1535–1556 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    He, B., Liao, L.Z., Han, D., Yang, H.: A new inexact alternating directions method for monotone variational inequalities. Math. Program. 92(1, Ser. A), 103–118 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    He, B., Yuan, X.: On the \(\cal{O}(1/n)\) convergence rate of the Douglas–Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Lin, T., Ma, S., Zhang, S.: An extragradient-based alternating direction method for convex minimization. Found. Comput. Math. 17(1), 35–59 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Ouyang, Y., Chen, Y., Lan, G., Pasiliao Jr., E.: An accelerated linearized alternating direction method of multipliers. SIAM J. Imaging Sci. 8(1), 644–681 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Shefi, R., Teboulle, M.: Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization. SIAM J. Optim. 24(1), 269–297 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    BoŢ, R.I., Csetnek, E.R.: ADMM for monotone operators: convergence analysis and rates. https://arxiv.org/pdf/1705.01913.pdf
  20. 20.
    Monteiro, R.D.C., Svaiter, B.F.: Iteration-complexity of block-decomposition algorithms and the alternating direction method of multipliers. SIAM J. Optim. 23(1), 475–507 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Solodov, M.V., Svaiter, B.F.: A hybrid approximate extragradient-proximal point algorithm using the enlargement of a maximal monotone operator. Set Valued Anal. 7(4), 323–345 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Zhang, X., Burger, M., Bresson, X., Osher, S.: Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM J. Imaging Sci. 3(3), 253–276 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    He, B., Yuan, X.: On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers. Numer. Math. 130(3), 567–577 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Cui, Y., Li, X., Sun, D., Toh, K.C.: On the convergence properties of a majorized ADMM for linearly constrained convex optimization problems with coupled objective functions. J. Optim. Theory Appl. 169(3), 1013–1041 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Gonçalves, M.L.N., Melo, J.G., Monteiro, R.D.C.: Extending the ergodic convergence rate of the proximal ADMM. https://arxiv.org/pdf/1611.02903.pdf
  26. 26.
    Shen, L., Pan, S.: Weighted iteration complexity of the sPADMM on the KKT residuals for convex composite optimization. https://arxiv.org/pdf/1611.03167.pdf
  27. 27.
    He, B., Yang, H.: Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities. Oper. Res. Lett. 23(3–5), 151–161 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    He, B.S., Yang, H., Wang, L.: Alternating direction method with self-adaptive penalty parameters for monotone variational inequalities. J. Optim. Theory Appl. 106(2), 337–356 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Solodov, M.V.: A class of decomposition methods for convex optimization and monotone variational inclusions via the hybrid inexact proximal point framework. Optim. Method Softw. 19(5), 557–575 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Lotito, P.A., Parente, L.A., Solodov, M.V.: A class of variable metric decomposition methods for monotone variational inclusions. J. Convex Anal. 16(3&4), 857–880 (2009)MathSciNetzbMATHGoogle Scholar
  31. 31.
    Banert, S., BoŢ, R.I., Csetnek, E.R.: Fixing and extending some recent results on the ADMM algorithm. https://arxiv.org/pdf/1612.05057.pdf
  32. 32.
    Monteiro, R.D.C., Svaiter, B.F.: On the complexity of the hybrid proximal extragradient method for the iterates and the ergodic mean. SIAM J. Optim. 20(6), 2755–2787 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    He, Y., Monteiro, R.D.C.: An accelerated HPE-type algorithm for a class of composite convex-concave saddle-point problems. SIAM J. Optim. 26(1), 29–56 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Marques Alves, M., Monteiro, R.D.C., Svaiter, B.F.: Regularized HPE-type methods for solving monotone inclusions with improved pointwise iteration-complexity bounds. SIAM J. Optim. 26(4), 2730–2743 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Monteiro, R.D.C., Svaiter, B.F.: Complexity of variants of Tseng’s modified F-B splitting and Korpelevich’s methods for hemivariational inequalities with applications to saddle-point and convex optimization problems. SIAM J. Optim. 21(4), 1688–1720 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Parente, L.A., Lotito, P.A., Solodov, M.V.: A class of inexact variable metric proximal point algorithms. SIAM J. Optim. 19(1), 240–260 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Rockafellar, R.T.: On the maximal monotonicity of subdifferential mappings. Pac. J. Math. 33, 209–216 (1970)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Burachik, R.S., Sagastizábal, C.A., Svaiter, B.F.: \(\epsilon \)-enlargements of maximal monotone operators: theory and applications. In: Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods (Lausanne, 1997), Appl. Optim., vol. 22, pp. 25–43. Kluwer Acad. Publ., Dordrecht (1999)Google Scholar
  39. 39.
    Burachik, R.S., Iusem, A.N., Svaiter, B.F.: Enlargement of monotone operators with applications to variational inequalities. Set Valued Anal. 5(2), 159–180 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Max L. N. Gonçalves
    • 1
  • Maicon Marques Alves
    • 2
  • Jefferson G. Melo
    • 1
  1. 1.IMEUniversidade Federal de GoiásGoiâniaBrazil
  2. 2.Departamento de MatemáticaUniversidade Federal de Santa CatarinaFlorianópolisBrazil

Personalised recommendations