Advertisement

Regularized Jacobi-type ADMM-methods for a class of separable convex optimization problems in Hilbert spaces

  • Eike Börgens
  • Christian KanzowEmail author
Article
  • 30 Downloads

Abstract

We consider a regularized version of a Jacobi-type alternating direction method of multipliers (ADMM) for the solution of a class of separable convex optimization problems in a Hilbert space. The analysis shows that this method is equivalent to the standard proximal-point method applied in a Hilbert space with a transformed scalar product. The method therefore inherits the known convergence results from the proximal-point method and allows suitable modifications to get a strongly convergent variant. Some additional properties are also shown by exploiting the particular structure of the ADMM-type solution method. Applications and numerical results are provided for the domain decomposition method and potential (generalized) Nash equilibrium problems in a Hilbert space setting.

Keywords

Alternating direction method of multipliers Hilbert space Proximal-point method Separable convex optimization Global convergence 

Notes

References

  1. 1.
    Attouch, H., Soueycatt, M.: Augmented Lagrangian and proximal alternating direction methods of multipliers in Hilbert spaces. Applications to games, PDE’s and control. Pac. J. Optim., 5(1):17–37, (2009)Google Scholar
  2. 2.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edn. CMS Books in Mathematics. Springer (2017)Google Scholar
  3. 3.
    Bertsekas, D.P.: Nonlinear Programming, 3rd edn. Athena Scientific, Belmont (2016)zbMATHGoogle Scholar
  4. 4.
    Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer Series in Operations Research. Springer, New York (2000)CrossRefGoogle Scholar
  5. 5.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)CrossRefzbMATHGoogle Scholar
  6. 6.
    Brézis, H., Lions, P.-L.: Produits infinis de résolvantes. Israel J. Math. 29(4), 329–345 (1978)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Börgens, E., Kanzow, C.: A distributed regularized Jacobi-type ADMM-method for generalized Nash equilibrium problems in Hilbert spaces. Numer. Funct. Anal. Optim. 39(12), 1–34 (2018)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Chen, C., He, B., Ye, Y., Yuan, X.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program., 155(1-2, Ser. A):57–79, (2016)Google Scholar
  9. 9.
    Deng, W., Lai, M.-J., Peng, Z., Yin, W.: Parallel multi-block ADMM with \(o(1/k)\) convergence. J. Sci. Comput. 71(2), 712–736 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Dobrowolski, M.: Angewandte Funktionalanalysis: Funktionalanalysis, Sobolev-Räume und elliptische Differentialgleichungen. Springer, Berlin (2010)CrossRefzbMATHGoogle Scholar
  11. 11.
    Dong, Y.: The proximal point algorithm revisited. J. Optim. Theory Appl. 161(2), 478–489 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Dong, Y.: Comments on “The proximal point algorithm revisited” [MR3193802]. J. Optim. Theory Appl. 166(1), 343–349 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Eckstein, J.: Approximate iterations in Bregman-function-based proximal algorithms. Math. Programming, 83(1, Ser. A):113–123, (1998)Google Scholar
  14. 14.
    Eckstein, J., Yao, W.: Understanding the convergence of the alternating direction method of multipliers: theoretical and computational perspectives. Pac. J. Optim. 11(4), 619–644 (2015)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. II. Springer, New York (2003)zbMATHGoogle Scholar
  16. 16.
    Glowinski, R., Le Tallec, P.: Augmented Lagrangian interpretation of the nonoverlapping Schwarz alternating method. Third International Symposium on Domain Decomposition Methods for Partial Differential Equations (Houston. TX, 1989), pp. 224–231. SIAM, Philadelphia, PA (1990)Google Scholar
  17. 17.
    Gu, G., He, B., Yuan, X.: Customized proximal point algorithms for linearly constrained convex minimization and saddle-point problems: a unified approach. Comput. Optim. Appl. 59(1–2), 135–161 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Halpern, B.: Fixed points of nonexpanding maps. Bull. Am. Math. Soc. 73, 957–961 (1967)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    He, B., Hou, L., Yuan, X.: On full Jacobian decomposition of the augmented Lagrangian method for separable convex programming. SIAM J. Optim. 25(4), 2274–2312 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    He, B., Tao, M., Yuan, X.: Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J. Optim. 22(2), 313–340 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    He, B., Tao, M., Yuan, X.: Convergence rate analysis for the alternating direction method of multipliers with a substitution procedure for separable convex programming. Math. Oper. Res. 42(3), 662–691 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    He, B., Xu, H.-K., Yuan, X.: On the proximal Jacobian decomposition of ALM for multiple-block separable convex minimization problems and its relationship to ADMM. J. Sci. Comput. 66(3), 1204–1217 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Hintermüller, M., Surowiec, T.: A PDE-constrained generalized Nash equilibrium problem with pointwise control and state constraints. Pac. J. Optim. 9(2), 251–273 (2013)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Hintermüller, M., Surowiec, T., Kämmler, A.: Generalized Nash equilibrium problems in Banach spaces: theory, Nikaido-Isoda-based path-following methods, and applications. SIAM J. Optim. 25(3), 1826–1856 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Ito, K., Kunisch, K.: Lagrange Multiplier Approach to Variational Problems and Applications. SIAM, Philadelphia, PA (2008)CrossRefzbMATHGoogle Scholar
  26. 26.
    Iusem, A., Otero, R. G.: Erratum: “Inexact versions of proximal point and augmented Lagrangian algorithms in Banach spaces” [Numer. Funct. Anal. Optim. 22 (2001), no. 5-6, 609–640; MR1849570 (2002e:90124)]. Numer. Funct. Anal. Optim., 23(1-2):227–228, (2002)Google Scholar
  27. 27.
    Kanzow, C., Karl, V., Steck, D., Wachsmuth, D.: The multiplier-penalty method for generalized Nash equilibrium problems in Banach spaces. Technical report, University of Würzburg, Institute of Mathematics, (2017)Google Scholar
  28. 28.
    Langtangen, H. P., Logg, A.: Solving PDEs in Python, volume 3 of Simula SpringerBriefs on Computing. Springer, Cham, (2016). The FEniCS tutorial IGoogle Scholar
  29. 29.
    Larsson, S., Thomée, V.: Partial differential equations with numerical methods, volume 45 of Texts in Applied Mathematics. Springer, Berlin, 2009. Paperback reprint of the 2003 editionGoogle Scholar
  30. 30.
    Lions, P.-L.: On the Schwarz alternating method. III. A variant for nonoverlapping subdomains. Third International Symposium on Domain Decomposition Methods for Partial Differential Equations (Houston. TX, 1989), pp. 202–223. SIAM, Philadelphia, PA (1990)Google Scholar
  31. 31.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999)CrossRefzbMATHGoogle Scholar
  32. 32.
    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Rockafellar, R. T.: Convex analysis. Princeton Landmarks in Mathematics. Princeton University Press, Princeton, NJ, 1997. Reprint of the 1970 original, Princeton PaperbacksGoogle Scholar
  34. 34.
    Tao, M.: Some parallel splitting methods for separable convex programming with the \(O(\frac{1}{t})\) convergence rate. Pac. J. Optim. 10(2), 359–384 (2014)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Tao, M., Yuan, X.: An inexact parallel splitting augmented Lagrangian method for monotone variational inequalities with separable structures. Comput. Optim. Appl. 52(2), 439–461 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Wang, J.J., Song, W.: An algorithm twisted from generalized ADMM for multi-block separable convex minimization models. J. Comput. Appl. Math. 309, 342–358 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Wang, X., Hong, M., Ma, S., Luo, Z.-Q.: Solving multiple-block separable convex minimization problems using two-block alternating direction method of multipliers. Pac. J. Optim. 11(4), 645–667 (2015)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Institute of MathematicsUniversity of WürzburgWürzburgGermany

Personalised recommendations