Skip to main content
SpringerLink
Log in

An inexact version of the symmetric proximal ADMM for solving separable convex optimization

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

In this paper, we propose and analyze an inexact version of the symmetric proximal alternating direction method of multipliers (ADMM) for solving linearly constrained optimization problems. Basically, the method allows its first subproblem to be solved inexactly in such way that a relative approximate criterion is satisfied. In terms of the iteration number k, we establish global \(\boldsymbol {\mathcal {O}} \mathbf {(1/ \sqrt {k})}\) pointwise and \({\boldsymbol {\mathcal {O}} \mathbf {(1/ {k}})}\) ergodic convergence rates of the method for a domain of the acceleration parameters, which is consistent with the largest known one in the exact case. Since the symmetric proximal ADMM can be seen as a class of ADMM variants, the new algorithm as well as its convergence rates generalize, in particular, many others in the literature. Numerical experiments illustrating the practical advantages of the method are reported. To the best of our knowledge, this work is the first one to study an inexact version of the symmetric proximal ADMM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Adona, V.A., Gonċalves, M.L.N., Melo, J.G.: Iteration-complexity analysis of a generalized alternating direction method of multipliers. J. Glob. Optim. 73, 331–348 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  2. Adona, V.A., Gonċalves, M.L.N., Melo, J.G.: A partially inexact proximal alternating direction method of multipliers and its iteration-complexity analysis. J. Optim Theory Appl. 182, 640–666 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  3. Adona, V.A., Gonċalves, M.L.N., Melo, J.G.: An inexact proximal generalized alternating direction method of multipliers. Comput Optim. Appl. 76, 621–647 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  4. Alves, M.M., Eckstein, J., Geremia, M., Melo, J.G.: Relative-error inertial-relaxed inexact versions of Douglas-Rachford and admm splitting algorithms. Comput. Optim Appl. 75, 389–422 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  5. Attouch, H., Soueycatt, M.: Augmented Lagrangian and proximal alternating direction methods of multipliers in Hilbert spaces. Applications to games, PDE’s and control. Pac. J Optim. 5, 17–37 (2008)

    MathSciNet  MATH  Google Scholar 

  6. Bai, J., Li, J., Xu, F., Zhang, H.: Generalized symmetric ADMM for separable convex optimization. Comput Optim. Appl. 70, 129–170 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  7. Banert, S., BoŢ, R.I., Csetnek, E.R.: Fixing and extending some recent results on the ADMM algorithm. Numer Algorith. 86, 1303–1325 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bertsekas, D.P.: Constrained Optimization and Lagrange Multiplier Methods. Academic Press, New York (1982)

    MATH  Google Scholar 

  9. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach. Learn. 3, 1–122 (2011)

    Article  MATH  Google Scholar 

  10. Chang, X., Liu, S., Zhao, P., Song, D.: A generalization of linearized alternating direction method of multipliers for solving two-block separable convex programming. J. Comput. Appl. Math. 357, 251–272 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  11. Chen, J., Wang, Y., He, H.: Convergence analysis of positive-indefinite proximal ADMM with a glowinski’s relaxation factor. Numer Algorith. 83, 1415–140 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  12. Eckstein, J.: Some saddle-function splitting methods for convex programming. Optim Method Softw. 4, 75–83 (1994)

    Article  MathSciNet  Google Scholar 

  13. Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Programming 55, 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  14. Eckstein, J., Yao, W.: Approximate ADMM algorithms derived from Lagrangian splitting. Comput Optim. Appl. 68, 363–405 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  15. Eckstein, J., Yao, W.: Relative-error approximate versions of Douglas–Rachford splitting and special cases of the. ADMM Math. Programming 170, 417–444 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  16. Fortin, M., Glowinski, R.: On decomposition-coordination methods using an augmented Lagrangian. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, vol. 15 of studies in mathematics and its applications, Elsevier, pp. 97–146 (1983)

  17. Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, vol. 15 of studies in mathematics and its applications, Elsevier, Amsterdam, pp. 299–331 (1983)

  18. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  19. Gao, B., Ma, F.: Symmetric alternating direction method with indefinite proximal regularization for linearly constrained convex optimization. J. Optim. Theory Appl. 176, 178–204 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  20. Glowinski, R.: Numerical methods for nonlinear variational problems, Springer Series in Computational Physics, Springer-Verlag (1984)

  21. Glowinski, R., Marroco, A.: Sur l’approximation,, par éléments finis d’ordre un, et la résolution, par penalisation-dualité, d’une classe de problèmes de Dirichlet non linéaires. R.A.I.R.O. 9, 41–76 (1975)

    MATH  Google Scholar 

  22. Gonċalves, M.L.N., Alves, M.M., Melo, J.G.: Pointwise and ergodic convergence rates of a variable metric proximal alternating direction method of multipliers. J. Optim Theory Appl. 177, 448–478 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  23. Gonċalves, M.L.N., Melo, J.G., Monteiro, R.D.C.: On the iteration-complexity of a non-Euclidean hybrid proximal extragradient framework and of a proximal. ADMM Optimization 69, 847–873 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  24. Hager, W.W., Zhang, H.: Convergence rates for an inexact ADMM applied to separable convex optimization. Comput Optim. Appl. 77, 729–754 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  25. He, B., Liu, H., Wang, Z., Yuan, X.: A strictly contractive Peaceman–Rachford splitting method for convex programming. SIAM J. Optim. 24, 1011–1040 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  26. He, B., Ma, F., Yuan, X.: Convergence study on the symmetric version of ADMM with larger step sizes. SIAM J. Imaging Sci. 9, 1467–1501 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  27. He, B., Yuan, X.: On non-ergodic convergence rate of Douglas-Rachford alternating direction method of multipliers. Numer Math. 130, 567–577 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  28. Ma, F.: Convergence study on the proximal alternating direction method with larger step size. Numer Algorith. 85, 399–425 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  29. Ng, M., Wang, F., Yuan, X.: Inexact alternating direction methods for image recovery. SIAM J. Sci. Comput. 33, 1643–1668 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  30. Nocedal, J., Wright, S.J.: Numerical Optimization 2nd. Springer, New York (2006)

    MATH  Google Scholar 

  31. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)

    Book  MATH  Google Scholar 

  32. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys D 60, 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  33. Shen, Y., Zuo, Y., Yu, A.: A partial PPa S-ADMM for multi-block for separable convex optimization with linear constraints. Optimization online (2020)

  34. Sun, H., Tian, M., Sun, M.: The symmetric ADMM with indefinite proximal regularization and its application. J. Inequal. Appl. 2017, 1–22 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  35. Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1, 248–272 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  36. Wu, Z., Li, M.: An LQP-based symmetric alternating direction method of multipliers with larger step sizes. J. Oper. Res. Soc. China 7, 365–383 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  37. Wu, Z., Li, M., Wang, D.Z.W., Han, D.: A symmetric alternating direction method of multipliers for separable nonconvex minimization problems. Asia-Pac. J. Oper. Res 34, 1750030 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  38. Wu, Z., Liu, F., Li, M.: A proximal Peaceman–Rachford splitting method for solving the multi-block separable convex minimization problems. Int. J. Comput. Math. 96, 708–728 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  39. Xie, J., Liao, A., Yang, X.: An inexact alternating direction method of multipliers with relative error criteria. Optim Lett. 11, 583–596 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  40. Yang, J., Yin, W., Zhang, Y., Wang, Y.: A fast algorithm for edge-preserving variational multichannel image restoration. SIAM J. Imaging Sci. 2, 569–592 (2009)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Funding

The work of these authors was supported in part by CNPq Grant 405349/2021-1 and 304133/2021-3.

Author information

Author notes

    Authors and Affiliations

    1. IME, Universidade Federal de Goias, Campus II, 131, CEP 74001-970, Goiânia, Goiás, Brazil

      Vando A. Adona & Max L. N. Gonçalves

    Authors
    1. Vando A. Adona
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Max L. N. Gonçalves
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Max L. N. Gonçalves.

    Ethics declarations

    Conflict of interest

    The authors declare no competing interests.

    Additional information

    Data availability

    The numerical codes of the current study are available from the corresponding author on reasonable request.

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Max L. N. Gonçalves contributed equally to this work.

    Rights and permissions

    Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Adona, V.A., Gonçalves, M.L.N. An inexact version of the symmetric proximal ADMM for solving separable convex optimization. Numer Algor 94, 1–28 (2023). https://doi.org/10.1007/s11075-022-01491-9

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11075-022-01491-9

    Keywords

    Mathematics Subject Classification (2010)

    Search

    Navigation