Skip to main content
SpringerLink
Log in

A partial Bregman ADMM with a general relaxation factor for structured nonconvex and nonsmooth optimization

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

In this paper, a partial Bregman alternating direction method of multipliers (ADMM) with a general relaxation factor \(\alpha \in (0,\frac{1+\sqrt{5}}{2})\) is proposed for structured nonconvex and nonsmooth optimization, where the objective function is the sum of a nonsmooth convex function and a smooth nonconvex function without coupled variables. We add a Bregman distance to alleviate the difficulty of solving the nonsmooth subproblem. For the smooth subproblem, we directly perform a gradient descent step of the augmented Lagrangian function, which makes the computational cost of each iteration of our method very cheap. To our knowledge, the nonconvex ADMM with a relaxation factor \(\alpha \ne 1\) in the literature has never been studied for the problem under consideration. Under some mild conditions, the boundedness of the generated sequence, the global convergence and the iteration complexity are established. The numerical results verify the efficiency and robustness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

The datasets tested in this study are openly available in the LIBSVM repository, https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/.

References

  1. Shen, Y., Wen, Z., Zhang, Y.: Augmented Lagrangian alternating direction method for matrix separation based on low-rank factorization. Optim. Methods Softw. 29(2), 239–263 (2014)

    Article  MathSciNet  Google Scholar 

  2. Xu, Y.: Alternating proximal gradient method for sparse nonnegative Tucker decomposition. Math. Program. Comput. 7(1), 39–70 (2015)

    Article  MathSciNet  Google Scholar 

  3. Allen, G.: Sparse higher-order principal components analysis. In: The 15th International conference on artificial intelligence and statistics (2012)

  4. Ames, B., Hong, M.: Alternating direction method of multipliers for penalized zero-variance discriminant analysis. Comput. Optim. Appl. 64(3), 725–754 (2016)

    Article  MathSciNet  Google Scholar 

  5. Hong, M., Luo, Z.-Q., Razaviyayn, M.M.: Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems. SIAM J. Optim. 26(1), 337–364 (2016)

    Article  MathSciNet  Google Scholar 

  6. Glowinski, R., Marrocco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de dirichlet non linéaires. ESAIM Math. Model. Numer. Anal. 9, 41–76 (1975)

    Google Scholar 

  7. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2, 17–40 (1976)

    Article  Google Scholar 

  8. Eckstein, J., Ferris, M.C.: Operator-splitting methods for monotone affine variational inequalities, with a parallel application to optimal control. INFORMS J. Comput. 10(2), 218–235 (1998)

    Article  MathSciNet  Google Scholar 

  9. Wang, Y., Yang, J., Yin, W., et al.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1, 248–272 (2008)

    Article  MathSciNet  Google Scholar 

  10. Boyd, S., Parikh, N., Chu, E., et al.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  11. He, B.S., Yuan, X.M.: On the \(O(1/n)\) convergence rate of the Douglas-Rachford alternating direction method. SIAM J. Numer. Anal. 50, 700–709 (2012)

    Article  MathSciNet  Google Scholar 

  12. Han, D.R., Yuan, X.M.: Local linear convergence of the alternating direction method of multipliers for quadratic programs. SIAM J. Numer. Anal. 51(6), 3446–3457 (2013)

    Article  MathSciNet  Google Scholar 

  13. He, B.S., Liu, H., Wang, Z.R., et al.: A strictly contractive Peaceman-Rachford splitting method for convex programming. SIAM J. Optim. 24(3), 1011–1040 (2014)

    Article  MathSciNet  Google Scholar 

  14. Chen, C., He, B.S., Ye, Y.Y., et al.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program. 155(1–2), 57–79 (2016)

    Article  MathSciNet  Google Scholar 

  15. Chen, L.M., Gu, Y.T.: The convergence guarantees of a non-convex approach for sparse recovery. IEEE Trans. Signal Process. 62(15), 3754–3767 (2014)

    Article  MathSciNet  Google Scholar 

  16. Wang, F., Xu, Z.B., Xu, H.-K.: Convergence of Bregman alternating direction method with multipliers for nonconvex composite problems. arXiv:1410.8625 (2014)

  17. Shen, X.Y., Chen, L.M., Gu, Y.T., et al.: Square-root lasso with non-convex regularization: an ADMM approach. IEEE Signal Process. Lett. 23(7), 934–938 (2016)

    Article  Google Scholar 

  18. Melo, J.G., Monteiro. R.D.C.: Iteration-complexity of a linearized proximal multiblock ADMM class for linearly constrained nonconvex optimization problems. Available on: http://www.optimization-online.org (2017)

  19. Goncalves, M.L.N., Melo, J.G., Monteiro, R.D.C.: Convergence rate bounds for a proximal ADMM with over-relaxation stepsize parameter for solving nonconvex linearly constrained problems. Pac. J. Optim. 15(3), 379–398 (2019)

    MathSciNet  Google Scholar 

  20. Wu, Z., Li, M., Wang, D.Z.W., et al.: A symmetric alternating direction method of multipliers for separable nonconvex minimization problems. Asia-Pac. J. Oper. Res. 34(6), 1750030 (2017)

    Article  MathSciNet  Google Scholar 

  21. Themelis, A., Patrinos, P.: Douglas-Rachford splitting and ADMM for nonconvex optimization: tight convergence results. SIAM J. Optim. 30(1), 149–181 (2020)

    Article  MathSciNet  Google Scholar 

  22. Boţ, R.I., Nguyen, D.K.: The proximal alternating direction method of multipliers in the nonconvex setting: convergence analysis and rates. Math. Oper. Res. 45(2), 682–712 (2020)

    Article  MathSciNet  Google Scholar 

  23. Jia, Z., Gao, X., Cai, X., et al.: Local linear convergence of the alternating direction method of multipliers for nonconvex separable optimization problems. J. Optim. Theory Appl. 188(1), 1–25 (2021)

    Article  MathSciNet  Google Scholar 

  24. Jia, Z., Gao, X., Cai, X., et al.: The convergence rate analysis of the symmetric ADMM for the nonconvex separable optimization problems. J. Ind. Manag. Optim. 17(4), 1943–1971 (2021)

    Article  MathSciNet  Google Scholar 

  25. Jia, Z., Huang, J., Wu, Z.: An incremental aggregated proximal ADMM for linearly constrained nonconvex optimization with application to sparse logistic regression problems. J. Comput. Appl. Math. 390, 113384 (2021)

    Article  MathSciNet  Google Scholar 

  26. Yashtini, M.: Convergence and rate analysis of a proximal linearized ADMM for nonconvex nonsmooth optimization. J. Global Optim. 84, 913–939 (2022)

    Article  MathSciNet  Google Scholar 

  27. Themelis, A., Stella, L., Patrinos, P.: Douglas-Rachford splitting and ADMM for nonconvex optimization: accelerated and Newton-type linesearch algorithms. Comput. Optim. Appl. 82, 395–440 (2022)

    Article  MathSciNet  Google Scholar 

  28. Han, D.R.: A survey on some recent developments of alternating direction method of multipliers. J. Oper. Res. Soc. China 10(1), 1–52 (2022)

    Article  MathSciNet  Google Scholar 

  29. Li, G.Y., Pong, T.K.: Global convergence of splitting methods for nonconvex composite optimization. SIAM J. Optim. 25(4), 2434–2460 (2015)

    Article  MathSciNet  Google Scholar 

  30. Yang, L., Pong, T.K., Chen, X.: Alternating direction method of multipliers for a class of nonconvex and nonsmooth problems with applications to background/foreground extraction. SIAM J. Imaging Sci. 10, 74–110 (2017)

    Article  MathSciNet  Google Scholar 

  31. Wang, F., Cao, W., Xu, Z.: Convergence of multi-block Bregman ADMM for nonconvex composite problems. Sci. China Inf. Sci. 61(12), 122101 (2018)

    Article  MathSciNet  Google Scholar 

  32. Wang, Y., Yin, W., Zeng, J.: Global convergence of ADMM in nonconvex nonsmooth optimization. J. Sci. Comput. 78, 29–63 (2019)

    Article  MathSciNet  Google Scholar 

  33. Lu, Y., Huang, M., Zhang, Y., et al.: A nonconvex ADMM for a class of sparse inverse semidefinite quadratic programming problems. Optimization 68(6), 1075–1105 (2019)

    Article  MathSciNet  Google Scholar 

  34. Jiang, B., Lin, T.Y., Ma, S.Q., et al.: Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis. Comput. Optim. Appl. 72, 115–157 (2019)

    Article  MathSciNet  Google Scholar 

  35. Jian, J., Liu, P., Yin, J., et al.: A QCQP-based splitting SQP algorithm for two-block nonconvex constrained optimization problems with application. J. Comput. Appl. Math. 390, 113368 (2021)

    Article  MathSciNet  Google Scholar 

  36. Jian, J.B., Zhang, C., Yin, J.H.: A Peaceman-Rachford splitting sequential quadratic programming method with double step-lengths for two-block nonconvex optimization (in Chinese). Sci. Sin. Math. 52, 1449–1476 (2022)

    Article  Google Scholar 

  37. Rockafellar, R.T., Wets, R.J.-B.: Variational analysis. Springer, Berlin (2009)

    Google Scholar 

  38. Bagirov, A., Karmitsa, N., Mäkelä, M.M.: Introduction to nonsmooth optimization: theory practice and software. Springer, New York (2014)

    Book  Google Scholar 

  39. Li, G., Pong, T.K.: Douglas-Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems. Math. Program. 159(1), 371–401 (2016)

    Article  MathSciNet  Google Scholar 

  40. Milzarek, A., Xiao, X., Cen, S., et al.: A stochastic semismooth Newton method for nonsmooth nonconvex optimization. SIAM J. Optim. 29(4), 2916–2948 (2019)

    Article  MathSciNet  Google Scholar 

  41. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 1–27 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the two anonymous referees for their constructive and pertinent suggestions that improved the quality of the paper significantly.

Funding

This work was supported by the National Natural Science Foundation of China (12171106), the Natural Science Foundation of Guangxi Province (2023GXNSFBA026029), the National Natural Science Foundation of China (12271113), Research Project of Guangxi Minzu University (2022KJQD03) and the Middle-aged and Young Teachers’ Basic Ability Promotion Project of Guangxi Province (2023KY0168).

Author information

Author notes

  1. Jianghua Yin and Chunming Tang have contributed equally to this work and should be regarded as co-first authors.

Authors and Affiliations

  1. School of Mathematics and Physics, Center for Applied Mathematics of Guangxi, Guangxi Minzu University, Nanning, 530006, China

    Jianghua Yin, Jinbao Jian & Qiongxuan Huang

  2. College of Mathematics and Information Science, Guangxi University, Nanning, 530004, China

    Chunming Tang

Authors
  1. Jianghua Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunming Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinbao Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiongxuan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jinbao Jian.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yin, J., Tang, C., Jian, J. et al. A partial Bregman ADMM with a general relaxation factor for structured nonconvex and nonsmooth optimization. J Glob Optim (2024). https://doi.org/10.1007/s10898-024-01384-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10898-024-01384-2

Keywords

Mathematics Subject Classification

Search

Navigation