Skip to main content
SpringerLink
Log in

Two-step inertial Bregman alternating minimization algorithm for nonconvex and nonsmooth problems

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

In this paper, we propose an algorithm combining Bregman alternating minimization algorithm with two-step inertial force for solving a minimization problem composed of two nonsmooth functions with a smooth one in the absence of convexity. For solving nonconvex and nonsmooth problems, we give an abstract convergence theorem for general descent methods satisfying a sufficient decrease assumption, and allowing a relative error tolerance. Our result holds under the assumption that the objective function satisfies the Kurdyka–Łojasiewicz inequality. The proposed algorithm is shown to satisfy the requirements of our abstract convergence theorem. The convergence is obtained provided an appropriate regularization of the objective function satisfies the Kurdyka–Łojasiewicz inequality. Finally, numerical results are reported to show the effectiveness of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Neumann, J.v.: Functional operators, Vol.22, Annals of Mathematics Studies, Princeton University Press, Princeton, NJ (1950)

  2. Combettes, P.L., Wajs, V.: Signal recovery by proximal forward-backward splitting. Multiscale Model Sim 4(4), 1168–1200 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  3. Donoho, D.L.: Compressed sensing. IEEE Trans. Inform. Theory 4, 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  4. Lions, P.L.: On the Schwarz alternating method. III, A variant for nonoverlapping subdomains. In: Chan, T.F., Glowinski, R., Périaux, J., Widlund, O. (eds.) Third Internat, pp. 202–231. Sympos. Domain Decomposition Methods for Partial Differential Equations, SIAM, Philadelphia (1990)

  5. Widrow, B., Wallach, E.: Adaptive Inverse Control. Prentice-Hall, Englewood Cliffs, NJ (1996)

    Google Scholar 

  6. Behmardi, B., Raich, R.: On provable exact low-rank recovery in topic models. In: proceedings of IEEE statistical signal processing workshop (SSP), Nice, (2011), 265-268

  7. Xu, H., Caramanis, C., Mannor, S.: Outlier-robust PCA: the high-dimensional case. IEEE Trans. Inform. Theor. 59, 546–572 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  8. Sra, S., Nowozin, S., Wright, S.J. (eds.): Optimization for Machine Learning. The MIT Press, Cambridge (2011)

    Google Scholar 

  9. Wang, F., Cao, W., Xu, Z.: Convergence of multi-block Bregman ADMM for nonconvex composite problems. Sci. China Inf. Sci. 61, 122101:1-122101:12 (2018)

    Article  MathSciNet  Google Scholar 

  10. Li, G., Pong, T.K.: Global convergence of splitting methods for nonconvex composite optimization. SIAM J. Optim. 25(4), 2434–2460 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  11. Hong, M., Luo, Z.Q., Razaviyayn, M.: Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems. SIAM J. Optim. 26, 337–364 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  12. Li, G., Pong, T.K.: Douglas-Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems. Math. Program. Ser. A 159, 371–401 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  13. Clason, C., Mazurenko, S., Valkonen, T.: Acceleration and global convergence of a first-order primal-dual method for nonconvex problems. SIAM J. Optim. 29(1), 933–963 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  14. Wu, Z., Li, M., Wang, D.Z.W., Han, D.: A symmetric alternating direction method of multipliers for separable nonconvex minimization problems, Asia Pac. J. Oper. Res. 34(6), 1750030 (2017)

    MathSciNet  MATH  Google Scholar 

  15. Wen, B., Chen, X., Pong, T.K.: A proximal difference-of-convex algorithm with extrapolation. Comput. Optim. Appl. 69, 297–324 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  16. Liu, Z., Wu, C., Zhao, Y.: A new globally convergent algorithm for non-Lipschitz \(l_p-l_q\) minimization. Adv. Comput. Math. 45, 1369–1399 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  17. Li, M., Wu, Z.: Convergence analysis of the generalized splitting methods for a class of nonconvex optimization problems. J. Optim. Theory Appl. 183(2), 535–565 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  18. Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-Łojasiewicz inequality. Math. Oper. Res. 35(2), 438–457 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  19. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. Ser. A 146, 459–494 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  20. Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4, 1–17 (1964)

    Article  Google Scholar 

  21. Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: inertial proximal algorithm for nonconvex optimization. SIAM J. Imaging Sci. 7(2), 1388–1419 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  22. Boţ, R.I., Csetnek, E.R.: An inertial Tseng’s type proximal algorithm for nonsmooth and nonconvex optimization problems. J. Optim. Theory Appl. 171(2), 600–616 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  23. Zhang, Y., He, S.: Inertial proximal alternating minimization for nonconvex and nonsmooth problems. J. Inequal. Appl. 2017, 232 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  24. Pock, T., Sabach, S.: Inertial proximal alternating linearized minimization (iPALM) for nonconvex and nonsmooth problems. SIAM J. Imaging Sci. 9, 1756–1787 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  25. Gao, X., Cai, X., Han, D.: A Gauss-Seidel type inertial proximal alternating linearized minimization for a class of nonconvex optimization problems. J. Global Optim. 76, 863–887 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  26. Polyak, B.T.: Introduction to Optimization, Optimization Software Inc., Publications Division: New York (1987)

  27. Liang, J., Fadili, J.M., Peyrë, G.: A multi-step inertial forward-backward splitting method for non-convex optimization, Advances in Neural Information Processing Systems, Advances in Neural Information Processing Systems 29-Proceedings of the 2016 conference, (2016), 4042-4050. Adv. neural inf. proces. syst

  28. Poon, C., Liang, J.: Geometry of first-order methods and adaptive acceleration. arXiv:2003.03910

  29. Mordukhovich, B.: Variational Analysis and Generalized Differentiation, I: Basic Theory. Grundlehren der Mathematischen Wissenschaften, Vol. 330. Springer-Verlag, Berlin (2006)

  30. Bolte, J., Daniilidis, A., Lewis, A.: The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17(4), 1205–1223 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  31. Rockafellar, R.T., Wets, R.: Variational analysis, Grundlehren der Mathematischen Wissenschaften, vol. 317. Springer, Berlin (1998)

    Google Scholar 

  32. Łojasiewicz, S., Une propriété topologique des sous-ensembles analytiques réels, in Les Équations aux Dérivées Partielles (Paris,: Éditions du centre National de la Recherche Scientifique. Paris 1963, 87–89 (1962)

  33. Kurdyka, K.: On gradients of functions definable in o-minimal structures. Ann. Inst. Fourier 48, 769–783 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  34. Xu, Y., Yin, W.: A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6, 1758–1789 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  35. Hsieh, Y.-P., Kao, Y.-C., Mahabadi, R.K., Yurtsever, A., Kyrillidis, A., Cevher, V.: A non-Euclidean gradient descent framework for non-convex matrix factorization. IEEE Trans. Signal Process 66, 5917–5926 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  36. Auslender, A., Teboulle, M.: Interior gradient and proximal methods for convex and conic optimization. SIAM J. Optim. 16, 697–725 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  37. Auslender, A., Teboulle, M.: Projected subgradient methods with non-Euclidean distances for non-differentiable convex minimization and variational inequalities. Math. Program 120, 27–48 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  38. Bauschke, H.H., Dao, M.N., Lindstrom, S.B.: Regularizing with Bregman-Moreau envelopes. SIAM J. Optim. 28, 3208–3228 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  39. Bolte, J., Sabach, S., Teboulle, M., Vaisbourd, Y.: First-order methods beyond convexity and Lipschitz gradient continuity with applications to quadratic inverse problems. SIAM J. Optim. 28, 2131–2151 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  40. Bauschke, H.H., Bolte, J., Chen, J., Teboulle, M., Wang, X.: On linear convergence of non-Euclidean gradient methods without strong convexity and Lipschitz gradient continuity. J. Optim. Theory Appl. 182, 1068–1087 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  41. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program. Ser. A 137, 91–129 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  42. Sunthrayuth, P., Jolaoso, L.O., Cholamjiak, P.: New Bregman projection methods for solving pseudo-monotone variational inequality problem. J. Appl. Math. Comput. (2021). https://doi.org/10.1007/s12190-021-01581-2

    Article  MATH  Google Scholar 

  43. Hieu, D.V., Cholamjiak, P.: Modified extragradient method with Bregman distance for variational inequalities. Appl. Anal. 101(2), 655–670 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  44. Xu, Z.B., Chang, X.Y., Xu, F.M., et al.: \(L_{1/2}\) regularization: a thresholding representation theory and a fast solver. IEEE Trans. Neural Netw. Learning Syst. 23, 1013–1027 (2012)

    Google Scholar 

  45. Zeng, J., Xu, Z., Zhang, B., et al.: Accelerated \(L_{1/2}\) regularization based SAR imaging via BCR and reduced Newton skills. Signal Process. 93(7), 1831–1844 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

We are grateful to Professor Chunlin Wu, Miss Yuan Li and Mr. Xuan Lin for valuable help in Numerical experiments. We express our thanks to two anonymous referees for their constructive suggestions, which significant improved the presentation of this paper.

Author information

Authors and Affiliations

  1. College of Science, Civil Aviation University of China, Tianjin, 300300, China

    Jing Zhao & Qiao-Li Dong

  2. Department of Mathematics and Engineering Sciences, Hellenic Military Academy, 16673, Vari Attikis, Greece

    Michael Th. Rassias

  3. Institute for Advanced Study, Program in Interdisciplinary Studies, 1 Einstein Dr, Princeton, NJ, 08540, USA

    Michael Th. Rassias

  4. Department of Mathematics, Luoyang Normal University, Luoyang, China

    Fenghui Wang

Authors
  1. Jing Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiao-Li Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Th. Rassias
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fenghui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiao-Li Dong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supported by Scientific Research Project of Tianjin Municipal Education Commission (2020ZD02)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, J., Dong, QL., Rassias, M.T. et al. Two-step inertial Bregman alternating minimization algorithm for nonconvex and nonsmooth problems. J Glob Optim 84, 941–966 (2022). https://doi.org/10.1007/s10898-022-01176-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-022-01176-6

Keywords

Mathematics Subject Classification

Search

Navigation