Skip to main content
SpringerLink
Log in

An alternating structure-adapted Bregman proximal gradient descent algorithm for constrained nonconvex nonsmooth optimization problems and its inertial variant

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

We consider the nonconvex nonsmooth minimization problem over abstract sets, whose objective function is the sum of a proper lower semicontinuous biconvex function of the entire variables and two smooth nonconvex functions of their private variables. Fully exploiting the problem structure, we propose an alternating structure-adapted Bregman proximal (ASABP for short) gradient descent algorithm, where the geometry of the abstract set and the function is captured by employing generalized Bregman function. Under the assumption that the underlying function satisfies the Kurdyka–Łojasiewicz property, we prove that each bounded sequence generated by ASABP globally converges to a critical point. We then adopt an inertial strategy to accelerate the ASABP algorithm (IASABP), and utilize a backtracking line search scheme to find “suitable” step sizes, making the algorithm efficient and robust. The global O(1/K) sublinear convergence rate measured by Bregman distance is also established. Furthermore, to illustrate the potential of ASABP and its inertial version (IASABP), we apply them to solving the Poisson linear inverse problem, and the results are promising.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Data availability

All data generated or analysed during this study are included in the paper.

References

  1. Ahookhosh, M., Hien, L., Gillis, N., Patrinos, P.: A block inertial Bregman proximal algorithm for nonsmooth nonconvex problems with application to symmetric nonnegative matrix tri-factorization. J. Optim. Theory Appl. 190, 234–258 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  2. Ahookhosh, M., Hien, L., Gillis, N., Patrinos, P.: Multi-block Bregman proximal alternating linearized minimization and its application to orthogonal nonnegative matrix factorization. Comput. Optim. Appl. 79, 681–775 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  3. Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-Łojasiewicz inequality. Math. Oper. Res. 35, 438–457 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  4. Attouch, H., Bolte, J., Svaiter, B.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized gauss-seidel methods. Math. Program. 137, 91–129 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bauschke, H., Bolte, J., Chen, J., Teboulle, M., Wang, X.: On linear convergence of non-Euclidean gradient methods without strong convexity and Lipschitz gradient continuity. J. Optim. Theory Appl. 182, 1068–1087 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  6. Bauschke, H., Bolte, J., Teboulle, M.: A descent lemma beyond Lipschitz gradient continuity: first-order methods revisited and applications. Math. Oper. Res. 42, 330–348 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  7. Benning, M., Betcke, M., Ehrhardt, M., Schonlieb, C.: Choose your path wisely: gradient descent in a Bregman distance framework. SIAM J. Imaging Sci. 14, 814–843 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bertero, M., Boccacci, P., Desiderà, G., Vicidomini, G.: Image deblurring with Poisson data: from cells to galaxies. Inverse Probl 25, 123006 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  9. Bertsekas, D.: Nonlinear programming. J. Oper. Res. Soc. 48, 334–334 (1997)

    Article  Google Scholar 

  10. Bertsekas, D., Tsitsiklis, J.: Parallel and Distributed Computation: Numerical Methods. Athena Scientific (2015)

  11. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  12. Bolte, J., Sabach, S., Teboulle, M., Vaisbourd, Y.: First order methods beyond convexity and Lipschitz gradient continuity with applications to quadratic inverse problems. SIAM J. Optim. 28, 2131–2151 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  13. Censor, Y., Iusem, A., Zenios, S.: An interior point method with Bregman functions for the variational inequality problem with paramonotone operators. Math. Program. 81, 373–400 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  14. Gao, X., Cai, X., Han, D.: A Gauss-Seidel type inertial proximal alternating linearized minimization for a class of nonconvex optimization problems. J. Global Optim. 76, 863–887 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  15. Han, D.: A new hybrid generalized proximal point algorithm for variational inequality problems. J. Global Optim. 26, 125–140 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  16. Hohage, T., Werner, F.: Inverse problems with Poisson data: statistical regularization theory, applications and algorithms. Inverse Probl 32, 093001 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  17. Lu, H., Freund, R., Nesterov, Y.: Relatively smooth convex optimization by first-order methods, and applications. SIAM J. Optim. 28, 333–354 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  18. Mukkamala, M., Ochs, P., Pock, T., Sabach, S.: Convex-concave backtracking for inertial Bregman proximal gradient algorithms in nonconvex optimization. SIAM J. Math. Data Sci. 2, 658–682 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  19. Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Springer (2003)

  20. Nikolova, M., Tan, P.: Alternating structure-adapted proximal gradient descent for nonconvex nonsmooth block-regularized problems. SIAM J. Optim. 29, 2053–2078 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  21. Pock, T., Sabach, S.: Inertial proximal alternating linearized minimization (iPALM) for nonconvex and nonsmooth problems. SIAM J. Imaging Sci. 9, 1756–1787 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  22. Polyak, B.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Maths. Math. Phys. 4, 1–17 (1964)

    Article  Google Scholar 

  23. Rockafellar, R.: Convex Analysis. Princeton University Press, Princeton (1970)

    Book  MATH  Google Scholar 

  24. Rockafellar, R., Wets, R.: Variational Analysis. Springer (2009)

  25. Teboulle, M.: A simplified view of first order methods for optimization. Math. Program. 170, 67–96 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  26. Wu, Z., Li, C., Li, M., Lim, A.: Inertial proximal gradient methods with bregman regularization for a class of nonconvex optimization problems. J. Global Optim. 79, 617–644 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  27. Xu, Y., Yin, W.: A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6, 1758–1789 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  28. Zhang, H., Dai, Y., Guo, L., Peng, W.: Proximal-like incremental aggregated gradient method with linear convergence under Bregman distance growth conditions. Math. Oper. Res. 46, 61–81 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  29. Zhang, X., Zhang, X., Li, X., Li, Z., Wang, S.: Classify social image by integrating multi-modal content. Multimed. Tools. Appl. 77, 7469–7485 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

The authors are grateful to the editor and the anonymous reviewer for their valuable comments and suggestions toward the improvement of this paper. This work was supported by National Natural Science Foundation of China (12201173, 12131004, 12126603).

Author information

Authors and Affiliations

  1. Institute of Mathematics, Hebei University of Technology, Tianjin, 300401, People’s Republic of China

    Xue Gao

  2. School of Mathematical Sciences, Jiangsu Key Lab for NSLSCS, Nanjing Normal University, Nanjing, 210023, People’s Republic of China

    Xingju Cai

  3. School of Computer Science and Technology, East China Normal University, Shanghai, 200026, People’s Republic of China

    Xiangfeng Wang

  4. School of Mathematical Sciences, LMIB, Beihang University, Beijing, 100191, People’s Republic of China

    Deren Han

Authors
  1. Xue Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xingju Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangfeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Deren Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deren Han.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, X., Cai, X., Wang, X. et al. An alternating structure-adapted Bregman proximal gradient descent algorithm for constrained nonconvex nonsmooth optimization problems and its inertial variant. J Glob Optim 87, 277–300 (2023). https://doi.org/10.1007/s10898-023-01300-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-023-01300-0

Keywords

Mathematics Subject Classification

Search

Navigation