Skip to main content
SpringerLink
Log in

Malitsky-Tam forward-reflected-backward splitting method for nonconvex minimization problems

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

We extend the Malitsky-Tam forward-reflected-backward (FRB) splitting method for inclusion problems of monotone operators to nonconvex minimization problems. By assuming the generalized concave Kurdyka-Łojasiewicz (KL) property of a quadratic regularization of the objective, we show that the FRB method converges globally to a stationary point of the objective and enjoys the finite length property. Convergence rates are also given. The sharpness of our approach is guaranteed by virtue of the exact modulus associated with the generalized concave KL property. Numerical experiments suggest that FRB is competitive compared to the Douglas-Rachford method and the Boţ-Csetnek inertial Tseng’s method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data availability

The data that support the findings of Table 1 are available from the corresponding author upon request.

Notes

  1. In the remainder, we shall omit adjectives “pointwise" and “setwise" whenever there is no ambiguity.

  2. We also performed simulations with much larger problem size (\(n=4000,5000,6000\)), in which case FRB, DR, and iTseng tend to stuck at stationary points while DRh can still hit global minimizers. We believe that this is due to its heuristics; see also [6, Section 5].

References

  1. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, Cham (2017)

    Book  Google Scholar 

  2. Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simulat. 4, 1168–1200 (2005)

    Article  MathSciNet  Google Scholar 

  3. Tseng, P.: A modified forward-backward splitting method for maximal monotone mappings. SIAM J. Control. Optim. 38, 431–446 (2000)

    Article  MathSciNet  Google Scholar 

  4. Malitsky, Y., Tam, M.K.: A forward-backward splitting method for monotone inclusions without cocoercivity. SIAM J. Optim. 30, 1451–1472 (2020)

    Article  MathSciNet  Google Scholar 

  5. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2, 183–202 (2009)

    Article  MathSciNet  Google Scholar 

  6. Li, G., Pong, T.K.: Douglas-Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems. Math. Program. 159, 371–401 (2016)

    Article  MathSciNet  Google Scholar 

  7. Wang, X., Wang, Z.: The exact modulus of the generalized Kurdyka-Łojasiewicz property. Math. Oper. Res. (2021). arXiv:2008.13257, to appear

  8. Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-Łojasiewicz inequality. Math. Oper. Res. 35, 438–457 (2010)

    Article  MathSciNet  Google Scholar 

  9. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel Methods. Math. Program. 137, 91–129 (2013)

    Article  MathSciNet  Google Scholar 

  10. Banert, S., Boţ, R.I.: A general double-proximal gradient algorithm for dc programming. Math. Program. 178, 301–326 (2019)

    Article  MathSciNet  Google Scholar 

  11. Bolte, J., Sabach, S., Teboulle, M., Vaisbourd, Y.: First order methods beyond convexity and Lipschitz gradient continuity with applications to quadratic inverse problems. SIAM J. Optim. 28, 2131–2151 (2018)

    Article  MathSciNet  Google Scholar 

  12. Boţ, R.I., Csetnek, E.R.: An inertial Tseng’s type proximal algorithm for nonsmooth and nonconvex optimization problems. J. Optim. Theory Appl. 171, 600–616 (2016)

    Article  MathSciNet  Google Scholar 

  13. Li, G., Liu, T., Pong, T.K.: Peaceman-Rachford splitting for a class of nonconvex optimization problems. Comput. Optim. Appl. 68, 407–436 (2017)

    Article  MathSciNet  Google Scholar 

  14. Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I: Basic Theory. Springer-Verlag, Berlin (2006)

    Book  Google Scholar 

  15. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer-Verlag, Berlin (1998)

    Book  Google Scholar 

  16. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014)

    Article  MathSciNet  Google Scholar 

  17. Bolte, J., Daniilidis, A., Lewis, A., Shiota, M.: Clarke subgradients of stratifiable functions. SIAM J. Optim. 18, 556–572 (2007)

    Article  MathSciNet  Google Scholar 

  18. Boţ, R.I., Nguyen, D.-K.: The proximal alternating direction method of multipliers in the nonconvex setting: convergence analysis and rates. Math. Oper. Res. 45, 682–712 (2020)

    Article  MathSciNet  Google Scholar 

  19. Bolte, J., Daniilidis, A., Ley, O., Mazet, L.: Characterizations of Łojasiewicz inequalities: subgradient flows, talweg, convexity. Trans. Am. Math. Soc. 362, 3319–3363 (2010)

    Article  Google Scholar 

  20. Łojasiewicz, S.: Une propriété topologique des sous-ensembles analytiques réels. Les équations aux dérivées partielles 117, 87–89 (1963)

    MATH  Google Scholar 

  21. Kurdyka, K.: On gradients of functions definable in o-minimal structures. Annales de l’institut Fourier 48, 769–783 (1998)

    Article  MathSciNet  Google Scholar 

  22. Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: inertial proximal algorithm for nonconvex optimization. SIAM J. Imag. Sci. 7, 1388–1419 (2014)

    Article  MathSciNet  Google Scholar 

  23. Böhm, A., Sedlmayer, M., Csetnek, E. R., Boţ, R. I.: Two steps at a time–taking gan training in stride with Tseng’s method, arXiv:2006.09033, (2020)

  24. Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. 116, 5–16 (2009)

    Article  MathSciNet  Google Scholar 

  25. Li, G., Pong, T.K.: Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods. Found. Comput. Math. 18, 1199–1232 (2018)

    Article  MathSciNet  Google Scholar 

  26. Boţ, R.I., Csetnek, E.R., Nguyen, D.-K.: A proximal minimization algorithm for structured nonconvex and nonsmooth problems. SIAM J. Optim. 29, 1300–1328 (2019)

    Article  MathSciNet  Google Scholar 

  27. Chen, C., Pong, T.K., Tan, L., Zeng, L.: A difference-of-convex approach for split feasibility with applications to matrix factorizations and outlier detection. J. Global Optim. 78, 107–136 (2020)

    Article  MathSciNet  Google Scholar 

  28. Bauschke, H.H., Luke, D.R., Phan, H.M., Wang, X.: Restricted normal cones and sparsity optimization with affine constraints. Found. Comput. Math. 14, 63–83 (2014)

    Article  MathSciNet  Google Scholar 

  29. Lu, Z., Zhang, Y.: Sparse approximation via penalty decomposition methods. SIAM J. Optim. 23, 2448–2478 (2013)

    Article  MathSciNet  Google Scholar 

  30. Boţ, R.I., Csetnek, E.R., László, S.C.: An inertial forward-backward algorithm for the minimization of the sum of two nonconvex functions. EURO J. Comput. Optimiz. 4, 3–25 (2016)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors thank the editor and the reviewers for helpful suggestions and constructive feedback which helped us to improve the presentation of the results. XW and ZW were partially supported by NSERC Discovery Grants.

Author information

Authors and Affiliations

  1. Mathematics, University of British Columbia, Kelowna, B.C., V1V 1V7, Canada

    Xianfu Wang & Ziyuan Wang

Authors
  1. Xianfu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziyuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianfu Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Wang, Z. Malitsky-Tam forward-reflected-backward splitting method for nonconvex minimization problems. Comput Optim Appl 82, 441–463 (2022). https://doi.org/10.1007/s10589-022-00364-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-022-00364-0

Keywords

Mathematics Subject Classification

Search

Navigation