Skip to main content
SpringerLink
Log in

Newton-Type Inertial Algorithms for Solving Monotone Equations Governed by Sums of Potential and Nonpotential Operators

  • Published:
Applied Mathematics & Optimization Aims and scope Submit manuscript

Abstract

In a Hilbert space setting, this paper is devoted to the study of a class of first-order algorithms which aim to solve structured monotone equations involving the sum of potential and nonpotential operators. Precisely, we are looking for the zeros of an operator \(A= \nabla f +B \), where \(\nabla f\) is the gradient of a differentiable convex function f, and B is a nonpotential monotone and cocoercive operator. This study is based on the inertial autonomous dynamic previously studied by the authors, which involves dampings controlled respectively by the Hessian of f, and by a Newton-type correction term attached to B. These geometric dampings attenuate the oscillations which occur with the inertial methods with viscous damping. Temporal discretization of this dynamic provides fully splitted proximal-gradient algorithms. Their convergence properties are proven using Lyapunov analysis. These results open the door to the design of first-order accelerated algorithms in numerical optimization taking into account the specific properties of potential and nonpotential terms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Abbas, B., Attouch, H., Svaiter, B.F.: Newton-like dynamics and forward-backward methods for structured monotone inclusions in Hilbert spaces. J. Optim. Theory Appl. 161(2), 331–360 (2014)

    Article  MathSciNet  Google Scholar 

  2. Adly, S., Attouch, H.: Finite convergence of proximal-gradient inertial algorithms combining dry friction with Hessian-driven damping. SIAM J. Optim. 30(3), 2134–2162 (2020)

    Article  MathSciNet  Google Scholar 

  3. Adly, S., Attouch, H., Vo V.N.: Asymptotic behavior of Newton-like inertial dynamics involving the sum of potential and nonpotential terms (2021). https://hal.archives-ouvertes.fr/hal-03213925

  4. Alecsa, C.D., László, S., Pinta, T.: An extension of the second order dynamical system that models Nesterov’s convex gradient method. Applied Mathematics and Optimization (2020). https://doi.org/10.1007/s00245-020-09692-1

  5. Alvarez, F., Attouch, H.: An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Valued Anal. 9(1–2), 3–11 (2001)

    Article  MathSciNet  Google Scholar 

  6. Alvarez, F., Attouch, H., Bolte, J., Redont, P.: A second-order gradient-like dissipative dynamical system with Hessian-driven damping. Application to optimization and mechanics. J. Math. Pures Appl. 81(8), 747–779 (2002)

    Article  MathSciNet  Google Scholar 

  7. Attouch, H., Cabot, A.: Convergence of a relaxed inertial proximal algorithm for maximally monotone operators. Math. Program. 184, 243–287 (2020)

    Article  MathSciNet  Google Scholar 

  8. Attouch, H., Chbani, Z., Fadili, J., Riahi, H.: First-order algorithms via inertial systems with Hessian-driven damping. Math. Program. (2020). https://doi.org/10.1007/s10107-020-01591-1, hal-02193846

  9. Attouch, H., Chbani, Z., Peypouquet, J., Redont, P.: Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity. Math. Program. Ser. B 168, 123–175 (2018)

    Article  MathSciNet  Google Scholar 

  10. Attouch, H., László, S.C.: Continuous newton-like inertial dynamics for monotone inclusions. Set-Valued Var. Anal. (2020). https://doi.org/10.1007/s11228-020-00564-y

  11. Attouch, H., László, S.C.: Newton-like inertial dynamics and proximal algorithms governed by maximally monotone operators. SIAM J. Optim. 30(4), 3252–3283 (2020)

    Article  MathSciNet  Google Scholar 

  12. Attouch, H., Maingé, P.E.: Asymptotic behavior of second order dissipative evolution equations combining potential with nonpotential effects. ESAIM Control Optim. Calc. Var. 17(3), 836–857 (2011)

    Article  MathSciNet  Google Scholar 

  13. Attouch, H., Maingé, P.E., Redont, P.: A second-order differential system with Hessian-driven damping; application to nonelastic shock laws. Differ. Equ. Appl. 4(1), 27–65 (2012)

    MathSciNet  MATH  Google Scholar 

  14. Attouch, H., Marques Alves, M., Svaiter, B.F.: A dynamic approach to a proximal-Newton method for monotone inclusions in Hilbert Spaces, with complexity \(\cal{O}(1/n^2)\). J. Convex Anal. 23(1), 139–180 (2016)

    MathSciNet  MATH  Google Scholar 

  15. Attouch, H., Peypouquet, J.: Convergence of inertial dynamics and proximal algorithms governed by maximal monotone operators. Math. Program. 174(1–2), 391–432 (2019)

    Article  MathSciNet  Google Scholar 

  16. Attouch, H., Peypouquet, J., Redont, P.: Fast convex minimization via inertial dynamics with Hessian driven damping. J. Differ. Equ. 261(10), 5734–5783 (2016)

    Article  Google Scholar 

  17. Attouch, H., Redont, P., Svaiter, B.F.: Global convergence of a closed-loop regularized Newton method for solving monotone inclusions in Hilbert spaces. J. Optim. Theory Appl. 157(3), 624–650 (2013)

    Article  MathSciNet  Google Scholar 

  18. Attouch, H., Svaiter, B.F.: A continuous dynamical Newton-Like approach to solving monotone inclusions. SIAM J. Control Optim. 49(2), 574–598 (2011)

    Article  MathSciNet  Google Scholar 

  19. Baillon, J.-B., Haddad, G.: Quelques propriétés des opérateurs angles-bornés et n-cycliquement monotones. Isr. J. Math. 26, 137–150 (1977)

    Article  Google Scholar 

  20. Bauschke, H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. CMS Books in Mathematics, Springer, Berlin (2011)

    Book  Google Scholar 

  21. Boţ, R.I., Csetnek, E.R.: Second order forward-backward dynamical systems for monotone inclusion problems. SIAM J. Control Optim. 54, 1423–1443 (2016)

    Article  MathSciNet  Google Scholar 

  22. Boţ, R.I., Csetnek, E.R., László, S.C.: Tikhonov regularization of a second order dynamical system with Hessian damping. Math. Program. (2020). https://doi.org/10.1007/s10107-020-01528-8

  23. Castera, C., Bolte, J., Févotte, C., Pauwels, E.: An Inertial Newton Algorithm for Deep Learning. (2019), HAL-02140748

  24. Kim, D.: Accelerated proximal point method for maximally monotone operators. Math. Program. Ser. A, (accepted)(2020). Preprint available at arXiv:1905.05149v3

  25. Lin, T., Jordan, M.I.: A control-theoretic perspective on optimal high-order optimization (2019). arXiv:1912.07168v1

  26. Nesterov, Y.: A method for solving the convex programming problem with convergence rate \(O(1/k^2)\). (Russian) Dokl. Akad. Nauk SSSR 269(3), 543–547 (1983)

    Google Scholar 

  27. Maingé, P.E.: First-order continuous Newton-like systems for monotone inclusions. SIAM J. Control Optim. 51(2), 1615–1638 (2013)

    Article  MathSciNet  Google Scholar 

  28. Shi, B., Du, S.S., Jordan, M.I., Su, W.J.: Understanding the acceleration phenomenon via high-resolution differential equations. Math. Program. (2018). https://doi.org/10.1007/s10107-021-01681-8

  29. Su, W., Boyd, S., Candès, E.J.: A differential equation for modeling Nesterov’s accelerated gradient method. J. Mach. Learn. Res. 17, 1–43 (2016)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratoire XLIM, Université de Limoges, 123, Avenue Albert Thomas, 87060, Limoges, France

    Samir Adly & Van Nam Vo

  2. IMAG, Université Montpellier, CNRS, Place Eugène Bataillon, 34095, Montpellier CEDEX 5, France

    Hedy Attouch

Authors
  1. Samir Adly
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hedy Attouch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Van Nam Vo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Samir Adly.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Hedy Attouch: Supported by COST Action: CA16228.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Adly, S., Attouch, H. & Vo, V.N. Newton-Type Inertial Algorithms for Solving Monotone Equations Governed by Sums of Potential and Nonpotential Operators. Appl Math Optim 85, 44 (2022). https://doi.org/10.1007/s00245-022-09846-3

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00245-022-09846-3

Keywords

Search

Navigation