Skip to main content

Advertisement

SpringerLink
Log in

Convergence of Non-smooth Descent Methods Using the Kurdyka–Łojasiewicz Inequality

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

We investigate the convergence of subgradient-oriented descent methods in non-smooth non-convex optimization. We prove convergence in the sense of subsequences for functions with a strict standard model, and we show that convergence to a single critical point may be guaranteed if the Kurdyka–Łojasiewicz inequality is satisfied. We show, by way of an example, that the Kurdyka–Łojasiewicz inequality alone is not sufficient to prove the convergence to critical points.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Algorithm 3

Similar content being viewed by others

References

  1. Dennis, J.E. Jr., Schnabel, R.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice Hall Series in Computational Mathematics. Prentice Hall, New York (1983)

    MATH  Google Scholar 

  2. Absil, P.A., Mahony, R., Andrews, B.: Convergence of the iterates of descent methods for analytic cost functions. SIAM J. Optim. 16(2), 531–547 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  3. Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program., Ser. B 116(1–2),, 5–16 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  4. Bolte, J., Daniilidis, A., Ley, O., Mazet, L.: Characterizations of Łojesiewicz inequalities: subgradient flows, talweg, convexity. Trans. Am. Math. Soc. 362(6), 3319–3363 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  5. Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka–Łojasiewicz inequality. Math. Oper. Res. 35(2), 438–457 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  6. Noll, D., Rondepierre, A.: Convergence of linesearch and trust-region methods using the Kurdyka–Łojasiewicz inequality. In: Bailey, D.H., Bauschke, H.H., Borwein, P., Garvan, F., Théra, M., Vanderwerff, J., Wolkowicz, H. (eds.) Computational and Analytical Mathematics. Proceedings in Mathematics and Statistics, vol. 50 (2013). In Honor of Jonathan Borwein’s 60th Birthday

    Google Scholar 

  7. Noll, D., Prot, O., Rondepierre, A.: A proximity control algorithm to minimize non-smooth non-convex functions. Pac. J. Optim. 4(3), 569–602 (2008)

    MathSciNet  Google Scholar 

  8. Bolte, J., Daniilidis, A., Lewis, A.: The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17(4), 1205–1223 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  9. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program., Ser. A 137(1), 91–129 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  10. Bolte, J., Daniilidis, A., Lewis, A., Shiota, M.: Clarke subgradients of stratifiable functions. SIAM J. Optim. 18(2), 556–572 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  11. Spingarn, J.E.: Submonotone subdifferentials of Lipschitz functions. Trans. Am. Math. Soc. 264, 77–89 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  12. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (2004)

    Google Scholar 

  13. Noll, D.: Cutting plane oracles to minimize non-smooth non-convex functions. Set-Valued Var. Anal. 18(3–4), 531–568 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  14. Daniilidis, A., Georgiev, P.: Approximate convexity and submonotonicity. J. Math. Anal. Appl. 291, 117–144 (2004)

    Article  MathSciNet  Google Scholar 

  15. Apkarian, P., Noll, D., Prot, O.: A trust region spectral bundle method for nonconvex eigenvalue optimization. SIAM J. Optim. 10(1), 281–306 (2008)

    Article  MathSciNet  Google Scholar 

  16. Apkarian, P., Noll, D., Prot, O.: A proximity control algorithm to minimize non-smooth and non-convex semi-infinite maximum eigenvalue functions. J. Convex Anal. 16, 641–666 (2009)

    MATH  MathSciNet  Google Scholar 

  17. Gabarrou, M., Noll, D., Alazard, D.: Design of a flight control architecture using a non-convex bundle method. Math. Control Signals Syst. 25(2), 257–290 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  18. Noll, D.: Bundle methods for non-convex minimization with inexact subgradient and function values. In: Bailey, D.H., Bauschke, H.H., Borwein, P., Garvan, F., Théra, M., Vanderwerff, J., Wolkowicz, H. (eds.) Computational and Analytical Mathematics. Springer Proceedings in Mathematics and Statistics, vol. 50 (2013). In Honor of Jonathan Borwein’s 60th Birthday

    Google Scholar 

  19. Alber, Y.I., Iusem, A.N., Solodov, M.V.: On the projected subgradient method for nonsmooth convex optimization in a Hilbert space. Math. Program. 81, 23–35 (1998)

    MATH  MathSciNet  Google Scholar 

  20. Ostrowski, A.M.: Solution of Equations in Euclidean and Banach Spaces. Academic Press, New York (1973)

    MATH  Google Scholar 

  21. Noll, D.: A bundle method for non-smooth and non-convex optimization. Talk at the 2009 ISMP, Chicago

  22. Hiriart-Urruty, J.-B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms, vol. I and II: Advanced Theory and Bundle Methods. Grundlehren der Mathematischen Wissenschaften, vol. 306. Springer, New York (1993)

    Google Scholar 

  23. Nesterov, Y.:. Private communication (2013)

  24. Gürbüzbalaban, M., Overton, M.L.: On Nesterov’s nonsmooth Chebyshev-Rosenbrock functions. Nonlinear Anal. 75, 1282–1289 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  25. Apkarian, P., Noll, D., Rondepierre, A.: Mixed H 2/H ∞ control via nonsmooth optimization. SIAM J. Control Optim. 47(3), 1516–1546 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  26. Polak, E.: Optimization: Algorithms, and Consistent Approximation. Springer, Berlin (1997)

    Book  MATH  Google Scholar 

  27. Sagastizábal, C., Solodov, M.: An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or filter. SIAM J. Optim. 16(1), 146–169 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  28. Apkarian, P., Noll, D.: Nonsmooth optimization for multiband frequency domain control design. Automatica 43, 724–731 (2007)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgements

The author was supported by the EADS Foundation grant Technicom.

Author information

Authors and Affiliations

  1. Institut de Mathématiques, Université Paul Sabatier, 31062, Toulouse, France

    Dominikus Noll

Authors
  1. Dominikus Noll
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dominikus Noll.

Additional information

Communicated by Hedy Attouch.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Noll, D. Convergence of Non-smooth Descent Methods Using the Kurdyka–Łojasiewicz Inequality. J Optim Theory Appl 160, 553–572 (2014). https://doi.org/10.1007/s10957-013-0391-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-013-0391-8

Keywords

Search

Navigation