Advertisement

Computational Optimization and Applications

, Volume 71, Issue 3, pp 879–913 | Cite as

A convergence analysis of the method of codifferential descent

  • M. V. DolgopolikEmail author
Article
  • 98 Downloads

Abstract

This paper is devoted to a detailed convergence analysis of the method of codifferential descent (MCD) developed by professor V.F. Demyanov for solving a large class of nonsmooth nonconvex optimization problems. We propose a generalization of the MCD that is more suitable for applications than the original method, and that utilizes only a part of a codifferential on every iteration, which allows one to reduce the overall complexity of the method. With the use of some general results on uniformly codifferentiable functions obtained in this paper, we prove the global convergence of the generalized MCD in the infinite dimensional case. Also, we propose and analyse a quadratic regularization of the MCD, which is the first general method for minimizing a codifferentiable function over a convex set. Apart from convergence analysis, we also discuss the robustness of the MCD with respect to computational errors, possible step size rules, and a choice of parameters of the algorithm. In the end of the paper we estimate the rate of convergence of the MCD for a class of nonsmooth nonconvex functions that arise, in particular, in cluster analysis. We prove that under some general assumptions the method converges with linear rate, and it convergence quadratically, provided a certain first order sufficient optimality condition holds true.

Keywords

Nonsmooth optimization Nonconvex optimization Codifferential Quasidifferential Method of codifferential descent 

Mathematics Subject Classification

90C56 49J52 

Notes

Acknowledgements

The author is sincerely grateful to his colleagues G.Sh. Tamasyan, A. Fominyh and T. Angelov for numerous useful discussions on the method of codifferential descent and its applications that played an important role in the preparation of this paper. Also, the author wishes to express his thanks to the anonymous referees for many thoughtful comments that helped to significantly improve the quality of the article, and to professor V.N. Malozemov for pointing out the fact that the PPP algorithm converges quadratically to a Chebyshev (Haar) point.

References

  1. 1.
    Demyanov, V.F.: Continuous generalized gradients for nonsmooth functions. In: Kurzhanski, A., Neumann, K., Pallaschke, D. (eds.) Optimization, Parallel Processing and Applications, pp. 24–27. Springer, Berlin (1988)CrossRefGoogle Scholar
  2. 2.
    Demyanov, V.F.: On codifferentiable functions. Vestn. Leningr. Univ. Math. 2, 22–26 (1988)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Demyanov, V.F.: Smoothness of nonsmooth functions. In: Clarke, F.H., Demyanov, V.F., Giannesssi, F. (eds.) Nonsmooth Optimization and Related Topics, pp. 79–88. Springer, Boston (1989)CrossRefGoogle Scholar
  4. 4.
    Demyanov, V.F., Rubinov, A.M.: Constructive Nonsmooth Analysis. Peter Lang, Frankfurt am Main (1995)zbMATHGoogle Scholar
  5. 5.
    Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I. Basic Theory. Springer, Berlin (2006)Google Scholar
  6. 6.
    Penot, J.-P.: Calculus Without Derivatives. Springer, New York (2013)CrossRefGoogle Scholar
  7. 7.
    Zaffaroni, A.: Continuous approximations, codifferentiable functions and minimization methods. In: Demyanov, V.F., Rubinov, A.M. (eds.) Quasidifferentiability and Related Topics, pp. 361–391. Kluwer Academic Publishers, Dordrecht (2000)CrossRefGoogle Scholar
  8. 8.
    Dolgopolik, M.V.: Codifferential calculus in normed spaces. J. Math. Sci. 173, 441–462 (2011)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Dolgopolik, M.V.: Abstract convex approximations of nonsmooth functions. Optimization 64, 1439–1469 (2015)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Demyanov, V.F., Stavroulakis, G.E., Polyakova, L.N., Panagiotopoulos, P.D.: Quasidifferentiability and Nonsmooth Modelling in Mechnics, Engineering and Economics. Kluwer Academic Publishers, Dordrecht (1996)CrossRefGoogle Scholar
  11. 11.
    Demyanov, V.F., Rubinov, A.M. (eds.): Quasidifferentiability and Related Topics. Kluwer Academic Publishers, Dordrecht (2000)zbMATHGoogle Scholar
  12. 12.
    Pallaschke, D., Recht, P., Urbański, R.: On locally-Lipschitz quasi-differentiable functions in Banach spaces. Optimization 17, 287–295 (1986)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Uderzo, A.: Fréchet quasidifferential calculus with applications to metric regularity of continuous maps. Optimization 54, 469–493 (2005)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Bagriov, A.M.: Numerical methods for minimizing quasidifferentiable functions: a survey and comparison. In: Demyanov, V.F., Rubinov, A.M. (eds.) Quasidifferentiability and Related Topics, pp. 33–71. Kluwer Academic Publishers, Dordrecht (2000)CrossRefGoogle Scholar
  15. 15.
    Bagirov, A.: A method for minimization of quasidifferentiable functions. Optim. Methods Softw. 17, 31–60 (2002)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Luderer, B., Weigelt, J.: A solution method for a special class of nondifferentiable unconstrained optimization problems. Comput. Optim. Appl. 24, 83–93 (2003)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Mäkelä, M.: Survey of bundle method for nonsmooth optimization. Optim. Methods Softw. 17, 1–29 (2002)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Haarala, N., Miettinen, K., Mäkelä, M.: Globally convergent limited memory bundle method for large-scale nonsmooth optimization. Math. Program. 109, 181–205 (2007)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Hare, W., Sagastizábal, C.: A redistributed proximal bundle method for nonconvex optimization. SIAM J. Optim. 20, 2442–2473 (2010)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Fuduli, A., Gaudioso, M., Nurminski, E.A.: A splitting bundle approach for non-smooth non-convex minimization. Optimization 64, 1131–1151 (2015)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Burke, J.V., Lewis, A.S., Overton, M.L.: A robust gradient sampling algorithm for nonsmooth, nonconvex optimization. SIAM J. Optim. 15, 751–779 (2005)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Kiwiel, K.C.: A nonderivative version of the gradient sampling algorithm for nonsmooth nonconvex optimization. SIAM J. Optim. 20, 1983–1994 (2010)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Hare, W., Nutini, J.: A derivative-free approximate gradient sampling algorithm for finite minimax problems. Comput. Optim. Appl. 56, 1–38 (2013)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Curtis, F.E., Que, X.: An adaptive gradient sampling algorithm for non-smooth optimization. Optim. Methods Softw. 28, 1302–1324 (2013)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Lewis, A.S., Overton, M.L.: Nonsmooth optimization via quasi-Newton methods. Math. Program. 141, 135–163 (2013)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Keskar, N., Wächter, A.: A limited-memory quasi-Newton algorithm for bound-constrained non-smooth optimization. Optim. Methods Softw. (2017).  https://doi.org/10.1080/10556788.2017.1378652 CrossRefGoogle Scholar
  27. 27.
    Bagirov, A.M., Karasözen, B., Sezer, M.: Discrete gradient method: derivative-free method for nonsmooth optimization. J. Optim. Theory Appl. 137, 317–334 (2008)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Bagirov, A.M., Rubinov, A.M., Zhang, J.: A multidimensional descent method for global optimization. Optimization 58, 611–625 (2009)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. SIAM, Philadelphia (2009)CrossRefGoogle Scholar
  30. 30.
    Karmitsa, N., Bagirov, A., Mäkelä, M.M.: Comparing different nonsmooth minimization methods and software. Optim. Methods Softw. 27, 131–153 (2012)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Bagirov, A., Napsu, K., Mäkelä, M.M.: Introduction to Nonsmooth Optimization. Springer, Cham (2014)CrossRefGoogle Scholar
  32. 32.
    Demyanov, V.F., Bagirov, A.M., Rubinov, A.M.: A method of truncated codifferential with applications to some problems of cluster analysis. J. Glob. Optim. 23, 63–80 (2002)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Bagirov, A.M., Ganjehlou, A.N., Ugon, J., Tor, A.H.: Truncated codifferential method for nonsmooth convex optimization. Pac. J. Optim. 6, 483–496 (2010)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Tor, A.H., Karasözen, B., Bagirov, A.: Truncated codifferential method for linearly constrained nonsmooth optimization. In: The ISI proceedings of 24th mini EURO conference “Continuous Optimization and Information-Based Technologies in the Financial Sector”, Izmir, Turkey, pp 87–93 (2010)Google Scholar
  35. 35.
    Bagirov, A.M., Ugon, J.: Codifferential method for minimizing nonsmooth DC functions. J. Glob. Optim. 50, 3–22 (2011)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Tor, A.H., Bagirov, A., Karasözen, B.: Aggregate codifferential method for nonsmooth DC optimization. J. Comput. Appl. Math. 259, 851–867 (2014)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Andramonov, M.Yu.: The method of confidence neighbourhoods for minimization of codifferentiable functions. Russ. Math. 48, 1–7 (2004)Google Scholar
  38. 38.
    Demyanov, V.F., Tamasyan, GSh: Exact penalty functions in isoperimetric problems. Optimization 60, 153–177 (2011)MathSciNetCrossRefGoogle Scholar
  39. 39.
    Demyanov, V.F., Tamasyan, GSh: Direct methods in the parametric moving boundary variational problem. Numer. Funct. Anal. Optim. 35, 932–961 (2014)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Fominyh, A.V., Karelin, V.V., Polyakova, L.N.: Application of the hypodifferential descent method to the problem of constructing an optimal control. Optim. Lett. (2017).  https://doi.org/10.1007/s11590-017-1222-x CrossRefzbMATHGoogle Scholar
  41. 41.
    Pschenichny, B.N., Danilin, Yu.M.: Numerical Methods in Extremal Problems, English edn. Mir Publishers, Moscow (1978)Google Scholar
  42. 42.
    Pironneau, O., Polak, E.: On the rate of convergence of certain methods of centers. Math. Program. 2, 230–258 (1972)MathSciNetCrossRefGoogle Scholar
  43. 43.
    Daugavet, V.A., Malozemov, V.N.: Quadratic rate of convergence of a linearization method for solving discrete minimax problems. USSR Comput. Math. Math. Phys. 21, 19–28 (1981)CrossRefGoogle Scholar
  44. 44.
    Polak, E.: Optimization, Algorithms and Consistent Approximations. Springer, New York (1997)zbMATHGoogle Scholar
  45. 45.
    Ioffe, A.D., Tihomirov, V.M.: Theory of Extremal Problems. North-Holland Publishing Company, Amsterdam (1979)zbMATHGoogle Scholar
  46. 46.
    Luderer, B.: Does the special choice of quasidifferentials influence necessary minimum conditions? In: Oettli, W., Pallaschke, D. (eds.) Advances in Optimization, pp. 256–266. Springer, Berlin (1992)CrossRefGoogle Scholar
  47. 47.
    Luderer, B., Rösiger, R., Würker, U.: On necessary minimum conditions in quasidifferential calculus: independence of the specific choice of quasidifferentials. Optimization 22, 643–660 (1991)MathSciNetCrossRefGoogle Scholar
  48. 48.
    Kuntz, L.: A charaterization of continuously codifferentiable function and some consequences. Optimization 22, 539–547 (1991)MathSciNetCrossRefGoogle Scholar
  49. 49.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley-Interscience, New York (1983)zbMATHGoogle Scholar
  50. 50.
    Pallaschke, D., Urbański, R.: Pairs of Compact Convex Sets: Fractional Arithmetic with Convex Sets. Kluwer Academic Publishers, Dordrecht (2002)CrossRefGoogle Scholar
  51. 51.
    Di Pillo, G., Facchinei, F.: Exact Barrier Function Methods for Lipschitz Programs. Appl. Math. Optim. 32, 1–31 (1995)MathSciNetCrossRefGoogle Scholar
  52. 52.
    Demyanov, V.F., Malozemov, V.N.: Introduction to Minimax. Wiley, New York (1990)Google Scholar
  53. 53.
    Hiriart-Urruty, J.-B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms I. Springer, Berlin (1993)CrossRefGoogle Scholar
  54. 54.
    Edwards, R.E.: Functional Analysis: Theory and Applications. Dover Publications, New York (1995)Google Scholar
  55. 55.
    Penot, J.-P.: On the convergence of descent algorithms. Comput. Optim. Appl. 23, 279–284 (2002)MathSciNetCrossRefGoogle Scholar
  56. 56.
    Dolgopolik, M.V.: Nonsmooth problems of calculus of variations via codifferentiation. ESAIM: Control Optim. Calc. Var. 20, 1153–1180 (2014)MathSciNetCrossRefGoogle Scholar
  57. 57.
    Dolgopolik, M.V.: Inhomogeneous convex approximations of nonsmooth functions. Russ. Math. 56, 28–42 (2012)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Saint Petersburg State UniversitySaint PetersburgRussia
  2. 2.Institute of Problems of Mechanical EngineeringRussian Academy of SciencesSaint PetersburgRussia

Personalised recommendations