Discrete Gradient Method: Derivative-Free Method for Nonsmooth Optimization

Article

Abstract

A new derivative-free method is developed for solving unconstrained nonsmooth optimization problems. This method is based on the notion of a discrete gradient. It is demonstrated that the discrete gradients can be used to approximate subgradients of a broad class of nonsmooth functions. It is also shown that the discrete gradients can be applied to find descent directions of nonsmooth functions. The preliminary results of numerical experiments with unconstrained nonsmooth optimization problems as well as the comparison of the proposed method with the nonsmooth optimization solver DNLP from CONOPT-GAMS and the derivative-free optimization solver CONDOR are presented.

Keywords

Nonsmooth optimization Derivative-free optimization Subdifferentials Discrete gradients 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Frangioni, A.: Generalized bundle methods. SIAM J. Optim. 113, 117–156 (2002) CrossRefMathSciNetGoogle Scholar
  2. 2.
    Gaudioso, M., Monaco, M.F.: A bundle type approach to the unconstrained minimization of convex nonsmooth functions. Math. Program. 23, 216–226 (1982) MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Hiriart-Urruty, J.B., Lemarechal, C.: Convex Analysis and Minimization Algorithms, vols. 1 and 2. Springer, Heidelberg (1993) Google Scholar
  4. 4.
    Kiwiel, K.C.: Methods of Descent for Nondifferentiable Optimization. Lecture Notes in Mathematics, vol. 1133. Springer, Berlin (1985) MATHGoogle Scholar
  5. 5.
    Lemarechal, C.: An extension of Davidon methods to nondifferentiable problems. In: Balinski, M.L., Wolfe, P. (eds.) Nondifferentiable Optimization. Mathematical Programming Study, vol. 3, pp. 95–109. North-Holland, Amsterdam (1975) Google Scholar
  6. 6.
    Mifflin, R.: An algorithm for constrained optimization with semismooth functions. Math. Oper. Res. 2, 191–207 (1977) MATHMathSciNetCrossRefGoogle Scholar
  7. 7.
    Zowe, J.: Nondifferentiable optimization: A motivation and a short introduction into the subgradient and the bundle concept. In: Schittkowski, K. (ed.) Computational Mathematical Programming. NATO SAI Series, vol. 15, pp. 323–356. Springer, New York (1985) Google Scholar
  8. 8.
    Wolfe, P.H.: A method of conjugate subgradients of minimizing nondifferentiable convex functions. Math. Program. Study 3, 145–173 (1975) MathSciNetGoogle Scholar
  9. 9.
    Polak, E., Royset, J.O.: Algorithms for finite and semi-infinite min-max-min problems using adaptive smoothing techniques. J. Optim. Theory Appl. 119, 421–457 (2003) MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Burke, J.V., Lewis, A.S., Overton, M.L.: A robust gradient sampling algorithm for nonsmooth, nonconvex optimization. SIAM J. Optim. 15, 751–779 (2005) MATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Audet, C., Dennis, J.E. Jr.: Analysis of generalized pattern searches. SIAM J. Optim. 13, 889–903 (2003) MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Torzcon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7, 1–25 (1997) CrossRefMathSciNetGoogle Scholar
  13. 13.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983) MATHGoogle Scholar
  14. 14.
    Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 15, 959–972 (1977) MATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Demyanov, V.F., Rubinov, A.M.: Constructive Nonsmooth Analysis. Peter Lang, Frankfurt am Main (1995) MATHGoogle Scholar
  16. 16.
    Bagirov, A.M., Rubinov, A.M., Soukhoroukova, A.V., Yearwood, J.: Supervised and unsupervised data classification via nonsmooth and global optimisation. TOP: Spanish Oper. Res. J. 11, 1–93 (2003) MATHMathSciNetGoogle Scholar
  17. 17.
    Bagirov, A.M., Yearwood, J.: A new nonsmooth optimisation algorithm for minimum sum-of-squares clustering problems. Eur. J. Oper. Res. 170, 578–596 (2006) MATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Bagirov, A.M.: Minimization methods for one class of nonsmooth functions and calculation of semi-equilibrium prices. In: Eberhard, A., et al. (eds.) Progress in Optimization: Contribution from Australasia, pp. 147–175. Kluwer Academic, Dordrecht (1999) Google Scholar
  19. 19.
    Bagirov, A.M.: Continuous subdifferential approximations and their applications. J. Math. Sci. 115, 2567–2609 (2003) MATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Wolfe, P.H.: Finding the nearest point in a polytope. Math. Program. 11, 128–149 (1976) MATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Frangioni, A.: Solving semidefinite quadratic problems within nonsmooth optimization algorithms. Comput. Oper. Res. 23, 1099–1118 (1996) MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Kiwiel, K.C.: A dual method for certain positive semidefinite quadratic programming problems. SIAM J. Sci. Stat. Comput. 10, 175–186 (1989) MATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Luks̃an, L., Vlc̃ek, J.: Test problems for nonsmooth unconstrained and linearly constrained optimization. Technical Report 78, Institute of Computer Science, Academy of Sciences of the Czech Republic (2000) Google Scholar
  24. 24.
    GAMS: The solver manuals. GAMS Development Corporation, Washington D.C. (2004) Google Scholar
  25. 25.
    Bergen, F.V.: CONDOR: a constrained, non-linear, derivative-free parallel optimizer for continuous, high computing load, noisy objective functions. Ph.D. thesis, Université Libre de Bruxelles, Belgium (2004) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Centre for Informatics and Applied Optimization, School of Information Technology and Mathematical SciencesUniversity of BallaratVictoriaAustralia
  2. 2.Department of Mathematics & Institute of Applied MathematicsMiddle East Technical UniversityAnkaraTurkey
  3. 3.Department of MathematicsMiddle East Technical UniversityAnkaraTurkey

Personalised recommendations