Nondifferentiable Optimization

Part of the NATO ASI Series book series (volume 15)


Nondifferentiable optimization NDO (also called nonsmooth optimization NSO) concerns problems in which the functions involved have discontinuous first derivatives. This causes classical methods to fail; hence nonsmooth problems require a new, a nonstandard approach. The paper tries to develop the basic features of the two main direct approaches in NDO, namely the Subgradient concept and the Bundle concept. Rather than collecting as many results in this area as possible, we will try to motivate and to help understanding the main underlying ideas.


Line Search Steep Descent Indirect Approach Exact Penalty Minimax Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. Ben-Tal and J. Zowe, “Necessary and sufficient optimality conditions for a class of nonsmooth minimization problems” Mathematical Programming 24 (1982) 70–91.CrossRefzbMATHMathSciNetGoogle Scholar
  2. [2]
    C. Charalambous and A. R. Conn, “An efficient method to solve the minimax problem directly”, SIAM Journal on Numerical Analysis 15 (1982) 162–187.CrossRefMathSciNetGoogle Scholar
  3. [3]
    A. R. Conn, “An efficient second order method to solve the (constrained) minimax problem”, University of Waterloo, Report CORR-7 9-5.Google Scholar
  4. [4]
    El-Attar, R. Vidyasagar and S. R. Dutta, “An algorithm for ℓ1 norm minimization with application to nonlinear ℓ1-approximation”, SIAM Journal on Numerical Analysis 16 (1979) 70–86.CrossRefzbMATHMathSciNetGoogle Scholar
  5. [5]
    Yu. M. Ermoliev, “Methods of solution of nonlinear extremal problems”, Cybernetics 24, 1–16.Google Scholar
  6. [6]
    R. Fletcher, Practical methods of optimization II (John Wiley & Sons, New York, 1981).Google Scholar
  7. [7]
    J. L. Goffin, “Nondifferentiable optimization and the relaxation method” in: C. Lemarèchal and R. Mifflin (eds.) Nonsmooth Optimization (Pergamon Press, New York, 1978).Google Scholar
  8. [8]
    J. L. Goffin, “On convergence rate of subgradient optimization methods”, Mathematical Programming 13 (1977) 329–347.CrossRefzbMATHMathSciNetGoogle Scholar
  9. [9]
    J. L. Goffin, “Convergence results in a class of variable metric subgradient methods” in: O. L. Mangasarian, R. R. Meyer and S. M. Robinson (eds.) Nonlinear Programming 4 (Academic Press, New York, 1981).Google Scholar
  10. [10]
    J. L. Goffin, “Convergence rates of the ellipsoid method on general convex functions”, Mathematics of Operations Research 8 (1983) 135–149.CrossRefzbMATHMathSciNetGoogle Scholar
  11. [11]
    M. Held, P. Wolfe and H. Crowder, “Validation of subgradient optimization”, Mathematical Programming 6 (1974) 62–88.CrossRefzbMATHMathSciNetGoogle Scholar
  12. [12]
    D. M. Himmelblau, Applied Nonlinear Programming (McGraw-Hill, 1972).Google Scholar
  13. [13]
    J. B. Hiriart-Urruty, “Lipschitz r-continuity of the approximate subdifferential of a convex function”, Mathematica Scandinavica 47 (1980) 123–134.zbMATHMathSciNetGoogle Scholar
  14. [14]
    R. Hornung, “An efficient algorithm for solving the discrete mini-sum problems,” Operations Research Letters 2 (1983) 115–118.CrossRefzbMATHMathSciNetGoogle Scholar
  15. [15]
    C. Lemarèchal, “Nondifferentiable optimization”, in: Dixon, Spedicato and Szegö H (eds.) Nonlinear Optimization, Theory and Algorithms (Birkhäuser, Boston, 1980).Google Scholar
  16. [16]
    C. Lemarèchal, “A view of line searches” in: W. Oettli and J. Stoer (eds.) Optimization and Optimal Control, Lecture Notes in Control and Information, (Springer, 1981).Google Scholar
  17. [17]
    C. Lemarèchal, “Numerical experiments in nonsmooth optimization” in: E. A. Nurminskii (eds.) Progress in Nondifferentiable Optimization (IIASA Proceedings, Laxenburg, 1982).Google Scholar
  18. [18]
    C. Lemarèchal and J. J. Strodiot, “Bundle methods, cutting plane algorithms and o-Newton directions”, Preprint, University of Namur, 1984.Google Scholar
  19. [19]
    C. Lemaréchal, J. J. Strodiot and A. Bihain, “On a Bundle algorithm for nonsmooth optimization” in: O. L. Mangasarian, R. R. Meyer and S. M. Robinson (eds.), Nonlinear Programming 4 (Academic Press, New York, 1981).Google Scholar
  20. [20]
    C. Lemarèchal and J. Zowe, “Some remarks on the construction of higher order algorithms in convex optimization”, Journal of Applied Mathematics and Optimization 10 (1983) 51-61.Google Scholar
  21. [21]
    A. Y. Levin, “On an algorithm for the minimization of convex functions”, Soviet Mathematical Doklady6, 286–290.Google Scholar
  22. [22]
    R. Mifflin, “An algorithm for constrained optimization with semi-smooth functions”, Mathematics of Operations Research 2 (1977) 191–207.CrossRefzbMATHMathSciNetGoogle Scholar
  23. [23]
    E. A. Nurminskii, “On ɛ-differential mappings and their applications in nondifferentiable optimization”, Working paper 78–58, IIASA, 1978.Google Scholar
  24. [24]
    E. A. Nurminskii (ed.), Progress in nondifferentiable optimization (IIASA Proceedings, Laxenburg, 1982).Google Scholar
  25. [25]
    T. Pietrzykowski, “An exact penalty method for constrained maxima”, SIAM Journal on Numerical Analysis 6 (1969) 299–304.CrossRefzbMATHMathSciNetGoogle Scholar
  26. [26]
    B. T. Poljak, “A general method of solving extremal problems”, Soviet Mathematical Doklady 8, 593–597.Google Scholar
  27. [27]
    B. T. Poljak, “Minimization of unsmooth functionals”, USSR Computational Mathematics and Mathematical Physics 9, 14–29.Google Scholar
  28. [28]
    B. N. Pshenichnyi, Necessary conditions for an extremum (Marcel Dekker, New York, 1971).Google Scholar
  29. [29]
    R. T. Rockafellar, Convex Analysis (Princeton University Press, 1972).Google Scholar
  30. [30]
    V. A. Shokov, “Note on minimization methods using space dilation”, Cybernetics 10, 4, 689–692.Google Scholar
  31. [31]
    N. Z. Shor, “The rate of convergence of the generalized gradient descent method”, Cybernetics 4, 3, 79–80.Google Scholar
  32. [32]
    N. Z. Shor, “Utilization of the operation of space dilation in the minimization of convex functions”, Cybernetics 6, 2, 102–108.Google Scholar
  33. [33]
    N. Z. Shor, “Convergence of a gradient method with space dilation in the directions of the difference between two successive gradients”, Cybernetics 11, 4, 564–570.Google Scholar
  34. [34]
    N. Z. Shor, “Cut-off method with space extension in convex programming problems”, Cybernetics 13, 94–96.Google Scholar
  35. [35]
    N. Z. Shor, “Generalized gradient methods of nondifferentiable optimization employing space dilation operations” in: A. Bachern, M. Grötschel and B. Körte (eds.) Mathematical programming, the state of the art (Springer, Berlin, 1983).Google Scholar
  36. [36]
    P. Wolfe, “A method of conjugate subgradients for minimizing nondifferentiable functions”, Mathematical Programming Study 3 (1975) 145–173.MathSciNetGoogle Scholar
  37. [37]
    R. S. Womersley, “Numerical methods for structured problems in nonsmooth optimization”, University of Dundee, 1981.Google Scholar
  38. [38]
    D. B. Yudin and A. S. Nemorovskii, “Estimation of the informational complexity of mathematical programming problems,” Matekon 13, 2–25, 25.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1985

Authors and Affiliations

  • J. Zowe
    • 1
  1. 1.University of BayreuthBayreuthW.-Germany

Personalised recommendations