Journal of Global Optimization

, Volume 24, Issue 2, pp 187–203 | Cite as

Augmented Lagrangian Duality and Nondifferentiable Optimization Methods in Nonconvex Programming

  • Rafail N. Gasimov


In this paper we present augmented Lagrangians for nonconvex minimization problems with equality constraints. We construct a dual problem with respect to the presented here Lagrangian, give the saddle point optimality conditions and obtain strong duality results. We use these results and modify the subgradient and cutting plane methods for solving the dual problem constructed. Algorithms proposed in this paper have some advantages. We do not use any convexity and differentiability conditions, and show that the dual problem is always concave regardless of properties the primal problem satisfies. The subgradient of the dual function along which its value increases is calculated without solving any additional problem. In contrast with the penalty or multiplier methods, for improving the value of the dual function, one need not to take the ‘penalty like parameter’ to infinity in the new methods. In both methods the value of the dual function strongly increases at each iteration. In the contrast, by using the primal-dual gap, the proposed algorithms possess a natural stopping criteria. The convergence theorem for the subgradient method is also presented.

Nonconvex programming Augmented Lagrangian Duality with zero gap Subgradient method Cutting plane method 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Andramanov, M.Yu., Rubinov, A.M. and Glover, B.M. (1997), Cutting Angle Method for Minimizing increasing Convex-Along-Rays Functions, Research Report 97/7, SITMS, University of Ballarat, Australia.Google Scholar
  2. Andramanov, M.Yu., Rubinov, A.M. and Glover, B.M. (1999), Cutting Angle Methods in Global Optimization, Applied Mathematics Letters 12, 95–100.Google Scholar
  3. Bazaraa, M.S., Sherali, H.D. and Shetty, C.M. (1993), Nonlinear Programming. Theory and Algorithms, John Wiley & Sons, Inc., New York.Google Scholar
  4. Bertsekas, D.P. (1995), Nonlinear Programming, Athena Scientific, Belmont, MA.Google Scholar
  5. Buys, J.D. (1972), Dual Algorithms for Constrained Optimization Problems, Doctoral Dissertation, University of Leiden, Leiden, the Netherlands.Google Scholar
  6. Cheney, E.W. and Goldstein, A.A. (1959), Newton's Method for Convex Programming and Tchebycheff Approximation, Numer. Math. 1, 253–268.Google Scholar
  7. Courant, R. (1943), Variational Methods for the Solution of Problems of Equilibrium and Vibrations, Bull. Amer. Math. Soc. 49, 1–23.Google Scholar
  8. Demyanov, V.F. (1968), Algorithm for some Minimax Problems, J. Computer and System Sciences 2, 342–380.Google Scholar
  9. Ekeland, I. and Temam, R. (1976) Convex Analysis and Variational Problems, Elsevier-North Holland, Amsterdam.Google Scholar
  10. Fletcher, R. (1970), A Class of Methods for Nonlinear Programming with Termination and Convergence Properties, in Abadie, J. (ed.), Integer and Nonlinear Programming, North Holland, Amsterdam, pp. 157–173.Google Scholar
  11. Floudas, C.A., et al. (1999), Handbook of Test Problems in Local and Global Optimization, Kluwer Academic Publishers, Dordrecht.Google Scholar
  12. Goffin, J.L. (1977), On Convergent Rates of Subgradient Optimization Methods, Math. Programming 13, 329–347.Google Scholar
  13. Haarhoff, P.C. and Buys, J.D. (1970), A New Method for the Optimization of a Nonlinear Function Subject to Nonlinear Constraints, Comput. J. 13, 178–184.Google Scholar
  14. Hestenes, M.R. (1969), Multiplier and Gradient Methods, J. Optim. Theory Appl. 4, 303–320.Google Scholar
  15. Himmelblau, D.M. (1972) Applied Nonlinear Optimization, McGraw-Hill Book Company, New York.Google Scholar
  16. Kelley, J.E. (1960), The Cutting-Plane Method for Solving Convex programs, J. Soc. Indust. Appl. Math. 8, 703–712.Google Scholar
  17. Khenkin, E.I. (1976), A Search Algorithm for General Problem of Mathematical Programming, USSR Journal of Computational Mathematics and Mathematical Physics 16, 61–71, (in Russian).Google Scholar
  18. Kort, B.W. and Bertsekas, D.P. (1972), A New Penalty Function Method for Constrained Minimization, Proc. IEEE Decision and Control Conference, New Orleans, LA, pp. 162–166.Google Scholar
  19. Krein, M.G. and Rutman, M.A. (1962), Linear Operators Leaving Invariant a Cone in a Banach Space, Trans. Amer. Math. Soc., Providence, RI, 10, 199–325Google Scholar
  20. Pallaschke, D. and Rolewicz, S. (1997), Foundations of Mathematical Optimization (Convex Analysis without Linearity), Kluwer Academic Publishers, Dordrecht.Google Scholar
  21. Pietrzykowski, T. (1969), An Exact Potential Method for Constrained Maxima, SIAM J. Numer. Anal. 6, 299–304.Google Scholar
  22. Polak, E. (1997), Optimization. Algorithms and Consistent Approximations, Springer, Berlin.Google Scholar
  23. Poljak, B.T. (1969a), Minimization of Unsmooth Functionals, Z. Vychislitelnoy Matematiki i Matematicheskoy Fiziki 9, 14–29.Google Scholar
  24. Poljak, B.T. (1969b), The Conjugate Gradient Method in Extremal Problems, Z. Vychislitelnoy Matematiki i Matematicheskoy Fiziki 9, 94–112.Google Scholar
  25. Poljak, B.T. (1970), Iterative Methods Using Lagrange Multipliers for Solving Extremal Problems with Constraints of the Equation Type, Z. Vyc¸islitelnoy Mat. i Mat. Fiziki 10, 1098–1106.Google Scholar
  26. Powell, M.J.D. (1969), A Method for Nonlinear Constraints in Minimization Problems, in Fletcher, R. (ed.), Optimization, Academic Press, New York, pp. 283–298.Google Scholar
  27. Rockafellar, R.T. (1970), Convex Analysis, Princeton University Press, Princeton, NJ.Google Scholar
  28. Rockafellar, R.T. (1993), Lagrange Multipliers and Optimality, SIAM Review 35, 183–238.Google Scholar
  29. Rockafellar, R.T. and Wets, R.J.-B. (1998), Variational Analysis, Springer, Berlin.Google Scholar
  30. Rubinov, A.M. (2000), Abstract Convexity and Global Optimization, Kluwer Academic Publishers, Dordrecht.Google Scholar
  31. Shor, N.Z. (1985), Minimization Methods for Nondifferentiable Functions, Springer, Berlin.Google Scholar
  32. Shor, N.Z. (1995), Dual Estimates in Multiextremal Problems, Journal of Global Optimzation 7, 75–91.Google Scholar
  33. Singer, I. (1997), Abstract Convex Analysis, John Wiley and Sons, Inc., New York.Google Scholar
  34. Wierzbicki, A.P. (1971), A Penalty Function Shifting Method in Constrained Static Optimization and its Convergence Properties, Arch. Automat. i Telemechaniki 16, 395–415.Google Scholar
  35. Zangwill, W.I. (1967), Nonlinear Programming via Penalty Functions, Management Sci. 13, 344–358.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Rafail N. Gasimov
    • 1
  1. 1.Department of Industrial EngineeringOsmangazi University, BademlikEskişehirTurkey

Personalised recommendations