A Deflected Subgradient Method Using a General Augmented Lagrangian Duality with Implications on Penalty Methods
We propose a duality scheme for solving constrained nonsmooth and nonconvex optimization problems. Our approach is to use a new variant of the deflected subgradient method for solving the dual problem. Our augmented Lagrangian function induces a primal–dual method with strong duality, that is, with zero duality gap. We prove that our method converges to a dual solution if and only if a dual solution exists. We also prove that all accumulation points of an auxiliary primal sequence are primal solutions. Our results apply, in particular, to classical penalty methods, since the penalty functions associated with these methods can be recovered as a special case of our augmented Lagrangians. Besides the classical augmenting terms given by the ℓ 1- or ℓ 2-norm forms, terms of many other forms can be used in our Lagrangian function. Using a practical selection of the step-size parameters, as well as various choices of the augmenting term, we demonstrate the method on test problems. Our numerical experiments indicate that it is more favourable to use an augmenting term of an exponential form rather than the classical ℓ 1- or ℓ 2-norm forms.
KeywordsPenalty Function Accumulation Point Dual Solution Strong Duality Subgradient Method
Unable to display preview. Download preview PDF.
- 1.d’Antonio, G., Frangioni, A.: Convergence analysis of deflected conditional approximate subgradient methods, SIAM Journal on Optimization 20, 357–386 (2009)Google Scholar
- 6.Burachik, R.S.: On primal convergence for augmented Lagrangian duality, Optimization (accepted for publication)Google Scholar
- 14.Gasimov R.N., Ismayilova, N.A.: The modified subgradient method for equality constrained nonconvex optimization problems. In: Qi, L., Teo, K.L., Yang, X.Q. (eds.) Optimization and Control with Applications, pp.257–269, Springer, New York (2005)Google Scholar
- 20.Polyak, B.T.: The conjugate gradient method in extremal problems, Zhurnal Vychislitelnoy Matematiki i Matematicheskoy Fiziki 9, 94–112 (1969)Google Scholar
- 21.Polyak, B.T.: Minimization of unsmooth functionals, Zhurnal Vychislitelnoy Matematiki i Matematicheskoy Fiziki 9, 14–29 (1969)Google Scholar
- 23.Polyak, B.T.: Introduction to Optimization, Optimization Software Inc., Publications Division, New York (1987)Google Scholar
- 24.Polyak, R.A.: Nonlinear rescaling vs. smoothing technique in convex optimization, Mathematical Programming A 92, 197–235 (2002)Google Scholar
- 34.Shor, N.Z.: Minimization Methods for Nondifferentiable Functions, Springer, Berlin (1985)Google Scholar