# Degenerate Problems with Nonisolated Solutions

## Abstract

The common feature of Newtonian methods discussed in the previous chapters is their local superlinear convergence under reasonable assumptions, which, however, always subsumed that the solution in question is isolated. On the other hand, sometimes one has to deal with problems (or even problem classes) whose solutions are not isolated, and thus are degenerate in some sense. One important example of degeneracy, to which we pay special attention, is the case of nonunique Lagrange multipliers associated with a solution of a given problem. We first put in evidence that in such cases, Newton-type methods for optimization are attracted to certain special Lagrange multipliers, called critical, which violate the second-order sufficient optimality conditions. This, in fact, is the reason for slow convergence typically observed in degenerate cases. To deal with degeneracy, we then consider special modifications of Newtonian methods; in particular, stabilized methods for variational problems and stabilized sequential quadratic programming for optimization. Finally, mathematical programs with complementarity constraints are discussed, which is an important class of inherently degenerate problems.

## References

- 1.W. Achtziger, C. Kanzow, Mathematical programs with vanishing constraints: optimality conditions and constraint qualifications. Math. Program.
**114**, 69–99 (2007)CrossRefMathSciNetGoogle Scholar - 2.W. Achtziger, T. Hoheisel, C. Kanzow, A smoothing-regularization approach to mathematical programs with vanishing constraints. Comput. Optim. Appl.
**55**, 733–767 (2013)CrossRefzbMATHMathSciNetGoogle Scholar - 6.M. Anitescu, Degenerate nonlinear programming with a quadratic growth condition. SIAM J. Optim.
**10**, 1116–1135 (2000)CrossRefzbMATHMathSciNetGoogle Scholar - 7.M. Anitescu,
*Nonlinear Programs with Unbounded Lagrange Multiplier Sets*(Mathematics and Computer Science Division, Argonne National Laboratories, Argonne, 2000). Preprint ANL/MCS-P796-0200Google Scholar - 8.M. Anitescu,
*On solving mathematical programs with complementarity constraints as nonlinear programs*(Mathematics and Computer Science Division, Argonne National Laboratories, Argonne, 2000). Preprint ANL/MCS-P864-1200Google Scholar - 9.M. Anitescu, A superlinearly convergent sequential quadratically constrained quadratic programming algorithm for degenerate nonlinear programming. SIAM J. Optim.
**12**, 949–978 (2002)CrossRefzbMATHMathSciNetGoogle Scholar - 10.M. Anitescu, On the rate of convergence of sequential quadratic programming with nondifferentiable exact penalty function in the presence of constrain degeneracy. Math. Program.
**92**, 359–386 (2002)CrossRefzbMATHMathSciNetGoogle Scholar - 11.M. Anitescu, On using the elastic mode in nonlinear programming approaches to mathematical programs with complementarity constraints. SIAM J. Optim.
**15**, 1203–1236 (2005)CrossRefzbMATHMathSciNetGoogle Scholar - 12.M. Anitescu, P. Tseng, S.J. Wright, Elastic-mode algorithms for mathematical programs with equilibrium constraints: global convergence and stationarity properties. Math. Program.
**110**, 337–371 (2007)CrossRefzbMATHMathSciNetGoogle Scholar - 13.A.V. Arutyunov,
*Optimality Conditions: Abnormal and Degenerate Problems*(Kluwer Academic, Dordrecht, 2000)CrossRefGoogle Scholar - 27.J.F. Bonnans, A. Shapiro,
*Perturbation Analysis of Optimization Problems*(Springer, New York, 2000)CrossRefzbMATHGoogle Scholar - 40.Y. Chen, M. Florian, The nonlinear bilevel programming problem: formulations, regularity and optimality conditions. Optimization
**32**, 193–209 (1995)CrossRefzbMATHMathSciNetGoogle Scholar - 56.S. Dempe,
*Foundations of Bilevel Programming*(Kluwer, Dordrecht, 2002)zbMATHGoogle Scholar - 68.F. Facchinei, J.-S. Pang,
*Finite-Dimensional Variational Inequalities and Complementarity Problems*(Springer, New York, 2003)Google Scholar - 73.D. Fernández, A quasi-Newton strategy for the sSQP method for variational inequality and optimization problems. Math. Program.
**137**, 199–223 (2013)CrossRefzbMATHMathSciNetGoogle Scholar - 75.D. Fernández, M. Solodov, Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems. Math. Program.
**125**, 47–73 (2010)CrossRefzbMATHMathSciNetGoogle Scholar - 76.D. Fernández, M. Solodov, Local convergence of exact and inexact augmented Lagrangian methods under the second-order sufficient optimality condition. SIAM J. Optim.
**22**, 384–407 (2012)CrossRefzbMATHMathSciNetGoogle Scholar - 78.D. Fernández, E.A. Pilotta, G.A. Torres, An inexact restoration strategy for the globalization of the sSQP method. Comput. Optim. Appl.
**54**, 595–617 (2013)CrossRefzbMATHMathSciNetGoogle Scholar - 79.M.C. Ferris, F. Tin-Loi, On the solution of a minimum weight elastoplastic problem involving displacement and complementarity constraints. Comput. Method. Appl. Mech. Eng.
**174**, 108–120 (1999)CrossRefGoogle Scholar - 84.A. Fischer, Modified Wilson’s method for nonlinear programs with nonunique multipliers. Math. Oper. Res.
**24**, 699–727 (1999)CrossRefzbMATHMathSciNetGoogle Scholar - 85.A. Fischer, Local behavior of an iterative framework for generalized equations with nonisolated solutions. Math. Program.
**94**, 91–124 (2002)CrossRefzbMATHMathSciNetGoogle Scholar - 92.R. Fletcher, S. Leyffer, Solving mathematical programs with equilibrium constraints as nonlinear programs. Optim. Method. Softw.
**19**, 15–40 (2004)CrossRefzbMATHMathSciNetGoogle Scholar - 95.R. Fletcher, S. Leyffer, D. Ralph, S. Scholtes, Local convergence of SQP methods for mathematical programs with equilibrium constraints. SIAM J. Optim.
**17**, 259–286 (2006)CrossRefzbMATHMathSciNetGoogle Scholar - 99.P.E. Gill, D.P. Robinson, A globally convergent stabilized SQP Method. SIAM J. Optim.
**23**, 1983–2010 (2013)CrossRefMathSciNetGoogle Scholar - 104.M. Golubitsky, D.G. Schaeffer,
*Singularities and Groups in Bifurcation Theory*, vol 1. (Springer, New York, 1985)CrossRefzbMATHGoogle Scholar - 112.W.W. Hager, Stabilized sequential quadratic programming. Comput. Optim. Appl.
**12**, 253–273 (1999)CrossRefzbMATHMathSciNetGoogle Scholar - 121.T. Hoheisel, C. Kanzow, First- and second-order optimality conditions for mathematical programs with vanishing constraints. Appl. Math.
**52**, 495–514 (2007)CrossRefzbMATHMathSciNetGoogle Scholar - 123.T. Hoheisel, C. Kanzow, On the Abadie and Guignard constraint qualifications for mathematical programs with vanishing constraints. Optimization
**58**, 431–448 (2009)CrossRefzbMATHMathSciNetGoogle Scholar - 126.A.F. Izmailov, On the Lagrange methods for finding degenerate solutions of constrained optimization problems. Comput. Math. Math. Phys.
**36**, 423–429 (1996)zbMATHMathSciNetGoogle Scholar - 127.A.F. Izmailov, Mathematical programs with complementarity constraints: regularity, optimality conditions, and sensitivity. Comput. Math. Math. Phys.
**44**, 1145–1164 (2004)MathSciNetGoogle Scholar - 128.A.F. Izmailov, On the analytical and numerical stability of critical Lagrange multipliers. Comput. Math. Math. Phys.
**45**, 930–946 (2005)MathSciNetGoogle Scholar - 132.A.F. Izmailov, M.M. Golishnikov, Newton-type methods for constrained optimization with nonregular constraints. Comput. Math. Math. Phys.
**46**, 1299–1319 (2006)CrossRefMathSciNetGoogle Scholar - 135.A.F. Izmailov, A.L. Pogosyan, Optimality conditions and Newton-type methods for mathematical programs with vanishing constraints. Comput. Math. Math. Phys.
**49**, 1128–1140 (2009)CrossRefMathSciNetGoogle Scholar - 137.A.F. Izmailov, A.L. Pogosyan, Active-set Newton methods for mathematical programs with vanishing constraints. Comput. Optim. Appl.
**53**, 425–452 (2012)CrossRefzbMATHMathSciNetGoogle Scholar - 138.A.F. Izmailov, M.V. Solodov, Superlinearly convergent algorithms for solving singular equations and smooth reformulations of complementarity problems. SIAM J. Optim.
**13**, 386–405 (2002)CrossRefzbMATHMathSciNetGoogle Scholar - 141.A.F. Izmailov, M.V. Solodov, Newton-type methods for optimization problems without constraint qualifications. SIAM J. Optim.
**15**, 210–228 (2004)CrossRefzbMATHMathSciNetGoogle Scholar - 143.A.F. Izmailov, M.V. Solodov, An active-set Newton method for mathematical programs with complementarity constraints. SIAM J. Optim.
**19**, 1003–1027 (2008)CrossRefzbMATHMathSciNetGoogle Scholar - 144.A.F. Izmailov, M.V. Solodov, Examples of dual behaviour of Newton-type methods on optimization problems with degenerate constraints. Comput. Optim. Appl.
**42**, 231–264 (2009)CrossRefzbMATHMathSciNetGoogle Scholar - 145.A.F. Izmailov, M.V. Solodov, Mathematical programs with vanishing constraints: optimality conditions, sensitivity, and a relaxation method. J. Optim. Theory Appl.
**142**, 501–532 (2009)CrossRefzbMATHMathSciNetGoogle Scholar - 146.A.F. Izmailov, M.V. Solodov, On attraction of Newton-type iterates to multipliers violating second-order sufficiency conditions. Math. Program.
**117**, 271–304 (2009)CrossRefzbMATHMathSciNetGoogle Scholar - 149.A.F. Izmailov, M.V. Solodov, On attraction of linearly constrained Lagrangian methods and of stabilized and quasi-Newton SQP methods to critical multipliers. Math. Program.
**126**, 231–257 (2011)CrossRefzbMATHMathSciNetGoogle Scholar - 150.A.F. Izmailov, M.V. Solodov, Stabilized SQP revisited. Math. Program.
**133**, 93–120 (2012)CrossRefzbMATHMathSciNetGoogle Scholar - 153.A.F. Izmailov, E.I. Uskov, Attraction of Newton method to critical Lagrange multipliers: fully quadratic case (2013). Available at http://www.optimization-online.org/DB_HTML/2013/02/3761.html
- 154.A.F. Izmailov, A.L. Pogosyan, M.V. Solodov, Semismooth SQP method for equality-constrained optimization problems with an application to the lifted reformulation of mathematical programs with complementarity constraints. Optim. Method. Softw.
**26**, 847–872 (2011)CrossRefzbMATHMathSciNetGoogle Scholar - 156.A.F. Izmailov, A.L. Pogosyan, M.V. Solodov, Semismooth Newton method for the lifted reformulation of mathematical programs with complementarity constraints. Comput. Optim. Appl.
**51**, 199–221 (2012)CrossRefzbMATHMathSciNetGoogle Scholar - 157.A.F. Izmailov, M.V. Solodov, E.I. Uskov, Augmented Lagrangian methods applied to optimization problems with degenerate constraints, including problems with complementarity constraints. SIAM J. Optim.
**22**, 1579–1606 (2012)CrossRefzbMATHMathSciNetGoogle Scholar - 167.T. Kato,
*Perturbation Theory for Linear Operators*(Springer, Berlin, 1984)zbMATHGoogle Scholar - 180.D.-H. Li, L. Qi, Stabilized SQP method via linear equations. Applied Mathematics Technical Reptort AMR00/5. University of New South Wales, Sydney, 2000Google Scholar
- 181.G.H. Lin, M. Fukushima, Hybrid approach with active set identification for mathematical programs with complementarity constraints. J. Optim. Theory Appl.
**128**, 1–28 (2006)CrossRefzbMATHMathSciNetGoogle Scholar - 184.Z.-Q. Luo, J.-S. Pang, D. Ralph,
*Mathematical Programs with Equilibrium Constraints*(Cambridge University Press, Cambridge, 1996)CrossRefGoogle Scholar - 185.MacMPEC. (2012) http://wiki.mcs.anl.gov/leyffer/index.php/MacMPEC
- 203.E.M.E. Mostafa, L.N. Vicente, S.J. Wright, Numerical behavior of a stabilized SQP method for degenerate NLP problems, in
*Global Optimization and Constraint Satisfaction*, ed. by C. Bliek, C. Jermann, A. Neumaier. Lecture Notes in Computer Science, vol 2861 (Springer, Berlin, 2003), pp. 123–141Google Scholar - 211.J.V. Outrata, M. Kocvara, J. Zowe,
*Nonsmooth Approach to Optimization Problems with Equilibrium Constraints: Theory, Applications and Numerical Results*(Kluwer Academic, Dordrecht, 1998)CrossRefzbMATHGoogle Scholar - 229.D. Ralph, Sequential quadratic programming for mathematical programs with linear complementarity constraints, in
*Computational Techniques and Applications: CTAC95*, ed. by R.L. May, A.K. Easton (World Scientific, Singapore, 1996), pp. 663–668Google Scholar - 240.H. Scheel, S. Scholtes, Mathematical programs with complementarity constraints: stationarity, optimality and sensitivity. Math. Oper. Res.
**25**, 1–22 (2000)CrossRefzbMATHMathSciNetGoogle Scholar - 241.S. Scholtes, M. Stöhr, How stringent is the linear independence assumption for mathematical programs with complementarity constraints? Math. Oper. Res.
**26**, 851–863 (2001)CrossRefzbMATHMathSciNetGoogle Scholar - 258.O. Stein, Lifting mathematical programs with complementarity constraints. Math. Program.
**131**, 71–94 (2012)CrossRefzbMATHMathSciNetGoogle Scholar - 269.S.J. Wright, Superlinear convergence of a stabilized SQP method to a degenerate solution. Comput. Optim. Appl.
**11**, 253–275 (1998)CrossRefzbMATHMathSciNetGoogle Scholar - 270.S.J. Wright, Modifying SQP for degenerate problems. SIAM J. Optim.
**13**, 470–497 (2002)CrossRefzbMATHMathSciNetGoogle Scholar - 271.S.J. Wright, Constraint identification and algorithm stabilization for degenerate nonlinear programs. Math. Program.
**95**, 137–160 (2003)CrossRefzbMATHMathSciNetGoogle Scholar - 272.S.J. Wright, An algorithm for degenerate nonlinear programming with rapid local convergence. SIAM J. Optim.
**15**, 673–696 (2005)CrossRefzbMATHMathSciNetGoogle Scholar