Advertisement

Degenerate Problems with Nonisolated Solutions

  • Alexey F. Izmailov
  • Mikhail V. Solodov
Chapter
Part of the Springer Series in Operations Research and Financial Engineering book series (ORFE)

Abstract

The common feature of Newtonian methods discussed in the previous chapters is their local superlinear convergence under reasonable assumptions, which, however, always subsumed that the solution in question is isolated. On the other hand, sometimes one has to deal with problems (or even problem classes) whose solutions are not isolated, and thus are degenerate in some sense. One important example of degeneracy, to which we pay special attention, is the case of nonunique Lagrange multipliers associated with a solution of a given problem. We first put in evidence that in such cases, Newton-type methods for optimization are attracted to certain special Lagrange multipliers, called critical, which violate the second-order sufficient optimality conditions. This, in fact, is the reason for slow convergence typically observed in degenerate cases. To deal with degeneracy, we then consider special modifications of Newtonian methods; in particular, stabilized methods for variational problems and stabilized sequential quadratic programming for optimization. Finally, mathematical programs with complementarity constraints are discussed, which is an important class of inherently degenerate problems.

References

  1. 1.
    W. Achtziger, C. Kanzow, Mathematical programs with vanishing constraints: optimality conditions and constraint qualifications. Math. Program. 114, 69–99 (2007)CrossRefMathSciNetGoogle Scholar
  2. 2.
    W. Achtziger, T. Hoheisel, C. Kanzow, A smoothing-regularization approach to mathematical programs with vanishing constraints. Comput. Optim. Appl. 55, 733–767 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 6.
    M. Anitescu, Degenerate nonlinear programming with a quadratic growth condition. SIAM J. Optim. 10, 1116–1135 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 7.
    M. Anitescu, Nonlinear Programs with Unbounded Lagrange Multiplier Sets (Mathematics and Computer Science Division, Argonne National Laboratories, Argonne, 2000). Preprint ANL/MCS-P796-0200Google Scholar
  5. 8.
    M. Anitescu, On solving mathematical programs with complementarity constraints as nonlinear programs (Mathematics and Computer Science Division, Argonne National Laboratories, Argonne, 2000). Preprint ANL/MCS-P864-1200Google Scholar
  6. 9.
    M. Anitescu, A superlinearly convergent sequential quadratically constrained quadratic programming algorithm for degenerate nonlinear programming. SIAM J. Optim. 12, 949–978 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 10.
    M. Anitescu, On the rate of convergence of sequential quadratic programming with nondifferentiable exact penalty function in the presence of constrain degeneracy. Math. Program. 92, 359–386 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 11.
    M. Anitescu, On using the elastic mode in nonlinear programming approaches to mathematical programs with complementarity constraints. SIAM J. Optim. 15, 1203–1236 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 12.
    M. Anitescu, P. Tseng, S.J. Wright, Elastic-mode algorithms for mathematical programs with equilibrium constraints: global convergence and stationarity properties. Math. Program. 110, 337–371 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  10. 13.
    A.V. Arutyunov, Optimality Conditions: Abnormal and Degenerate Problems (Kluwer Academic, Dordrecht, 2000)CrossRefGoogle Scholar
  11. 27.
    J.F. Bonnans, A. Shapiro, Perturbation Analysis of Optimization Problems (Springer, New York, 2000)CrossRefzbMATHGoogle Scholar
  12. 40.
    Y. Chen, M. Florian, The nonlinear bilevel programming problem: formulations, regularity and optimality conditions. Optimization 32, 193–209 (1995)CrossRefzbMATHMathSciNetGoogle Scholar
  13. 56.
    S. Dempe, Foundations of Bilevel Programming (Kluwer, Dordrecht, 2002)zbMATHGoogle Scholar
  14. 68.
    F. Facchinei, J.-S. Pang, Finite-Dimensional Variational Inequalities and Complementarity Problems (Springer, New York, 2003)Google Scholar
  15. 73.
    D. Fernández, A quasi-Newton strategy for the sSQP method for variational inequality and optimization problems. Math. Program. 137, 199–223 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  16. 75.
    D. Fernández, M. Solodov, Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems. Math. Program. 125, 47–73 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 76.
    D. Fernández, M. Solodov, Local convergence of exact and inexact augmented Lagrangian methods under the second-order sufficient optimality condition. SIAM J. Optim. 22, 384–407 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 78.
    D. Fernández, E.A. Pilotta, G.A. Torres, An inexact restoration strategy for the globalization of the sSQP method. Comput. Optim. Appl. 54, 595–617 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  19. 79.
    M.C. Ferris, F. Tin-Loi, On the solution of a minimum weight elastoplastic problem involving displacement and complementarity constraints. Comput. Method. Appl. Mech. Eng. 174, 108–120 (1999)CrossRefGoogle Scholar
  20. 84.
    A. Fischer, Modified Wilson’s method for nonlinear programs with nonunique multipliers. Math. Oper. Res. 24, 699–727 (1999)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 85.
    A. Fischer, Local behavior of an iterative framework for generalized equations with nonisolated solutions. Math. Program. 94, 91–124 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  22. 92.
    R. Fletcher, S. Leyffer, Solving mathematical programs with equilibrium constraints as nonlinear programs. Optim. Method. Softw. 19, 15–40 (2004)CrossRefzbMATHMathSciNetGoogle Scholar
  23. 95.
    R. Fletcher, S. Leyffer, D. Ralph, S. Scholtes, Local convergence of SQP methods for mathematical programs with equilibrium constraints. SIAM J. Optim. 17, 259–286 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  24. 99.
    P.E. Gill, D.P. Robinson, A globally convergent stabilized SQP Method. SIAM J. Optim. 23, 1983–2010 (2013)CrossRefMathSciNetGoogle Scholar
  25. 104.
    M. Golubitsky, D.G. Schaeffer, Singularities and Groups in Bifurcation Theory, vol 1. (Springer, New York, 1985)CrossRefzbMATHGoogle Scholar
  26. 112.
    W.W. Hager, Stabilized sequential quadratic programming. Comput. Optim. Appl. 12, 253–273 (1999)CrossRefzbMATHMathSciNetGoogle Scholar
  27. 121.
    T. Hoheisel, C. Kanzow, First- and second-order optimality conditions for mathematical programs with vanishing constraints. Appl. Math. 52, 495–514 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  28. 123.
    T. Hoheisel, C. Kanzow, On the Abadie and Guignard constraint qualifications for mathematical programs with vanishing constraints. Optimization 58, 431–448 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  29. 126.
    A.F. Izmailov, On the Lagrange methods for finding degenerate solutions of constrained optimization problems. Comput. Math. Math. Phys. 36, 423–429 (1996)zbMATHMathSciNetGoogle Scholar
  30. 127.
    A.F. Izmailov, Mathematical programs with complementarity constraints: regularity, optimality conditions, and sensitivity. Comput. Math. Math. Phys. 44, 1145–1164 (2004)MathSciNetGoogle Scholar
  31. 128.
    A.F. Izmailov, On the analytical and numerical stability of critical Lagrange multipliers. Comput. Math. Math. Phys. 45, 930–946 (2005)MathSciNetGoogle Scholar
  32. 132.
    A.F. Izmailov, M.M. Golishnikov, Newton-type methods for constrained optimization with nonregular constraints. Comput. Math. Math. Phys. 46, 1299–1319 (2006)CrossRefMathSciNetGoogle Scholar
  33. 135.
    A.F. Izmailov, A.L. Pogosyan, Optimality conditions and Newton-type methods for mathematical programs with vanishing constraints. Comput. Math. Math. Phys. 49, 1128–1140 (2009)CrossRefMathSciNetGoogle Scholar
  34. 137.
    A.F. Izmailov, A.L. Pogosyan, Active-set Newton methods for mathematical programs with vanishing constraints. Comput. Optim. Appl. 53, 425–452 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  35. 138.
    A.F. Izmailov, M.V. Solodov, Superlinearly convergent algorithms for solving singular equations and smooth reformulations of complementarity problems. SIAM J. Optim. 13, 386–405 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  36. 141.
    A.F. Izmailov, M.V. Solodov, Newton-type methods for optimization problems without constraint qualifications. SIAM J. Optim. 15, 210–228 (2004)CrossRefzbMATHMathSciNetGoogle Scholar
  37. 143.
    A.F. Izmailov, M.V. Solodov, An active-set Newton method for mathematical programs with complementarity constraints. SIAM J. Optim. 19, 1003–1027 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  38. 144.
    A.F. Izmailov, M.V. Solodov, Examples of dual behaviour of Newton-type methods on optimization problems with degenerate constraints. Comput. Optim. Appl. 42, 231–264 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  39. 145.
    A.F. Izmailov, M.V. Solodov, Mathematical programs with vanishing constraints: optimality conditions, sensitivity, and a relaxation method. J. Optim. Theory Appl. 142, 501–532 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  40. 146.
    A.F. Izmailov, M.V. Solodov, On attraction of Newton-type iterates to multipliers violating second-order sufficiency conditions. Math. Program. 117, 271–304 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  41. 149.
    A.F. Izmailov, M.V. Solodov, On attraction of linearly constrained Lagrangian methods and of stabilized and quasi-Newton SQP methods to critical multipliers. Math. Program. 126, 231–257 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  42. 150.
    A.F. Izmailov, M.V. Solodov, Stabilized SQP revisited. Math. Program. 133, 93–120 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  43. 153.
    A.F. Izmailov, E.I. Uskov, Attraction of Newton method to critical Lagrange multipliers: fully quadratic case (2013). Available at http://www.optimization-online.org/DB_HTML/2013/02/3761.html
  44. 154.
    A.F. Izmailov, A.L. Pogosyan, M.V. Solodov, Semismooth SQP method for equality-constrained optimization problems with an application to the lifted reformulation of mathematical programs with complementarity constraints. Optim. Method. Softw. 26, 847–872 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  45. 156.
    A.F. Izmailov, A.L. Pogosyan, M.V. Solodov, Semismooth Newton method for the lifted reformulation of mathematical programs with complementarity constraints. Comput. Optim. Appl. 51, 199–221 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  46. 157.
    A.F. Izmailov, M.V. Solodov, E.I. Uskov, Augmented Lagrangian methods applied to optimization problems with degenerate constraints, including problems with complementarity constraints. SIAM J. Optim. 22, 1579–1606 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  47. 167.
    T. Kato, Perturbation Theory for Linear Operators (Springer, Berlin, 1984)zbMATHGoogle Scholar
  48. 180.
    D.-H. Li, L. Qi, Stabilized SQP method via linear equations. Applied Mathematics Technical Reptort AMR00/5. University of New South Wales, Sydney, 2000Google Scholar
  49. 181.
    G.H. Lin, M. Fukushima, Hybrid approach with active set identification for mathematical programs with complementarity constraints. J. Optim. Theory Appl. 128, 1–28 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  50. 184.
    Z.-Q. Luo, J.-S. Pang, D. Ralph, Mathematical Programs with Equilibrium Constraints (Cambridge University Press, Cambridge, 1996)CrossRefGoogle Scholar
  51. 185.
  52. 203.
    E.M.E. Mostafa, L.N. Vicente, S.J. Wright, Numerical behavior of a stabilized SQP method for degenerate NLP problems, in Global Optimization and Constraint Satisfaction, ed. by C. Bliek, C. Jermann, A. Neumaier. Lecture Notes in Computer Science, vol 2861 (Springer, Berlin, 2003), pp. 123–141Google Scholar
  53. 211.
    J.V. Outrata, M. Kocvara, J. Zowe, Nonsmooth Approach to Optimization Problems with Equilibrium Constraints: Theory, Applications and Numerical Results (Kluwer Academic, Dordrecht, 1998)CrossRefzbMATHGoogle Scholar
  54. 229.
    D. Ralph, Sequential quadratic programming for mathematical programs with linear complementarity constraints, in Computational Techniques and Applications: CTAC95, ed. by R.L. May, A.K. Easton (World Scientific, Singapore, 1996), pp. 663–668Google Scholar
  55. 240.
    H. Scheel, S. Scholtes, Mathematical programs with complementarity constraints: stationarity, optimality and sensitivity. Math. Oper. Res. 25, 1–22 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
  56. 241.
    S. Scholtes, M. Stöhr, How stringent is the linear independence assumption for mathematical programs with complementarity constraints? Math. Oper. Res. 26, 851–863 (2001)CrossRefzbMATHMathSciNetGoogle Scholar
  57. 258.
    O. Stein, Lifting mathematical programs with complementarity constraints. Math. Program. 131, 71–94 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  58. 269.
    S.J. Wright, Superlinear convergence of a stabilized SQP method to a degenerate solution. Comput. Optim. Appl. 11, 253–275 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  59. 270.
    S.J. Wright, Modifying SQP for degenerate problems. SIAM J. Optim. 13, 470–497 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  60. 271.
    S.J. Wright, Constraint identification and algorithm stabilization for degenerate nonlinear programs. Math. Program. 95, 137–160 (2003)CrossRefzbMATHMathSciNetGoogle Scholar
  61. 272.
    S.J. Wright, An algorithm for degenerate nonlinear programming with rapid local convergence. SIAM J. Optim. 15, 673–696 (2005)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Alexey F. Izmailov
    • 1
  • Mikhail V. Solodov
    • 2
  1. 1.Moscow State UniversityMoscowRussia
  2. 2.Instituto de Matemática Pura e AplicadaRio de JaneiroBrazil

Personalised recommendations