Elements of Optimization Theory and Variational Analysis

  • Alexey F. Izmailov
  • Mikhail V. Solodov
Part of the Springer Series in Operations Research and Financial Engineering book series (ORFE)


In this chapter we state the problem settings that will be investigated in the book and discuss some theoretical issues necessary for the subsequent analysis of numerical methods. We emphasize that the concept of this chapter is rather minimalistic. We provide only the material that would be directly used later on and prove only those facts which cannot be regarded as “standard” (i.e., their proofs cannot be found in well-established sources). For the material that we regard as rather standard, we limit our presentation to the statements, references, and comments.


Variational Inequality Maximal Monotone Constraint Qualification Mathematical Programming Problem Contingent Cone 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 13.
    A.V. Arutyunov, Optimality Conditions: Abnormal and Degenerate Problems (Kluwer Academic, Dordrecht, 2000)CrossRefGoogle Scholar
  2. 14.
    A.V. Arutyunov, A.F. Izmailov, Sensitivity analysis for cone-constrained optimization problems under the relaxed constraint qualifications. Math. Oper. Res. 30, 333–353 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 19.
    D.P. Bertsekas, Nonlinear Programming, 2nd edn. (Athena, Belmont, 1999)zbMATHGoogle Scholar
  4. 26.
    J.F. Bonnans, Local analysis of Newton-type methods for variational inequalities and nonlinear programming. Appl. Math. Optim. 29, 161–186 (1994)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 27.
    J.F. Bonnans, A. Shapiro, Perturbation Analysis of Optimization Problems (Springer, New York, 2000)CrossRefzbMATHGoogle Scholar
  6. 28.
    J.F. Bonnans, A. Sulem, Pseudopower expansion of solutions of generalized equations and constrained optimization. Math. Program. 70, 123–148 (1995)zbMATHMathSciNetGoogle Scholar
  7. 29.
    J.F. Bonnans, J.Ch. Gilbert, C. Lemaréchal, C. Sagastizábal, Numerical Optimization: Theoretical and Practical Aspects, 2nd edn. (Springer, Berlin, 2006)Google Scholar
  8. 43.
    F.H. Clarke, On the inverse function theorem. Pacisic J. Math. 64, 97–102 (1976)CrossRefzbMATHGoogle Scholar
  9. 44.
    F.H. Clarke, Optimization and Nonsmooth Analysis (Wiley, New York, 1983)zbMATHGoogle Scholar
  10. 46.
    R.W. Cottle, J.-S. Pang, R.E. Stone, The Linear Complementarity Problem (SIAM, Philadelphia, 2009)CrossRefzbMATHGoogle Scholar
  11. 62.
    A.L. Dontchev, R.T. Rockafellar, Implicit Functions and Solution Mappings (Springer, New York, 2009)CrossRefzbMATHGoogle Scholar
  12. 63.
    A.L. Dontchev, R.T. Rockafellar, Newton’s method for generalized equations: a sequential implicit function theorem. Math. Program. 123, 139–159 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  13. 64.
    H.G. Eggleston, Convexity (Cambridge University Press, Cambridge, 1958)CrossRefzbMATHGoogle Scholar
  14. 65.
    L.C. Evans, R.F. Gariepy, Measure Theory and Fine Properties of Functions (CRC Press, Boca Raton, 1992)zbMATHGoogle Scholar
  15. 66.
    M. Fabian, D. Preiss, On the Clarke’s generalized Jacobian, in Proceedings of the 14th Winter School on Abstract Analysis, Circolo Matematico di Palermo, Palermo, 1987. Rendiconti del Circolo Matematico di Palermo, Ser. II, Number 14, pp. 305–307Google Scholar
  16. 68.
    F. Facchinei, J.-S. Pang, Finite-Dimensional Variational Inequalities and Complementarity Problems (Springer, New York, 2003)Google Scholar
  17. 77.
    D. Fernández, A.F. Izmailov, M.V. Solodov, Sharp primal superlinear convergence results for some Newtonian methods for constrained optimization. SIAM J. Optim. 20, 3312–3334 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 83.
    A. Fischer, Solution of monotone complementarity problems with locally Lipschitzian function. Math. Program. 76, 513–532 (1997)zbMATHGoogle Scholar
  19. 85.
    A. Fischer, Local behavior of an iterative framework for generalized equations with nonisolated solutions. Math. Program. 94, 91–124 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  20. 98.
    J. Gauvin, A necessary and sufficient regularity conditions to have bounded multipliers in nonconvex programming. Math. Program. 12, 136–138 (1977)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 113.
    W.W. Hager, M.S. Gowda, Stability in the presence of degeneracy and error estimation. Math. Program. 85, 181–192 (1999)CrossRefzbMATHMathSciNetGoogle Scholar
  22. 119.
    J.-B. Hiriart-Urruty, Mean-value theorems in nonsmooth analysis. Numer. Func. Anal. Optim. 2, 1–30 (1980)CrossRefzbMATHMathSciNetGoogle Scholar
  23. 120.
    J.-B. Hiriart-Urruty, J.-J. Strodiot, V.H. Nguyen, Generalized Hessian matrix and second-order optimality conditions for problems with C 1, 1 data. Appl. Math. Optim. 11, 43–56 (1984)CrossRefzbMATHMathSciNetGoogle Scholar
  24. 128.
    A.F. Izmailov, On the analytical and numerical stability of critical Lagrange multipliers. Comput. Math. Math. Phys. 45, 930–946 (2005)MathSciNetGoogle Scholar
  25. 129.
    A.F. Izmailov, Sensitivity of solutions to systems of optimality conditions under the violation of constraint qualifications. Comput. Math. Math. Phys. 47, 533–554 (2007)CrossRefMathSciNetGoogle Scholar
  26. 130.
    A.F. Izmailov, Solution sensitivity for Karush–Kuhn–Tucker systems with nonunique Lagrange multipliers. Optimization 59, 747–775 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  27. 142.
    A.F. Izmailov, M.V. Solodov, A note on solution sensitivity for Karush–Kuhn–Tucker systems. Math. Meth. Oper. Res. 61, 347–363 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  28. 150.
    A.F. Izmailov, M.V. Solodov, Stabilized SQP revisited. Math. Program. 133, 93–120 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  29. 155.
    A.F. Izmailov, A.S. Kurennoy, M.V. Solodov, A note on upper Lipschitz stability, error bounds, and critical multipliers for Lipschitz-continuous KKT systems. Math. Program. 142, 591–604 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  30. 166.
    W. Karush, Minima of functions of several variables with inequalities as side constraints. Technical Report (master’s thesis), Department of Mathematics, University of Chicago, 1939Google Scholar
  31. 168.
    D. Klatte, Upper Lipschitz behaviour of solutions to perturbed C 1, 1 programs. Math. Program. 88, 285–311 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
  32. 169.
    D. Klatte, B. Kummer, Nonsmooth Equations in Optimization: Regularity, Calculus, Methods and Applications (Kluwer Academic, Dordrecht, 2002)Google Scholar
  33. 170.
    D. Klatte, K. Tammer, On the second order sufficient conditions to perturbed C 1, 1 optimization problems. Optimization 19, 169–180 (1988)CrossRefzbMATHMathSciNetGoogle Scholar
  34. 171.
    M. Kojima, Strongly stable stationary solutions in nonlinear programs, in Analysis and Computation of Fixed Points, ed. by S.M. Robinson (Academic, NewYork, 1980), pp. 93–138Google Scholar
  35. 174.
    H.W. Kuhn, A.W. Tucker, Non-linear programming. in Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, ed. by J. Neyman (University of California Press, Berkeley, 1951), pp. 481–493Google Scholar
  36. 178.
    A. Levy, Solution sensitivity from general principles. SIAM J. Control Optim. 40, 1–38 (2001)CrossRefzbMATHMathSciNetGoogle Scholar
  37. 186.
    O.L. Mangasarian, Nonlinear Programming (SIAM, Philadelphia, 1994)CrossRefzbMATHGoogle Scholar
  38. 187.
    O.L. Mangasarian, S. Fromovitz, The Fritz John necessary optimality conditions in the presence of equality and inequality constraints. J. Math. Anal. Appl. 17, 37–47 (1967)CrossRefzbMATHMathSciNetGoogle Scholar
  39. 199.
    R. Mifflin, Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 15, 959–972 (1977)CrossRefzbMATHMathSciNetGoogle Scholar
  40. 201.
    B.S. Mordukhovich, Variational Analysis and Generalized Differentiation (Springer, Berlin, 2006)Google Scholar
  41. 208.
    J. Nocedal, S.J. Wright, Numerical Optimization, 2nd edn. (Springer, New York, 2006)zbMATHGoogle Scholar
  42. 213.
    J.-S. Pang, L. Qi, Nonsmooth equations: motivation and algorithms. SIAM J. Optim. 3, 443–465 (1993)CrossRefzbMATHMathSciNetGoogle Scholar
  43. 222.
    L. Qi, Convergence analysis of some algorithms for solving nonsmooth equations. Math. Oper. Res. 18, 227–244 (1993)CrossRefzbMATHMathSciNetGoogle Scholar
  44. 224.
    L. Qi, H. Jiang, Semismooth Karush–Kuhn–Tucker equations and convergence analysis of Newton and quasi-Newton methods for solving these equations. Math. Oper. Res. 22, 301–325 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  45. 225.
    L. Qi, J. Sun, A nonsmooth version of Newton’s method. Math. Program. 58, 353–367 (1993)CrossRefzbMATHMathSciNetGoogle Scholar
  46. 227.
    H. Rademacher, Über partielle und totale differenzierbarkeit I. Math. Ann. 89, 340–359 (1919)CrossRefMathSciNetGoogle Scholar
  47. 233.
    S.M. Robinson, Stability theory for systems of inequalities, Part II. Differentiable nonlinear systems, SIAM J. Numer. Anal. 13, 497–513 (1976)Google Scholar
  48. 234.
    S.M. Robinson, Strongly regular generalized equations. Math. Oper. Res. 5, 43–62 (1980)CrossRefzbMATHMathSciNetGoogle Scholar
  49. 239.
    R.T. Rockafellar, R.J.-B. Wets, Variational Analysis (Springer, Berlin, 1998)CrossRefzbMATHGoogle Scholar
  50. 244.
    A. Shapiro, On concepts of directional differentiability. J. Optim. Theory Appl. 66, 477–487 (1990)CrossRefzbMATHMathSciNetGoogle Scholar
  51. 245.
    A. Shapiro, Sensitivity analysis of generalized equations. J. Math. Sci. 115, 2554–2565 (2003)CrossRefzbMATHMathSciNetGoogle Scholar
  52. 250.
    M.V. Solodov, Constraint qualifications, in Wiley Encyclopedia of Operations Research and Management Science, ed. by J.J. Cochran (Wiley, London, 2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Alexey F. Izmailov
    • 1
  • Mikhail V. Solodov
    • 2
  1. 1.Moscow State UniversityMoscowRussia
  2. 2.Instituto de Matemática Pura e AplicadaRio de JaneiroBrazil

Personalised recommendations