Mathematical Programming

, Volume 168, Issue 1–2, pp 673–716 | Cite as

Approximations and generalized Newton methods

  • Diethard KlatteEmail author
  • Bernd Kummer
Full Length Paper Series B


We present approaches to (generalized) Newton methods in the framework of generalized equations \(0\in f(x)+M(x)\), where f is a function and M is a multifunction. The Newton steps are defined by approximations \({\hat{f}}\) of f and the solutions of \(0\in {\hat{f}}(x)+M(x)\). We give a unified view of the local convergence analysis of such methods by connecting a certain type of approximation with the desired kind of convergence and different regularity conditions for \(f+M\). Our paper is, on the one hand, thought as a survey of crucial parts of the topic, where we mainly use concepts and results of the monograph (Klatte and Kummer in Nonsmooth equations in optimization: regularity, calculus, methods and applications, Kluwer Academic Publishers, Dordrecht, 2002). On the other hand, we present original results and new features. They concern the extension of convergence results via Newton maps (Klatte and Kummer in Nonsmooth equations in optimization: regularity, calculus, methods and applications, Kluwer Academic Publishers, Dordrecht, 2002; Kummer, in: Oettli, Pallaschke (eds) Advances in optimization, Springer, Berlin, 1992) from equations to generalized equations both for linear and nonlinear approximations \({\hat{f}}\), and relations between semi-smoothness, Newton maps and directional differentiability of f. We give a Kantorovich-type statement, valid for all sequences of Newton iterates under metric regularity, and recall and extend results on multivalued approximations for general inclusions \(0\in F(x)\). Equations with continuous, non-Lipschitzian f are considered, too.


Generalized Newton method Local convergence Inclusion Generalized equation Regularity Newton map Nonlinear approximation Successive approximation 

Mathematics Subject Classification

49J53 49K40 90C31 65J05 



The authors are indebted to the referees for their constructive comments that significantly improved the presentation.


  1. 1.
    Adly, S., Cibulka, R., van Ngai, H.: Newton’s method for solving inclusion using set-valued approximations. SIAM J. Optim. 25, 159–184 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Artacho, F.J.A., Belyakov, M., Dontchev, A.L., Lopez, M.: Local convergence of quasi-Newton methods under metric regularity. Comput. Optim. Appl. 58, 225–247 (2014)Google Scholar
  3. 3.
    Argyros, I.K.: Convergence and Applications of Newton-Type Iterations. Springer, New York (2010)Google Scholar
  4. 4.
    Aubin, J.-P., Ekeland, I.: Applied Nonlinear Analysis. Wiley, New York (1984)zbMATHGoogle Scholar
  5. 5.
    Bartle, R.G.: Newton’s method in Banach spaces. Proc. Am. Math. Soc. 5, 827–831 (1955)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Broyden, C.G.: A class of methods for solving nonlinear simultaneous equations. Math. Comput. 19, 577–593 (1965)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Bütikofer, S., Klatte, D.: A nonsmooth Newton method with path search and its use in solving \(C^{1,1}\) programs and semi-infinite problems. SIAM J. Optim. 20, 2381–2412 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Cibulka, R., Dontchev, A.L., Geoffroy, M.H.: Inexact Newton methods and Dennis–More theorems for nonsmooth generalized equations. SIAM J. Control Optim. 53, 1003–1019 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Cibulka, R., Dontchev, A.L., Preininger, J., Roubal, T., Veliov, V.: Kantorovich-type theorems for generalized equations. Research Report 2015-16, TU Wien (2015)Google Scholar
  10. 10.
    Clarke, F.H.: On the inverse function theorem. Pac. J. Math. 64, 97–102 (1976)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)zbMATHGoogle Scholar
  12. 12.
    Dontchev, A.L.: Local convergence of the Newton method for generalized equation. C. R. Acad. Sci. Paris Ser. I 322, 327–331 (1996)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Dontchev, A.L.: Local Analysis of a Newton-type Method Based on Partial Linearization. Lectures in Applied Mathematics, vol. 32, pp. 295–306 (1996)Google Scholar
  14. 14.
    Dontchev, A.L., Rockafellar, R.T.: Characterizations of strong regularity for variational inequalities over polyhedral convex sets. SIAM J. Optim. 6, 1087–1105 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Dontchev, A.L., Rockafellar, R.T.: Implicit Functions and Solution Mappings. A View from Variational Analysis. Springer, Dortrecht (2009). (2nd edition 2014)CrossRefzbMATHGoogle Scholar
  16. 16.
    Dontchev, A.L., Rockafellar, R.T.: Newton’s method for generalized equations: a sequential implicit function theorem. Math. Program. Ser. B 123, 139–159 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementary Problems, vol. II. Springer, New York (2003)zbMATHGoogle Scholar
  18. 18.
    Fusek, P.: Isolated zeros of Lipschitzian metrically regular R\(^n\) functions. Optimization 49, 425–446 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Griewank, A.: The local convergence of Broyden-like methods on Lipschitzian problems in Hilbert spaces. SIAM J. Numer. Anal. 24, 684–705 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Griewank, A.: On stable piecewise linearization and generalized algorithmic differentiation. Optim. Methods Softw. 28, 1139–1178 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Hintermüller, M., Ito, K., Kunisch, K.: The primal-dual active set strategy as a semi-smooth Newton method. SIAM J. Optim. 13, 865–888 (2003)CrossRefzbMATHGoogle Scholar
  22. 22.
    Hoheisel, T., Kanzow, C., Mordukhovich, B.S., Phan, H.: Generalized Newton’s method based on graphical derivatives. Nonlinear Anal. 75, 1324–1340 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Ito, K., Kunisch, K.: Lagrange Multiplier Approach to Variational Problems and Applications. SIAM, Philadelphia (2008)CrossRefzbMATHGoogle Scholar
  24. 24.
    Izmailov, A.F., Solodov, M.V.: Newton-Type Methods for Optimization and Variational Problems. Springer, Cham (2014)CrossRefzbMATHGoogle Scholar
  25. 25.
    Jongen, HTh, Jonker, P., Twilt, F.: Nonlinear Optimization in \(R^n\), I: Morse Theory, Chebychev Approximation. Peter Lang, Bern (1983)zbMATHGoogle Scholar
  26. 26.
    Jongen, HTh, Jonker, P., Twilt, F.: Nonlinear Optimization in \(R^n\), II: Transversality, Flows, Parametric Aspects. Peter Lang, Bern (1986)zbMATHGoogle Scholar
  27. 27.
    Josephy, N.H.: Newton’s method for generalized equations and the PIES energy model. Ph.D. Dissertation, Department of Industrial Engineering, University of Wisconsin-Madison (1979)Google Scholar
  28. 28.
    Kantorovich, L.W., Akilov, G.P.: Funktionalanalysis in normierten Räumen. Akademie, Berlin (1964)zbMATHGoogle Scholar
  29. 29.
    Klatte, D., Kruger, A., Kummer, B.: From convergence principles to stability and optimality conditions. J. Convex Anal. 19, 1043–1072 (2012)MathSciNetzbMATHGoogle Scholar
  30. 30.
    King, A.J., Rockafellar, R.T.: Sensitivity analysis for nonsmooth generalized equations. Math. Program. 55(2), 193–212 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Klatte, D., Kummer, B.: Nonsmooth Equations in Optimization: Regularity, Calculus, Methods and Applications. Kluwer Academic Publishers, Dordrecht (2002)zbMATHGoogle Scholar
  32. 32.
    Klatte, D., Kummer, B.: Strong Lipschitz stability of stationary solutions for nonlinear programs and variational inequalities. SIAM J. Optim. 16, 96–119 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Klatte, D., Kummer, B.: Newton methods for stationary points: an elementary view of regularity conditions and solution schemes. Optimization 56, 441–462 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Klatte, D., Kummer, B.: Aubin property and uniqueness of solutions in cone constrained optimization. Math. Methods Oper. Res. 77, 291–304 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Kojima, M., Shindo, S.: Extension of Newton and Quasi-Newton methods to systems of \(PC^1\) equations. J. Oper. Res. Soc. Jpn. 29, 352–374 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Kummer, B.: Newton’s method for non-differentiable functions. In: Guddat, J., et al. (eds.) Advances in Mathematical Optimization. Series in Mathematical Research, vol. 45, pp. 114–125. Akademie, Berlin (1988)Google Scholar
  37. 37.
    Kummer, B.: Lipschitzian inverse functions, directional derivatives and application in \({C}^{1,1}\) optimization. J. Optim. Theory Appl. 70, 559–580 (1991)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Kummer, B.: Newton’s method based on generalized derivatives for nonsmooth functions: convergence analysis. In: Oettli, W., Pallaschke, D. (eds.) Advances in Optimization, pp. 171–194. Springer, Berlin (1992)CrossRefGoogle Scholar
  39. 39.
    Kummer, B.: Approximation of multifunctions and superlinear convergence. In: Durier, R., Michelot, C. (eds.) Recent Developments in Optimization. Lecture Notes in Economics and Mathematical Systems, vol. 429, pp. 243–251. Springer, Berlin (1995)CrossRefGoogle Scholar
  40. 40.
    Kummer, B.: Lipschitzian and pseudo-Lipschitzian inverse functions and applications to nonlinear programming. In: Fiacco, A.V. (ed.) Mathematical Programming with Data Perturbations, pp. 201–222. M. Dekker, New York (1997)Google Scholar
  41. 41.
    Kummer, B.: Newton’s method for continuous functions? script for students. (2012)
  42. 42.
    Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 15, 957–972 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Mordukhovich, B.S.: Complete characterization of openness, metric regularity and Lipschitzian properties of multifunctions. Trans. Am. Math. Soc. 340, 1–35 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Ortega, J.M.: The Newton–Kantorovich theorem. Am. Math. Mon. 75, 658–660 (1968)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Pang, J.-S., Qi, L.: Nonsmooth equations: motivation and algorithms. SIAM J. Optim. 3, 443–465 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Potra, F.A., Ptak, V.: Nondiscrete Induction and Iterative Processes. Research Notes in Mathematics, vol. 103. Pitman, Boston (1984)Google Scholar
  47. 47.
    Qi, L., Sun, L.: A nonsmooth version of Newton’s method. Math. Program. 58, 353–367 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    Rademacher, H.: Ueber partielle und totale Differenzierbarkeit von Funktionen mehrerer Variabeln und ueber die Transformation der Doppelintegrale. Math. Ann. 79, 340–359 (1919)MathSciNetCrossRefzbMATHGoogle Scholar
  49. 49.
    Robinson, S.M.: Extension of Newton’s method to nonlinear functions with values in a cone. Numer. Math. 19, 341–347 (1972)MathSciNetCrossRefzbMATHGoogle Scholar
  50. 50.
    Robinson, S.M.: Generalized equations and their solutions. Part I: Basic theory. Math. Program. Study 10, 128–141 (1979)CrossRefzbMATHGoogle Scholar
  51. 51.
    Robinson, S.M.: Strongly regular generalized equations. Math. Oper. Res. 5, 43–62 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  52. 52.
    Robinson, S.M.: Newton’s method for a class of nonsmooth functions. Set Valued Anal. 2, 291–305 (1994). (Preprint 1988)MathSciNetCrossRefzbMATHGoogle Scholar
  53. 53.
    Rockafellar, R.T.: Proto-differentiability of set-valued mappings and its applications in optimization. In: Analyse non linéaire (Perpignan 1987), Ann. Inst. H. Poincaré, Anal. Non Linéaire, vol. 6 (suppl.), pp. 448–482 (1989)Google Scholar
  54. 54.
    Rockafellar, R.T., Wets, J.-B.: Variational Analysis. Springer, Berlin (1998)CrossRefzbMATHGoogle Scholar
  55. 55.
    Scholtes, S.: Introduction to Piecewise Differentiable Equations. Springer, New York (2012). (Preprint 1994)CrossRefzbMATHGoogle Scholar
  56. 56.
    Shapiro, A.: On concepts of directional differentiability. J. Optim. Theory Appl. 66, 477–487 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  57. 57.
    Sun, D., Qi, L.: On NCP functions. Comput. Optim. Appl. 13, 201–220 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  58. 58.
    Ulbrich, M.: Semismooth Newton methods for operator equations in function spaces. SIAM J. Optim. 13, 805–841 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  59. 59.
    Ulbrich, M.: Semismooth Newton Methods for Variational Inequalities and Constrained Optimization Problems in Function Spaces. SIAM, Philadelphia (2011)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany and Mathematical Optimization Society 2017

Authors and Affiliations

  1. 1.IBWUniversität ZürichZurichSwitzerland
  2. 2.Institut für MathematikHumboldt-Universität zu BerlinBerlinGermany

Personalised recommendations