Abstract
We present approaches to (generalized) Newton methods in the framework of generalized equations \(0\in f(x)+M(x)\), where f is a function and M is a multifunction. The Newton steps are defined by approximations \({\hat{f}}\) of f and the solutions of \(0\in {\hat{f}}(x)+M(x)\). We give a unified view of the local convergence analysis of such methods by connecting a certain type of approximation with the desired kind of convergence and different regularity conditions for \(f+M\). Our paper is, on the one hand, thought as a survey of crucial parts of the topic, where we mainly use concepts and results of the monograph (Klatte and Kummer in Nonsmooth equations in optimization: regularity, calculus, methods and applications, Kluwer Academic Publishers, Dordrecht, 2002). On the other hand, we present original results and new features. They concern the extension of convergence results via Newton maps (Klatte and Kummer in Nonsmooth equations in optimization: regularity, calculus, methods and applications, Kluwer Academic Publishers, Dordrecht, 2002; Kummer, in: Oettli, Pallaschke (eds) Advances in optimization, Springer, Berlin, 1992) from equations to generalized equations both for linear and nonlinear approximations \({\hat{f}}\), and relations between semi-smoothness, Newton maps and directional differentiability of f. We give a Kantorovich-type statement, valid for all sequences of Newton iterates under metric regularity, and recall and extend results on multivalued approximations for general inclusions \(0\in F(x)\). Equations with continuous, non-Lipschitzian f are considered, too.
Similar content being viewed by others
Change history
25 October 2019
In Subsection 3.2 of the paper [5]
25 October 2019
In Subsection 3.2 of the paper [5]
References
Adly, S., Cibulka, R., van Ngai, H.: Newton’s method for solving inclusion using set-valued approximations. SIAM J. Optim. 25, 159–184 (2015)
Artacho, F.J.A., Belyakov, M., Dontchev, A.L., Lopez, M.: Local convergence of quasi-Newton methods under metric regularity. Comput. Optim. Appl. 58, 225–247 (2014)
Argyros, I.K.: Convergence and Applications of Newton-Type Iterations. Springer, New York (2010)
Aubin, J.-P., Ekeland, I.: Applied Nonlinear Analysis. Wiley, New York (1984)
Bartle, R.G.: Newton’s method in Banach spaces. Proc. Am. Math. Soc. 5, 827–831 (1955)
Broyden, C.G.: A class of methods for solving nonlinear simultaneous equations. Math. Comput. 19, 577–593 (1965)
Bütikofer, S., Klatte, D.: A nonsmooth Newton method with path search and its use in solving \(C^{1,1}\) programs and semi-infinite problems. SIAM J. Optim. 20, 2381–2412 (2010)
Cibulka, R., Dontchev, A.L., Geoffroy, M.H.: Inexact Newton methods and Dennis–More theorems for nonsmooth generalized equations. SIAM J. Control Optim. 53, 1003–1019 (2015)
Cibulka, R., Dontchev, A.L., Preininger, J., Roubal, T., Veliov, V.: Kantorovich-type theorems for generalized equations. Research Report 2015-16, TU Wien (2015)
Clarke, F.H.: On the inverse function theorem. Pac. J. Math. 64, 97–102 (1976)
Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)
Dontchev, A.L.: Local convergence of the Newton method for generalized equation. C. R. Acad. Sci. Paris Ser. I 322, 327–331 (1996)
Dontchev, A.L.: Local Analysis of a Newton-type Method Based on Partial Linearization. Lectures in Applied Mathematics, vol. 32, pp. 295–306 (1996)
Dontchev, A.L., Rockafellar, R.T.: Characterizations of strong regularity for variational inequalities over polyhedral convex sets. SIAM J. Optim. 6, 1087–1105 (1996)
Dontchev, A.L., Rockafellar, R.T.: Implicit Functions and Solution Mappings. A View from Variational Analysis. Springer, Dortrecht (2009). (2nd edition 2014)
Dontchev, A.L., Rockafellar, R.T.: Newton’s method for generalized equations: a sequential implicit function theorem. Math. Program. Ser. B 123, 139–159 (2010)
Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementary Problems, vol. II. Springer, New York (2003)
Fusek, P.: Isolated zeros of Lipschitzian metrically regular R\(^n\) functions. Optimization 49, 425–446 (2001)
Griewank, A.: The local convergence of Broyden-like methods on Lipschitzian problems in Hilbert spaces. SIAM J. Numer. Anal. 24, 684–705 (1987)
Griewank, A.: On stable piecewise linearization and generalized algorithmic differentiation. Optim. Methods Softw. 28, 1139–1178 (2013)
Hintermüller, M., Ito, K., Kunisch, K.: The primal-dual active set strategy as a semi-smooth Newton method. SIAM J. Optim. 13, 865–888 (2003)
Hoheisel, T., Kanzow, C., Mordukhovich, B.S., Phan, H.: Generalized Newton’s method based on graphical derivatives. Nonlinear Anal. 75, 1324–1340 (2012)
Ito, K., Kunisch, K.: Lagrange Multiplier Approach to Variational Problems and Applications. SIAM, Philadelphia (2008)
Izmailov, A.F., Solodov, M.V.: Newton-Type Methods for Optimization and Variational Problems. Springer, Cham (2014)
Jongen, HTh, Jonker, P., Twilt, F.: Nonlinear Optimization in \(R^n\), I: Morse Theory, Chebychev Approximation. Peter Lang, Bern (1983)
Jongen, HTh, Jonker, P., Twilt, F.: Nonlinear Optimization in \(R^n\), II: Transversality, Flows, Parametric Aspects. Peter Lang, Bern (1986)
Josephy, N.H.: Newton’s method for generalized equations and the PIES energy model. Ph.D. Dissertation, Department of Industrial Engineering, University of Wisconsin-Madison (1979)
Kantorovich, L.W., Akilov, G.P.: Funktionalanalysis in normierten Räumen. Akademie, Berlin (1964)
Klatte, D., Kruger, A., Kummer, B.: From convergence principles to stability and optimality conditions. J. Convex Anal. 19, 1043–1072 (2012)
King, A.J., Rockafellar, R.T.: Sensitivity analysis for nonsmooth generalized equations. Math. Program. 55(2), 193–212 (1992)
Klatte, D., Kummer, B.: Nonsmooth Equations in Optimization: Regularity, Calculus, Methods and Applications. Kluwer Academic Publishers, Dordrecht (2002)
Klatte, D., Kummer, B.: Strong Lipschitz stability of stationary solutions for nonlinear programs and variational inequalities. SIAM J. Optim. 16, 96–119 (2005)
Klatte, D., Kummer, B.: Newton methods for stationary points: an elementary view of regularity conditions and solution schemes. Optimization 56, 441–462 (2007)
Klatte, D., Kummer, B.: Aubin property and uniqueness of solutions in cone constrained optimization. Math. Methods Oper. Res. 77, 291–304 (2013)
Kojima, M., Shindo, S.: Extension of Newton and Quasi-Newton methods to systems of \(PC^1\) equations. J. Oper. Res. Soc. Jpn. 29, 352–374 (1986)
Kummer, B.: Newton’s method for non-differentiable functions. In: Guddat, J., et al. (eds.) Advances in Mathematical Optimization. Series in Mathematical Research, vol. 45, pp. 114–125. Akademie, Berlin (1988)
Kummer, B.: Lipschitzian inverse functions, directional derivatives and application in \({C}^{1,1}\) optimization. J. Optim. Theory Appl. 70, 559–580 (1991)
Kummer, B.: Newton’s method based on generalized derivatives for nonsmooth functions: convergence analysis. In: Oettli, W., Pallaschke, D. (eds.) Advances in Optimization, pp. 171–194. Springer, Berlin (1992)
Kummer, B.: Approximation of multifunctions and superlinear convergence. In: Durier, R., Michelot, C. (eds.) Recent Developments in Optimization. Lecture Notes in Economics and Mathematical Systems, vol. 429, pp. 243–251. Springer, Berlin (1995)
Kummer, B.: Lipschitzian and pseudo-Lipschitzian inverse functions and applications to nonlinear programming. In: Fiacco, A.V. (ed.) Mathematical Programming with Data Perturbations, pp. 201–222. M. Dekker, New York (1997)
Kummer, B.: Newton’s method for continuous functions? script for students. http://www2.mathematik.hu-berlin.de/kummer/scripts/ContNewton.pdf (2012)
Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 15, 957–972 (1977)
Mordukhovich, B.S.: Complete characterization of openness, metric regularity and Lipschitzian properties of multifunctions. Trans. Am. Math. Soc. 340, 1–35 (1993)
Ortega, J.M.: The Newton–Kantorovich theorem. Am. Math. Mon. 75, 658–660 (1968)
Pang, J.-S., Qi, L.: Nonsmooth equations: motivation and algorithms. SIAM J. Optim. 3, 443–465 (1993)
Potra, F.A., Ptak, V.: Nondiscrete Induction and Iterative Processes. Research Notes in Mathematics, vol. 103. Pitman, Boston (1984)
Qi, L., Sun, L.: A nonsmooth version of Newton’s method. Math. Program. 58, 353–367 (1993)
Rademacher, H.: Ueber partielle und totale Differenzierbarkeit von Funktionen mehrerer Variabeln und ueber die Transformation der Doppelintegrale. Math. Ann. 79, 340–359 (1919)
Robinson, S.M.: Extension of Newton’s method to nonlinear functions with values in a cone. Numer. Math. 19, 341–347 (1972)
Robinson, S.M.: Generalized equations and their solutions. Part I: Basic theory. Math. Program. Study 10, 128–141 (1979)
Robinson, S.M.: Strongly regular generalized equations. Math. Oper. Res. 5, 43–62 (1980)
Robinson, S.M.: Newton’s method for a class of nonsmooth functions. Set Valued Anal. 2, 291–305 (1994). (Preprint 1988)
Rockafellar, R.T.: Proto-differentiability of set-valued mappings and its applications in optimization. In: Analyse non linéaire (Perpignan 1987), Ann. Inst. H. Poincaré, Anal. Non Linéaire, vol. 6 (suppl.), pp. 448–482 (1989)
Rockafellar, R.T., Wets, J.-B.: Variational Analysis. Springer, Berlin (1998)
Scholtes, S.: Introduction to Piecewise Differentiable Equations. Springer, New York (2012). (Preprint 1994)
Shapiro, A.: On concepts of directional differentiability. J. Optim. Theory Appl. 66, 477–487 (1990)
Sun, D., Qi, L.: On NCP functions. Comput. Optim. Appl. 13, 201–220 (1999)
Ulbrich, M.: Semismooth Newton methods for operator equations in function spaces. SIAM J. Optim. 13, 805–841 (2003)
Ulbrich, M.: Semismooth Newton Methods for Variational Inequalities and Constrained Optimization Problems in Function Spaces. SIAM, Philadelphia (2011)
Acknowledgements
The authors are indebted to the referees for their constructive comments that significantly improved the presentation.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Klatte, D., Kummer, B. Approximations and generalized Newton methods. Math. Program. 168, 673–716 (2018). https://doi.org/10.1007/s10107-017-1194-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-017-1194-8
Keywords
- Generalized Newton method
- Local convergence
- Inclusion
- Generalized equation
- Regularity
- Newton map
- Nonlinear approximation
- Successive approximation