Abstract
In this paper we survey numerical methods for solving nonlinear systems of equations F (x) = 0, where F: R n— R n. We are especially interested in large problems. We describe modern implementations of the main local algorithms, as well as their globally convergent counterparts.
Work supported by FAPESP (PT NO 90-3724-6) FAEP-UNICAMP, FINEP and CNPq.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abaffy, J. [1992]: Superlinear convergence theorems for Newton-type methods for nonlinear systems of equations, JOTA, 73, pp. 269–277
Abaffy, J.; Broyden, C.G.; Spedicato, E. [1984]: A class of direct methods for linear equations, Numerische Mathematik 45, pp. 361–376
Abaffy, J.; Galantai, A.; Spedicato, E. [1987]: The local convergence of ABS method for nonlinear algebraic system, Numerische Mathematik 51, pp. 429–439
Abaffy, J.; Spedicato, E. [1989]: ABS Projection Algorithms, Mathematical Techniques for Linear and Nonlinear Equations, Ellis Horwood, Chichester
Axelsson, O. [1985]: A survey of preconditioned iterative methods for linear systems of equations, BIT 25, pp. 166–187
Barnes, J.G.P. [1965]: An algorithm for solving nonlinear equations based on the secant method, Computer Journal 8, pp. 66–72
Bertsekas, D.P.; Tsitsiklis, C. [1989]: Parallel and Distributed Computation. Numerical Methods, The Johns Hopkins University Press, New York
Brent, R.P. [1973]: Some efficient algorithms for solving systems of nonlinear equations, SIAM Journal on Numerical Analysis 10, pp. 327–344
Brown, K.M. [1969]: A quadratically convergent Newton-like method based upon Gaussian elimination, SIAM Journal on Numerical Analysis 6, pp. 560–569
Broyden, C.G. [1965]: A class of methods for solving nonlinear simultaneous equations, Mathematics of Computation 19, pp. 577–593
Broyden, C.G.; Dennis Jr., J.E.; More, J.J. [1973]: On the local and superlinear convergence of quasi-Newton methods, Journal of the Institute of Mathematics and its Applications 12, pp. 223–245
Burmeister, W.; Schmidt, J.W. [1978]: On the k-order of coupled sequences arising in single-step type methods, Numerische Mathematik 33, pp. 653–661
Chadee, F.F. [1985]: Sparse quasi-Newton methods and the continuation problem, T.R. S.O.L. 85–8, Department of Operations Research, Stanford University
Chow, S.N.; Mallet-Paret, J.; Yorke, J.A. [1978]: Finding zeros of maps: Homotopy methods that are constructive with probability one, Mathematics of Computation 32, pp. 887–899
Cimmino, G. [1938]: Calcolo approsimato per la soluzione dei sistemi di equazioni lineari, La Ricerca Scientifica Ser II, Ano IV 1 pp. 326–333
Coleman, T. F.; Garbow, B. S.; Moré, J. J. [1984]: Software for estimating sparse Jacobian matrices, ACM Trans. Math. Software 11, pp. 363–378
Coleman, T. F.; Moré, J. J. [1983]: Estimation of sparse Jacobian matrices and graph coloring problems, SIAM Journal on Numerical Analysis 20, pp. 187–209
Curtis, A. R.; Powell, M. J. D.; Reid, J. K. [1974], On the estimation of sparse Jacobian matrices, Journal of the Institute of Mathematics and its Applications 13, pp. 117–120
Davidenko, D.F. [1953]: On the approximate solution of nonlinear equations, Ukrain. Mat. Z. 5, pp. 196 -206
Dembo, R.S.; Eisenstat, S.C.; Steihaug, T. [1982]: Inexact Newton methods, SIAM Journal on Numerical Analysis 19, pp. 400–408
Dennis Jr., J.E.; Martinez, J.M.; Zhang, X. [1992]: Triangular Decomposition Methods for Reducible Nonlinear Systems of Equations, Technical Report, Department of Mathematical Sciences, Rice University
Dennis Jr., J.E.; Marwil, E.S. [1982]: Direct secant updates of matrix factorizations, Mathematics of Computation 38, pp. 459–476
Dennis Jr., J.E.; More, J.J. [1974]: A characterization of superlinear convergence and its application to quasi-Newton methods, Mathematics of Computation 28, pp. 549 -560
Dennis Jr., J.E.; More, J.J. [1977]: Quasi-Newton methods, motivation and theory, SIAM Review 19, pp. 46–89
Dennis Jr., J.E.; Schnabel, R.B. [1979]: Least change secant updates for quasi-Newton methods, SIAM Review 21, pp. 443–459
Dennis Jr., J.E.; Schnabel, R.B. [1983]: Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs
Dennis Jr., J.E.; Walker, H.F. [1981]: Convergence theorems for least-change secant update methods, SIAM Journal on Numerical Analysis 18, pp. 949–987
Deuflhard, P. [1991]: Global inexact Newton methods for very large scale nonlinear problems Impact of Computing in Science and Engineering 3, pp. 366–393
Diniz-Ehrhardt, M.A.; Martinez, J.M. [1992]: A parallel projection method for over determined nonlinear systems of equations, to appear in Numerical Algorithms
Duff, I.S. [1977]: MA28-a set of Fortran subroutrines for sparse unsymmetric linear equations. AERE R8730, HMSO, London
Duff, I.S.; Erisman, A.M.; Reid, J.K. [1989]: Direct Methods for Sparse Matrices, Oxford Scientific Publications
Eisenstat, S.C.; Walker, H.F. [1993]: Globally convergent inexact Newton methods, to appear in SIAM Journal on Optimization
Eriksson, J. [1976]: A note on the decomposition of systems of sparse nonlinear equations, BIT 16, pp. 462–465
Fletcher, R. [1987]: Practical Methods of Optimization (2nd edition), John Wiley and Sons, New York
Forster, W. [1993]: Homotopy methods, to appear in Handbook of Global Optimization, Kluwer
Friedlander, A.; Gomes-Ruggiero, M.A.; Marténez, J.M.; Santos, S.A. [1993]: A globally convergent method for solving nonlinear systems using a trust-region strategy, Technical Report, Department of Applied Mathematics, University of Campinas
Gay, D.M. [1975]: Brown’s method and some generalizations with applications to minimization problems, Ph D Thesis, Computer Science Department, Cornell University, Ithaca, New York
Gay, D.M. [1979]: Some convergence properties of Broy den’s method, SIAM Journal on Numerical Analysis 16, pp. 623–630
Gay, D.M.; Schnabel, R.B. [1978]: Solving systems of nonlinear equations by Broy-den’s method with projected updates, in Nonlinear Programming 3, edited by O. Mangasarian, R. Meyer and S. Robinson, Academic Press, New York, pp. 245–281
George, A.; Ng, E. [1987]: Symbolic factorization for sparse Gaussian elimination with partial pivoting, SIAM Journal on Scientific and Statistical Computing 8, pp. 877–898
Golub, G.H.; Van Loan, Ch.F. [1989]: Matrix Computations, The Johns Hopkins University Press, Baltimore and London.
Gomes-Ruggiero, M.A.; Martinez, J.M. [1992]: The Column-Updating Method for solving nonlinear equations in Hilbert space, Mathematical Modelling and Numerical Analysis 26, pp 309–330
Gomes-Ruggiero, M.A.; Martinez, J.M.; Moretti, A.C. [1992]: Comparing algorithms for solving sparse nonlinear systems of equations, SIAM Journal on Scientific and Statistical Computing 13, pp. 459–483
Gragg, W.B.; Stewart, G.W. [1976]: A stable variant of the secant method for solving nonlinear equations, SIAM Journal on Numerical Analysis 13, pp. 127–140
Griewank, A. [1986]: The solution of boundary value problems by Broyden based secant methods, CTAC-85. Proceedings of the Computational Techniques and Applications Conference, J. Noye and R. May (eds.), North Holland, Amsterdam
Griewank, A. [1992]: Achieving Logarithmic Growth of Temporal and Spacial Complexity in Reverse Automatic Differentiation, Optimization Methods and Software 1, pp. 35–54
Griewank, A.; Toint, Ph.L. [1982a]: On the unconstrained optimization of partially separable functions, in Nonlinear Optimization 1981, edited by M.J.D. Powell, Academic Press, New York
Griewank, A.; Toint, Ph.L. [1982b]: Partitioned variable metric for large structured optimization problems, Numerische Mathematik 39, pp. 119–137
Griewank, A.; Toint, Ph.L. [1982c]: Local convergence analysis for partitioned quasi-Newton updates, Numerische Mathematik 39, pp. 429–448
Griewank, A.; Toint, Ph.L. [1984]: Numerical experiments with partially separable optimization problems, in Numerical Analysis Proceedings Dundee 1983, edited by D.F. Griffiths, Lecture Notes in Mathematics vol. 1066, Springer-Verlag, Berlin, pp. 203–220
Grippo, L.; Lampariello, F.; Lucidi, S. [1986]: A nonmonotone line search technique for Newton’s method, SIAM Journal on Numerical Analysis 23, pp. 707 – 716
Hart, W.E.; Soul, S.O.W. [1973]: Quasi-Newton methods for discretized nonlinear boundary value problems, J. Inst. Math. Applies. 11, pp. 351–359
Hestenes, M.R.; Stiefel, E. [1952]: Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards B49, pp. 409–436
Hoyer, W. [1987]: Quadratically convergent decomposition algorithms for nonlinear systems with special structure, Inform. Tech. Univ. Dresden 07, pp. 23–87
Hoyer, W.; Schmidt, J.W. [1984]: Newton-type decomposition methods for equations arising in network analysis, Z. Angew-Math. Mech. 64, pp. 397–405
Hoyer, W.; Schmidt, J.W.; Shabani, N. [1989]: Superlinearly convergent decompositon methods for block-tridiagonal nonlinear systems of equations, Numerical Functional Analysis and Optimization 10 (9 & 10), pp. 961–975
Iri, M. [1984]: Simultaneous computations of functions, partial derivatives and estimates of rounding errors. Complexity and Practicality, Japan Journal of Applied Mathematics 1, pp. 223–252
Johnson, G.W.; Austria, N.H. [1983]: A quasi-Newton method employing direct secant updates of matrix factorizations, SIAM Journal on Numerical Analysis 20, pp. 315–325
Kaczmarz, S. [1937]: Angenaherte Auflösung von Systemen linearer Gleichungen, Bull. Acad. Polon. Sei. Lett. A35, pp. 355–357
Kelley, C.T.; Sachs, E.W. [1987]: A quasi-Newton method for elliptic boundary value problems, SIAM Journal on Numerical Analysis 24, pp. 516–531
Lahaye, E. [1934]: Une méthode de résolution d’une catégorie d’equations transcendantes, Comptes Rendus Acad. Sci. Paris 198, pp. 1840–1842
Lopes, T.L.; Martínez, J.M. [1980]: Combination of the Sequential Secant Method and Broyden’s method with projected updates, Computing 25, pp. 379–386
Martínez, J.M. [1979a]: Three new algorithms based on the sequential secant method, BIT 19, pp. 236–243
Martínez, J.M. [1979b]: On the order of convergence of Broyden-Gay-Schnabel’s method, Commentationes Mathematicae Universitatis Carolinae 19, pp. 107–118
Martiínez, J.M. [1979c]: Generalization of the methods of Brent and Brown for solving nonlinear simultaneous equations, SIAM Journal on Numerical Analysis 16, pp. 434–448
Martínez, J.M. [1980]: Solving nonlinear simultaneous equations with a generalization of Brent’s method, BIT 20, pp. 501–510
Martínez, J.M. [1983]: A quasi-Newton method with a new updating for the LDU factorization of the approximate Jacobian, Matemática Aplicada e Computacional 2, pp. 131–142
Martínez, J.M. [1984]: A quasi-Newton method with modification of one column per iteration, Computing 33, pp. 353–362
Martínez, J.M. [1986a]: The method of Successive Orthogonal Projections for solving nonlinear simultaneous equations, Calcolo 23, pp. 93–105
Martínez, J.M. [1986b]: Solving systems of nonlinear simultaneous equations by means of an accelerated Successive Orthogonal Projections Method, Computational and Applied Mathematics 165, pp. 169–179
Martínez, J.M. [1986c]: Solution of nonlinear systems of equations by an optimal projection method, Computing 37, pp. 59–70
Martínez, J.M. [1987]: Quasi-Newton Methods with Factorization Scaling for Solving Sparse Nonlinear Systems of Equations, Computing 38, pp. 133–141
Martínez, J.M. [1990a]: A family of quasi-Newton methods for nonlinear equations with direct secant updates of matrix factorizations, SIAM Journal on Numerical Analysis 27, pp. 1034–1049
Martínez, J.M. [1990b]: Local convergence theory of inexact Newton methods based on structured least change updates, Mathematics of Computation 55, pp. 143–168
Martínez, J.M. [1991]: Quasi-Newton Methods for Solving Underdetermined Nonlinear Simultaneous Equations, Journal of Computational and Applied Mathematics 34, pp. 171–190
Martínez, J.M. [1992a]: On the relation between two local convergence theories of least change secant update methods, Mathematics of Computation 59, pp. 457–481
Martínez, J.M. [1992b]: A Theory of Secant Preconditioners, to appear in Mathematics of Computation
Martínez, J.M. [1992c]: On the Convergence of the Column-Updating Methods, Technical Report, Department of Applied Mathematics, University of Campinas
Martínez, J.M. [1992d]: Fixed-Point Quasi-Newton Methods, SIAM Journal on Numerical Analysis 29, pp. 1413–1434
Martínez, J.M. [1992e]: SOR-Secant Methods, to appear in SIAM Journal on Numerical Analysis
Martínez, J.M.; Zambaldi, M.C. [1992]: An inverse Column-Updating Method for solving Large-Scale Nonlinear Systems of Equations, to appear in Optimization Methods and Software
Matthies, H.; Strang, G. [1979]: The solution of nonlinear finite element equations, International Journal of Numerical Methods in Engineering 14, pp. 1613–1626
Mc Cormick, S.F. [1977]: The method of Kaczmarz and row-orthogonalization for solving linear equations and least-squares problems in Hilbert space, Indiana University Mathematical Journal 26, pp. 1137–1150.
Meyn, K-H [1983]: Solution of underdetermined nonlinear equations by stationary iteration, Numerische Mathematik 42, pp. 161–172
Milnor, J.W. [1969]: Topology from the differential viewpoint, The University Press of Virginia, Charlottesville, Virginia
Moré, J.J. [1989]: A collection of nonlinear model problems, Preprint MCS-P60–0289, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, Illinois
Nazareth, L.; Nocedal, J. [1978]: A study of conjugate gradient methods, Report SOL 78–29, Department of Operations Research, Stanford University
Nash, S.G. [1985]: Preconditioning of Truncated Newton methods, SIAM Journal on Scientific and Statistical Computing 6, pp. 599 -616
Ortega, J.M.; Rheinboldt, W.G. [1970]: Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, NY
Ostrowski, A.M. [1973]: Solution of Equations in Euclidean and Banach Spaces, Academic Press, New York.
Rail, L.B. [1984]: Differentiation in PASCAL-SC: Type Gradient, ACM Transactions on Mathematical Software 10, pp. 161–184.
Rail, L.B. [1987]: Optimal Implementation of Differentiation Arithmetic, in Computer Arithmetic, Scientific Computation and Programming Languages, U. Külisch (ed.), Teubner, Stuttgart
Rheinboldt, W.C. [1986]: Numerical Analysis of Parametrized Nonlinear Equations, J. Wiley, Interscience, New York
Saad, Y., Schultz, M.H. [1986]: GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM Journal on Numerical Analysis 7, pp. 856–869
Schmidt, J.W. [1987]: A class of superlinear decomposition methods in nonlinear equations, Numerical Functional Analysis and Optimization 9, pp. 629–645
Schmidt J.W., Hoyer, W., Haufe, Ch. [1985]: Consistent approximations in Newton-type decomposition methods, Numerische Mathematik 47, pp. 413–425
Schnabel, R.B., Frank, P.D. [1984]: Tensor methods for nonlinear equations, SIAM Journal on Numerical Analysis 21, pp. 815–843
Shamanskii, V.E. [1967]: A modification of Newton’s method, Ukrain Mat. Z. 19, pp. 133–138
Schwandt, H. [1984]: An interval arithmetic approach for the construction of an almost globally convergent method for the solution of the nonlinear Poisson equation on the unit square, SIAM Journal on Scientific and Statistical Computing 5, pp. 427–452
Schwetlick, H. [1978]: Numerische Losung Nichtlinearer Gleichungen, Deutscher Verlag der Wissenschaften, Berlin
Spedicato, E. [1991] (editor): Computer Algorithms for Solving Linear Algebraic Equations. The State of Art, NATO ASI Series, Series F: Computer and Systems Sciences, Vol. 77, Springer Verlag, Berlin
Spedicato, E.; Chen, Z.; Deng, N. [1992]: A class of difference ABS-type algorithms for a nonlinear system of equations, Technical Report, Department of Mathematics, University of Bergamo
Tewarson, R.P. [1988]: A new Quasi-Newton algorithm, Applied Mathematics Letters 1, pp. 101–104
Tewarson, R.P.; Zhang, Y. [1987]: Sparse Quasi-Newton LDU update, International Journal on Numerical Methods in Engineering 24, pp. 1093–1100
Tompkins, C. [1955]: Projection methods in calculation, Proceedings Second Symposium on Linear Programming, Washington D.C., pp. 425–448
Walker, H.F.; Watson, L.T. [1989]: Least-Change Update Methods for underdetermined systems, Research Report, Department of Mathematics, Utah State University
Watson, L.T. [1979]: An algorithm that is globally convergent with probability one for a class of nonlinear two-point boundary value problems, SIAM Journal on Numerical Analysis 16, pp. 394–401
Watson, L.T. [1980]: Solving finite difference approximations to nonlinear two-point boundary value problems by a homotopy method, SIAM Journal on Scientific and Statistical Computing 1, pp. 467–480
Toint, Ph.L. [1986]: Numerical solution of large sets of algebraic nonlinear equations, Mathematics of Computation 16, pp. 175–189
Watson, L.T. [1983]: Engineering applications of the Chow-Yorke algorithm, in Homotopy Methods and Global Convergence (B.C. Eaves and M.J. Todd eds.), Plenum, New York
Watson, L.T.; Billups, S.C.; Morgan, A.P. [1987]: Algorithm 652: HOMPACK: A suite of codes for globally convergent homotopy algorithms, ACM Trans. Math. Software 13, pp. 281–310
Watson, L.T.; Scott, M.R. [1987]: Solving spline-collocation approximations to nonlinear two-point boundary value problems by a homotopy method, Applied Mathematics and Computation 24, pp. 333–357
Watson, L.T.; Wang, C.Y. [1981]: A homotopy method applied to elastica problems, International Journal on Solid Structures 17, pp. 29–37
Wolfe P. [1959]: The secant method for solving nonlinear equations, Communications ACM 12, pp. 12–13
Zambaldi, M.C. [1990]: Estruturas estáticas e dinamicas para resolver sistemas não lineares esparsos, Tese de Mestrado, Departamento de Matemática Aplicada, Universidade Estadual de Campinas, Campinas, Brazil
Zlatev, Z.; Wasniewski, J.; Schaumburg, K. [1981]: Y12M. Solution of large and sparse systems of linear algebraic equations, Lecture Notes in Computer Science 121, Springer-Verlag, New York, Berlin, Heidelberg and Tokyo
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1994 Kluwer Academic Publishers
About this chapter
Cite this chapter
Martínez, J.M. (1994). Algorithms for Solving Nonlinear Systems of Equations. In: Spedicato, E. (eds) Algorithms for Continuous Optimization. NATO ASI Series, vol 434. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-0369-2_4
Download citation
DOI: https://doi.org/10.1007/978-94-009-0369-2_4
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-6652-5
Online ISBN: 978-94-009-0369-2
eBook Packages: Springer Book Archive