Numerical Algorithms

, Volume 4, Issue 2, pp 241–262 | Cite as

A parallel projection method for overdetermined nonlinear systems of equations

  • Maria A. Diniz-Ehrhardt
  • José Mario Martínez


We consider overdetermined nonlinear systems of equationsF(x)=0, whereF: ℝ n → ℝ m ,m≥n. For this type of systems we define “weighted least square distance” (WLSD) solutions, which represent an alternative to classical least squares solutions and to other solutions based on residual normas. We introduce a generalization of the classical method of Cimmino for linear systems and we prove local convergence results. We introduce a practical strategy for improving the global convergence properties of the method. Finally, numerical experiments are presented.


Linear System Numerical Experiment Nonlinear System Classical Method Projection Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    H.T. Banks and K. Kunisch,Estimation Techniques for Distributed Parameter Systems (Birkhäuser, Boston, Basel and Berlin, 1989).Google Scholar
  2. [2]
    A. Ben Israel: A Newton-Raphson method for the solution of systems of equations, J. Math. Anal. Appl. 15 (1966) 243–252.CrossRefGoogle Scholar
  3. [3]
    T.N. Bishop et al., Tomographic determination of velocity and depth in laterally varying media, Geophysics 50 (1985) 903–923.CrossRefGoogle Scholar
  4. [4]
    Å. Björck, Least squares methods, inHandbook of Numerical Analysis vol. 1, eds. P. G. Ciarlet and J.L. Lions (Elsevier/North-Holland, 1987).Google Scholar
  5. [5]
    C.G. Broyden, J.E. Dennis Jr. and J.J. Moré, On the local and superlinear convergence of quasi-Newton methods, J. Inst. Math. Appl. 12 (1973) 223–245.Google Scholar
  6. [6]
    Y. Censor, Row-action methods for huge and sparse systems and their applications, SIAM Rev. 23 (1981) 444–466.CrossRefGoogle Scholar
  7. [7]
    Y. Censor, D.E. Gustafson, A. Lent and H. Tuy, A new approach to the emission computerized tomography problem: simultaneous calculation of attenuation and activity coefficients, IEEE Trans. Nucl. Sci. NS-26 (1979) 2775–2779.Google Scholar
  8. [8]
    G. Cimmino, Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari, La Ricerca Scientifica Ser II, Anno IV 1 (1938) 326–333.Google Scholar
  9. [9]
    J.E. Dennis Jr. and R.B. Schnabel,Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Prentice-Hall, Englewood Cliffs, NJ, 1983).Google Scholar
  10. [10]
    A.R. De Pierro and A.N. Iusem, On the set of weighted least squares solutions of systems of convex inequalities, Comm. Math. Univ. Carol. 25 (1984) 667–678.Google Scholar
  11. [11]
    A.R. De Pierro and A.N. Iusem, A parallel projection method of finding a common point of a family of convex sets, Pesquisa Operacional 5 (1985) 1–20.Google Scholar
  12. [12]
    A.R. De Pierro and A.N. Iusem, A simultaneous projections method for linear inequalities, Lin. Alg. Appl. 64 (1985) 243–253.CrossRefGoogle Scholar
  13. [13]
    I.S. Duff, MA28—a set of FORTRAN subroutines for sparse unsymmetric linear equations, AERE R8730 (HMSO, London, 1977).Google Scholar
  14. [14]
    I.S. Duff, A.M. Erisman and J.K. Reid,Direct Methods for Sparse Matrices (Clarendon Press, Oxford, 1986).Google Scholar
  15. [15]
    A. George and M.T. Health, Solution of sparse linear least squares using Givens rotations, Lin. Alg. Appl. 34 (1980) 69–83.CrossRefGoogle Scholar
  16. [16]
    A. George, M. T. Heath and E. Ng, A comparison of some methods for solving sparse leastsquares problems, SIAM J. Sci. Stat. Comput. 4 (1983) 177–187.CrossRefGoogle Scholar
  17. [17]
    J.A. George and J.W. Liu,Computer Solution of Large Sparse Positive Definite Systems (Prentice-Hall, Englewood Cliffs, NJ, 1981).Google Scholar
  18. [18]
    G.H. Golub and Ch.F. Van Loan,Matrix Computations (The Johns Hopkins University Press, Baltimore and London, 1989).Google Scholar
  19. [19]
    M.A. Gomes-Ruggiero, J.M. Martínez and A.C. Moretti, Comparing algorithms for solving sparse nonlinear systems of equations, SIAM J. Sci. Stat. Comput. 13 (1992) 459–483.CrossRefGoogle Scholar
  20. [20]
    M.T. Heath, Numerical methods for large sparse linear least squares problems, SIAM J. Sci. Stat. Comput. 5 (1984) 497–513.CrossRefGoogle Scholar
  21. [21]
    L.W.B. Leite and J.W.D. Leão, Ridge regression applied to the inversion of two-dimensional aeromagnetic anomalies, Geophysics 50 (1985) 1294–1306.CrossRefGoogle Scholar
  22. [22]
    K. Levenberg, A method for the solution of certain problems in least squares, Quart. J. Appl. Math. 2 (1944) 164–168.Google Scholar
  23. [23]
    D. Marquardt, An algorithm for least-squares estimation of nonlinear parameters, SIMA J. Appl. Math. 11 (1963) 431–441.CrossRefGoogle Scholar
  24. [24]
    J.M. Martínez, An algorithm for solving sparse nonlinear least squares problems, Computing 39 (1987) 307–325.Google Scholar
  25. [25]
    J.M. Martínez, Quasi-Newton methods for solving underdetermined nonlinear simultaneous equations, J. Comput. Appl. Math. 34 (1991) 171–190.CrossRefGoogle Scholar
  26. [26]
    J.M. Martínez and R.J. Sampaio, Parallel and sequential Kaczmarz methods for solving underdetermined nonlinear equations, J. Comp. Appl. Math. 15 (1986) 311–321.CrossRefGoogle Scholar
  27. [27]
    J.M. Martínez and R.F. Santos, An algorithm for solving nonlinear least-squares problems with a new curvilinear search, Computing 44 (1990) 83–90.Google Scholar
  28. [28]
    J.J. Moré, The Levenberg-Marquardt algorithm: implementation and theory, in:Numerical Analysis, Dundee 1977, Lecture Notes in Mathematics 630, ed. G.A. Watson (Springer, Berlin, 1978) pp. 105–116.Google Scholar
  29. [29]
    J.J. Moré, Recent developments in algorithms and software for trust region methods, in:Mathematical Programming Bonn 1982-The State of Art, eds. A. Bachem, M. Grötschel and B. Korte (Springer, 1983).Google Scholar
  30. [30]
    L.B. Pedersen, Interpretation of potential field data. A generalized inverse approach, Geophys. Prosp. 25 (1977) 199–230.Google Scholar
  31. [31]
    V. Pereyra, Numerical methods for inverse problems in 3-D geophysical modeling, Appl. Numer. Math. 4 (1988) 97–139.CrossRefGoogle Scholar
  32. [32]
    V. Pereyra and S.J. Wright, Three-dimensional inversion of travel time data for structurally complex geology, Preprint MCS P174 0890, Mathematics and Computer Science Division, Argonne National Laboratory (1990).Google Scholar
  33. [33]
    R.J. Santos, Métodos iterativos para problemas inversos, Ph.D. Thesis, Department of Applied Mathematics, University of Campinas (1992).Google Scholar
  34. [34]
    H.F. Walker and L.T. Watson, Least-change secant update methods for underdetermined systems, SIAM J. Numer. Anal. 27 (1990) 1227–1262.CrossRefGoogle Scholar

Copyright information

© J.C. Baltzer AG Science Publishers 1993

Authors and Affiliations

  • Maria A. Diniz-Ehrhardt
    • 1
  • José Mario Martínez
    • 1
  1. 1.Department of Applied Mathematics, IMECC-UNICAMPState University of CampinasCampinasBrazil

Personalised recommendations