Numerical experiments with variations of the Gauss-Newton algorithm for nonlinear least squares

  • E. Spedicato
  • M. T. Vespucci
Contributed Papers


In this paper, the classical Gauss-Newton method for the unconstrained least squares problem is modified by introducing a quasi-Newton approximation to the second-order term of the Hessian. Various quasi-Newton formulas are considered, and numerical experiments show that most of them are more efficient on large residual problems than the Gauss-Newton method and a general purpose minimization algorithm based upon the BFGS formula. A particular quasi-Newton formula is shown numerically to be superior. Further improvements are obtained by using a line search that exploits the special form of the function.

Key Words

Mathematical programming nonlinear programming nonlinear least squares quasi-Newton methods 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Greenstadt, J.,Variations on Variable Metric Methods, Mathematics of Computation, Vol. 24, pp. 1–18, 1970.Google Scholar
  2. 2.
    Oren, S. S.,Self-Scaling Variable Metric Methods for Unconstrained Minimization, Stanford University, PhD Thesis, 1972.Google Scholar
  3. 3.
    McKeown, J. J.,On Algorithms for Sums of Squares Problems, Toward Global Optimization, Edited by L. C. W. Dixon and G. P. Szegö, North-Holland, Amsterdam, Holland, 1975.Google Scholar
  4. 4.
    Meyer, R. R.,Theoretical and Computational Aspects of Nonlinear Regression, Nonlinear Programming, Edited by J. Rosen, O. Mangasarian, and K. Ritter, Academic Press, New York, New York, 1970.Google Scholar
  5. 5.
    Spedicato, E.,Parameter Estimation and Least Squares, Numerical Techniques for Stochastic Systems, Edited by F. Archetti and M. Cugiani, North-Holland, Amsterdam, Holland, 1980.Google Scholar
  6. 6.
    Al-Baali, M., andFletcher, R.,Variational Methods for Nonlinear Least Squares, Journal of the Operational Research Society, Vol. 36, pp. 405–421, 1985.Google Scholar
  7. 7.
    Dennis, J.,Some Computational Techniques for the Nonlinear Least Squares Problem, Numerical Solution of Nonlinear Algebraic Equations, Edited by G. D. Byrne and C. A. Hall, Academic Press, New York, New York, 1974.Google Scholar
  8. 8.
    Betts, J. T.,Solving the Nonlinear Least Squares Problem: Application of a General Method, Journal of Optimization Theory and Applications, Vol. 18, pp. 469–483, 1976.Google Scholar
  9. 9.
    Biggs, M. C.,The Estimation of the Hessian Matrix in Nonlinear Least Squares Problems with Nonzero Residuals, Mathematical Programming, Vol. 12, pp. 67–80, 1977.Google Scholar
  10. 10.
    Dennis, J., Gay, D., andWelsch, R.,An Adaptive Nonlinear Least Squares Algorithm, ACM Transactions on Mathematical Software, Vol. 7, pp. 348–368, 1981.Google Scholar
  11. 11.
    Oren, S. S., andSpedicato, E.,Optimal Conditioning of Self-Scaling Variable Metric Algorithms, Mathematical Programming, Vol. 10, pp. 70–90, 1976.Google Scholar
  12. 12.
    Shanno, D., andPhua, K.,Numerical Comparison of Several Variable Metric Methods, University of Arizona, Management and Information Sciences Report No. 22, 1977.Google Scholar
  13. 13.
    Lindstrom, P., andWedin, P.,A New Line Search Algorithm for Nonlinear Least Squares Problems, University of Umea, Institute of Information Processing, Report No. 82, 1981.Google Scholar
  14. 14.
    Abaffy, J., andSpedicato, E.,A Code for General Linear Systems, Consiglio Nazionale delle Richerche, Istituto per le Applicazioni del Calcolo, Monografia di Software Matematico No. 21, 1983.Google Scholar
  15. 15.
    Vespucci, M. T.,A Modified Gauss-Newton Code for Large Residual Least Squares, Istituto Universitario di Bergamo, Dipartimento di Matematica, Statistica, Informatica e Applicazioni, Quaderno No. 4, 1986.Google Scholar

Copyright information

© Plenum Publishing Corporation 1988

Authors and Affiliations

  • E. Spedicato
    • 1
  • M. T. Vespucci
    • 1
  1. 1.Department of MathematicsUniversity of BergamoBergamoItaly

Personalised recommendations