Abstract
In this paper, the classical Gauss-Newton method for the unconstrained least squares problem is modified by introducing a quasi-Newton approximation to the second-order term of the Hessian. Various quasi-Newton formulas are considered, and numerical experiments show that most of them are more efficient on large residual problems than the Gauss-Newton method and a general purpose minimization algorithm based upon the BFGS formula. A particular quasi-Newton formula is shown numerically to be superior. Further improvements are obtained by using a line search that exploits the special form of the function.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Greenstadt, J.,Variations on Variable Metric Methods, Mathematics of Computation, Vol. 24, pp. 1–18, 1970.
Oren, S. S.,Self-Scaling Variable Metric Methods for Unconstrained Minimization, Stanford University, PhD Thesis, 1972.
McKeown, J. J.,On Algorithms for Sums of Squares Problems, Toward Global Optimization, Edited by L. C. W. Dixon and G. P. Szegö, North-Holland, Amsterdam, Holland, 1975.
Meyer, R. R.,Theoretical and Computational Aspects of Nonlinear Regression, Nonlinear Programming, Edited by J. Rosen, O. Mangasarian, and K. Ritter, Academic Press, New York, New York, 1970.
Spedicato, E.,Parameter Estimation and Least Squares, Numerical Techniques for Stochastic Systems, Edited by F. Archetti and M. Cugiani, North-Holland, Amsterdam, Holland, 1980.
Al-Baali, M., andFletcher, R.,Variational Methods for Nonlinear Least Squares, Journal of the Operational Research Society, Vol. 36, pp. 405–421, 1985.
Dennis, J.,Some Computational Techniques for the Nonlinear Least Squares Problem, Numerical Solution of Nonlinear Algebraic Equations, Edited by G. D. Byrne and C. A. Hall, Academic Press, New York, New York, 1974.
Betts, J. T.,Solving the Nonlinear Least Squares Problem: Application of a General Method, Journal of Optimization Theory and Applications, Vol. 18, pp. 469–483, 1976.
Biggs, M. C.,The Estimation of the Hessian Matrix in Nonlinear Least Squares Problems with Nonzero Residuals, Mathematical Programming, Vol. 12, pp. 67–80, 1977.
Dennis, J., Gay, D., andWelsch, R.,An Adaptive Nonlinear Least Squares Algorithm, ACM Transactions on Mathematical Software, Vol. 7, pp. 348–368, 1981.
Oren, S. S., andSpedicato, E.,Optimal Conditioning of Self-Scaling Variable Metric Algorithms, Mathematical Programming, Vol. 10, pp. 70–90, 1976.
Shanno, D., andPhua, K.,Numerical Comparison of Several Variable Metric Methods, University of Arizona, Management and Information Sciences Report No. 22, 1977.
Lindstrom, P., andWedin, P.,A New Line Search Algorithm for Nonlinear Least Squares Problems, University of Umea, Institute of Information Processing, Report No. 82, 1981.
Abaffy, J., andSpedicato, E.,A Code for General Linear Systems, Consiglio Nazionale delle Richerche, Istituto per le Applicazioni del Calcolo, Monografia di Software Matematico No. 21, 1983.
Vespucci, M. T.,A Modified Gauss-Newton Code for Large Residual Least Squares, Istituto Universitario di Bergamo, Dipartimento di Matematica, Statistica, Informatica e Applicazioni, Quaderno No. 4, 1986.
Author information
Authors and Affiliations
Additional information
Communicated by L. C. W. Dixon
Rights and permissions
About this article
Cite this article
Spedicato, E., Vespucci, M.T. Numerical experiments with variations of the Gauss-Newton algorithm for nonlinear least squares. J Optim Theory Appl 57, 323–339 (1988). https://doi.org/10.1007/BF00938543
Issue Date:
DOI: https://doi.org/10.1007/BF00938543