Abstract
Recent theoretical and practical investigations have shown that the Gauss-Newton algorithm is the method of choice for the numerical solution of nonlinear least squares parameter estimation problems. It is shown that when line searches are included, the Gauss-Newton algorithm behaves asymptotically like steepest descent, for a special choice of parameterization. Based on this a conjugate gradient acceleration is developed. It converges fast also for those large residual problems, where the original Gauss-Newton algorithm has a slow rate of convergence. Several numerical test examples are reported, verifying the applicability of the theory.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
O. Axelsson,Solution of linear systems of equations: iterative methods, pp. 1–51 of Sparse Matrix Techniques ed. by V. A. Barker, Lecture Notes in Mathematics 572, Springer-Verlag Berlin - Heidelberg - New York 1977.
P. Baptist and J. Stoer,On the relation between quadratic termination and convergence properties of minimization algorithms, Part I.Theory, Num. Math. 28, 343–366 (1977), Part II,Applications, Num. Math. 28, 367–391 (1977).
Y. Bard,Nonlinear parameter estimation, Acad. Press, New York 1974.
J. E. Dennis,Some computational techniques for the nonlinear least squares problem, pp. 157–183 inNumerical solution of systems of nonlinear algebraic equations, edited by G. D. Byrne and C. A. Hall, Acad. Press New York 1973.
G. E. Forsythe,On the asymptotic directions of the s-dimensional optimum gradient method, Num. Math. 11, 57–76 (1968).
A. S. Householder,The theory of matrices in numerical analysis, Blaisdell, New York 1964.
D. R. Kincaid and D. M. Young,Survey of iterative methods, to appear in Encyclopedia of computer science and technology. Marcel Dekker, New York.
D. G. Luenberger,Introduction to linear and nonlinear programming, Addison-Wesley, Reading Mass. 1973.
H. Ramsin and P. Å. Wedin,A comparison of some algorithms for the non-linear least squares problem, BIT 17, 72–90 (1977).
P. Å. Wedin,Perturbation theory for pseudoinverses, BIT 13, 217–232 (1973).
P. Å. Wedin,On the Gauss-Newton method for the non-linear least squares problem, ITM working paper 24, Inst. f. Tillämpad Matematik, Stockholm 1974.
A. Ruhe,Accelerated Gauss-Newton algorithms for nonlinear least squares problems, Tech Rep UMINF-67.78, Dept. of Information processing, Umeå 1978.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Ruhe, A. Accelerated Gauss-Newton algorithms for nonlinear least squares problems. BIT 19, 356–367 (1979). https://doi.org/10.1007/BF01930989
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01930989