Abstract
Computational experiments by McKeown [11] have shown that specialised methods, based on the Gauss—Newton iteration, are not necessarily the best choice for minimising functions that are sums of squared terms. Difficulties arise when the Gauss—Newton approach does not yield a good approximation to the second derivative matrix of the function: and this is more likely to happen when the function value at the optimum is not near zero and the terms in the sum of squares are significantly nonlinear. This paper considers some specialised methods for the nonlinear least squares problem which seek to improve the Gauss—Newton estimate of the Hessian matrix without explicitly calculating second derivatives.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
M.C. Biggs, “A note on minimisation algorithms which make use of nonquadratic properties of the objective function”,Journal of the Institute of Mathematics and its Applications 12 (1973) 337.
M.C. Biggs, “Some improvements to the OPTIMA subroutines”, Numerical Optimisation Centre Technical Report No. 69, The Hatfield Polytechnic, Hertfordshire (1975).
K.M. Brown and J.E. Dennis, “New computational algorithms for minimising a sum of squares of nonlinear functions”, Computer Science Department Report 71-6, Yale University New Haven, CT (1971).
R. Coleman, British Aircraft Corporation, private communication.
J.E. Dennis, “Some computational techniques for the nonlinear least squares problem”, in: G.D. Byrne and C.A. Hall, eds.,Numerical solution of nonlinear algebraic equations (Academic Press, New York, 1974).
L.C.W. Dixon, “The behaviour of variable metric algorithms when applied to three illconditioned nonquadratic functions”, Numerical Optimisation Centre Technical Report No. 32, The Hatfield Polytechnic, Hertfordshire (1971).
L.C.W. Dixon, “Nonlinear optimisation; a survey of the state of the art”, in: D.J. Evans, ed.,Software for numerical mathematics (Academic Press, New York, 1974).
R. Fletcher, “A modified Marquardt subroutine for nonlinear least squares”, UKAEA Research Report AERE R6799 (1971).
S.E. Hersom and F.H. Reynolds, “An application of numerical optimisation to the analysis of accelerated life test data”, presented at the conference on the use of digital computers in measurement, York University (1973).
J.J. McKeown, “A comparison of methods for solving parameter estimation problems”, Paper PV-9, 3rd IFAC symposium on identification and system parameter estimation, The Hague (1973).
J.J. McKeown, “Specialised versus general purpose algorithms for minimising functions that are sums of squared terms”,Mathematical Programming 9 (1975) 57.
J.J. McKeown, “On the choice of algorithms for sums of squares problems”, in: L.C.W. Dixon and G.P. Szego, eds.,Towards global optimisation (North-Holland, Amsterdam, 1975).
R.R. Meyer, “Theoretical and computational aspects of nonlinear regression”, in: J. Rosen, O. Mangasarian and K. Ritter, eds.,Nonlinear programming (Academic Press, New York, 1970).
M.J.D. Powell, “A new algorithm for unconstrained optimisation”, in: J. Rosen, O. Mangasarian and K. Ritter, eds.,Nonlinear Programming (Academic Press, New York, 1970).
M.J.D. Powell, “VA05A”, AERE Harwell subroutine library (1969).
P. Wolfe, “Another variable metric method”, IBM Working Paper (1967).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bartholomew-Biggs, M.C. The estimation of the hessian matrix in nonlinear least squares problems with non-zero residuals. Mathematical Programming 12, 67–80 (1977). https://doi.org/10.1007/BF01593770
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01593770