Advertisement

Mathematical Programming

, Volume 12, Issue 1, pp 67–80 | Cite as

The estimation of the hessian matrix in nonlinear least squares problems with non-zero residuals

  • M. C. Bartholomew-Biggs
Article

Abstract

Computational experiments by McKeown [11] have shown that specialised methods, based on the Gauss—Newton iteration, are not necessarily the best choice for minimising functions that are sums of squared terms. Difficulties arise when the Gauss—Newton approach does not yield a good approximation to the second derivative matrix of the function: and this is more likely to happen when the function value at the optimum is not near zero and the terms in the sum of squares are significantly nonlinear. This paper considers some specialised methods for the nonlinear least squares problem which seek to improve the Gauss—Newton estimate of the Hessian matrix without explicitly calculating second derivatives.

Keywords

Good Choice Mathematical Method Computational Experiment Hessian Matrix Newton Iteration 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    M.C. Biggs, “A note on minimisation algorithms which make use of nonquadratic properties of the objective function”,Journal of the Institute of Mathematics and its Applications 12 (1973) 337.Google Scholar
  2. [2]
    M.C. Biggs, “Some improvements to the OPTIMA subroutines”, Numerical Optimisation Centre Technical Report No. 69, The Hatfield Polytechnic, Hertfordshire (1975).Google Scholar
  3. [3]
    K.M. Brown and J.E. Dennis, “New computational algorithms for minimising a sum of squares of nonlinear functions”, Computer Science Department Report 71-6, Yale University New Haven, CT (1971).Google Scholar
  4. [4]
    R. Coleman, British Aircraft Corporation, private communication.Google Scholar
  5. [5]
    J.E. Dennis, “Some computational techniques for the nonlinear least squares problem”, in: G.D. Byrne and C.A. Hall, eds.,Numerical solution of nonlinear algebraic equations (Academic Press, New York, 1974).Google Scholar
  6. [6]
    L.C.W. Dixon, “The behaviour of variable metric algorithms when applied to three illconditioned nonquadratic functions”, Numerical Optimisation Centre Technical Report No. 32, The Hatfield Polytechnic, Hertfordshire (1971).Google Scholar
  7. [7]
    L.C.W. Dixon, “Nonlinear optimisation; a survey of the state of the art”, in: D.J. Evans, ed.,Software for numerical mathematics (Academic Press, New York, 1974).Google Scholar
  8. [8]
    R. Fletcher, “A modified Marquardt subroutine for nonlinear least squares”, UKAEA Research Report AERE R6799 (1971).Google Scholar
  9. [9]
    S.E. Hersom and F.H. Reynolds, “An application of numerical optimisation to the analysis of accelerated life test data”, presented at the conference on the use of digital computers in measurement, York University (1973).Google Scholar
  10. [10]
    J.J. McKeown, “A comparison of methods for solving parameter estimation problems”, Paper PV-9, 3rd IFAC symposium on identification and system parameter estimation, The Hague (1973).Google Scholar
  11. [11]
    J.J. McKeown, “Specialised versus general purpose algorithms for minimising functions that are sums of squared terms”,Mathematical Programming 9 (1975) 57.Google Scholar
  12. [12]
    J.J. McKeown, “On the choice of algorithms for sums of squares problems”, in: L.C.W. Dixon and G.P. Szego, eds.,Towards global optimisation (North-Holland, Amsterdam, 1975).Google Scholar
  13. [13]
    R.R. Meyer, “Theoretical and computational aspects of nonlinear regression”, in: J. Rosen, O. Mangasarian and K. Ritter, eds.,Nonlinear programming (Academic Press, New York, 1970).Google Scholar
  14. [14]
    M.J.D. Powell, “A new algorithm for unconstrained optimisation”, in: J. Rosen, O. Mangasarian and K. Ritter, eds.,Nonlinear Programming (Academic Press, New York, 1970).Google Scholar
  15. [15]
    M.J.D. Powell, “VA05A”, AERE Harwell subroutine library (1969).Google Scholar
  16. [16]
    P. Wolfe, “Another variable metric method”, IBM Working Paper (1967).Google Scholar

Copyright information

© The Mathematical Programming Society 1977

Authors and Affiliations

  • M. C. Bartholomew-Biggs
    • 1
  1. 1.The Hatfield PolytechnicHatfieldUK

Personalised recommendations