Journal of Optimization Theory and Applications

, Volume 18, Issue 2, pp 187–197 | Cite as

New least-square algorithms

  • W. C. Davidon
Contributed Papers


New algorithms are presented for approximating the minimum of the sum of squares ofM real and differentiable functions over anN-dimensional space. These algorithms update estimates for the location of a minimum after each one of the functions and its first derivatives are evaluated, in contrast with other least-square algorithms which evaluate allM functions and their derivatives at one point before using any of this information to make an update. These new algorithms give estimates which fluctuate about a minimum rather than converging to it. For many least-square problems, they give an adequate approximation for the solution more quickly than do other algorithms.

Key Words

Least-square methods variable-metric methods gradient methods nonlinear programming 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Raphson, J.,Analysis Aequationum Universalis, London, England, 1690.Google Scholar
  2. 2.
    Gauss, K. F.,Theoria Motus Corporum Coelestium, Werke 7, pp. 240–254, Königlichen Gesellschaft der Wissenschaften, Goettingen, Germany, 1809.Google Scholar
  3. 3.
    Levenberg, K.,A Method for the Solution of Certain Non-linear Problems in Least Squares, Quarterly of Applied Mathematics, Vol. 2, pp. 164–168, 1944.Google Scholar
  4. 4.
    Marquardt, D. W.,An Algorithm for Least Squares Estimation, SIAM Journal on Applied Mathematics, Vol. 11, pp. 431–441, 1963.Google Scholar
  5. 5.
    Parker, F. D.,Inverses of Vandermonde Matrices, American Mathematical Monthly, Vol. 71, pp. 410–411, 1964.Google Scholar
  6. 6.
    Box, M. J.,A Comparison of Several Current Optimization Methods, Computer Journal, Vol. 9, pp. 67–77, 1966.Google Scholar
  7. 7.
    Brown, K., andDennis, J.,Derivative-Free Analogues of the Levenberg-Marquardt and Gauss Algorithms for Non-linear Least Squares Approximation, IBM Philadelphia Scientific Center, Technical Report No. 320-2994, 1970.Google Scholar
  8. 8.
    Fletcher, R., andPowell, M. J. D.,A Rapidly Convergent Descent Method for Minimization, Computer Journal, Vol. 6, pp. 163–168, 1963.Google Scholar
  9. 9.
    Brown, K., andDennis, J.,New Computational Algorithms for Minimizing a Sum of Squares of Non-linear Functions, Yale University, Department of Computer Science, Research Report No. 71-6, 1971.Google Scholar
  10. 10.
    Cheswick, J., Davidon, W., andSchoen, C.,Application of a New Least Squares Method to Structure Refinement, Acta Crystallographica (to appear).Google Scholar

Copyright information

© Plenum Publishing Corporation 1976

Authors and Affiliations

  • W. C. Davidon
    • 1
  1. 1.Haverford CollegeHaverford

Personalised recommendations