Skip to main content
SpringerLink
Log in

Recent advances in unconstrained optimization

  • Published:
Mathematical Programming Submit manuscript

Abstract

We survey the development of algorithms and theory for the unconstrained optimization problem during the years 1967–1970. Therefore (except for one remark) the material is taken from papers that have already been published. This exception is an explanation of some numerical difficulties that can occur when using Davidon's (1959) variable metric algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. D.J. Wilde and C.S. Beightler,Foundations of optimization (Prentice-Hall Inc., Englewood Cliffs, N.J., 1967).

    Google Scholar 

  2. J. Kowalik and M.R. Osborne,Methods for unconstrained optimization problems (Elsevier Publishing Co. Inc., New York, 1968).

    Google Scholar 

  3. M.J. Box, D. Davies and W.H. Swann,Nonlinear optimization techniques (Oliver and Boyd Ltd., Edinburgh, 1969).

    Google Scholar 

  4. M.J.D. Powell, “A survey of numerical methods for unconstrained optimization”,SIAM Review 12 (1970) 79–97.

    Google Scholar 

  5. H.Y. Huang, “Unified approach to quadratically convergent algorithms for function minimization”,J.O.T.A. 5 (1970) 405–423.

    Google Scholar 

  6. W.C. Davidon, “Variable metric method for minimization”, A.E.C. Research and Development Rep. ANL-5990 (Rev.) (1959).

  7. R. Fletcher and M.J.D. Powell, “A rapidly convergent descent method for minimization”,Computer Journal 6 (1963) 163–168.

    Google Scholar 

  8. J. Greenstadt, “Variations on variable metric methods”,Mathematics of Computation 24 (1970) 1–22.

    Google Scholar 

  9. M.J.D. Powell “A new algorithm for unconstrained optimization“, Report No. T.P. 393, A.E.R.E. Harwell (1970).

    Google Scholar 

  10. M.J.D. Powell, “An efficient method for finding the minimum of a function of several variables without calculating derivatives”,Computer Journal 7 (1964) 155–162

    Google Scholar 

  11. G.W. Stewart, “A modification of Davidon's minimization method to accept difference approximations of derivatives”,Journal of the A.C.M. 14 (1967) 72–83.

    Google Scholar 

  12. S.A. Lill. “A modified Davidon method for finding the minimum of a function using difference approximations for derivatives”,Computer Journal 13 (1970) 111–113.

    Google Scholar 

  13. R. Fletcher, “A review of methods for unconstrained optimization”, in:Optimization Ed. R. Fletcher (Academic Press, London, 1969) 1–12.

    Google Scholar 

  14. M.J. Box, “A comparison of several current optimization methods, and the use of transformations in constrained problems”,Computer Journal 9 (1966) 67–77.

    Google Scholar 

  15. G. Berman, “Lattice approximations to the minima of functions of several variables”,Journal of the A.C.M. 16 (1969) 286–294.

    Google Scholar 

  16. A.V. Fiacco and G.P. McCormick,Nonlinear programming: sequential unconstrained minimization techniques (John Wiley and Sons Inc., New York, 1968).

    Google Scholar 

  17. D.H. Winfield, “Function minimization without derivatives by a sequence of quadratic programming problems”, Technical Report No. 537, Engineering and Applied Physics Division, Harvard University (1967).

  18. W.I. Zangwill, “Minimizing a function without calculating derivatives”,Computer Journal 10 (1967) 293–296.

    Google Scholar 

  19. S.M. Goldfeld, R.E. Quandt and H.F. Trotter, “Maximization by improved quadratic hill-climbing and other methods”, Econometric Research Memo, No. 95, Princeton University (1968).

  20. J. Greenstadt, “On the relative efficiencies of gradient methods”,Mathematics of Computation 21 (1967) 360–367.

    Google Scholar 

  21. G.E. Myers, “Properties of the conjugate gradient and Davidon methods”,J.O.T.A. 2 (1968) 209–219.

    Google Scholar 

  22. C.G. Broyden, “The convergence of a class of double-rank minimization algorithms 1. General considerations”,J.I.M.A. 6 (1970) 76–90.

    Google Scholar 

  23. J.D. Pearson, “Variable metric methods of minimization”,Computer Journal 12 (1969) 171–178.

    Google Scholar 

  24. R. Fletcher and C.M. Reeves, “Function minimization by conjugate gradients”,Computer Journal 7 (1964) 149–154.

    Google Scholar 

  25. D. Goldfarb, “Sufficient conditions for the convergence of a variable metric algorithm”, in:Optimization, Ed. R. Fletcher (Academic Press, London, 1969) 273–281.

    Google Scholar 

  26. R. Fletcher, “A new approach to variable metric algorithms”,Computer Journal 13 (1970) 317–322.

    Google Scholar 

  27. M.J.D. Powell, “On the convergence of the variable metric algorithm“, Report No. T.P. 382, A.E.R.E. Harwell (1969).

    Google Scholar 

  28. Y. Bard, “On a numerical instability of Davidon-like methods”,Mathematics of Computation 22 (1968) 665–666.

    Google Scholar 

  29. G.P. McCormick and J.D. Pearson, “Variable metric methods and unconstrained optimization”, in:Optimization, Ed. R. Fletcher (Academic Press, London, 1969) 307–325.

    Google Scholar 

  30. G.P. McCormick “The rate of convergence of the reset Davidon variable metric method”, MRC Report No. 1012, Wisconsin (1969).

  31. C.G. Broyden, “Quasi-Newton methods and their application to function minimization”,Mathematics of Computation 21 (1967) 368–381.

    Google Scholar 

  32. C.G. Broyden, “The convergence of a class of double-rank minimization algorithms 2. The new algorithm”,J.I.M.A. 6 (1970) 222–231.

    Google Scholar 

  33. D. Goldfarb, “A family of variable metric methods derived by variational means”,Mathematics of Computation 24 (1970) 23–26.

    Google Scholar 

  34. D.F. Shanno, “Conditioning of quasi-Newton methods for function minimization”, Center for Mathematical Studies Report No. 6910, University of Chicago (1969).

  35. D.F. Shanno and P.C. Kettler, “Optimal conditioning of quasi-Newton methods”, Center for Mathematical Studies Report No. 6937, University of Chicago (1969).

  36. H.J. Kelley and G.E. Myers, “Conjugate direction methods for parameter optimization”, presented at the 18th Congress of the International Astronautical Federation, Belgrade, 1967.

  37. M.R. Hestenes, “Multiplier and gradient methods”, in:Computing methods in optimization problems 2, Eds. L.A. Zadeh et al. (Academic Press, New York, 1969).

    Google Scholar 

  38. A. Miele and J.W. Cantrell, “Study on a memory gradient method for the minimization of functions”J.O.T.A. 3 (1969) 459–470.

    Google Scholar 

  39. A. Miele and J.W. Cantrell, “Memory gradient method for the minimization of functions”, in:Symposium on Optimization, Eds. Balakrishnan et al. (Springer-Verlag, Berlin, 1970) 252–263.

    Google Scholar 

  40. J.W. Cantrell, “Relation between the memory gradient method and the Fletcher-Reeves method”,J.O.T.A. 4 (1969) 67–71.

    Google Scholar 

  41. D.F. Shanno, “Inverse quasi-Newton methods”, Center for Mathematical Studies Report No. 6938, University of Chicago (1969).

  42. W.C. Davidon, “Variance algorithm for minimization”,Computer Journal 10 (1968) 406–410.

    Google Scholar 

  43. B.A. Murtagh and R.W.H. Sargent, “A constrained minimization method with quadratic convergence”, in:Optimization, Ed. R. Fletcher (Academic Press, London, 1969) 215–246.

    Google Scholar 

  44. P. Wolfe, “Another variable metric method” (1967) working paper.

  45. M.J.D. Powell, “Rank one methods for unconstrained optimization”, in:Integer and nonlinear programming, Ed. J. Abadie (North-Holland Publishing Co., Amsterdam, 1970) 139–156.

    Google Scholar 

  46. W.C. Davidon, “Variance algorithms for minimization”, in:Optimization, Ed. R. Fletcher (Academic Press, London, 1969) 13–20.

    Google Scholar 

  47. B.A. Murtagh and R.W.H. Sargent, “Computational experience with quadratically convergent minimization methods”,Computer Journal 13 (1970) 185–194.

    Google Scholar 

  48. Y. Bard, “Comparison of gradient methods for the solution of non-linear parameter estimation problems”,SIAM Numerical Analysis 7 (1970) 157–186.

    Google Scholar 

  49. S.M. Goldfeld, R.E. Quandt and H.F. Trotter, “Maximization by quadratic hill-climbing”,Econometrica 34 (1966) 541–551.

    Google Scholar 

  50. P. Wolfe, “Convergence conditions for ascent methods”,SIAM Review 11 (1969) 226–235.

    Google Scholar 

  51. A.A. Goldstein and J.F. Price, “An effective algorithm for minimization”,Numerische Mathematik 10 (1967) 184–189.

    Google Scholar 

  52. M.J.D. Powell, “A Fortran subroutine for unconstrained minimization, requiring first derivatives of the objective function”, Report No. R.6469, A.E.R.E. Harwell (1970).

    Google Scholar 

  53. J.M. Bennett, “Triangular factors of modified matrices”,Numerische Mathematik 7 (1965) 217–221.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. A.E.R.E., Harwell, Berks., Great Britain

    M. J. D. Powell

Authors
  1. M. J. D. Powell
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Powell, M.J.D. Recent advances in unconstrained optimization. Mathematical Programming 1, 26–57 (1971). https://doi.org/10.1007/BF01584071

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01584071

Keywords

Search

Navigation