Mathematical Programming

, Volume 138, Issue 1–2, pp 475–500

Beyond symmetric Broyden for updating quadratic models in minimization without derivatives

Full Length Paper Series A

Abstract

Some highly successful algorithms for unconstrained minimization without derivatives construct changes to the variables by applying trust region methods to quadratic approximations to the objective function \({F (\underline{x}), \underline{x} \in \mathcal{R}^n}\). A quadratic model has (n + 1) (n + 2)/2 independent parameters, but each new model may interpolate only 2n + 1 values of F, for instance. The symmetric Broyden method takes up the remaining freedom by minimizing the Frobenius norm of the difference between the second derivative matrices of the old and new models, which usually works well in practice. We consider an extension of this technique that combines changes in first derivatives with changes in second derivatives. A simple example suggests that the extension does bring some advantages, but numerical experiments on three test problems with up to 320 variables are disappointing. On the other hand, rates of convergence are investigated numerically when F is a homogeneous quadratic function, which allows very high accuracy to be achieved in practice, the initial and final errors in the variables being about 10 and 10−5000, respectively. It is clear in some of these experiments that the extension does reduce the number of iterations. The main difficulty in the work was finding a way of implementing the extension sufficiently accurately in only \({\mathcal{O}( n^2 )}\) operations on each iteration. A version of the truncated conjugate gradient procedure is suitable, that is used in the numerical experiments, and that is described in detail in an appendix.

Keywords

Minimization without derivatives Quadratic models Symmetric Broyden Truncated conjugate gradients 

Mathematics Subject Classification (2000)

65K05 90C30 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Broyden C.G., Dennis J.E., Moré J.J.: On the local and superlinear convergence of quasi-Newton methods. J. Inst. Math. Appl. 12, 223–245 (1973)MathSciNetMATHCrossRefGoogle Scholar
  2. 2.
    Buckley, A.G.: Test functions for unconstrained minimization. Technical Report 1989 CS-3, Dalhousie University, Canada (1989)Google Scholar
  3. 3.
    Conn A.R., Gould N.I.M., Lescrenier M., Toint Ph.L.: Performance of a multifrontal scheme for partially separable optimization. In: Gomez, S., Hennart, J.-P. (eds) Advances in Optimization and Numerical Analysis, pp. 79–96. Kluwer, Dordrecht (1994)CrossRefGoogle Scholar
  4. 4.
    Conn A.R., Scheinberg K., Vicente L.N.: Introduction to Derivative-Free Optimization. SIAM, Philadelphia (2009)MATHCrossRefGoogle Scholar
  5. 5.
    Fletcher R.: Practical Methods of Optimization. Wiley, Chichester (1987)MATHGoogle Scholar
  6. 6.
    Powell M.J.D.: Least Frobenius norm updating of quadratic models that satisfy interpolation conditions. Math. Program. B 100, 183–215 (2004)MATHCrossRefGoogle Scholar
  7. 7.
    Powell M.J.D.: On updating the inverse of a KKT matrix. In: Yuan, Y. (eds) Numerical Linear Algebra and Optimization, pp. 56–78. Science Press, Beijing (2004)Google Scholar
  8. 8.
    Powell M.J.D.: The NEWUOA software for unconstrained optimization without derivatives. In: Di Pillo, G., Roma, M. (eds) Large-Scale Optimization, pp. 255–297. Springer, New York (2006)CrossRefGoogle Scholar
  9. 9.
    Powell, M.J.D.: The BOBYQA algorithm for bound constrained optimization without derivatives. Report No. DAMTP 2009/NA06, CMS, University of Cambridge (2009)Google Scholar

Copyright information

© Springer and Mathematical Optimization Society 2012

Authors and Affiliations

  1. 1.Department of Applied Mathematics and Theoretical PhysicsUniversity of CambridgeCambridgeUK

Personalised recommendations