Beyond symmetric Broyden for updating quadratic models in minimization without derivatives
 M. J. D. Powell
 … show all 1 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
Some highly successful algorithms for unconstrained minimization without derivatives construct changes to the variables by applying trust region methods to quadratic approximations to the objective function \({F (\underline{x}), \underline{x} \in \mathcal{R}^n}\) . A quadratic model has (n + 1) (n + 2)/2 independent parameters, but each new model may interpolate only 2n + 1 values of F, for instance. The symmetric Broyden method takes up the remaining freedom by minimizing the Frobenius norm of the difference between the second derivative matrices of the old and new models, which usually works well in practice. We consider an extension of this technique that combines changes in first derivatives with changes in second derivatives. A simple example suggests that the extension does bring some advantages, but numerical experiments on three test problems with up to 320 variables are disappointing. On the other hand, rates of convergence are investigated numerically when F is a homogeneous quadratic function, which allows very high accuracy to be achieved in practice, the initial and final errors in the variables being about 10 and 10^{−5000}, respectively. It is clear in some of these experiments that the extension does reduce the number of iterations. The main difficulty in the work was finding a way of implementing the extension sufficiently accurately in only \({\mathcal{O}( n^2 )}\) operations on each iteration. A version of the truncated conjugate gradient procedure is suitable, that is used in the numerical experiments, and that is described in detail in an appendix.
 Broyden C.G., Dennis J.E., Moré J.J.: On the local and superlinear convergence of quasiNewton methods. J. Inst. Math. Appl. 12, 223–245 (1973) CrossRef
 Buckley, A.G.: Test functions for unconstrained minimization. Technical Report 1989 CS3, Dalhousie University, Canada (1989)
 Conn A.R., Gould N.I.M., Lescrenier M., Toint Ph.L.: Performance of a multifrontal scheme for partially separable optimization. In: Gomez, S., Hennart, J.P. (eds) Advances in Optimization and Numerical Analysis, pp. 79–96. Kluwer, Dordrecht (1994) CrossRef
 Conn A.R., Scheinberg K., Vicente L.N.: Introduction to DerivativeFree Optimization. SIAM, Philadelphia (2009) CrossRef
 Fletcher R.: Practical Methods of Optimization. Wiley, Chichester (1987)
 Powell M.J.D.: Least Frobenius norm updating of quadratic models that satisfy interpolation conditions. Math. Program. B 100, 183–215 (2004) CrossRef
 Powell M.J.D.: On updating the inverse of a KKT matrix. In: Yuan, Y. (eds) Numerical Linear Algebra and Optimization, pp. 56–78. Science Press, Beijing (2004)
 Powell M.J.D.: The NEWUOA software for unconstrained optimization without derivatives. In: Di Pillo, G., Roma, M. (eds) LargeScale Optimization, pp. 255–297. Springer, New York (2006) CrossRef
 Powell, M.J.D.: The BOBYQA algorithm for bound constrained optimization without derivatives. Report No. DAMTP 2009/NA06, CMS, University of Cambridge (2009)
 Title
 Beyond symmetric Broyden for updating quadratic models in minimization without derivatives
 Journal

Mathematical Programming
Volume 138, Issue 12 , pp 475500
 Cover Date
 20130401
 DOI
 10.1007/s101070110510y
 Print ISSN
 00255610
 Online ISSN
 14364646
 Publisher
 SpringerVerlag
 Additional Links
 Topics
 Keywords

 Minimization without derivatives
 Quadratic models
 Symmetric Broyden
 Truncated conjugate gradients
 65K05
 90C30
 Industry Sectors
 Authors

 M. J. D. Powell ^{(1)}
 Author Affiliations

 1. Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Wilberforce Road, Cambridge, CB3 0WA, UK