Abstract
We consider iterative trust region algorithms for the unconstrained minimization of an objective function \(F ( \underline{x})\), \(\underline{x}\in \mathcal{R}^{n}\), when F is differentiable but no derivatives are available, and when each model of F is a linear or a quadratic polynomial. The models interpolate F at n+1 points, which defines them uniquely when they are linear polynomials. In the quadratic case, second derivatives of the models are derived from information from previous iterations, but there are so few data that typically only the magnitudes of second derivative estimates are correct. Nevertheless, numerical results show that much faster convergence is achieved when quadratic models are employed instead of linear ones. Just one new value of F is calculated on each iteration. Changes to the variables are either trust region steps or are designed to maintain suitable volumes and diameters of the convex hulls of the interpolation points. It is proved that, if F is bounded below, if ∇2 F is also bounded, and if the number of iterations is infinite, then the sequence of gradients \(\underline{\nabla}F ( \underline{x}_{\,k} )\), k=1,2,3,…, converges to zero, where \(\underline{x}_{\,k}\) is the centre of the trust region of the k-th iteration.
Similar content being viewed by others
References
Conn, A.R., Scheinberg, K., Vicente, L.N.: On the convergence of derivative-free methods for unconstrained optimization. In: Buhmann, M.D., Iserles, A. (eds.) Approximation Theory and Optimization, pp. 83–108. Cambridge University Press, Cambridge (1997)
Conn, A.R., Scheinberg, K., Vicente, L.N.: Global convergence of general derivative-free trust-region algorithms to first and second order critical points. SIAM J. Optim. 20, 387–415 (2009)
Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. SIAM, Philadelphia (2009)
Powell, M.J.D.: A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Gomez, S., Hennart, J.-P. (eds.) Advances in Optimization and Numerical Analysis, pp. 51–67. Kluwer Academic, Dordrecht (1994)
Powell, M.J.D.: The NEWUOA software for unconstrained optimization without derivatives. In: Di Pillo, G., Roma, M. (eds.) Large-Scale Optimization, pp. 255–297. Springer, New York (2006)
Acknowledgement
The author is grateful to two referees for their helpful advice and comments.
Author information
Authors and Affiliations
Corresponding author
Additional information
Presented at the Workshop on Nonlinear Optimization, Variational Inequalities and Equilibrium Problems (Erice, Italy, 2010).
Rights and permissions
About this article
Cite this article
Powell, M.J.D. On the convergence of trust region algorithms for unconstrained minimization without derivatives. Comput Optim Appl 53, 527–555 (2012). https://doi.org/10.1007/s10589-012-9483-x
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-012-9483-x