Abstract
We consider derivative free methods based on sampling approaches for nonlinear optimization problems where derivatives of the objective function are not available and cannot be directly approximated. We show how the bounds on the error between an interpolating polynomial and the true function can be used in the convergence theory of derivative free sampling methods. These bounds involve a constant that reflects the quality of the interpolation set. The main task of such a derivative free algorithm is to maintain an interpolation sampling set so that this constant remains small, and at least uniformly bounded. This constant is often described through the basis of Lagrange polynomials associated with the interpolation set. We provide an alternative, more intuitive, definition for this concept and show how this constant is related to the condition number of a certain matrix. This relation enables us to provide a range of algorithms whilst maintaining the interpolation set so that this condition number or the geometry constant remain uniformly bounded. We also derive bounds on the error between the model and the function and between their derivatives, directly in terms of this condition number and of this geometry constant.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alberto, P., Nogueira, F., Rocha, H., Vicente, LN.: Pattern search methods for user-provided points: Application to molecular geometry problems. SIAM J. Optim. 14, 1216–1236 (2004)
Audet, C., Dennis, J.E.:Analysis of generalized pattern searches. SIAM J. Optim. 13, 889–903
de Boor C.D. (1994). Gaussian elimination by segments and multivariate polynomial interpolation. In: Zahar R.V.M. (eds). Approximation and Computation: A Festschrift in Honor of Walter Gautschi. Birkhäuser Verlag, pp. 87–96
Bortz, D.M., Kelley, C.T.: The Simplex Gradient and Noisy Optimization Problems, vol. 24, pp. 77–90. Birkhäuser, Boston (1998)
Ciarlet, P.G., Raviart, P.A.:General Lagrange and Hermite interpolation in R n with applications to finite element methods. Arch. Rational Mech. Anal. 46, 177–199 (1972)
Colson, B.: Trust-region algorithms for derivative-free optimization and nonlinear bilevel programming. Ph.D. thesis, Département de Mathématique, FUNDP, Namur (2003)
Conn A.R., Gould N.I.M., Toint Ph.L. (2000) Trust-Region Methods. MPS-SIAM Series on Optimization. SIAM, Philadelphia
Conn A.R., Scheinberg K., Toint Ph.L. (1997) On the convergence of derivative-free methods for unconstrained optimization. In: Powell M.J.D., Buhmann M.D., Iserles A. (eds). Approximation Theory and Optimization. Tributes to. Cambridge University Press, Cambridge, pp. 83–108
Conn, A.R., Scheinberg, K., Toint, Ph.L.: Recent progress in unconstrained nonlinear optimization without derivatives. Math. Program. 79, 397–414 (1997)
Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice–Hall, Englewood Cliffs, (republished by SIAM, Philadelphia, in 1996, as Classics in Applied Mathematics, vol. 16) (1983)
Kelley C.T. (1999) Iterative Methods for Optimization. SIAM, Philadelphia
Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: New perspectives on some classical and modern methods. SIAM Rev. 45, 385–482 (2003)
Marazzi, M., Nocedal, J.: Wedge trust region methods for derivative free optimization. Math. Program. 91, 289–300 (2002)
Oeuvray, R.: Trust-region methods based on radial basis functions with application to biomedical imaging. Ph.D. thesis, EPFL, Lausanne (2005)
Oeuvray, R., Bierlaire, M.: BOOSTERS: A derivative-free algorithm based on radial basis functions. Submitted for publication (2005)
Powell, M.J.D.: Direct search algorithms for optimization calculations. Acta Numerica 7, 287–336 (1998)
Powell, M.J.D.:On the Lagrange functions of quadratic models that are defined by interpolation. Optim. Methods Softw. 16, 289–309 (2001)
Powell, M.J.D.:On trust region methods for unconstrained minimization without derivatives.Math. Program. 97, 605–623 (2003)
Price, C.J., Coope, I.D.: Frames and grids in unconstrained and linearly constrained optimization: A non-smooth approach. SIAM J. Optim. 14, 415–438 (2003)
Sauer, T.: On multivariate Lagrange interpolation. Math. Comp. 64, 1147–1170 (1995)
Torczon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7, 1–25 (1997)
Wright, M.H.: Direct search methods: Once scorned, now respectable. In: Griffiths, D.F., Watson, G.A. (eds.) Numerical Analysis 1995, Proceedings of the 1995 Dundee Biennal Conference in Numerical Analysis, pp. 191–208. Pitman Res. Notes Math. Ser. vol. 344. CRC Press, Boca Raton (1996)
Author information
Authors and Affiliations
Corresponding author
Additional information
Dedicated to Clovis Gonzaga on the occasion of his 60th birthday.
Support for L. N. Vicente was provided by Centro de Matemática da Universidade de Coimbra, by FCT under grants POCTI/35059/MAT/2000 and POCTI/59442/MAT/2004, by the European Union under grant IST-2000-26063, and by Fundação Calouste Gulbenkian. Most of this paper was written while this author was visiting IBM T.J. Watson Research Center.
Rights and permissions
About this article
Cite this article
Conn, A.R., Scheinberg, K. & Vicente, L.N. Geometry of interpolation sets in derivative free optimization. Math. Program. 111, 141–172 (2008). https://doi.org/10.1007/s10107-006-0073-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-006-0073-5