Abstract.
A common question asked by users of direct search algorithms is how to use derivative information at iterates where it is available. This paper addresses that question with respect to Generalized Pattern Search (GPS) methods for unconstrained and linearly constrained optimization. Specifically, this paper concentrates on the GPS pollstep. Polling is done to certify the need to refine the current mesh, and it requires O(n) function evaluations in the worst case. We show that the use of derivative information significantly reduces the maximum number of function evaluations necessary for pollsteps, even to a worst case of a single function evaluation with certain algorithmic choices given here. Furthermore, we show that rather rough approximations to the gradient are sufficient to reduce the pollstep to a single function evaluation. We prove that using these less expensive pollsteps does not weaken the known convergence properties of the method, all of which depend only on the pollstep.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abramson, M.A.: Mixed variable optimization of a load-bearing thermal insulation system. Technical Report TR02-13, Department of Computational and Applied Mathematics, Rice University, Houston Texas, 2002. To appear in Optim. Eng.
Abramson, M.A.: Nonlinear optimization with mixed variables and derivatives – Matlab © (NOMADm). Software. Available for download at http://en.afit.edu/ENC/Faculty/MAbramson/abramson.html, 2002
Abramson, M.A.: Pattern search algorithms for mixed variable general constrained optimization problems. PhD thesis, Rice University, Department of Computational and Applied Mathematics, Houston, Texas, 2002. Also appears as CAAM Technical Report TR-02-11
Alexandrov, N., Dennis, J.E. Jr., Lewis, R., Torczon, V.: A trust region framework for managing the use of approximation models in optimization. Struct. Optim. 15, 16–23 (1998)
Audet, C.: Convergence results for pattern search algorithms are tight, Technical Report G-2002-56, Les Cahiers du GERAD, Montréal, Canada, 2002. To appear in Optim. Eng.
Audet, C., Dennis, J.E. Jr.: Pattern search algorithms for mixed variable programming. SIAM J. Optim. 11, 573–594 (2000)
Audet, C., Dennis, J.E. Jr.: A pattern search filter method for nonlinear programming without derivatives. Technical Report TR00-09, Department of Computational and Applied Mathematics. Rice University, Houston Texas, (2000)
Audet, C., Dennis, J.E. Jr.: Analysis of generalized pattern searches. SIAM J. Optim. 13, 889–903 (2003)
Bongartz, I., Conn, A.R., Gould, N., Toint, Ph.L.: CUTE: Constrained and unconstrained testing environment. ACM Trans. Math. Softw. 21, 123–160 (1995)
Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Serafini, D.B., Torczon, V., Trosset M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Optim. 17, 1–13 (1999)
Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Moore, D.W., Serafini, D.B.: Managing surrogate objectives to optimize a helicopter rotor design – further experiment., AIAA Paper 98-4717, St. Louis, September 1998 (1999)
Byrd, R.H., Tapia R.A.: An extension of Curry’s theorem to steepest descent in normed linear spaces. Math. Program. 9, 247–254 (1975)
Clarke, F.H.: Optimization and Nonsmooth Analysis. SIAM Classics in Applied Mathematics, Vol. 5, Philadelphia, 1990
Conn, A.R., Gould, N.I.M., Toint, Ph.L.: A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J. Numer. Anal. 28, 545–572 (1991)
Davis, C.: Theory of positive linear dependence. Am. J. Math. 76, 733–746 (1954)
Dennis, J.E. Jr., Torczon, V.: Direct search methods on parallel machines. SIAM J. Optim. 1, 448–474 (1991)
Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. 91, 239–269 (2002)
Fletcher, R, Leyffer, S., Toint, Ph.L.: On the global convergence of an SLP-filter algorithm, Report NA/183, Dundee University, Dept. of Mathematics, 1998
Fletcher, R, Gould, N.I.M., Leyffer, S., Toint, Ph.L.: On the global convergence of trust-region SQP-filter algorithms for general nonlinear programming, Report 99/03, Department of Mathematics, FUNDP, Namur (B), 1999
Kokkolaras, M., Audet, C., Dennis, J.E. Jr.: Mixed variable optimization of the number and composition of heat intercepts in a thermal insulation system. Optim. Eng. 2, 5–29 (2001)
Lewis, R.M., Torczon V.: Rank ordering and positive basis in pattern search algorithms. Technical Report TR-96-71, ICASE NASA Langley Research Center, 1996
Lewis, R.M., Torczon, V.: A globally convergent augmented Lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Optim. 12, 1075–1089 (2002)
Lewis, R.M., Torczon, V.: Pattern search algorithms for bound constrained minimization. SIAM J. Optim. 9, 1082–1099 (1999)
Lewis, R.M., Torczon, V.: Pattern search methods for linearly constrained minimization. SIAM J. Optim. 10, 917–941 (2000)
McKay, M.D., Conover, W.J., Beckman, R.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21, 239–245 (1979)
Mohammadi, B., Pironneau, O.: Applied Shape Optimization for Fluids. Oxford University Press, Oxford, 2001
Soto, O., Löhner, R.: CFD optimization using an incomplete–gradient adjoint formulation. Int. J. Num. Meth. Eng. 51, 735–753 (2001)
Stein, M.: Large sample properties of simulations using Latin hypercube sampling. Technometrics 29, 143–151 (1987)
Torczon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7, 1–25 (1997)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Abramson, M., Audet, C. & Dennis, J. Generalized pattern searches with derivative information. Math. Program., Ser. B 100, 3–25 (2004). https://doi.org/10.1007/s10107-003-0484-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-003-0484-5