# Dynamic scaling in the mesh adaptive direct search algorithm for blackbox optimization

- 489 Downloads
- 4 Citations

## Abstract

Blackbox optimization deals with situations in which the objective function and constraints are typically computed by launching a time-consuming computer simulation. The subject of this work is the mesh adaptive direct search (mads) class of algorithms for blackbox optimization. We propose a way to dynamically scale the mesh, which is the discrete spatial structure on which mads relies, so that it automatically adapts to the characteristics of the problem to solve. Another objective of the paper is to revisit the mads method in order to ease its presentation and to reflect recent developments. This new presentation includes a nonsmooth convergence analysis. Finally, numerical tests are conducted to illustrate the efficiency of the dynamic scaling, both on academic test problems and on a supersonic business jet design problem.

## Keywords

Blackbox optimization Derivative-free optimization Mesh adaptive direct search Dynamic scaling## Mathematics Subject Classification

90C30 90C56 65K05 62P30## Notes

### Acknowledgments

The authors wish to thank Michael Kokkolaras for making the Supersonic Business Jet Design problem available. We also thank the referees for their constructive comments. Work of the first author was supported by NSERC Grant 239436. The second author was supported by NSERC Grant 418250. The first and second authors are supported by AFOSR FA9550-12-1-0198.

## References

- Abramson MA, Audet C, Dennis JE Jr, Le Digabel S (2009) OrthoMADS: a deterministic MADS instance with orthogonal directions. SIAM J Optim 20(2):948–966MathSciNetCrossRefzbMATHGoogle Scholar
- Adjengue L, Audet C, Ben I, Yahia IB (2014) A variance-based method to rank input variables of the mesh adaptive direct search algorithm. Optim Lett 8(5):1599–1610MathSciNetCrossRefzbMATHGoogle Scholar
- Agte JS, Sobieszczanski-Sobieski J, Sandusky RRJ (1999) Supersonic business jet design through bilevel integrated system synthesis. In: Proceedings of the World Aviation Conference, volume SAE Paper No. 1999–01-5622, San Francisco, CA, 1999. MCB University Press, BradfordGoogle Scholar
- Audet C (2004) Convergence results for generalized pattern search algorithms are tight. Optim Eng 5(2):101–122MathSciNetCrossRefzbMATHGoogle Scholar
- Audet C, Béchard V, Le Digabel S (2008) Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J Glob Optim 41(2):299–318MathSciNetCrossRefzbMATHGoogle Scholar
- Audet C, Dennis JE Jr (2006) Mesh adaptive direct search algorithms for constrained optimization. SIAM J Optim 17(1):188–217MathSciNetCrossRefzbMATHGoogle Scholar
- Audet C, Dennis JE Jr (2009) A Progressive Barrier for Derivative-Free Nonlinear Programming. SIAM J Optim 20(1):445–472MathSciNetCrossRefzbMATHGoogle Scholar
- Audet C, Dennis JE Jr, Le Digabel S (2008) Parallel space decomposition of the mesh adaptive direct search algorithm. SIAM J Optim 19(3):1150–1170MathSciNetCrossRefzbMATHGoogle Scholar
- Audet C, Dennis JE Jr, Le Digabel S (2010) Globalization strategies for mesh adaptive direct search. Comput Optim Appl 46(2):193–215MathSciNetCrossRefzbMATHGoogle Scholar
- Audet C, Ianni A, Le Digabel S, Tribes C (2014) Reducing the number of function evaluations in mesh adaptive direct search algorithms. SIAM J Optim 24(2):621–642MathSciNetCrossRefzbMATHGoogle Scholar
- Box GEP, Muller ME (1958) A note on the generation of random normal deviates. Ann Math Stat 29(2):610–611CrossRefzbMATHGoogle Scholar
- Conn AR, Le Digabel S (2013) Use of quadratic models with mesh-adaptive direct search for constrained black box optimization. Optim Methods Softw 28(1):139–158MathSciNetCrossRefzbMATHGoogle Scholar
- Conn AR, Scheinberg K, Vicente LN (2009) Introduction to derivative-free optimization. MOS-SIAM series on optimization. SIAM, PhiladelphiaCrossRefzbMATHGoogle Scholar
- Coope ID, Price CJ (2000) Frame-based methods for unconstrained optimization. J Optim Theory Appl 107(2):261–274MathSciNetCrossRefzbMATHGoogle Scholar
- Cramer EJ, Dennis JE Jr, Frank PD, Lewis RM, Shubin GR (1994) Problem formulation for multidisciplinary optimization. SIAM J Optim 4(4):754–776MathSciNetCrossRefzbMATHGoogle Scholar
- Dennis Jr JE, Schnabel RB (1996) Numerical methods for unconstrained optimization and nonlinear equations. Prentice-Hall, Englewood Cliffs, NJ, 1983. Reissued in 1996 by SIAM Publications, Philadelphia, as, Vol. 16 in the series Classics in Applied MathematicsGoogle Scholar
- Fletcher R (1980) Practical methods of optimization, volume 1: unconstrained optimization. Wiley, ChichesterzbMATHGoogle Scholar
- García-Palomares UM, Rodríguez JF (2002) New sequential and parallel derivative-free algorithms for unconstrained optimization. SIAM J Optim 13(1):79–96MathSciNetCrossRefzbMATHGoogle Scholar
- Gill PE, Murray W, Wright MH (1981) Practical optimization. Academic Press, LondonzbMATHGoogle Scholar
- Gould N, Orban D, Toint P (2015) CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput Optim Appl 60(3):545–557. Code available at http://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki
- Halton JH (1960) On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals. Numer Math 2(1):84–90MathSciNetCrossRefzbMATHGoogle Scholar
- Hough PD, Kolda TG, Torczon V (2001) Asynchronous parallel pattern search for nonlinear optimization. SIAM J Sci Comput 23(1):134–156MathSciNetCrossRefzbMATHGoogle Scholar
- Kolda TG, Lewis RM, Torczon V (2003) Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev 45(3):385–482MathSciNetCrossRefzbMATHGoogle Scholar
- Kolda TG, Lewis RM, Torczon V (2006) Stationarity results for generating set search for linearly constrained optimization. SIAM J Optim 17(4):943–968MathSciNetCrossRefzbMATHGoogle Scholar
- Le Digabel S (2011) Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans Math Softw 37(4):44:1–44:15MathSciNetCrossRefGoogle Scholar
- Liseikin VD (2009) Grid generation methods. Springer, BerlinzbMATHGoogle Scholar
- Lucidi S, Sciandrone M (2002) On the global convergence of derivative-free methods for unconstrained optimization. SIAM J Optim 13(1):97–116MathSciNetCrossRefzbMATHGoogle Scholar
- Marsaglia G (1972) Choosing a point from the surface of a sphere. Ann Math Stat 43(2):645–646CrossRefzbMATHGoogle Scholar
- Martínez JM, Sobral FNC (2013) Constrained derivative-free optimization on thin domains. J Glob Optim 56(3):1217–1232MathSciNetCrossRefzbMATHGoogle Scholar
- McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245MathSciNetzbMATHGoogle Scholar
- Mladenović N, Hansen P (1997) Variable neighborhood search. Comput Oper Res 24(11):1097–1100MathSciNetCrossRefzbMATHGoogle Scholar
- Moré JJ, Wild SM (2009) Benchmarking derivative-free optimization algorithms. SIAM J Optim 20(1):172–191MathSciNetCrossRefzbMATHGoogle Scholar
- Nocedal J, Wright SJ (1999) Numerical optimization. Springer Series in Operations Research. Springer, New YorkCrossRefzbMATHGoogle Scholar
- Papalambros PY, Wilde DJ (2000) Principles of optimal design: modeling and computation, 2nd edn. Cambridge University Press, CambridgeCrossRefzbMATHGoogle Scholar
- Torczon V (1997) On the convergence of pattern search algorithms. SIAM J Optim 7(1):1–25MathSciNetCrossRefzbMATHGoogle Scholar
- Tosserams S, Kokkolaras M, Etman LFP, Rooda JE (2010) A non-hierarchical formulation of analytical target cascading. J Mech Des 132(5):051002-1–051002-13CrossRefGoogle Scholar
- Van Dyke B, Asaki TJ (2013) Using QR decomposition to obtain a new instance of mesh adaptive direct search with uniformly distributed polling directions. J Optim Theory Appl 159(3):805–821MathSciNetCrossRefzbMATHGoogle Scholar
- Van Loan CF (1985) How near is a stable matrix to an unstable matrix. Contemp Math 47:465–478MathSciNetCrossRefzbMATHGoogle Scholar
- Vicente LN, Custódio AL (2012) Analysis of direct searches for discontinuous functions. Math Programm 133(1–2):299–325MathSciNetCrossRefzbMATHGoogle Scholar