# Dynamic scaling in the mesh adaptive direct search algorithm for blackbox optimization

- 436 Downloads
- 2 Citations

## Abstract

Blackbox optimization deals with situations in which the objective function and constraints are typically computed by launching a time-consuming computer simulation. The subject of this work is the mesh adaptive direct search (mads) class of algorithms for blackbox optimization. We propose a way to dynamically scale the mesh, which is the discrete spatial structure on which mads relies, so that it automatically adapts to the characteristics of the problem to solve. Another objective of the paper is to revisit the mads method in order to ease its presentation and to reflect recent developments. This new presentation includes a nonsmooth convergence analysis. Finally, numerical tests are conducted to illustrate the efficiency of the dynamic scaling, both on academic test problems and on a supersonic business jet design problem.

## Keywords

Blackbox optimization Derivative-free optimization Mesh adaptive direct search Dynamic scaling## Mathematics Subject Classification

90C30 90C56 65K05 62P30## Notes

### Acknowledgments

The authors wish to thank Michael Kokkolaras for making the Supersonic Business Jet Design problem available. We also thank the referees for their constructive comments. Work of the first author was supported by NSERC Grant 239436. The second author was supported by NSERC Grant 418250. The first and second authors are supported by AFOSR FA9550-12-1-0198.

## References

- Abramson MA, Audet C, Dennis JE Jr, Le Digabel S (2009) OrthoMADS: a deterministic MADS instance with orthogonal directions. SIAM J Optim 20(2):948–966MathSciNetCrossRefMATHGoogle Scholar
- Adjengue L, Audet C, Ben I, Yahia IB (2014) A variance-based method to rank input variables of the mesh adaptive direct search algorithm. Optim Lett 8(5):1599–1610MathSciNetCrossRefMATHGoogle Scholar
- Agte JS, Sobieszczanski-Sobieski J, Sandusky RRJ (1999) Supersonic business jet design through bilevel integrated system synthesis. In: Proceedings of the World Aviation Conference, volume SAE Paper No. 1999–01-5622, San Francisco, CA, 1999. MCB University Press, BradfordGoogle Scholar
- Audet C (2004) Convergence results for generalized pattern search algorithms are tight. Optim Eng 5(2):101–122MathSciNetCrossRefMATHGoogle Scholar
- Audet C, Béchard V, Le Digabel S (2008) Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J Glob Optim 41(2):299–318MathSciNetCrossRefMATHGoogle Scholar
- Audet C, Dennis JE Jr (2006) Mesh adaptive direct search algorithms for constrained optimization. SIAM J Optim 17(1):188–217MathSciNetCrossRefMATHGoogle Scholar
- Audet C, Dennis JE Jr (2009) A Progressive Barrier for Derivative-Free Nonlinear Programming. SIAM J Optim 20(1):445–472MathSciNetCrossRefMATHGoogle Scholar
- Audet C, Dennis JE Jr, Le Digabel S (2008) Parallel space decomposition of the mesh adaptive direct search algorithm. SIAM J Optim 19(3):1150–1170MathSciNetCrossRefMATHGoogle Scholar
- Audet C, Dennis JE Jr, Le Digabel S (2010) Globalization strategies for mesh adaptive direct search. Comput Optim Appl 46(2):193–215MathSciNetCrossRefMATHGoogle Scholar
- Audet C, Ianni A, Le Digabel S, Tribes C (2014) Reducing the number of function evaluations in mesh adaptive direct search algorithms. SIAM J Optim 24(2):621–642MathSciNetCrossRefMATHGoogle Scholar
- Box GEP, Muller ME (1958) A note on the generation of random normal deviates. Ann Math Stat 29(2):610–611CrossRefMATHGoogle Scholar
- Conn AR, Le Digabel S (2013) Use of quadratic models with mesh-adaptive direct search for constrained black box optimization. Optim Methods Softw 28(1):139–158MathSciNetCrossRefMATHGoogle Scholar
- Conn AR, Scheinberg K, Vicente LN (2009) Introduction to derivative-free optimization. MOS-SIAM series on optimization. SIAM, PhiladelphiaCrossRefMATHGoogle Scholar
- Coope ID, Price CJ (2000) Frame-based methods for unconstrained optimization. J Optim Theory Appl 107(2):261–274MathSciNetCrossRefMATHGoogle Scholar
- Cramer EJ, Dennis JE Jr, Frank PD, Lewis RM, Shubin GR (1994) Problem formulation for multidisciplinary optimization. SIAM J Optim 4(4):754–776MathSciNetCrossRefMATHGoogle Scholar
- Dennis Jr JE, Schnabel RB (1996) Numerical methods for unconstrained optimization and nonlinear equations. Prentice-Hall, Englewood Cliffs, NJ, 1983. Reissued in 1996 by SIAM Publications, Philadelphia, as, Vol. 16 in the series Classics in Applied MathematicsGoogle Scholar
- Fletcher R (1980) Practical methods of optimization, volume 1: unconstrained optimization. Wiley, ChichesterMATHGoogle Scholar
- García-Palomares UM, Rodríguez JF (2002) New sequential and parallel derivative-free algorithms for unconstrained optimization. SIAM J Optim 13(1):79–96MathSciNetCrossRefMATHGoogle Scholar
- Gill PE, Murray W, Wright MH (1981) Practical optimization. Academic Press, LondonMATHGoogle Scholar
- Gould N, Orban D, Toint P (2015) CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput Optim Appl 60(3):545–557. Code available at http://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki
- Halton JH (1960) On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals. Numer Math 2(1):84–90MathSciNetCrossRefMATHGoogle Scholar
- Hough PD, Kolda TG, Torczon V (2001) Asynchronous parallel pattern search for nonlinear optimization. SIAM J Sci Comput 23(1):134–156MathSciNetCrossRefMATHGoogle Scholar
- Kolda TG, Lewis RM, Torczon V (2003) Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev 45(3):385–482MathSciNetCrossRefMATHGoogle Scholar
- Kolda TG, Lewis RM, Torczon V (2006) Stationarity results for generating set search for linearly constrained optimization. SIAM J Optim 17(4):943–968MathSciNetCrossRefMATHGoogle Scholar
- Le Digabel S (2011) Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans Math Softw 37(4):44:1–44:15MathSciNetCrossRefGoogle Scholar
- Liseikin VD (2009) Grid generation methods. Springer, BerlinMATHGoogle Scholar
- Lucidi S, Sciandrone M (2002) On the global convergence of derivative-free methods for unconstrained optimization. SIAM J Optim 13(1):97–116MathSciNetCrossRefMATHGoogle Scholar
- Marsaglia G (1972) Choosing a point from the surface of a sphere. Ann Math Stat 43(2):645–646CrossRefMATHGoogle Scholar
- Martínez JM, Sobral FNC (2013) Constrained derivative-free optimization on thin domains. J Glob Optim 56(3):1217–1232MathSciNetCrossRefMATHGoogle Scholar
- McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245MathSciNetMATHGoogle Scholar
- Mladenović N, Hansen P (1997) Variable neighborhood search. Comput Oper Res 24(11):1097–1100MathSciNetCrossRefMATHGoogle Scholar
- Moré JJ, Wild SM (2009) Benchmarking derivative-free optimization algorithms. SIAM J Optim 20(1):172–191MathSciNetCrossRefMATHGoogle Scholar
- Nocedal J, Wright SJ (1999) Numerical optimization. Springer Series in Operations Research. Springer, New YorkCrossRefMATHGoogle Scholar
- Papalambros PY, Wilde DJ (2000) Principles of optimal design: modeling and computation, 2nd edn. Cambridge University Press, CambridgeCrossRefMATHGoogle Scholar
- Torczon V (1997) On the convergence of pattern search algorithms. SIAM J Optim 7(1):1–25MathSciNetCrossRefMATHGoogle Scholar
- Tosserams S, Kokkolaras M, Etman LFP, Rooda JE (2010) A non-hierarchical formulation of analytical target cascading. J Mech Des 132(5):051002-1–051002-13CrossRefGoogle Scholar
- Van Dyke B, Asaki TJ (2013) Using QR decomposition to obtain a new instance of mesh adaptive direct search with uniformly distributed polling directions. J Optim Theory Appl 159(3):805–821MathSciNetCrossRefMATHGoogle Scholar
- Van Loan CF (1985) How near is a stable matrix to an unstable matrix. Contemp Math 47:465–478MathSciNetCrossRefMATHGoogle Scholar
- Vicente LN, Custódio AL (2012) Analysis of direct searches for discontinuous functions. Math Programm 133(1–2):299–325MathSciNetCrossRefMATHGoogle Scholar