Optimization Letters

, Volume 12, Issue 4, pp 675–689 | Cite as

Robust optimization of noisy blackbox problems using the Mesh Adaptive Direct Search algorithm

  • Charles Audet
  • Amina Ihaddadene
  • Sébastien Le DigabelEmail author
  • Christophe Tribes
Original Paper


Blackbox optimization problems are often contaminated with numerical noise, and direct search methods such as the Mesh Adaptive Direct Search (MADS) algorithm may get stuck at solutions artificially created by the noise. We propose a way to smooth out the objective function of an unconstrained problem using previously evaluated function evaluations, rather than resampling points. The new algorithm, called Robust-MADS is applied to a collection of noisy analytical problems from the literature and on an optimization problem to tune the parameters of a trust-region method.


Robust optimization Direct search Blackbox optimization MADS 



This work was supported in part by FRQNT Grant 2015-PR-182098 and NSERC Grant RDCPJ 490744-15 in collaboration with Rio Tinto and Hydro-Québec.


  1. 1.
    Abramson, M.A., Audet, C., Dennis Jr., J.E., Le Digabel, S.: OrthoMADS: a deterministic MADS instance with orthogonal directions. SIAM J. Optim. 20(2), 948–966 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Audet, C., Dennis Jr., J.E.: Analysis of generalized pattern searches. SIAM J. Optim. 13(3), 889–903 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Audet, C., Dennis Jr., J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Audet, C., Dennis Jr., J.E.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(1), 445–472 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Audet, C., Dennis Jr., J.E., Le Digabel, S.: Parallel space decomposition of the mesh adaptive direct search algorithm. SIAM J. Optim. 19(3), 1150–1170 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Audet, C., Ianni, A., Le Digabel, S., Tribes, C.: Reducing the number of function evaluations in Mesh Adaptive Direct Search algorithms. SIAM J. Optim. 24(2), 621–642 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Audet, C., Orban, D.: Finding optimal algorithmic parameters using derivative-free optimization. SIAM J. Optim. 17(3), 642–664 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Billups, S.C., Larson, J., Graf, P.: Derivative-free optimization of expensive functions with computational error using weighted regression. SIAM J. Optim. 23(1), 27–53 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Huyer, W., Neumaier, A.: SNOBFIT—stable noisy optimization by branch and fit. ACM Trans. Math. Softw. 35(2), 9:1–9:25 (2008)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Chen, R., Menickelly, M., Scheinberg, K.: Stochastic optimization using a trust-region method and random models. Math. Program, 1–41 (2016)Google Scholar
  11. 11.
    Choi, T.D., Kelley, C.T.: Superlinear convergence and implicit filtering. SIAM J. Optim. 10(4), 1149–1162 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Deng, G., Ferris, M.C.: Adaptation of the UOBYQA algorithm for noisy functions. In: Proceedings of the 38th Conference on Winter Simulation, WSC ’06, pp. 312–319. Winter Simulation Conference (2006)Google Scholar
  13. 13.
    Yang, D., Flockton, S.J.: Evolutionary algorithms with a coarse-to-fine function smoothing. IEEE Int. Conf. Evol. Comput. 2, 657–662 (1995)Google Scholar
  14. 14.
    Elster, C., Neumaier, A.: A grid algorithm for bound constrained optimization of noisy functions. IMA J. Numer. Anal. 15(4), 585–608 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Epanechnikov, V.A.: Non-parametric estimation of a multivariate probability density. Theory Probab. Appl. 14(1), 153–158 (1969)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Gould, N.I.M., Orban, D., Toint, Ph.L.: CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60(3), 545–557 (2015).
  17. 17.
    Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79(1), 157–181 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Kostrowicki, J., Piela, L., Cherayil, B.J., Scheraga, H.A.: Performance of the diffusion equation method in searches for optimum structures of clusters of Lennard–Jones atoms. J. Phys. Chem. 95(10), 4113–4119 (1991)CrossRefGoogle Scholar
  19. 19.
    Larson, J., Billups, S.C.: Stochastic derivative-free optimization using a trust region framework. Comput. Optim. Appl. 64(3), 619–645 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Le Digabel, S.: Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37(4), 44:1–44:15 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Le Digabel, S., Wild, S.M.: A Taxonomy of Constraints in Simulation-Based Optimization. Technical Report G-2015-57, Les cahiers du GERAD (2015)Google Scholar
  22. 22.
    Li, J., Wu, C., Wu, Z., Long, Q.: Gradient-free method for nonsmooth distributed optimization. J. Global Optim. 61(2), 325–340 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Liu, Q., Zeng, J., Yang, G.: MrDIRECT: a multilevel robust DIRECT algorithm for global optimization problems. J. Global Optim. 62(2), 205–227 (2015)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Moré, J.J., Wild, S.M.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20(1), 172–191 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Powell, M.J.D.: UOBYQA: unconstrained optimization by quadratic approximation. Math. Program. 92(3), 555–582 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Selvan, S.E., Borckmans, P.B., Chattopadhyay, A., Absil, P.-A.: Spherical mesh adaptive direct search for separating quasi-uncorrelated sources by range-based independent component analysis. Neural Comput. 25(9), 2486–2522 (2013)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Shao, C.-S., Byrd, R.H., Eskow, E., Schnabel, R.B.: Global optimization for molecular clusters using a new smoothing approach. In: Biegler, L., Lorenz, T., Conn, A.R., Coleman, T.F., Santosa, F.N. (eds.) Large-Scale Optimization with Applications, Volume 94 of The IMA Volumes in Mathematics and its Applications, pp. 163–199. Springer, New York (1997)CrossRefGoogle Scholar
  28. 28.
    Sriver, T.A., Chrissis, J.W., Abramson, M.A.: Pattern search ranking and selection algorithms for mixed variable simulation-based optimization. Eur. J. Oper. Res. 198(3), 878–890 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Van Dyke, B., Asaki, T.J.: Using QR decomposition to obtain a new instance of Mesh Adaptive Direct Search with uniformly distributed polling directions. J. Optim. Theory Appl. 159(3), 805–821 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Wei, F., Wang, Y., Meng, Z.: A smoothing function method with uniform design for global optimization. Pac. J. Optim. 10(2), 385–399 (2014)MathSciNetzbMATHGoogle Scholar
  31. 31.
    Wu, Z.: The effective energy transformation scheme as a special continuation approach to global optimization with application to molecular conformation. SIAM J. Optim. 6(3), 748–768 (1996)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Département de mathématiques et génie industriel, École Polytechnique de MontréalGERADMontrealCanada

Personalised recommendations