Skip to main content
SpringerLink
Log in

A Survey on Direct Search Methods for Blackbox Optimization and Their Applications

  • Chapter
  • First Online:
Mathematics Without Boundaries

Abstract

Blackbox optimization typically arises when the functions defining the objective and constraints of an optimization problem are computed through a computer simulation. The blackbox is expensive to compute, can have limited precision and can be contaminated with numerical noise. It may also fail to return a valid output, even when the input appears acceptable. Launching twice the simulation from the same input may produce different outputs. These unreliable properties are frequently encountered when dealing with real optimization problems. The term blackbox is used to indicate that the internal structure of the target problem, such as derivatives or their approximations, cannot be exploited as it may be unknown, hidden, unreliable, or inexistent. There are situations where some structure such as bounds or linear constraints may be exploited and in some cases a surrogate of the problem is supplied or a model may be constructed and trusted. This chapter surveys algorithms for this class of problems, including a supporting convergence analysis based on the nonsmooth calculus. The chapter also lists numerous published applications of these methods to real optimization problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abhishek, K., Leyffer, S., Linderoth, J.T.: Modeling without categorical variables: a mixed-integer nonlinear program for the optimization of thermal insulation systems. Optim. Eng. 11, 185–212 (2010). doi:10.1007/s11081-010-9109-z

    MathSciNet  MATH  Google Scholar 

  2. Abramson, M.A.: Mixed variable optimization of a load-bearing thermal insulation system using a filter pattern search algorithm. Optim. Eng. 5(2), 157–177 (2004)

    MathSciNet  MATH  Google Scholar 

  3. Abramson, M.A.: Second-order behavior of pattern search. SIAM J. Optim. 16(2), 315–330 (2005)

    MathSciNet  Google Scholar 

  4. Abramson, M.A., Audet, C.: Convergence of mesh adaptive direct search to second-order stationary points. SIAM J. Optim. 17(2), 606–619 (2006)

    MathSciNet  MATH  Google Scholar 

  5. Abramson, M.A., Audet, C., Dennis, J.E. Jr.: Filter pattern search algorithms for mixed variable constrained optimization problems. Pac. J. Optim. 3(3), 477–500 (2007)

    MathSciNet  MATH  Google Scholar 

  6. Abramson, M.A., Brezhneva, O.A., Dennis, J.E. Jr., Pingel, R.L.: Pattern search in the presence of degenerate linear constraints. Optim. Methods Softw. 23(3), 297–319 (2008)

    MathSciNet  MATH  Google Scholar 

  7. Abramson, M.A., Audet, C., Chrissis, J.W., Walston, J.G.: Mesh adaptive direct search algorithms for mixed variable optimization. Optim. Lett. 3(1), 35–47 (2009)

    MathSciNet  MATH  Google Scholar 

  8. Abramson, M.A., Audet, C., Dennis, J.E. Jr., Le Digabel, S.: OrthoMADS: A deterministic MADS instance with orthogonal directions. SIAM J. Optim. 20(2), 948–966 (2009)

    MathSciNet  MATH  Google Scholar 

  9. Abramson, M.A., Audet, C., Couture, G., Dennis, J.E. Jr., Le Digabel, S., Tribes, C.: The NOMAD project (2014). Software available at http://www.gerad.ca/nomad

  10. Alarie, S., Audet, C., Garnier, V., Le Digabel, S., Leclaire, L.A.: Snow water equivalent estimation using blackbox optimization. Pac. J. Optim. 9(1), 1–21 (2013)

    MathSciNet  MATH  Google Scholar 

  11. Alberto, P., Nogueira, F., Rocha, H., Vicente, L.N.: Pattern search methods for user-provided points: Application to molecular geometry problems. SIAM J. Optim. 14(4), 1216–1236 (2004)

    MathSciNet  MATH  Google Scholar 

  12. Aspentech (2014). http://www.aspentech.com/

  13. Audet, C.: Convergence results for pattern search algorithms are tight. Optim. Eng. 5(2), 101–122 (2004)

    MathSciNet  MATH  Google Scholar 

  14. Audet, C.: A short proof on the cardinality of maximal positive bases. Optim. Lett. 5(1), 191–194 (2011)

    MathSciNet  MATH  Google Scholar 

  15. Audet, C., Dennis, J.E. Jr.: Pattern search algorithms for mixed variable programming. SIAM J. Optim. 11(3), 573–594 (2001)

    MathSciNet  Google Scholar 

  16. Audet, C., Dennis, J.E. Jr.: Analysis of generalized pattern searches. SIAM J. Optim. 13(3), 889–903 (2003)

    MathSciNet  MATH  Google Scholar 

  17. Audet, C., Dennis, J.E. Jr.: A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14(4), 980–1010 (2004)

    MathSciNet  MATH  Google Scholar 

  18. Audet, C., Dennis, J.E. Jr.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)

    MathSciNet  MATH  Google Scholar 

  19. Audet, C., Dennis, J.E. Jr.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(4), 445–472 (2009)

    MathSciNet  MATH  Google Scholar 

  20. Audet, C., Le Digabel, S.: The mesh adaptive direct search algorithm for periodic variables. Pac. J. Optim. 8(1), 103–119 (2012)

    MathSciNet  MATH  Google Scholar 

  21. Audet, C., Orban, D.: Finding optimal algorithmic parameters using derivative-free optimization. SIAM J. Optim. 17(3), 642–664 (2006)

    MathSciNet  MATH  Google Scholar 

  22. Audet, C., Béchard, V., Chaouki, J.: Spent potliner treatment process optimization using a MADS algorithm. Optim. Eng. 9(2), 143–160 (2008)

    MATH  Google Scholar 

  23. Audet, C., Béchard, V., Le Digabel, S.: Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J. Glob. Optim. 41(2), 299–318 (2008)

    MATH  Google Scholar 

  24. Audet, C., Custódio, A.L., Dennis, J.E. Jr.: Erratum: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 18(4), 1501–1503 (2008)

    Google Scholar 

  25. Audet, C., Dennis, J.E. Jr., Le Digabel, S.: Parallel space decomposition of the mesh adaptive direct search algorithm. SIAM J. Optim. 19(3), 1150–1170 (2008)

    MathSciNet  MATH  Google Scholar 

  26. Audet, C., Savard, G., Zghal, W.: Multiobjective optimization through a series of single-objective formulations. SIAM J. Optim. 19(1), 188–210 (2008)

    MathSciNet  MATH  Google Scholar 

  27. Audet, C., Dang, C.-K., Orban, D.: Algorithmic parameter optimization of the DFO method with the OPAL framework. In: Naono, K., Teranishi, K., Cavazos, J., Suda, R. (eds.) Software Automatic Tuning: From Concepts to State-of-the-Art Results, Chap. 15, pp. 255–274. Springer, Berlin (2010)

    Google Scholar 

  28. Audet, C., Dennis, J.E. Jr., Le Digabel, S.: Globalization strategies for mesh adaptive direct search. Comput. Optim. Appl. 46(2), 193–215 (2010)

    MathSciNet  MATH  Google Scholar 

  29. Audet, C., Savard, G., Zghal, W.: A mesh adaptive direct search algorithm for multiobjective optimization. Eur. J. Oper. Res. 204(3), 545–556 (2010)

    MathSciNet  MATH  Google Scholar 

  30. Audet, C., Dennis, J.E. Jr., Le Digabel, S.: Trade-off studies in blackbox optimization. Optim. Methods Softw. 27(4–5), 613–624 (2012)

    MathSciNet  MATH  Google Scholar 

  31. Audet, C., Dang, C.-K., Orban, D.: Efficient use of parallelism in algorithmic parameter optimization applications. Optim. Lett. 7(3), 421–433 (2013)

    MathSciNet  MATH  Google Scholar 

  32. Balaprakash, P. Wild, S.M., Norris, B.: Spapt: Search problems in automatic performance tuning. Proc. Comput. Sci. 9, 1959–1968 (2012). Proceedings of the International Conference on Computational Science, ICCS (2012)

    Google Scholar 

  33. Björkman, M., Holmström, K.: Global optimization of costly nonconvex functions using radial basis functions. Optim. Eng. 1, 373–397 (2000)

    MathSciNet  MATH  Google Scholar 

  34. Booker, A.J.: Well-conditioned Kriging models for optimization of computer simulations. Technical Report M&CT-TECH-00-002, Boeing Computer Services, Research and Technology, M/S 7L–68, Seattle, Washington 98124 (2000)

    Google Scholar 

  35. Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Moore, D.W., Serafini, D.B.: Managing surrogate objectives to optimize a helicopter rotor design – further experiments. AIAA Paper 1998–4717, Presented at the 8th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, St. Louis (1998)

    Google Scholar 

  36. Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Serafini, D.B., Torczon, V.:. Optimization using surrogate objectives on a helicopter test example. In: Borggaard, J., Burns, J., Cliff, E., Schreck, S. (eds.) Optimal Design and Control. Progress in Systems and Control Theory, pp. 49–58. Birkhäuser, Cambridge (1998)

    Google Scholar 

  37. Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Serafini, D.B., Torczon, V., Trosset, M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Multidiscip. Optim. 17(1), 1–13 (1999)

    Google Scholar 

  38. Box, G.E.P.: Evolutionary operation: A method for increasing industrial productivity. Appl. Stat. 6, 81–101 (1957)

    Google Scholar 

  39. Caleanu, C.-D., Mao, X., Pradel, G., Moga, S., Xue, Y.: Combined pattern search optimization of feature extraction and classification parameters in facial recognition. Pattern Recognit. Lett. 32(9), 1250–1255 (2011)

    Google Scholar 

  40. Chetehouna, K., Sero-Guillaume, O., Sochet, I., Degiovanni, A.: On the experimental determination of flame front positions and of propagation parameters for a fire. Int. J. Therm. Sci. 47(9), 1148–1157 (2008)

    Google Scholar 

  41. Choi, T.D., Kelley, C.T.: Superlinear convergence and implicit filtering. SIAM J. Optim. 10(4), 1149–1162 (2000)

    MathSciNet  MATH  Google Scholar 

  42. Choquette, Y., Lavigne, P., Ducharme, P., Houdayer, A., Martin, J.-P.: Apparatus and Method for Monitoring Snow Water Equivalent and Soil Moisture Content Using Natural Gamma Radiation, September 2010. US Patent No. 7800051 B2

    Google Scholar 

  43. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983). Reissued in 1990 by SIAM Publications, Philadelphia, as Vol. 5 in the series Classics in Applied Mathematics

    Google Scholar 

  44. Conn, A.R., Le Digabel, S.: Use of quadratic models with mesh-adaptive direct search for constrained black box optimization. Optim. Methods Softw. 28(1), 139–158 (2013)

    MathSciNet  MATH  Google Scholar 

  45. Conn, A.R., Scheinberg, K., Toint, Ph.L.: On the convergence of derivative-free methods for unconstrained optimization. In: Buhmann, M.D., Iserles, A. (eds.) Approximation Theory and Optimization: Tributes to M.J.D. Powell, pp. 83–108. Cambridge University Press, Cambridge (1997)

    Google Scholar 

  46. Conn, A.R., Scheinberg, K., Vicente, L.N.: Global convergence of general derivative-free trust-region algorithms to first and second order critical points. SIAM J. Optim. 20(1), 387–415 (2009)

    MathSciNet  MATH  Google Scholar 

  47. Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MOS/SIAM Series on Optimization. SIAM, Philadelphia (2009)

    MATH  Google Scholar 

  48. Conn, A.R., Scheinberg, K., Toint, Ph.L.: DFO (derivative free optimization) (2014). Software available at http://www.coin-or.org

  49. Coope, I.D., Price, C.J.: Frame-based methods for unconstrained optimization. J. Optim. Theory Appl. 107(2), 261–274 (2000)

    MathSciNet  MATH  Google Scholar 

  50. Courbariaux, Y., Chaouki, J., Guy, C.: Update on spent potliners treatments: Kinetics of cyanides destruction at high temperature. Ind. Eng. Chem. Res. 43(18), 5828–5837 (2004)

    Google Scholar 

  51. Cramer, E.J., Gablonsky, J.M.: Effective parallel optimization of complex computer simulations. In: Proceedings of the 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference (2004)

    Google Scholar 

  52. Custódio, A.L., Madeira, J.F.A., Vaz, A.I.F., Vicente, L.N.: Direct multisearch for multiobjective optimization. SIAM J. Optim. 21(3), 1109–1140 (2011)

    MathSciNet  MATH  Google Scholar 

  53. Custódio, A.L., Rocha, H., Vicente, L.N.: Incorporating minimum Frobenius norm models in direct search. Comput. Optim. Appl. 46(2), 265–278 (2010)

    MathSciNet  MATH  Google Scholar 

  54. Das, I., Dennis, J.E. Jr.: Normal-boundary intersection: A new method for generating the pareto surface in nonlinear multicriteria optimization problems. SIAM J. Optim. 8(3), 631–657 (1998)

    MathSciNet  MATH  Google Scholar 

  55. Davis, C.: Theory of positive linear dependence. Am. J. Math. 76, 733–746 (1954)

    MATH  Google Scholar 

  56. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)

    Google Scholar 

  57. Dennis, J.E. Jr., Torczon, V.: Direct search methods on parallel machines. SIAM J. Optim. 1(4), 448–474 (1991)

    MathSciNet  MATH  Google Scholar 

  58. Dennis, J.E. Jr., Torczon, V.: Managing approximation models in optimization. In: Alexandrov, N.M., Hussaini, M.Y. (eds.) Multidisciplinary Design Optimization: State of the Art, pp. 330–347. SIAM, Philadelphia (1997)

    Google Scholar 

  59. Dennis, J.E. Jr., Woods, D.J.: Optimization on microcomputers: The Nelder–Mead simplex algorithm. In: Wouk, A. (ed.) New Computing Environments: Microcomputers in Large-Scale Computing, pp. 116–122. Society for Industrial and Applied Mathematics, Philadelphia (1987)

    Google Scholar 

  60. Dennis, J.E. Jr., Wu, Z.: Parallel Continuous Optimization. Sourcebook of Parallel Computing, pp. 649–670. Morgan Kaufmann, San Francisco (2003)

    Google Scholar 

  61. Dennis, J.E. Jr., Price, C.J., Coope, I.D.: Direct search methods for nonlinearly constrained optimization using filters and frames. Optim. Eng. 5(2), 123–144 (2004)

    MathSciNet  MATH  Google Scholar 

  62. Dolan, E.D., Lewis, R.M., Torczon, V.: On the local convergence of pattern search. SIAM J. Optim. 14(2), 567–583 (2003)

    MathSciNet  MATH  Google Scholar 

  63. FactSage (2014). http://www.factsage.com/

  64. Fermi, E., Metropolis, N.: Numerical solution of a minimum problem. Los Alamos Unclassified Report LA–1492, Los Alamos National Laboratory, Los Alamos (1952)

    Google Scholar 

  65. Ferris, M.C., Mangasarian, O.L.: Parallel variable distribution. SIAM J. Optim. 4(4), 815–832 (1994)

    MathSciNet  MATH  Google Scholar 

  66. Finkel, D.E., Kelley, C.T.: Convergence analysis of the DIRECT algorithm. Technical Report CRSC-TR04-28, Center for Research in Scientific Computation (2004)

    Google Scholar 

  67. Finkel, D.E., Kelley, C.T.: Convergence analysis of sampling methods for perturbed Lipschitz functions. Pac. J. Optim. 5(2), 339–350 (2009)

    MathSciNet  MATH  Google Scholar 

  68. Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. Ser. A 91, 239–269 (2002)

    MathSciNet  MATH  Google Scholar 

  69. Fletcher, R., Leyffer, S., Toint, Ph.L.: On the global convergence of an SLP–filter algorithm. Technical Report NA/183, Department of Mathematics, Dundee University (1998)

    Google Scholar 

  70. Fourer, R., Orban, D.: Dr. Ampl – a meta-solver for optimization problem analysis. Comput. Manag. Sci. 7(4), 437–463 (2009)

    Google Scholar 

  71. Fowler, K.R., Kelley, C.T., Miller, C.T., Kees, C.E., Darwin, R.W., Reese, J.P., Farthing, M.W., Reed, M.S.C.: Solution of a well-field design problem with implicit filtering. Optim. Eng. 5(2), 207–234 (2004)

    MathSciNet  MATH  Google Scholar 

  72. Fowler, K.R., Reese, J.P., Kees, C.E., Dennis, J.E. Jr., Kelley, C.T., Miller, C.T., Audet, C., Booker, A.J., Couture, G., Darwin, R.W., Farthing, M.W., Finkel, D.E., Gablonsky, J.M., Gray, G., Kolda, T.G.: Comparison of derivative-free optimization methods for groundwater supply and hydraulic capture community problems. Adv. Water Resour. 31(5), 743–757 (2008)

    Google Scholar 

  73. Gablonsky, J.M., Kelley, C.T.: A locally-biased form of the DIRECT algorithm. J. Glob. Optim. 21, 27–37 (2001)

    MathSciNet  MATH  Google Scholar 

  74. García-Palomares, U.M., Rodríguez, J.F.: New sequential and parallel derivative-free algorithms for unconstrained optimization. SIAM J. Optim. 13(1), 79–96 (2002)

    MathSciNet  MATH  Google Scholar 

  75. Gheribi, A.E., Robelin, C., Le Digabel, S., Audet, C., Pelton, A.D.: Calculating all local minima on liquidus surfaces using the factsage software and databases and the mesh adaptive direct search algorithm. J. Chem. Thermodyn. 43(9), 1323–1330 (2011)

    Google Scholar 

  76. Gheribi, A.E., Audet, C., Le Digabel, S., Bélisle, E., Bale, C.W., Pelton, A.D.: Calculating optimal conditions for alloy and process design using thermodynamic and properties databases, the FactSage software and the Mesh Adaptive Direct Search algorithm. CALPHAD 36, 135–143 (2012)

    Google Scholar 

  77. Gilmore, P., Kelley, C.T.: An implicit filtering algorithm for optimization of functions with many local minima. SIAM J. Optim. 5(2), 269–285 (1995)

    MathSciNet  MATH  Google Scholar 

  78. Gould, N.I.M., Orban, D., Toint, Ph.L.: CUTEr (and SifDec): A constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)

    MathSciNet  MATH  Google Scholar 

  79. Gould, N.I.M., Orban, D., Sartenaer, A., Toint, Ph.L.: Sensitivity of trust-region algorithms on their parameters. 4OR 3(3), 227–241 (2005)

    Google Scholar 

  80. Gramacy, R.B., Le Digabel, S.: The mesh adaptive direct search algorithm with treed Gaussian process surrogates. Technical Report G-2011-37, Les cahiers du GERAD (2011)

    Google Scholar 

  81. Gramacy, R.B., Taddy, M.A., Wild, S.M.: Variable selection and sensitivity analysis via dynamic trees with an application to computer code performance tuning. Technical Report 1108.4739, arXiv, August 2011

    Google Scholar 

  82. Gray, G.A., Kolda, T.G.: Algorithm 856: APPSPACK 4.0: Asynchronous parallel pattern search for derivative-free optimization. ACM Trans. Math. Softw. 32(3), 485–507 (2006)

    Google Scholar 

  83. Griffin, J.D., Kolda, T.G.: Nonlinearly-constrained optimization using heuristic penalty methods and asynchronous parallel generating set search. Appl. Math. Res. Express 25(5), 36–62 (2010)

    MathSciNet  Google Scholar 

  84. Griffin, J.D., Kolda, T.G., Lewis, R.M.: Asynchronous parallel generating set search for linearly-constrained optimization. SIAM J. Sci. Comput. 30(4), 1892–1924 (2008)

    MathSciNet  Google Scholar 

  85. Hansen, P., Mladenović, N.: Variable neighborhood search: principles and applications. Eur. J. Oper. Res. 130(3), 449–467 (2001)

    MATH  Google Scholar 

  86. Hare, W.L., Macklem, M.: Derivative-free optimization methods for finite minimax problems. Optim. Methods Softw. 28(2), 300–312 (2013)

    MathSciNet  MATH  Google Scholar 

  87. Hayes, R.E., Bertrand, F.H., Audet, C., Kolaczkowski, S.T.: Catalytic combustion kinetics: Using a direct search algorithm to evaluate kinetic parameters from light-off curves. Canad. J. Chem. Eng. 81(6), 1192–1199 (2003)

    Google Scholar 

  88. Higham, N.J.: Optimization by direct search in matrix computations. SIAM J. Matrix Anal. Appl. 14, 317–333 (1993)

    MathSciNet  MATH  Google Scholar 

  89. Hock, W., Schittkowski, K.: Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems, vol. 187. Springer, Berlin (1981)

    Google Scholar 

  90. Hooke, R., Jeeves, T.A.: Direct search solution of numerical and statistical problems. J. Assoc. Comput. Mach. 8(2), 212–229 (1961)

    MATH  Google Scholar 

  91. Hough, P.D., Kolda, T.G., Torczon, V.: Asynchronous parallel pattern search for nonlinear optimization. SIAM J. Sci. Comput. 23(1), 134–156 (2001)

    MathSciNet  MATH  Google Scholar 

  92. Jahn, J.: Vector Optimization: Theory, Applications, and Extensions. Springer, Berlin (2004)

    Google Scholar 

  93. Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79(1), 157–181 (1993)

    MathSciNet  MATH  Google Scholar 

  94. Kelley, C.T.: Iterative Methods for Optimization. Frontiers in Applied Mathematics, vol. 18. SIAM, Philadelphia (1999)

    Google Scholar 

  95. Kokkolaras, M., Audet, C., Dennis, J.E. Jr.: Mixed variable optimization of the number and composition of heat intercepts in a thermal insulation system. Optim. Eng. 2(1), 5–29 (2001)

    MathSciNet  MATH  Google Scholar 

  96. Kolda, T.G.: Revisiting asynchronous parallel pattern search for nonlinear optimization. SIAM J. Optim. 16(2), 563–586 (2005)

    MathSciNet  MATH  Google Scholar 

  97. Kolda, T.G., Torczon, V.: On the convergence of asynchronous parallel pattern search. SIAM J. Optim. 14(4), 939–964 (2004)

    MathSciNet  MATH  Google Scholar 

  98. Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: New perspectives on some classical and modern methods. SIAM Rev. 45(3), 385–482 (2003)

    MathSciNet  MATH  Google Scholar 

  99. Kolda, T.G., Lewis, R.M., Torczon, V.: A generating set direct search augmented Lagrangian algorithm for optimization with a combination of general and linear constraints. Technical Report SAND2006-5315, Sandia National Laboratories, USA (2006)

    Google Scholar 

  100. Kolda, T.G., Lewis, R.M., Torczon, V.: Stationarity results for generating set search for linearly constrained optimization. SIAM J. Optim. 17(4), 943–968 (2006)

    MathSciNet  MATH  Google Scholar 

  101. Le Digabel, S.: Algorithm 909: NOMAD: Nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37(4), 44:1–44:15 (2011)

    Google Scholar 

  102. Audet, C., Le Digabel, S., Tribes, C.: NOMAD user guide. Technical Report G-2009-37, Les cahiers du GERAD (2009)

    Google Scholar 

  103. Leach, E.B.: A note on inverse function theorem. In: Proceedings of the AMS, vol. 12, pp. 694–697 (1961)

    MathSciNet  MATH  Google Scholar 

  104. Lewis, R.M., Torczon, V.: Rank ordering and positive bases in pattern search algorithms. Technical Report 96–71, Institute for Computer Applications in Science and Engineering, Mail Stop 132C, pp. 23681–2199. NASA Langley Research Center, Hampton, VA (1996)

    Google Scholar 

  105. Lewis, R.M., Torczon, V.: Pattern search algorithms for bound constrained minimization. SIAM J. Optim. 9(4), 1082–1099 (1999)

    MathSciNet  MATH  Google Scholar 

  106. Lewis, R.M., Torczon, V.: Pattern search methods for linearly constrained minimization. SIAM J. Optim. 10(3), 917–941 (2000)

    MathSciNet  MATH  Google Scholar 

  107. Lewis, R.M., Torczon, V.: A globally convergent augmented Lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Optim. 12(4), 1075–1089 (2002)

    MathSciNet  MATH  Google Scholar 

  108. Lewis, R.M., Shepherd, A., Torczon, V.: Implementing generating set search methods for linearly constrained optimization. SIAM J. Sci. Comput. 29(6), 2507–2530 (2007)

    MathSciNet  MATH  Google Scholar 

  109. Li, M., Li, Y.F., Wu, Y.J., Cai, S., Zhu, N.Y., Rezzak, N., Schrimpf, R.D., Fleetwood, D.M., Wang, J.Q., Cheng, X.X., Wang, Y., Wang, D.L., Hao, Y.: Including Radiation Effects and Dependencies on Process-Related Variability in Advanced Foundry SPICE Models Using a New Physical Model and Parameter Extraction Approach. IEEE Trans. Nuclear Sci. 58(6, Part 1), 2876–2882 (2011). IEEE Radiation Effects Data Workshop (REDW)/48th IEEE International Nuclear and Space Radiation Effects Conference (NSREC), pp. 25–29, Las Vegas (2011)

    Google Scholar 

  110. Liuzzi, G., Lucidi, S., Sciandrone, M.: A derivative-free algorithm for linearly constrained finite minimax problems. SIAM J. Optim. 16(4), 1054–1075 (2006)

    MathSciNet  MATH  Google Scholar 

  111. Lophaven, S., Nielsen, H., Søondergaard, J.: Dace: A matlab kriging toolbox version 2.0. Technical Report IMM-REP-2002-12, Informatics and Mathematical Modelling, Technical University of Denmark (2002)

    Google Scholar 

  112. Lucidi, S., Sciandrone, M.: On the global convergence of derivative-free methods for unconstrained optimization. SIAM J. Optim. 13, 97–116 (2002)

    MathSciNet  MATH  Google Scholar 

  113. Marsden, A.L.: Aerodynamic noise control by optimal shape design. Ph.D. thesis, Stanford University (2004)

    Google Scholar 

  114. Marsden, A.L., Wang, M., Dennis, J.E. Jr., Moin, P.: Optimal aeroacoustic shape design using the surrogate management framework. Optim. Eng. 5(2), 235–262 (2004)

    MathSciNet  MATH  Google Scholar 

  115. Marsden, A.L., Wang, M., Dennis, J.E. Jr., Moin, P.: Suppression of airfoil vortex-shedding noise via derivative-free optimization. Phys. Fluids 16(10), L83–L86 (2004)

    Google Scholar 

  116. Marsden, A.L., Wang, M., Dennis, J.E. Jr., Moin, P.: Trailing-edge noise reduction using derivative-free optimization and large-eddy simulation. J. Fluid Mech. 572, 13–36 (2007)

    MathSciNet  MATH  Google Scholar 

  117. Marsden, A.L., Feinstein, J.A., Taylor, C.A.: A computational framework for derivative-free optimization of cardiovascular geometries. Comput. Methods Appl. Mech. Eng. 197(21–24), 1890–1905 (2008)

    MathSciNet  MATH  Google Scholar 

  118. Martin, J.-P., Houdayer, A., Lebel, C., Choquette, Y., Lavigne, P., Ducharme, P.: An unattended gamma monitor for the determination of snow water equivalent (SWE) using the natural ground gamma radiation. In: Nuclear Science Symposium Conference Record, pp. 983–988. IEEE (2008)

    Google Scholar 

  119. Marty, A.: Optimisation du placement et de l’assignation de fréquence d’antennes dans un réseau de télécommunications. Master’s thesis, École Polytechnique de Montréal (2011)

    Google Scholar 

  120. McKinnon, K.I.M.: Convergence of the Nelder-Mead simplex method to a nonstationary point. SIAM J. Optim. 9, 148–158 (1998)

    MathSciNet  MATH  Google Scholar 

  121. Mladenović, N., Hansen, P.: Variable neighborhood search. Comput. Oper. Res. 24(11), 1097–1100 (1997)

    MathSciNet  MATH  Google Scholar 

  122. Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7(4), 308–313 (1965)

    MATH  Google Scholar 

  123. Oeuvray, R., Bierlaire, M.: BOOSTERS: A derivative-free algorithm based on radial basis functions. Int. J. Model. Simul. 29(1), 26–36 (2009)

    Google Scholar 

  124. Orban, D.: Templating and automatic code generation for performance with Python. Technical Report G-2011-30, Les cahiers du GERAD (2011)

    Google Scholar 

  125. Polak, E., Wetter, M.: Precision control for generalized pattern search algorithms with adaptive precision function evaluations. SIAM J. Optim. 16(3), 650–669 (2006)

    MathSciNet  MATH  Google Scholar 

  126. Prechtel, M., Leugering, G., Steinmann, P., Stingl, M.: Towards optimization of crack resistance of composite materials by adjustment of fiber shapes. Eng. Fract. Mech. 78(6), 944–960 (2011)

    Google Scholar 

  127. Price, C.J., Coope, I.D.: Frames and grids in unconstrained and linearly constrained optimization: A nonsmooth approach. SIAM J. Optim. 14, 415–438 (2003)

    MathSciNet  MATH  Google Scholar 

  128. Regis, R.G., Shoemaker, C.A.: Constrained global optimization of expensive black box functions using radial basis functions. J. Glob. Optim. 31, 153–171 (2005)

    MathSciNet  MATH  Google Scholar 

  129. Renaud, E., Robelin, C., Gheribi, A.E., Chartrand, P.: Thermodynamic evaluation and optimization of the Li, Na, K, Mg, Ca, Sr // F, Cl reciprocal system. J. Chem. Thermodyn. 43(8), 1286–1298 (2011)

    Google Scholar 

  130. Rockafellar, R.T.: Generalized directional derivatives and subgradients of nonconvex functions. Canad. J. Math. 32(2), 257–280 (1980)

    MathSciNet  MATH  Google Scholar 

  131. Sankaran, S., Marsden, A.L.: The impact of uncertainty on shape optimization of idealized bypass graft models in unsteady flow. Phys. Fluids 22(12), 121902 (2010)

    Google Scholar 

  132. Sankaran, S., Audet, C., Marsden, A.L.: A method for stochastic constrained optimization using derivative-free surrogate pattern search and collocation. J. Comput. Phys. 229(12), 4664–4682 (2010)

    MATH  Google Scholar 

  133. Serafini, D.B.: A framework for managing models in nonlinear optimization of computationally expensive functions. Ph.D. thesis, Department of Computational and Applied Mathematics, Rice University (1998)

    Google Scholar 

  134. Søndergaard, J.: Optimization using surrogate models—by the space mapping technique. Ph.D. thesis, Informatics and Mathematical Modelling, Technical University of Denmark (2003)

    Google Scholar 

  135. Spillane, M.C., Gica, E., Titov, V.V.: Tsunameter Network Design for the U.S. DART Array. AGU Fall Meeting Abstracts, p. A1368 (December 2009)

    Google Scholar 

  136. Stephens, C.P., Baritompa, W.: Global optimization requires global information. J. Optim. Theory Appl. 96, 575–588 (1998)

    MathSciNet  MATH  Google Scholar 

  137. Sun, J., Salvucci, G.D., Entekhabi, D., Farhadi, L.: Parameter estimation of coupled water and energy balance models based on stationary constraints of surface states. Water Resour. Res. 47, 1–16 (2011)

    Google Scholar 

  138. Sweatlock, L.A., Diest, K., Marthaler, D.E.: Metamaterials design using gradient-free numerical optimization. J. Appl. Phys. 108(8), 1–5 (2010)

    Google Scholar 

  139. Tang, B.: Orthogonal array-based Latin hypercubes. J. Am. Stat. Assoc. 88(424), 1392–1397 (1993)

    MATH  Google Scholar 

  140. Torczon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7(1), 1–25 (1997)

    MathSciNet  MATH  Google Scholar 

  141. Torres, R., Bès, C., Chaptal, J., Hiriart-Urruty, J.-B.: Optimal, environmentally-friendly departure procedures for civil aircraft. J. Aircr. 48(1), 11–22 (2011)

    Google Scholar 

  142. Vicente, L.N., Custódio, A.L.: Analysis of direct searches for discontinuous functions. Math. Program. 133(1–2), 299–325 (2012)

    MathSciNet  MATH  Google Scholar 

  143. Wild, S.M., Shoemaker, C.A.: Global convergence of radial basis function trust region derivative-free algorithms. SIAM J. Optim. 21(3), 761–781 (2011)

    MathSciNet  MATH  Google Scholar 

  144. Yang, W., Feinstein, J.A., Marsden, A.L.: Constrained optimization of an idealized y-shaped baffle for the fontan surgery at rest and exercise. Comput. Methods Appl. Mech. Eng. 199(33–36), 2135–2149 (2010)

    MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by NSERC grant 239436 and AFOSR FA9550-12-1-0198.

Author information

Authors and Affiliations

  1. Département de Mathématiques et de Génie Industriel, École Polytechnique de Montréal and GERAD, C.P. 6079, Succ. Centre-ville, Montréal, QC, Canada, H3C 3A7

    Charles Audet

Authors
  1. Charles Audet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles Audet .

Editor information

Editors and Affiliations

  1. Dept. Industrial & Systems Engineering, University of Florida, Gainesville, Florida, USA

    Panos M. Pardalos

  2. Department of Mathematics, National Technical University of Athens, Athens, Greece

    Themistocles M. Rassias

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media New York

About this chapter

Cite this chapter

Audet, C. (2014). A Survey on Direct Search Methods for Blackbox Optimization and Their Applications. In: Pardalos, P., Rassias, T. (eds) Mathematics Without Boundaries. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-1124-0_2

Download citation

Publish with us

Policies and ethics

Search

Navigation