Advertisement

A Survey on Direct Search Methods for Blackbox Optimization and Their Applications

  • Charles Audet
Chapter

Abstract

Blackbox optimization typically arises when the functions defining the objective and constraints of an optimization problem are computed through a computer simulation. The blackbox is expensive to compute, can have limited precision and can be contaminated with numerical noise. It may also fail to return a valid output, even when the input appears acceptable. Launching twice the simulation from the same input may produce different outputs. These unreliable properties are frequently encountered when dealing with real optimization problems. The term blackbox is used to indicate that the internal structure of the target problem, such as derivatives or their approximations, cannot be exploited as it may be unknown, hidden, unreliable, or inexistent. There are situations where some structure such as bounds or linear constraints may be exploited and in some cases a surrogate of the problem is supplied or a model may be constructed and trusted. This chapter surveys algorithms for this class of problems, including a supporting convergence analysis based on the nonsmooth calculus. The chapter also lists numerous published applications of these methods to real optimization problems.

Keywords

Blackbox optimization Direct search methods Derivative-free optimization Surrogate models Nonsmooth analysis Applications 

Notes

Acknowledgements

This work was supported by NSERC grant 239436 and AFOSR FA9550-12-1-0198.

References

  1. 1.
    Abhishek, K., Leyffer, S., Linderoth, J.T.: Modeling without categorical variables: a mixed-integer nonlinear program for the optimization of thermal insulation systems. Optim. Eng. 11, 185–212 (2010). doi:10.1007/s11081-010-9109-zMathSciNetzbMATHGoogle Scholar
  2. 2.
    Abramson, M.A.: Mixed variable optimization of a load-bearing thermal insulation system using a filter pattern search algorithm. Optim. Eng. 5(2), 157–177 (2004)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Abramson, M.A.: Second-order behavior of pattern search. SIAM J. Optim. 16(2), 315–330 (2005)MathSciNetGoogle Scholar
  4. 4.
    Abramson, M.A., Audet, C.: Convergence of mesh adaptive direct search to second-order stationary points. SIAM J. Optim. 17(2), 606–619 (2006)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Abramson, M.A., Audet, C., Dennis, J.E. Jr.: Filter pattern search algorithms for mixed variable constrained optimization problems. Pac. J. Optim. 3(3), 477–500 (2007)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Abramson, M.A., Brezhneva, O.A., Dennis, J.E. Jr., Pingel, R.L.: Pattern search in the presence of degenerate linear constraints. Optim. Methods Softw. 23(3), 297–319 (2008)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Abramson, M.A., Audet, C., Chrissis, J.W., Walston, J.G.: Mesh adaptive direct search algorithms for mixed variable optimization. Optim. Lett. 3(1), 35–47 (2009)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Abramson, M.A., Audet, C., Dennis, J.E. Jr., Le Digabel, S.: OrthoMADS: A deterministic MADS instance with orthogonal directions. SIAM J. Optim. 20(2), 948–966 (2009)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Abramson, M.A., Audet, C., Couture, G., Dennis, J.E. Jr., Le Digabel, S., Tribes, C.: The NOMAD project (2014). Software available at http://www.gerad.ca/nomad
  10. 10.
    Alarie, S., Audet, C., Garnier, V., Le Digabel, S., Leclaire, L.A.: Snow water equivalent estimation using blackbox optimization. Pac. J. Optim. 9(1), 1–21 (2013)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Alberto, P., Nogueira, F., Rocha, H., Vicente, L.N.: Pattern search methods for user-provided points: Application to molecular geometry problems. SIAM J. Optim. 14(4), 1216–1236 (2004)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Aspentech (2014). http://www.aspentech.com/
  13. 13.
    Audet, C.: Convergence results for pattern search algorithms are tight. Optim. Eng. 5(2), 101–122 (2004)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Audet, C.: A short proof on the cardinality of maximal positive bases. Optim. Lett. 5(1), 191–194 (2011)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Audet, C., Dennis, J.E. Jr.: Pattern search algorithms for mixed variable programming. SIAM J. Optim. 11(3), 573–594 (2001)MathSciNetGoogle Scholar
  16. 16.
    Audet, C., Dennis, J.E. Jr.: Analysis of generalized pattern searches. SIAM J. Optim. 13(3), 889–903 (2003)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Audet, C., Dennis, J.E. Jr.: A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14(4), 980–1010 (2004)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Audet, C., Dennis, J.E. Jr.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Audet, C., Dennis, J.E. Jr.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(4), 445–472 (2009)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Audet, C., Le Digabel, S.: The mesh adaptive direct search algorithm for periodic variables. Pac. J. Optim. 8(1), 103–119 (2012)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Audet, C., Orban, D.: Finding optimal algorithmic parameters using derivative-free optimization. SIAM J. Optim. 17(3), 642–664 (2006)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Audet, C., Béchard, V., Chaouki, J.: Spent potliner treatment process optimization using a MADS algorithm. Optim. Eng. 9(2), 143–160 (2008)zbMATHGoogle Scholar
  23. 23.
    Audet, C., Béchard, V., Le Digabel, S.: Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J. Glob. Optim. 41(2), 299–318 (2008)zbMATHGoogle Scholar
  24. 24.
    Audet, C., Custódio, A.L., Dennis, J.E. Jr.: Erratum: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 18(4), 1501–1503 (2008)Google Scholar
  25. 25.
    Audet, C., Dennis, J.E. Jr., Le Digabel, S.: Parallel space decomposition of the mesh adaptive direct search algorithm. SIAM J. Optim. 19(3), 1150–1170 (2008)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Audet, C., Savard, G., Zghal, W.: Multiobjective optimization through a series of single-objective formulations. SIAM J. Optim. 19(1), 188–210 (2008)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Audet, C., Dang, C.-K., Orban, D.: Algorithmic parameter optimization of the DFO method with the OPAL framework. In: Naono, K., Teranishi, K., Cavazos, J., Suda, R. (eds.) Software Automatic Tuning: From Concepts to State-of-the-Art Results, Chap. 15, pp. 255–274. Springer, Berlin (2010)Google Scholar
  28. 28.
    Audet, C., Dennis, J.E. Jr., Le Digabel, S.: Globalization strategies for mesh adaptive direct search. Comput. Optim. Appl. 46(2), 193–215 (2010)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Audet, C., Savard, G., Zghal, W.: A mesh adaptive direct search algorithm for multiobjective optimization. Eur. J. Oper. Res. 204(3), 545–556 (2010)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Audet, C., Dennis, J.E. Jr., Le Digabel, S.: Trade-off studies in blackbox optimization. Optim. Methods Softw. 27(4–5), 613–624 (2012)MathSciNetzbMATHGoogle Scholar
  31. 31.
    Audet, C., Dang, C.-K., Orban, D.: Efficient use of parallelism in algorithmic parameter optimization applications. Optim. Lett. 7(3), 421–433 (2013)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Balaprakash, P. Wild, S.M., Norris, B.: Spapt: Search problems in automatic performance tuning. Proc. Comput. Sci. 9, 1959–1968 (2012). Proceedings of the International Conference on Computational Science, ICCS (2012)Google Scholar
  33. 33.
    Björkman, M., Holmström, K.: Global optimization of costly nonconvex functions using radial basis functions. Optim. Eng. 1, 373–397 (2000)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Booker, A.J.: Well-conditioned Kriging models for optimization of computer simulations. Technical Report M&CT-TECH-00-002, Boeing Computer Services, Research and Technology, M/S 7L–68, Seattle, Washington 98124 (2000)Google Scholar
  35. 35.
    Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Moore, D.W., Serafini, D.B.: Managing surrogate objectives to optimize a helicopter rotor design – further experiments. AIAA Paper 1998–4717, Presented at the 8th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, St. Louis (1998)Google Scholar
  36. 36.
    Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Serafini, D.B., Torczon, V.:. Optimization using surrogate objectives on a helicopter test example. In: Borggaard, J., Burns, J., Cliff, E., Schreck, S. (eds.) Optimal Design and Control. Progress in Systems and Control Theory, pp. 49–58. Birkhäuser, Cambridge (1998)Google Scholar
  37. 37.
    Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Serafini, D.B., Torczon, V., Trosset, M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Multidiscip. Optim. 17(1), 1–13 (1999)Google Scholar
  38. 38.
    Box, G.E.P.: Evolutionary operation: A method for increasing industrial productivity. Appl. Stat. 6, 81–101 (1957)Google Scholar
  39. 39.
    Caleanu, C.-D., Mao, X., Pradel, G., Moga, S., Xue, Y.: Combined pattern search optimization of feature extraction and classification parameters in facial recognition. Pattern Recognit. Lett. 32(9), 1250–1255 (2011)Google Scholar
  40. 40.
    Chetehouna, K., Sero-Guillaume, O., Sochet, I., Degiovanni, A.: On the experimental determination of flame front positions and of propagation parameters for a fire. Int. J. Therm. Sci. 47(9), 1148–1157 (2008)Google Scholar
  41. 41.
    Choi, T.D., Kelley, C.T.: Superlinear convergence and implicit filtering. SIAM J. Optim. 10(4), 1149–1162 (2000)MathSciNetzbMATHGoogle Scholar
  42. 42.
    Choquette, Y., Lavigne, P., Ducharme, P., Houdayer, A., Martin, J.-P.: Apparatus and Method for Monitoring Snow Water Equivalent and Soil Moisture Content Using Natural Gamma Radiation, September 2010. US Patent No. 7800051 B2Google Scholar
  43. 43.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983). Reissued in 1990 by SIAM Publications, Philadelphia, as Vol. 5 in the series Classics in Applied MathematicsGoogle Scholar
  44. 44.
    Conn, A.R., Le Digabel, S.: Use of quadratic models with mesh-adaptive direct search for constrained black box optimization. Optim. Methods Softw. 28(1), 139–158 (2013)MathSciNetzbMATHGoogle Scholar
  45. 45.
    Conn, A.R., Scheinberg, K., Toint, Ph.L.: On the convergence of derivative-free methods for unconstrained optimization. In: Buhmann, M.D., Iserles, A. (eds.) Approximation Theory and Optimization: Tributes to M.J.D. Powell, pp. 83–108. Cambridge University Press, Cambridge (1997)Google Scholar
  46. 46.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Global convergence of general derivative-free trust-region algorithms to first and second order critical points. SIAM J. Optim. 20(1), 387–415 (2009)MathSciNetzbMATHGoogle Scholar
  47. 47.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MOS/SIAM Series on Optimization. SIAM, Philadelphia (2009)zbMATHGoogle Scholar
  48. 48.
    Conn, A.R., Scheinberg, K., Toint, Ph.L.: DFO (derivative free optimization) (2014). Software available at http://www.coin-or.org
  49. 49.
    Coope, I.D., Price, C.J.: Frame-based methods for unconstrained optimization. J. Optim. Theory Appl. 107(2), 261–274 (2000)MathSciNetzbMATHGoogle Scholar
  50. 50.
    Courbariaux, Y., Chaouki, J., Guy, C.: Update on spent potliners treatments: Kinetics of cyanides destruction at high temperature. Ind. Eng. Chem. Res. 43(18), 5828–5837 (2004)Google Scholar
  51. 51.
    Cramer, E.J., Gablonsky, J.M.: Effective parallel optimization of complex computer simulations. In: Proceedings of the 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference (2004)Google Scholar
  52. 52.
    Custódio, A.L., Madeira, J.F.A., Vaz, A.I.F., Vicente, L.N.: Direct multisearch for multiobjective optimization. SIAM J. Optim. 21(3), 1109–1140 (2011)MathSciNetzbMATHGoogle Scholar
  53. 53.
    Custódio, A.L., Rocha, H., Vicente, L.N.: Incorporating minimum Frobenius norm models in direct search. Comput. Optim. Appl. 46(2), 265–278 (2010)MathSciNetzbMATHGoogle Scholar
  54. 54.
    Das, I., Dennis, J.E. Jr.: Normal-boundary intersection: A new method for generating the pareto surface in nonlinear multicriteria optimization problems. SIAM J. Optim. 8(3), 631–657 (1998)MathSciNetzbMATHGoogle Scholar
  55. 55.
    Davis, C.: Theory of positive linear dependence. Am. J. Math. 76, 733–746 (1954)zbMATHGoogle Scholar
  56. 56.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)Google Scholar
  57. 57.
    Dennis, J.E. Jr., Torczon, V.: Direct search methods on parallel machines. SIAM J. Optim. 1(4), 448–474 (1991)MathSciNetzbMATHGoogle Scholar
  58. 58.
    Dennis, J.E. Jr., Torczon, V.: Managing approximation models in optimization. In: Alexandrov, N.M., Hussaini, M.Y. (eds.) Multidisciplinary Design Optimization: State of the Art, pp. 330–347. SIAM, Philadelphia (1997)Google Scholar
  59. 59.
    Dennis, J.E. Jr., Woods, D.J.: Optimization on microcomputers: The Nelder–Mead simplex algorithm. In: Wouk, A. (ed.) New Computing Environments: Microcomputers in Large-Scale Computing, pp. 116–122. Society for Industrial and Applied Mathematics, Philadelphia (1987)Google Scholar
  60. 60.
    Dennis, J.E. Jr., Wu, Z.: Parallel Continuous Optimization. Sourcebook of Parallel Computing, pp. 649–670. Morgan Kaufmann, San Francisco (2003)Google Scholar
  61. 61.
    Dennis, J.E. Jr., Price, C.J., Coope, I.D.: Direct search methods for nonlinearly constrained optimization using filters and frames. Optim. Eng. 5(2), 123–144 (2004)MathSciNetzbMATHGoogle Scholar
  62. 62.
    Dolan, E.D., Lewis, R.M., Torczon, V.: On the local convergence of pattern search. SIAM J. Optim. 14(2), 567–583 (2003)MathSciNetzbMATHGoogle Scholar
  63. 63.
    FactSage (2014). http://www.factsage.com/
  64. 64.
    Fermi, E., Metropolis, N.: Numerical solution of a minimum problem. Los Alamos Unclassified Report LA–1492, Los Alamos National Laboratory, Los Alamos (1952)Google Scholar
  65. 65.
    Ferris, M.C., Mangasarian, O.L.: Parallel variable distribution. SIAM J. Optim. 4(4), 815–832 (1994)MathSciNetzbMATHGoogle Scholar
  66. 66.
    Finkel, D.E., Kelley, C.T.: Convergence analysis of the DIRECT algorithm. Technical Report CRSC-TR04-28, Center for Research in Scientific Computation (2004)Google Scholar
  67. 67.
    Finkel, D.E., Kelley, C.T.: Convergence analysis of sampling methods for perturbed Lipschitz functions. Pac. J. Optim. 5(2), 339–350 (2009)MathSciNetzbMATHGoogle Scholar
  68. 68.
    Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. Ser. A 91, 239–269 (2002)MathSciNetzbMATHGoogle Scholar
  69. 69.
    Fletcher, R., Leyffer, S., Toint, Ph.L.: On the global convergence of an SLP–filter algorithm. Technical Report NA/183, Department of Mathematics, Dundee University (1998)Google Scholar
  70. 70.
    Fourer, R., Orban, D.: Dr. Ampl – a meta-solver for optimization problem analysis. Comput. Manag. Sci. 7(4), 437–463 (2009)Google Scholar
  71. 71.
    Fowler, K.R., Kelley, C.T., Miller, C.T., Kees, C.E., Darwin, R.W., Reese, J.P., Farthing, M.W., Reed, M.S.C.: Solution of a well-field design problem with implicit filtering. Optim. Eng. 5(2), 207–234 (2004)MathSciNetzbMATHGoogle Scholar
  72. 72.
    Fowler, K.R., Reese, J.P., Kees, C.E., Dennis, J.E. Jr., Kelley, C.T., Miller, C.T., Audet, C., Booker, A.J., Couture, G., Darwin, R.W., Farthing, M.W., Finkel, D.E., Gablonsky, J.M., Gray, G., Kolda, T.G.: Comparison of derivative-free optimization methods for groundwater supply and hydraulic capture community problems. Adv. Water Resour. 31(5), 743–757 (2008)Google Scholar
  73. 73.
    Gablonsky, J.M., Kelley, C.T.: A locally-biased form of the DIRECT algorithm. J. Glob. Optim. 21, 27–37 (2001)MathSciNetzbMATHGoogle Scholar
  74. 74.
    García-Palomares, U.M., Rodríguez, J.F.: New sequential and parallel derivative-free algorithms for unconstrained optimization. SIAM J. Optim. 13(1), 79–96 (2002)MathSciNetzbMATHGoogle Scholar
  75. 75.
    Gheribi, A.E., Robelin, C., Le Digabel, S., Audet, C., Pelton, A.D.: Calculating all local minima on liquidus surfaces using the factsage software and databases and the mesh adaptive direct search algorithm. J. Chem. Thermodyn. 43(9), 1323–1330 (2011)Google Scholar
  76. 76.
    Gheribi, A.E., Audet, C., Le Digabel, S., Bélisle, E., Bale, C.W., Pelton, A.D.: Calculating optimal conditions for alloy and process design using thermodynamic and properties databases, the FactSage software and the Mesh Adaptive Direct Search algorithm. CALPHAD 36, 135–143 (2012)Google Scholar
  77. 77.
    Gilmore, P., Kelley, C.T.: An implicit filtering algorithm for optimization of functions with many local minima. SIAM J. Optim. 5(2), 269–285 (1995)MathSciNetzbMATHGoogle Scholar
  78. 78.
    Gould, N.I.M., Orban, D., Toint, Ph.L.: CUTEr (and SifDec): A constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)MathSciNetzbMATHGoogle Scholar
  79. 79.
    Gould, N.I.M., Orban, D., Sartenaer, A., Toint, Ph.L.: Sensitivity of trust-region algorithms on their parameters. 4OR 3(3), 227–241 (2005)Google Scholar
  80. 80.
    Gramacy, R.B., Le Digabel, S.: The mesh adaptive direct search algorithm with treed Gaussian process surrogates. Technical Report G-2011-37, Les cahiers du GERAD (2011)Google Scholar
  81. 81.
    Gramacy, R.B., Taddy, M.A., Wild, S.M.: Variable selection and sensitivity analysis via dynamic trees with an application to computer code performance tuning. Technical Report 1108.4739, arXiv, August 2011Google Scholar
  82. 82.
    Gray, G.A., Kolda, T.G.: Algorithm 856: APPSPACK 4.0: Asynchronous parallel pattern search for derivative-free optimization. ACM Trans. Math. Softw. 32(3), 485–507 (2006)Google Scholar
  83. 83.
    Griffin, J.D., Kolda, T.G.: Nonlinearly-constrained optimization using heuristic penalty methods and asynchronous parallel generating set search. Appl. Math. Res. Express 25(5), 36–62 (2010)MathSciNetGoogle Scholar
  84. 84.
    Griffin, J.D., Kolda, T.G., Lewis, R.M.: Asynchronous parallel generating set search for linearly-constrained optimization. SIAM J. Sci. Comput. 30(4), 1892–1924 (2008)MathSciNetGoogle Scholar
  85. 85.
    Hansen, P., Mladenović, N.: Variable neighborhood search: principles and applications. Eur. J. Oper. Res. 130(3), 449–467 (2001)zbMATHGoogle Scholar
  86. 86.
    Hare, W.L., Macklem, M.: Derivative-free optimization methods for finite minimax problems. Optim. Methods Softw. 28(2), 300–312 (2013)MathSciNetzbMATHGoogle Scholar
  87. 87.
    Hayes, R.E., Bertrand, F.H., Audet, C., Kolaczkowski, S.T.: Catalytic combustion kinetics: Using a direct search algorithm to evaluate kinetic parameters from light-off curves. Canad. J. Chem. Eng. 81(6), 1192–1199 (2003)Google Scholar
  88. 88.
    Higham, N.J.: Optimization by direct search in matrix computations. SIAM J. Matrix Anal. Appl. 14, 317–333 (1993)MathSciNetzbMATHGoogle Scholar
  89. 89.
    Hock, W., Schittkowski, K.: Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems, vol. 187. Springer, Berlin (1981)Google Scholar
  90. 90.
    Hooke, R., Jeeves, T.A.: Direct search solution of numerical and statistical problems. J. Assoc. Comput. Mach. 8(2), 212–229 (1961)zbMATHGoogle Scholar
  91. 91.
    Hough, P.D., Kolda, T.G., Torczon, V.: Asynchronous parallel pattern search for nonlinear optimization. SIAM J. Sci. Comput. 23(1), 134–156 (2001)MathSciNetzbMATHGoogle Scholar
  92. 92.
    Jahn, J.: Vector Optimization: Theory, Applications, and Extensions. Springer, Berlin (2004)Google Scholar
  93. 93.
    Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79(1), 157–181 (1993)MathSciNetzbMATHGoogle Scholar
  94. 94.
    Kelley, C.T.: Iterative Methods for Optimization. Frontiers in Applied Mathematics, vol. 18. SIAM, Philadelphia (1999)Google Scholar
  95. 95.
    Kokkolaras, M., Audet, C., Dennis, J.E. Jr.: Mixed variable optimization of the number and composition of heat intercepts in a thermal insulation system. Optim. Eng. 2(1), 5–29 (2001)MathSciNetzbMATHGoogle Scholar
  96. 96.
    Kolda, T.G.: Revisiting asynchronous parallel pattern search for nonlinear optimization. SIAM J. Optim. 16(2), 563–586 (2005)MathSciNetzbMATHGoogle Scholar
  97. 97.
    Kolda, T.G., Torczon, V.: On the convergence of asynchronous parallel pattern search. SIAM J. Optim. 14(4), 939–964 (2004)MathSciNetzbMATHGoogle Scholar
  98. 98.
    Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: New perspectives on some classical and modern methods. SIAM Rev. 45(3), 385–482 (2003)MathSciNetzbMATHGoogle Scholar
  99. 99.
    Kolda, T.G., Lewis, R.M., Torczon, V.: A generating set direct search augmented Lagrangian algorithm for optimization with a combination of general and linear constraints. Technical Report SAND2006-5315, Sandia National Laboratories, USA (2006)Google Scholar
  100. 100.
    Kolda, T.G., Lewis, R.M., Torczon, V.: Stationarity results for generating set search for linearly constrained optimization. SIAM J. Optim. 17(4), 943–968 (2006)MathSciNetzbMATHGoogle Scholar
  101. 101.
    Le Digabel, S.: Algorithm 909: NOMAD: Nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37(4), 44:1–44:15 (2011)Google Scholar
  102. 102.
    Audet, C., Le Digabel, S., Tribes, C.: NOMAD user guide. Technical Report G-2009-37, Les cahiers du GERAD (2009)Google Scholar
  103. 103.
    Leach, E.B.: A note on inverse function theorem. In: Proceedings of the AMS, vol. 12, pp. 694–697 (1961)MathSciNetzbMATHGoogle Scholar
  104. 104.
    Lewis, R.M., Torczon, V.: Rank ordering and positive bases in pattern search algorithms. Technical Report 96–71, Institute for Computer Applications in Science and Engineering, Mail Stop 132C, pp. 23681–2199. NASA Langley Research Center, Hampton, VA (1996)Google Scholar
  105. 105.
    Lewis, R.M., Torczon, V.: Pattern search algorithms for bound constrained minimization. SIAM J. Optim. 9(4), 1082–1099 (1999)MathSciNetzbMATHGoogle Scholar
  106. 106.
    Lewis, R.M., Torczon, V.: Pattern search methods for linearly constrained minimization. SIAM J. Optim. 10(3), 917–941 (2000)MathSciNetzbMATHGoogle Scholar
  107. 107.
    Lewis, R.M., Torczon, V.: A globally convergent augmented Lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Optim. 12(4), 1075–1089 (2002)MathSciNetzbMATHGoogle Scholar
  108. 108.
    Lewis, R.M., Shepherd, A., Torczon, V.: Implementing generating set search methods for linearly constrained optimization. SIAM J. Sci. Comput. 29(6), 2507–2530 (2007)MathSciNetzbMATHGoogle Scholar
  109. 109.
    Li, M., Li, Y.F., Wu, Y.J., Cai, S., Zhu, N.Y., Rezzak, N., Schrimpf, R.D., Fleetwood, D.M., Wang, J.Q., Cheng, X.X., Wang, Y., Wang, D.L., Hao, Y.: Including Radiation Effects and Dependencies on Process-Related Variability in Advanced Foundry SPICE Models Using a New Physical Model and Parameter Extraction Approach. IEEE Trans. Nuclear Sci. 58(6, Part 1), 2876–2882 (2011). IEEE Radiation Effects Data Workshop (REDW)/48th IEEE International Nuclear and Space Radiation Effects Conference (NSREC), pp. 25–29, Las Vegas (2011)Google Scholar
  110. 110.
    Liuzzi, G., Lucidi, S., Sciandrone, M.: A derivative-free algorithm for linearly constrained finite minimax problems. SIAM J. Optim. 16(4), 1054–1075 (2006)MathSciNetzbMATHGoogle Scholar
  111. 111.
    Lophaven, S., Nielsen, H., Søondergaard, J.: Dace: A matlab kriging toolbox version 2.0. Technical Report IMM-REP-2002-12, Informatics and Mathematical Modelling, Technical University of Denmark (2002)Google Scholar
  112. 112.
    Lucidi, S., Sciandrone, M.: On the global convergence of derivative-free methods for unconstrained optimization. SIAM J. Optim. 13, 97–116 (2002)MathSciNetzbMATHGoogle Scholar
  113. 113.
    Marsden, A.L.: Aerodynamic noise control by optimal shape design. Ph.D. thesis, Stanford University (2004)Google Scholar
  114. 114.
    Marsden, A.L., Wang, M., Dennis, J.E. Jr., Moin, P.: Optimal aeroacoustic shape design using the surrogate management framework. Optim. Eng. 5(2), 235–262 (2004)MathSciNetzbMATHGoogle Scholar
  115. 115.
    Marsden, A.L., Wang, M., Dennis, J.E. Jr., Moin, P.: Suppression of airfoil vortex-shedding noise via derivative-free optimization. Phys. Fluids 16(10), L83–L86 (2004)Google Scholar
  116. 116.
    Marsden, A.L., Wang, M., Dennis, J.E. Jr., Moin, P.: Trailing-edge noise reduction using derivative-free optimization and large-eddy simulation. J. Fluid Mech. 572, 13–36 (2007)MathSciNetzbMATHGoogle Scholar
  117. 117.
    Marsden, A.L., Feinstein, J.A., Taylor, C.A.: A computational framework for derivative-free optimization of cardiovascular geometries. Comput. Methods Appl. Mech. Eng. 197(21–24), 1890–1905 (2008)MathSciNetzbMATHGoogle Scholar
  118. 118.
    Martin, J.-P., Houdayer, A., Lebel, C., Choquette, Y., Lavigne, P., Ducharme, P.: An unattended gamma monitor for the determination of snow water equivalent (SWE) using the natural ground gamma radiation. In: Nuclear Science Symposium Conference Record, pp. 983–988. IEEE (2008)Google Scholar
  119. 119.
    Marty, A.: Optimisation du placement et de l’assignation de fréquence d’antennes dans un réseau de télécommunications. Master’s thesis, École Polytechnique de Montréal (2011)Google Scholar
  120. 120.
    McKinnon, K.I.M.: Convergence of the Nelder-Mead simplex method to a nonstationary point. SIAM J. Optim. 9, 148–158 (1998)MathSciNetzbMATHGoogle Scholar
  121. 121.
    Mladenović, N., Hansen, P.: Variable neighborhood search. Comput. Oper. Res. 24(11), 1097–1100 (1997)MathSciNetzbMATHGoogle Scholar
  122. 122.
    Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7(4), 308–313 (1965)zbMATHGoogle Scholar
  123. 123.
    Oeuvray, R., Bierlaire, M.: BOOSTERS: A derivative-free algorithm based on radial basis functions. Int. J. Model. Simul. 29(1), 26–36 (2009)Google Scholar
  124. 124.
    Orban, D.: Templating and automatic code generation for performance with Python. Technical Report G-2011-30, Les cahiers du GERAD (2011)Google Scholar
  125. 125.
    Polak, E., Wetter, M.: Precision control for generalized pattern search algorithms with adaptive precision function evaluations. SIAM J. Optim. 16(3), 650–669 (2006)MathSciNetzbMATHGoogle Scholar
  126. 126.
    Prechtel, M., Leugering, G., Steinmann, P., Stingl, M.: Towards optimization of crack resistance of composite materials by adjustment of fiber shapes. Eng. Fract. Mech. 78(6), 944–960 (2011)Google Scholar
  127. 127.
    Price, C.J., Coope, I.D.: Frames and grids in unconstrained and linearly constrained optimization: A nonsmooth approach. SIAM J. Optim. 14, 415–438 (2003)MathSciNetzbMATHGoogle Scholar
  128. 128.
    Regis, R.G., Shoemaker, C.A.: Constrained global optimization of expensive black box functions using radial basis functions. J. Glob. Optim. 31, 153–171 (2005)MathSciNetzbMATHGoogle Scholar
  129. 129.
    Renaud, E., Robelin, C., Gheribi, A.E., Chartrand, P.: Thermodynamic evaluation and optimization of the Li, Na, K, Mg, Ca, Sr // F, Cl reciprocal system. J. Chem. Thermodyn. 43(8), 1286–1298 (2011)Google Scholar
  130. 130.
    Rockafellar, R.T.: Generalized directional derivatives and subgradients of nonconvex functions. Canad. J. Math. 32(2), 257–280 (1980)MathSciNetzbMATHGoogle Scholar
  131. 131.
    Sankaran, S., Marsden, A.L.: The impact of uncertainty on shape optimization of idealized bypass graft models in unsteady flow. Phys. Fluids 22(12), 121902 (2010)Google Scholar
  132. 132.
    Sankaran, S., Audet, C., Marsden, A.L.: A method for stochastic constrained optimization using derivative-free surrogate pattern search and collocation. J. Comput. Phys. 229(12), 4664–4682 (2010)zbMATHGoogle Scholar
  133. 133.
    Serafini, D.B.: A framework for managing models in nonlinear optimization of computationally expensive functions. Ph.D. thesis, Department of Computational and Applied Mathematics, Rice University (1998)Google Scholar
  134. 134.
    Søndergaard, J.: Optimization using surrogate models—by the space mapping technique. Ph.D. thesis, Informatics and Mathematical Modelling, Technical University of Denmark (2003)Google Scholar
  135. 135.
    Spillane, M.C., Gica, E., Titov, V.V.: Tsunameter Network Design for the U.S. DART Array. AGU Fall Meeting Abstracts, p. A1368 (December 2009)Google Scholar
  136. 136.
    Stephens, C.P., Baritompa, W.: Global optimization requires global information. J. Optim. Theory Appl. 96, 575–588 (1998)MathSciNetzbMATHGoogle Scholar
  137. 137.
    Sun, J., Salvucci, G.D., Entekhabi, D., Farhadi, L.: Parameter estimation of coupled water and energy balance models based on stationary constraints of surface states. Water Resour. Res. 47, 1–16 (2011)Google Scholar
  138. 138.
    Sweatlock, L.A., Diest, K., Marthaler, D.E.: Metamaterials design using gradient-free numerical optimization. J. Appl. Phys. 108(8), 1–5 (2010)Google Scholar
  139. 139.
    Tang, B.: Orthogonal array-based Latin hypercubes. J. Am. Stat. Assoc. 88(424), 1392–1397 (1993)zbMATHGoogle Scholar
  140. 140.
    Torczon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7(1), 1–25 (1997)MathSciNetzbMATHGoogle Scholar
  141. 141.
    Torres, R., Bès, C., Chaptal, J., Hiriart-Urruty, J.-B.: Optimal, environmentally-friendly departure procedures for civil aircraft. J. Aircr. 48(1), 11–22 (2011)Google Scholar
  142. 142.
    Vicente, L.N., Custódio, A.L.: Analysis of direct searches for discontinuous functions. Math. Program. 133(1–2), 299–325 (2012)MathSciNetzbMATHGoogle Scholar
  143. 143.
    Wild, S.M., Shoemaker, C.A.: Global convergence of radial basis function trust region derivative-free algorithms. SIAM J. Optim. 21(3), 761–781 (2011)MathSciNetzbMATHGoogle Scholar
  144. 144.
    Yang, W., Feinstein, J.A., Marsden, A.L.: Constrained optimization of an idealized y-shaped baffle for the fontan surgery at rest and exercise. Comput. Methods Appl. Mech. Eng. 199(33–36), 2135–2149 (2010)zbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Département de Mathématiques et de Génie IndustrielÉcole Polytechnique de Montréal and GERADMontréalCanada

Personalised recommendations