A Survey on Direct Search Methods for Blackbox Optimization and Their Applications

Chapter

Abstract

Blackbox optimization typically arises when the functions defining the objective and constraints of an optimization problem are computed through a computer simulation. The blackbox is expensive to compute, can have limited precision and can be contaminated with numerical noise. It may also fail to return a valid output, even when the input appears acceptable. Launching twice the simulation from the same input may produce different outputs. These unreliable properties are frequently encountered when dealing with real optimization problems. The term blackbox is used to indicate that the internal structure of the target problem, such as derivatives or their approximations, cannot be exploited as it may be unknown, hidden, unreliable, or inexistent. There are situations where some structure such as bounds or linear constraints may be exploited and in some cases a surrogate of the problem is supplied or a model may be constructed and trusted. This chapter surveys algorithms for this class of problems, including a supporting convergence analysis based on the nonsmooth calculus. The chapter also lists numerous published applications of these methods to real optimization problems.

Keywords

Blackbox optimization Direct search methods Derivative-free optimization Surrogate models Nonsmooth analysis Applications 

Notes

Acknowledgements

This work was supported by NSERC grant 239436 and AFOSR FA9550-12-1-0198.

References

  1. 1.
    Abhishek, K., Leyffer, S., Linderoth, J.T.: Modeling without categorical variables: a mixed-integer nonlinear program for the optimization of thermal insulation systems. Optim. Eng. 11, 185–212 (2010). doi:10.1007/s11081-010-9109-zMathSciNetMATHGoogle Scholar
  2. 2.
    Abramson, M.A.: Mixed variable optimization of a load-bearing thermal insulation system using a filter pattern search algorithm. Optim. Eng. 5(2), 157–177 (2004)MathSciNetMATHGoogle Scholar
  3. 3.
    Abramson, M.A.: Second-order behavior of pattern search. SIAM J. Optim. 16(2), 315–330 (2005)MathSciNetGoogle Scholar
  4. 4.
    Abramson, M.A., Audet, C.: Convergence of mesh adaptive direct search to second-order stationary points. SIAM J. Optim. 17(2), 606–619 (2006)MathSciNetMATHGoogle Scholar
  5. 5.
    Abramson, M.A., Audet, C., Dennis, J.E. Jr.: Filter pattern search algorithms for mixed variable constrained optimization problems. Pac. J. Optim. 3(3), 477–500 (2007)MathSciNetMATHGoogle Scholar
  6. 6.
    Abramson, M.A., Brezhneva, O.A., Dennis, J.E. Jr., Pingel, R.L.: Pattern search in the presence of degenerate linear constraints. Optim. Methods Softw. 23(3), 297–319 (2008)MathSciNetMATHGoogle Scholar
  7. 7.
    Abramson, M.A., Audet, C., Chrissis, J.W., Walston, J.G.: Mesh adaptive direct search algorithms for mixed variable optimization. Optim. Lett. 3(1), 35–47 (2009)MathSciNetMATHGoogle Scholar
  8. 8.
    Abramson, M.A., Audet, C., Dennis, J.E. Jr., Le Digabel, S.: OrthoMADS: A deterministic MADS instance with orthogonal directions. SIAM J. Optim. 20(2), 948–966 (2009)MathSciNetMATHGoogle Scholar
  9. 9.
    Abramson, M.A., Audet, C., Couture, G., Dennis, J.E. Jr., Le Digabel, S., Tribes, C.: The NOMAD project (2014). Software available at http://www.gerad.ca/nomad
  10. 10.
    Alarie, S., Audet, C., Garnier, V., Le Digabel, S., Leclaire, L.A.: Snow water equivalent estimation using blackbox optimization. Pac. J. Optim. 9(1), 1–21 (2013)MathSciNetMATHGoogle Scholar
  11. 11.
    Alberto, P., Nogueira, F., Rocha, H., Vicente, L.N.: Pattern search methods for user-provided points: Application to molecular geometry problems. SIAM J. Optim. 14(4), 1216–1236 (2004)MathSciNetMATHGoogle Scholar
  12. 12.
    Aspentech (2014). http://www.aspentech.com/
  13. 13.
    Audet, C.: Convergence results for pattern search algorithms are tight. Optim. Eng. 5(2), 101–122 (2004)MathSciNetMATHGoogle Scholar
  14. 14.
    Audet, C.: A short proof on the cardinality of maximal positive bases. Optim. Lett. 5(1), 191–194 (2011)MathSciNetMATHGoogle Scholar
  15. 15.
    Audet, C., Dennis, J.E. Jr.: Pattern search algorithms for mixed variable programming. SIAM J. Optim. 11(3), 573–594 (2001)MathSciNetGoogle Scholar
  16. 16.
    Audet, C., Dennis, J.E. Jr.: Analysis of generalized pattern searches. SIAM J. Optim. 13(3), 889–903 (2003)MathSciNetMATHGoogle Scholar
  17. 17.
    Audet, C., Dennis, J.E. Jr.: A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14(4), 980–1010 (2004)MathSciNetMATHGoogle Scholar
  18. 18.
    Audet, C., Dennis, J.E. Jr.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)MathSciNetMATHGoogle Scholar
  19. 19.
    Audet, C., Dennis, J.E. Jr.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(4), 445–472 (2009)MathSciNetMATHGoogle Scholar
  20. 20.
    Audet, C., Le Digabel, S.: The mesh adaptive direct search algorithm for periodic variables. Pac. J. Optim. 8(1), 103–119 (2012)MathSciNetMATHGoogle Scholar
  21. 21.
    Audet, C., Orban, D.: Finding optimal algorithmic parameters using derivative-free optimization. SIAM J. Optim. 17(3), 642–664 (2006)MathSciNetMATHGoogle Scholar
  22. 22.
    Audet, C., Béchard, V., Chaouki, J.: Spent potliner treatment process optimization using a MADS algorithm. Optim. Eng. 9(2), 143–160 (2008)MATHGoogle Scholar
  23. 23.
    Audet, C., Béchard, V., Le Digabel, S.: Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J. Glob. Optim. 41(2), 299–318 (2008)MATHGoogle Scholar
  24. 24.
    Audet, C., Custódio, A.L., Dennis, J.E. Jr.: Erratum: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 18(4), 1501–1503 (2008)Google Scholar
  25. 25.
    Audet, C., Dennis, J.E. Jr., Le Digabel, S.: Parallel space decomposition of the mesh adaptive direct search algorithm. SIAM J. Optim. 19(3), 1150–1170 (2008)MathSciNetMATHGoogle Scholar
  26. 26.
    Audet, C., Savard, G., Zghal, W.: Multiobjective optimization through a series of single-objective formulations. SIAM J. Optim. 19(1), 188–210 (2008)MathSciNetMATHGoogle Scholar
  27. 27.
    Audet, C., Dang, C.-K., Orban, D.: Algorithmic parameter optimization of the DFO method with the OPAL framework. In: Naono, K., Teranishi, K., Cavazos, J., Suda, R. (eds.) Software Automatic Tuning: From Concepts to State-of-the-Art Results, Chap. 15, pp. 255–274. Springer, Berlin (2010)Google Scholar
  28. 28.
    Audet, C., Dennis, J.E. Jr., Le Digabel, S.: Globalization strategies for mesh adaptive direct search. Comput. Optim. Appl. 46(2), 193–215 (2010)MathSciNetMATHGoogle Scholar
  29. 29.
    Audet, C., Savard, G., Zghal, W.: A mesh adaptive direct search algorithm for multiobjective optimization. Eur. J. Oper. Res. 204(3), 545–556 (2010)MathSciNetMATHGoogle Scholar
  30. 30.
    Audet, C., Dennis, J.E. Jr., Le Digabel, S.: Trade-off studies in blackbox optimization. Optim. Methods Softw. 27(4–5), 613–624 (2012)MathSciNetMATHGoogle Scholar
  31. 31.
    Audet, C., Dang, C.-K., Orban, D.: Efficient use of parallelism in algorithmic parameter optimization applications. Optim. Lett. 7(3), 421–433 (2013)MathSciNetMATHGoogle Scholar
  32. 32.
    Balaprakash, P. Wild, S.M., Norris, B.: Spapt: Search problems in automatic performance tuning. Proc. Comput. Sci. 9, 1959–1968 (2012). Proceedings of the International Conference on Computational Science, ICCS (2012)Google Scholar
  33. 33.
    Björkman, M., Holmström, K.: Global optimization of costly nonconvex functions using radial basis functions. Optim. Eng. 1, 373–397 (2000)MathSciNetMATHGoogle Scholar
  34. 34.
    Booker, A.J.: Well-conditioned Kriging models for optimization of computer simulations. Technical Report M&CT-TECH-00-002, Boeing Computer Services, Research and Technology, M/S 7L–68, Seattle, Washington 98124 (2000)Google Scholar
  35. 35.
    Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Moore, D.W., Serafini, D.B.: Managing surrogate objectives to optimize a helicopter rotor design – further experiments. AIAA Paper 1998–4717, Presented at the 8th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, St. Louis (1998)Google Scholar
  36. 36.
    Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Serafini, D.B., Torczon, V.:. Optimization using surrogate objectives on a helicopter test example. In: Borggaard, J., Burns, J., Cliff, E., Schreck, S. (eds.) Optimal Design and Control. Progress in Systems and Control Theory, pp. 49–58. Birkhäuser, Cambridge (1998)Google Scholar
  37. 37.
    Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Serafini, D.B., Torczon, V., Trosset, M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Multidiscip. Optim. 17(1), 1–13 (1999)Google Scholar
  38. 38.
    Box, G.E.P.: Evolutionary operation: A method for increasing industrial productivity. Appl. Stat. 6, 81–101 (1957)Google Scholar
  39. 39.
    Caleanu, C.-D., Mao, X., Pradel, G., Moga, S., Xue, Y.: Combined pattern search optimization of feature extraction and classification parameters in facial recognition. Pattern Recognit. Lett. 32(9), 1250–1255 (2011)Google Scholar
  40. 40.
    Chetehouna, K., Sero-Guillaume, O., Sochet, I., Degiovanni, A.: On the experimental determination of flame front positions and of propagation parameters for a fire. Int. J. Therm. Sci. 47(9), 1148–1157 (2008)Google Scholar
  41. 41.
    Choi, T.D., Kelley, C.T.: Superlinear convergence and implicit filtering. SIAM J. Optim. 10(4), 1149–1162 (2000)MathSciNetMATHGoogle Scholar
  42. 42.
    Choquette, Y., Lavigne, P., Ducharme, P., Houdayer, A., Martin, J.-P.: Apparatus and Method for Monitoring Snow Water Equivalent and Soil Moisture Content Using Natural Gamma Radiation, September 2010. US Patent No. 7800051 B2Google Scholar
  43. 43.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983). Reissued in 1990 by SIAM Publications, Philadelphia, as Vol. 5 in the series Classics in Applied MathematicsGoogle Scholar
  44. 44.
    Conn, A.R., Le Digabel, S.: Use of quadratic models with mesh-adaptive direct search for constrained black box optimization. Optim. Methods Softw. 28(1), 139–158 (2013)MathSciNetMATHGoogle Scholar
  45. 45.
    Conn, A.R., Scheinberg, K., Toint, Ph.L.: On the convergence of derivative-free methods for unconstrained optimization. In: Buhmann, M.D., Iserles, A. (eds.) Approximation Theory and Optimization: Tributes to M.J.D. Powell, pp. 83–108. Cambridge University Press, Cambridge (1997)Google Scholar
  46. 46.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Global convergence of general derivative-free trust-region algorithms to first and second order critical points. SIAM J. Optim. 20(1), 387–415 (2009)MathSciNetMATHGoogle Scholar
  47. 47.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MOS/SIAM Series on Optimization. SIAM, Philadelphia (2009)MATHGoogle Scholar
  48. 48.
    Conn, A.R., Scheinberg, K., Toint, Ph.L.: DFO (derivative free optimization) (2014). Software available at http://www.coin-or.org
  49. 49.
    Coope, I.D., Price, C.J.: Frame-based methods for unconstrained optimization. J. Optim. Theory Appl. 107(2), 261–274 (2000)MathSciNetMATHGoogle Scholar
  50. 50.
    Courbariaux, Y., Chaouki, J., Guy, C.: Update on spent potliners treatments: Kinetics of cyanides destruction at high temperature. Ind. Eng. Chem. Res. 43(18), 5828–5837 (2004)Google Scholar
  51. 51.
    Cramer, E.J., Gablonsky, J.M.: Effective parallel optimization of complex computer simulations. In: Proceedings of the 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference (2004)Google Scholar
  52. 52.
    Custódio, A.L., Madeira, J.F.A., Vaz, A.I.F., Vicente, L.N.: Direct multisearch for multiobjective optimization. SIAM J. Optim. 21(3), 1109–1140 (2011)MathSciNetMATHGoogle Scholar
  53. 53.
    Custódio, A.L., Rocha, H., Vicente, L.N.: Incorporating minimum Frobenius norm models in direct search. Comput. Optim. Appl. 46(2), 265–278 (2010)MathSciNetMATHGoogle Scholar
  54. 54.
    Das, I., Dennis, J.E. Jr.: Normal-boundary intersection: A new method for generating the pareto surface in nonlinear multicriteria optimization problems. SIAM J. Optim. 8(3), 631–657 (1998)MathSciNetMATHGoogle Scholar
  55. 55.
    Davis, C.: Theory of positive linear dependence. Am. J. Math. 76, 733–746 (1954)MATHGoogle Scholar
  56. 56.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)Google Scholar
  57. 57.
    Dennis, J.E. Jr., Torczon, V.: Direct search methods on parallel machines. SIAM J. Optim. 1(4), 448–474 (1991)MathSciNetMATHGoogle Scholar
  58. 58.
    Dennis, J.E. Jr., Torczon, V.: Managing approximation models in optimization. In: Alexandrov, N.M., Hussaini, M.Y. (eds.) Multidisciplinary Design Optimization: State of the Art, pp. 330–347. SIAM, Philadelphia (1997)Google Scholar
  59. 59.
    Dennis, J.E. Jr., Woods, D.J.: Optimization on microcomputers: The Nelder–Mead simplex algorithm. In: Wouk, A. (ed.) New Computing Environments: Microcomputers in Large-Scale Computing, pp. 116–122. Society for Industrial and Applied Mathematics, Philadelphia (1987)Google Scholar
  60. 60.
    Dennis, J.E. Jr., Wu, Z.: Parallel Continuous Optimization. Sourcebook of Parallel Computing, pp. 649–670. Morgan Kaufmann, San Francisco (2003)Google Scholar
  61. 61.
    Dennis, J.E. Jr., Price, C.J., Coope, I.D.: Direct search methods for nonlinearly constrained optimization using filters and frames. Optim. Eng. 5(2), 123–144 (2004)MathSciNetMATHGoogle Scholar
  62. 62.
    Dolan, E.D., Lewis, R.M., Torczon, V.: On the local convergence of pattern search. SIAM J. Optim. 14(2), 567–583 (2003)MathSciNetMATHGoogle Scholar
  63. 63.
    FactSage (2014). http://www.factsage.com/
  64. 64.
    Fermi, E., Metropolis, N.: Numerical solution of a minimum problem. Los Alamos Unclassified Report LA–1492, Los Alamos National Laboratory, Los Alamos (1952)Google Scholar
  65. 65.
    Ferris, M.C., Mangasarian, O.L.: Parallel variable distribution. SIAM J. Optim. 4(4), 815–832 (1994)MathSciNetMATHGoogle Scholar
  66. 66.
    Finkel, D.E., Kelley, C.T.: Convergence analysis of the DIRECT algorithm. Technical Report CRSC-TR04-28, Center for Research in Scientific Computation (2004)Google Scholar
  67. 67.
    Finkel, D.E., Kelley, C.T.: Convergence analysis of sampling methods for perturbed Lipschitz functions. Pac. J. Optim. 5(2), 339–350 (2009)MathSciNetMATHGoogle Scholar
  68. 68.
    Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. Ser. A 91, 239–269 (2002)MathSciNetMATHGoogle Scholar
  69. 69.
    Fletcher, R., Leyffer, S., Toint, Ph.L.: On the global convergence of an SLP–filter algorithm. Technical Report NA/183, Department of Mathematics, Dundee University (1998)Google Scholar
  70. 70.
    Fourer, R., Orban, D.: Dr. Ampl – a meta-solver for optimization problem analysis. Comput. Manag. Sci. 7(4), 437–463 (2009)Google Scholar
  71. 71.
    Fowler, K.R., Kelley, C.T., Miller, C.T., Kees, C.E., Darwin, R.W., Reese, J.P., Farthing, M.W., Reed, M.S.C.: Solution of a well-field design problem with implicit filtering. Optim. Eng. 5(2), 207–234 (2004)MathSciNetMATHGoogle Scholar
  72. 72.
    Fowler, K.R., Reese, J.P., Kees, C.E., Dennis, J.E. Jr., Kelley, C.T., Miller, C.T., Audet, C., Booker, A.J., Couture, G., Darwin, R.W., Farthing, M.W., Finkel, D.E., Gablonsky, J.M., Gray, G., Kolda, T.G.: Comparison of derivative-free optimization methods for groundwater supply and hydraulic capture community problems. Adv. Water Resour. 31(5), 743–757 (2008)Google Scholar
  73. 73.
    Gablonsky, J.M., Kelley, C.T.: A locally-biased form of the DIRECT algorithm. J. Glob. Optim. 21, 27–37 (2001)MathSciNetMATHGoogle Scholar
  74. 74.
    García-Palomares, U.M., Rodríguez, J.F.: New sequential and parallel derivative-free algorithms for unconstrained optimization. SIAM J. Optim. 13(1), 79–96 (2002)MathSciNetMATHGoogle Scholar
  75. 75.
    Gheribi, A.E., Robelin, C., Le Digabel, S., Audet, C., Pelton, A.D.: Calculating all local minima on liquidus surfaces using the factsage software and databases and the mesh adaptive direct search algorithm. J. Chem. Thermodyn. 43(9), 1323–1330 (2011)Google Scholar
  76. 76.
    Gheribi, A.E., Audet, C., Le Digabel, S., Bélisle, E., Bale, C.W., Pelton, A.D.: Calculating optimal conditions for alloy and process design using thermodynamic and properties databases, the FactSage software and the Mesh Adaptive Direct Search algorithm. CALPHAD 36, 135–143 (2012)Google Scholar
  77. 77.
    Gilmore, P., Kelley, C.T.: An implicit filtering algorithm for optimization of functions with many local minima. SIAM J. Optim. 5(2), 269–285 (1995)MathSciNetMATHGoogle Scholar
  78. 78.
    Gould, N.I.M., Orban, D., Toint, Ph.L.: CUTEr (and SifDec): A constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)MathSciNetMATHGoogle Scholar
  79. 79.
    Gould, N.I.M., Orban, D., Sartenaer, A., Toint, Ph.L.: Sensitivity of trust-region algorithms on their parameters. 4OR 3(3), 227–241 (2005)Google Scholar
  80. 80.
    Gramacy, R.B., Le Digabel, S.: The mesh adaptive direct search algorithm with treed Gaussian process surrogates. Technical Report G-2011-37, Les cahiers du GERAD (2011)Google Scholar
  81. 81.
    Gramacy, R.B., Taddy, M.A., Wild, S.M.: Variable selection and sensitivity analysis via dynamic trees with an application to computer code performance tuning. Technical Report 1108.4739, arXiv, August 2011Google Scholar
  82. 82.
    Gray, G.A., Kolda, T.G.: Algorithm 856: APPSPACK 4.0: Asynchronous parallel pattern search for derivative-free optimization. ACM Trans. Math. Softw. 32(3), 485–507 (2006)Google Scholar
  83. 83.
    Griffin, J.D., Kolda, T.G.: Nonlinearly-constrained optimization using heuristic penalty methods and asynchronous parallel generating set search. Appl. Math. Res. Express 25(5), 36–62 (2010)MathSciNetGoogle Scholar
  84. 84.
    Griffin, J.D., Kolda, T.G., Lewis, R.M.: Asynchronous parallel generating set search for linearly-constrained optimization. SIAM J. Sci. Comput. 30(4), 1892–1924 (2008)MathSciNetGoogle Scholar
  85. 85.
    Hansen, P., Mladenović, N.: Variable neighborhood search: principles and applications. Eur. J. Oper. Res. 130(3), 449–467 (2001)MATHGoogle Scholar
  86. 86.
    Hare, W.L., Macklem, M.: Derivative-free optimization methods for finite minimax problems. Optim. Methods Softw. 28(2), 300–312 (2013)MathSciNetMATHGoogle Scholar
  87. 87.
    Hayes, R.E., Bertrand, F.H., Audet, C., Kolaczkowski, S.T.: Catalytic combustion kinetics: Using a direct search algorithm to evaluate kinetic parameters from light-off curves. Canad. J. Chem. Eng. 81(6), 1192–1199 (2003)Google Scholar
  88. 88.
    Higham, N.J.: Optimization by direct search in matrix computations. SIAM J. Matrix Anal. Appl. 14, 317–333 (1993)MathSciNetMATHGoogle Scholar
  89. 89.
    Hock, W., Schittkowski, K.: Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems, vol. 187. Springer, Berlin (1981)Google Scholar
  90. 90.
    Hooke, R., Jeeves, T.A.: Direct search solution of numerical and statistical problems. J. Assoc. Comput. Mach. 8(2), 212–229 (1961)MATHGoogle Scholar
  91. 91.
    Hough, P.D., Kolda, T.G., Torczon, V.: Asynchronous parallel pattern search for nonlinear optimization. SIAM J. Sci. Comput. 23(1), 134–156 (2001)MathSciNetMATHGoogle Scholar
  92. 92.
    Jahn, J.: Vector Optimization: Theory, Applications, and Extensions. Springer, Berlin (2004)Google Scholar
  93. 93.
    Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79(1), 157–181 (1993)MathSciNetMATHGoogle Scholar
  94. 94.
    Kelley, C.T.: Iterative Methods for Optimization. Frontiers in Applied Mathematics, vol. 18. SIAM, Philadelphia (1999)Google Scholar
  95. 95.
    Kokkolaras, M., Audet, C., Dennis, J.E. Jr.: Mixed variable optimization of the number and composition of heat intercepts in a thermal insulation system. Optim. Eng. 2(1), 5–29 (2001)MathSciNetMATHGoogle Scholar
  96. 96.
    Kolda, T.G.: Revisiting asynchronous parallel pattern search for nonlinear optimization. SIAM J. Optim. 16(2), 563–586 (2005)MathSciNetMATHGoogle Scholar
  97. 97.
    Kolda, T.G., Torczon, V.: On the convergence of asynchronous parallel pattern search. SIAM J. Optim. 14(4), 939–964 (2004)MathSciNetMATHGoogle Scholar
  98. 98.
    Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: New perspectives on some classical and modern methods. SIAM Rev. 45(3), 385–482 (2003)MathSciNetMATHGoogle Scholar
  99. 99.
    Kolda, T.G., Lewis, R.M., Torczon, V.: A generating set direct search augmented Lagrangian algorithm for optimization with a combination of general and linear constraints. Technical Report SAND2006-5315, Sandia National Laboratories, USA (2006)Google Scholar
  100. 100.
    Kolda, T.G., Lewis, R.M., Torczon, V.: Stationarity results for generating set search for linearly constrained optimization. SIAM J. Optim. 17(4), 943–968 (2006)MathSciNetMATHGoogle Scholar
  101. 101.
    Le Digabel, S.: Algorithm 909: NOMAD: Nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37(4), 44:1–44:15 (2011)Google Scholar
  102. 102.
    Audet, C., Le Digabel, S., Tribes, C.: NOMAD user guide. Technical Report G-2009-37, Les cahiers du GERAD (2009)Google Scholar
  103. 103.
    Leach, E.B.: A note on inverse function theorem. In: Proceedings of the AMS, vol. 12, pp. 694–697 (1961)MathSciNetMATHGoogle Scholar
  104. 104.
    Lewis, R.M., Torczon, V.: Rank ordering and positive bases in pattern search algorithms. Technical Report 96–71, Institute for Computer Applications in Science and Engineering, Mail Stop 132C, pp. 23681–2199. NASA Langley Research Center, Hampton, VA (1996)Google Scholar
  105. 105.
    Lewis, R.M., Torczon, V.: Pattern search algorithms for bound constrained minimization. SIAM J. Optim. 9(4), 1082–1099 (1999)MathSciNetMATHGoogle Scholar
  106. 106.
    Lewis, R.M., Torczon, V.: Pattern search methods for linearly constrained minimization. SIAM J. Optim. 10(3), 917–941 (2000)MathSciNetMATHGoogle Scholar
  107. 107.
    Lewis, R.M., Torczon, V.: A globally convergent augmented Lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Optim. 12(4), 1075–1089 (2002)MathSciNetMATHGoogle Scholar
  108. 108.
    Lewis, R.M., Shepherd, A., Torczon, V.: Implementing generating set search methods for linearly constrained optimization. SIAM J. Sci. Comput. 29(6), 2507–2530 (2007)MathSciNetMATHGoogle Scholar
  109. 109.
    Li, M., Li, Y.F., Wu, Y.J., Cai, S., Zhu, N.Y., Rezzak, N., Schrimpf, R.D., Fleetwood, D.M., Wang, J.Q., Cheng, X.X., Wang, Y., Wang, D.L., Hao, Y.: Including Radiation Effects and Dependencies on Process-Related Variability in Advanced Foundry SPICE Models Using a New Physical Model and Parameter Extraction Approach. IEEE Trans. Nuclear Sci. 58(6, Part 1), 2876–2882 (2011). IEEE Radiation Effects Data Workshop (REDW)/48th IEEE International Nuclear and Space Radiation Effects Conference (NSREC), pp. 25–29, Las Vegas (2011)Google Scholar
  110. 110.
    Liuzzi, G., Lucidi, S., Sciandrone, M.: A derivative-free algorithm for linearly constrained finite minimax problems. SIAM J. Optim. 16(4), 1054–1075 (2006)MathSciNetMATHGoogle Scholar
  111. 111.
    Lophaven, S., Nielsen, H., Søondergaard, J.: Dace: A matlab kriging toolbox version 2.0. Technical Report IMM-REP-2002-12, Informatics and Mathematical Modelling, Technical University of Denmark (2002)Google Scholar
  112. 112.
    Lucidi, S., Sciandrone, M.: On the global convergence of derivative-free methods for unconstrained optimization. SIAM J. Optim. 13, 97–116 (2002)MathSciNetMATHGoogle Scholar
  113. 113.
    Marsden, A.L.: Aerodynamic noise control by optimal shape design. Ph.D. thesis, Stanford University (2004)Google Scholar
  114. 114.
    Marsden, A.L., Wang, M., Dennis, J.E. Jr., Moin, P.: Optimal aeroacoustic shape design using the surrogate management framework. Optim. Eng. 5(2), 235–262 (2004)MathSciNetMATHGoogle Scholar
  115. 115.
    Marsden, A.L., Wang, M., Dennis, J.E. Jr., Moin, P.: Suppression of airfoil vortex-shedding noise via derivative-free optimization. Phys. Fluids 16(10), L83–L86 (2004)Google Scholar
  116. 116.
    Marsden, A.L., Wang, M., Dennis, J.E. Jr., Moin, P.: Trailing-edge noise reduction using derivative-free optimization and large-eddy simulation. J. Fluid Mech. 572, 13–36 (2007)MathSciNetMATHGoogle Scholar
  117. 117.
    Marsden, A.L., Feinstein, J.A., Taylor, C.A.: A computational framework for derivative-free optimization of cardiovascular geometries. Comput. Methods Appl. Mech. Eng. 197(21–24), 1890–1905 (2008)MathSciNetMATHGoogle Scholar
  118. 118.
    Martin, J.-P., Houdayer, A., Lebel, C., Choquette, Y., Lavigne, P., Ducharme, P.: An unattended gamma monitor for the determination of snow water equivalent (SWE) using the natural ground gamma radiation. In: Nuclear Science Symposium Conference Record, pp. 983–988. IEEE (2008)Google Scholar
  119. 119.
    Marty, A.: Optimisation du placement et de l’assignation de fréquence d’antennes dans un réseau de télécommunications. Master’s thesis, École Polytechnique de Montréal (2011)Google Scholar
  120. 120.
    McKinnon, K.I.M.: Convergence of the Nelder-Mead simplex method to a nonstationary point. SIAM J. Optim. 9, 148–158 (1998)MathSciNetMATHGoogle Scholar
  121. 121.
    Mladenović, N., Hansen, P.: Variable neighborhood search. Comput. Oper. Res. 24(11), 1097–1100 (1997)MathSciNetMATHGoogle Scholar
  122. 122.
    Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7(4), 308–313 (1965)MATHGoogle Scholar
  123. 123.
    Oeuvray, R., Bierlaire, M.: BOOSTERS: A derivative-free algorithm based on radial basis functions. Int. J. Model. Simul. 29(1), 26–36 (2009)Google Scholar
  124. 124.
    Orban, D.: Templating and automatic code generation for performance with Python. Technical Report G-2011-30, Les cahiers du GERAD (2011)Google Scholar
  125. 125.
    Polak, E., Wetter, M.: Precision control for generalized pattern search algorithms with adaptive precision function evaluations. SIAM J. Optim. 16(3), 650–669 (2006)MathSciNetMATHGoogle Scholar
  126. 126.
    Prechtel, M., Leugering, G., Steinmann, P., Stingl, M.: Towards optimization of crack resistance of composite materials by adjustment of fiber shapes. Eng. Fract. Mech. 78(6), 944–960 (2011)Google Scholar
  127. 127.
    Price, C.J., Coope, I.D.: Frames and grids in unconstrained and linearly constrained optimization: A nonsmooth approach. SIAM J. Optim. 14, 415–438 (2003)MathSciNetMATHGoogle Scholar
  128. 128.
    Regis, R.G., Shoemaker, C.A.: Constrained global optimization of expensive black box functions using radial basis functions. J. Glob. Optim. 31, 153–171 (2005)MathSciNetMATHGoogle Scholar
  129. 129.
    Renaud, E., Robelin, C., Gheribi, A.E., Chartrand, P.: Thermodynamic evaluation and optimization of the Li, Na, K, Mg, Ca, Sr // F, Cl reciprocal system. J. Chem. Thermodyn. 43(8), 1286–1298 (2011)Google Scholar
  130. 130.
    Rockafellar, R.T.: Generalized directional derivatives and subgradients of nonconvex functions. Canad. J. Math. 32(2), 257–280 (1980)MathSciNetMATHGoogle Scholar
  131. 131.
    Sankaran, S., Marsden, A.L.: The impact of uncertainty on shape optimization of idealized bypass graft models in unsteady flow. Phys. Fluids 22(12), 121902 (2010)Google Scholar
  132. 132.
    Sankaran, S., Audet, C., Marsden, A.L.: A method for stochastic constrained optimization using derivative-free surrogate pattern search and collocation. J. Comput. Phys. 229(12), 4664–4682 (2010)MATHGoogle Scholar
  133. 133.
    Serafini, D.B.: A framework for managing models in nonlinear optimization of computationally expensive functions. Ph.D. thesis, Department of Computational and Applied Mathematics, Rice University (1998)Google Scholar
  134. 134.
    Søndergaard, J.: Optimization using surrogate models—by the space mapping technique. Ph.D. thesis, Informatics and Mathematical Modelling, Technical University of Denmark (2003)Google Scholar
  135. 135.
    Spillane, M.C., Gica, E., Titov, V.V.: Tsunameter Network Design for the U.S. DART Array. AGU Fall Meeting Abstracts, p. A1368 (December 2009)Google Scholar
  136. 136.
    Stephens, C.P., Baritompa, W.: Global optimization requires global information. J. Optim. Theory Appl. 96, 575–588 (1998)MathSciNetMATHGoogle Scholar
  137. 137.
    Sun, J., Salvucci, G.D., Entekhabi, D., Farhadi, L.: Parameter estimation of coupled water and energy balance models based on stationary constraints of surface states. Water Resour. Res. 47, 1–16 (2011)Google Scholar
  138. 138.
    Sweatlock, L.A., Diest, K., Marthaler, D.E.: Metamaterials design using gradient-free numerical optimization. J. Appl. Phys. 108(8), 1–5 (2010)Google Scholar
  139. 139.
    Tang, B.: Orthogonal array-based Latin hypercubes. J. Am. Stat. Assoc. 88(424), 1392–1397 (1993)MATHGoogle Scholar
  140. 140.
    Torczon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7(1), 1–25 (1997)MathSciNetMATHGoogle Scholar
  141. 141.
    Torres, R., Bès, C., Chaptal, J., Hiriart-Urruty, J.-B.: Optimal, environmentally-friendly departure procedures for civil aircraft. J. Aircr. 48(1), 11–22 (2011)Google Scholar
  142. 142.
    Vicente, L.N., Custódio, A.L.: Analysis of direct searches for discontinuous functions. Math. Program. 133(1–2), 299–325 (2012)MathSciNetMATHGoogle Scholar
  143. 143.
    Wild, S.M., Shoemaker, C.A.: Global convergence of radial basis function trust region derivative-free algorithms. SIAM J. Optim. 21(3), 761–781 (2011)MathSciNetMATHGoogle Scholar
  144. 144.
    Yang, W., Feinstein, J.A., Marsden, A.L.: Constrained optimization of an idealized y-shaped baffle for the fontan surgery at rest and exercise. Comput. Methods Appl. Mech. Eng. 199(33–36), 2135–2149 (2010)MATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Département de Mathématiques et de Génie IndustrielÉcole Polytechnique de Montréal and GERADMontréalCanada

Personalised recommendations