Journal of Global Optimization

, Volume 41, Issue 2, pp 299–318 | Cite as

Nonsmooth optimization through Mesh Adaptive Direct Search and Variable Neighborhood Search

  • Charles Audet
  • Vincent Béchard
  • Sébastien Le DigabelEmail author


This paper proposes a way to combine the Mesh Adaptive Direct Search (MADS) algorithm, which extends the Generalized Pattern Search (GPS) algorithm, with the Variable Neighborhood Search (VNS) metaheuristic, for nonsmooth constrained optimization. The resulting algorithm retains the convergence properties of MADS, and allows the far reaching exploration features of VNS to move away from local solutions. The paper also proposes a generic way to use surrogate functions in the VNS search. Numerical results illustrate advantages and limitations of this method.


Nonsmooth optimization Mesh Adaptive Direct Search Generalized Pattern Search Variable Neighborhood Search 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abramson M.A. (2004). Mixed variable optimization of a load-bearing thermal insulation system using a filter pattern search algorithm. Optim. Eng. 5(2): 157–177 CrossRefGoogle Scholar
  2. 2.
    Abramson M.A. and Audet C. (2006). Second-order convergence of mesh-adaptive direct search. SIAM J. Optim. 17(2): 606–619 CrossRefGoogle Scholar
  3. 3.
    Alberto P., Nogueira F., Rocha U. and Vicente L.N. (2004). Pattern search methods for user-provided points: application to molecular geometry problems. SIAM J. Optim. 14(4): 1216–1236 CrossRefGoogle Scholar
  4. 4.
    Audet C., Brimberg J., Hansen P., Le Digabel S. and Mladenović N. (2004). Pooling problem: alternate formulations and solution methods. Manage. Sci. 50(6): 761–776 CrossRefGoogle Scholar
  5. 5.
    Audet, C., Couture, G., Dennis, J.E. Jr.: NOMAD Project (LTMADS Package). Software available at Scholar
  6. 6.
    Audet C. and Dennis J.E. (2000). Pattern search algorithms for mixed variable programming. SIAM J. Optim. 11(3): 573–594 CrossRefGoogle Scholar
  7. 7.
    Audet C. and Dennis J.E. (2003). Analysis of generalized pattern searches. SIAM J. Optim. 13(3): 889–903 CrossRefGoogle Scholar
  8. 8.
    Audet C. and Dennis J.E. (2004). A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14(4): 980–1010 CrossRefGoogle Scholar
  9. 9.
    Audet C. and Dennis J.E. (2006). Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1): 188–217 CrossRefGoogle Scholar
  10. 10.
    Audet C. and Orban D. (2006). Finding optimal algorithmic parameters using the mesh adaptive direct search algorithm. SIAM J. Optim. 17(3): 642–664 CrossRefGoogle Scholar
  11. 11.
    Béchard, V., Audet, C., Chaouki, J.: Robust optimization of chemical processes using a MADS algorithm. Technical reportG–2005–16, Les Cahiers du GERAD, Montréal (2005)Google Scholar
  12. 12.
    Booker, A.J., Dennis, J.E. Jr., Frank, P.D., Moore, D.W., Serafini, D.B.: Managing surrogate objectives to optimize a helicopter rotor design—further experiments. AIAA Paper 1998–4717, Presented at the 8th AIAA/ISSMO symposium on multidisciplinary analysis and optimization, St. Louis (1998)Google Scholar
  13. 13.
    Booker A.J., Frank P.D., Serafini D.B., Torczon V. and Dennis J.E. (1998). Optimization using surrogate objectives on a helicopter test example. In: Borggaard, J., Burns, J., Cliff, E., and Schreck, S. (eds) Optimal Des. Control, Progress in Systems and Control Theory, pp 49–58. Birkhäuser, Cambridge Google Scholar
  14. 14.
    Booker A.J., Frank P.D., Serafini D.B., Torczon V., Trosset M.W. and Dennis J.E. (1999). A rigorous framework for optimization of expensive functions by surrogates. Struct. Optim. 17(1): 1–13 CrossRefGoogle Scholar
  15. 15.
    Bowden, R.O., Hall, J.D.: Simulation optimization research and development. Simulation conference, 1693–1698, Presented at the 1998 winter simulation conference (1998)Google Scholar
  16. 16.
    Brimberg, J., Mladenović, N.: A variable neighbourhood algorithm for solving the continuous location-allocation problem. In: Hamacher, D. (ed.) S tud. Location Anal. vol. 10, pp. 1–12, Athens, Greece (1996)Google Scholar
  17. 17.
    Caporossi G. and Hansen P. (2000). Variable neighborhood search for extremal graphs 1: the AutoGraphiX system. Discrete Math. 212: 29–44 CrossRefGoogle Scholar
  18. 18.
    Clarke F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983). Reissued in 1990 by SIAM Publications, Philadelphia, as vol. 5 in the series Classics in Applied MathematicsGoogle Scholar
  19. 19.
    Douglas J.M. (1988). Conceptual Design of Chemical Processes. McGraw-Hill, New York Google Scholar
  20. 20.
    Drazić, M., Lavor, C., Maculan, N., Mladenović, N.: A Continuous VNS Heuristic for Finding the Tridimensional Structure of a Molecule. Technical Report G–2004–22, Les Cahiers du GERAD, Montréal (2004)Google Scholar
  21. 21.
    Finkel, D.E., Kelley, C.T.: Convergence analysis of the direct algorithm. SIAM J. Optim. 2004 (to appear)Google Scholar
  22. 22.
    Fletcher R. and Leyffer S. (2002). Nonlinear programming without a penalty function. Math. Program. Series A 91: 239–269 CrossRefGoogle Scholar
  23. 23.
    Fowler K.R., Kelley C.T., Miller C.T., Kees C.E., Darwin R.W., Reese J.P., Farthing M.W. and Reed M.S.C. (2004). Solution of a well-field design problem with implicit filtering. Optim. Eng. 5(2): 207–234 CrossRefGoogle Scholar
  24. 24.
    Hansen P. and Mladenović N. (2001). J-MEANS: a new local search heuristic for minimum sum of squares clustering. Pattern Recogn. 34(2): 405–413 CrossRefGoogle Scholar
  25. 25.
    Hansen P. and Mladenović N. (2001). Variable neighborhood search: principles and applications. Eur. J. Oper. Res. 130(3): 449–467 CrossRefGoogle Scholar
  26. 26.
    Hansen P., Mladenović N. and Perez-Britos D. (2001). Variable neighborhood decomposition search. J. Heuristics 7(4): 335–350 CrossRefGoogle Scholar
  27. 27.
    Hansen P., Mladenović N. and Urosevic D. (2004). Variable neighborhood search for the maximum clique. Discrete Appl. Math. 145(1): 117–125 CrossRefGoogle Scholar
  28. 28.
    Hayes R.E., Bertrand F.H., Audet C. and Kolaczkowski S.T. (2003). Catalytic combustion kinetics: Using a direct search algorithm to evaluate kinetic parameters from light-off curves. Can. J. Chem. Eng. 81(6): 1192–1199 CrossRefGoogle Scholar
  29. 29.
    Himmelblau D.M., Edgar T.F. and Lasdon L.S. (2003). Optimization of Chemical Processes, 2nd edn. McGraw-Hill, New York Google Scholar
  30. 30.
    Jones D.R., Perttunen C.D. and Stuckman B.E. (1993). Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79(1): 157–181 CrossRefGoogle Scholar
  31. 31.
    Kokkolaras M., Audet C. and Dennis J.E. (2001). Mixed variable optimization of the number and composition of heat intercepts in a thermal insulation system. Optim. Eng. 2(1): 5–29 CrossRefGoogle Scholar
  32. 32.
    Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45(3):385–482 (electronic) (2003)Google Scholar
  33. 33.
    Lewis R.M., Torczon V. and Trosset M.W. (2000). Direct search methods: Then and now. J. Comput. Appl. Math. 124(1–2): 191–207 CrossRefGoogle Scholar
  34. 34.
    Lophaven, S., Nielsen, H., Sondergaard, J.: DACE—A Matlab Kriging toolbox, version 2.0. Technical report IMM-REP-2002-12, Informatics and mathematical modelling, Technical University of Denmark (2002)Google Scholar
  35. 35.
    Marsden A.L., Wang M., Moin P. and Dennis J.E. (2004). Optimal aeroacoustic shape design using the surrogate management framework. Optim. Eng. 5(2): 235–262 CrossRefGoogle Scholar
  36. 36.
    Mladenović N. and Hansen P. (1997). Variable neighborhood search. Comput. Oper. Res. 24(11): 1097–1100 CrossRefGoogle Scholar
  37. 37.
    Perez, R., Liu, H.H.T., Behdinan, K.: Evaluation of multidisciplinary optimization approaches for aircraft conceptual design. In: AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Albany, NY, September (2004)Google Scholar
  38. 38.
    Seader J.D., Sieder W.D. and Lewin D.R. (1999). Process Design Principles: Synthesis, Analysis and Evaluation. John Wiley and Sons Inc., New York Google Scholar
  39. 39.
    Serafini, D.B.: A framework for managing models in nonlinear optimization of computationally expensive functions. Ph.D. thesis, Department of Mathematical Sciences, Rice University, Houston, Texas (1998)Google Scholar
  40. 40.
    Snyder J.D. and Subramaniam B. (1994). A novel reverse flow strategy for ethylbenzene dehydrogenation in a packed-bed reactor. Chem. Eng. Sci. 49: 5585–5601 CrossRefGoogle Scholar
  41. 41.
    Sobieszczanski-Sobieski, J., Agte, J.S., Sandusky, R.R. Jr.: Bi-level integrated system synthesis (BLISS). Technical Report NASA/TM-1998-208715, NASA, Langley Research Center, August 1998Google Scholar
  42. 42.
    Stein M. (1987). Large sample properties of simulations using latin hypercube sampling. Technometrics 29(2): 143–151 CrossRefGoogle Scholar
  43. 43.
    Tang B. (1993). Orthogonal array-based latin hypercubes. J. Am. Stat. Assoc. 88(424): 1392–1397 CrossRefGoogle Scholar
  44. 44.
    Timmerhaus K.D., Peters M.S. and West R.E. (2003). Plant Design and Economics for Chemical Engineers, 5th edn. McGraw-Hill, New York Google Scholar
  45. 45.
    Torczon V. (1997). On the convergence of pattern search algorithms. SIAM J. Optim. 7(1): 1–25 CrossRefGoogle Scholar
  46. 46.
    Trefethen, N.: The hundred dollar hundred digit challenge. SIAM News 35(1), January–February 2002. Scholar
  47. 47.
    Vaz A.I.F. and Vicente L.N. (2007). A particle swarm pattern search method for bound constrained global optimization. J. Global Optim. 39(2): 197–219 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2007

Authors and Affiliations

  • Charles Audet
    • 1
  • Vincent Béchard
    • 1
  • Sébastien Le Digabel
    • 1
    Email author
  1. 1.Département de Mathématiques et de Génie IndustrielEcole Polytechnique de Montréal and GERADMontrealCanada

Personalised recommendations