Skip to main content
SpringerLink
Log in

Mesh-based Nelder–Mead algorithm for inequality constrained optimization

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

Despite the lack of theoretical and practical convergence support, the Nelder–Mead (NM) algorithm is widely used to solve unconstrained optimization problems. It is a derivative-free algorithm, that attempts iteratively to replace the worst point of a simplex by a better one. The present paper proposes a way to extend the NM algorithm to inequality constrained optimization. This is done through a search step of the mesh adaptive direct search (Mads) algorithm, inspired by the NM algorithm. The proposed algorithm does not suffer from the NM lack of convergence, but instead inherits from the totality of the Mads convergence analysis. Numerical experiments show an important improvement in the quality of the solutions produced using this search step.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Audet, C., Béchard, V., Le Digabel, S.: Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J. Global Optim. 41(2), 299–318 (2008)

    Article  MathSciNet  Google Scholar 

  2. Audet, C., Dennis Jr., J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)

    Article  MathSciNet  Google Scholar 

  3. Audet, C., Dennis Jr., J.E.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(1), 445–472 (2009)

    Article  MathSciNet  Google Scholar 

  4. Audet, C., Dennis Jr., J.E., Le Digabel, S.: Parallel space decomposition of the mesh adaptive direct search algorithm. SIAM J. Optim. 19(3), 1150–1170 (2008)

    Article  MathSciNet  Google Scholar 

  5. Audet, C., Hare, W.: Derivative-Free and Blackbox Optimization. Springer Series in Operations Research and Financial Engineering. Springer, Berlin (2017)

    Book  Google Scholar 

  6. Audet, C., Ianni, A., Le Digabel, S., Tribes, C.: Reducing the number of function evaluations in mesh adaptive direct search algorithms. SIAM J. Optim. 24(2), 621–642 (2014)

    Article  MathSciNet  Google Scholar 

  7. Audet, C., Kokkolaras, M., Le Digabel, S., Talgorn, B.: Order-based error for managing ensembles of surrogates in derivative-free optimization. J. Global Optim. 70(3), 645–675 (2018)

    Article  MathSciNet  Google Scholar 

  8. Audet, C., Le Digabel, S., Tribes, C.: Dynamic scaling in the mesh adaptive direct search algorithm for blackbox optimization. Optim. Eng. 17(2), 333–358 (2016)

    Article  MathSciNet  Google Scholar 

  9. Brea, E.: An extension of Nelder–Mead method to nonlinear mixed-integer optimization problems. Rev. Int. Métod. Numér. Cálc Diseño Ing. 29(3), 163–174 (2013)

    MathSciNet  Google Scholar 

  10. Bűrmen, Á., Puhan, J., Tuma, T.: Grid restrained Nelder–Mead algorithm. Comput. Optim. Appl. 34(3), 359–375 (2006)

    Article  MathSciNet  Google Scholar 

  11. Chang, K.H.: Stochastic Nelder–Mead simplex method—a new globally convergent direct search method for simulation optimization. Eur. J. Oper. Res. 220(3), 684–694 (2012)

    Article  MathSciNet  Google Scholar 

  12. Chen, X., Wang, N.: Optimization of short-time gasoline blending scheduling problem with a DNA based hybrid genetic algorithm. Chem. Eng. Process. 49(10), 1076–1083 (2010)

    Article  Google Scholar 

  13. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983). (reissued in 1990 by SIAM Publications, Philadelphia, as Vol. 5 in the series Classics in Applied Mathematics)

    MATH  Google Scholar 

  14. Conn, A.R., Le Digabel, S.: Use of quadratic models with mesh-adaptive direct search for constrained black box optimization. Optim. Methods Softw. 28(1), 139–158 (2013)

    Article  MathSciNet  Google Scholar 

  15. Coope, I.D., Price, C.J.: Frame-based methods for unconstrained optimization. J. Optim. Theory Appl. 107(2), 261–274 (2000)

    Article  MathSciNet  Google Scholar 

  16. Craig, J.: Bluebird developer manual. http://www.civil.uwaterloo.ca/jrcraig/pdf/bluebird_developers_manual.pdf (2002). Accessed 11 June 2018

  17. Dennis Jr., J.E., Woods, D.J.: Optimization on microcomputers: The Nelder–Mead simplex algorithm. In: Wouk, A. (ed.) New Computing Environments: Microcomputers in Large-Scale Computing, pp. 116–122. Society for Industrial and Applied Mathematics, Philadelphia, PA (1987)

    Google Scholar 

  18. Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. Ser. A 91, 239–269 (2002)

    Article  MathSciNet  Google Scholar 

  19. Fletcher, R., Leyffer, S., Toint, PhL: On the global convergence of a filter-SQP algorithm. SIAM J. Optim. 13(1), 44–59 (2002)

    Article  MathSciNet  Google Scholar 

  20. Gao, F., Han, L.: Implementing the Nelder–Mead simplex algorithm with adaptive parameters. Comput. Optim. Appl. 51(1), 259–277 (2012)

    Article  MathSciNet  Google Scholar 

  21. Gould, N.I.M., Orban, D., Toint, PhL: CUTEr (and SifDec): a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)

    Article  Google Scholar 

  22. Hedar, A.-R.: Global optimization test problems. http://www-optima.amp.i.kyoto-u.ac.jp/member/student/hedar/Hedar_files/TestGO.htm (2017). Accessed 11 June 2018

  23. Hock, W., Schittkowski, K.: Test Examples for Nonlinear Programming Codes, Lecture Notes in Economics and Mathematical Systems, vol. 187. Springer, Berlin (1981)

    Book  Google Scholar 

  24. Jahn, J.: Introduction to the Theory of Nonlinear Optimization. Springer, Berlin (1994)

    Book  Google Scholar 

  25. Jamil, M., Yang, X.-S.: A literature survey of benchmark functions for global optimisation problems. Int. J. Math. Modell. Numer. Optim. 4(2), 150–194 (2013)

    MATH  Google Scholar 

  26. Kelley, C.T.: Detection and remediation of stagnation in the Nelder–Mead algorithm using a sufficient decrease condition. SIAM J. Optim. 10(1), 43–55 (1999)

    Article  MathSciNet  Google Scholar 

  27. Kitayama, S., Arakawa, M., Yamazaki, K.: Sequential approximate optimization using radial basis function network for engineering optimization. Optim. Eng. 12(4), 535–557 (2011)

    Article  Google Scholar 

  28. Lagarias, J.C., Reeds, J.A., Wright, M.H., Wright, P.E.: Convergence properties of the Nelder–Mead simplex method in low dimensions. SIAM J. Optim. 9, 112–147 (1998)

    Article  MathSciNet  Google Scholar 

  29. Le Digabel, S.: Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37(4), 44:1–44:15 (2011)

    Article  MathSciNet  Google Scholar 

  30. Lukšan, L., Vlček, J.: Test problems for nonsmooth unconstrained and linearly constrained optimization. Technical Report V-798, ICS AS CR (2000)

  31. Matott, L.S., Leung, K., Sim, J.: Application of MATLAB and Python optimizers to two case studies involving groundwater flow and contaminant transport modeling. Comput. Geosci. 37(11), 1894–1899 (2011)

    Article  Google Scholar 

  32. McKinnon, K.I.M.: Convergence of the Nelder–Mead simplex method to a nonstationary point. SIAM J. Optim. 9(1), 148–158 (1998)

    Article  MathSciNet  Google Scholar 

  33. Mezura-Montes, E., Coello, C.A.: Useful infeasible solutions in engineering optimization with evolutionary algorithms. In: Proceedings of the 4th Mexican International Conference on Advances in Artificial Intelligence, MICAI’05, pp. 652–662, Springer, Berlin (2005)

    MATH  Google Scholar 

  34. Mladenović, N., Petrović, J., Kovačević-Vujčić, V., Čangalović, M.: Solving spread spectrum radar polyphase code design problem by tabu search and variable neighbourhood search. Eur. J. Oper. Res. 151(2), 389–399 (2003)

    Article  MathSciNet  Google Scholar 

  35. Moré, J.J., Garbow, B.S., Hillstrom, Kenneth E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7(1), 17–41 (1981)

    Article  MathSciNet  Google Scholar 

  36. Moré, J.J., Wild, S.M.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20(1), 172–191 (2009)

    Article  MathSciNet  Google Scholar 

  37. Nazareth, L., Tseng, P.: Gilding the lily: a variant of the Nelder–Mead algorithm based on golden-section search. Comput. Optim. Appl. 22, 133–144 (2002)

    Article  MathSciNet  Google Scholar 

  38. Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7(4), 308–313 (1965)

    Article  MathSciNet  Google Scholar 

  39. Pigache, F., Messine, F., Nogarede, B.: Optimal design of piezoelectric transformers: a rational approach based on an analytical model and a deterministic global optimization. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 54(7), 1293–1302 (2007)

    Article  Google Scholar 

  40. Price, C.J., Coope, I.D., Byatt, D.: A convergent variant of the Nelder–Mead algorithm. J. Optim. Theory Appl. 113(1), 5–19 (2002)

    Article  MathSciNet  Google Scholar 

  41. Rodríguez, J.F., Renaud, J.E., Watson, L.T.: Trust region augmented Lagrangian methods for sequential response surface approximation and optimization. J. Mech. Des. 120(1), 58–66 (1998). 03

    Article  Google Scholar 

  42. Strasser, M.: Übertrangung des Optimierungsverfahrens von Nelder und Mead auf restringierte Probleme. Diploma thesis, Numerical Mathematics Group, Technical University of Darmstadt, Germany (1994)

  43. Tao, J., Wang, N.: DNA double helix based hybrid GA for the gasoline blending recipe optimization problem. Chem. Eng. Technol. 31(3), 440–451 (2008)

    Article  Google Scholar 

  44. Tribes, C., Dubé, J.-F., Trépanier, J.-Y.: Decomposition of multidisciplinary optimization problems: formulations and application to a simplified wing design. Eng. Optim. 37(8), 775–796 (2005)

    Article  MathSciNet  Google Scholar 

  45. Tseng, P.: Fortified-descent simplicial search method: a general approach. SIAM J. Optim. 10(1), 269–288 (1999)

    Article  MathSciNet  Google Scholar 

  46. Wang, K., Wang, N.: A novel RNA genetic algorithm for parameter estimation of dynamic systems. Chem. Eng. Res. Des. 88(11), 1485–1493 (2010)

    Article  Google Scholar 

  47. Zhao, J., Wang, N.: A bio-inspired algorithm based on membrane computing and its application to gasoline blending scheduling. Comput. Chem. Eng. 35(2), 272–283 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by NSERC CRD Grant (#RDCPJ 490744 - 15) in collaboration with Hydro-Québec and Rio Tinto. The authors wish to thank Shawn Mattot, Genetha Gray, and Stefan Wild for making the LOCKWOOD problem available.

Author information

Authors and Affiliations

  1. GERAD and Département de Mathématiques et Génie Industriel, École Polytechnique de Montréal, C.P. 6079, Succ. Centre-ville, Montreal, QC, H3C 3A7, Canada

    Charles Audet & Christophe Tribes

Authors
  1. Charles Audet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christophe Tribes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christophe Tribes.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Audet, C., Tribes, C. Mesh-based Nelder–Mead algorithm for inequality constrained optimization. Comput Optim Appl 71, 331–352 (2018). https://doi.org/10.1007/s10589-018-0016-0

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-018-0016-0

Keywords

Search

Navigation