Skip to main content
SpringerLink
Log in

Implementing the Nelder-Mead simplex algorithm with adaptive parameters

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

In this paper, we first prove that the expansion and contraction steps of the Nelder-Mead simplex algorithm possess a descent property when the objective function is uniformly convex. This property provides some new insights on why the standard Nelder-Mead algorithm becomes inefficient in high dimensions. We then propose an implementation of the Nelder-Mead method in which the expansion, contraction, and shrink parameters depend on the dimension of the optimization problem. Our numerical experiments show that the new implementation outperforms the standard Nelder-Mead method for high dimensional problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Andrei, N.: Convex functions. Adv. Model. Optim. 9(2), 257–267 (2007)

    MATH  MathSciNet  Google Scholar 

  2. Byatt, D.: Convergent variants of the Nelder-Mead algorithm. Master’s Thesis, University of Canterbury, Christchurch, New Zealand (2000)

  3. Byrd, R., Nocedal, J., Zhu, C.: Towards a discrete Newton method with memory for large-scale optimization. In: Di Pillo, G., Giannessi, F. (eds.) Nonlinear Optimization and Applications. Plenum, New York (1996)

    Google Scholar 

  4. Dennis, J.E. Jr., Torczon, V.: Direct search methods on parallel machines. SIAM J. Optim. 1, 448–474 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  5. Dennis, J.E. Jr., Woods, D.J.: Optimization on microcomputers: The Nelder-Mead simplex algorithm. In: Wouk, A. (ed.) New Computing Environments: Microcomputers in Large-Scale Scientific Computing. SIAM, Philadelphia (1987)

    Google Scholar 

  6. Han, L., Neumann, M.: Effect of dimensionality on the Nelder-Mead simplex method. Optim. Methods Softw. 21(1), 1–16 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  7. Kelley, C.T.: Iterative Methods for Optimization. SIAM Publications, Philadelphia (1999)

    Book  MATH  Google Scholar 

  8. Kelley, C.T.: Detection and remediation of stagnation in the Nelder-Mead algorithm using a sufficient decrease condition. SIAM J. Optim. 10, 43–55 (2000)

    Article  Google Scholar 

  9. Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45, 385–482 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  10. Lagarias, J.C., Reeds, J.A., Wright, M.H., Wright, P.: Convergence properties of the Nelder-Mead simplex algorithm in low dimensions. SIAM J. Optim. 9, 112–147 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  11. Math Works: MATLAB 6, Release 12, The Math Works, Natick, MA (2000)

  12. Mckinnon, K.I.M.: Convergence of the Nelder-Mead simplex method to a nonstationary point. SIAM J. Optim. 9, 148–158 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  13. Moré, J.J., Garbow, B.S., Hillstrom, B.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7(1), 17–41 (1981)

    Article  MATH  Google Scholar 

  14. Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7, 308–313 (1965)

    MATH  Google Scholar 

  15. Price, C.J., Coope, I.D., Byatt, D.: A convergent variant of the Nelder-Mead algorithm. JOTA 113, 5–19 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  16. Torczon, V.: Multi-directional Search: A Direct Search Algorithm for Parallel Machines. Ph.D. Thesis, Rice University, TX (1989)

  17. Tseng, P.: Fortified-descent simplicial search method: a general approach. SIAM J. Optim. 10, 269–288 (2000)

    Article  MathSciNet  Google Scholar 

  18. Woods, D.J.: An Iterative Approach for Solving Multi-objective Optimization Problems. Ph.D. Thesis, Rice University, TX (1985)

  19. Wright, M.H.: Direct search methods: Once scorned, now respectable. In: Griffiths, D.F., Watson, G.A. (eds.) Numerical Analysis 1995: Proceedings of the 1995 Dundee Biennial Conference in Numerical Analysis, pp. 191–208. Addison Wesley Longman, Harlow (1996)

    Google Scholar 

  20. Wright, M.H.: N&M@42: Nelder-Mead at 42″, a talk given at University of Waterloo, June 2007

  21. Zalinescu, C.: On uniformly convex functions. J. Math. Anal. Appl. 95, 344–374 (1983)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Idaho, Moscow, ID, 83844, USA

    Fuchang Gao

  2. Department of Mathematics, University of Michigan-Flint, Flint, MI, 48502, USA

    Lixing Han

Authors
  1. Fuchang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lixing Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lixing Han.

Additional information

F. Gao was supported in part by NSF Grant DMS-0405855.

L. Han was supported in part by a Research and Creative Activities Grant from UM-Flint.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gao, F., Han, L. Implementing the Nelder-Mead simplex algorithm with adaptive parameters. Comput Optim Appl 51, 259–277 (2012). https://doi.org/10.1007/s10589-010-9329-3

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-010-9329-3

Keywords

Search

Navigation