Computational Economics

, Volume 43, Issue 4, pp 447–461 | Cite as

Nelder-Mead Simplex Optimization Routine for Large-Scale Problems: A Distributed Memory Implementation

  • Kyle Klein
  • Julian Neira


The Nelder-Mead simplex method is an optimization routine that works well with irregular objective functions. For a function of \(n\) parameters, it compares the objective function at the \(n+1\) vertices of a simplex and updates the worst vertex through simplex search steps. However, a standard serial implementation can be prohibitively expensive for optimizations over a large number of parameters. We describe an implementation of the Nelder-Mead method in parallel using a distributed memory. For \(p\) processors, each processor is assigned \((n+1)/p\) vertices at each iteration. Each processor then updates its worst local vertices, communicates the results, and a new simplex is formed with the vertices from all processors. We also describe how the algorithm can be implemented with only two MPI commands. In simulations, our implementation exhibits large speedups and is scalable to large problem sizes.


Parallel computing Optimization algorithms Nelder-Mead 

JEL Classification



  1. Aldrich, E. M., Fernandez-Villaverde, J., Ronald Gallant, A., & Rubio-Ramirez, J. F. (2011). Tapping the supercomputer under your desk: Solving dynamic equilibrium models with graphics processors. Journal of Economic Dynamics and Control, 35, 386–393.CrossRefGoogle Scholar
  2. Beaumont, P. M., & Bradshaw, P. M. (1995). A distributed parallel genetic algorithm for solving optimal growth models. Computational Economics, 8, 159–179.CrossRefGoogle Scholar
  3. Creel, M. (2005). User-friendly parallel computations with econometric examples. Computational Economics, 26, 107–128.Google Scholar
  4. Dennis, J. E, Jr, & Torczon, V. (1991). Direct search methods on parallel machines. SIAM Journal on Optimization, 1, 448–474.CrossRefGoogle Scholar
  5. Ferrall, C. (2005). Solving finite mixture models: Efficient computation in economics under serial and parallel execution. Computational Economics, 25, 343–379.CrossRefGoogle Scholar
  6. Huggett, M., Ventura, G., & Yaron, A. (2006). Human capital and earnings distribution dynamics. Journal of Monetary Economics, 53, 265–290.CrossRefGoogle Scholar
  7. Lawver, D. (2012). Measuring quality increases in the medical sector. Santa Barbara: University of California Santa Barbara.Google Scholar
  8. Lee, D., & Wiswall, M. (2007). A parallel implementation of the simplex function minimization routine. Computational Economics, 30, 171–187.CrossRefGoogle Scholar
  9. Nelder, J. A., & Mead, R. (1965). A simplex method for function minimization. The Computer Journal, 7, 308–313.CrossRefGoogle Scholar
  10. Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (2007). Numerical recipes: The art of scientific computing (3rd ed.). Cambridge University Press.Google Scholar
  11. Swann, C. A. (2002). Maximum likelihood estimation using parallel computing: An introduction to MPI. Computational Economics, 19, 145–178.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.U.C. Santa BarbaraSanta BarbaraUSA

Personalised recommendations