Skip to main content

Experimental Comparisons of Derivative Free Optimization Algorithms

(Invited Talk)

  • Conference paper
Experimental Algorithms (SEA 2009)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 5526))

Included in the following conference series:


In this paper, the performances of the quasi-Newton BFGS algorithm, the NEWUOA derivative free optimizer, the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), the Differential Evolution (DE) algorithm and Particle Swarm Optimizers (PSO) are compared experimentally on benchmark functions reflecting important challenges encountered in real-world optimization problems. Dependence of the performances in the conditioning of the problem and rotational invariance of the algorithms are in particular investigated.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others


  1. Scheinberg, K., Conn, A.R., Toint, P.L.: Recent progress in unconstrained nonlinear optimization without derivatives. Mathematical Programming 79(3), 397–415 (1997)

    MathSciNet  MATH  Google Scholar 

  2. Torczon, V.: On the convergence of pattern search algorithms. SIAM Journal on optimization 7(1), 1–25 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  3. Powell, M.J.D.: The NEWUOA software for unconstrained optimization without derivatives. In: Large Scale Nonlinear Optimization, pp. 255–297 (2006)

    Google Scholar 

  4. Suganthan, P.N., Hansen, N., Liang, J.J., Deb, K., Chen, Y.P., Auger, A., Tiwari, S.: Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Technical report, Nanyang Technological University, Singapore and KanGAL Report Number 2005005 (Kanpur Genetic Algorithms Laboratory, IIT Kanpur) (May 2005)

    Google Scholar 

  5. Rechenberg, I.: Evolutionsstrategie: Optimierung Technischer Systeme nach Prinzipien der Biologischen Evolution. Frommann-Holzboog (1973)

    Google Scholar 

  6. Schwefel, H.-P.: Evolution and Optimum Seeking. Sixth-Generation Computer Technology Series. John Wiley & Sons, Chichester (1995)

    MATH  Google Scholar 

  7. Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159–195 (2001)

    Article  Google Scholar 

  8. Hansen, N.: The CMA evolution strategy: a comparing review. In: Lozano, J.A., Larranaga, P., Inza, I., Bengoetxea, E. (eds.) Towards a new evolutionary computation. Advances on estimation of distribution algorithms, pp. 75–102. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  9. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, 1995, vol. 4, pp. 1942–1948 (1995)

    Google Scholar 

  10. Storn, R., Price, K.: Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J. of Global Optimization 11(4), 341–359 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  11. Hansen, N., Ros, R., Mauny, N., Schoenauer, M., Auger, A.: PSO facing non-separable and ill-conditioned problems. Research Report RR-6447, INRIA (2008)

    Google Scholar 

  12. Wilke, D.N., Kok, S., Groenwold, A.A.: Comparison of linear and classical velocity update rules in particle swarm optimization: Notes on scale and frame invariance. Int. J. Numer. Meth. Engng. 70, 985–1008 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  13. Shang, Y.-W., Qiu, Y.-H.: A note on the extended rosenbrock function. Evol. Comput. 14(1), 119–126 (2006)

    Article  Google Scholar 

  14. Hansen, N., Kern, S.: Evaluating the CMA evolution strategy on multimodal test functions. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 282–291. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  15. Auger, A., Hansen, N.: Performance evaluation of an advanced local search evolutionary algorithm. In: Proceedings of the IEEE Congress on Evolutionary Computation (2005)

    Google Scholar 

  16. Feoktistov, V.: Differential Evolution: In Search of Solutions. In: Optimization and Its Applications. Springer, New York (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Auger, A., Hansen, N., Perez Zerpa, J.M., Ros, R., Schoenauer, M. (2009). Experimental Comparisons of Derivative Free Optimization Algorithms. In: Vahrenhold, J. (eds) Experimental Algorithms. SEA 2009. Lecture Notes in Computer Science, vol 5526. Springer, Berlin, Heidelberg.

Download citation

  • DOI:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-02010-0

  • Online ISBN: 978-3-642-02011-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics