Experimental Comparisons of Derivative Free Optimization Algorithms

(Invited Talk)
  • A. Auger
  • N. Hansen
  • J. M. Perez Zerpa
  • R. Ros
  • M. Schoenauer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5526)

Abstract

In this paper, the performances of the quasi-Newton BFGS algorithm, the NEWUOA derivative free optimizer, the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), the Differential Evolution (DE) algorithm and Particle Swarm Optimizers (PSO) are compared experimentally on benchmark functions reflecting important challenges encountered in real-world optimization problems. Dependence of the performances in the conditioning of the problem and rotational invariance of the algorithms are in particular investigated.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Scheinberg, K., Conn, A.R., Toint, P.L.: Recent progress in unconstrained nonlinear optimization without derivatives. Mathematical Programming 79(3), 397–415 (1997)MathSciNetMATHGoogle Scholar
  2. 2.
    Torczon, V.: On the convergence of pattern search algorithms. SIAM Journal on optimization 7(1), 1–25 (1997)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Powell, M.J.D.: The NEWUOA software for unconstrained optimization without derivatives. In: Large Scale Nonlinear Optimization, pp. 255–297 (2006)Google Scholar
  4. 4.
    Suganthan, P.N., Hansen, N., Liang, J.J., Deb, K., Chen, Y.P., Auger, A., Tiwari, S.: Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Technical report, Nanyang Technological University, Singapore and KanGAL Report Number 2005005 (Kanpur Genetic Algorithms Laboratory, IIT Kanpur) (May 2005)Google Scholar
  5. 5.
    Rechenberg, I.: Evolutionsstrategie: Optimierung Technischer Systeme nach Prinzipien der Biologischen Evolution. Frommann-Holzboog (1973)Google Scholar
  6. 6.
    Schwefel, H.-P.: Evolution and Optimum Seeking. Sixth-Generation Computer Technology Series. John Wiley & Sons, Chichester (1995)MATHGoogle Scholar
  7. 7.
    Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159–195 (2001)CrossRefGoogle Scholar
  8. 8.
    Hansen, N.: The CMA evolution strategy: a comparing review. In: Lozano, J.A., Larranaga, P., Inza, I., Bengoetxea, E. (eds.) Towards a new evolutionary computation. Advances on estimation of distribution algorithms, pp. 75–102. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, 1995, vol. 4, pp. 1942–1948 (1995)Google Scholar
  10. 10.
    Storn, R., Price, K.: Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J. of Global Optimization 11(4), 341–359 (1997)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Hansen, N., Ros, R., Mauny, N., Schoenauer, M., Auger, A.: PSO facing non-separable and ill-conditioned problems. Research Report RR-6447, INRIA (2008)Google Scholar
  12. 12.
    Wilke, D.N., Kok, S., Groenwold, A.A.: Comparison of linear and classical velocity update rules in particle swarm optimization: Notes on scale and frame invariance. Int. J. Numer. Meth. Engng. 70, 985–1008 (2007)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Shang, Y.-W., Qiu, Y.-H.: A note on the extended rosenbrock function. Evol. Comput. 14(1), 119–126 (2006)CrossRefGoogle Scholar
  14. 14.
    Hansen, N., Kern, S.: Evaluating the CMA evolution strategy on multimodal test functions. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 282–291. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  15. 15.
    Auger, A., Hansen, N.: Performance evaluation of an advanced local search evolutionary algorithm. In: Proceedings of the IEEE Congress on Evolutionary Computation (2005)Google Scholar
  16. 16.
    Feoktistov, V.: Differential Evolution: In Search of Solutions. In: Optimization and Its Applications. Springer, New York (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • A. Auger
    • 1
    • 2
  • N. Hansen
    • 1
    • 2
  • J. M. Perez Zerpa
    • 1
  • R. Ros
    • 1
  • M. Schoenauer
    • 1
    • 2
  1. 1.TAO Projetct-Team, INRIA Saclay – Île-de-France LRI, Bat 490 Univ. Paris-SudOrsay CedexFrance
  2. 2.Microsoft Research-INRIA Joint CentreOrsay CedexFrance

Personalised recommendations