General Lower Bounds for Evolutionary Algorithms

  • Olivier Teytaud
  • Sylvain Gelly
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4193)


Evolutionary optimization, among which genetic optimization, is a general framework for optimization. It is known (i) easy to use (ii) robust (iii) derivative-free (iv) unfortunately slow. Recent work [8] in particular show that the convergence rate of some widely used evolution strategies (evolutionary optimization for continuous domains) can not be faster than linear (i.e. the logarithm of the distance to the optimum can not decrease faster than linearly), and that the constant in the linear convergence (i.e. the constant C such that the distance to the optimum after n steps is upper bounded by C n ) unfortunately converges quickly to 1 as the dimension increases to ∞. We here show a very wide generalization of this result: all comparison-based algorithms have such a limitation. Note that our result also concerns methods like the Hooke & Jeeves algorithm, the simplex method, or any direct search method that only compares the values to previously seen values of the fitness. But it does not cover methods that use the value of the fitness (see [5] for cases in which the fitness-values are used), even if these methods do not use gradients. The former results deal with convergence with respect to the number of comparisons performed, and also include a very wide family of algorithms with respect to the number of function-evaluations. However, there is still place for faster convergence rates, for more original algorithms using the full ranking information of the population and not only selections among the population. We prove that, at least in some particular cases, using the full ranking information can improve these lower bounds, and ultimately provide superlinear convergence results.


Convergence Rate Evolutionary Algorithm Evolutionary Computation Continuous Case Superlinear Convergence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Auger, A.: Convergence results for (1,λ)-SA-ES using the theory of ϕ-irreducible markov chains. Theoretical Computer Science (in press, 2005)Google Scholar
  2. 2.
    Auger, A., Jebalia, M., Teytaud, O.: Xse: quasi-random mutations for evolution strategies. In: Proceedings of Evolutionary Algorithms, pages 12 (2005)Google Scholar
  3. 3.
    Devroye, L., Györfi, L., Lugosi, G.: A probabilistic Theory of Pattern Recognition. Springer, Heidelberg (1997)Google Scholar
  4. 4.
    Droste, S.: Not all linear functions are equally difficult for the compact genetic algorithm. In: Proc. of the Genetic and Evolutionary Computation COnference (GECCO 2005), pp. 679–686 (2005)Google Scholar
  5. 5.
    Droste, S., Jansen, T., Wegener, I.: Upper and lower bounds for randomized search heuristics in black-box optimization (2003)Google Scholar
  6. 6.
    Feller, W.: An introduction to Probability Theory and its Applications. Wiley, Chichester (1968)MATHGoogle Scholar
  7. 7.
    Hooke, R., Jeeves, T.A.: Direct search solution of numerical and statistical problems. Journal of the ACM 8, 212–229 (1961)MATHCrossRefGoogle Scholar
  8. 8.
    Jagerskupper, J., Witt, C.: Runtime analysis of a (mu+1)es for the sphere function. Technical report (2005)Google Scholar
  9. 9.
    Nelder, J., Mead, R.: A simplex method for function minimization. Computer Journal 7, 308–311 (1965)MATHGoogle Scholar
  10. 10.
    Rudolph, G.: Convergence rates of evolutionary algorithms for a class of convex objective functions. Control and Cybernetics 26(3), 375–390 (1997)MATHMathSciNetGoogle Scholar
  11. 11.
    Teytaud, O., Gelly, S., Mary, J.: On the ultimate convergence rates for isotropic algorithms and the best choices among various forms of isotropy, ppsn (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Olivier Teytaud
    • 1
  • Sylvain Gelly
    • 1
  1. 1.TAO (Inria), LRI, UMR 8623(CNRS – Univ. Paris-Sud)Univ. Paris-SudOrsayFrance

Personalised recommendations