Lower Bounds for Evolution Strategies Using VC-Dimension

  • Olivier Teytaud
  • Hervé Fournier
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5199)

Abstract

We derive lower bounds for comparison-based or selection-based algorithms, improving existing results in the continuous setting, and extending them to non-trivial results in the discrete case. This is achieved by considering the VC-dimension of the level sets of the fitness functions; results are then obtained through the use of Sauer’s lemma. In the special case of optimization of the sphere function, improved lower bounds are obtained by bounding the possible number of sign conditions realized by some systems of equations.

Keywords

Evolution Strategies Convergence ratio VC-dimension Sign conditions 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Arnold, D.V.: Optimal weighted recombination. In: Wright, A.H., Vose, M.D., De Jong, K.A., Schmitt, L.M. (eds.) FOGA 2005. LNCS, vol. 3469, pp. 215–237. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  2. 2.
    Auger, A.: Convergence results for (1,λ)-SA-ES using the theory of ϕ-irreducible Markov chains. Theoretical Computer Science 334(1-3), 35–69 (2005)CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Bäck, T., Hoffmeister, F., Schwefel, H.-P.: Extended selection mechanisms in genetic algorithms. In: Belew, R.K., Booker, L.B. (eds.) Proceedings of the Fourth International Conference on Genetic Algorithms, pp. 92–99. Morgan Kaufmann Publishers, San Mateo (1991)Google Scholar
  4. 4.
    Baker, J.E.: Reducing bias and inefficiency in the selection algorithm. In: Proceedings of the Second International Conference on Genetic Algorithms on Genetic algorithms and their application, pp. 14–21. Lawrence Erlbaum Associates, Inc., Mahwah (1987)Google Scholar
  5. 5.
    Beyer, H.-G.: Toward a theory of evolution strategies: On the benefit of sex - the (μ/μ,λ)-theory. Evolutionary Computation 3(1), 81–111 (1995)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Beyer, H.-G., Schwefel, H.-P.: Evolution strategies: a comprehensive introduction. Natural Computing 1(1), 3–52 (2002)CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Devroye, L., Györfi, L., Lugosi, G.: A probabilistic Theory of Pattern Recognition. Springer, Heidelberg (1997)Google Scholar
  8. 8.
    Droste, S.: Not all linear functions are equally difficult for the compact genetic algorithm. In: Proc. of the Genetic and Evolutionary Computation COnference (GECCO 2005), pp. 679–686 (2005)Google Scholar
  9. 9.
    Feller, W.: An introduction to Probability Theory and its Applications. Wiley, Chichester (1968)MATHGoogle Scholar
  10. 10.
    Gelly, S., Ruette, S., Teytaud, O.: Comparison-based algorithms are robust and randomized algorithms are anytime. Evolutionary Computation Journal (MIT Press), Special issue on bridging Theory and Practice 15(4), 411–434 (2007)Google Scholar
  11. 11.
    Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159–195 (2001)CrossRefGoogle Scholar
  12. 12.
    Hooke, R., Jeeves, T.A.: ”Direct search” solution of numerical and statistical problems. Journal of the ACM 8(2), 212–229 (1961)CrossRefMATHGoogle Scholar
  13. 13.
    Jägersküpper, J., Witt, C.: Rigorous runtime analysis of a (μ + 1)ES for the sphere function. In: GECCO, pp. 849–856 (2005)Google Scholar
  14. 14.
    Matoušek, J.: Lectures on Discrete Geometry. Graduate Texts in Mathematics, vol. 212. Springer, Heidelberg (2002)CrossRefMATHGoogle Scholar
  15. 15.
    Rechenberg, I.: Evolutionstrategie: Optimierung Technischer Systeme nach Prinzipien des Biologischen Evolution. Fromman-Holzboog Verlag, Stuttgart (1973)Google Scholar
  16. 16.
    Rudolph, G.: Convergence rates of evolutionary algorithms for a class of convex objective functions. Control and Cybernetics 26(3), 375–390 (1997)MATHMathSciNetGoogle Scholar
  17. 17.
    Sauer, N.: On the density of families of sets. Journal of Combinatorial Theory, Ser. A 13(1), 145–147 (1972)CrossRefMATHGoogle Scholar
  18. 18.
    Teytaud, O., Gelly, S.: General lower bounds for evolutionary algorithms. In: Proceedings of PPSN, pp. 21–31 (2006)Google Scholar
  19. 19.
    Vapnik, V.N., Chervonenkis, A.Ya.: On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications XVI(2), 264–280 (1971)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Whitley, D.: The GENITOR algorithm and selection pressure: Why rank-based allocation of reproductive trials is best. In: Schaffer, J.D. (ed.) Proceedings of the Third International Conference on Genetic Algorithms, pp. 116–121. Morgan Kaufmann, San Mateo (1989)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Olivier Teytaud
    • 1
  • Hervé Fournier
    • 2
  1. 1.TAO (Inria), LRI, UMR 8623 (CNRS - Univ. Paris-Sud) ,Bât 490, Univ. Paris-SudOrsayFrance
  2. 2.Laboratoire PRiSM, CNRS UMR 8144 and Univ. Versailles St-Quentin en YvelinesVersaillesFrance

Personalised recommendations