On the Ultimate Convergence Rates for Isotropic Algorithms and the Best Choices Among Various Forms of Isotropy
In this paper, we show universal lower bounds for isotropic algorithms, that hold for any algorithm such that each new point is the sum of one already visited point plus one random isotropic direction multiplied by any step size (whenever the step size is chosen by an oracle with arbitrarily high computational power). The bound is 1–O(1/d) for the constant in the linear convergence (i.e. the constant C such that the distance to the optimum after n steps is upper bounded by C n ), as already seen for some families of evolution strategies in [19,12], in contrast with 1–O(1) for the reverse case of a random step size and a direction chosen by an oracle with arbitrary high computational power. We then recall that isotropy does not uniquely determine the distribution of a sample on the sphere and show that the convergence rate in isotropic algorithms is improved by using stratified or antithetic isotropy instead of naive isotropy. We show at the end of the paper that beyond the mathematical proof, the result holds on experiments. We conclude that one should use antithetic-isotropy or stratified-isotropy, and never standard-isotropy.
KeywordsConvergence Rate Unit Sphere Sphere Function Linear Convergence Random Independent Sampling
Unable to display preview. Download preview PDF.
- 1.Auger, A.: Convergence results for (1,λ)-SA-ES using the theory of ϕ-irreducible markov chains. Theoretical Computer Science (in press, 2005)Google Scholar
- 2.Auger, A., Jebalia, M., Teytaud, O.: Xse: quasi-random mutations for evolution strategies. In: Proceedings of Evolutionary Algorithms, pages 12 (2005)Google Scholar
- 3.Beyer, H.-G.: The Theory of Evolutions Strategies. Springer, Heidelberg (2001)Google Scholar
- 6.Conway, J.H., Sloane, N.J.: Sphere packings, lattices and groups (1998)Google Scholar
- 8.Frahm, G., Junker, M.: Generalized elliptical distributions: Models and estimation. Technical Report 0037 (2003)Google Scholar
- 11.Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 11(1) (2003)Google Scholar
- 12.Jagerskupper, J.: In between progress rate and stochastic convergence. Dagstuhl’s seminar (2006)Google Scholar
- 13.Literka: A remarkable monotonic property of the gamma function. Technical report (2005)Google Scholar
- 15.Salomon, R.: The deterministic genetic algorithm: Implementation details and some results (1999)Google Scholar
- 17.Teytaud, O., Gelly, S.: General lower bounds for evolutionary algorithms, ppsn (2006)Google Scholar
- 19.Witt, C., Jägersküpper, J.: Rigorous runtime analysis of a (mu+1) es for the sphere function. In: Proceedings of the 2005 Conference on Genetic and Evolutionary Computation, pp. 849–856 (2005)Google Scholar