Abstract
Randomized direct-search methods for the optimization of a function f:ℝn→ℝ given by a black box for f-evaluations are investigated. We consider the cumulative step-size adaptation (CSA) for the variance of multivariate zero-mean normal distributions. Those are commonly used to sample new candidate solutions within metaheuristics, in particular within the CMA Evolution Strategy (CMA-ES), a state-of-the-art direct-search method. Though the CMA-ES is very successful in practical optimization, its theoretical foundations are very limited because of the complex stochastic process it induces. To forward the theory on this successful method, we propose two simplifications of the CSA used within CMA-ES for step-size control. We show by experimental and statistical evaluation that they perform sufficiently similarly to the original CSA (in the considered scenario), so that a further theoretical analysis is in fact reasonable. Furthermore, we outline in detail a probabilistic/theoretical runtime analysis for one of the two CSA-derivatives.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Hansen, N., Ostermeier, A.: Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation. In: Proc. IEEE Int’l Conference on Evolutionary Computation (ICEC), pp. 312–317 (1996)
Hansen, N.: List of references to various applications of CMA-ES (2008), http://www.bionik.tu-berlin.de/user/niko/cmaapplications.pdf
Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) Evolutionary Algorithm. Theoretical Computer Science 276, 51–82 (2002)
Jägersküpper, J.: Algorithmic analysis of a basic evolutionary algorithm for continuous optimization. Theoretical Computer Science 379, 329–347 (2007)
Rechenberg, I.: Cybernetic solution path of an experimental problem. Royal Aircraft Establishment (1965)
Schwefel, H.-P.: Numerical Optimization of Computer Models. Wiley, New York (1981)
Jägersküpper, J.: Analysis of a simple evolutionary algorithm for minimization in Euclidean spaces. In: Baeten, J.C.M., Lenstra, J.K., Parrow, J., Woeginger, G.J. (eds.) ICALP 2003. LNCS, vol. 2719, pp. 1068–1079. Springer, Heidelberg (2003)
Jägersküpper, J.: How the (1+1) ES using isotropic mutations minimizes positive definite quadratic forms. Theoretical Computer Science 361, 38–56 (2005)
Beyer, H.-G.: The Theory of Evolution Strategies. Springer, Heidelberg (2001)
Jägersküpper, J.: Probabilistic runtime analysis of \((1\overset{+}{,}\lambda)\) ES using isotropic mutations. In: Proc. 2006 Genetic and Evolutionary Computation Conference (GECCo), pp. 461–468. ACM Press, New York (2006)
Beyer, H.-G., Arnold, D.V.: Performance analysis of evolutionary optimization with cumulative step-length adaptation. IEEE Transactions on Automatic Control, 617–622 (2004)
Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: New perspectives on some classical and modern methods. SIAM Review 45, 385–482 (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jägersküpper, J., Preuss, M. (2008). Empirical Investigation of Simplified Step-Size Control in Metaheuristics with a View to Theory. In: McGeoch, C.C. (eds) Experimental Algorithms. WEA 2008. Lecture Notes in Computer Science, vol 5038. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-68552-4_20
Download citation
DOI: https://doi.org/10.1007/978-3-540-68552-4_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-68548-7
Online ISBN: 978-3-540-68552-4
eBook Packages: Computer ScienceComputer Science (R0)