Abstract
Multi-Modal Optimization (MMO) is ubiquitous in engineering, machine learning and artificial intelligence applications. Many algorithms have been proposed for multimodal optimization, and many of them are based on restart strategies. However, only few works address the issue of initialization in restarts. Furthermore, very few comparisons have been done, between different MMO algorithms, and against simple baseline methods. This paper proposes an analysis of restart strategies, and provides a restart strategy for any local search algorithm for which theoretical guarantees are derived. This restart strategy is to decrease some ’step-size’, rather than to increase the population size, and it uses quasi-random initialization, that leads to a rigorous proof of improvement with respect to random restarts or restarts with constant initial step-size. Furthermore, when this strategy encapsulates a (1+1)-ES with 1/5th adaptation rule, the resulting algorithm outperforms state of the art MMO algorithms while being computationally faster.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Augr, A., Hansen, N.: A restart CMA evolution strategy with increasing population size. In: Proceedings of the IEEE Congress on Evolutionary Computation, CEC 2005, pp. 1769–1776 (2005)
Auger, A., Jebalia, M., Teytaud, O.: Algorithms (X, sigma, eta): Quasi-random Mutations for Evolution Strategies. In: Talbi, E.-G., Liardet, P., Collet, P., Lutton, E., Schoenauer, M. (eds.) EA 2005. LNCS, vol. 3871, pp. 296–307. Springer, Heidelberg (2006)
Auger, A., Teytaud, O.: Continuous lunches are free plus the design of optimal optimization algorithms. Algorithmica 57(1), 121–146 (2010)
Beyer, H.-G., Sendhoff, B.: Covariance Matrix Adaptation Revisited – The CMSA Evolution Strategy –. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN X. LNCS, vol. 5199, pp. 123–132. Springer, Heidelberg (2008)
Beyer, H.-G.: The Theory of Evolution Strategies. Springer, Heidelberg (2001)
DeJong, K.: The Analysis of the Behavior of a Class of Genetic Adaptive Systems. PhD thesis, University of Michigan, Ann Harbor (1975); Dissertation Abstract International 36(10), 5140B (University Microfilms No 76-9381)
Georgieva, A., Jordanov, I.: A hybrid meta-heuristic for global optimisation using low-discrepancy sequences of points. Computers and Operations Research - Special Issue on Hybrid Metaheuristics (in press)
Goldberg, D.E., Richardson, J.: Genetic algorithms with sharing for multimodalfunction optimization. In: Grefenstette, J.J. (ed.) ICGA, pp. 41–49. Lawrence Erlbaum Associates (1987)
Hansen, N., Kern, S.: Evaluating the CMA Evolution Strategy on Multimodal Test Functions. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN VIII. LNCS, vol. 3242, pp. 282–291. Springer, Heidelberg (2004)
Harik, G.: Finding multiple solutions in problems of bounded difficulty. Technical Report 94002, University of Illinois at Urbana Champaign (1994)
Harik, G.: Finding multimodal solutions using restricted tournament selection. In: Eshelman, L.J. (ed.) Proc. Sixth International Conference on Genetic Algorithms, pp. 24–31. Morgan Kaufmann (1995)
Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. of Global Optimization 13(4), 455–492 (1998)
Kimura, S., Matsumura, K.: Genetic algorithms using low-discrepancy sequences. In: GECCO, pp. 1341–1346 (2005)
L’Ecuyer, P., Lemieux, C.: Recent Advances in Randomized Quasi-Monte Carlo Methods, pp. 419–474. Kluwer Academic (2002)
Li, J., Balazs, M., Parks, G., Clarkson, P.: A species conserving genetic algorithm for multimodal function optimization. Evolutionary Computation 10(3), 207–234 (2002)
Lindemann, S.R., LaValle, S.M.: Incremental low-discrepancy lattice methods for motion planning. In: Proceedings IEEE International Conference on Robotics and Automation, pp. 2920–2927 (2003)
Mahfoud, S.W.: Niching methods for genetic algorithms. PhD thesis, University of Illinois at Urbana-Champaign, Champaign, IL, USA (1995)
Mengshoel, O.J., Goldberg, D.E.: Probabilistic crowding: Deterministic crowding with probabilisitic replacement. In: Banzhaf, W., Daida, J., Eiben, A.E., Garzon, M.H., Honavar, V., Jakiela, M., Smith, R.E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference, Orlando, Florida, USA, July 13-17, vol. 1, pp. 409–416. Morgan Kaufmann (1999)
Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo Methods. SIAM, Philadelphia (1992)
Owen, A.B.: Quasi-Monte Carlo sampling. In: Jensen, H.W. (ed.) Monte Carlo Ray Tracing: Siggraph 2003 Course 44, pp. 69–88. SIGGRAPH (2003)
Pétrowski, A.: A clearing procedure as a niching method for genetic algorithms. In: International Conference on Evolutionary Computation, pp. 798–803 (1996)
Singh, G., Kalyanmoy Deb, D.: Comparison of multi-modal optimization algorithms based on evolutionary algorithms. In: GECCO 2006: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, pp. 1305–1312. ACM, New York (2006)
Teytaud, F., Teytaud, O.: Log( λ ) Modifications for Optimal Parallelism. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN XI. LNCS, vol. 6238, pp. 254–263. Springer, Heidelberg (2010)
Teytaud, O.: When Does Quasi-random Work? In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN X. LNCS, vol. 5199, pp. 325–336. Springer, Heidelberg (2008)
Teytaud, O., Gelly, S.: DCMA: yet another derandomization in covariance-matrix-adaptation. In: Thierens, D., et al. (eds.) ACM-GECCO 2007, pp. 955–963. ACM, New York (2007)
van den Bergh, F., Engelbrecht, A.: Cooperative learning in neural networks using particle swarm optimizers (2000)
Vandewoestyne, B., Cools, R.: Good permutations for deterministic scrambled halton sequences in terms of l2-discrepancy. Computational and Applied Mathematics 189(1,2), 341–361 (2006) bibitemiccama Vandewoestyne, B., Cools, R.: Good permutations for deterministic scrambled halton sequences in terms of l2-discrepancy. Computational and Applied Mathematics 189(1,2), 341–361 (2006)
Villemonteix, J., Vazquez, E., Walter, E.: An informational approach to the global optimization of expensive-to-evaluate functions. Journal of Global Optimization 44(4), 509–534 (2009)
Yin, X., Germay, N.: A fast genetic algorithm with sharing scheme using cluster analysis methods in multimodal function optimization. In: Albrecht, R.F., Steele, N.C., Reeves, C.R. (eds.) Artificial Neural Nets and Genetic Algorithms, pp. 450–457. Springer (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Schoenauer, M., Teytaud, F., Teytaud, O. (2012). A Rigorous Runtime Analysis for Quasi-Random Restarts and Decreasing Stepsize. In: Hao, JK., Legrand, P., Collet, P., Monmarché, N., Lutton, E., Schoenauer, M. (eds) Artificial Evolution. EA 2011. Lecture Notes in Computer Science, vol 7401. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35533-2_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-35533-2_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35532-5
Online ISBN: 978-3-642-35533-2
eBook Packages: Computer ScienceComputer Science (R0)