Abstract
Choosing the right selection rate is a long standing issue in evolutionary computation. In the continuous unconstrained case, we prove mathematically that a single parent \(\mu =1\) leads to a sub-optimal simple regret in the case of the sphere function. We provide a theoretically-based selection rate \(\mu /\lambda \) that leads to better progress rates. With our choice of selection rate, we get a provable regret of order \(O(\lambda ^{-1})\) which has to be compared with \(O(\lambda ^{-2/d})\) in the case where \(\mu =1\). We complete our study with experiments to confirm our theoretical claims.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arnold, D.V.: Optimal weighted recombination. In: Wright, A.H., Vose, M.D., De Jong, K.A., Schmitt, L.M. (eds.) FOGA 2005. LNCS, vol. 3469, pp. 215–237. Springer, Heidelberg (2005). https://doi.org/10.1007/11513575_12
Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. JMLR 13, 281–305 (2012)
Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13, 281–305 (2012)
Beyer, H.G., Schwefel, H.P.: Evolution strategies -a comprehensive introduction. Natural Comput. Int. J. 1(1), 3–52 (2002). https://doi.org/10.1023/A:1015059928466
Beyer, H.-G., Sendhoff, B.: Covariance matrix adaptation revisited – the CMSA evolution strategy –. In: Rudolph, G., Jansen, T., Beume, N., Lucas, S., Poloni, C. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 123–132. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87700-4_13
Bousquet, O., Gelly, S., Karol, K., Teytaud, O., Vincent, D.: Critical hyper-parameters: No random, no cry (2017, preprint). https://arxiv.org/pdf/1706.03200.pdf
Bubeck, S., Munos, R., Stoltz, G.: Pure exploration in multi-armed bandits problems. In: Gavaldà, R., Lugosi, G., Zeugmann, T., Zilles, S. (eds.) ALT 2009. LNCS (LNAI), vol. 5809, pp. 23–37. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04414-4_7
Escalante, H., Reyes, A.M.: Evolution strategies. CCC-INAOE tutorial (2013)
Fournier, H., Teytaud, O.: Lower bounds for comparison based evolution strategies using VC-dimension and sign patterns. Algorithmica (2010)
Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 11(1), 159–195 (2003)
Hansen, N., Arnold, D.V., Auger, A.: Evolution strategies. In: Kacprzyk, J., Pedrycz, W. (eds.) Springer Handbook of Computational Intelligence, pp. 871–898. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-43505-2_44
Jebalia, M., Auger, A.: Log-linear convergence of the scale-invariant (\({\mu }/{\mu }_{w{\lambda }}\))-ES and optimal \({\mu }\) for intermediate recombination for large population sizes. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN 2010. LNCS, vol. 6238, pp. 52–62. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15844-5_6
McKay, M.D., Beckman, R.J., Conover, W.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2), 239–245 (1979)
Niederreiter, H.: Random Number Generation and quasi-Monte Carlo Methods. Society for Industrial and Applied Mathematics, Philadelphia (1992)
Rapin, J., Teytaud, O.: Nevergrad - A gradient-free optimization platform. https://GitHub.com/FacebookResearch/Nevergrad (2018)
Teytaud, F.: A new selection ratio for large population sizes. In: Di Chio, C., et al. (eds.) EvoApplications 2010. LNCS, vol. 6024, pp. 452–460. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12239-2_47
Teytaud, F., Teytaud, O.: Why one must use reweighting in estimation of distribution algorithms. In: Genetic and Evolutionary Computation Conference, GECCO 2009, Proceedings, Montreal, Québec, Canada, 8–12 July 2009, pp. 453–460 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Meunier, L., Chevaleyre, Y., Rapin, J., Royer, C.W., Teytaud, O. (2020). On Averaging the Best Samples in Evolutionary Computation. In: Bäck, T., et al. Parallel Problem Solving from Nature – PPSN XVI. PPSN 2020. Lecture Notes in Computer Science(), vol 12270. Springer, Cham. https://doi.org/10.1007/978-3-030-58115-2_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-58115-2_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58114-5
Online ISBN: 978-3-030-58115-2
eBook Packages: Computer ScienceComputer Science (R0)