Skip to main content

On Averaging the Best Samples in Evolutionary Computation

  • Conference paper
  • First Online:
Parallel Problem Solving from Nature – PPSN XVI (PPSN 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12270))

Included in the following conference series:

Abstract

Choosing the right selection rate is a long standing issue in evolutionary computation. In the continuous unconstrained case, we prove mathematically that a single parent \(\mu =1\) leads to a sub-optimal simple regret in the case of the sphere function. We provide a theoretically-based selection rate \(\mu /\lambda \) that leads to better progress rates. With our choice of selection rate, we get a provable regret of order \(O(\lambda ^{-1})\) which has to be compared with \(O(\lambda ^{-2/d})\) in the case where \(\mu =1\). We complete our study with experiments to confirm our theoretical claims.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Arnold, D.V.: Optimal weighted recombination. In: Wright, A.H., Vose, M.D., De Jong, K.A., Schmitt, L.M. (eds.) FOGA 2005. LNCS, vol. 3469, pp. 215–237. Springer, Heidelberg (2005). https://doi.org/10.1007/11513575_12

    Chapter  Google Scholar 

  2. Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. JMLR 13, 281–305 (2012)

    MathSciNet  MATH  Google Scholar 

  3. Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13, 281–305 (2012)

    MathSciNet  MATH  Google Scholar 

  4. Beyer, H.G., Schwefel, H.P.: Evolution strategies -a comprehensive introduction. Natural Comput. Int. J. 1(1), 3–52 (2002). https://doi.org/10.1023/A:1015059928466

    Article  MathSciNet  MATH  Google Scholar 

  5. Beyer, H.-G., Sendhoff, B.: Covariance matrix adaptation revisited – the CMSA evolution strategy –. In: Rudolph, G., Jansen, T., Beume, N., Lucas, S., Poloni, C. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 123–132. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87700-4_13

    Chapter  Google Scholar 

  6. Bousquet, O., Gelly, S., Karol, K., Teytaud, O., Vincent, D.: Critical hyper-parameters: No random, no cry (2017, preprint). https://arxiv.org/pdf/1706.03200.pdf

  7. Bubeck, S., Munos, R., Stoltz, G.: Pure exploration in multi-armed bandits problems. In: Gavaldà, R., Lugosi, G., Zeugmann, T., Zilles, S. (eds.) ALT 2009. LNCS (LNAI), vol. 5809, pp. 23–37. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04414-4_7

    Chapter  Google Scholar 

  8. Escalante, H., Reyes, A.M.: Evolution strategies. CCC-INAOE tutorial (2013)

    Google Scholar 

  9. Fournier, H., Teytaud, O.: Lower bounds for comparison based evolution strategies using VC-dimension and sign patterns. Algorithmica (2010)

    Google Scholar 

  10. Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 11(1), 159–195 (2003)

    Article  Google Scholar 

  11. Hansen, N., Arnold, D.V., Auger, A.: Evolution strategies. In: Kacprzyk, J., Pedrycz, W. (eds.) Springer Handbook of Computational Intelligence, pp. 871–898. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-43505-2_44

    Chapter  Google Scholar 

  12. Jebalia, M., Auger, A.: Log-linear convergence of the scale-invariant (\({\mu }/{\mu }_{w{\lambda }}\))-ES and optimal \({\mu }\) for intermediate recombination for large population sizes. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN 2010. LNCS, vol. 6238, pp. 52–62. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15844-5_6

    Chapter  Google Scholar 

  13. McKay, M.D., Beckman, R.J., Conover, W.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2), 239–245 (1979)

    MathSciNet  MATH  Google Scholar 

  14. Niederreiter, H.: Random Number Generation and quasi-Monte Carlo Methods. Society for Industrial and Applied Mathematics, Philadelphia (1992)

    Book  Google Scholar 

  15. Rapin, J., Teytaud, O.: Nevergrad - A gradient-free optimization platform. https://GitHub.com/FacebookResearch/Nevergrad (2018)

  16. Teytaud, F.: A new selection ratio for large population sizes. In: Di Chio, C., et al. (eds.) EvoApplications 2010. LNCS, vol. 6024, pp. 452–460. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12239-2_47

    Chapter  Google Scholar 

  17. Teytaud, F., Teytaud, O.: Why one must use reweighting in estimation of distribution algorithms. In: Genetic and Evolutionary Computation Conference, GECCO 2009, Proceedings, Montreal, Québec, Canada, 8–12 July 2009, pp. 453–460 (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Laurent Meunier .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Meunier, L., Chevaleyre, Y., Rapin, J., Royer, C.W., Teytaud, O. (2020). On Averaging the Best Samples in Evolutionary Computation. In: Bäck, T., et al. Parallel Problem Solving from Nature – PPSN XVI. PPSN 2020. Lecture Notes in Computer Science(), vol 12270. Springer, Cham. https://doi.org/10.1007/978-3-030-58115-2_46

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58115-2_46

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58114-5

  • Online ISBN: 978-3-030-58115-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics