Skip to main content
SpringerLink
Log in

Improving Nevergrad’s Algorithm Selection Wizard NGOpt Through Automated Algorithm Configuration

  • Conference paper
  • First Online:
Parallel Problem Solving from Nature – PPSN XVII (PPSN 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13398))

Included in the following conference series:

Abstract

Algorithm selection wizards are effective and versatile tools that automatically select an optimization algorithm given high-level information about the problem and available computational resources, such as number and type of decision variables, maximal number of evaluations, possibility to parallelize evaluations, etc. State-of-the-art algorithm selection wizards are complex and difficult to improve. We propose in this work the use of automated configuration methods for improving their performance by finding better configurations of the algorithms that compose them. In particular, we use elitist iterated racing (irace) to find CMA configurations for specific artificial benchmarks that replace the hand-crafted CMA configurations currently used in the NGOpt wizard provided by the Nevergrad platform. We discuss in detail the setup of irace for the purpose of generating configurations that work well over the diverse set of problem instances within each benchmark. Our approach improves the performance of the NGOpt wizard, even on benchmark suites that were not part of the tuning by irace.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aziz-Alaoui, A., Doerr, C., Dréo, J.: Towards large scale automated algorithm design by integrating modular benchmarking frameworks. In: Chicano, F., Krawiec, K. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO Companion 2021, New York, NY, pp. 1365–1374. ACM Press (2021). https://doi.org/10.1145/3449726.3463155

  2. Bezerra, L.C.T., López-Ibáñez, M., Stützle, T.: Automatically designing state-of-the-art multi- and many-objective evolutionary algorithms. Evol. Comput. 28(2), 195–226 (2020). https://doi.org/10.1162/evco_a_00263

    Article  Google Scholar 

  3. Birattari, M.: Tuning Metaheuristics: A Machine Learning Perspective, Studies in Computational Intelligence, vol. 197. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-00483-4

  4. Cartis, C., Fiala, J., Marteau, B., Roberts, L.: Improving the flexibility and robustness of model-based derivative-free optimization solvers (2018)

    Google Scholar 

  5. Hansen, N., Auger, A., Finck, S., Ros, R.: Real-parameter black-box optimization benchmarking 2009: Experimental setup. Technical report, RR-6828, INRIA, France (2009). https://hal.inria.fr/inria-00362633/document

  6. Hansen, N., Auger, A., Ros, R., Mersmann, O., Tušar, T., Brockhoff, D.: COCO: a platform for comparing continuous optimizers in a black-box setting. Optim. Meth. Software 36(1), 1–31 (2020). https://doi.org/10.1080/10556788.2020.1808977

    Article  MathSciNet  MATH  Google Scholar 

  7. Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 9(2), 159–195 (2001). https://doi.org/10.1162/106365601750190398

    Article  Google Scholar 

  8. Hoos, H.H.: Automated algorithm configuration and parameter tuning. In: Hamadi, Y., Monfroy, E., Saubion, F. (eds.) Autonomous Search, pp. 37–71. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21434-9_3

    Chapter  Google Scholar 

  9. Huang, C., Li, Y., Yao, X.: A survey of automatic parameter tuning methods for metaheuristics. IEEE Trans. Evol. Comput. 24(2), 201–216 (2020). https://doi.org/10.1109/TEVC.2019.2921598

    Article  Google Scholar 

  10. Johnson, S.G.: The nlopt nonlinear-optimization package (1994). http://github.com/stevengj/nlopt

  11. Kerschke, P., Hoos, H.H., Neumann, F., Trautmann, H.: Automated algorithm selection: survey and perspectives. Evol. Comput. 27(1), 3–45 (2019). https://doi.org/10.1162/evco_a_00242

    Article  Google Scholar 

  12. KhudaBukhsh, A.R., Xu, L., Hoos, H.H., Leyton-Brown, K.: SATenstein: automatically building local search SAT solvers from components. Artif. Intell. 232, 20–42 (2016). https://doi.org/10.1016/j.artint.2015.11.002

    Article  MathSciNet  MATH  Google Scholar 

  13. Liao, T., Molina, D., Stützle, T.: Performance evaluation of automatically tuned continuous optimizers on different benchmark sets. Appl. Soft Comput. 27, 490–503 (2015)

    Article  Google Scholar 

  14. Liao, T., Montes de Oca, M.A., Stützle, T.: Computational results for an automatically tuned CMA-ES with increasing population size on the CEC 2005 benchmark set. Soft Comput. 17(6), 1031–1046 (2013). https://doi.org/10.1007/s00500-012-0946-x

  15. Liao, T., Stützle, T., Montes de Oca, M.A., Dorigo, M.: A unified ant colony optimization algorithm for continuous optimization. Eur. J. Oper. Res. 234(3), 597–609 (2014)

    Article  MathSciNet  Google Scholar 

  16. López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Stützle, T., Birattari, M.: The irace pacskage: iterated racing for automatic algorithm configuration. Oper. Res. Perspect. 3, 43–58 (2016). https://doi.org/10.1016/j.orp.2016.09.002

    Article  MathSciNet  Google Scholar 

  17. López-Ibáñez, M., Liao, T., Stützle, T.: On the anytime behavior of IPOP-CMA-ES. In: Coello, C.A.C., Cutello, V., Deb, K., Forrest, S., Nicosia, G., Pavone, M. (eds.) PPSN 2012. LNCS, vol. 7491, pp. 357–366. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32937-1_36

    Chapter  Google Scholar 

  18. López-Ibáñez, M., Stützle, T.: The automatic design of multi-objective ant colony optimization algorithms. IEEE Trans. Evol. Comput. 16(6), 861–875 (2012). https://doi.org/10.1109/TEVC.2011.2182651

    Article  Google Scholar 

  19. Mascia, F., López-Ibáñez, M., Dubois-Lacoste, J., Stützle, T.: Grammar-based generation of stochastic local search heuristics through automatic algorithm configuration tools. Comput. Oper. Res. 51, 190–199 (2014). https://doi.org/10.1016/j.cor.2014.05.020

    Article  MathSciNet  MATH  Google Scholar 

  20. Meunier, L., et al.: Black-box optimization revisited: improving algorithm selection wizards through massive benchmarking. IEEE Trans. Evol. Comput. 26(3), 490–500 (2022). https://doi.org/10.1109/TEVC.2021.3108185

    Article  Google Scholar 

  21. Pagnozzi, F., Stützle, T.: Automatic design of hybrid stochastic local search algorithms for permutation flowshop problems. Eur. J. Oper. Res. 276, 409–421 (2019). https://doi.org/10.1016/j.ejor.2019.01.018

    Article  MathSciNet  MATH  Google Scholar 

  22. Rapin, J., Teytaud, O.: Nevergrad: a gradient-free optimization platform (2018). https://github.com/FacebookResearch/Nevergrad

  23. Rice, J.R.: The algorithm selection problem. Adv. Comput. 15, 65–118 (1976)

    Article  Google Scholar 

  24. Ros, R., Hansen, N.: A simple modification in CMA-ES achieving linear time and space complexity. In: Rudolph, G., Jansen, T., Beume, N., Lucas, S., Poloni, C. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 296–305. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87700-4_30

    Chapter  Google Scholar 

  25. Schede, E., et al.: A survey of methods for automated algorithm configuration (2022). https://doi.org/10.48550/ARXIV.2202.01651

  26. Styles, J., Hoos, H.H.: Ordered racing protocols for automatically configuring algorithms for scaling performance. In: Blum, C., Alba, E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2013, New York, NY, pp. 551–558. ACM Press (2013). ISBN 978-1-4503-1963-8, https://doi.org/10.1145/2463372.2463438

  27. Videau, M., Leite, A., Teytaud, O., Schoenauer, M.: Multi-objective genetic programming for explainable reinforcement learning. In: Medvet, E., Pappa, G., Xue, B. (eds.) EuroGP 2022. LNCS, vol. 13223, pp. 256–281. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-02056-8_18

    Chapter  Google Scholar 

  28. Xu, L., Hoos, H.H., Leyton-Brown, K.: Hydra: automatically configuring algorithms for portfolio-based selection. In: Fox, M., Poole, D. (eds.) Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press (2010)

    Google Scholar 

  29. Xu, L., Hutter, F., Hoos, H.H., Leyton-Brown, K.: SATzilla: portfolio-based algorithm selection for SAT. J. Artif. Intell. Res. 32, 565–606 (2008). https://doi.org/10.1613/jair.2490

    Article  MATH  Google Scholar 

Download references

Acknowledgments

M. López-Ibáñez is a “Beatriz Galindo” Senior Distinguished Researcher (BEAGAL 18/00053) funded by the Spanish Ministry of Science and Innovation (MICINN). C. Doerr is supported by the Paris Ile-de-France Region (AlgoSelect) and by the INS2I institute of CNRS (RandSearch). T. Eftimov, A. Nikolikj, and G. Cenikj is supported by the Slovenian Research Agency: research core fundings No. P2-0098 and project No. N2-0239. G. Cenikj is also supported by the Ad Futura grant for postgraduate study.

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Engineering, Ss. Cyril and Methodius Skopje, Skopje, North Macedonia

    Risto Trajanov

  2. Jožef Stefan Institute, Ljubljana, Slovenia

    Ana Nikolikj, Gjorgjina Cenikj & Tome Eftimov

  3. Jožef Stefan International Postgraduate School, Ljubljana, Slovenia

    Ana Nikolikj & Gjorgjina Cenikj

  4. Université Littoral Côte d’Opale, Dunkirk, France

    Fabien Teytaud

  5. Facebook AI Research, Paris, France

    Mathurin Videau & Olivier Teytaud

  6. ITIS Software, Universidad de Málaga, Málaga, Spain

    Manuel López-Ibáñez

  7. Sorbonne Université, CNRS, LIP6, Paris, France

    Carola Doerr

Authors
  1. Risto Trajanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ana Nikolikj
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gjorgjina Cenikj
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fabien Teytaud
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mathurin Videau
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Olivier Teytaud
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tome Eftimov
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Manuel López-Ibáñez
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Carola Doerr
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tome Eftimov .

Editor information

Editors and Affiliations

  1. TU Dortmund, Dortmund, Germany

    Günter Rudolph

  2. Leiden University, Leiden, The Netherlands

    Anna V. Kononova

  3. Shinshu University, Nagano, Japan

    Hernán Aguirre

  4. Technische Universität Dresden, Dresden, Germany

    Pascal Kerschke

  5. University of Stirling, Stirling, UK

    Gabriela Ochoa

  6. Jožef Stefan Institute, Ljubljana, Slovenia

    Tea Tušar

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Trajanov, R. et al. (2022). Improving Nevergrad’s Algorithm Selection Wizard NGOpt Through Automated Algorithm Configuration. In: Rudolph, G., Kononova, A.V., Aguirre, H., Kerschke, P., Ochoa, G., Tušar, T. (eds) Parallel Problem Solving from Nature – PPSN XVII. PPSN 2022. Lecture Notes in Computer Science, vol 13398. Springer, Cham. https://doi.org/10.1007/978-3-031-14714-2_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-14714-2_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-14713-5

  • Online ISBN: 978-3-031-14714-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation