The importance of implementation details and parameter settings in black-box optimization: a case study on Gaussian estimation-of-distribution algorithms and circles-in-a-square packing problems
- 168 Downloads
We consider a scalable problem that has strong ties with real-world problems, can be compactly formulated and efficiently evaluated, yet is not trivial to solve and has interesting characteristics that differ from most commonly used benchmark problems: packing n circles in a square (CiaS). Recently, a first study that used basic Gaussian EDAs indicated that typically suggested algorithmic parameter settings do not necessarily transfer well to the CiaS problem. In this article, we consider also AMaLGaM, an enhanced Gaussian EDA, as well as arguably the most powerful real-valued black-box optimization algorithm to date, CMA-ES, which can also be seen as a further enhanced Gaussian EDA. We study whether the well-known performance on typical benchmark problems extends to the CiaS problem. We find that although the enhancements over a basic Gaussian EDA result in superior performance, the further efficiency enhancements in CMA-ES are not highly impactful. Instead, the most impactful features are how constraint handling is performed, how large the population size is, whether a full covariance matrix is used and whether restart techniques are used. We further show that a previously published version of AMaLGaM that does not require the user to set the the population size parameter is capable of solving the problem and we derive the scalability of the required number of function evaluations to solve the problem up to 99.99 % of the known optimal value for up to 30 circles.
KeywordsBlack-box optimization Evolutionary computation Estimation-of-distribution algorithms Parameter tuning Constraint handling Circles-in-a-square packing
Compliance with ethical standards
Conflict of interest
Peter A. N. Bosman, Marcus Gallagher declare that they have no conflict of interest.
This article does not contain any studies with human participants or animals performed by any of the authors.
- Gallagher M (2009) Investigating circles in a square packing problems as a realistic benchmark for continuous metaheuristic optimization algorithms. In: MIC 2009: the VIII metaheuristics international conference, Hamburg, GermanyGoogle Scholar
- Hansen N (2006) The CMA evolution strategy: a comparing review. In: Lozano JA, Larrañaga P, Inza I, Bengoetxea E (eds) Towards a new evolutionary computation. Advances in estimation of distribution algorithms. Springer, Berlin, pp 75–102Google Scholar
- Hansen N, Auger A, Ros R, Finck S, Pošík P (2010) Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009. In: Branke J et al (eds) Proceedings of the genetic and evolutionary computation conference—GECCO-2010. ACM Press, New York, pp 1689–1696Google Scholar
- Larrañaga P, Lozano JA, Bengoetxea E (2001) Estimation of distribution algorithm based on multivariate normal and Gaussian networks. Technical Report KZZA-IK-1-01. University of the Basque Country, Department of Computer Science and Artificial IntelligenceGoogle Scholar
- Larrañaga P, Lozan JA (2001) Estimation of distribution algorithms. A new tool for evolutionary computation. Kluwer Academic, LondonGoogle Scholar
- Mahfoud S (2000) Niching methods. In: Bäck DFT, Michalewicz Z (eds) Evolutionary computation 2–advanced algorithms and operations, Chap 13. IOP Publishing Ltd, Bristol, pp 87–92Google Scholar
- Morgan R, Gallagher M (2014) Fitness landscape analysis of circles in a square packing problems. In: Dick G et al (eds) Simulated evolution and learning—SEAL-2014. Springer, Berlin, pp 455–466Google Scholar
- Specht E (2013) http://www.packomania.com/
- Szabó PG, Markót MC, Csendes T (2005) Global optimization in geometry—circle packing into the square. In: Audet P, Hansen P, Savard P (eds) Essays and surveys in global optimization. Springer, USGoogle Scholar