Soft Computing

, Volume 22, Issue 4, pp 1209–1223 | Cite as

The importance of implementation details and parameter settings in black-box optimization: a case study on Gaussian estimation-of-distribution algorithms and circles-in-a-square packing problems

  • Peter A. N. BosmanEmail author
  • Marcus Gallagher
Methodologies and Application


We consider a scalable problem that has strong ties with real-world problems, can be compactly formulated and efficiently evaluated, yet is not trivial to solve and has interesting characteristics that differ from most commonly used benchmark problems: packing n circles in a square (CiaS). Recently, a first study that used basic Gaussian EDAs indicated that typically suggested algorithmic parameter settings do not necessarily transfer well to the CiaS problem. In this article, we consider also AMaLGaM, an enhanced Gaussian EDA, as well as arguably the most powerful real-valued black-box optimization algorithm to date, CMA-ES, which can also be seen as a further enhanced Gaussian EDA. We study whether the well-known performance on typical benchmark problems extends to the CiaS problem. We find that although the enhancements over a basic Gaussian EDA result in superior performance, the further efficiency enhancements in CMA-ES are not highly impactful. Instead, the most impactful features are how constraint handling is performed, how large the population size is, whether a full covariance matrix is used and whether restart techniques are used. We further show that a previously published version of AMaLGaM that does not require the user to set the the population size parameter is capable of solving the problem and we derive the scalability of the required number of function evaluations to solve the problem up to 99.99 % of the known optimal value for up to 30 circles.


Black-box optimization Evolutionary computation Estimation-of-distribution algorithms Parameter tuning Constraint handling Circles-in-a-square packing 


Compliance with ethical standards

Conflict of interest

Peter A. N. Bosman, Marcus Gallagher declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. Addis B, Locatelli M, Schoen F (2008) Disk packing in a square: a new global optimization approach. INFORMS J Comput 20(4):516–524MathSciNetCrossRefzbMATHGoogle Scholar
  2. Bosman P, Grahl J, Thierens D (2013) Benchmarking parameter-free AMaLGaM on functions with and without noise. Evolut Comput 21(3):445–469CrossRefGoogle Scholar
  3. Bosman PAN, Thierens D (2000) Expanding from discrete to continuous estimation of distribution algorithms: the IDEA. In: Schoenauer M et al (eds) Parallel problem solving from nature—PPSN VI. Springer, Berlin, pp 767–776CrossRefGoogle Scholar
  4. Castillo I, Kampas FJ, Pintér JD (2008) Solving circle packing problems by global optimization: numerical results and industrial applications. Eur J Oper Res 191:786–802MathSciNetCrossRefzbMATHGoogle Scholar
  5. Coello CAC (2002) Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art. Comput Methods Appl Mech Eng 191(11–12):12451287MathSciNetzbMATHGoogle Scholar
  6. Deb K (2000) An efficient constraint handling method for genetic algorithms. Comput Methods Appl Mech Eng 186(2–4):311–338CrossRefzbMATHGoogle Scholar
  7. Gallagher M (2009) Investigating circles in a square packing problems as a realistic benchmark for continuous metaheuristic optimization algorithms. In: MIC 2009: the VIII metaheuristics international conference, Hamburg, GermanyGoogle Scholar
  8. Gallagher M (2012) Beware the parameters: estimation of distribution algorithms applied to circles in a square packing. In: Coello CAC et al (eds) Parallel problem solving from nature—PPSN XII. Springer, Berlin, pp 478–487CrossRefGoogle Scholar
  9. Grosso A, Jamali A, Locatelli M, Schoen F (2010) Solving the problem of packing equal and unequal circles in a circular container. J Glob Optim 47(1):63–81MathSciNetCrossRefzbMATHGoogle Scholar
  10. Hansen N (2006) The CMA evolution strategy: a comparing review. In: Lozano JA, Larrañaga P, Inza I, Bengoetxea E (eds) Towards a new evolutionary computation. Advances in estimation of distribution algorithms. Springer, Berlin, pp 75–102Google Scholar
  11. Hansen N, Auger A, Ros R, Finck S, Pošík P (2010) Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009. In: Branke J et al (eds) Proceedings of the genetic and evolutionary computation conference—GECCO-2010. ACM Press, New York, pp 1689–1696Google Scholar
  12. Larrañaga P, Lozano JA, Bengoetxea E (2001) Estimation of distribution algorithm based on multivariate normal and Gaussian networks. Technical Report KZZA-IK-1-01. University of the Basque Country, Department of Computer Science and Artificial IntelligenceGoogle Scholar
  13. Larrañaga P, Lozan JA (2001) Estimation of distribution algorithms. A new tool for evolutionary computation. Kluwer Academic, LondonGoogle Scholar
  14. Mahfoud S (2000) Niching methods. In: Bäck DFT, Michalewicz Z (eds) Evolutionary computation 2–advanced algorithms and operations, Chap 13. IOP Publishing Ltd, Bristol, pp 87–92Google Scholar
  15. Morgan R, Gallagher M (2014) Fitness landscape analysis of circles in a square packing problems. In: Dick G et al (eds) Simulated evolution and learning—SEAL-2014. Springer, Berlin, pp 455–466Google Scholar
  16. Pelikan M, Sastry K, Cantú-Paz E (2006) Scalable optimization via probabilistic modeling: from algorithms to applications. Springer, BerlinCrossRefzbMATHGoogle Scholar
  17. Szabó PG, Markót MC, Csendes T (2005) Global optimization in geometry—circle packing into the square. In: Audet P, Hansen P, Savard P (eds) Essays and surveys in global optimization. Springer, USGoogle Scholar
  18. Whitley D, Mathias K, Rana S, Dzubera J (1996) Evaluating evolutionary algorithms. Artif Intell 85(1–2):245–276CrossRefGoogle Scholar
  19. Yu E, Suganthan P (2010) Ensemble of niching algorithms. Inf Sci 180(15):2815–2833MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Peter A.N. Bosman Centrum Wiskunde & Informatica (CWI)AmsterdamThe Netherlands
  2. 2.School of Information Technology and Electrical EngineeringThe University of QueenslandBrisbaneAustralia

Personalised recommendations