Algorithm Configuration Landscapes:

More Benign Than Expected?
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11102)


Automated algorithm configuration procedures make use of powerful meta-heuristics to determine parameter settings that often substantially improve the performance of highly heuristic, state-of-the-art algorithms for prominent \(\mathcal {NP}\)-hard problems, such as the TSP, SAT and mixed integer programming (MIP). These meta-heuristics were originally designed for combinatorial optimization problems with vast and challenging search landscapes. Their use in automated algorithm configuration implies that algorithm configuration landscapes are assumed to be similarly complex; however, to the best of our knowledge no work has been done to support or reject this hypothesis. We address this gap by investigating the response of varying individual numerical parameters while fixing the remaining parameters at optimized values. We present evidence that most parameters exhibit uni-modal and often even convex responses, indicating that algorithm configuration landscapes are likely much more benign than previously believed.



YP was supported by an NSERC Vanier Scholarship. HH acknowledges funding through an NSERC Discovery Grant, CFI JLEF funding and startup funding from Universiteit Leiden.


  1. 1.
    IBM Corp: IBM ILOG CPLEX Optimizer (2018). Accessed 30 Mar 2018
  2. 2.
    Ansótegui, C., Sellmann, M., Tierney, K.: A gender-based genetic algorithm for the automatic configuration of algorithms. In: Gent, I.P. (ed.) CP 2009. LNCS, vol. 5732, pp. 142–157. Springer, Heidelberg (2009). Scholar
  3. 3.
    Biere, A.: CaDiCaL, Lingeling, Plingeling, Treengeling and YalSAT entering the SAT Competition 2017. In: Proceedings of SAT Competition 2017: Solver and Benchmark Descriptions, pp. 14–15 (2017)Google Scholar
  4. 4.
    Dubois-Lacoste, J., Hoos, H., Stützle, T.: On the empirical scaling behaviour of state-of-the-art local search algorithms for the Euclidean TSP. In: Proceedings of GECCO, pp. 377–384 (2015)Google Scholar
  5. 5.
    Falkner, S., Lindauer, M., Hutter, F.: SpySMAC: automated configuration and performance analysis of SAT solvers. In: Heule, M., Weaver, S. (eds.) SAT 2015. LNCS, vol. 9340, pp. 215–222. Springer, Cham (2015). Scholar
  6. 6.
    Fawcett, C., Hoos, H.: Analysing differences between algorithm configurations through ablation. JOH 22(4), 431–458 (2016)Google Scholar
  7. 7.
    Helsgaun, K.: An effective implementation of the Lin-Kernighan traveling salesman heuristic. EJOR 126, 106–130 (2000)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Hoos, H., Stützle, T.: Stochastic Local Search: Foundations & Applications. Morgan Kaufmann Publishers Inc., San Francisco (2005)zbMATHGoogle Scholar
  9. 9.
    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011). Scholar
  10. 10.
    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Identifying key algorithm parameters and instance features using forward selection. In: Nicosia, G., Pardalos, P. (eds.) LION 2013. LNCS, vol. 7997, pp. 364–381. Springer, Heidelberg (2013). Scholar
  11. 11.
    Hutter, F., Hoos, H., Leyton-Brown, K.: An efficient approach for assessing hyperparameter importance. In: Proceedings of ICML, pp. 754–762 (2014)Google Scholar
  12. 12.
    Hutter, F., Hoos, H., Leyton-Brown, K., Stützle, T.: ParamILS: an automatic algorithm configuration framework. JAIR 36, 267–306 (2009)CrossRefGoogle Scholar
  13. 13.
    Hutter, F., et al.: AClib: a benchmark library for algorithm configuration. In: Proceedings of LION, pp. 36–40 (2014)Google Scholar
  14. 14.
    López-Ibáñez, M., Dubois-Lacoste, J., Cáceres, L., Stützle, T., Birattari, M.: The irace package: iterated racing for automatic algorithm configuration. ORP 3, 43–58 (2016)MathSciNetGoogle Scholar
  15. 15.
    Mu, Z., Hoos, H.: Empirical scaling analyser: an automated system for empirical analysis of performance scaling. In: Proceedings of GECCO, pp. 771–772 (2015)Google Scholar
  16. 16.
    Mu, Z., Hoos, H.H., Stützle, T.: The impact of automated algorithm configuration on the scaling behaviour of state-of-the-art inexact TSP solvers. In: Festa, P., Sellmann, M., Vanschoren, J. (eds.) LION 2016. LNCS, vol. 10079, pp. 157–172. Springer, Cham (2016). Scholar
  17. 17.
    Nagata, Y., Kobayashi, S.: A powerful genetic algorithm using edge assembly crossover for the traveling salesman problem. INFORMS JOC 25(2), 346–363 (2013)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Soos, M.: CryptoMiniSat v4. In: Proceedings of SAT Competition 2014: Solver and Benchmark Descriptions, p. 23 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Computer ScienceThe University of British ColumbiaVancouverCanada
  2. 2.LIACSUniversiteit LeidenLeidenThe Netherlands

Personalised recommendations