Parallel Algorithm Configuration

  • Frank Hutter
  • Holger H. Hoos
  • Kevin Leyton-Brown
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7219)


State-of-the-art algorithms for solving hard computational problems often expose many parameters whose settings critically affect empirical performance. Manually exploring the resulting combinatorial space of parameter settings is often tedious and unsatisfactory. Automated approaches for finding good parameter settings are becoming increasingly prominent and have recently lead to substantial improvements in the state of the art for solving a variety of computationally challenging problems. However, running such automated algorithm configuration procedures is typically very costly, involving many thousands of invocations of the algorithm to be configured. Here, we study the extent to which parallel computing can come to the rescue. We compare straightforward parallelization by multiple independent runs with a more sophisticated method of parallelizing the model-based configuration procedure SMAC. Empirical results for configuring the MIP solver CPLEX demonstrate that near-optimal speedups can be obtained with up to 16 parallel workers, and that 64 workers can still accomplish challenging configuration tasks that previously took 2 days in 1–2 hours. Overall, we show that our methods make effective use of large-scale parallel resources and thus substantially expand the practical applicability of algorithm configuration methods.


Parallel Algorithm Desirability Function Time Budget Wall Clock Time Good Parameter Setting 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hutter, F., Babić, D., Hoos, H.H., Hu, A.J.: Boosting verification by automatic tuning of decision procedures. In: Proc. of FMCAD 2007, pp. 27–34. IEEE Computer Society (2007)Google Scholar
  2. 2.
    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Automated Configuration of Mixed Integer Programming Solvers. In: Lodi, A., Milano, M., Toth, P. (eds.) CPAIOR 2010. LNCS, vol. 6140, pp. 186–202. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    Fawcett, C., Helmert, M., Hoos, H.H., Karpas, E., Röger, G., Seipp, J.: FD-Autotune: Domain-specific configuration using fast-downward. In: Proc. of ICAPS-PAL 2011, 8 p. (2011)Google Scholar
  4. 4.
    Hutter, F., Hoos, H.H., Leyton-Brown, K., Stützle, T.: ParamILS: an automatic algorithm configuration framework. Journal of Artificial Intelligence Research 36, 267–306 (2009)zbMATHGoogle Scholar
  5. 5.
    Ansótegui, C., Sellmann, M., Tierney, K.: A Gender-Based Genetic Algorithm for the Automatic Configuration of Algorithms. In: Gent, I.P. (ed.) CP 2009. LNCS, vol. 5732, pp. 142–157. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  6. 6.
    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential Model-Based Optimization for General Algorithm Configuration. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  7. 7.
    Hoos, H.H., Stützle, T.: Local search algorithms for SAT: An empirical evaluation. Journal of Automated Reasoning 24(4), 421–481 (2000)zbMATHCrossRefGoogle Scholar
  8. 8.
    Gomes, C.P., Selman, B., Crato, N., Kautz, H.: Heavy-tailed phenomena in satisfiability and constraint satisfaction problems. Journal of Algorithms 24(1) (2000)Google Scholar
  9. 9.
    Ribeiro, C.C., Rosseti, I., Vallejos, R.: On the Use of Run Time Distributions to Evaluate and Compare Stochastic Local Search Algorithms. In: Stützle, T., Birattari, M., Hoos, H.H. (eds.) SLS 2009. LNCS, vol. 5752, pp. 16–30. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  10. 10.
    Hoos, H.H., Stützle, T.: Towards a characterisation of the behaviour of stochastic local search algorithms for SAT. Artificial Intelligence 112(1-2), 213–232 (1999)zbMATHMathSciNetCrossRefGoogle Scholar
  11. 11.
    Hutter, F.: Automated Configuration of Algorithms for Solving Hard Computational Problems. PhD thesis, University Of British Columbia, Department of Computer Science, Vancouver, Canada (October 2009)Google Scholar
  12. 12.
    López-Ibáñez, M., Dubois-Lacoste, J., Stützle, T., Birattari, M.: The irace package, iterated race for automatic algorithm configuration. Technical Report TR/IRIDIA/2011-004, IRIDIA, Université Libre de Bruxelles, Belgium (2011)Google Scholar
  13. 13.
    Jones, D.R.: A taxonomy of global optimization methods based on response surfaces. Journal of Global Optimization 21(4), 345–383 (2001)zbMATHMathSciNetCrossRefGoogle Scholar
  14. 14.
    Schonlau, M., Welch, W.J., Jones, D.R.: Global versus local search in constrained optimization of computer models. In: Flournoy, N., Rosenberger, W.F., Wong, W.K. (eds.) New Developments and Applications in Experimental Design, vol. 34, pp. 11–25. Institute of Mathematical Statistics, Hayward (1998)Google Scholar
  15. 15.
    Srinivas, N., Krause, A., Kakade, S., Seeger, M.: Gaussian process optimization in the bandit setting: No regret and experimental design. In: Proc. of ICML 2010 (2010)Google Scholar
  16. 16.
    Ginsbourger, D., Le Riche, R., Carraro, L.: Kriging Is Well-Suited to Parallelize Optimization. In: Tenne, Y., Goh, C.-K. (eds.) Computational Intel. in Expensive Opti. Prob. ALO, vol. 2, pp. 131–162. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  17. 17.
    Nell, C., Fawcett, C., Hoos, H.H., Leyton-Brown, K.: HAL: A Framework for the Automated Analysis and Design of High-Performance Algorithms. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 600–615. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  18. 18.
    Balint, A., Diepold, D., Gall, D., Gerber, S., Kapler, G., Retz, R.: EDACC - An Advanced Platform for the Experiment Design, Administration and Analysis of Empirical Algorithms. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 586–599. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  19. 19.
    Atamtürk, A.: On the facets of the mixed–integer knapsack polyhedron. Mathematical Programming 98, 145–175 (2003)zbMATHMathSciNetCrossRefGoogle Scholar
  20. 20.
    Atamtürk, A., Muñoz, J.C.: A study of the lot-sizing polytope. Mathematical Programming 99, 443–465 (2004)zbMATHMathSciNetCrossRefGoogle Scholar
  21. 21.
    Leyton-Brown, K., Pearson, M., Shoham, Y.: Towards a universal test suite for combinatorial auction algorithms. In: Proc. of EC 2000, pp. 66–76 (2000)Google Scholar
  22. 22.
    Gomes, C.P., van Hoeve, W.-J., Sabharwal, A.: Connections in Networks: A Hybrid Approach. In: Trick, M.A. (ed.) CPAIOR 2008. LNCS, vol. 5015, pp. 303–307. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  23. 23.
    Cote, M., Gendron, B., Rousseau, L.: Grammar-based integer programing models for multi-activity shift scheduling. Technical Report CIRRELT-2010-01, Centre interuniversitaire de recherche sur les réseaux d’entreprise, la logistique et le transport (2010)Google Scholar
  24. 24.
    Ahmadizadeh, K., Dilkina, B., Gomes, C.P., Sabharwal, A.: An Empirical Study of Optimization for Maximizing Diffusion in Networks. In: Cohen, D. (ed.) CP 2010. LNCS, vol. 6308, pp. 514–521. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Frank Hutter
    • 1
  • Holger H. Hoos
    • 1
  • Kevin Leyton-Brown
    • 1
  1. 1.University of British ColumbiaVancouverCanada

Personalised recommendations