An Experimental Study of Adaptive Capping in irace

  • Leslie Pérez CáceresEmail author
  • Manuel López-Ibáñez
  • Holger Hoos
  • Thomas Stützle
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10556)


The \({\textsf {irace}} \) package is a widely used for automatic algorithm configuration and implements various iterated racing procedures. The original \({\textsf {irace}} \) was designed for the optimisation of the solution quality reached within a given running time, a situation frequently arising when configuring algorithms such as stochastic local search procedures. However, when applied to configuration scenarios that involve minimising the running time of a given target algorithm, \({\textsf {irace}} \) falls short of reaching the performance of other general-purpose configuration approaches, since it tends to spend too much time evaluating poor configurations. In this article, we improve the efficacy of \({\textsf {irace}} \) in running time minimisation by integrating an adaptive capping mechanism into \({\textsf {irace}} \), inspired by the one used by ParamILS. We demonstrate that the resulting \({\textsf {irace}}_{\textsf {cap}} \) reaches performance levels competitive with those of state-of-the-art algorithm configurators that have been designed to perform well on running time minimisation scenarios. We also investigate the behaviour of \({\textsf {irace}}_{\textsf {cap}} \) in detail and contrast different ways of integrating adaptive capping.



This research was supported through funding through COMEX project (P7/36) within the Interuniversity Attraction Poles Programme of the Belgian Science Policy Office. Thomas Stützle acknowledges support from the Belgian F.R.S.-FNRS, of which he is a Senior Research Associate. Holger Hoos acknowledges support through an NSERC Discovery Grant.


  1. 1.
    Ansótegui, C., Malitsky, Y., Samulowitz, H., Sellmann, M., Tierney, K.: Model-based genetic algorithms for algorithm configuration. In: IJCAI 2015, pp. 733–739. IJCAI/AAAI Press, Menlo Park (2015)Google Scholar
  2. 2.
    Ansótegui, C., Sellmann, M., Tierney, K.: A gender-based genetic algorithm for the automatic configuration of algorithms. In: Gent, I.P. (ed.) CP 2009. LNCS, vol. 5732, pp. 142–157. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-04244-7_14 CrossRefGoogle Scholar
  3. 3.
    Babić, D., Hutter, F.: Spear theorem prover. In: SAT 2008: Proceedings of the SAT 2008 Race (2008)Google Scholar
  4. 4.
    Balaprakash, P., Birattari, M., Stützle, T.: Improvement strategies for the F-Race algorithm: sampling design and iterative refinement. In: Bartz-Beielstein, T., Blesa Aguilera, M.J., Blum, C., Naujoks, B., Roli, A., Rudolph, G., Sampels, M. (eds.) HM 2007. LNCS, vol. 4771, pp. 108–122. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-75514-2_9 CrossRefGoogle Scholar
  5. 5.
    Biere, A.: Yet another local search solver and lingeling and friends entering the SAT competition 2014. In: Belov, A., et al. (ed.) Proceedings of SAT Competition 2014. Science Series of Publications B, vol. B-2014-2, pp. 39–40. University of Helsinki (2014)Google Scholar
  6. 6.
    Birattari, M.: The Problem of Tuning Metaheuristics as Seen from a Machine Learning Perspective. Ph.D. thesis, Université Libre de Bruxelles, Belgium (2004)Google Scholar
  7. 7.
    Birattari, M., Yuan, Z., Balaprakash, P., Stützle, T.: F-race and iterated F-race: an overview. In: Bartz-Beielstein, T., Chiarandini, M., Paquete, L., Preuss, M. (eds.) Experimental Methods for the Analysis of Optimization Algorithms, pp. 311–336. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-02538-9_13 CrossRefGoogle Scholar
  8. 8.
    Hoos, H.H.: Automated algorithm configuration and parameter tuning. In: Hamadi, Y., Monfroy, E., Saubion, F. (eds.) Autonomous Search, pp. 37–71. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-21434-9_3 Google Scholar
  9. 9.
    Hoos, H.H.: Programming by optimization. Commun. ACM 55(2), 70–80 (2012)CrossRefGoogle Scholar
  10. 10.
    Hoos, H.H., Stützle, T.: Stochastic Local Search-Foundations and Applications. Morgan Kaufmann Publishers, San Francisco (2005)zbMATHGoogle Scholar
  11. 11.
    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Automated configuration of mixed integer programming solvers. In: Lodi, A., Milano, M., Toth, P. (eds.) CPAIOR 2010. LNCS, vol. 6140, pp. 186–202. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-13520-0_23 CrossRefGoogle Scholar
  12. 12.
    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-25566-3_40 CrossRefGoogle Scholar
  13. 13.
    Hutter, F., Hoos, H.H., Leyton-Brown, K., Stützle, T.: ParamILS: an automatic algorithm configuration framework. J. Artif. Intell. Res. 36, 267–306 (2009)zbMATHGoogle Scholar
  14. 14.
    Hutter, F., Lindauer, M.T., Balint, A., Bayless, S., Hoos, H.H., Leyton-Brown, K.: The configurable SAT solver challenge (CSSC). Artif. Intell. 243, 1–25 (2017)CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    Hutter, F., López-Ibáñez, M., Fawcett, C., Lindauer, M., Hoos, H.H., Leyton-Brown, K., Stützle, T.: AClib: a benchmark library for algorithm configuration. In: Pardalos, P.M., Resende, M.G.C., Vogiatzis, C., Walteros, J.L. (eds.) LION 2014. LNCS, vol. 8426, pp. 36–40. Springer, Cham (2014). doi: 10.1007/978-3-319-09584-4_4 Google Scholar
  16. 16.
  17. 17.
    López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Stützle, T., Birattari, M.: The irace package: iterated racing for automatic algorithm configuration. Oper. Res. Perspect. 3, 43–58 (2016)CrossRefMathSciNetGoogle Scholar
  18. 18.
    López-Ibáñez, M., Dubois-Lacoste, J., Stützle, T., Birattari, M.: The irace package, iterated race for automatic algorithm configuration. Technical report TR/IRIDIA/2011-004, IRIDIA, Université Libre de Bruxelles, Belgium (2011)Google Scholar
  19. 19.
    Lourenço, H.R., Martin, O., Stützle, T.: Iterated local search: framework and applications. In: Gendreau, M., Potvin, J.Y. (eds.) Handbook of Metaheuristics. International Series in Operations Research & Management Science, vol. 146, pp. 363–397. Springer, Boston (2010). doi: 10.1007/978-1-4419-1665-5_12 CrossRefGoogle Scholar
  20. 20.
    Pérez Cáceres, L., López-Ibáñez, M., Hoos, H.H., Stützle, T.: An experimental study of adaptive capping in irace: Supplementary material (2017).
  21. 21.
    Pérez Cáceres, L., López-Ibáñez, M., Stützle, T.: An analysis of parameters of irace. In: Blum, C., Ochoa, G. (eds.) EvoCOP 2014. LNCS, vol. 8600, pp. 37–48. Springer, Heidelberg (2014). doi: 10.1007/978-3-662-44320-0_4 Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Leslie Pérez Cáceres
    • 1
    Email author
  • Manuel López-Ibáñez
    • 2
  • Holger Hoos
    • 3
  • Thomas Stützle
    • 1
  1. 1.IRIDIAUniversité Libre de BruxellesBrusselsBelgium
  2. 2.Alliance Manchester Business SchoolUniversity of ManchesterManchesterUK
  3. 3.Computer Science DepartmentUniversity of British ColumbiaVancouverCanada

Personalised recommendations