Abstract
Finding appropriate values for the parameters of an algorithm is a challenging, important, and time consuming task. While typically parameters are tuned by hand, recent studies have shown that automatic tuning procedures can effectively handle this task and often find better parameter settings. F-Race has been proposed specifically for this purpose and it has proven to be very effective in a number of cases. F-Race is a racing algorithm that starts by considering a number of candidate parameter settings and eliminates inferior ones as soon as enough statistical evidence arises against them. In this paper, we propose two modifications to the usual way of applying F-Race that on the one hand, make it suitable for tuning tasks with a very large number of initial candidate parameter settings and, on the other hand, allow a significant reduction of the number of function evaluations without any major loss in solution quality. We evaluate the proposed modifications on a number of stochastic local search algorithms and we show their effectiveness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Adenso-Diaz, B., Laguna, M.: Fine-tuning of algorithms using fractional experimental designs and local search. Operations Research 54(1), 99–114 (2006)
Barr, R., Golden, B., Kelly, J., Rescende, M., Stewart, W.: Designing and reporting on computational experiments with heuristic methods. Journal of Heuristics 1(1), 9–32 (1995)
Birattari, M., Stützle, T., Paquete, L., Varrentrapp, K.: A racing algorithm for configuring metaheuristics. In: Langdon, W.B. (ed.) Proceedings of the Genetic and Evolutionary Computation Conference, pp. 11–18. Morgan Kaufmann, San Francisco, CA, USA (2002)
Birattari, M.: The Problem of Tuning Metaheuristics as Seen from a Machine Learning Perspective. PhD thesis, Université Libre de Bruxelles, Brussels, Belgium (2004)
Becker, S., Gottlieb, J., Stützle, T.: Applications of racing algorithms: An industrial perspective. In: Talbi, E.-G., Liardet, P., Collet, P., Lutton, E., Schoenauer, M. (eds.) EA 2005. LNCS, vol. 3871, pp. 271–283. Springer, Heidelberg (2006)
Chiarandini, M., Birattari, M., Socha, K., Rossi-Doria, O.: An effective hybrid algorithm for university course timetabling. Journal of Scheduling 9(5), 403–432 (2006)
Pellegrini, P., Birattari, M.: The relevance of tuning the parameters of metaheuristics. A case study: The vehicle routing problem with stochastic demand. Technical Report TR/IRIDIA/2006-008, IRIDIA, Université Libre de Bruxelles, Brussels, Belgium (2006)
Zlochin, M., Birattari, M., Meuleau, N., Dorigo, M.: Model-based search for combinatorial optimization: A critical survey. Annals of Operations Research 131, 373–395 (2004)
Maron, O., Moore, A.: Hoeffding races: Accelerating model selection search for classification and function approximation. In: Cowan, J.D., Tesauro, G., Alspector, J. (eds.) NIPS, vol. 6, pp. 59–66. Morgan Kaufmann, San Francisco, CA, USA (1994)
Moore, A., Lee, M.: Efficient algorithms for minimizing cross validation error. In: Proceedings of the Eleventh International Conference on Machine Learning, pp. 190–198. Morgan Kaufmann, San Francisco, CA, USA (1994)
Conover, W.J.: Practical Nonparametric Statistics, 3rd edn. John Wiley & Sons, New York,USA (1999)
Hoos, H., Stützle, T.: Stochastic Local Search: Foundations and Applications. Morgan Kaufmann, San Francisco, CA, USA (2005)
Stützle, T., Hoos, H.: \(\cal MAX\)–\(\cal MIN\) Ant System. Future Generation Computer System 16(8), 889–914 (2000)
Birattari, M., Balaprakash, P., Stützle, T., Dorigo, M.: Estimation-based local search for stochastic combinatorial optimization. Technical Report TR/IRIDIA/2007-003, IRIDIA, Université Libre de Bruxelles, Brussels, Belgium (2007)
Birattari, M.: The race package for R. Racing methods for the selection of the best. Technical Report TR/IRIDIA/2003-37, IRIDIA, Université Libre de Bruxelles, Brussels, Belgium (2003), Package available at: http://cran.r-project.org/src/contrib/Descriptions/race.html
Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, Cambridge, MA (2004)
Johnson, D.S., McGeoch, L.A., Rego, C., Glover, F.: 8th DIMACS implementation challenge (2001)
Kohavi, R., John, G.: Automatic parameter selection by minimizing estimated error. In: Prieditis, A., Russell, S. (eds.) Proceedings of the Twelfth International Conference on Machine Learning, pp. 304–312 (1995)
Boyan, J., Moore, A.: Using prediction to improve combinatorial optimization search. In: Sixth International Workshop on Artificial Intelligence and Statistics (1997)
Audet, C., Orban, D.: Finding optimal algorithmic parameters using the mesh adaptive direct search algorithm. SIAM Journal on Optimization 17(3), 642–664 (2006)
Beielstein, T., Parsopoulos, K., Vrahatis, M.: Tuning PSO parameters through sensitivity analysis. Technical report, Collaborative Research Center 531 Computational Intelligence CI–124/02 (2002)
Hutter, F., Hoos, H., Stützle, T.: Automatic algorithm configuration based on local search. In: AAAI-2007, AAAI Press, Menlo Park, CA, USA (2007)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Balaprakash, P., Birattari, M., Stützle, T. (2007). Improvement Strategies for the F-Race Algorithm: Sampling Design and Iterative Refinement. In: Bartz-Beielstein, T., et al. Hybrid Metaheuristics. HM 2007. Lecture Notes in Computer Science, vol 4771. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75514-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-540-75514-2_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-75513-5
Online ISBN: 978-3-540-75514-2
eBook Packages: Computer ScienceComputer Science (R0)