Skip to main content
SpringerLink
Log in

F-Race and Iterated F-Race: An Overview

  • Chapter
  • First Online:
Experimental Methods for the Analysis of Optimization Algorithms

Abstract

Algorithms for solving hard optimization problems typically have several parameters that need to be set appropriately such that some aspect of performance is optimized. In this chapter, we review F-Race, a racing algorithm for the task of automatic algorithm configuration. F-Race is based on a statistical approach for selecting the best configuration out of a set of candidate configurations under stochastic evaluations. We review the ideas underlying this technique and discuss an extension of the initial F-Race algorithm, which leads to a family of algorithms that we call iterated F-Race. Experimental results comparing one specific implementation of iterated F-Race to the original F-Race algorithm confirm the potential of this family of algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Adenso-Diaz B, Laguna M (2006) Fine-tuning of algorithms using fractional experimental designs and local search. Operations Research 54(1):99–114

    Article  MATH  Google Scholar 

  • Balaprakash P, Birattari M, Stützle T (2007) Improvement strategies for the FRace algorithm: Sampling design and iterative refinement. In: Bartz-Beielstein T, et al. (eds) Hybrid Metaheuristics, 4th InternationalWorkshop,HM2007, Lecture Notes in Computer Science, vol 4771, Springer, Berlin, Germany, pp. 108–122

    Google Scholar 

  • Balaprakash P, Birattari M, Stützle T, Dorigo M (2009a) Adaptive sample size and importance sampling in estimation-based local search for the probabilistic traveling salesman problem. European Journal of Operational Research 199(1):98–110

    Article  MATH  MathSciNet  Google Scholar 

  • Balaprakash P, Birattari M, Stützle T, Yuan Z, Dorigo M (2009b) Ant colony optimization and estimation-based local search for the probabilistic traveling salesman problem. Swarm Intelligence 3(3):223–242

    Article  Google Scholar 

  • Bartz-Beielstein T (2006) Experimental Research in Evolutionary Computation. Springer, Berlin, Germany

    MATH  Google Scholar 

  • Becker S (2004) Racing-Verfahren für Tourenplanungsprobleme. Diplomarbeit, Technische Universität Darmstadt, Darmstadt, Germany

    Google Scholar 

  • Becker S, Gottlieb J, Stützle T (2005) Applications of racing algorithms: An industrial perspective. In: Talbi EG, et al. (eds) Artificial Evolution: 7th International Conference, Evolution Artificielle, EA 2005, Springer Verlag, Berlin, Germany, Lille, France, Lecture Notes in Computer Science, vol 3871, pp. 271–283

    Google Scholar 

  • den Besten ML (2004) Simple metaheuristics for scheduling. an empirical investigation into the application of iterated local search to deterministic scheduling problems with tardiness penalities. PhD thesis, FG Intellektik, FB Informatik, TU Darmstadt

    Google Scholar 

  • Billingsley P (1986) Probability and Measure, 2nd edn. Wiley, New York, NY, USA

    MATH  Google Scholar 

  • Bin Hussin MS, Stützle T, Birattari M (2007) A study of stochastic local search algorithms for the quadratic assignment problems. In: Ridge E, et al. (eds) Proceedings of SLS-DS 2007, Doctoral Symposium on Engineering Stochastic Local Search Algorithms, Brussels, Belgium, pp. 11–15

    Google Scholar 

  • Birattari M (2004a) On the estimation of the expected performance of a metaheuristic on a class of instances. How many instances, how many runs? Tech. Rep. TR/IRIDIA/2004-001, IRIDIA, Université Libre de Bruxelles, Brussels, Belgium

    Google Scholar 

  • Birattari M (2004b) The problem of tuning metaheuristics as seen from a machine learning perspective. PhD thesis, Université Libre de Bruxelles, Brussels, Belgium

    Google Scholar 

  • Birattari M(2009) Tuning Metaheuristics: A Machine Learning Perspective, Studies in Computational Intelligence, vol 197. Springer, Berlin, Germany

    Google Scholar 

  • Birattari M, Stützle T, Paquete L, Varrentrapp K (2002) A racing algorithm for configuring metaheuristics. In: Langdon WB, et al. (eds) GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, Morgan Kaufmann Publishers, San Francisco, CA, pp. 11–18

    Google Scholar 

  • Birattari M, Balaprakash P, Dorigo M (2007) The ACO/F-Race algorithm for combinatorial optimization under uncertainty. In: Doerner KF, et al. (eds) Metaheuristics - Progress in Complex Systems Optimization, Operations Research/Computer Science Interfaces Series, Springer, Berlin, Germany, pp. 189–203

    Google Scholar 

  • Blum C, Socha K (2005) Training feed-forward neural networks with ant colony optimization: An application to pattern classification. In: Nedjah N, et al. (eds) Proceedings of Fifth International Conference on Hybrid Intelligent Systems (HIS’05), IEEE Computer Society, Los Alamitos, CA, USA, pp. 233–238

    Google Scholar 

  • Burnham K, Anderson D (2002) Model selection and multimodel inference: a practical information-theoretic approach. Springer

    MATH  Google Scholar 

  • Caelen O, Bontempi G (2005) How to allocate a restricted budget of leave-one-out assessments for effective model selection in machine learning: a comparison of state-of-the-art techniques. In: Verbeeck K, et al. (eds) Proceedings of the 17th Belgian-Dutch Conference on Artificial Intelligence (BNAIC’05), Brussels, Belgium, pp. 51–58

    Google Scholar 

  • Chiarandini M (2005) Stochastic local search methods for highly constrained combinatorial optimisation problems. PhD thesis, Technische Universität Darmstadt, Darmstadt, Germany

    Google Scholar 

  • Chiarandini M, Stützle T (2002) Experimental evaluation of course timetabling algorithms. Tech. Rep. AIDA-02-05, FG Intellektik, FB Informatik, Technische Universität Darmstadt, Darmstadt, Germany

    Google Scholar 

  • Chiarandini M, Stützle T (2007) Stochastic local search algorithms for graph set t-colouring and frequency assignment. Constraints 12(3):371–403

    Article  MATH  MathSciNet  Google Scholar 

  • Chiarandini M, Birattari M, Socha K, Rossi-Doria O (2006) An effective hybrid algorithm for university course timetabling. Journal of Scheduling 9(5):403–432

    Article  MATH  MathSciNet  Google Scholar 

  • Conover WJ (1999) Practical Nonparametric Statistics, 3rd edn. Wiley, New York, NY, USA

    Google Scholar 

  • Dean A, Voss D (1999) Design and Analysis of Experiments. Springer, New York, NY, USA

    Book  MATH  Google Scholar 

  • Di Gaspero L, Roli A (2008) Stochastic local search for large-scale instances of the haplotype inference problem by pure parsimony. Journal of Algorithms 63(1–3):55–69

    Article  MATH  Google Scholar 

  • Di Gaspero L, di Tollo G, Roli A, Schaerf A (2007) Hybrid local search for constrained financial portfolio selection problems. In: Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems, Lecture Notes in Computer Science, vol 4510, Springer Verlag, Berlin, Germany, pp. 44–58

    Google Scholar 

  • Dorigo M, Stützle T (2004) Ant colony optimization. MIT Press, Cambridge, MA

    Book  MATH  Google Scholar 

  • Hoos HH, Stützle T (2004) Stochastic Local Search. Foundations and Applications. Morgan Kaufmann, San Francisco, CA, USA

    Google Scholar 

  • Hutter F, Hoos HH, Stützle T (2007) Automatic algorithm configuration based on local search. In: Holte RC, et al. (eds) Proceedings of the 22nd Conference on Artificial Intelligence (AAAI), AAAI Press / The MIT Press, Menlo Park, CA, USA, pp. 1152–1157

    Google Scholar 

  • Johnson DS, McGeoch LA, Rego C, Glover F (2001) 8th DIMACS implementation challenge. http://www.research.att.com/~dsj/chtsp/ (webpage last visited in April 2009)

  • Lenne R, Solnon C, Stützle T, Tannier E, Birattari M (2007) Effective stochastic local search algorithms for the genomic median problem. In: Ridge E, et al. (eds) Proceedings of SLS-DS 2007, Doctoral Symposium on Engineering Stochastic Local Search Algorithms, Brussels, Belgium, pp. 1–5

    Google Scholar 

  • Manfrin M (2003) Metaeuristiche per la costruzione degli orari dei corsi universitari. Tesi di Laurea, Università degli Studi di Firenze, Firenze, Italy, in Italian

    Google Scholar 

  • Maron O (1994) Hoeffding races: Model selection for MRI classification. Master’s thesis, The Massachusetts Institute of Technology, Cambridge, MA, USA

    Google Scholar 

  • Maron O, Moore AW (1994) Hoeffding races: Accelerating model selection search for classification and function approximation. In: Cowan JD, et al. (eds) Advances in Neural Information Processing Systems, Morgan Kaufmann, San Francisco, CA, USA, vol 6, pp. 59–66

    Google Scholar 

  • Maron O, Moore AW (1997) The racing algorithm: Model selection for lazy learners. Artificial Intelligence Review 11(1–5):193–225

    Article  Google Scholar 

  • Montgomery DC (2000) Design and Analysis of Experiments, 5th edn. Wiley, New York, NY, USA

    Google Scholar 

  • Nouyan S (2008) Teamwork in a swarm of robots – an experiment in search and retrieval. PhD thesis, Université Libre de Bruxelles, Brussels, Belgium

    Google Scholar 

  • Nouyan S, Campo A, Dorigo M (2008) Path formation in a robot swarm. Swarm Intelligence 2(1):1–23

    Article  Google Scholar 

  • Papoulis A (1991) Probability, Random Variables, and Stochastic Processes, 3rd edn. McGraw-Hill, New York, NY, USA

    Google Scholar 

  • Pellegrini P (2005) Application of two nearest neighbor approaches to a rich vehicle routing problem. Tech. Rep. TR/IRIDIA/2005-15, IRIDIA, Université Libre de Bruxelles, Belgium

    Google Scholar 

  • Philemotte C, Bersini H (2008) The gestalt heuristic: learning the right level of abstraction to better search the optima. Tech. Rep. TR/IRIDIA/2008-021, IRIDIA, Université Libre de Bruxelles, Belgium

    Google Scholar 

  • Risler M, Chiarandini M, Paquete L, Schiavinotto T, Stützle T (2004) An algorithm for the car sequencing problem of the ROADEF 2005 challenge. Tech. Rep. AIDA–04–06, FG Intellektik, TU Darmstadt, Darmstadt, Germany

    Google Scholar 

  • Rossi-Doria O, Sampels M, Birattari M, Chiarandini M, Dorigo M, Gambardella LM, Knowles J, Manfrin M, Mastrolilli M, Paechter B, Paquete L, Stützle T (2003) A comparison of the performance of different metaheuristics on the timetabling problem. In: Burke E, et al. (eds) Practice and Theory of Automated Timetabling IV, Springer Verlag, Berlin, Germany, Lecture Notes in Computer Science, vol 2740, pp. 329–351

    Google Scholar 

  • Schiavinotto T, Stützle T (2004) The linear ordering problem: Instances, search space analysis and algorithms. Journal of Mathematical Modelling and Algorithms 3(4):367–402

    Article  MATH  MathSciNet  Google Scholar 

  • Sheskin D (2000) Handbook of Parametric and Nonparametric Statistical Procedures, 2nd edn. Chapman & Hall/CRC, Boca Raton, FL, USA

    MATH  Google Scholar 

  • Siegel S, Castellan NJ Jr (1988) Nonparametric Statistics for the Behavioral Sciences, 2nd edn. McGraw-Hill, New York, NY, USA

    Google Scholar 

  • Socha K, Blum C (2007) An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training. Neural Computing and Applications 16(3):235–247

    Article  Google Scholar 

  • Stützle T, Hoos HH (2000) MAX–MIN ant system. Future Generation Computer Systems 16(8):889–914

    Article  Google Scholar 

  • Yuan B, Gallagher M (2004) Statistical racing techniques for improved empirical evaluation of evolutionary algorithms. In: Yao X, et al. (eds) Parallel Problem Solving from Nature - PPSN VIII, Lecture Notes in Computer Science, vol 3242, Springer Verlag, Berlin, Germany, pp. 172–181

    Chapter  Google Scholar 

  • Yuan B, Gallagher M (2005) A hybrid approach to parameter tuning in genetic algorithms. In: Proceedings of the IEEE Congress in Evolutionary Computation (CEC’05), IEEE Press, Piscataway, NJ, vol 2, pp. 1096–1103

    Google Scholar 

  • Yuan B, Gallagher M (2007) Combining Meta-EAs and racing for difficult EA parameter tuning tasks. In: Parameter Setting in Evolutionary Algorithms, Studies in Computational Intelligence, vol 54, Springer Verlag, Berlin, Germany, pp. 121–142

    Google Scholar 

  • Yuan Z, Fügenschuh A, Homfeld H, Balaprakash P, Stützle T, Schoch M (2008) Iterated greedy algorithms for a real-world cyclic train scheduling problem. In: Blesa MJ, et al. (eds) Hybrid Metaheuristics, 5th International Workshop, HM 2008, Springer Verlag, Berlin, Germany, Lecture Notes in Computer Science, vol 5296, pp. 102–116

    Google Scholar 

  • Zlochin M, Birattari M, Meuleau N, Dorigo M (2004) Model-based search for combinatorial optimization: A critical survey. Annals of Operations Research 131(1–4):373–395

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work has been supported by META-X, an ARC project funded by the French Community of Belgium. The authors acknowledge support from the fund for scientific research FRS-FNRS of the French Community of Belgium, of which they are research associates (M.B. and T.S.), or aspirant (Z.Y.), respectively.

Author information

Authors and Affiliations

  1. IRIDIA, CoDE, Université Libre de Bruxelles, Brussels, Belgium

    Mauro Birattari, Zhi Yuan, Prasanna Balaprakash & Thomas Stützle

Authors
  1. Mauro Birattari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhi Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Prasanna Balaprakash
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Stützle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Mauro Birattari , Zhi Yuan , Prasanna Balaprakash or Thomas Stützle .

Editor information

Editors and Affiliations

  1. Institute of Computer Science, Faculty of Computer Science, Cologne University of Applied Sciences, Steinmüllerallee 1, Gummersbach, 51643, Germany

    Thomas Bartz-Beielstein

  2. , Department of Mathematics, University of Southern Denmark, Campusvej 55, Odense, 5230, Denmark

    Marco Chiarandini

  3. CISUC, Department of Informatics Engineering, University of Coimbra, Coimbra, 3030-290, Portugal

    Luís Paquete

  4. Algorithm Engineering, TU Dortmund, Otto-Hahn-Str. 14, Dortmund, 44227, Germany

    Mike Preuss

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Birattari, M., Yuan, Z., Balaprakash, P., Stützle, T. (2010). F-Race and Iterated F-Race: An Overview. In: Bartz-Beielstein, T., Chiarandini, M., Paquete, L., Preuss, M. (eds) Experimental Methods for the Analysis of Optimization Algorithms. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02538-9_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-02538-9_13

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-02537-2

  • Online ISBN: 978-3-642-02538-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation