• Thomas Bartz-BeielsteinEmail author
  • Marco ChiarandiniEmail author
  • Luís PaqueteEmail author
  • Mike PreussEmail author


Theory and experiments are complementary ways to analyze optimization algorithms. Experiments can also live a life of their own and produce learning without need to follow or test a theory. Yet, in order to make conclusions based on experiments trustworthy, reliable, and objective a systematic methodology is needed. In the natural sciences, this methodology relies on the mathematical framework of statistics. This book collects the results of recent research that focused on the application of statistical principles to the specific task of analyzing optimization algorithms.


Multiobjective Optimization Solution Quality Algorithm Performance Vehicle Route Problem Biobjective Optimization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Ackermann R (1989) The new experimentalism. British Journal for the Philosophy of Science 40:185–190CrossRefGoogle Scholar
  2. Bartz-Beielstein T (2006) Experimental Research in Evolutionary Computation— The New Experimentalism. Natural Computing Series, Springer, Berlin, Heidelberg, New YorkGoogle Scholar
  3. Bartz-Beielstein T (2008) How experimental algorithmics can benefit from Mayo’s extensions to Neyman-Pearson theory of testing.Synthese 163(3):385–396, DOI 10.1007/s11229-007-9297-zzbMATHCrossRefMathSciNetGoogle Scholar
  4. Birattari M (2005) Tuning Metaheuristics. Springer, Berlin, Heidelberg, New YorkzbMATHGoogle Scholar
  5. Burke EK, Gendreau M, Gendron B, Rousseau LM (eds) (2008) Proceedings of the 7th International Conference on the Practice and Theory of Automated Timetabling, PATAT 2008, Université de Montréal, Montreal, CA, available online: ProceedingsGoogle Scholar
  6. Chalmers AF (1999) What Is This Thing Called Science. University of Queensland Press, St. Lucia, AustraliaGoogle Scholar
  7. Dorigo M, Stützle T (2004) Ant Colony Optimization. MIT Press, Cambridge, MA, USAzbMATHCrossRefGoogle Scholar
  8. Golden B, Raghavan S, Wasil E (eds) (2008) The Vehicle Routing Problem: Latest Advances and New Challenges, Operations Research/Computer Science Interfaces Series, vol 43. SpringerzbMATHGoogle Scholar
  9. Hacking I (2001) An Introduction to Probability and Inductive Logic. Cambridge University Press, Cambridge, UKzbMATHGoogle Scholar
  10. Hooker J (1996) Testing heuristics: We have it all wrong. Journal of Heuristics 1(1):33–42CrossRefGoogle Scholar
  11. Johnson DS (2002) A theoretician’s guide to the experimental analysis of algorithms. In: Goldwasser MH, Johnson DS, McGeoch CC (eds) Data Structures, Near Neighbor Searches, and Methodology: Fifth and Sixth DIMACS Implementation Challenges, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol 59, American Mathematical Society, pp 215–250Google Scholar
  12. Kirkpatrick S, Gelatt D, Vecchi MP (1983) Optimization by simulated annealing. Science 220(4598):671–680CrossRefMathSciNetGoogle Scholar
  13. Mayo DG (1983) An objective theory of statistical testing. Synthese 57:297–340CrossRefMathSciNetGoogle Scholar
  14. Mayo DG (1996) Error and the Growth of Experimental Knowledge. The University of Chicago Press, Chicago ILGoogle Scholar
  15. McGeoch CC (1996) Toward an experimental method for algorithm simulation. INFORMS Journal on Computing 8(1):1–15, (this journal issue contains also com1 mentaries by Pierre L’Ecuyer, James B. Orlin and Douglas R. Shier, and a rejoinder by C. McGeoch)zbMATHCrossRefMathSciNetGoogle Scholar
  16. Motwani R, Raghavan P (1995) Randomized Algorithms. Cambridge University Press, Cambridge, UKzbMATHGoogle Scholar
  17. Schwefel HP (1995) Evolution and Optimum Seeking. Sixth-Generation Computer Technology, Wiley, New York NYGoogle Scholar
  18. Van Hentenryck P, Michel L (2005) Constraint-Based Local Search. The MIT Press, Cambridge, USAGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  1. 1.Institute of Computer ScienceCologne University of Applied SciencesGummersbachGermany
  2. 2.Department of Mathematics and Computer ScienceUniversity of Southern DenmarkOdenseDenmark
  3. 3.CISUC, Department of Informatics EngineeringUniversity of CoimbraCoimbraPortugal
  4. 4.Algorithm Engineering, TU DortmundDortmundGermany

Personalised recommendations