Comparison of Metaheuristics

Part of the International Series in Operations Research & Management Science book series (ISOR, volume 146)


Metaheuristics are truly diverse in nature—under the overarching theme of performing operations to escape local optima, algorithms as different as ant colony optimization, tabu search, harmony search, and genetic algorithms have emerged. Due to the unique functionality of each type of metaheuristic, comparison of metaheuristics is in many ways more difficult than other algorithmic comparisons. In this chapter, we discuss techniques for meaningful comparison of metaheuristics. We discuss how to create and classify instances in a new testbed and how to make sure other researchers have access to the problems for future metaheuristic comparisons. Further, we discuss the disadvantages of large parameter sets and how to measure complicating parameter interactions in a metaheuristic’s parameter space. Last, we discuss how to compare metaheuristics in terms of both solution quality and runtime.


Parameter Interaction Tabu Search Problem Instance Solution Quality Orienteering Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Ahuja, R., Orlin, J.: Use of representative operation counts in computational testing of algorithms. INFORMS J. Comput. 8(3), 318–330 (1996)CrossRefGoogle Scholar
  2. 2.
    Archetti, C., Feillet, D., Hertz, A., Speranza, M.G.: The capacitated team orienteering and profitable tour problems. J. Oper. Res. Soc. 60(6), 831–842 (2009)CrossRefGoogle Scholar
  3. 3.
    Bailey, D.: Twelve ways to fool the masses when giving performance results on parallel computers. Supercomput. Rev. 4(8), 54–55 (1991)Google Scholar
  4. 4.
    Bull, M., Smith, L., Pottage, L., Freeman, R.: Benchmarking Java against C and Fortran for scientific applications. In: ACM 2001 Java Grande/ISCOPE Conference, pp. 97–105. ACM, New York (2001)Google Scholar
  5. 5.
    Chao, I.M.: Algorithms and solutions to multi-level vehicle routing problems. Ph.D. thesis, University of Maryland, College Park, MD (1993)Google Scholar
  6. 6.
    Chen, S., Golden, B., Wasil, E.: The split delivery vehicle routing problem: applications, algorithms, test problems, and computational results. Networks 49, 318–329 (2007)CrossRefGoogle Scholar
  7. 7.
    Christofides, N., Eilon, S.: An algorithm for the vehicle dispatching problem. Oper. Res. Q. 20(3), 309–318 (1969)CrossRefGoogle Scholar
  8. 8.
    Coello, C.: Evolutionary multi-objective optimization: a historical view of the field. IEEE Comput. Intel. Mag. 1(1), 28–36 (2006)CrossRefGoogle Scholar
  9. 9.
    Coy, S., Golden, B., Runger, G., Wasil, E.: Using experimental design to find effective parameter settings for heuristics. J. Heuristics 7(1), 77–97 (2001)CrossRefGoogle Scholar
  10. 10.
    Deb, K., Agarwal, S.: Understanding interactions among genetic algorithm parameters. In: Foundations of Genetic Algorithms, pp. 265–286. Morgan Kauffman, San Mateo, CA (1998)Google Scholar
  11. 11.
    Dongarra, J.: Performance of various computers using standard linear equations software. Tech. rep., University of Tennessee (2009)Google Scholar
  12. 12.
    Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT, Cambridge (2004)CrossRefGoogle Scholar
  13. 13.
    Fischetti, M., Salazar González, J.J., Toth, P.: A branch-and-cut algorithm for the symmetric generalized traveling salesman problem. Oper. Res. 45(3), 378–394 (1997)CrossRefGoogle Scholar
  14. 14.
    Gamvros, I., Golden, B., Raghavan, S.: The multilevel capacitated minimum spanning tree problem. INFORMS J. Comput. 18(3), 348–365 (2006)CrossRefGoogle Scholar
  15. 15.
    Gendreau, M., Laporte, G., Semet, F.: A tabu search heuristic for the undirected selective travelling salesman problem. Eur. J. Oper. Res. 106(2–3), 539–545 (1998)CrossRefGoogle Scholar
  16. 16.
    Glover, F.: Tabu search: a tutorial. Interfaces 20(4), 74–94 (1990)CrossRefGoogle Scholar
  17. 17.
    Hartmann, S., Kolisch, R.: Experimental evaluation of state-of-the-art heuristics for the resource-constrained project scheduling problem. Eur. J. Oper. Res. 127(2), 394–407 (2000)CrossRefGoogle Scholar
  18. 18.
    Hollasch, S.: Four-space visualization of 4d objects. Ph.D. thesis, Arizona State University, Tempe, Arizona (1991)Google Scholar
  19. 19.
    Jans, R., Degraeve, Z.: Meta-heuristics for dynamic lot sizing: a review and comparison of solution approaches. Eur. J. Oper. Res. 177(3), 1855–1875 (2007)CrossRefGoogle Scholar
  20. 20.
    Li, F., Golden, B., Wasil, E.: Very large-scale vehicle routing: new test problems, algorithms, and results. Comput. Oper. Res. 32(5), 1165–1179 (2005)CrossRefGoogle Scholar
  21. 21.
    Li, F., Golden, B., Wasil, E.: The open vehicle routing problem: algorithms, large-scale test problems, and computational results. Comput. Oper. Res. 34(10), 2918–2930 (2007)CrossRefGoogle Scholar
  22. 22.
    Li, F., Golden, B., Wasil, E.: A record-to-record travel algorithm for solving the heterogeneous fleet vehicle routing problem. Comput. Oper. Res. 34(9), 2734–2742 (2007)CrossRefGoogle Scholar
  23. 23.
    Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs. Springer, New York (1996)Google Scholar
  24. 24.
    Montgomery, D.: Design and Analysis of Experiments. Wiley, New York (2006)Google Scholar
  25. 25.
    Nummela, J., Julstrom, B.: An effective genetic algorithm for the minimum-label spanning tree problem. In: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, pp. 553–557. ACM, New York (2006)Google Scholar
  26. 26.
    Paquete, L., Stützle, T.: Design and analysis of stochastic local search for the multiobjective traveling salesman problem. Comput. Oper. Res. 36(9), 2619–2631 (2009)CrossRefGoogle Scholar
  27. 27.
    Plackett, R., Burman, J.: The design of optimum multifactorial experiments. Biometrika 33, 305–325 (1946)CrossRefGoogle Scholar
  28. 28.
    Reinelt, G.: TSPLIB—a traveling salesman problem library. ORSA J. Comput. 3(4), 376–384 (1991)Google Scholar
  29. 29.
    Sawai, H., Kizu, S.: Parameter-free genetic algorithm inspired by “disparity theory of evolution”. In: Eiben, A., Bäck, T., Schoenauer, M., Schwefel H.P., (eds.) Parallel Problem Solving from Nature – PPSN V, LNCS, vol. 1498, pp. 702–711. Springer Berlin / Heidelberg (1998)CrossRefGoogle Scholar
  30. 30.
    Silberholz, J., Golden, B.: The effective application of a new approach to the generalized orienteering problem. J. Heuristics 16(3), 393–415 (2010)CrossRefGoogle Scholar
  31. 31.
    Wang, Q., Sun, X., Golden, B.L.: Using artificial neural networks to solve generalized orienteering problems. In: Dagli, C., Akay, M., Chen, C., Fernández, B., Ghosh J., (eds.) Intelligent Engineering Systems Through Artificial Neural Networks, vol. 6, pp. 1063–1068. ASME Press, New York (1996)Google Scholar
  32. 32.
    Xiong, Y., Golden, B., Wasil, E.: A one-parameter genetic algorithm for the minimum labeling spanning tree problem. IEEE Trans. Evol. Comput. 9(1), 55–60 (2005)CrossRefGoogle Scholar
  33. 33.
    Xu, J., Kelly, J.: A network flow-based tabu search heuristic for the vehicle routing problem. Transp. Sci. 30(4), 379–393 (1996)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Center for Scientific Computing and Mathematical Modeling, University of MarylandCollege ParkUSA
  2. 2.R.H. Smith School of Business, University of MarylandCollege ParkUSA

Personalised recommendations