Journal of Heuristics

, Volume 24, Issue 3, pp 295–320 | Cite as

A case study of algorithm selection for the traveling thief problem

  • Markus WagnerEmail author
  • Marius Lindauer
  • Mustafa Mısır
  • Samadhi Nallaperuma
  • Frank Hutter


Many real-world problems are composed of several interacting components. In order to facilitate research on such interactions, the Traveling Thief Problem (TTP) was created in 2013 as the combination of two well-understood combinatorial optimization problems. With this article, we contribute in four ways. First, we create a comprehensive dataset that comprises the performance data of 21 TTP algorithms on the full original set of 9720 TTP instances. Second, we define 55 characteristics for all TPP instances that can be used to select the best algorithm on a per-instance basis. Third, we use these algorithms and features to construct the first algorithm portfolios for TTP, clearly outperforming the single best algorithm. Finally, we study which algorithms contribute most to this portfolio.


Combinatorial optimization Instance analysis Algorithm portfolio 


  1. Applegate, D., Cook, W.J., Rohe, A.: Chained Lin–Kernighan for large traveling salesman problems. J. Comput. 15(1), 82–92 (2003)MathSciNetzbMATHGoogle Scholar
  2. Beasley, E.J.: Or-library: distributing test problems by electronic mail. J. Oper. Res. Soc. 41(11), 1069–1072 (1990)CrossRefGoogle Scholar
  3. Bell, J.E., McMullen, P.R.: Ant colony optimization techniques for the vehicle routing problem. Adv. Eng. Inform. 18(1), 41–48 (2004)CrossRefGoogle Scholar
  4. Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability, Frontiers in Artificial Intelligence and Applications, vol. 185. IOS Press, Amsterdam (2009)Google Scholar
  5. Bischl, B., Kerschke, P., Kotthoff, L., Lindauer, M., Malitsky, Y., Frechétte, A., Hoos, H., Hutter, F., Leyton-Brown, K., Tierney, K., Vanschoren, J.: ASlib: a benchmark library for algorithm selection. Artif. Intell. 237, 41–58 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  6. Blank, J., Deb, K., Mostaghim, S.: Solving the Bi-objective Traveling Thief Problem with Multi-objective Evolutionary Algorithms. Springer, Berlin (2017)CrossRefGoogle Scholar
  7. Bonyadi, M.R., Michalewicz, Z., Barone, L.: The travelling thief problem: the first step in the transition from theoretical problems to realistic problems. In: Congress on Evolutionary Computation, pp. 1037–1044. IEEE, (2013)Google Scholar
  8. Bonyadi, M.R., Michalewicz, Z., Przybylek, M.R., Wierzbicki, A.: Socially inspired algorithms for the TTP. In: Genetic and Evolutionary Computation Conference, pp. 421–428. ACM, (2014)Google Scholar
  9. Bonyadi, M.R., Michalewicz, Z., Neumann, F., Wagner, M.: Evolutionary computation for multicomponent problems: opportunities and future directions. CoRR abs/1606.06818. (2016)
  10. Brazdil, P., Giraud-Carrier, C., Soares, C., Vilalta, R.: Metalearning: Applications to Data Mining, 1st edn. Springer, Berlin (2008)zbMATHGoogle Scholar
  11. Breimann, L.: Random forests. Mach. Learn. J. 45, 5–32 (2001)CrossRefGoogle Scholar
  12. Chalkiadakis, G., Elkind, E., Wooldridge, M.: Computational Aspects of Cooperative Game Theory, Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, San Rafael (2011)zbMATHGoogle Scholar
  13. Chand, S., Wagner, M.: Fast heuristics for the multiple traveling thieves problem. In: Genetic and Evolutionary Computation Conference (GECCO), pp. 293–300. ACM, (2016)Google Scholar
  14. Dantzig, G.B., Ramser, J.H.: The truck dispatching problem. Manag. Sci. 6(1), 80–91 (1959)MathSciNetCrossRefzbMATHGoogle Scholar
  15. El Yafrani, M., Ahiod, B.: Population-based versus single-solution heuristics for the travelling thief problem. In: Genetic and Evolutionary Computation Conference (GECCO), pp. 317–324 . ACM, (2016)Google Scholar
  16. Faulkner, H., Polyakovskiy, S., Schultz, T., Wagner, M.: Approximate approaches to the traveling thief problem. In: Genetic and Evolutionary Computation Conference, pp. 385–392. ACM, (2015)Google Scholar
  17. Frechette, A., Kotthoff, L., Rahwan, T., Hoos, H., Leyton-Brown, K., Michalak, T.: Using the shapley value to analyze algorithm portfolios. In: 30th AAAI Conference on Artificial Intelligence (2016)Google Scholar
  18. Hoos, H., Lindauer, M., Schaub, T.: Claspfolio 2: advances in algorithm selection for answer set programming. Theory Pract. Logic Program. 14, 569–585 (2014)CrossRefzbMATHGoogle Scholar
  19. Hoos, H., Kaminski, R., Lindauer, M., Schaub, T.: Aspeed: solver scheduling via answer set programming. Theory Pract. Logic Program. 15, 117–142 (2015)CrossRefzbMATHGoogle Scholar
  20. Huberman, B., Lukose, R., Hogg, T.: An economic approach to hard computational problems. Science 275, 51–54 (1997)CrossRefGoogle Scholar
  21. Hutter, F., Hoos, H., Leyton-Brown, K., Stützle, T.: ParamILS: an automatic algorithm configuration framework. J. Artif. Intell. Res. 36, 267–306 (2009)zbMATHGoogle Scholar
  22. Hutter, F., Hoos, H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello C (ed.) Proceedings of the Fifth International Conference on Learning and Intelligent Optimization (LION’11). Lecture Notes in Computer Science, vol. 6683, pp. 507–523. Springer, (2011)Google Scholar
  23. Hutter, F., Xu, L., Hoos, H., Leyton-Brown, K.: Algorithm runtime prediction: methods and evaluation. Artif. Intell. 206, 79–111 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  24. Kadioglu, S., Malitsky, Y., Sellmann, M., Tierney, K.: ISAC—instance-specific algorithm configuration. In: Coelho H, Studer R, Wooldridge M (eds.) Proceedings of the Nineteenth European Conference on Artificial Intelligence (ECAI’10), pp. 751–756. IOS Press, (2010)Google Scholar
  25. Kadioglu, S., Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm selection and scheduling. In: Lee J (ed.) Proceedings of the Seventeenth International Conference on Principles and Practice of Constraint Programming (CP’11). Lecture Notes in Computer Science, vol. 6876, pp. 454–469. Springer, (2011)Google Scholar
  26. Klamroth, K., Mostaghim, S., Naujoks, B., Poles, S., Purshouse, R., Rudolph, G., Ruzika, S., Sayn, S., Wiecek, M.M., Yao, X.: Multiobjective optimization for interwoven systems. J. Multi Criteria Decis. Anal. 24(1–2), 71–81 (2017)CrossRefGoogle Scholar
  27. Koch, T., Achterberg, T., Andersen, E., Bastert, O., Berthold, T., Bixby, R.E., Danna, E., Gamrath, G., Gleixner, A.M., Heinz, S., Lodi, A., Mittelmann, H., Ralphs, T., Salvagnin, D., Steffy, D.E., Wolter, K.: MIPLIB 2010. Math. Program. Comput. 3(2), 103–163 (2011)MathSciNetCrossRefGoogle Scholar
  28. Kotthoff, L.: Algorithm selection for combinatorial search problems: a survey. In: Bessiere C, De Raedt L, Kotthoff L, Nijssen S, O’Sullivan B, Pedreschi D (eds.) Data Mining and Constraint Programming, pp. 149–190. Springer (2016)Google Scholar
  29. Laporte, G.: The vehicle routing problem: an overview of exact and approximate algorithms. Eur. J. Oper. Res. 59(3), 345–358 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  30. Leyton-Brown, K., Nudelman, E., Shoham, Y.: Learning the empirical hardness of optimization problems: the case of combinatorial auctions. In: Hentenryck PV (ed.) Principles and Practice of Constraint Programming—CP 2002. Lecture Notes in Computer Science, vol. 2470, pp. 556–572. Springer, (2002)Google Scholar
  31. Lindauer, M., Hoos, H., Hutter, F., Schaub, T.: Autofolio: an automatically configured algorithm selector. J. Artif. Intell. 53, 745–778 (2015)Google Scholar
  32. Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm portfolios based on cost-sensitive hierarchical clustering. In: Rossi F (ed.) Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI’13), pp. 608–614. (2013)Google Scholar
  33. Maratea, M., Pulina, L., Ricca, F.: A multi-engine approach to answer-set programming. Theory Pract. Logic Program. 14, 841–868 (2014)MathSciNetCrossRefGoogle Scholar
  34. Martello, S., Pisinger, D., Toth, P.: Dynamic programming and strong bounds for the 0–1 knapsack problem. Manag. Sci. 45(3), 414–424 (1999)CrossRefzbMATHGoogle Scholar
  35. Mei, Y., Li, X., Yao, X.: Improving efficiency of heuristics for the large scale traveling thief problem. In: Simulated Evolution and Learning. LNCS, vol. 8886, pp. 631–643 Springer (2014a)Google Scholar
  36. Mei, Y., Li, X., Yao, X.: On investigation of interdependence between sub-problems of the TTP. Soft Comput. 20(1), 157–172 (2014b)CrossRefGoogle Scholar
  37. Mersmann, O., Bischl, B., Bossek, J., Trautmann, H., Wagner, M., Neumann, F.: Local search and the traveling salesman problem: A feature-based characterization of problem hardness. In: Hamadi Y, Schoenauer M (eds.) Learning and Intelligent Optimization: 6th International Conference (LION 6), pp. 115–129. Springer, (2012)Google Scholar
  38. Mersmann, O., Bischl, B., Trautmann, H., Wagner, M., Bossek, J., Neumann, F.: A novel feature-based approach to characterize algorithm performance for the traveling salesperson problem. Ann. Math. Artif. Intell. 69(2), 151–182 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  39. Michalewicz, Z.: Ubiquity symposium: evolutionary computation and the processes of life: the emperor is naked: evolutionary algorithms for real-world applications. Ubiquity 2012(November), 3:1–3:13 (2012)CrossRefGoogle Scholar
  40. Michalewicz, Z., Fogel, D.B.: How to Solve It—Modern Heuristics: Second, Revised and Extended, 2nd edn. Springer, Berlin (2004)CrossRefzbMATHGoogle Scholar
  41. Mısır, M., Sebag, M.: Algorithm selection as a collaborative filtering problem. Technical report. INRIA-Saclay. (2013)
  42. Nallaperuma, S., Wagner, M., Neumann, F.: Ant colony optimisation and the traveling salesperson problem: Hardness, features and parameter settings. In: Proceedings of the 15th Annual Conference Companion on Genetic and Evolutionary Computation, ACM, New York, NY, USA, GECCO ’13 Companion, pp. 13–14. (2013a)Google Scholar
  43. Nallaperuma, S., Wagner, M., Neumann, F., Bischl, B., Mersmann, O., Trautmann, H.: A feature-based comparison of local search and the christofides algorithm for the travelling salesperson problem. In: Proceedings of the Twelfth Workshop on Foundations of Genetic Algorithms XII, ACM, New York, NY, USA, FOGA XII ’13, pp. 147–160. (2013b)Google Scholar
  44. Nallaperuma, S., Wagner, M., Neumann, F.: Parameter prediction based on features of evolved instances for ant colony optimization and the traveling salesperson problem. In: Parallel Problem Solving from Nature PPSN XIII. LNCS, vol. 8672. pp. 100–109. Springer, (2014)Google Scholar
  45. Nallaperuma, S., Wagner, M., Neumann, F.: Analyzing the effects of instance features and algorithm parameters for max min ant system and the traveling salesperson problem. Front. Robot. AI 2, 18 (2015)CrossRefGoogle Scholar
  46. Polyakovskiy, S., Neumann, F.: Packing while traveling: Mixed integer programming for a class of nonlinear knapsack problems. In: Integration of AI and OR Techniques in Constraint Programming. LNCS, vol. 9075, pp. 330–344. Springer, (2015)Google Scholar
  47. Polyakovskiy, S., Bonyadi, M.R., Wagner, M., Michalewicz, Z., Neumann, F.: A comprehensive benchmark set and heuristics for the traveling thief problem. In: Genetic and Evolutionary Computation Conference, pp. 477–484. ACM, (2014a)Google Scholar
  48. Polyakovskiy, S., Bonyadi, M.R., Wagner, M., Michalewicz, Z., Neumann, F.: TTP Test Data. (2014b)
  49. Reinelt, G.: TSPLIB—a traveling salesman problem library. ORSA J. Comput. 3(4), 376–384 (1991)CrossRefzbMATHGoogle Scholar
  50. Rice, J.: The algorithm selection problem. Adv. Comput. 15, 65–118 (1976)CrossRefGoogle Scholar
  51. Rizzoli, A.E., Montemanni, R., Lucibello, E., Gambardella, L.M.: Ant colony optimization for real-world vehicle routing problems. Swarm Intell. 1(2), 135–151 (2007)CrossRefGoogle Scholar
  52. Smith-Miles, K.: Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Comput. Surv. 41(1), 6 (2008)CrossRefGoogle Scholar
  53. Smith-Miles, K., Baatar, D., Wreford, B., Lewis, R.: Towards objective measures of algorithm performance across instance space. Comput. OR 45, 12–24 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  54. Stützle, T., Hoos, H.H.: MAX–MIN ant system. J. Future Gener. Comput. Syst. 16, 889–914 (2000)CrossRefzbMATHGoogle Scholar
  55. van Rijn, J., Abdulrahman, S., Brazdil, P., Vanschoren, J.: Fast algorithm selection using learning curves. In: Fromont É, Bie TD, van Leeuwen M (eds.) Proceedings of the international symposium on Advances in Intelligent Data Analysis (IDA). Lecture Notes in Computer Science, vol. 9385, pp. 298–309. Springer, (2015)Google Scholar
  56. Vilalta, R., Drissi, Y.: A perspective view and survey of meta-learning. Artif. Intell. Rev. 18(2), 77–95 (2002)CrossRefGoogle Scholar
  57. Wagner, M.: Stealing Items More Efficiently with Ants, A Swarm Intelligence Approach to the Travelling Thief Problem. Springer, Cham (2016)CrossRefGoogle Scholar
  58. Weise, T., Zapf, M., Chiong, R., Nebro, A.J.: Why is optimization difficult? In: Chiong, R. (ed.) Nature-Inspired Algorithms for Optimisation, pp. 1–50. Springer, Heidelberg (2009)Google Scholar
  59. Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: SATzilla: portfolio-based algorithm selection for SAT. J. Artif. Intell. Res. 32, 565–606 (2008)zbMATHGoogle Scholar
  60. Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: Hydra-MIP: automated algorithm configuration and selection for mixed integer programming. In: RCRA workshop on Experimental Evaluation of Algorithms for Solving Problems with Combinatorial Explosion at the International Joint Conference on Artificial Intelligence (IJCAI). (2011)Google Scholar
  61. Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: Evaluating component solver contributions to portfolio-based algorithm selectors. In: Cimatti A, Sebastiani R (eds.) Proceedings of the Fifteenth International Conference on Theory and Applications of Satisfiability Testing (SAT’12). Lecture Notes in Computer Science, vol. 7317, pp. 228–241. Springer, (2012)Google Scholar
  62. Yafrani, M.E., Chand, S., Neumann, A., Wagner, M.: A Case Study of Multi-objectiveness in Multi-component Problems. (2017)

Copyright information

© Her Majesty the Queen in Right of Australia 2017

Authors and Affiliations

  1. 1.Optimisation and Logistics Group, School of Computer ScienceThe University of AdelaideAdelaideAustralia
  2. 2.Institut für InformatikAlbert-Ludwigs-Universität FreiburgFreiburgGermany
  3. 3.Institute of Machine Learning and Computational IntelligenceNanjing University of Aeronautics and AstronauticsNanjingChina
  4. 4.Department of Computer ScienceUniversity of SheffieldSheffieldUK

Personalised recommendations