A case study of algorithm selection for the traveling thief problem

Abstract

Many real-world problems are composed of several interacting components. In order to facilitate research on such interactions, the Traveling Thief Problem (TTP) was created in 2013 as the combination of two well-understood combinatorial optimization problems. With this article, we contribute in four ways. First, we create a comprehensive dataset that comprises the performance data of 21 TTP algorithms on the full original set of 9720 TTP instances. Second, we define 55 characteristics for all TPP instances that can be used to select the best algorithm on a per-instance basis. Third, we use these algorithms and features to construct the first algorithm portfolios for TTP, clearly outperforming the single best algorithm. Finally, we study which algorithms contribute most to this portfolio.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Notes

  1. 1.

    http://challenge.icon-fet.eu/.

  2. 2.

    As available at the TTP project page: http://cs.adelaide.edu.au/~optlog/research/ttp.php.

  3. 3.

    For a more detailed description, we refer the interested reader to Polyakovskiy et al. (2014b, 2014a).

  4. 4.

    https://cran.r-project.org/web/packages/tspmeta/.

  5. 5.

    http://www.ml4aad.org/flexfolio/.

  6. 6.

    See TTP-2016 at http://www.aslib.net.

  7. 7.

    We use the worst possible performance as the performance of an empty portfolio: \(\text {Oracle}(\{\},\mathcal {I}) = -1\).

References

  1. Applegate, D., Cook, W.J., Rohe, A.: Chained Lin–Kernighan for large traveling salesman problems. J. Comput. 15(1), 82–92 (2003)

    MathSciNet  MATH  Google Scholar 

  2. Beasley, E.J.: Or-library: distributing test problems by electronic mail. J. Oper. Res. Soc. 41(11), 1069–1072 (1990)

    Article  Google Scholar 

  3. Bell, J.E., McMullen, P.R.: Ant colony optimization techniques for the vehicle routing problem. Adv. Eng. Inform. 18(1), 41–48 (2004)

    Article  Google Scholar 

  4. Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability, Frontiers in Artificial Intelligence and Applications, vol. 185. IOS Press, Amsterdam (2009)

    Google Scholar 

  5. Bischl, B., Kerschke, P., Kotthoff, L., Lindauer, M., Malitsky, Y., Frechétte, A., Hoos, H., Hutter, F., Leyton-Brown, K., Tierney, K., Vanschoren, J.: ASlib: a benchmark library for algorithm selection. Artif. Intell. 237, 41–58 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  6. Blank, J., Deb, K., Mostaghim, S.: Solving the Bi-objective Traveling Thief Problem with Multi-objective Evolutionary Algorithms. Springer, Berlin (2017)

    Google Scholar 

  7. Bonyadi, M.R., Michalewicz, Z., Barone, L.: The travelling thief problem: the first step in the transition from theoretical problems to realistic problems. In: Congress on Evolutionary Computation, pp. 1037–1044. IEEE, (2013)

  8. Bonyadi, M.R., Michalewicz, Z., Przybylek, M.R., Wierzbicki, A.: Socially inspired algorithms for the TTP. In: Genetic and Evolutionary Computation Conference, pp. 421–428. ACM, (2014)

  9. Bonyadi, M.R., Michalewicz, Z., Neumann, F., Wagner, M.: Evolutionary computation for multicomponent problems: opportunities and future directions. CoRR abs/1606.06818. http://arxiv.org/abs/1606.06818 (2016)

  10. Brazdil, P., Giraud-Carrier, C., Soares, C., Vilalta, R.: Metalearning: Applications to Data Mining, 1st edn. Springer, Berlin (2008)

    Google Scholar 

  11. Breimann, L.: Random forests. Mach. Learn. J. 45, 5–32 (2001)

    Article  Google Scholar 

  12. Chalkiadakis, G., Elkind, E., Wooldridge, M.: Computational Aspects of Cooperative Game Theory, Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, San Rafael (2011)

    Google Scholar 

  13. Chand, S., Wagner, M.: Fast heuristics for the multiple traveling thieves problem. In: Genetic and Evolutionary Computation Conference (GECCO), pp. 293–300. ACM, (2016)

  14. Dantzig, G.B., Ramser, J.H.: The truck dispatching problem. Manag. Sci. 6(1), 80–91 (1959)

    MathSciNet  Article  MATH  Google Scholar 

  15. El Yafrani, M., Ahiod, B.: Population-based versus single-solution heuristics for the travelling thief problem. In: Genetic and Evolutionary Computation Conference (GECCO), pp. 317–324 . ACM, (2016)

  16. Faulkner, H., Polyakovskiy, S., Schultz, T., Wagner, M.: Approximate approaches to the traveling thief problem. In: Genetic and Evolutionary Computation Conference, pp. 385–392. ACM, (2015)

  17. Frechette, A., Kotthoff, L., Rahwan, T., Hoos, H., Leyton-Brown, K., Michalak, T.: Using the shapley value to analyze algorithm portfolios. In: 30th AAAI Conference on Artificial Intelligence (2016)

  18. Hoos, H., Lindauer, M., Schaub, T.: Claspfolio 2: advances in algorithm selection for answer set programming. Theory Pract. Logic Program. 14, 569–585 (2014)

    Article  MATH  Google Scholar 

  19. Hoos, H., Kaminski, R., Lindauer, M., Schaub, T.: Aspeed: solver scheduling via answer set programming. Theory Pract. Logic Program. 15, 117–142 (2015)

    Article  MATH  Google Scholar 

  20. Huberman, B., Lukose, R., Hogg, T.: An economic approach to hard computational problems. Science 275, 51–54 (1997)

    Article  Google Scholar 

  21. Hutter, F., Hoos, H., Leyton-Brown, K., Stützle, T.: ParamILS: an automatic algorithm configuration framework. J. Artif. Intell. Res. 36, 267–306 (2009)

    MATH  Google Scholar 

  22. Hutter, F., Hoos, H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello C (ed.) Proceedings of the Fifth International Conference on Learning and Intelligent Optimization (LION’11). Lecture Notes in Computer Science, vol. 6683, pp. 507–523. Springer, (2011)

  23. Hutter, F., Xu, L., Hoos, H., Leyton-Brown, K.: Algorithm runtime prediction: methods and evaluation. Artif. Intell. 206, 79–111 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  24. Kadioglu, S., Malitsky, Y., Sellmann, M., Tierney, K.: ISAC—instance-specific algorithm configuration. In: Coelho H, Studer R, Wooldridge M (eds.) Proceedings of the Nineteenth European Conference on Artificial Intelligence (ECAI’10), pp. 751–756. IOS Press, (2010)

  25. Kadioglu, S., Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm selection and scheduling. In: Lee J (ed.) Proceedings of the Seventeenth International Conference on Principles and Practice of Constraint Programming (CP’11). Lecture Notes in Computer Science, vol. 6876, pp. 454–469. Springer, (2011)

  26. Klamroth, K., Mostaghim, S., Naujoks, B., Poles, S., Purshouse, R., Rudolph, G., Ruzika, S., Sayn, S., Wiecek, M.M., Yao, X.: Multiobjective optimization for interwoven systems. J. Multi Criteria Decis. Anal. 24(1–2), 71–81 (2017)

    Article  Google Scholar 

  27. Koch, T., Achterberg, T., Andersen, E., Bastert, O., Berthold, T., Bixby, R.E., Danna, E., Gamrath, G., Gleixner, A.M., Heinz, S., Lodi, A., Mittelmann, H., Ralphs, T., Salvagnin, D., Steffy, D.E., Wolter, K.: MIPLIB 2010. Math. Program. Comput. 3(2), 103–163 (2011)

    MathSciNet  Article  Google Scholar 

  28. Kotthoff, L.: Algorithm selection for combinatorial search problems: a survey. In: Bessiere C, De Raedt L, Kotthoff L, Nijssen S, O’Sullivan B, Pedreschi D (eds.) Data Mining and Constraint Programming, pp. 149–190. Springer (2016)

  29. Laporte, G.: The vehicle routing problem: an overview of exact and approximate algorithms. Eur. J. Oper. Res. 59(3), 345–358 (1992)

    MathSciNet  Article  MATH  Google Scholar 

  30. Leyton-Brown, K., Nudelman, E., Shoham, Y.: Learning the empirical hardness of optimization problems: the case of combinatorial auctions. In: Hentenryck PV (ed.) Principles and Practice of Constraint Programming—CP 2002. Lecture Notes in Computer Science, vol. 2470, pp. 556–572. Springer, (2002)

  31. Lindauer, M., Hoos, H., Hutter, F., Schaub, T.: Autofolio: an automatically configured algorithm selector. J. Artif. Intell. 53, 745–778 (2015)

    Google Scholar 

  32. Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm portfolios based on cost-sensitive hierarchical clustering. In: Rossi F (ed.) Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI’13), pp. 608–614. (2013)

  33. Maratea, M., Pulina, L., Ricca, F.: A multi-engine approach to answer-set programming. Theory Pract. Logic Program. 14, 841–868 (2014)

    MathSciNet  Article  Google Scholar 

  34. Martello, S., Pisinger, D., Toth, P.: Dynamic programming and strong bounds for the 0–1 knapsack problem. Manag. Sci. 45(3), 414–424 (1999)

    Article  MATH  Google Scholar 

  35. Mei, Y., Li, X., Yao, X.: Improving efficiency of heuristics for the large scale traveling thief problem. In: Simulated Evolution and Learning. LNCS, vol. 8886, pp. 631–643 Springer (2014a)

  36. Mei, Y., Li, X., Yao, X.: On investigation of interdependence between sub-problems of the TTP. Soft Comput. 20(1), 157–172 (2014b)

    Article  Google Scholar 

  37. Mersmann, O., Bischl, B., Bossek, J., Trautmann, H., Wagner, M., Neumann, F.: Local search and the traveling salesman problem: A feature-based characterization of problem hardness. In: Hamadi Y, Schoenauer M (eds.) Learning and Intelligent Optimization: 6th International Conference (LION 6), pp. 115–129. Springer, (2012)

  38. Mersmann, O., Bischl, B., Trautmann, H., Wagner, M., Bossek, J., Neumann, F.: A novel feature-based approach to characterize algorithm performance for the traveling salesperson problem. Ann. Math. Artif. Intell. 69(2), 151–182 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  39. Michalewicz, Z.: Ubiquity symposium: evolutionary computation and the processes of life: the emperor is naked: evolutionary algorithms for real-world applications. Ubiquity 2012(November), 3:1–3:13 (2012)

    Article  Google Scholar 

  40. Michalewicz, Z., Fogel, D.B.: How to Solve It—Modern Heuristics: Second, Revised and Extended, 2nd edn. Springer, Berlin (2004)

    Google Scholar 

  41. Mısır, M., Sebag, M.: Algorithm selection as a collaborative filtering problem. Technical report. INRIA-Saclay. http://hal.inria.fr/hal-00922840 (2013)

  42. Nallaperuma, S., Wagner, M., Neumann, F.: Ant colony optimisation and the traveling salesperson problem: Hardness, features and parameter settings. In: Proceedings of the 15th Annual Conference Companion on Genetic and Evolutionary Computation, ACM, New York, NY, USA, GECCO ’13 Companion, pp. 13–14. (2013a)

  43. Nallaperuma, S., Wagner, M., Neumann, F., Bischl, B., Mersmann, O., Trautmann, H.: A feature-based comparison of local search and the christofides algorithm for the travelling salesperson problem. In: Proceedings of the Twelfth Workshop on Foundations of Genetic Algorithms XII, ACM, New York, NY, USA, FOGA XII ’13, pp. 147–160. (2013b)

  44. Nallaperuma, S., Wagner, M., Neumann, F.: Parameter prediction based on features of evolved instances for ant colony optimization and the traveling salesperson problem. In: Parallel Problem Solving from Nature PPSN XIII. LNCS, vol. 8672. pp. 100–109. Springer, (2014)

  45. Nallaperuma, S., Wagner, M., Neumann, F.: Analyzing the effects of instance features and algorithm parameters for max min ant system and the traveling salesperson problem. Front. Robot. AI 2, 18 (2015)

    Article  Google Scholar 

  46. Polyakovskiy, S., Neumann, F.: Packing while traveling: Mixed integer programming for a class of nonlinear knapsack problems. In: Integration of AI and OR Techniques in Constraint Programming. LNCS, vol. 9075, pp. 330–344. Springer, (2015)

  47. Polyakovskiy, S., Bonyadi, M.R., Wagner, M., Michalewicz, Z., Neumann, F.: A comprehensive benchmark set and heuristics for the traveling thief problem. In: Genetic and Evolutionary Computation Conference, pp. 477–484. ACM, (2014a)

  48. Polyakovskiy, S., Bonyadi, M.R., Wagner, M., Michalewicz, Z., Neumann, F.: TTP Test Data. http://cs.adelaide.edu.au/~optlog/research/ttp.php (2014b)

  49. Reinelt, G.: TSPLIB—a traveling salesman problem library. ORSA J. Comput. 3(4), 376–384 (1991)

    Article  MATH  Google Scholar 

  50. Rice, J.: The algorithm selection problem. Adv. Comput. 15, 65–118 (1976)

    Article  Google Scholar 

  51. Rizzoli, A.E., Montemanni, R., Lucibello, E., Gambardella, L.M.: Ant colony optimization for real-world vehicle routing problems. Swarm Intell. 1(2), 135–151 (2007)

    Article  Google Scholar 

  52. Smith-Miles, K.: Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Comput. Surv. 41(1), 6 (2008)

    Article  Google Scholar 

  53. Smith-Miles, K., Baatar, D., Wreford, B., Lewis, R.: Towards objective measures of algorithm performance across instance space. Comput. OR 45, 12–24 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  54. Stützle, T., Hoos, H.H.: MAX–MIN ant system. J. Future Gener. Comput. Syst. 16, 889–914 (2000)

    Article  MATH  Google Scholar 

  55. van Rijn, J., Abdulrahman, S., Brazdil, P., Vanschoren, J.: Fast algorithm selection using learning curves. In: Fromont É, Bie TD, van Leeuwen M (eds.) Proceedings of the international symposium on Advances in Intelligent Data Analysis (IDA). Lecture Notes in Computer Science, vol. 9385, pp. 298–309. Springer, (2015)

  56. Vilalta, R., Drissi, Y.: A perspective view and survey of meta-learning. Artif. Intell. Rev. 18(2), 77–95 (2002)

    Article  Google Scholar 

  57. Wagner, M.: Stealing Items More Efficiently with Ants, A Swarm Intelligence Approach to the Travelling Thief Problem. Springer, Cham (2016)

    Google Scholar 

  58. Weise, T., Zapf, M., Chiong, R., Nebro, A.J.: Why is optimization difficult? In: Chiong, R. (ed.) Nature-Inspired Algorithms for Optimisation, pp. 1–50. Springer, Heidelberg (2009)

    Google Scholar 

  59. Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: SATzilla: portfolio-based algorithm selection for SAT. J. Artif. Intell. Res. 32, 565–606 (2008)

    MATH  Google Scholar 

  60. Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: Hydra-MIP: automated algorithm configuration and selection for mixed integer programming. In: RCRA workshop on Experimental Evaluation of Algorithms for Solving Problems with Combinatorial Explosion at the International Joint Conference on Artificial Intelligence (IJCAI). (2011)

  61. Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: Evaluating component solver contributions to portfolio-based algorithm selectors. In: Cimatti A, Sebastiani R (eds.) Proceedings of the Fifteenth International Conference on Theory and Applications of Satisfiability Testing (SAT’12). Lecture Notes in Computer Science, vol. 7317, pp. 228–241. Springer, (2012)

  62. Yafrani, M.E., Chand, S., Neumann, A., Wagner, M.: A Case Study of Multi-objectiveness in Multi-component Problems. http://cs.adelaide.edu.au/~optlog/research/combinatorial.php (2017)

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Markus Wagner.

Additional information

M. Wagner was supported by the Australian Research Council (DE160100850) and by a Priority Partner Grant by the University of Adelaide, Australia. M. Lindauer and F. Hutter were supported by the DFG (German Research Foundation) under Emmy Noether Grant HU 1900/2-1. M. Mısır was supported by the Nanjing University of Aeronautics and Astronautics Starter Research Fund.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wagner, M., Lindauer, M., Mısır, M. et al. A case study of algorithm selection for the traveling thief problem. J Heuristics 24, 295–320 (2018). https://doi.org/10.1007/s10732-017-9328-y

Download citation

Keywords

  • Combinatorial optimization
  • Instance analysis
  • Algorithm portfolio