Skip to main content

A case study of algorithm selection for the traveling thief problem

Journal of Heuristics volume 24, pages 295–320 (2018)Cite this article

Abstract

Many real-world problems are composed of several interacting components. In order to facilitate research on such interactions, the Traveling Thief Problem (TTP) was created in 2013 as the combination of two well-understood combinatorial optimization problems. With this article, we contribute in four ways. First, we create a comprehensive dataset that comprises the performance data of 21 TTP algorithms on the full original set of 9720 TTP instances. Second, we define 55 characteristics for all TPP instances that can be used to select the best algorithm on a per-instance basis. Third, we use these algorithms and features to construct the first algorithm portfolios for TTP, clearly outperforming the single best algorithm. Finally, we study which algorithms contribute most to this portfolio.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Notes

  1. http://challenge.icon-fet.eu/.

  2. As available at the TTP project page: http://cs.adelaide.edu.au/~optlog/research/ttp.php.

  3. For a more detailed description, we refer the interested reader to Polyakovskiy et al. (2014b, 2014a).

  4. https://cran.r-project.org/web/packages/tspmeta/.

  5. http://www.ml4aad.org/flexfolio/.

  6. See TTP-2016 at http://www.aslib.net.

  7. We use the worst possible performance as the performance of an empty portfolio: \(\text {Oracle}(\{\},\mathcal {I}) = -1\).

References

  • Applegate, D., Cook, W.J., Rohe, A.: Chained Lin–Kernighan for large traveling salesman problems. J. Comput. 15(1), 82–92 (2003)

    MathSciNet  MATH  Google Scholar 

  • Beasley, E.J.: Or-library: distributing test problems by electronic mail. J. Oper. Res. Soc. 41(11), 1069–1072 (1990)

    Article  Google Scholar 

  • Bell, J.E., McMullen, P.R.: Ant colony optimization techniques for the vehicle routing problem. Adv. Eng. Inform. 18(1), 41–48 (2004)

    Article  Google Scholar 

  • Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability, Frontiers in Artificial Intelligence and Applications, vol. 185. IOS Press, Amsterdam (2009)

    Google Scholar 

  • Bischl, B., Kerschke, P., Kotthoff, L., Lindauer, M., Malitsky, Y., Frechétte, A., Hoos, H., Hutter, F., Leyton-Brown, K., Tierney, K., Vanschoren, J.: ASlib: a benchmark library for algorithm selection. Artif. Intell. 237, 41–58 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  • Blank, J., Deb, K., Mostaghim, S.: Solving the Bi-objective Traveling Thief Problem with Multi-objective Evolutionary Algorithms. Springer, Berlin (2017)

    Book  Google Scholar 

  • Bonyadi, M.R., Michalewicz, Z., Barone, L.: The travelling thief problem: the first step in the transition from theoretical problems to realistic problems. In: Congress on Evolutionary Computation, pp. 1037–1044. IEEE, (2013)

  • Bonyadi, M.R., Michalewicz, Z., Przybylek, M.R., Wierzbicki, A.: Socially inspired algorithms for the TTP. In: Genetic and Evolutionary Computation Conference, pp. 421–428. ACM, (2014)

  • Bonyadi, M.R., Michalewicz, Z., Neumann, F., Wagner, M.: Evolutionary computation for multicomponent problems: opportunities and future directions. CoRR abs/1606.06818. http://arxiv.org/abs/1606.06818 (2016)

  • Brazdil, P., Giraud-Carrier, C., Soares, C., Vilalta, R.: Metalearning: Applications to Data Mining, 1st edn. Springer, Berlin (2008)

    MATH  Google Scholar 

  • Breimann, L.: Random forests. Mach. Learn. J. 45, 5–32 (2001)

    Article  Google Scholar 

  • Chalkiadakis, G., Elkind, E., Wooldridge, M.: Computational Aspects of Cooperative Game Theory, Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, San Rafael (2011)

    MATH  Google Scholar 

  • Chand, S., Wagner, M.: Fast heuristics for the multiple traveling thieves problem. In: Genetic and Evolutionary Computation Conference (GECCO), pp. 293–300. ACM, (2016)

  • Dantzig, G.B., Ramser, J.H.: The truck dispatching problem. Manag. Sci. 6(1), 80–91 (1959)

    MathSciNet  Article  MATH  Google Scholar 

  • El Yafrani, M., Ahiod, B.: Population-based versus single-solution heuristics for the travelling thief problem. In: Genetic and Evolutionary Computation Conference (GECCO), pp. 317–324 . ACM, (2016)

  • Faulkner, H., Polyakovskiy, S., Schultz, T., Wagner, M.: Approximate approaches to the traveling thief problem. In: Genetic and Evolutionary Computation Conference, pp. 385–392. ACM, (2015)

  • Frechette, A., Kotthoff, L., Rahwan, T., Hoos, H., Leyton-Brown, K., Michalak, T.: Using the shapley value to analyze algorithm portfolios. In: 30th AAAI Conference on Artificial Intelligence (2016)

  • Hoos, H., Lindauer, M., Schaub, T.: Claspfolio 2: advances in algorithm selection for answer set programming. Theory Pract. Logic Program. 14, 569–585 (2014)

    Article  MATH  Google Scholar 

  • Hoos, H., Kaminski, R., Lindauer, M., Schaub, T.: Aspeed: solver scheduling via answer set programming. Theory Pract. Logic Program. 15, 117–142 (2015)

    Article  MATH  Google Scholar 

  • Huberman, B., Lukose, R., Hogg, T.: An economic approach to hard computational problems. Science 275, 51–54 (1997)

    Article  Google Scholar 

  • Hutter, F., Hoos, H., Leyton-Brown, K., Stützle, T.: ParamILS: an automatic algorithm configuration framework. J. Artif. Intell. Res. 36, 267–306 (2009)

    MATH  Google Scholar 

  • Hutter, F., Hoos, H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello C (ed.) Proceedings of the Fifth International Conference on Learning and Intelligent Optimization (LION’11). Lecture Notes in Computer Science, vol. 6683, pp. 507–523. Springer, (2011)

  • Hutter, F., Xu, L., Hoos, H., Leyton-Brown, K.: Algorithm runtime prediction: methods and evaluation. Artif. Intell. 206, 79–111 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  • Kadioglu, S., Malitsky, Y., Sellmann, M., Tierney, K.: ISAC—instance-specific algorithm configuration. In: Coelho H, Studer R, Wooldridge M (eds.) Proceedings of the Nineteenth European Conference on Artificial Intelligence (ECAI’10), pp. 751–756. IOS Press, (2010)

  • Kadioglu, S., Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm selection and scheduling. In: Lee J (ed.) Proceedings of the Seventeenth International Conference on Principles and Practice of Constraint Programming (CP’11). Lecture Notes in Computer Science, vol. 6876, pp. 454–469. Springer, (2011)

  • Klamroth, K., Mostaghim, S., Naujoks, B., Poles, S., Purshouse, R., Rudolph, G., Ruzika, S., Sayn, S., Wiecek, M.M., Yao, X.: Multiobjective optimization for interwoven systems. J. Multi Criteria Decis. Anal. 24(1–2), 71–81 (2017)

    Article  Google Scholar 

  • Koch, T., Achterberg, T., Andersen, E., Bastert, O., Berthold, T., Bixby, R.E., Danna, E., Gamrath, G., Gleixner, A.M., Heinz, S., Lodi, A., Mittelmann, H., Ralphs, T., Salvagnin, D., Steffy, D.E., Wolter, K.: MIPLIB 2010. Math. Program. Comput. 3(2), 103–163 (2011)

    MathSciNet  Article  Google Scholar 

  • Kotthoff, L.: Algorithm selection for combinatorial search problems: a survey. In: Bessiere C, De Raedt L, Kotthoff L, Nijssen S, O’Sullivan B, Pedreschi D (eds.) Data Mining and Constraint Programming, pp. 149–190. Springer (2016)

  • Laporte, G.: The vehicle routing problem: an overview of exact and approximate algorithms. Eur. J. Oper. Res. 59(3), 345–358 (1992)

    MathSciNet  Article  MATH  Google Scholar 

  • Leyton-Brown, K., Nudelman, E., Shoham, Y.: Learning the empirical hardness of optimization problems: the case of combinatorial auctions. In: Hentenryck PV (ed.) Principles and Practice of Constraint Programming—CP 2002. Lecture Notes in Computer Science, vol. 2470, pp. 556–572. Springer, (2002)

  • Lindauer, M., Hoos, H., Hutter, F., Schaub, T.: Autofolio: an automatically configured algorithm selector. J. Artif. Intell. 53, 745–778 (2015)

    Google Scholar 

  • Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm portfolios based on cost-sensitive hierarchical clustering. In: Rossi F (ed.) Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI’13), pp. 608–614. (2013)

  • Maratea, M., Pulina, L., Ricca, F.: A multi-engine approach to answer-set programming. Theory Pract. Logic Program. 14, 841–868 (2014)

    MathSciNet  Article  Google Scholar 

  • Martello, S., Pisinger, D., Toth, P.: Dynamic programming and strong bounds for the 0–1 knapsack problem. Manag. Sci. 45(3), 414–424 (1999)

    Article  MATH  Google Scholar 

  • Mei, Y., Li, X., Yao, X.: Improving efficiency of heuristics for the large scale traveling thief problem. In: Simulated Evolution and Learning. LNCS, vol. 8886, pp. 631–643 Springer (2014a)

  • Mei, Y., Li, X., Yao, X.: On investigation of interdependence between sub-problems of the TTP. Soft Comput. 20(1), 157–172 (2014b)

    Article  Google Scholar 

  • Mersmann, O., Bischl, B., Bossek, J., Trautmann, H., Wagner, M., Neumann, F.: Local search and the traveling salesman problem: A feature-based characterization of problem hardness. In: Hamadi Y, Schoenauer M (eds.) Learning and Intelligent Optimization: 6th International Conference (LION 6), pp. 115–129. Springer, (2012)

  • Mersmann, O., Bischl, B., Trautmann, H., Wagner, M., Bossek, J., Neumann, F.: A novel feature-based approach to characterize algorithm performance for the traveling salesperson problem. Ann. Math. Artif. Intell. 69(2), 151–182 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  • Michalewicz, Z.: Ubiquity symposium: evolutionary computation and the processes of life: the emperor is naked: evolutionary algorithms for real-world applications. Ubiquity 2012(November), 3:1–3:13 (2012)

    Article  Google Scholar 

  • Michalewicz, Z., Fogel, D.B.: How to Solve It—Modern Heuristics: Second, Revised and Extended, 2nd edn. Springer, Berlin (2004)

    Book  MATH  Google Scholar 

  • Mısır, M., Sebag, M.: Algorithm selection as a collaborative filtering problem. Technical report. INRIA-Saclay. http://hal.inria.fr/hal-00922840 (2013)

  • Nallaperuma, S., Wagner, M., Neumann, F.: Ant colony optimisation and the traveling salesperson problem: Hardness, features and parameter settings. In: Proceedings of the 15th Annual Conference Companion on Genetic and Evolutionary Computation, ACM, New York, NY, USA, GECCO ’13 Companion, pp. 13–14. (2013a)

  • Nallaperuma, S., Wagner, M., Neumann, F., Bischl, B., Mersmann, O., Trautmann, H.: A feature-based comparison of local search and the christofides algorithm for the travelling salesperson problem. In: Proceedings of the Twelfth Workshop on Foundations of Genetic Algorithms XII, ACM, New York, NY, USA, FOGA XII ’13, pp. 147–160. (2013b)

  • Nallaperuma, S., Wagner, M., Neumann, F.: Parameter prediction based on features of evolved instances for ant colony optimization and the traveling salesperson problem. In: Parallel Problem Solving from Nature PPSN XIII. LNCS, vol. 8672. pp. 100–109. Springer, (2014)

  • Nallaperuma, S., Wagner, M., Neumann, F.: Analyzing the effects of instance features and algorithm parameters for max min ant system and the traveling salesperson problem. Front. Robot. AI 2, 18 (2015)

    Article  Google Scholar 

  • Polyakovskiy, S., Neumann, F.: Packing while traveling: Mixed integer programming for a class of nonlinear knapsack problems. In: Integration of AI and OR Techniques in Constraint Programming. LNCS, vol. 9075, pp. 330–344. Springer, (2015)

  • Polyakovskiy, S., Bonyadi, M.R., Wagner, M., Michalewicz, Z., Neumann, F.: A comprehensive benchmark set and heuristics for the traveling thief problem. In: Genetic and Evolutionary Computation Conference, pp. 477–484. ACM, (2014a)

  • Polyakovskiy, S., Bonyadi, M.R., Wagner, M., Michalewicz, Z., Neumann, F.: TTP Test Data. http://cs.adelaide.edu.au/~optlog/research/ttp.php (2014b)

  • Reinelt, G.: TSPLIB—a traveling salesman problem library. ORSA J. Comput. 3(4), 376–384 (1991)

    Article  MATH  Google Scholar 

  • Rice, J.: The algorithm selection problem. Adv. Comput. 15, 65–118 (1976)

    Article  Google Scholar 

  • Rizzoli, A.E., Montemanni, R., Lucibello, E., Gambardella, L.M.: Ant colony optimization for real-world vehicle routing problems. Swarm Intell. 1(2), 135–151 (2007)

    Article  Google Scholar 

  • Smith-Miles, K.: Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Comput. Surv. 41(1), 6 (2008)

    Article  Google Scholar 

  • Smith-Miles, K., Baatar, D., Wreford, B., Lewis, R.: Towards objective measures of algorithm performance across instance space. Comput. OR 45, 12–24 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  • Stützle, T., Hoos, H.H.: MAX–MIN ant system. J. Future Gener. Comput. Syst. 16, 889–914 (2000)

    Article  MATH  Google Scholar 

  • van Rijn, J., Abdulrahman, S., Brazdil, P., Vanschoren, J.: Fast algorithm selection using learning curves. In: Fromont É, Bie TD, van Leeuwen M (eds.) Proceedings of the international symposium on Advances in Intelligent Data Analysis (IDA). Lecture Notes in Computer Science, vol. 9385, pp. 298–309. Springer, (2015)

  • Vilalta, R., Drissi, Y.: A perspective view and survey of meta-learning. Artif. Intell. Rev. 18(2), 77–95 (2002)

    Article  Google Scholar 

  • Wagner, M.: Stealing Items More Efficiently with Ants, A Swarm Intelligence Approach to the Travelling Thief Problem. Springer, Cham (2016)

    Book  Google Scholar 

  • Weise, T., Zapf, M., Chiong, R., Nebro, A.J.: Why is optimization difficult? In: Chiong, R. (ed.) Nature-Inspired Algorithms for Optimisation, pp. 1–50. Springer, Heidelberg (2009)

    Google Scholar 

  • Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: SATzilla: portfolio-based algorithm selection for SAT. J. Artif. Intell. Res. 32, 565–606 (2008)

    MATH  Google Scholar 

  • Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: Hydra-MIP: automated algorithm configuration and selection for mixed integer programming. In: RCRA workshop on Experimental Evaluation of Algorithms for Solving Problems with Combinatorial Explosion at the International Joint Conference on Artificial Intelligence (IJCAI). (2011)

  • Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: Evaluating component solver contributions to portfolio-based algorithm selectors. In: Cimatti A, Sebastiani R (eds.) Proceedings of the Fifteenth International Conference on Theory and Applications of Satisfiability Testing (SAT’12). Lecture Notes in Computer Science, vol. 7317, pp. 228–241. Springer, (2012)

  • Yafrani, M.E., Chand, S., Neumann, A., Wagner, M.: A Case Study of Multi-objectiveness in Multi-component Problems. http://cs.adelaide.edu.au/~optlog/research/combinatorial.php (2017)

Download references

Author information

Authors and Affiliations

  1. Optimisation and Logistics Group, School of Computer Science, The University of Adelaide, Adelaide, Australia

    Markus Wagner

  2. Institut für Informatik, Albert-Ludwigs-Universität Freiburg, Freiburg, Germany

    Marius Lindauer & Frank Hutter

  3. Institute of Machine Learning and Computational Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Mustafa Mısır

  4. Department of Computer Science, University of Sheffield, Sheffield, UK

    Samadhi Nallaperuma

Authors
  1. Markus Wagner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marius Lindauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mustafa Mısır
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Samadhi Nallaperuma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Frank Hutter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Markus Wagner.

Additional information

M. Wagner was supported by the Australian Research Council (DE160100850) and by a Priority Partner Grant by the University of Adelaide, Australia. M. Lindauer and F. Hutter were supported by the DFG (German Research Foundation) under Emmy Noether Grant HU 1900/2-1. M. Mısır was supported by the Nanjing University of Aeronautics and Astronautics Starter Research Fund.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wagner, M., Lindauer, M., Mısır, M. et al. A case study of algorithm selection for the traveling thief problem. J Heuristics 24, 295–320 (2018). https://doi.org/10.1007/s10732-017-9328-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10732-017-9328-y

Keywords

  • Combinatorial optimization
  • Instance analysis
  • Algorithm portfolio