Skip to main content

Advertisement

SpringerLink
Log in

A hybrid algorithm using a genetic algorithm and multiagent reinforcement learning heuristic to solve the traveling salesman problem

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In recent years, hybrid genetic algorithms (GAs) have received significant interest and are widely being used to solve real-world problems. The hybridization of heuristic methods aims at incorporating benefits of stand-alone heuristics in order to achieve better results for the optimization problem. In this paper, we propose a hybridization of GAs and Multiagent Reinforcement Learning (MARL) heuristic for solving Traveling Salesman Problem (TSP). The hybridization process is implemented by producing the initial population of GA, using MARL heuristic. In this way, GA with a novel crossover operator, which we have called Smart Multi-point crossover, acts as tour improvement heuristic and MARL acts as construction heuristic. Numerical results based on several TSP datasets taken from the TSPLIB demonstrate that proposed method found optimum solution of many TSP datasets and near optimum of the others and enable to compete with nine state-of-the-art algorithms, in terms of solution quality and CPU time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Helsgaun K (2000) An effective implementation of the Lin-Kernighan traveling salesman heuristic. Eur J Oper Res 126(1):106–130

    Article  MathSciNet  Google Scholar 

  2. Laporte G (1992) The traveling salesman problem: an overview of exact and approximate algorithms. Eur J Oper Res 59(2):231–247

    Article  MathSciNet  Google Scholar 

  3. Johnson DS, McGeoch LA (1997) The traveling salesman problem: a case study in local optimization. Local Search Comb Optim 1:215–310

    MATH  Google Scholar 

  4. Johnson DS, McGeoch LA (2007) Experimental analysis of heuristics for the STSP. In: The traveling salesman problem and its variations. Springer, US, pp 369–443

    Chapter  Google Scholar 

  5. Lenstra JK (1997) Local search in combinatorial optimization. Princeton University Press, Princeton

    MATH  Google Scholar 

  6. Dorigo M, Gambardella LM (1997) Ant colonies for the traveling salesman problem. BioSystems 43(2):73–81

    Article  Google Scholar 

  7. Dorigo M, Gambardella LM (1997) Ant colony system: a cooperative learning approach for the traveling salesman problem. IEEE Trans Evolut Comput 1(1):53–66

    Article  Google Scholar 

  8. Dorigo M, Gambardella LM (2016) Ant-Q: a reinforcement learning approach to the traveling salesman problem. In: Proceedings of the twelfth international conference on machine learning (ML-95), pp 252–260

  9. Gündüz M, Kiran MS, Özceylan E (2015) A hierarchic approach based on swarm intelligence to solve traveling salesman problem. Turk J Electr Eng Comput Sci 23(1):103–117

    Article  Google Scholar 

  10. Dong GF, Guo WW, Tickle K (2012) Solving the traveling salesman problem using cooperative genetic ant systems. Expert Syst Appl 39(5):5006–5011

    Article  Google Scholar 

  11. Yong W (2015) Hybrid max-min ant system with four vertices and three lines inequality for traveling salesman problem. Soft Comput 19(3):585–596

    Article  Google Scholar 

  12. Haykin S, Network N (2004) A comprehensive foundation. Neural Netw 2:2004

    Google Scholar 

  13. Budinich M (1996) A self-organizing neural network for the traveling salesman problem that is competitive with simulated annealing. Neural Comput 8(2):416–424

    Article  Google Scholar 

  14. Li R, Qiao J, Li W (2016) A modified hopfield neural network for solving TSP problem. In: 12th world congress on proceedings of the in intelligent control and automation (WCICA), 2016, IEEE

  15. Masutti TA, de Castro LN (2009) A self-organizing neural network using ideas from the immune system to solve the traveling salesman problem. Inf Sci 179(10):1454–1468

    Article  MathSciNet  Google Scholar 

  16. Créput JC, Koukam A (2009) A memetic neural network for the Euclidean traveling salesman problem. Neurocomputing 72(4):1250–1264

    Article  Google Scholar 

  17. Thanh PD, Binh HTT, Lam BT (2015) New mechanism of combination crossover operators in genetic algorithm for solving the traveling salesman problem. In: Knowledge and systems engineering. Springer International Publishing, pp 367–379

  18. Tsai CW, Tseng SP, Chiang MC, Yang CS, Hong TP (2014) A high-performance genetic algorithm: using traveling salesman problem as a case. Sci World J 2014:14, Article ID 178621. doi:10.1155/2014/178621

    Article  Google Scholar 

  19. Sallabi OM, El-Haddad Y (2009) An improved genetic algorithm to solve the traveling salesman problem. World Acad Sci, Eng Technol 52(3):471–474

    Google Scholar 

  20. Chen SM, Chien CY (2011) Solving the traveling salesman problem based on the genetic simulated annealing ant colony system with particle swarm optimization techniques. Expert Syst Appl 38(12):14439–14450

    Article  Google Scholar 

  21. Malek M, Guruswamy M, Pandya M, Owens H (1989) Serial and parallel simulated annealing and tabu search algorithms for the traveling salesman problem. Ann Oper Res 21(1):59–84

    Article  MathSciNet  Google Scholar 

  22. Geng X, Chen Z, Yang W, Shi D, Zhao K (2011) Solving the traveling salesman problem based on an adaptive simulated annealing algorithm with greedy search. Appl Soft Comput 11(4):3680–3689

    Article  Google Scholar 

  23. Lin Y, Bian Z, Liu X (2016) Developing a dynamic neighborhood structure for an adaptive hybrid simulated annealing–tabu search algorithm to solve the symmetrical traveling salesman problem. Appl Soft Comput 49:937–952

    Article  Google Scholar 

  24. Zhan SH, Lin J, Zhang ZJ, Zhong YW (2016) List-based simulated annealing algorithm for traveling salesman problem. Comput Intell Neurosci 2016:8

    Article  Google Scholar 

  25. Fiechter CN (1994) A parallel tabu search algorithm for large traveling salesman problems. Discret Appl Math 51(3):243–267

    Article  MathSciNet  Google Scholar 

  26. Misevičius A (2015) Using iterated tabu search for the traveling salesman problem. Inf Technol Control 32:3

    Google Scholar 

  27. Wong LP, Low MYH, Chong CS (2008) A bee colony optimization algorithm for traveling salesman problem. In: Proceedings of the second Asia international conference on modelling and simulation, IEEE

  28. Meng L, Yin S, Hu X (2016) A new method used for traveling salesman problem based on discrete artificial bee colony algorithm. TELKOMNIKA Telecommun Comput Electron Control 14(1):342–348

    Article  Google Scholar 

  29. Shi XH, Liang YC, Lee HP, Lu C, Wang QX (2007) Particle swarm optimization-based algorithms for TSP and generalized TSP. Inf Process Lett 103(5):169–176

    Article  MathSciNet  Google Scholar 

  30. Merz P, Freisleben B (1997) Genetic local search for the TSP: new results. In: IEEE international conference on proceedings of the in evolutionary computation, 1997, IEEE, pp 159–164

  31. White CM, Yen GG (2004) A hybrid evolutionary algorithm for traveling salesman problem. In: Congress on proceedings of the in evolutionary computation, 2004. CEC2004, IEEE, vol 2, pp 1473–1478

  32. Machado TR, Lopes HS (2005) A hybrid particle swarm optimization model for the traveling salesman problem. In: Adaptive and natural computing algorithms. Springer, Vienna, pp 255–258

    Chapter  Google Scholar 

  33. Yang J, Wu C, Lee HP, Liang Y (2008) Solving traveling salesman problems using generalized chromosome genetic algorithm. Prog Nat Sci 18(7):887–892

    Article  MathSciNet  Google Scholar 

  34. Wang Y (2014) The hybrid genetic algorithm with two local optimization strategies for traveling salesman problem. Comput Ind Eng 70:124–133

    Article  Google Scholar 

  35. Deng Y, Liu Y, Zhou D (2015) An improved genetic algorithm with initial population strategy for symmetric TSP. Math Probl Eng 2015:212794

    Google Scholar 

  36. de Lima Junior FC, de Melo JD, Neto ADD (2007) Using q-learning algorithm for initialization of the grasp metaheuristic and genetic algorithm. In: Proceedings of the in 2007 international joint conference on neural networks, IEEE, pp 1243–1248

  37. Liu F, Zeng G (2009) Study of genetic algorithm with reinforcement learning to solve the TSP. Expert Syst Appl 36(3):6995–7001

    Article  MathSciNet  Google Scholar 

  38. dos Santos JPQ, de Lima FC, Magalhaes RM, de Melo JD, Neto ADD (2009) A parallel hybrid implementation using genetic algorithm, GRASP and reinforcement learning. In: Proceedings of the In 2009 international joint conference on neural networks. IEEE, pp 2798–2803

  39. Alipour MM, Razavi SN (2015) A new multiagent reinforcement learning algorithm to solve the symmetric traveling salesman problem. Multiagent Grid Syst 11(2):107–119

    Article  Google Scholar 

  40. Jünger M, Reinelt G, Rinaldi G (1995) The traveling salesman problem. Handb Oper Res Manag Sci 7:225–330

    MathSciNet  MATH  Google Scholar 

  41. Holland JH (1975) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. U Michigan Press, Ann Arbor

    MATH  Google Scholar 

  42. Goldberg DE, Lingle R (1985) Alleles, loci, and the traveling salesman problem. In: Proceedings of an international conference on genetic algorithms and their applications, vol 154. Lawrence Erlbaum, Hillsdale, pp 154–159

  43. Davis L (1985) Applying adaptive algorithms to epistatic domains. In: Proceedings of the in IJCAI, vol. 85, pp 162–164

  44. Oliver IM, Smith D, Holland JR (1987) Study of permutation crossover operators on the traveling salesman problem. In: Proceedings of the in genetic algorithms and their applications: proceedings of the second international conference on genetic algorithms: July 28-31, 1987 at the Massachusetts Institute of Technology, Cambridge, MA. Hillsdale, NJ: L. Erlhaum Associates, 1987

  45. Whitley LD, Starkweather T, Fuquay DA (1989) Scheduling problems and traveling salesmen: the genetic edge recombination operator. In: Proceedings of the In ICGA, vol 89, pp 133–40

  46. Kaya M (2011) The effects of two new crossover operators on genetic algorithm performance. Appl Soft Comput 11(1):881–890

    Article  Google Scholar 

  47. Alipour MM, Razavi SN (2016) A novel local search heuristic based on nearest insertion into the convex hull for solving euclidean TSP. Int J Oper Res, (Under Publishing)

  48. Russell S, Norvig P (2010) Artificial intelligence: a modern approach. Prentice Hall, Englewood Cliffs

    MATH  Google Scholar 

  49. Claus C, Boutilier C (1998) The dynamics of reinforcement learning in cooperative multiagent systems. In: Proceedings of the In AAAI/IAAI, pp 746–752)

  50. Reinelt G TSPLIB is a library of sample benchmark instances for the TSP (and related problems) from various sources and of various types’ [online] http://comopt.ifi.uniheidelberg.de/software/TSPLIB95/. Accessed Feb 2016

  51. Tokic M, Schwenker F, Palm G (2013) Meta-learning of exploration and exploitation parameters with replacing eligibility traces. In: Proceedings of the In IAPR international workshop on partially supervised learning. Springer, Berlin, pp 68–79

    Google Scholar 

  52. Kobayashi K, Mizoue H, Kuremoto T, Obayashi M (2009) A meta-learning method based on temporal difference error. In: Proceedings of the in international conference on neural information processing. Springer, Berlin, pp 530–537

    Chapter  Google Scholar 

  53. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT press, Cambridge

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran

    Mir Mohammad Alipour, Seyed Naser Razavi, Mohammad Reza Feizi Derakhshi & Mohammad Ali Balafar

Authors
  1. Mir Mohammad Alipour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seyed Naser Razavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Reza Feizi Derakhshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohammad Ali Balafar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mir Mohammad Alipour.

Ethics declarations

Conflict of interest

We (Author A, Mir Mohammad Alipour; Author B, Seyed Naser Razavi; Author C, Mohammad Reza Feizi Derakhshi and Author D, Mohammad Ali Balafar) wish to confirm that there are no known conflicts of interest associated with this article and there has been no significant financial support for this work that could have influenced its outcome.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alipour, M.M., Razavi, S.N., Feizi Derakhshi, M.R. et al. A hybrid algorithm using a genetic algorithm and multiagent reinforcement learning heuristic to solve the traveling salesman problem. Neural Comput & Applic 30, 2935–2951 (2018). https://doi.org/10.1007/s00521-017-2880-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-017-2880-4

Keywords

Search

Navigation