Advertisement

A New Approach to Improve the Ant Colony System Performance: Learning Levels

  • Laura Cruz R.
  • Juan J. Gonzalez B.
  • José F. Delgado Orta
  • Barbara A. Arrañaga C.
  • Hector J. Fraire H.
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5572)

Abstract

In this paper a hybrid ant colony system algorithm is presented. A new approach to update the pheromone trails, denominated learning levels, is incorporated. Learning levels is based on the distributed Q-learning algorithm, a variant of reinforcement learning, which is incorporated to the basic ant colony algorithm. The hybrid algorithm is used to solve the Vehicle Routing Problem with Time Windows. Experimental results with the Solomon’s dataset of instances reveal that learning levels improve execution time and quality, respect to the basic ant colony system algorithm, 0.15% for traveled distance and 0.6% in vehicles used. Now we are applying the hybrid ant colony system in other domains.

Keywords

Ant Colony System (ACS) Distribued Q-learning (DQL) vehicle routing problem (VRP) 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Toth, P., Vigo, D.: The Vehicle Routing Problem. In: Proc. SIAM Monographs on Discrete Mathematics and Applications (2002)Google Scholar
  2. 2.
    Bent, R., Van Hentenryck, P.: A Two-Stage Hybrid Local Search for the Vehicle Routing Problem with Time Windows. Transportation Science (2001)Google Scholar
  3. 3.
    Bräysy, O.: A Reactive Variable Neighborhood Search Algorithm for the Vehicle Routing Problem with Time Windows. Tech. report, SINTEF Applied Mathematics, Department of Optimization (2001)Google Scholar
  4. 4.
    Pisinger, D., Ropke, S.: A General Heuristic for Vehicle Routing Problems. Tech. report, Dept. of Computer Science, Univ. Copenhagen (2005)Google Scholar
  5. 5.
    Gambardella, L., Taillard, E., Agazzi, G.: MACS-VRPTW: A Múltiple Ant Colony System for Vehicle Routing Problems with Time Windows. Tech. report IDSIA-06-99, IDSIA (1999)Google Scholar
  6. 6.
    Homberger, J., Gehring, H.: Two Evolutionary Metaheuristics for the Vehicle Routing Problems with Time Windows. INFOR 37, 297–318 (1999)Google Scholar
  7. 7.
    Berger, J., Barkaoui, M.: A Memetic Algorithm for the Vehicle Routing Problem with Time Windows. In: The 7th International Command and Control Research and Technology Symposium (2002)Google Scholar
  8. 8.
    Rochat, Y., Taillard, E.: Probabilistic Diversification and Intensification in Local Search for Vehicle Routing. Journal of Heuristics 1, 147–167 (1995)CrossRefMATHGoogle Scholar
  9. 9.
    Taillard, E., et al.: A Tabu Search Heuristic for the Vehicle Routing Problem with Soft Time Windows. Transportation Science 31, 170–186 (1997)CrossRefMATHGoogle Scholar
  10. 10.
    Cordeau, F., et al.: The VRP with time windows. Technical Report Cahiers du GERAD G-99-13, Ecole des Hautes ´Etudes Commerciales de Montreal (1999)Google Scholar
  11. 11.
    Potvin, J.Y., Rousseau, J.M.: An Exchange Heuristic for Routeing Problems with Time Windows. Journal of the Operational Research Society 46, 1433–1446 (1995)CrossRefMATHGoogle Scholar
  12. 12.
    Dorigo, M., Gambardella, L.M.: Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem. Technical Report TR/IRIDIA/1996-5, IRIDIA, Université Libre de Bruxelles (1996)Google Scholar
  13. 13.
    Dantzig, G.B., Ramser, J.H.: The Truck Dispatching Problem. Management Science 6(1), 80–91 (1959)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Jong, C., Kant, G., Vliet, A.V.: On Finding Minimal Route Duration in the Vehicle Routing Problem with Multiple Time Windows. Tech. report. Dept. of Computer Science, Utrecht Univ. (1996)Google Scholar
  15. 15.
    Shaw, P.: Using Constraint Programming and Local Search Methods to Solve Vehicle Routing Problems. In: Maher, M.J., Puget, J.-F. (eds.) CP 1998. LNCS, vol. 1520, pp. 417–431. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  16. 16.
    Dorronsoro, B.: The VRP Web (2006), http://neo.lcc.uma.es/radi-aeb/WebVRP/
  17. 17.
    Dorigo, M.: Positive Feedback as a Search Strategy. Technical Report. No. 91-016. Politecnico Di Milano, Italy (1991)Google Scholar
  18. 18.
    Colorni, A., Dorigo, M., Matienzo, V.: Distributed Optimization by Ant Colonies. In: Varela, F.J., Bourgine, P. (eds.) Proc. First European Conference on Artificial Life, pp. 134–142. MIT Press, Cambridge (1992)Google Scholar
  19. 19.
    Colorni, A., Dorigo, M., Maniezzo, V.: An Investigation of Some Properties of an Ant Algorithm. In: Manner, R., Manderick, B. (eds.) Proceedings of PPSN-II, Second International Conference on Parallel Problem Solving from Nature, pp. 509–520. Elsevier, Amsterdam (1992)Google Scholar
  20. 20.
    Gambardella, L.M., Dorigo, M.: Ant-Q: A Reinforcement Learning Approach to the Traveling Salesman Problem. In: Prieditis, A., Russell, S. (eds.) Proceedings of ML1995, Twelfth International Conference on Machine Learning, Tahoe City, CA, pp. 252–260. Morgan Kaufmann, San Francisco (1995)Google Scholar
  21. 21.
    Dorigo, M., Gambardella, L.M.: Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem. Technical Report TR/IRIDIA/1996-5, IRIDIA, Université Libre de Bruxelles (1996)Google Scholar
  22. 22.
    Stützle, T., Hoos, H.H.: Improving the Ant System: A detailed report on the MAXMIN Ant System. Technical report AIDA-96-12, FG Intellektik, FB Informatik, TU Darmstadt (1996)Google Scholar
  23. 23.
    Bullnheimer, B., Hartl, R.F., Strauss, C.: A New Rank Based Version of the Ant System: A Computational Study, Technical report, Institute of Management Science, University of Vienna, Austria (1997)Google Scholar
  24. 24.
    Mariano, C., Morales, E.F.: DQL: A New Updating Strategy for Reinforcement Learning Based on Q-Learning. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS, vol. 2167, pp. 324–335. Springer, Heidelberg (2001)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Laura Cruz R.
    • 1
  • Juan J. Gonzalez B.
    • 1
  • José F. Delgado Orta
    • 1
  • Barbara A. Arrañaga C.
    • 1
  • Hector J. Fraire H.
    • 1
  1. 1.Instituto Tecnológico de Ciudad MaderoCiudad Madero TamaulipasMéxico

Personalised recommendations