A Cross Entropy Multiagent Learning Algorithm for Solving Vehicle Routing Problems with Time Windows

  • Tai-Yu Ma
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6971)


The vehicle routing problem with time windows (VRPTW) has been the subject of intensive study because of its importance in real applications. In this paper, we propose a cross entropy multiagent learning algorithm, which considers an optimum solution as a rare event to be learned. The routing policy is node-distributed, controlled by a set of parameterized probability distribution functions. Based on the performance of experienced tours of vehicle agents, these parameters are updated iteratively by minimizing Kullback-Leibler cross entropy in order to generate better solutions in next iterations. When applying the proposed algorithm on Solomon’s 100-customer problem set, it shows outperforming results in comparison with the classical cross entropy approach. Moreover, this method needs only very small number of parameter settings. Its implementation is also relatively simple and flexible to solve other vehicle routing problems under various dynamic scenarios.


Local Search Vehicle Route Problem Local Search Algorithm Local Search Procedure Cross Entropy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cordeau, J.F., Desaulniers, G., Desrosiers, J., Solomon, M.M., Soumis, F.: The VRP with time windows. In: Toth, P., Vigo, D. (eds.) The Vehicle Routing Problem. SIAM Monographs on Discrete Mathematics and Applications, pp. 157–194 (2002)Google Scholar
  2. 2.
    Braysy, O., Gendreau, M.: Vehicle routing problem with time windows. Part I: Route Construction and Local Search Algorithms. Transportation Science 39, 104–118 (2005)zbMATHGoogle Scholar
  3. 3.
    Braysy, O., Gendreau, M.: Vehicle routing problem with time windows. Part II: Metaheuristics. Transportation Science 39, 119–139 (2005)zbMATHGoogle Scholar
  4. 4.
    Golden, B., Raghavan, S., Wasil, E. (eds.): The vehicle routing problem, latest advances and new challenges. Operations Research/Computer Science Interfaces Series, vol. 43. Springer, Berlin (2008)Google Scholar
  5. 5.
    Ibaraki, T., Imahori, S., Kubo, M., Masuda, T., Uno, T., Yagiura, M.: Effective local search algorithms for routing and scheduling problems with general time window constraints. Transportation Science 39(2), 206–232 (2005)CrossRefGoogle Scholar
  6. 6.
    Hashimoto, H., Ibaraki, T., Imahori, S., Yagiura, M.: The vehicle routing problem with exible time windows and traveling times. Discrete Applied Mathematics 154, 2271–2290 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Pisinger, D., Ropke, S.: A general heuristic for vehicle routing problems. Computers & Operations Research 34, 2403–2435 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Braysy, O., Dullaert, W., Gendreau, M.: Evolutionary algorithms for the vehicle routing problem with time windows. Journal of Heuristics 10, 587–611 (2004)CrossRefGoogle Scholar
  9. 9.
    Homberger, J., Gehring, H.: A two-phase hybrid metaheuristic for the vehicle routing problem with time windows. European Journal of Operational Research 162, 220–238 (2005)CrossRefzbMATHGoogle Scholar
  10. 10.
    Martin, O., Otto, S.W., Felten, E.W.: Large-step Markov chains for the TSP incorporating local search heuristic. Operation Research Letters 11, 219–224 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Hashimoto, H., Yagiura, M., Ibaraki, T.: An iterated local search algorithm for the time-dependent vehicle routing problem with time windows. Discrete Optimization 5, 434–456 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Helsgaun, K.: An effective implementation of the Lin-Kernighan traveling salesman heuristic. Datalogiske skrifter, Writings on Computer Science, no. 81. Roskilde University (1999)Google Scholar
  13. 13.
    Braysy, O., Hasle, G., Dullaert, W.: A multi-start local search algorithm for the vehicle routing problem with time windows. European Journal of Operational Research 159, 586–605 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Rubinstein, R.Y.: The Cross-Entropy Method for Combinatorial and Continuous Optimization. Methodology and Computing in Applied Probability 2, 127–190 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Irnich, S., Funke, B., Grunert, T.: Sequential search and its application to vehicle-routing problems. Computers & Operations Research 33, 2405–2429 (2006)CrossRefzbMATHGoogle Scholar
  16. 16.
    Barbucha, D., Jedrzejowicz, P.: Multi-agent platform for solving the dynamic vehicle routing problem. In: Proc.of 11th Int. IEEE Conf. on Intelligent Transportation Systems, pp. 517–522 (2008)Google Scholar
  17. 17.
    Vokrinek, J., Komenda, A., Pechoucek, M.: Agents Towards Vehicle Routing Problems. In: Proc. of 9th Int. Conf. on Autonomous Agents and Multiagent Systems, pp. 773–780 (2010)Google Scholar
  18. 18.
    Davidson, P., Henesey, L., Ramstedt, L., Tornquist, J., Wernstedt, F.: An analysis of agent-based approaches to transport logistics. Trans. Res. Part C 13, 255–271 (2005)CrossRefzbMATHGoogle Scholar
  19. 19.
    De Boer, P.T., Kroese, D.P., Mannor, S., Rubinstein, R.Y.: A Tutorial on the Cross-Entropy Method. Annals of Operations Research 134(1), 19–67 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Solomon, M.M.: Algorithms for the vehicle routing and scheduling problems with time window constraints. Operations Research 35, 254–265 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Rubinstein, R.Y., Kroese, D.K.: Simulation and the Monte Carlo Method. Wiley Series in Probability and Statistics (2008)Google Scholar
  22. 22.
    Croes, G.: A method for solving traveling-salesman problems. Operations Research 6, 791–812 (1958)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Jepsen, M., Petersen, B., Spoorendonk, S., Pisinger, D.: Subset-Row Inequalities Applied to the Vehicle-Routing Problem with Time Windows. Operations Research 56(2), 497–511 (2008)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Tai-Yu Ma

There are no affiliations available

Personalised recommendations