Logistics Research

, Volume 3, Issue 1, pp 19–27 | Cite as

Routing automated guided vehicles in container terminals through the Q-learning technique

Original Paper


This paper suggests a routing method for automated guided vehicles in port terminals that uses the Q-learning technique. One of the most important issues for the efficient operation of an automated guided vehicle system is to find shortest routes for the vehicles. In this paper, we determine shortest-time routes inclusive of the expected waiting times instead of simple shortest-distance routes, which are usually used in practice. For the determination of the total travel time, the waiting time must be estimated accurately. This study proposes a method for estimating for each vehicle the waiting time that results from the interferences among vehicles during travelling. The estimation of the waiting times is achieved by using the Q-learning technique and by constructing the shortest-time routing matrix for each given set of positions of quay cranes. An experiment was performed to evaluate the performance of the learning algorithm and to compare the performance of the learning-based routes with that of the shortest-distance routes by a simulation study.


AGV Reinforcement learning Shortest pats Estimation of waiting times AGV Container terminal 



This study was partially supported by the Korean-German international symposium program of KOSEF in Korea and DFG in Germany and by a Korea Research Foundation Grant that was funded by the Korean Government (MOEHRD) (The Regional Research Universities Program/Institute of Logistics Information Technology).


  1. 1.
    Broadbent AJ, Besant CB, Premi SK, Walker SP (1985) Free ranging AGV systems: promises, problems and pathways. In: Proceeding of the 2nd international conference on automated materials handling, pp 221–237Google Scholar
  2. 2.
    Evers JJM, Koppers SAJ (1996) Automated guided vehicle traffic control at a container terminal. Transp Res A 30(1):21–34Google Scholar
  3. 3.
    Gaskins RJ, Tanchoco JMA (1987) Flow path design for automated guided vehicle systems. Int J Prod Res 25(5):667–676CrossRefGoogle Scholar
  4. 4.
    Kim CW, Tanchoco JMA (1991) Conflict free shortest time bi-directional AGV routing. Int J Prod Res 29(12):2377–2391MATHCrossRefGoogle Scholar
  5. 5.
    Lim JK, Lim JM, Yoshimoto K, Kim KH, Takahashi T (2002) A construction algorithm for designing guide paths of automated guided vehicle system. Int J Prod Res 40(15):3981–3994MATHCrossRefGoogle Scholar
  6. 6.
    Mahadevan S (1996) Average reward reinforcement learning; foundation, algorithms, and empirical results. Mach Learn 22(1):159–195Google Scholar
  7. 7.
    Mitchell TM (1997) Machine learning. McGraw-hill, New YorkGoogle Scholar
  8. 8.
    Oboth C, Batta R, Karwan M (1999) Dynamic conflict free routing of automated guided vehicles. Int J Prod Res 37(9):2003–2030MATHCrossRefGoogle Scholar
  9. 9.
    Qiu L, Hsu WJ, Huang SY, Wang H (2002) Scheduling and routing algorithms for AGV’s: a survey. Int J Prod Res 40(3):745–760MATHCrossRefGoogle Scholar
  10. 10.
    Rajotia S, Shanker K, Batra JL (1998) A semi-dynamic time window constrained routing strategy in an AGV system. Int J Prod Res 36(1):35–50MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  1. 1.Seaport InnovationsTamswegAustria
  2. 2.Department of Industrial EngineeringPusan National UniversityBusanKorea
  3. 3.Lehrstuhl für Logistik, FB 7University of BremenBremenGermany

Personalised recommendations