Abstract
This paper suggests a routing method for automated guided vehicles in port terminals that uses the Q-learning technique. One of the most important issues for the efficient operation of an automated guided vehicle system is to find shortest routes for the vehicles. In this paper, we determine shortest-time routes inclusive of the expected waiting times instead of simple shortest-distance routes, which are usually used in practice. For the determination of the total travel time, the waiting time must be estimated accurately. This study proposes a method for estimating for each vehicle the waiting time that results from the interferences among vehicles during travelling. The estimation of the waiting times is achieved by using the Q-learning technique and by constructing the shortest-time routing matrix for each given set of positions of quay cranes. An experiment was performed to evaluate the performance of the learning algorithm and to compare the performance of the learning-based routes with that of the shortest-distance routes by a simulation study.
Similar content being viewed by others
References
Broadbent AJ, Besant CB, Premi SK, Walker SP (1985) Free ranging AGV systems: promises, problems and pathways. In: Proceeding of the 2nd international conference on automated materials handling, pp 221–237
Evers JJM, Koppers SAJ (1996) Automated guided vehicle traffic control at a container terminal. Transp Res A 30(1):21–34
Gaskins RJ, Tanchoco JMA (1987) Flow path design for automated guided vehicle systems. Int J Prod Res 25(5):667–676
Kim CW, Tanchoco JMA (1991) Conflict free shortest time bi-directional AGV routing. Int J Prod Res 29(12):2377–2391
Lim JK, Lim JM, Yoshimoto K, Kim KH, Takahashi T (2002) A construction algorithm for designing guide paths of automated guided vehicle system. Int J Prod Res 40(15):3981–3994
Mahadevan S (1996) Average reward reinforcement learning; foundation, algorithms, and empirical results. Mach Learn 22(1):159–195
Mitchell TM (1997) Machine learning. McGraw-hill, New York
Oboth C, Batta R, Karwan M (1999) Dynamic conflict free routing of automated guided vehicles. Int J Prod Res 37(9):2003–2030
Qiu L, Hsu WJ, Huang SY, Wang H (2002) Scheduling and routing algorithms for AGV’s: a survey. Int J Prod Res 40(3):745–760
Rajotia S, Shanker K, Batra JL (1998) A semi-dynamic time window constrained routing strategy in an AGV system. Int J Prod Res 36(1):35–50
Acknowledgments
This study was partially supported by the Korean-German international symposium program of KOSEF in Korea and DFG in Germany and by a Korea Research Foundation Grant that was funded by the Korean Government (MOEHRD) (The Regional Research Universities Program/Institute of Logistics Information Technology).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jeon, S.M., Kim, K.H. & Kopfer, H. Routing automated guided vehicles in container terminals through the Q-learning technique. Logist. Res. 3, 19–27 (2011). https://doi.org/10.1007/s12159-010-0042-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12159-010-0042-5