Routing automated guided vehicles in container terminals through the Q-learning technique
- 261 Downloads
This paper suggests a routing method for automated guided vehicles in port terminals that uses the Q-learning technique. One of the most important issues for the efficient operation of an automated guided vehicle system is to find shortest routes for the vehicles. In this paper, we determine shortest-time routes inclusive of the expected waiting times instead of simple shortest-distance routes, which are usually used in practice. For the determination of the total travel time, the waiting time must be estimated accurately. This study proposes a method for estimating for each vehicle the waiting time that results from the interferences among vehicles during travelling. The estimation of the waiting times is achieved by using the Q-learning technique and by constructing the shortest-time routing matrix for each given set of positions of quay cranes. An experiment was performed to evaluate the performance of the learning algorithm and to compare the performance of the learning-based routes with that of the shortest-distance routes by a simulation study.
KeywordsAGV Reinforcement learning Shortest pats Estimation of waiting times AGV Container terminal
This study was partially supported by the Korean-German international symposium program of KOSEF in Korea and DFG in Germany and by a Korea Research Foundation Grant that was funded by the Korean Government (MOEHRD) (The Regional Research Universities Program/Institute of Logistics Information Technology).
- 1.Broadbent AJ, Besant CB, Premi SK, Walker SP (1985) Free ranging AGV systems: promises, problems and pathways. In: Proceeding of the 2nd international conference on automated materials handling, pp 221–237Google Scholar
- 2.Evers JJM, Koppers SAJ (1996) Automated guided vehicle traffic control at a container terminal. Transp Res A 30(1):21–34Google Scholar
- 6.Mahadevan S (1996) Average reward reinforcement learning; foundation, algorithms, and empirical results. Mach Learn 22(1):159–195Google Scholar
- 7.Mitchell TM (1997) Machine learning. McGraw-hill, New YorkGoogle Scholar