Q(λ) Based Vector Direction for Path Planning Problem of Autonomous Mobile Robots

Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 107)


This paper presents a novel algorithm to improve the efficiency of path planning for autonomous mobile robots. In an obstacle-free environment, the path planning of a robot is attained by following the vector direction from its current position to the goal position. In an obstacle environment, while following the vector direction, a robot has to avoid obstacles by rotating the moving direction. To accomplish the obstacle avoidance task for the mobile robot, the Q(λ) algorithm is employed to train the robot to learn suitable moving directions. Experimental results show that the proposed algorithm is soundness and completeness with a fast learning rate in the large environment of states and obstacles.


Reinforcement learning Q-learning Q(λ) algorithm Path planning Mobile robots 



This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science, and Technology (2010-0012609).


  1. 1.
    Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285Google Scholar
  2. 2.
    Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. The MIT Press, CambridgeGoogle Scholar
  3. 3.
    Watkins C (1989) Learning from delayed rewards. Dissertation, Ph.D., King’s CollegeGoogle Scholar
  4. 4.
    Smart WD, Kaelbling LP (2002) Effective reinforcement learning for mobile robots. In: IEEE international conference on robotics and automation (ICRA’02), vol 4. IEEE Press, Washington, pp 3404–3410Google Scholar
  5. 5.
    Zamstein L, Arroyo A, Schwartz E, Keen S, Sutton B, Gandhi G (2006) Koolio: path planning using reinforcement learning on a real robot platform. In: 19th Florida conference on recent advances in robotics, Miami, May 2006Google Scholar
  6. 6.
    Chakraborty IG, Das PK, Konar A, Janarthanan R (2010) Extended Q-learning algorithm for path-planning of a mobile robot. LNCS, vol 6457. Springer, Heidelberg, pp 379–383Google Scholar
  7. 7.
    Vien NA, Viet NH, Lee SG, Chung TC (2007) Obstacle avoidance path planning for mobile robot based on ant-q reinforcement learning algorithm. LNCS, vol 4491. Springer, Heidelberg, pp 704–713Google Scholar
  8. 8.
    Mohammad AKJ, Mohammad AR, Lara Q (2011) Reinforcement based mobile robot navigation in dynamic environment. Robotics Comput-Integr Manuf 27:135–149CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  • Hyun Ju Hwang
    • 1
  • Hoang Huu Viet
    • 1
  • TaeChoong Chung
    • 1
  1. 1.Artificial Intelligence Lab, Department of Computer Engineering, School of Electronics and InformationKyung Hee UniversityYonginSouth Korea

Personalised recommendations