Fast-Maneuvering Target Seeking Based on Double-Action Q-Learning
In this paper, a reinforcement learning method called DAQL is proposed to solve the problem of seeking and homing onto a fast maneuvering target, within the context of mobile robots. This Q-learning based method considers both target and obstacle actions when determining its own action decisions, which enables the agent to learn more effectively in a dynamically changing environment. It particularly suits fast-maneuvering target cases, in which maneuvers of the target are unknown a priori. Simulation result depicts that the proposed method is able to choose a less convoluted path to reach the target when compared to the ideal proportional navigation (IPN) method in handling fast maneuvering and randomly moving target. Furthermore, it can learn to adapt to the physical limitation of the system and do not require specific initial conditions to be satisfied for successful navigation towards the moving target.
KeywordsMoving object navigation reinforcement learning Q-learning
Unable to display preview. Download preview PDF.
- 12.Borg, J.M., Mehrandezh, M., Fenton, R.G., Benhabib, B.: An Ideal Proportional Navigation Guidance system for moving object interception-robotic experiments. In: Systems, Man, and Cybernetics, 2000 IEEE International Conference, vol. 5, pp. 3247–3252 (2000)Google Scholar
- 14.Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. MIT Press, Cambridge (1998)Google Scholar
- 15.Kaelbling, L.P.: Learning in embedded systems. MIT Press, Cambridge (1993)Google Scholar
- 16.Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)Google Scholar
- 17.Sutton, R.S.: Reinforcement Learning. The International Series in Engineering and Computer Science, vol. 173. Kluwer Academic Publishers, Dordrecht (1992)Google Scholar
- 18.Ngai, D.C.K., Yung, N.H.C.: Double action Q-learning for obstacle avoidance in a dynamically changing environment. In: Proceedings of the 2005 IEEE Intelligent Vehicles Symposium, Las Vegas, pp. 211–216 (2005)Google Scholar
- 19.Ngai, D.C.K., Yung, N.H.C.: Performance Evaluation of Double Action Q-Learning in Moving Obstacle Avoidance Problem. In: Proceedings of the 2005 IEEE International Conference on Systems, Man, and Cybernetics, Hawaii, October 2005, pp. 865–870 (2005)Google Scholar