Abstract
A algorithm of dynamic multi-step reinforcement learning based on virtual potential field path planning is proposed in this paper. Firstly, it is constructed the virtual potential field according to the known information. And then in view of \( Q \) learning algorithm of the \( Q\left( \lambda \right) \) algorithm, a multi-step reinforcement learning algorithm is proposed in this paper. It can update current \( Q \) value used of future dynamic \( k \) steps according to the current environment status. At the same time, the convergence is analyzed. Finally the simulation experiments are done. It shows that the proposed algorithm and convergence and so on are more efficiency than similar algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Scaramuzza, D., Achtelik, M.C., Doitsidis, L., et al.: Vision-controlled micro flying robots: from system design to autonomous navigation and mapping in GPS-denied environments. IEEE Robot. Autom. Mag. 21(3), 26–40 (2014)
Chinnaiah, M.C., Savitri, T.S., Kumar, P.R.: A novel approach in navigation of FPGA robots in robust indoor environment. In: 2015 International Conference on Advanced Robotics and Intelligent Systems (ARIS), vol. 21, pp. 1–6, May 2015
Song, X., Fang, H., Jiao, X., Wang, Y.: Autonomous mobile robot navigation using machine learning. In: 2012 IEEE 6th International Conference on Information and Automation for Sustainability (ICIAfS), vol. 21, pp. 135–140, September 2012
Zuo, B., Chen, J., Wang, L., Wang, Y.: A reinforcement learning based robotic navigation system. In: 2014 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 3452–3457, 5–8 October 2014
Raja, P., Pugazhenthi, S.: Optimal path planning of mobile robots: a review. Int. J. Phys. Sci. 7(9), 1314–1320 (2012)
Niewola, A., Podsedkowski, L.: Nonholonomic mobile robot path planning with linear computational complexity graph searching algorithm. In: 2015 10th International Workshop on Robot Motion and Control (RoMoCo), pp. 217–222. IEEE (2015)
Kong, J., Ding, M., Li, X., et al.: An algorithm for mobile robot path planning using wireless sensor networks. In: 2015 IEEE International Conference on Mechatronics and Automation (ICMA), pp. 2238–2242. IEEE (2015)
Wu, Z., Feng, L.: Obstacle prediction-based dynamic path planning for a mobile robot. Int. J. Adv. Comput. Technol. 4(3), 23–31 (2012)
Hussein, A., Mostafa, H., Badrel-din, M., et al.: Metaheuristic optimization approach to mobile robot path planning. In: 2012 International Conference on Engineering and Technology (ICET), pp. 1–6. IEEE (2012)
Lee, D.W., Seo, S.W., Sim, K.B.: Online evolution for cooperative behavior in group robot systems. Int. J. Control Autom. Syst. 6(2), 282–287 (2008)
Schaal, S., Atkeson, C.: Learning control in robotics. IEEE Robot. Autom. Mag. 17(3), 20–29 (2010)
Liu, C., Xu, X., Hu, D.: Multiobjective reinforcement learning: a comprehensive overview. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 99(4), 1–13 (2013)
Andersen, K.T., Zeng, Y., Christensen, D.D., et al.: Experiments with online reinforcement learning in real-time strategy games. Appl. Artifi. Intell. 23(9), 855–871 (2009)
Lin, L., Xie, H., Zhang, D.: Supervised neural Q-learning based motion control for bionic underwater robots. J. Bionic Eng. 7(Sup), 177–184 (2010)
Hwang, K.S., Lin, H.Y., Hsu, Y.P., et al.: Self-organizing state aggregation for architecture design of Q-learning. Inf. Sci. 181(13), 2813–2822 (2011)
Santos, M., Martin, H.J.A., Lopez, V., et al.: Dyna-H: a heuristic planning reinforcement learning algorithm applied to role-playing game strategy decision systems. Knowl.-Based Syst. 32(1), 28–36 (2012)
Chen, C., Dong, D., Li, H., et al.: Hybrid MDP based integrated hierarchical Q-learning. Sci. China Inf. Sci. 54(11), 2279–2294 (2011)
Grezs, M., Kudenko, D.: Online learning of shaping rewards in reinforcement learning. Neural Netw. 23(4), 541–550 (2010)
Konidaris, G., Scheidwasser, I., Barto, A.G.: Transfer in reinforcement learning via shared features. J. Mach. Learn. Res. 13(1), 1333–1371 (2012)
Megherbi, D.B., Kim, M.A.: Collaborative distributed multi-agent reinforcement learning technique for dynamic agent shortest path planning via selected sub-goals in complex cluttered environments. In: 2015 IEEE International Inter-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pp. 118–124. IEEE (2015)
Acknowledgements
This work was supported in part by national natural science foundation of china (61602187), Science and Guangdong science and technology project (2016A040403122), Guangzhou science and technology project-science research project, (201707010482).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, J., Qi, W., Lu, X. (2017). Multi-step Reinforcement Learning Algorithm of Mobile Robot Path Planning Based on Virtual Potential Field. In: Zou, B., Han, Q., Sun, G., Jing, W., Peng, X., Lu, Z. (eds) Data Science. ICPCSEE 2017. Communications in Computer and Information Science, vol 728. Springer, Singapore. https://doi.org/10.1007/978-981-10-6388-6_45
Download citation
DOI: https://doi.org/10.1007/978-981-10-6388-6_45
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-6387-9
Online ISBN: 978-981-10-6388-6
eBook Packages: Computer ScienceComputer Science (R0)