Abstract
Obstacle avoidance and path planning of unmanned aerial vehicles (UAVs) is an essential and challenging task, especially in the unknown environment with dynamic obstacles. To address this problem, a method of UAV path planning based on Deep Q-Learning is proposed. The experience replay mechanism is introduced in the deep reinforcement learning (DRL) process, and a value network is established to calculate the optimal value for the action of the UAV. The optimal flight policy of the UAV is determined through the \(\epsilon \)-greed algorithm. The experimental results show that the UAV with well-trained model can avoid the obstacles in motion perfectly, and the cruise time is reduced by half compared with the untrained UAV.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dorling, K., Heinrichs, J., Messier, G.G.: Vehicle routing problems for drone delivery. IEEE Trans. Syst. Man Cybern. 47(1), 70–85 (2017). https://doi.org/10.1109/TSMC.2016.2582745
Yin, C., Xiao, Z., Cao, X., et al.: Offline and online search: UAV multiobjective path planning under dynamic urban environment. IEEE Internet Things J. 5(2), 546–558 (2018). https://doi.org/10.1109/JIOT.2017.2717078
Bevacqua, G., Cacace, J., Finzi, A., Lippiello, V.: Mixed-initiative planning and execution for multiple drones in search and rescue missions. In: Twenty-Fifth International Conference on Automated Planning and Scheduling. AAAI Publications (2015)
Chen, Y.B., Luo, G.C., Mei, Y.S., et al.: UAV path planning using artificial potential field method updated by optimal control theory. Int. J. Syst. Sci. 47(6), 1407–1420 (2016). https://doi.org/10.1080/00207721.2014.929191
Bortoff, S.A.: Path Planning for UAV. In: Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No. 00CH36334) (2000)
Peng, X., Xu, D., Zhang, F.: UAV online path planning based on dynamic multiobjective evolutionary algorithm. In: Proceedings of the 30th Chinese Control Conference, pp. 5424–5429. IEEE (2011)
Ragi, S., Chong, E.K.P.: UAV path planning in a dynamic environment via partially observable Markov decision process. IEEE Trans. Aerosp. Electron. Syst. 49(4), 2397–2412 (2013). https://doi.org/10.1109/TAES.2013.6621824
Roberge, V., Tarbouchi, M., Labonte, G.: Comparison of parallel genetic algorithm and particle swarm optimization for real-time UAV path planning. IEEE Trans. Industr. Inf. 9(1), 132–141 (2013). https://doi.org/10.1109/TII.2012.2198665
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015). https://doi.org/10.1038/nature14236
Chen, Y.F., Liu, M., Everett, M., et al.: Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 285–292. IEEE (2017). https://doi.org/10.1109/ICRA.2017.7989037
Chen, C., Liu, Y., Kreiss, S., et al.: Crowd-robot interaction: crowd-aware robot navigation with attention-based deep reinforcement learning. arXiv preprint arXiv:1809.08835 (2018)
Wan, Q., Liu, W., Xu, L., et al.: Research on optimal strategy planning of uncertain environment BDI agent based on Q-learning. Comput. Eng. Sci. 41(01), 166–172 (2019)
Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015)
Nair, A., McGrew, B., Andrychowicz, M., et al.: Overcoming exploration in reinforcement learning with demonstrations. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 6292–6299. IEEE (2018)
Hu, Y., Da, Q., Zeng, A., et al.: Reinforcement learning to rank in e-commerce search engine: formalization, analysis, and application. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 368–377. ACM (2018)
Luo, W., Tang, Q., Fu, C., Eberhard, P.: Deep-Sarsa based multi-UAV path planning and obstacle avoidance in a dynamic environment. In: International Conference on Sensing and Imaging. Advances in Swarm Intelligence, pp. 102–111 (2018). https://doi.org/10.1007/978-3-319-93818-9_10
van den Berg, J., Guy, S.J., Lin, M., Manocha, D.: Reciprocal \(n\)-body collision avoidance. In: Robotics Research. Springer Tracts in Advanced Robotics, vol. 70, pp. 3–19 (2011). https://doi.org/10.1007/978-3-642-19457-3_1
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, G., Zheng, X., Zhao, H., Zhao, Q., Zhang, C., Zhang, B. (2020). Unmanned Aerial Vehicles Path Planning Based on Deep Reinforcement Learning. In: Liu, Y., Wang, L., Zhao, L., Yu, Z. (eds) Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. ICNC-FSKD 2019. Advances in Intelligent Systems and Computing, vol 1074. Springer, Cham. https://doi.org/10.1007/978-3-030-32456-8_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-32456-8_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32455-1
Online ISBN: 978-3-030-32456-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)