Abstract
Providing a collision-free, safe and efficient optimal trajectory for unmanned aerial vehicles (UAVs) in an unknown dynamic environment is one of the most important issues for researchers. In this paper, a trajectory planning approach for UAV in unknown dynamic environment based on deep reinforcement learning (DRL) is proposed. This study models trajectory planning of UAV as a discrete-time, discrete-action problem, and then proposes an improved deep Q network (IDQN) algorithm to solve it. The IDQN algorithm adds the track angle information of UAV to the reward function to speed up the learning process, furthermore, it also improves the action selection strategy and learning rate setting. Besides, in simulation, the paper considers the trajectory constraints of UAV in order to make the obtained trajectory have better practical availability. Simulation results demonstrate the effectiveness of the IDQN algorithm to implement UAV trajectory planning with constraints in unknown dynamic environments. Meanwhile, comparison with the classical DQN (CDQN) algorithm is conducted to further explore the advantage of the method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Xu Z, Wei R, Zhang Q et al (2016) Obstacle avoidance algorithm for UAVs in unknown environment based on distributional perception and decision making. In: IEEE Chinese Guidance, Navigation and Control Conference (CGNCC). IEEE, pp 1072–1075
Zhao Y, Zheng Z, Liu Y (2018) Survey on computational-intelligence-based UAV path planning. Knowl-Based Syst 158:54–64
Quan L (2012) Mobile robot path planning based on reinforcement learning in an unknown environment. Diss
Mnih V, Kavukcuoglu K, Silver D et al (2013) Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602
Mnih V, Kavukcuoglu K, Silver D et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529
Lillicrap TP, Hunt JJ, Pritzel A et al (2015) Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971
Heess N, Sriram S, Lemmon J et al (2017) Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv:1707.02286
Mnih V, Badia AP, Mirza M et al (2016) Asynchronous methods for deep reinforcement learn-ing. In: International conference on machine learning, pp 1928–1937
Jaderberg M, Mnih V, Czarnecki WM et al (2016) Reinforcement learning with unsupervised auxiliary tasks. arXiv preprint arXiv:1611.05397
Polvara R, Patacchiola M, Sharma S et al (2017) Autonomous quadrotor landing using deep reinforcement learning. arXiv preprint arXiv:1709.03339
Huang H, Yang Y, Gui G et al (2019) Deep reinforcement learning for UAV Navigation through Massive MIMO. arXiv preprint arXiv:1901.10832
Rodriguez-Ramos A, Sampedro C, Bavle H et al (2019) A deep reinforcement learning strategy for UAV autonomous landing on a moving platform. J Intell Robot Syst 93(1–2):351–366
Zeng J, Qin L, Hu Y et al (2019) Integrating a path planner and an adaptive motion controller for navigation in dynamic environments. Appl Sci 9(7):1384
Wang X, Jiang Z, Xie R, Chen H (2012) Engine constraints in UAV route planning (Doctoral Dissertation)
Ng AY, Jordan MI (2003) Shaping and policy search in reinforcement learning. University of California, Berkeley
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, J., Wang, W., Wu, Q. (2020). Trajectory Planning of UAV in Unknown Dynamic Environment with Deep Reinforcement Learning. In: Jia, Y., Du, J., Zhang, W. (eds) Proceedings of 2019 Chinese Intelligent Systems Conference. CISC 2019. Lecture Notes in Electrical Engineering, vol 593. Springer, Singapore. https://doi.org/10.1007/978-981-32-9686-2_54
Download citation
DOI: https://doi.org/10.1007/978-981-32-9686-2_54
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-32-9685-5
Online ISBN: 978-981-32-9686-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)