Collision-Free Trajectory Generation and Tracking for UAVs Using Markov Decision Process in a Cluttered Environment
- 186 Downloads
Abstract
A collision-free trajectory generation and tracking method capable of re-planning unmanned aerial vehicle (UAV) trajectories can increase flight safety and decrease the possibility of mission failures. In this paper, a Markov decision process (MDP) based algorithm combined with backtracking method is presented to create a safe trajectory in the case of hostile environments. Subsequently, a differential flatness method is adopted to smooth the profile of the rerouted trajectory for satisfying the UAV physical constraints. Lastly, a flight controller based on passivity-based control (PBC) is designed to maintain UAV’s stability and trajectory tracking performance. Simulation results demonstrate that the UAV with the proposed strategy is capable of avoiding obstacles in a hostile environment.
Keywords
Collision-free Differential flatness Markov decision process (MDP) Passivity-based control (PBC) Unmanned aerial vehicle (UAV)Preview
Unable to display preview. Download preview PDF.
Notes
Acknowledgments
This work was supported in part by the Natural Sciences and Engineering Research Council of Canada, in part by the National Natural Science Foundation of China under Grant 61573282 and Grant 61603130. The authors would like to express their sincere gratitude to the Editor-in-Chief, the Guest Editors, and the anonymous reviewers whose insightful comments have helped to improve the quality of this paper considerably.
References
- 1.Gundlach, J.: Designing Unmanned Aircraft Systems: A Comprehensive Approach. American Institute of Aeronautics and Astronautics, Reston (2012)CrossRefGoogle Scholar
- 2.Halit, E., Kemal, L.: 3D path planning for multiple UAVs for maximum information collection. J. Intell. Robot. Syst. 73(1–4), 737–762 (2014)Google Scholar
- 3.Angelov, P.: Sense and Avoid in UAS: Research and Applications. Wiley, Hoboken (2012)CrossRefGoogle Scholar
- 4.Yang, K., Gan, S.K., Sukkarieh, S.: An efficient path planning and control algorithm for RUAV’s in unknown and cluttered environment. J. Intell. Robot. Syst. 57, 101–122 (2010)CrossRefzbMATHGoogle Scholar
- 5.Yu, X., Zhang, Y.M.: Sense and avoid technologies with applications to unmanned aircraft systems: Review and prospects. Prog. Aerosp. Sci. 74, 152–166 (2015)CrossRefGoogle Scholar
- 6.Gui, Y., Guo, P., Zhang, H., Lei, Z., Du, X., Du, J., Yu, Q.: Airborne vision-based navigation method for UAV accuracy landing using infrared lamps. J. Intell. Robot. Syst. 72(2), 197–218 (2013)CrossRefGoogle Scholar
- 7.Kuchar, J.K., Yang, L.C.: A review of conflict detection and resolution modeling methods. IEEE Trans. Intell. Trans. Syst. 1(4), 179–189 (2000)CrossRefGoogle Scholar
- 8.Akmeliawati, R., Mareels, I.M.Y.: Nonlinear energy-based control method for aircraft automatic landing systems. IEEE Trans. Control Syst. Technol. 18(4), 871–884 (2010)CrossRefGoogle Scholar
- 9.Khatib, O.: Real time obstacle avoidance for manipulators and bile Robots. Int. J. Rob. Res. 5(1), 90–99 (1986)CrossRefGoogle Scholar
- 10.Lavalle, S.: Planning Algorithms. Cambridge University Press, Cambridge (2006)CrossRefzbMATHGoogle Scholar
- 11.Chuang, J.H., Ahuja, N.: An analytically tractable potential field model of free space and its application in obstacle avoidance. IEEE Trans. Syst. Man. Cybern. B. Cybern. 28, 729–736 (1998)CrossRefGoogle Scholar
- 12.Geiger, B., Horn, J., Delullo, A., Niessner, A., Long, L.: Optimal path planning of UAV using direct collocation with nonlinear programming. In: Proceedings of the AIAA Guidance, Navigation, and Control Conference, Keystone, Colorado (2006)Google Scholar
- 13.Sridhar, B., Ng, H.K., Chen, N.Y.: Aircraft trajectory optimization and contrails avoidance in the presence of winds. J. Guid. Control. Dyn. 34(5), 1577–1583 (2011)CrossRefGoogle Scholar
- 14.Schrijver, A.: Theory of Linear and Integer Programming. Wiley, Hoboken (1998)zbMATHGoogle Scholar
- 15.Nikolos, I.K., Valavanis, K.P., Tsourveloudis, N.C., Kostaras, A.N.: Evolutionary algorithm based offline/online path planner for UAV navigation. IEEE Trans. Syst. B Man. Cybern. 33(6), 898–912 (2003)CrossRefGoogle Scholar
- 16.Son, Y.S., Baldick, R.: Hybrid coevolutionary programming for nash equilibrium search in games with local optima. IEEE Trans. Evol. Comput. 8(4), 305–315 (2004)CrossRefGoogle Scholar
- 17.Kim, D.H., Shin, S.: Self-organization of decentralized swarm agents based on modified particle swarm algorithm. J. Intell. Robot. Syst. 46(2), 129–149 (2006)CrossRefGoogle Scholar
- 18.Mauro, P., Conway, B.A.: Particle swarm optimization applied to space trajectories. J. Guid. Control Dyn. 33(5), 1429–1441 (2010)CrossRefGoogle Scholar
- 19.Pinto, A.M., Moreira, A.P., Costa, P.G.: A localization method based on map-matching and particle swarm optimization. J. Intell. Robot. Syst. 77(2), 313–326 (2015)CrossRefGoogle Scholar
- 20.Karaboga, D., Basturk, B.: A powerful and efficient algorithm for numerical function optimization: Artificial bee colony (ABC) algorithm. J. Global. Optim. 39(3), 459–471 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
- 21.Fu, Y., Zhang, Y.M., Yu, X.: An advanced sense and collision avoidance strategy for Unmanned Aerial Vehicles in landing phase. IEEE Aerosp. Electron. Syst. Mag. 31(9), 40–52 (2016)CrossRefGoogle Scholar
- 22.Bhattacharya, P., Gavrilova, M.L.: Roadmap-based path planning-using the Voronoi diagram for a clearance-based shortest path. IEEE Robot. Autom. Mag. 15(2), 58–66 (2008)CrossRefGoogle Scholar
- 23.Pehlivanoglu, Y.V.: A new vibrational generic algorithm enhanced with a Voronoi diagram for path planning of autonomous UAV. Aerosp. Sci. Technol. 16(1), 47–55 (2012)CrossRefGoogle Scholar
- 24.Sridharan, K., Priya, T.K.: The design of a hardware accelerator for real-time complete visibility graph construction and efficient FPGA implementation. IEEE Trans. Ind. Electron. 52(4), 1185–1187 (2005)CrossRefGoogle Scholar
- 25.Kavraki, L.E., Švestka, P., Latombe, J.C.: Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Trans. Robot. Autom. 12, 566–580 (1994)CrossRefGoogle Scholar
- 26.Kavraki, L.E., Svestka, P., Latombe, J.C., Overmars, M.H.: Randomized preprocessing of configuration for fast path planning. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 2138–2146, San Diego (1994)Google Scholar
- 27.Puterman, M.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, Hoboken (2005)zbMATHGoogle Scholar
- 28.Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
- 29.Lian, Z.T., Deshmukh, A.: Performance prediction of an unmanned airborne vehicle multi-agent system. Eur. J. Oper. Res. 172(2), 680–695 (2006)CrossRefzbMATHGoogle Scholar
- 30.Billingsley, T.B., Kochenderfer, M.J., Chryssanthacopoulos, J.P.: Collision avoidance for general aviation. IEEE Aerosp. Electron. Syst. Mag. 27(7), 1–17 (2011)Google Scholar
- 31.Ure, N.K., Chowdhary, G., Chen, Y.F., How, J.P., Vian, J.: Distributed learning for planning under uncertainty problems with heterogeneous teams. J. Intell. Robot. Syst. 74(1–2), 529–544 (2014)CrossRefGoogle Scholar
- 32.Fu, Y., Yu, X., Zhang, Y.M.: Sense and collision avoidance of Unmanned Aerial Vehicles using Markov decision process and flatness approach. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 714–719. Lijiang (2015)Google Scholar
- 33.Miele, A.: Flight Mechanics: Theory of Flight Paths. Courier Dover Publications, New York (2016)Google Scholar
- 34.Bai, H., Hsu, D., Kochenderfer, M.J., et al.: Unmanned aircraft collision avoidance using continuous-state POMDPs. Robot. Auton. Syst. 1, 1–8 (2012)Google Scholar
- 35.Chamseddine, A., Zhang, Y.M., Rabbath, C.A., Theilliol, D.: Trajectory planning and re-planning strategies applied to a quadrotor unmanned aerial vehicle. J. Guid. Control. Dyn. 35(5), 1667–1671 (2012)CrossRefGoogle Scholar
- 36.Yao, P., Wang, H., Su, Z.: Real-time path planning of unmanned aerial vehicle for target tracking and obstacle avoidance in complex dynamic environment. Aerosp. Sci. Technol. 47, 269–279 (2015)CrossRefGoogle Scholar
- 37.Powell, W.B.: Approximate Dynamic Programming. Wiley-Interscience, Hoboken (2008)Google Scholar