Advertisement

Journal of Intelligent & Robotic Systems

, Volume 93, Issue 1–2, pp 17–32 | Cite as

Collision-Free Trajectory Generation and Tracking for UAVs Using Markov Decision Process in a Cluttered Environment

  • Xiang Yu
  • Xiaobin Zhou
  • Youmin ZhangEmail author
Article
First Online:
  • 186 Downloads

Abstract

A collision-free trajectory generation and tracking method capable of re-planning unmanned aerial vehicle (UAV) trajectories can increase flight safety and decrease the possibility of mission failures. In this paper, a Markov decision process (MDP) based algorithm combined with backtracking method is presented to create a safe trajectory in the case of hostile environments. Subsequently, a differential flatness method is adopted to smooth the profile of the rerouted trajectory for satisfying the UAV physical constraints. Lastly, a flight controller based on passivity-based control (PBC) is designed to maintain UAV’s stability and trajectory tracking performance. Simulation results demonstrate that the UAV with the proposed strategy is capable of avoiding obstacles in a hostile environment.

Keywords

Collision-free Differential flatness Markov decision process (MDP) Passivity-based control (PBC) Unmanned aerial vehicle (UAV) 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgments

This work was supported in part by the Natural Sciences and Engineering Research Council of Canada, in part by the National Natural Science Foundation of China under Grant 61573282 and Grant 61603130. The authors would like to express their sincere gratitude to the Editor-in-Chief, the Guest Editors, and the anonymous reviewers whose insightful comments have helped to improve the quality of this paper considerably.

References

  1. 1.
    Gundlach, J.: Designing Unmanned Aircraft Systems: A Comprehensive Approach. American Institute of Aeronautics and Astronautics, Reston (2012)CrossRefGoogle Scholar
  2. 2.
    Halit, E., Kemal, L.: 3D path planning for multiple UAVs for maximum information collection. J. Intell. Robot. Syst. 73(1–4), 737–762 (2014)Google Scholar
  3. 3.
    Angelov, P.: Sense and Avoid in UAS: Research and Applications. Wiley, Hoboken (2012)CrossRefGoogle Scholar
  4. 4.
    Yang, K., Gan, S.K., Sukkarieh, S.: An efficient path planning and control algorithm for RUAV’s in unknown and cluttered environment. J. Intell. Robot. Syst. 57, 101–122 (2010)CrossRefzbMATHGoogle Scholar
  5. 5.
    Yu, X., Zhang, Y.M.: Sense and avoid technologies with applications to unmanned aircraft systems: Review and prospects. Prog. Aerosp. Sci. 74, 152–166 (2015)CrossRefGoogle Scholar
  6. 6.
    Gui, Y., Guo, P., Zhang, H., Lei, Z., Du, X., Du, J., Yu, Q.: Airborne vision-based navigation method for UAV accuracy landing using infrared lamps. J. Intell. Robot. Syst. 72(2), 197–218 (2013)CrossRefGoogle Scholar
  7. 7.
    Kuchar, J.K., Yang, L.C.: A review of conflict detection and resolution modeling methods. IEEE Trans. Intell. Trans. Syst. 1(4), 179–189 (2000)CrossRefGoogle Scholar
  8. 8.
    Akmeliawati, R., Mareels, I.M.Y.: Nonlinear energy-based control method for aircraft automatic landing systems. IEEE Trans. Control Syst. Technol. 18(4), 871–884 (2010)CrossRefGoogle Scholar
  9. 9.
    Khatib, O.: Real time obstacle avoidance for manipulators and bile Robots. Int. J. Rob. Res. 5(1), 90–99 (1986)CrossRefGoogle Scholar
  10. 10.
    Lavalle, S.: Planning Algorithms. Cambridge University Press, Cambridge (2006)CrossRefzbMATHGoogle Scholar
  11. 11.
    Chuang, J.H., Ahuja, N.: An analytically tractable potential field model of free space and its application in obstacle avoidance. IEEE Trans. Syst. Man. Cybern. B. Cybern. 28, 729–736 (1998)CrossRefGoogle Scholar
  12. 12.
    Geiger, B., Horn, J., Delullo, A., Niessner, A., Long, L.: Optimal path planning of UAV using direct collocation with nonlinear programming. In: Proceedings of the AIAA Guidance, Navigation, and Control Conference, Keystone, Colorado (2006)Google Scholar
  13. 13.
    Sridhar, B., Ng, H.K., Chen, N.Y.: Aircraft trajectory optimization and contrails avoidance in the presence of winds. J. Guid. Control. Dyn. 34(5), 1577–1583 (2011)CrossRefGoogle Scholar
  14. 14.
    Schrijver, A.: Theory of Linear and Integer Programming. Wiley, Hoboken (1998)zbMATHGoogle Scholar
  15. 15.
    Nikolos, I.K., Valavanis, K.P., Tsourveloudis, N.C., Kostaras, A.N.: Evolutionary algorithm based offline/online path planner for UAV navigation. IEEE Trans. Syst. B Man. Cybern. 33(6), 898–912 (2003)CrossRefGoogle Scholar
  16. 16.
    Son, Y.S., Baldick, R.: Hybrid coevolutionary programming for nash equilibrium search in games with local optima. IEEE Trans. Evol. Comput. 8(4), 305–315 (2004)CrossRefGoogle Scholar
  17. 17.
    Kim, D.H., Shin, S.: Self-organization of decentralized swarm agents based on modified particle swarm algorithm. J. Intell. Robot. Syst. 46(2), 129–149 (2006)CrossRefGoogle Scholar
  18. 18.
    Mauro, P., Conway, B.A.: Particle swarm optimization applied to space trajectories. J. Guid. Control Dyn. 33(5), 1429–1441 (2010)CrossRefGoogle Scholar
  19. 19.
    Pinto, A.M., Moreira, A.P., Costa, P.G.: A localization method based on map-matching and particle swarm optimization. J. Intell. Robot. Syst. 77(2), 313–326 (2015)CrossRefGoogle Scholar
  20. 20.
    Karaboga, D., Basturk, B.: A powerful and efficient algorithm for numerical function optimization: Artificial bee colony (ABC) algorithm. J. Global. Optim. 39(3), 459–471 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Fu, Y., Zhang, Y.M., Yu, X.: An advanced sense and collision avoidance strategy for Unmanned Aerial Vehicles in landing phase. IEEE Aerosp. Electron. Syst. Mag. 31(9), 40–52 (2016)CrossRefGoogle Scholar
  22. 22.
    Bhattacharya, P., Gavrilova, M.L.: Roadmap-based path planning-using the Voronoi diagram for a clearance-based shortest path. IEEE Robot. Autom. Mag. 15(2), 58–66 (2008)CrossRefGoogle Scholar
  23. 23.
    Pehlivanoglu, Y.V.: A new vibrational generic algorithm enhanced with a Voronoi diagram for path planning of autonomous UAV. Aerosp. Sci. Technol. 16(1), 47–55 (2012)CrossRefGoogle Scholar
  24. 24.
    Sridharan, K., Priya, T.K.: The design of a hardware accelerator for real-time complete visibility graph construction and efficient FPGA implementation. IEEE Trans. Ind. Electron. 52(4), 1185–1187 (2005)CrossRefGoogle Scholar
  25. 25.
    Kavraki, L.E., Švestka, P., Latombe, J.C.: Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Trans. Robot. Autom. 12, 566–580 (1994)CrossRefGoogle Scholar
  26. 26.
    Kavraki, L.E., Svestka, P., Latombe, J.C., Overmars, M.H.: Randomized preprocessing of configuration for fast path planning. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 2138–2146, San Diego (1994)Google Scholar
  27. 27.
    Puterman, M.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, Hoboken (2005)zbMATHGoogle Scholar
  28. 28.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  29. 29.
    Lian, Z.T., Deshmukh, A.: Performance prediction of an unmanned airborne vehicle multi-agent system. Eur. J. Oper. Res. 172(2), 680–695 (2006)CrossRefzbMATHGoogle Scholar
  30. 30.
    Billingsley, T.B., Kochenderfer, M.J., Chryssanthacopoulos, J.P.: Collision avoidance for general aviation. IEEE Aerosp. Electron. Syst. Mag. 27(7), 1–17 (2011)Google Scholar
  31. 31.
    Ure, N.K., Chowdhary, G., Chen, Y.F., How, J.P., Vian, J.: Distributed learning for planning under uncertainty problems with heterogeneous teams. J. Intell. Robot. Syst. 74(1–2), 529–544 (2014)CrossRefGoogle Scholar
  32. 32.
    Fu, Y., Yu, X., Zhang, Y.M.: Sense and collision avoidance of Unmanned Aerial Vehicles using Markov decision process and flatness approach. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp 714–719. Lijiang (2015)Google Scholar
  33. 33.
    Miele, A.: Flight Mechanics: Theory of Flight Paths. Courier Dover Publications, New York (2016)Google Scholar
  34. 34.
    Bai, H., Hsu, D., Kochenderfer, M.J., et al.: Unmanned aircraft collision avoidance using continuous-state POMDPs. Robot. Auton. Syst. 1, 1–8 (2012)Google Scholar
  35. 35.
    Chamseddine, A., Zhang, Y.M., Rabbath, C.A., Theilliol, D.: Trajectory planning and re-planning strategies applied to a quadrotor unmanned aerial vehicle. J. Guid. Control. Dyn. 35(5), 1667–1671 (2012)CrossRefGoogle Scholar
  36. 36.
    Yao, P., Wang, H., Su, Z.: Real-time path planning of unmanned aerial vehicle for target tracking and obstacle avoidance in complex dynamic environment. Aerosp. Sci. Technol. 47, 269–279 (2015)CrossRefGoogle Scholar
  37. 37.
    Powell, W.B.: Approximate Dynamic Programming. Wiley-Interscience, Hoboken (2008)Google Scholar

Copyright information

© Springer Nature B.V. 2018

Authors and Affiliations

  1. 1.Department of Mechanical, Industrial and Aerospace EngineeringConcordia UniversityMontrealCanada
  2. 2.College of Mechanical and Vehicle EngineeringHunan UniversityChangshaChina

Personalised recommendations

Cite article

Buy options