Skip to main content

A Deep Reinforcement Learning Strategy for UAV Path Following Control Under Sensor Fault

  • Conference paper
  • First Online:
Advances in Guidance, Navigation and Control

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 644))

  • 210 Accesses

Abstract

Unmanned Aerial Vehicles (UAVs) have been extensively used in civil and industrial applications due to the rapid development of the involved technologies. Especially, using deep reinforcement learning methods for motion control acquires a major progress recently since deep Q-learning has successfully applied to the continuous action domain problem. This paper proposes a new Deep Deterministic Policy Gradient (DDPG) algorithm for path following control problem of UAV with sensor faults. Firstly, the model of UAV path following problem has been established. After that, the DDPG framework is constructed. Then, the proposed DDPG algorithm is formulated to the path following problem. Finally, simulation results are carried out to show the efficiency and effectiveness of the proposed methodology.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 429.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 549.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 549.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Liu, Z.X., Yuan, C., Zhang, Y.M.: Active fault-tolerant control of unmanned quadrotor helicopter using linear parameter varying technique. J. Intell. Robot. Syst. (2017)

    Google Scholar 

  2. Qu, Y.H., Zhang, Y.T., Zhang, Y.M.: Optimal flight path planning for UAVs in 3-D threat environment. In: International Conference on Unmanned Aircraft Systems (ICUAS), Orlando, USA (2014)

    Google Scholar 

  3. Liu, Z.X., Zhang, Y.M., Yu, X., Yuan, C.: Unmanned surface vehicles: an overview of developments and challenges. Annu. Rev. Control (2016)

    Google Scholar 

  4. Qu, Y.H., Zhang, Y.T., Zhang, Y.M.: A UAV solution of regional surveillance based on pheromones and artificial potential field theory. In: International Conference on Unmanned Aircraft Systems (ICUAS), Denver, USA (2015)

    Google Scholar 

  5. Li, P., Yu, X., Peng, X.Y., Zheng, Z.Q., Zhang, Y.M.: Fault-tolerant cooperative control for multiple UAVs based on sliding mode techniques. Sci. China Inf. Sci. (2017)

    Google Scholar 

  6. Zhang, Y.T., Zhang, Y.M., Liu, Z.X., Yu, Z.Q., Qu, Y.H.: Line-of-sight path following control on UAV with sideslip estimation and compensation. In: 37th Chinese Control Conference (CCC), Wuhan, China (2018)

    Google Scholar 

  7. Fossen, T.I., Breivik, M., Skjetne, R.: Line-of-sight path following of underactuated marine craft. In: 6th IFAC Conference on Manoeuvring and Control of Marine Craft, Girona, Spain (2003)

    Google Scholar 

  8. Breivik, M.: Topics in guided motion control of marine vehicles. In: Tapir Uttrykk (2010)

    Google Scholar 

  9. Rodriguez-Ramos, A., Sampedro, C., Bavle, H., De La Puente, P., Campoy, P.: A deep reinforcement learning strategy for UAV autonomous landing on a moving platform. J. Intell. Robot. Syst. (2019)

    Google Scholar 

  10. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G, Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. Nature (2015)

    Google Scholar 

  11. Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., Riedmiller, M.: Deterministic policy gradient algorithms (2014)

    Google Scholar 

  12. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. ArXiv preprint (2015)

    Google Scholar 

  13. Rysdyk, R.: Unmanned aerial vehicle path following for target observation in wind. J. Guid. Control Dyn. (2006)

    Google Scholar 

  14. Qie, H., Shi, D.X., Shen, T.L., Xu, X.H., Li, Y., Wang, L.J.: Joint optimization of multi-UAV target assignment and path planning based on multi-agent reinforcement learning. IEEE Access (2019)

    Google Scholar 

  15. Wawrzynski, P.: Control policy with autocorrelated noise in reinforcement learning for robotics. Int. J. Mach. Learn. Comput. (2015)

    Google Scholar 

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China (No. 61833013 and 61573282) and Natural Sciences and Engineering Research Council of Canada.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Youmin Zhang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, Y., Zhang, Y., Yu, Z. (2022). A Deep Reinforcement Learning Strategy for UAV Path Following Control Under Sensor Fault. In: Yan, L., Duan, H., Yu, X. (eds) Advances in Guidance, Navigation and Control . Lecture Notes in Electrical Engineering, vol 644. Springer, Singapore. https://doi.org/10.1007/978-981-15-8155-7_432

Download citation

Publish with us

Policies and ethics