Skip to main content
SpringerLink
Log in

Image-Based Visual Servo Tracking Control of a Ground Moving Target for a Fixed-Wing Unmanned Aerial Vehicle

  • Regular Paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper proposes a new control method for the ground moving target tracking problem by a fixed-wing unmanned aerial vehicle (UAV) with a monocular pan-tilt camera. By utilizing the image-based visual servoing, the control law can be directly designed in the image plane, thereby avoiding errors caused by the 3D position calculation. Based on that, we design a control framework to integrate the control of the UAV and the pan-tilt, which enables the UAV to track the target while maintaining the feature point near the image center. Furthermore, considering that the low-cost pan-tilt camera we use has restricted characteristics, we present a deterministic finite automata model to transit the states of tracking when the pan-tilt attitude reaches saturation, thereby improving the tracking ability of the UAV for the moving target. The stability proof of the controller is given, and extensive experiments of hardware-in-the-loop (HIL) simulation and real flights are provided. The results show that the proposed method can achieve continuous robust tracking of the ground moving target.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Availability of Data and Materials

The data used to support the findings of this study are available from the corresponding author upon request.

References

  1. Stöcker, C., Bennett, R., Nex, F., et al.: Review of the current state of UAV regulations[J]. Remote Sens. 9(5), 459 (2017)

    Article  Google Scholar 

  2. Tang, L., Shao, G.: Drone remote sensing for forestry research and practices[J]. J. Forest Res. 26(4), 791–797 (2015)

    Article  Google Scholar 

  3. Colorado, J., Mondragon, I., Rodriguez, J., et al.: Geo-mapping and visual stitching to support landmine detection using a low-cost UAV[j]. Int. J. Adv. Robot. Syst. 12(9), 125 (2015)

    Article  Google Scholar 

  4. Muchiri, N., Kimathi, S.: A review of applications and potential applications of UAV[C] (2016)

  5. Wang, S., Jiang, F., Zhang, B., et al.: Development of UAV-based target tracking and recognition systems[J]. IEEE Transactions on Intelligent Transportation Systems (2019)

  6. Xiang, T., Jiang, F., Lan, G., et al.: UAV Based target tracking and recognition[C]. In: 2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), IEEE, pp. 400–405 (2016)

  7. Zhang, M., Liu, H.H.T.: Cooperative tracking a moving target using multiple fixed-wing UAVs[j]. J. Intell. Robot. Syst. 81(3-4), 505–529 (2016)

    Article  Google Scholar 

  8. Shin, H.S., Garcia, A.J., Alvarez, S.: Information-driven persistent sensing of a non-cooperative mobile target using UAVs[j]. J. Intell. Robot. Syst. 92(3-4), 629–643 (2018)

    Article  Google Scholar 

  9. Hutchinson, S., Hager, G.D., Corke, P.I.: A tutorial on visual servo control[J]. IEEE Trans. Robot. Autom. 12(5), 651–670 (1996)

    Article  Google Scholar 

  10. Chaumette, F., Hutchinson, S.: Visual servo control. I. Basic approaches[J]. IEEE Robot. Autom. Magaz. 13(4), 82–90 (2006)

    Article  Google Scholar 

  11. Chaumette, F., Hutchinson, S.: Visual servo control. II. Advanced approaches [Tutorial][J]. IEEE Robot Autom. Magaz. 14(1), 109–118 (2007)

    Article  Google Scholar 

  12. Saxena, A., Pandya, H., Kumar, G., et al.: Exploring convolutional networks for end-to-end visual servoing[C]. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), IEEE, pp. 3817–3823 (2017)

  13. Bateux, Q., Marchand, E., Leitner, J., et al.: Training deep neural networks for visual servoing[C]. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE, pp. 1–8 (2018)

  14. Guo, Y., Liu, Y., Oerlemans, A., et al.: Deep learning for visual understanding: a review[J]. Neurocomputing 187, 27–48 (2016)

    Article  Google Scholar 

  15. Zhang, D., Qi, H., Wu, X., et al.: The quadrotor dynamic modeling and indoor target tracking control method[J]. Math. Probl. Eng. 2014 (2014)

  16. Popova, M.G., Liu, H.H.: Position-based visual servoing for target tracking by a quadrotor UAV[c] (2016)

  17. Zhao, W., Liu, H., Lewis, F.L., et al.: Robust visual servoing control for ground target tracking of quadrotors[J]. IEEE Transactions on Control Systems Technology (2019)

  18. Kim, J.W., Shim, D.H.: A vision-based target tracking control system of a quadrotor by using a tablet computer[C]. In: 2013 International Conference on Unmanned Aircraft Systems (ICUAS), IEEE, pp 1165–1172 (2013)

  19. Serra, P., Cunha, R., Hamel, T., et al.: Landing of a quadrotor on a moving target using dynamic image-based visual servo control[J]. IEEE Trans. Robot. 32(6), 1524–1535 (2016)

    Article  Google Scholar 

  20. Zheng, D., Wang, H., Chen, W.: Image-based visual tracking of a moving target for a quadrotor[C]. In: 2017 11th Asian Control Conference (ASCC), IEEE, pp 198–203 (2017)

  21. Kassab, M.A., Maher, A., Elkazzaz, F., et al.: UAV Target tracking by detection via deep neural networks[C]. In: 2019 IEEE International Conference on Multimedia and Expo (ICME), IEEE, pp. 139–144 (2019)

  22. Hossain, S., Lee, D.: Deep learning-based real-time multiple-object detection and tracking from aerial imagery via a flying robot with GPU-based embedded devices[J]. Sensors 19(15), 3371 (2019)

    Article  Google Scholar 

  23. Arreola, L., Gudiño, G., Flores, G.: Object recognition and tracking using Haar-like Features Cascade Classifiers: Application to a quad-rotor UAV[J]. arXiv:1903.03947 (2019)

  24. Li, Z., Hovakimyan, N., Dobrokhodov, V., et al.: Vision-based target tracking and motion estimation using a small UAV[c]. In: 49th IEEE Conference on Decision and Control (CDC), IEEE, pp. 2505–2510 (2010)

  25. Choi, H., Kim, Y.: UAV Guidance using a monocular-vision sensor for aerial target tracking[J]. Control. Eng. Pract. 22, 10–19 (2014)

    Article  Google Scholar 

  26. Wang, X., Zhu, H., Zhang, D., et al.: Vision-based detection and tracking of a mobile ground target using a fixed-wing UAV[j]. Int. J. Adv. Robot. Syst. 11(9), 156 (2014)

    Article  Google Scholar 

  27. Le Bras, F., Hamel, T., Mahony, R.: Image-based visual servo control for circular trajectories for a fixed-wing aircraft[C]. In: Proceedings of the 48h IEEE Conference on decision and control (CDC) held jointly with 2009 28th Chinese Control Conference, IEEE, 3430–3435 (2009)

  28. Peliti, P., Rosa, L., Oriolo, G., et al.: Vision-based loitering over a target for a fixed-wing UAV[j]. IFAC Proc. Vol. 45(22), 51–57 (2012)

    Article  Google Scholar 

  29. Yang, L., Liu, Z., Wang, X., et al.: An optimized image-based visual servo control for fixed-wing unmanned aerial vehicle target tracking with fixed camera[J]. IEEE Access 7, 68455–68468 (2019)

    Article  Google Scholar 

  30. Yang, L., Liu, Z., Wang, G., et al.: Image-based visual servo control for ground target tracking using Fixed-Wing UAV with Pan-Tilt Camera[C] (2020)

  31. Redmon, J., Farhadi, A.: Yolov3: An incremental improvement[J]. arXiv:1804.02767(2018)

  32. Tian, Y., Yang, G., Wang, Z., et al.: Apple detection during different growth stages in orchards using the improved YOLO-v3 model[J]. Comput Electron Agricult 157, 417–426 (2019)

    Article  Google Scholar 

  33. Choi, J, Chun, D, Kim, H, et al.: Gaussian yolov3: An accurate and fast object detector using localization uncertainty for autonomous driving[C]. In: Proceedings of the IEEE International Conference on Computer Vision, pp 502–511 (2019)

  34. Wang, J., Wang, N., Li, L., et al.: Real-time behavior detection and judgment of egg breeders based on YOLO v3[J]. Neural Comput. Applic. 32(10), 5471–5481 (2020)

    Article  MathSciNet  Google Scholar 

  35. Techy, L., Woolsey, C.A.: Minimum-time path planning for unmanned aerial vehicles in steady uniform winds[J]. J. Guid. Control Dynam. 32(6), 1736–1746 (2009)

    Article  Google Scholar 

  36. Sujit, P.B., Saripalli, S., Sousa, J.B.: Unmanned aerial vehicle path following: A survey and analysis of algorithms for fixed-wing unmanned aerial vehicless[J]. IEEE Control. Syst. Mag. 34(1), 42–59 (2014)

    Article  MathSciNet  Google Scholar 

  37. Zhao, S., Wang, X., Zhang, D., et al.: Curved path following control for fixed-wing unmanned aerial vehicles with control constraint[J]. J. Intell. Robot. Syst. 89(1-2), 107–119 (2018)

    Article  Google Scholar 

  38. Khalil, H.K.: Nonlinear control[M] Pearson Higher Ed (2014)

  39. Wang, X., Shen, L., Liu, Z., et al.: Coordinated flight control of miniature fixed-wing UAV swarms: methods and experiments[J]. Sci. China Inform. Sci. 62(11), 212204 (2019)

    Article  MathSciNet  Google Scholar 

  40. Liu, Z., Wang, X., Shen, L., et al.: Mission-Oriented Miniature Fixed-Wing UAV Swarms: a Multilayered and Distributed Architecture[J]. IEEE Transactions on Systems, Man, and Cybernetics, Systems (2020)

  41. Liu, H., Wang, X., Zhong, Y.: Robust position control of a lab helicopter under wind disturbances[J]. IET Control Theory Appl. 8(15), 1555–1565 (2014)

    Article  Google Scholar 

  42. Jin, M., Kang, S.H., Chang, P.H., et al.: Robust control of robot manipulators using inclusive and enhanced time delay control[J]. IEEE/ASME Trans. Mechatron. 22(5), 2141–2152 (2017)

    Article  Google Scholar 

  43. Shao, X., Liu, N., Wang, Z., et al.: Neuroadaptive integral robust control of visual quadrotor for tracking a moving object[J]. Mech. Syst. Signal Process. 136, 106513 (2020)

    Article  Google Scholar 

Download references

Funding

This work is funded by the National Natural Science Foundation of China (61906209) and (61973309).

Author information

Authors and Affiliations

  1. College of Intelligence Science and Technology, National University of Defense Technology, Changsha, 410073, China

    Lingjie Yang, Zhihong Liu, Xiangke Wang, Xianguo Yu, Guanzheng Wang & Lincheng Shen

Authors
  1. Lingjie Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhihong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangke Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xianguo Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guanzheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lincheng Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

L.Y., Z.L. and X.W. conceived the idea. L.Y. collected the data, conducted analyses and wrote the manuscript. Z.L. and X.W. polished the manuscript. X.Y. and G.W. assisted conducting the experiments. All authors commented on, discussed, and edited the manuscript.

Corresponding author

Correspondence to Zhihong Liu.

Ethics declarations

Consent to Participate

Informed consent was obtained from all individual participants included in the study.

Consent to Publish

All the participants have consented to the submission of the data to the journal.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MP4 17.2 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, L., Liu, Z., Wang, X. et al. Image-Based Visual Servo Tracking Control of a Ground Moving Target for a Fixed-Wing Unmanned Aerial Vehicle. J Intell Robot Syst 102, 81 (2021). https://doi.org/10.1007/s10846-021-01425-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-021-01425-y

Keywords

Search

Navigation