Skip to main content
SpringerLink
Log in

Fisher pruning for developing real-time UAV trackers

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Unmanned aerial vehicle (UAV)-based tracking has shown large potential in various domains such as transportation, logistics, public safety, and more. However, deploying deep learning (DL)-based tracking algorithms on UAVs is challenging because of limitations in computing resources, battery capacity, and maximum load. Discriminative correlation filter (DCF)-based trackers have become a popular choice in the UAV tracking community owing to their ability to provide superior efficiency while consuming fewer resources. However, the limited representation learning ability of DCF-based trackers leads to lower precision in complex scenarios compared to DL-based methods. Filter pruning is a prevalent practice for deploying deep neural networks on edge devices with constrained resources, and it may be an effective way to solve problems encountered when deploying deep learning trackers on UAVs. However, the application of filter pruning to UAV tracking is underexplored, and a straightforward and useful pruning standard is desirable. This paper proposes using Fisher pruning to reduce the SiamFC++ model for UAV tracking, resulting in the F-SiamFC++ tracker. The proposed tracker achieves a remarkable balance between precision and efficiency, as demonstrated through exhaustive experiments on four popular UAV benchmarks: UAVDT, DTB70, UAV123@10fps, and Vistrone2018, showing state-of-the-art performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Fu, C., Li, B., Ding, F., Lin, F., Lu, G.: Correlation filters for unmanned aerial vehicle-based aerial tracking: a review and experimental evaluation. pp. 2–387 (2021)

  2. Cao, Z., Fu, C., Ye, J., Li, B., Li, Y.: Hift: hierarchical feature transformer for aerial tracking. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 15457–15466 (2021)

  3. Li, Y., Fu, C., Ding, F., Huang, Z., Lu, G.: Autotrack: towards high-performance visual tracking for uav with automatic spatio-temporal regularization. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11923–11932 (2020)

  4. Huang, Z., Fu, C., Li, Y., Lin, F., Lu, P.: Learning aberrance repressed correlation filters for real-time uav tracking. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2891–2900 (2019)

  5. Li, S., Liu, Y., Zhao, Q., Feng, Z.: Learning residue-aware correlation filters and refining scale estimates with the grabcut for real-time uav tracking. In: 2021 International Conference on 3D Vision (3DV), pp. 1238–1248 (2021)

  6. Li, S., Liu, Y., Zhao, Q., Feng, Z.: Learning residue-aware correlation filters and refining scale for real-time uav tracking. Pattern Recogn. 127, 108614 (2022)

    Article  Google Scholar 

  7. Du, D., Qi, Y., Yu, H., Yang, Y., Duan, K., Li, G., Zhang, W., Huang, Q., Tian, Q.: The unmanned aerial vehicle benchmark: object detection and tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 375–391 (2018)

  8. Choudhary, T., Mishra, V., Goswami, A., Sarangapani, J.: A comprehensive survey on model compression and acceleration. Artif. Intell. Rev. 53(7), 5113–5155 (2020)

    Article  Google Scholar 

  9. Wang, H., Qin, C., Zhang, Y., Fu, Y.: Emerging paradigms of neural network pruning, arXiv preprint arXiv:2103.06460 (2021)

  10. Theis, L., Korshunova, I., Tejani, A., Huszár, F.: Faster gaze prediction with dense networks and fisher pruning, arXiv preprint arXiv:1801.05787 (2018)

  11. Xu, Y., Wang, Z., Li, Z., Yuan, Y., Yu, G.: Siamfc++: towards robust and accurate visual tracking with target estimation guidelines. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-20), vol. 34, pp. 12549–12556 (2020)

  12. Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P. H.: Fully-convolutional siamese networks for object tracking. In: European conference on computer vision (ECCV), pp. 850–865. Springer(2016)

  13. Wu, W., Zhong, P., Li, S.: Fisher pruning for real-time uav tracking. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–7 (2022)

  14. Bolme, D.S., Beveridge, J.R., Draper, B.A., Lui, Y.M.: Visual object tracking using adaptive correlation filters. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2544–2550 (2010)

  15. Chicco, D.: Siamese neural networks: an overview. Artif. Neural Networks 73–94 (2021)

  16. Zhang, S., Chi, C., Yao, Y., Lei, Z., Li, S.Z.: Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9756–9765 (2020)

  17. Li, B., Yan, J., Wu, W., Zhu, Z., Hu, X.: High performance visual tracking with siamese region proposal network. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8971–8980 (2018)

  18. Zhu, Z., Wang, Q., Li, B., Wu, W., Yan, J., Hu, W.: Distractor-aware siamese networks for visual object tracking. In: European Conference on Computer Vision (ECCV), pp. 101–117 (2018)

  19. Wang, Q., Zhang, L., Bertinetto, L., Hu, W., Torr, P.H.: Fast online object tracking and segmentation: a unifying approach. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1328–1338 (2019)

  20. Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., Yan, J.: Siamrpn++: evolution of siamese visual tracking with very deep networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4282–4291 (2019)

  21. Zhang, Z., Peng, H.: Deeper and wider siamese networks for real-time visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4591–4600 (2019)

  22. Guo, D., Wang, J., Cui, Y., Wang, Z., Chen, S.: Siamcar: siamese fully convolutional classification and regression for visual tracking. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6268–6276 (2019)

  23. Chen, Z., Zhong, B., Li, G., Zhang, S., Ji, R.: Siamese box adaptive network for visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6668–6677 (2020)

  24. Danelljan, M., Bhat, G., Khan, F.S., Felsberg, M.: Atom: accurate tracking by overlap maximization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4660–4669 (2019)

  25. Bhat, G., Danelljan, M., Gool, L.V., Timofte, R.: Learning discriminative model prediction for tracking. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6182–6191 (2019)

  26. Bhat, G., Danelljan, M., Van Gool, L., Timofte, R.: Know your surroundings: exploiting scene information for object tracking. In: European Conference on Computer Vision (ECCV), pp. 205–221. Springer (2020)

  27. Mayer, C., Danelljan, M., Paudel, D.P., Van Gool, L.: Learning target candidate association to keep track of what not to track. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 13444–13454 (2021)

  28. Blalock, D., Ortiz, J.J.G., Frankle, J., Guttag, J.: What is the state of neural network pruning? arXiv preprint arXiv:2003.03033 (2020)

  29. Lin, M., Ji, R., Wang, Y., Zhang, Y., Zhang, B., Tian, Y., Shao, L.: Hrank: filter pruning using high-rank feature map. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1529–1538 (2020)

  30. Wang, H., Qin, C., Zhang, Y., Fu, Y.: Neural pruning via growing regularization, arXiv preprint arXiv:2012.09243 (2020)

  31. Liu, L., Zhang, S., Kuang, Z., Zhou, A., Xue, J.-H., Wang, X., Chen, Y., Yang, W.,Liao, Q., Zhang, W.: Group fisher pruning for practical network compression. In: International Conference on Machine Learning (ICML), PMLR, pp. 7021–7032 (2021)

  32. Zegers, P.: Fisher information properties. Entropy 17(7), 4918–4939 (2015)

    Article  MathSciNet  Google Scholar 

  33. Li, S., Yeung, D.Y.: Visual object tracking for unmanned aerial vehicles: A benchmark and new motion models. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI-17), pp. 4140–4146 (2017)

  34. Mueller, M., Smith, N., Ghanem, B.: A benchmark and simulator for uav tracking. Far East J. Math. Sci. 2(2), 445–461 (2016)

    Google Scholar 

  35. Wen, L., Zhu, P., Du, D., et al. Visdrone-sot2018: the vision meets drone single-object tracking challenge results. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pp. 469–495 (2018)

  36. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 37(3), 583–596 (2015)

    Article  Google Scholar 

  37. Danelljan, M., Hager, G., Khan, F.S., Felsberg, M.: Adaptive decontamination of the training set: a unified formulation for discriminative visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1430–1438 (2016)

  38. Mueller, M., Smith, N., Ghanem, B.: Context-aware correlation filter tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1387–1395 (2017)

  39. Galoogahi, H.K., Fagg, A., Lucey, S.: Learning background-aware correlation filters for visual tracking. In: IEEE International Conference on Computer Vision (ICCV), pp. 1144–1152 (2017)

  40. Danelljan, M., Bhat, G., Khan, F.S., Felsberg, M.: Eco: efficient convolution operators for tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6931–6939 (2017)

  41. Wang, N., Zhou, W., Tian, Q., Hong, R., Wang, M., Li, H.: Multi-cue correlation filters for robust visual tracking. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4844–4853 (2018)

  42. Li, F., Tian, C., Zuo, W., Zhang, L., Yang, M.-H.: Learning spatial-temporal regularized correlation filters for visual tracking. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4904–4913 (2018)

  43. Voigtlaender, P., Luiten, J., Torr, P.H., Leibe, B.: Siam r-cnn: visual tracking by re-detection. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6578–6588 (2020)

  44. Lukezic, A., Matas, J., Kristan, M.: D3s—a discriminative single shot segmentation tracker. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7133–7142 (2020)

  45. Danelljan, M., Gool, L.V., Timofte, R.: Probabilistic regression for visual tracking. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7183–7192 (2020)

  46. Guo, D., Shao, Y., Cui, Y., Wang, Z., Zhang, L., Shen, C.: Graph attention tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9543–9552 (2021)

  47. Yan, B., Peng, H., Wu, K., Wang, D., Fu, J., Lu, H.: Lighttrack: finding lightweight neural networks for object tracking via one-shot architecture search. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15180–15189 (2021)

  48. Chen, X., Yan, B., Zhu, J., Wang, D., Yang, X., Lu, H.: Transformer tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8126–8135 (2021)

  49. Zhou, Z., Pei, W., Li, X., Wang, H., Zheng, F., He, Z.: Saliency-associated object tracking. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9866–9875 (2021)

  50. Zhang, Z., Liu, Y., Wang, X., Li, B., Hu, W.: Learn to match: automatic matching network design for visual tracking. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 13339–13348 (2021)

  51. Wang, X., Zeng, D., Zhao, Q., Li, S.: Rank-based filter pruning for real-time uav tracking. In: IEEE International Conference on Multimedia and Expo (ICME), pp. 01–06 (2022)

Download references

Acknowledgements

Thanks to the support by Guangxi Key Laboratory of Embedded Technology and Intelligent System, the Guangxi Science and Technology Base and Talent Special Project (No. 2021AC9330), the National Natural Science Foundation of China (No. 62176170, 62066042, 61971005), and Sichuan Province Key Research and Development Project (No. 2020YJ0282).

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Guilin University of Technology, Guilin, 541000, China

    Pengzhi Zhong, Wanying Wu & Shuiwang Li

  2. College of Computer Science, Sichuan University, Chengdu, 30332, China

    Xiaowei Dai & Qijun Zhao

Authors
  1. Pengzhi Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wanying Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaowei Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qijun Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shuiwang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

PZ, WW and XD designed the research. SL and QZ guided the research. PZ, WW and XD drafted the manuscript. SL revised and finalized the paper.

Corresponding author

Correspondence to Shuiwang Li.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhong, P., Wu, W., Dai, X. et al. Fisher pruning for developing real-time UAV trackers. J Real-Time Image Proc 20, 91 (2023). https://doi.org/10.1007/s11554-023-01348-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-023-01348-x

Keywords

Search

Navigation