Skip to main content

Accelerated duality-aware correlation filters for visual tracking

Neural Computing and Applications volume 34, pages 6241–6256 (2022)Cite this article

Abstract

Correlation filters (CF) based tracking methods have attracted considerable attentions for their competitive performance. However, the inherent issues of boundary effect and filter degradation, as well as the scale variation, degrade the tracking accuracy. In addition, the frame-by-frame updating strategy limits the tracking speed, especially in those deep features-based CF trackers. To address these issues, we propose a novel tracker, namely Accelerated Duality-aware Correlation Filters (ADCF), in this paper. In the proposed tracker, dual correlation filters, i.e., translation filter and scale filter, are designed for target localization and scale estimation, respectively. A spatio-temporal regularization term is employed to suppress the boundary effect and filter degradation. Moreover, a model updating strategy named Sparse learning-based Average Peak-to-Correlation Energy (S-APCE) is proposed to accelerate the tracking speed. Finally, an Alternating Direction Method of Multipliers (ADMM) formulation is developed to optimize the ADCF efficiently. Extensive experimental results over six tracking benchmarks prove that the proposed tracker outperforms the state-of-the-art (SOTA) trackers in tracking accuracy and speed.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. https://www.vlfeat.org/matconvnet/.

References

  1. Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr PHS (2016) Staple: complementary learners for real-time tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1401–1409, 10.1109/CVPR.2016.156

  2. Bhat G, Johnander J, Danelljan M, Khan FS, Felsberg M (2018) Unveiling the power of deep tracking. In: Proceedings of the european conference on computer vision, pp 493–509, https://doi.org/10.1007/978-3-030-01216-8_30

  3. Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2544–2550, https://doi.org/10.1109/CVPR.2010.5539960

  4. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, vol 3. Now Publishers Inc, MA

    MATH  Google Scholar 

  5. Dai K, Wang D, Lu H, Sun C, Li J (2019) Visual tracking via adaptive spatially-regularized correlation filters. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4665–4674, https://doi.org/10.1109/CVPR.2019.00480

  6. Danelljan M, Häger G, Khan F, Felsberg M (2014) Accurate scale estimation for robust visual tracking. In: Proceedings of the British machine vision conference, https://doi.org/10.5244/C.28.65

  7. Danelljan M, Khan FS, Felsberg M, v d Weijer J (2014) Adaptive color attributes for real-time visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1090–1097, https://doi.org/10.1109/CVPR.2014.143

  8. Danelljan M, Häger G, Khan FS, Felsberg M (2015a) Convolutional features for correlation filter based visual tracking. In: Proceedings of the international conference on computer vision workshops, pp 621–629, https://doi.org/10.1109/ICCVW.2015.84

  9. Danelljan M, Häger G, Khan FS, Felsberg M (2015b) Learning spatially regularized correlation filters for visual tracking. In: Proceedings of the international conference on computer vision, pp 4310–4318, https://doi.org/10.1109/ICCV.2015.490

  10. Danelljan M, Robinson A, Shahbaz Khan F, Felsberg M (2016) Beyond correlation filters: Learning continuous convolution operators for visual tracking. In: Proceedings of the european conference on computer Vision, pp 472–488, https://doi.org/10.1007/978-3-319-46454-1_29

  11. Danelljan M, Bhat G, Khan FS, Felsberg M (2017a) Eco: efficient convolution operators for tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6931–6939, https://doi.org/10.1109/CVPR.2017.733

  12. Danelljan M, Häger G, Khan FS, Felsberg M (2017) Discriminative scale space tracking. IEEE Trans Pattern Anal Mach Intell 39(8):1561–1575. https://doi.org/10.1109/TPAMI.2016.2609928

    Article  Google Scholar 

  13. Fiaz M, Mahmood A, Javed S, Jung SK (2019) Handcrafted and deep trackers: Recent visual object tracking approaches and trends. ACM Comput Surv. https://doi.org/10.1145/3309665

    Article  Google Scholar 

  14. Galoogahi HK, Sim T, Lucey S (2013) Multi-channel correlation filters. In: Proceedings of the international conference on computer vision, pp 3072–3079, https://doi.org/10.1109/ICCV.2013.381

  15. Galoogahi HK, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. In: Proceedings of the international conference on computer vision, pp 1144–1152, https://doi.org/10.1109/ICCV.2017.129

  16. He Z, Fan Y, Zhuang J, Dong Y, Bai H (2017) Correlation filters with weighted convolution responses. In: Proceedings of the international conference on computer vision workshops, pp 1992–2000, https://doi.org/10.1109/ICCVW.2017.233

  17. Henriques JF, Caseiro R, Martins P, Batista J (2012) Exploiting the circulant structure of tracking-by-detection with kernels. In: Proceedings of the european conference on computer vision, pp 702–715, https://doi.org/10.1007/978-3-642-33765-9_50

  18. Huang Z, Fu C, Li Y, Lin F, Lu P (2019) Learning aberrance repressed correlation filters for real-time uav tracking. In: Proceedings of the international conference on computer vision, pp 2891–2900, https://doi.org/10.1109/ICCV.2019.00298

  19. Kristan M, Leonardis A, Matas Jea (2016) The visual object tracking vot2016 challenge results. In: Proceedings of the european conference on computer vision workshops, pp 777–823, https://doi.org/10.1007/978-3-319-46448-0_27

  20. Li F, Tian C, Zuo W, Zhang L, Yang M (2018) Learning spatial-temporal regularized correlation filters for visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4904–4913, 10.1109/CVPR.2018.00515

  21. Li G, Peng M, Nai K, Li Z, Li K (2020) Multi-view correlation tracking with adaptive memory-improved update model. Neural Comput Appl 32(13):9047–9063. https://doi.org/10.1007/s00521-019-04413-4

    Article  Google Scholar 

  22. Li P, Wang D, Wang L, Lu H (2018) Deep visual tracking: review and experimental comparison. Pattern Recogn 76:323–338. https://doi.org/10.1016/j.patcog.2017.11.007

    Article  Google Scholar 

  23. Li Y, Fu C, Ding F, Huang Z, Lu G (2020) Autotrack: Towards high-performance visual tracking for uav with automatic spatio-temporal regularization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 11920–11929, 10.1109/CVPR42600.2020.01194

  24. Liang P, Blasch E, Ling H (2015) Encoding color information for visual tracking: algorithms and benchmark. IEEE Trans Image Process 24(12):5630–5644. https://doi.org/10.1109/TIP.2015.2482905

    Article  MathSciNet  MATH  Google Scholar 

  25. Liu S, Liu D, Muhammad K, Ding W (2021) Effective template update mechanism in visual tracking with background clutter. Neurocomputing 458:615–625. https://doi.org/10.1016/j.neucom.2019.12.143

    Article  Google Scholar 

  26. Liu S, Wang S, Liu X, Gandomi AH, Daneshmand M, Muhammad K, De Albuquerque VHC (2021) Human memory update strategy: a multi-layer template update mechanism for remote visual monitoring. IEEE Trans Multimed 23:2188–2198. https://doi.org/10.1109/TMM.2021.3065580

    Article  Google Scholar 

  27. Liu T, Wang G, Yang Q (2015) Real-time part-based visual tracking via adaptive correlation filters. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4902–4912, https://doi.org/10.1109/CVPR.2015.7299124

  28. Ma C, Yang X, Chongyang Zhang, Yang M (2015) Long-term correlation tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5388–5396, https://doi.org/10.1109/CVPR.2015.7299177

  29. Marvasti-Zadeh SM, Cheng L, Ghanei-Yakhdan H, Kasaei S (2021) Deep learning for visual tracking: a comprehensive survey. IEEE Trans Intell Transport Sys. https://doi.org/10.1109/TITS.2020.3046478

    Article  Google Scholar 

  30. Morrison SWJ (1950) Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. Ann Maths Stat 21(1):124–127. https://doi.org/10.2307/2236561

    Article  MathSciNet  MATH  Google Scholar 

  31. Mueller M, Smith N, Ghanem B (2016) A benchmark and simulator for uav tracking. In: Proceedings of the european conference on computer vision, pp 445–461, https://doi.org/10.1007/978-3-319-46448-0_27

  32. Mueller M, Smith N, Ghanem B (2017) Context-aware correlation filter tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1387–1395, https://doi.org/10.1109/CVPR.2017.152

  33. Sun Y, Sun C, Wang D, He Y, Lu H (2019) Roi pooled correlation filters for visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5776–5784, https://doi.org/10.1109/CVPR.2019.00593

  34. Tang M, Yu B, Zhang F, Wang J (2018) High-speed tracking with multi-kernel correlation filters. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4874–4883, https://doi.org/10.1109/CVPR.2018.00512

  35. Wang M, Liu Y, Huang Z (2017) Large margin object tracking with circulant feature maps. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4800–4808, https://doi.org/10.1109/CVPR.2017.510

  36. Wang N, Shi J, Yeung D, Jia J (2015) Understanding and diagnosing visual tracking systems. In: Proceedings of the IEEE international conference on computer vision, pp 3101–3109, https://doi.org/10.1109/ICCV.2015.355

  37. Wang N, Zhou W, Tian Q, Hong R, Wang M, Li H (2018) Multi-cue correlation filters for robust visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4844–4853, https://doi.org/10.1109/CVPR.2018.00509

  38. Wang N, Song Y, Ma C, Zhou W, Liu W, Li H (2019a) Unsupervised deep tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1308–1317, https://doi.org/10.1109/CVPR.2019.00140

  39. Wang P, Sun M, Wang H, Li X, Yang Y (2020) Convolution operators for visual tracking based on spatial-temporal regularization. Neural Comput Appl 32(10):5339–5351. https://doi.org/10.1007/s00521-020-04704-1

    Article  Google Scholar 

  40. Wang X, Hou Z, Yu W, Jin Z, Zha Y, Qin X (2019) Online scale adaptive visual tracking based on multilayer convolutional features. IEEE Trans Cybernet 49(1):146–158. https://doi.org/10.1109/TCYB.2017.2768570

    Article  Google Scholar 

  41. Wu Y, Lim J, Yang M (2013) Online object tracking: A benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2411–2418, https://doi.org/10.1109/CVPR.2013.312

  42. Wu Y, Lim J, Yang M (2015) Object tracking benchmark. IEEE Trans Patt Anal Mach Intell 37(9):1834–1848. https://doi.org/10.1109/TPAMI.2014.2388226

    Article  Google Scholar 

  43. Xu T, Feng Z, Wu X, Kittler J (2019a) Joint group feature selection and discriminative filter learning for robust visual object tracking. In: Proceedings of the international conference on computer vision, pp 7949–7959, https://doi.org/10.1109/ICCV.2019.00804

  44. Xu T, Feng Z, Wu X, Kittler J (2019) Learning adaptive discriminative correlation filters via temporal consistency preserving spatial feature selection for robust visual object tracking. IEEE Trans Image Process 28(11):5596–5609. https://doi.org/10.1109/TIP.2019.2919201

    Article  MathSciNet  MATH  Google Scholar 

  45. Yun S, Choi J, Yoo Y, Yun K, Choi JY (2017) Action-decision networks for visual tracking with deep reinforcement learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1349–1358, https://doi.org/10.1109/CVPR.2017.148

  46. Zhang L, Varadarajan J, Suganthan PN, Ahuja N, Moulin P (2017a) Robust visual tracking using oblique random forests. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5825–5834, https://doi.org/10.1109/CVPR.2017.617

  47. Zhang S, Lu W, Xing W, Zhang L (2020) Learning scale-adaptive tight correlation filter for object tracking. IEEE Trans Cybernet 50(1):270–283. https://doi.org/10.1109/TCYB.2018.2868782

    Article  Google Scholar 

  48. Zhang T, Xu C, Yang M (2017b) Multi-task correlation particle filter for robust object tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4819–4827, https://doi.org/10.1109/CVPR.2017.512

  49. Zhang T, Xu C, Yang M (2019) Robust structural sparse tracking. IEEE Trans Patt Anal Machine Intell 41(2):473–486. https://doi.org/10.1109/TPAMI.2018.2797082

    Article  Google Scholar 

  50. Zhu G, Porikli F, Li H (2016) Beyond local search: tracking objects everywhere with instance-specific proposals. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 943–951, https://doi.org/10.1109/CVPR.2016.108

  51. Zhu G, Zhang Z, Wang J, Wu Y, Lu H (2019) Dynamic collaborative tracking. IEEE Trans Neural Networks Learn Sys 30(10):3035–3046. https://doi.org/10.1109/TNNLS.2018.2861838

    Article  Google Scholar 

Download references

Acknowledgements

Many thanks to the anonymous reviewers and editors for the valuable comments and suggestions. This work is partially supported by the National Natural Science Foundation of China (Nos. 61801272 and 61601266), and Natural Science Foundation of Shandong Province (Nos. ZR2020MF127 and ZR2021QD041).

Author information

Authors and Affiliations

  1. School of Electrical and Electronic Engineering, Shandong University of Technology, Zibo, 255000, China

    Libin Xu & Mingliang Gao

  2. School of Engineering, University of British Columbia Okanagan, Kelowna, V1V 1V7, Canada

    Zheng Liu

  3. School of Electronic Engineering and Computer Science, Queen Mary University of London, London, E1 4NS, UK

    Qilei Li

  4. Department of Embedded Systems Engineering, Incheon National University, Incheon, 22012, South Korea

    Gwanggil Jeon

Authors
  1. Libin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingliang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qilei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gwanggil Jeon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingliang Gao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Xu, L., Gao, M., Liu, Z. et al. Accelerated duality-aware correlation filters for visual tracking. Neural Comput & Applic 34, 6241–6256 (2022). https://doi.org/10.1007/s00521-021-06794-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06794-x

Keywords

  • Visual tracking
  • Correlation filters
  • Spatio-temporal regularization
  • Online model update