Advertisement

Exploiting the Anisotropy of Correlation Filter Learning for Visual Tracking

  • Yao SuiEmail author
  • Ziming Zhang
  • Guanghui Wang
  • Yafei Tang
  • Li Zhang
Article

Abstract

Correlation filtering based tracking model has received significant attention and achieved great success in terms of both tracking accuracy and computational complexity. However, due to the limitation of the loss function, current correlation filtering paradigm could not reliably respond to the abrupt appearance changes of the target object. This study focuses on improving the robustness of the correlation filter learning. An anisotropy of the filter response is observed and analyzed for the correlation filtering based tracking model, through which the overfitting issue of previous methods is alleviated. Three sparsity related loss functions are proposed to exploit the anisotropy, leading to three implementations of visual trackers, correspondingly resulting in improved overall tracking performance. A large number of experiments are conducted and these experimental results demonstrate that the proposed approach greatly improves the robustness of the learned correlation filter. The proposed trackers performs comparably against state-of-the-art tracking methods on four latest standard tracking benchmark datasets.

Keywords

Object tracking Anisotropy Correlation filtering Loss function Sparsity Robustness Sensitivity 

Notes

References

  1. Bach, F., Jenatton, R., Mairal, J., & Obozinski, G. (2011). Convex optimization with sparsity-inducing norms. Optimization for Machine Learning, 5, 19–53.zbMATHGoogle Scholar
  2. Beck, A., & Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1), 183–202.MathSciNetCrossRefzbMATHGoogle Scholar
  3. Bertinetto, L., Henriques, J., Valmadre, J., Torr, P., & Vedaldi, A. (2016a). SiameseFC-ResNet. In ECCV VOT workshop.Google Scholar
  4. Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., & Torr, P. (2016b). Staple: Complementary learners for real-time tracking. In CVPR.Google Scholar
  5. Bertinetto, L., Valmadre, J., Miksik, O., Golodetz, S., & Torr, P. H. (2015). The importance of estimating object extent when tracking with correlation filters. In ICCV VOT workshop.Google Scholar
  6. Bibi, A., Mueller, M., & Ghanem, B. (2016). Target response adaptation for correlation filter tracking. In ECCV.Google Scholar
  7. Bogun, I., & Ribeiro, E. (2015). Structure tracker with the robust Kalman filter. In ICCV VOT workshop.Google Scholar
  8. Bolme, D., Beveridge, J. R., Draper, Ba., & Lui, Y. M. (2010). Visual object tracking using adaptive correlation filters. In CVPR.Google Scholar
  9. Chen, B., Wang, L., & Lu, H. (2017). FSTC. In ICCV VOT workshop Google Scholar
  10. Chen, K., & Tao, W. (2016). Convolutional regression for visual tracking. arXiv.Google Scholar
  11. Chi, Z., Lu, H., Wang, L., & Sun, C. (2016). Dual deep network tracker. In ECCV VOT workshop.Google Scholar
  12. Danelljan, M., Bhat, G., Khan, S., & Felsberg, M. (2017a). Efficient convolution operator tracker: Hand crafted. In ICCV VOT workshop.Google Scholar
  13. Danelljan, M., Ghat, G., Khan, F., & Felsberg, M. (2017b). ECO: Efficient convolution operators for tracking. In CVPR.Google Scholar
  14. Danelljan, M., Gustav, H., Khan, F. S., & Felsberg, M. (2015). Learning spatially regularized correlation filters for visual tracking. In ICCV.Google Scholar
  15. Danelljan, M., Häger, G., Khan, F.S., & Felsberg, M. (2014a). Accurate scale estimation for robust visual tracking. In BMVC.Google Scholar
  16. Danelljan, M., Khan, F. S., Felsberg, M., & Weijer, J. V. D. (2014b). Adaptive color attributes for real-time visual tracking. In CVPR.Google Scholar
  17. Danelljan, M., Robinson, A., Shahbaz, K., & Felsberg, M. (2016). Beyond correlation filters: Learning continuous convolution operators for visual tracking. In ECCV.Google Scholar
  18. Duffner, S., & Garcia, C. (2015). Using discriminative motion context for online visual object tracking. IEEE Transactions on Circuits and Systems for Video Technology (TCVST), 26(12), 2215–2225.CrossRefGoogle Scholar
  19. Gao, J., Zhang, T., Xu, C., & Liu, B. (2016). Discriminative deep correlation tracking. In ECCV VOT workshop.Google Scholar
  20. Gundogdu, E., & Alatan, A. (2017). Good features to correlate for visual tracking. arXiv.Google Scholar
  21. Hare, S., Saffari, A., & Torr, P. (2011). Struck: Structured output tracking with kernels. In ICCV.Google Scholar
  22. He, Z., Fan, Y., & Zhuang, J. (2017). CFWCR. In ICCV VOT workshop.Google Scholar
  23. Henriques, F., Caseiro, R., Martins, P., & Batista, J. (2012). Exploiting the circulant structure of tracking-by-detection with kernels. In ECCV.Google Scholar
  24. Henriques, J., Caseiro, R., Martins, P., & Batista, J. (2015). High-speed tracking with kernelized correlation filters. IEEE TPAMI, 37(3), 583–596.CrossRefGoogle Scholar
  25. Hu, T., Du, D., Wen, L., Li, W., Qi, H., & Lyu, S. (2016). Geometric structure hyper-graph based tracker version 2. In ECCV VOT Workshop.Google Scholar
  26. Hua, Y., Alahari, K., & Schmid, C. (2015). Online object tracking with proposal selection. In ICCV.Google Scholar
  27. Kalal, Z., Mikolajczyk, K., & Matas, J. (2012). Tracking–learning-detection. IEEE TPAMI, 34(7), 1409–1422.CrossRefGoogle Scholar
  28. Kristan, M., Matas, J., Leonardis, A., Vojir, T., Pflugfelder, R., Fernandez, G., et al. (2016). A novel performance evaluation methodology for single-target trackers. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 38(11), 2137–2155.CrossRefGoogle Scholar
  29. Kwon, J., & Lee, K. (2010). Visual tracking decomposition. In CVPR.Google Scholar
  30. Lee, H., & Kim, D. (2016). Salient region based tracker. In ECCV VOT workshop.Google Scholar
  31. Lee, J. Y., Choi, S., Jeong, J. C., Kim, J. W., & Cho, J. I. (2015). Scaled SumShift tracker. In ICCV VOT workshop.Google Scholar
  32. Lee, J. Y., Choi, S., Jeong, J. C., Kim, J. W., & Cho, J. I. (2016). SumShift tracker with kernelized correlation filter. In ECCV VOT workshop.Google Scholar
  33. Lee, J. Y., & Yu, W. (2011). Visual tracking by partition-based histogram backprojection and maximum support criteria. In IEEE international conference on robotics and biomimetics.Google Scholar
  34. Li, Y., & Zhu, J. (2014). A scale adaptive kernel correlation filter tracker with feature integration. In ECCV workshop.Google Scholar
  35. Li, Y., & Zhu, J. (2015). NSAMF. In ICCV VOT workshop.Google Scholar
  36. Liu, S., Zhang, T., Cao, X., & Xu, C. (2016). Structural correlation filter for robust visual tracking. In CVPR.Google Scholar
  37. Liu, T., Wang, G., & Yang, Q. (2015). Real-time part-based visual tracking via adaptive correlation filters. In CVPR.Google Scholar
  38. Lukezic, A., Cehovin, L., & Kristan, M. (2015). Layered deformable parts tracker. In ICCV VOT workshop.Google Scholar
  39. Lukezic, A., Cehovin, L., & Kristan, M. (2016). Deformable parts correlation filters for robust visual tracking. arXiv.Google Scholar
  40. Lukezic, A., Vojir, T., Cehovin, L., Matas, J., & Kristan, M. (2017a). Discriminative correlation filter with channel and spatial reliability. In CVPR.Google Scholar
  41. Lukezic, A., Vojir, T., Cehovin, L., Matas, J., & Kristan, M. (2017b). Discriminative correlation filter with channel and spatial reliability: Fast. In ICCV VOT workshop.Google Scholar
  42. Ma, C., Huang, J. B., Yang, X., & Yang, M. H. (2015a). Hierarchical convolutional features for visual tracking. In ICCV.Google Scholar
  43. Ma, C., Yang, X., Zhang, C., & Yang, Mh. (2015b). Long-term correlation tracking. In CVPR.Google Scholar
  44. Mei, X., & Ling, H. (2011). Robust visual tracking and vehicle classification via sparse representation. IEEE TPAMI, 33(11), 2259–2272.CrossRefGoogle Scholar
  45. Mocanu, B., Tapu, R., & Zaharia, T. (2017). Adaptive single object tracking using offline learned motion and visual similar patterns. In ICCV VOT workshop.Google Scholar
  46. Nam, B. M. H., & Han, B. (2016a). Modeling and propagating CNNs in a tree structure for visual tracking. arXiv.Google Scholar
  47. Nam, H., & Han, B. (2016b). Learning multi-domain convolutional neural networks for visual tracking. In CVPR.Google Scholar
  48. Poostchi, M., Palaniappan, K., Seetharaman, G., & Gao, K. (2017). Spatial pyramid context-aware tracker. In ICCV VOT workshop.Google Scholar
  49. Possegger, H., Mauthner, T., & Bischof, H. (2015). In defense of color-based model-free tracking. In CVPR.Google Scholar
  50. Qi, Y., Qin, L., Zhang, S., & Huang, Q. (2016a). Scale-and-state aware tracker. In ECCV VOT workshop.Google Scholar
  51. Qi, Y., Zhang, S., Qin, L., Yao, H., Huang, Q., Lim, J., et al. (2016b). Hedged deep tracking. In CVPR.Google Scholar
  52. Singh, S., & Mishra, D. (2017). gNetTracker. In ICCV VOT workshop.Google Scholar
  53. Smeulders, A. W. M., Chu, D. M., Cucchiara, R., Calderara, S., Dehghan, A., & Shah, M. (2014). Visual tracking: An experimental survey. IEEE TPAMI, 36(7), 1442–1468.CrossRefGoogle Scholar
  54. Sui, Y., Tang, Y., & Zhang, L. (2015a). Discriminative low-rank tracking. In ICCV.Google Scholar
  55. Sui, Y., Tang, Y., Zhang, L., & Wang, G. (2018a). Visual tracking via subspace learning: A discriminative approach. International Journal of Computer Vision (IJCV), 126(5), 515–536.MathSciNetCrossRefGoogle Scholar
  56. Sui, Y., Wang, G., Tang, Y., & Zhang, L. (2016a). Tracking completion. In ECCV.Google Scholar
  57. Sui, Y., Wang, G., & Zhang, L. (2018b). Correlation filter learning toward peak strength for visual tracking. IEEE Transactions on Cybernetics, 48(4), 1290–1303.CrossRefGoogle Scholar
  58. Sui, Y., Wang, G., Zhang, L., & Yang, M. H. (2018c). Exploiting spatial–temporal locality of tracking via structured dictionary learning. IEEE Transactions on Image Processing (TIP), 27(3), 1282–1296.MathSciNetCrossRefGoogle Scholar
  59. Sui, Y., & Zhang, L. (2015). Visual tracking via locally structured Gaussian process regression. IEEE SPL, 22(9), 1331–1335.Google Scholar
  60. Sui, Y., & Zhang, L. (2016). Robust tracking via locally structured representation. IJCV, 119(2), 110–144.MathSciNetCrossRefzbMATHGoogle Scholar
  61. Sui, Y., Zhang, S., & Zhang, L. (2015b). Robust visual tracking via sparsity-induced subspace learning. IEEE TIP, 24(12), 4686–4700.MathSciNetGoogle Scholar
  62. Sui, Y., Zhang, Z., Wang, G., Tang, Y., & Zhang, L. (2016b). Real-time visual tracking: Promoting the robustness of correlation filter learning. In ECCV.Google Scholar
  63. Sui, Y., Zhao, X., Zhang, S., Yu, X., Zhao, S., & Zhang, L. (2015c). Self-expressive tracking. Pattern Recognit., 48(9), 2872–2884.CrossRefGoogle Scholar
  64. Sun, C., Liu, J., Lu, H., & Yang, M. H. (2017). Learning spatial-aware regressions for visual tracking. In ICCV VOT workshop.Google Scholar
  65. Tang, M., & Feng, J. (2015). Multi-kernel correlation filter for visual tracking. In ICCV.Google Scholar
  66. Vojir, T., Matas, J., & Noskova, J. (2015). Online adaptive hidden Markov model for multi-tracker fusion. arXiv.Google Scholar
  67. Vojir, T., Noskova, J., & Matas, J. (2014). Robust scale-adaptive mean-shift for tracking. Pattern Recognit. Lett., 40, 250–258.CrossRefGoogle Scholar
  68. Walsh, R., & Mederios, H. (2016). CF2 with Response Information Failure Detection. In ECCV VOT workshop.Google Scholar
  69. Wang, D., Lu, H., & Yang, M. H. (2013). Least soft-thresold squares tracking. In CVPR.Google Scholar
  70. Wang, L., Lu, H., Wang, Y., & Sun, C. (2016a). Multi-level deep feature tracker. In ECCV VOT workshop.Google Scholar
  71. Wang, L., Ouyang, W., Wang, X., & Lu, H. (2015a). Visual tracking with fully convolutional networks. In ICCV.Google Scholar
  72. Wang, L., Ouyang, W., Wang, X., & Lu, H. (2016b). STCT: Sequentially training convolutional networks for visual tracking. In CVPR.Google Scholar
  73. Wang, N., Huang, Z., Li, S., & Yeung, D. Y. (2015b). Ensemble-based tracking: Aggregating crowdsourced structured time series data. In ICML.Google Scholar
  74. Wang, N., Li, S., Gupta, A., & Yeung, D. Y. (2015c). Transferring rich feature hierarchies for robust visual tracking. arXiv.Google Scholar
  75. Wang, N., Zhou, W., & Li, H. (2017a). Dual deep network tracker. In ICCV VOT workshop.Google Scholar
  76. Wang, Q., Gao, J., Xing, J., Zhang, M., Z. Z., & Hu, W. (2017b). SiamDCF. In ICCV VOT workshop.Google Scholar
  77. Wen, L., Du, D., Li, S., Chang, C.M., Lyu, S., & Huang, Q. (2016). Structure hyper-graph based correlation filter tracker. In ECCV VOT workshop.Google Scholar
  78. Wright, J., Ma, Y., Mairal, J., & Sapiro, G. (2010). Sparse representation for computer vision and pattern recognition. Proceedings of The IEEE, 98(6), 1031–1044.CrossRefGoogle Scholar
  79. Wu, Y., Lim, J., & Yang, M. H. (2013). Online object tracking: A benchmark. In CVPR.Google Scholar
  80. Wu, Y., Lim, J., & Yang, M. H. (2015). Object tracking benchmark. IEEE TPAMI, 37(9), 1834–1848.CrossRefGoogle Scholar
  81. Xu, Z., Li, Y., & Zhu, J. (2016). An improved STAPLE tracker with multiple feature integration. In ECCV VOT Workshop.Google Scholar
  82. Yang, L., Liu, R., Zhang, D., & Zhang, L. (2017). Deep location-specific tracking. In ICCV VOT workshop.Google Scholar
  83. Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking: A Survey. ACM Computing Surveys, 38(4), 13–57.CrossRefGoogle Scholar
  84. Zhang, J., Ma, S., & Sclaroff, S. (2014a). MEEM: Robust tracking via multiple experts using entropy minimization. In ECCV.Google Scholar
  85. Zhang, K., Zhang, L., Liu, Q., Zhang, D., & Yang, M. H. (2014b). Fast visual tracking via dense spatio-temporal context learning. In ECCV.Google Scholar
  86. Zhang, M., Xing, J., Gao, J., & Hu, W. (2016). Fully-functional correlation filtering-based tracker. In ECCV VOT workshop.Google Scholar
  87. Zhang, M., Xing, J., Gao, J., Shi, X., Wang, Q., & Hu, W. (2015a). Rotation adaptive joint scale-spatial correlation filter based tracker. In ICCV VOT workshop.Google Scholar
  88. Zhang, S., Sui, Y., Zhao, S., Yu, X., & Zhang, L. (2015b). Multi-local-task learning with global regularization for object tracking. Pattern Recognit., 48(12), 3881–3894.CrossRefGoogle Scholar
  89. Zhang, S., Zhao, S., Sui, Y., & Zhang, L. (2015c). Single object tracking with fuzzy least squares support vector machine. IEEE TIP, 24(12), 5723–5738.MathSciNetGoogle Scholar
  90. Zhang, T., Gao, J., & Xu, C. (2017a). Robust correlation particle filter. In ICCV VOT workshop.Google Scholar
  91. Zhang, T., Ghanem, B., & Liu, S. (2012a). Robust visual tracking via multi-task sparse learning. In CVPR.Google Scholar
  92. Zhang, T., Ghanem, B., Liu, S., & Ahuja, N. (2012b). Low-rank sparse learning for robust visual tracking. In ECCV.Google Scholar
  93. Zhang, T., Liu, S., Xu, C., Yan, S., Ghanem, B., Ahuja, N., & Yang, Mh. (2015d). Structural sparse tracking. In CVPR.Google Scholar
  94. Zhang, T., Xu, C., & Yang, M. H. (2017b). Multi-task correlation particle filter for robust object tracking. In CVPR.Google Scholar
  95. Zhu, G., Porikli, F., & Li, H. (2016). Beyond local search: Tracking objects everywhere with instance-specific proposals. CVPR.Google Scholar
  96. Zhu, Z., Huang, G., Zou, W., & Du, D., Huang, C. (2017). UCT. In ICCV VOT workshop.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Yao Sui
    • 1
    Email author
  • Ziming Zhang
    • 2
  • Guanghui Wang
    • 3
  • Yafei Tang
    • 4
  • Li Zhang
    • 5
  1. 1.Harvard Medical SchoolHarvard UniversityBostonUSA
  2. 2.Mitsubishi Electric Research Laboratories (MERL)CambridgeUSA
  3. 3.Department of Electrical Engineering and Computer ScienceUniversity of KansasLawrenceUSA
  4. 4.China Unicom Research InstituteBeijingChina
  5. 5.Department of Electronic EngineeringTsinghua UniversityBeijingChina

Personalised recommendations