Abstract
Since the OTB100 benchmark dataset is released, it has been widely used in a large number of researches on object tracking for performance evaluation. However, the existing datasets are insufficient to evaluate trackers in handling different challenging factors. In this paper, we present the first dataset and benchmark for tracking objects with abrupt motion (AMTSet). The dataset consists of 50 videos of special scenes from our real life, such as camera switching, sudden dynamic change, low frame rate video, etc., which are quite challenging in object tracking. Boundary boxes over 10K frames are marked manually, and all of them are manually labelled for common attributes of object tracking, such as scale variation, illumination variation, occlusion, motion blur, etc. We benchmark the dataset on 36 representative trackers and rank them according to the tracking conditions and results. Furthermore, we propose an evaluation measure for object tracking to better highlight the performances of the trackers against abrupt motion. Our goal is to supplement the existing baseline datasets and provide researchers with more perfect baseline data in order to better evaluate the performance of different trackers.
Similar content being viewed by others
References
Babenko B, Yang MH, Belongie S (2011) Robust Object Tracking with Online Multiple Instance Learning. IEEE Trans Pattern Anal Mach Intell 33(8):1619–1632
Bao C, Wu Y, Ling H, Ji H (2012) Real time robust l1 tracker using accelerated proximal gradient approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1830–1837
Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr PH (2016) Staple: complementary learners for real-time tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1401–1409
Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PH (2016) Fully-convolutional siamese networks for object tracking. In: European conference on computer vision. Springer, pp 850–865
Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2544–2550
Chen Z, Zhong B, Li G, Zhang S, Ji R (2020) Siamese box network for visual tracking. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6668–6677
Collins R, Zhou X, Teh SK (2005) An open source tracking testbed and evaluation web site. In: Proceedings of the IEEE international workshop. Performance evaluation of track. Surveillance (PETS). IEEE, pp 1–8
Comaniciu D, Meer P (2002) Mean shift: A robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619
Dai K, Wang D, Lu H, Sun C, Li J (2019) Visual tracking via adaptive spatially-regularized correlation filters. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 4670–4679
Danelljan M, Bhat G, Shahbaz Khan F, Felsberg M (2017) Eco: Efficient convolution operators for tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). pp 6638–6646
Danelljan M, Häger G., Khan F, Felsberg M (2014) Accurate scale estimation for robust visual tracking. In: Proceedings of the British machine vision conference (BMVC). pp 1–11
Danelljan M, Häger G., Khan FS, Felsberg M (2017) Discriminative scale space tracking. IEEE Trans Pattern Anal Mach Intell 39(8):1561–1575
Danelljan M, Hager G, Shahbaz Khan F, Felsberg M (2015) Convolutional features for correlation filter based visual tracking. In: Proceedings of the IEEE international conference on computer vision workshop (ICCVW). IEEE, pp 58–66
Danelljan M, Hager G, Shahbaz Khan F, Felsberg M (2015) Learning spatially regularized correlation filters for visual tracking. In: Proceedings of IEEE International conference on computer vision. IEEE, pp 4310–4318
Danelljan M, Robinson A, Khan FS, Felsberg M (2016) Beyond correlation filters: learning continuous convolution operators for visual tracking. In: Proceedings of the European conference on computer vision (ECCV). Springer, pp 472–488
Danelljan M, Shahbaz Khan F, Felsberg M, Van de Weijer J (2014) Adaptive color attributes for real-time visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1090–1097
Dinh TB, Vo N, Medioni G (2011) Context tracker: exploring supporters and distracters in unconstrained environments. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1177–1184
Fan H, Lin L, Yang F, Chu P, Deng G, Yu S, Bai H, Xu Y, Liao C, Ling H (2019) A high-quality benchmark for large-scale single object tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 5369–5378
Feng W, Han R, Guo Q, Zhu J, Wang S (2019) Dynamic saliency-aware regularization for correlation filter-based object tracking. IEEE Trans Image Process 28(7):3232–3245
Fu C, Xu J, Lin F, Guo F, Liu T, Zhang Z (2020) Object saliency-aware dual regularized correlation filter for real-time aerial tracking. IEEE Trans Geosci Remote Sens 58(12):8940–8951
Gao J, Zhang T, Xu C (2019) Graph convolutional tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 4649–4659
Guo D, Wang J, Cui Y, Wang Z, Chen S (2020) Siamcar: Siamese fully convolutional classification and regression for visual tracking. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 6269–6277
Guo Q, Feng W, Zhou C, Huang R, Wan L, Wang S (2017) Learning dynamic siamese network for visual object tracking. In: Proceedings of IEEE international conference on computer vision. pp 1763–1771
Gupta M, Kumar S, Behera L, Subramanian VK (2016) A novel vision-based tracking algorithm for a human-following mobile robot. IEEE Trans on Syst Man Cybern Syst 47(7):1415–1427
Hadfield S, Lebeda K, Bowden R (2014) The visual object tracking VOT2014 challenge results. In: Proceedings of the European conference on computer vision (ECCV), visual object tracking challenge workshop
Han R, Guo Q, Feng W (2018) Content-related spatial regularization for visual object tracking. In: Proceedings of the IEEE international conference on multimedia and expo (ICME). IEEE, pp 1– 6
Held D, Thrun S, Savarese S (2016) Learning to track at 100 fps with deep regression networks. In: Proceedings of the European conference on computer vision (ECCV). Springer, pp 749–765
Henriques JF, Caseiro R, Martins P, Batista J (2012) Exploiting the circulant structure of tracking-by-detection with kernels. In: Proceedings of the European conference on computer vision (ECCV). Springer, pp 702–715
Henriques JF, Caseiro R, Martins P, Batista J (2014) High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell 37(3):583–596
Huang L, Zhao X, Huang K (2019) Got-10k: A large high-diversity benchmark for generic object tracking in the wild. IEEE Trans Pattern Anal Mach Intell :1–17. https://doi.org/10.1109/TPAMI.2019.2957464
Jia X, Lu H, Yang MH (2012) Visual tracking via adaptive structural local sparse appearance model. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1822–1829
Kalal Z, Mikolajczyk K, Matas J (2011) Tracking-learning-detection. IEEE Trans Pattern Anal Mach Intell 34(7):1409–1422
Kiani Galoogahi H, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. In: Proceedings of IEEE international conference on computer vision (ICCV). IEEE, pp 1135–1143
Kristan M, Leonardis A, Matas J, Felsberg M, Pflugfelder R, Cehovin Zajc L, Vojir T, Hager G, Lukezic A, Eldesokey A et al (2017) The visual object tracking vot2017 challenge results. In: Proceedings of the IEEE international conference on computer vision workshop (ICCVW). pp 1949–1972
Li B, Wu W, Wang Q, Zhang F, Xing J, Yan J (2019) Siamrpn++: Evolution of siamese visual tracking with very deep networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 4282–4291
Li B, Yan J, Wu W, Zhu Z, Hu X (2018) High performance visual tracking with siamese region proposal network. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 8971–8980
Li F, Tian C, Zuo W, Zhang L, Yang MH (2018) Learning spatial-temporal regularized correlation filters for visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 4904–4913
Li F, Wu X, Zuo W, Zhang D, Zhang L (2020) Remove cosine window from correlation filter-based visual trackers: when and how. IEEE Trans Image Process 29:7045–7060
Li P, Wang D, Wang L, Lu H (2018) Deep visual tracking: review and experimental comparison. Pattern Recognit 76:323–338
Li Y, Zhu J (2014) A scale adaptive kernel correlation filter tracker with feature integration. In: Proceedings of the European conference on computer vision (ECCV). Springer, pp 254–265
Liang P, Blasch E, Ling H (2015) Encoding color information for visual tracking: Algorithms and benchmark. IEEE Trans Image Process 24(12):5630–5644
Lin Z (2019) Intelligent optimization analysis of target tracking based on computer vision. In: 2019 4th international conference on mechanical, control and computer engineering (ICMCCE). IEEE, pp 765–7654
Liu J, Peter C, Robert C, Yanxi L (2013) Tracking sports players with context-conditioned motion models. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1830–1837
Lu H, Wang D (2019) Online visual tracking. Springer, Singapore
Ma C, Huang JB, Yang X, Yang MH (2015) Hierarchical convolutional features for visual tracking. In: Proceedings IEEE international conference on computer vision (ICCV). IEEE, pp 3074–3082
Mueller M, Smith N, Ghanem B (2016) A benchmark and simulator for uav tracking. In: European conference on computer vision. Springer, pp 445–461
Mueller M, Smith N, Ghanem B (2017) Context-aware correlation filter tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1396–1404
Muller M, Bibi A, Giancola S, Alsubaihi S, Ghanem B (2018) Trackingnet: a large-scale dataset and benchmark for object tracking in the wild. In: Proceedings of the european conference on computer vision (ECCV), pp 300–317
Nam H, Han B (2016) Learning multi-domain convolutional neural networks for visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 4293–4302
Oron S, Bar-Hillel A, Levi D, Avidan S (2015) Locally orderless tracking. Int J Comput Vision 111(2):213–228
Pan S, Shi L, Guo S (2015) A kinect-based real-time compressive tracking prototype system for amphibious spherical robots. Sensors 15(4):8232–8252
Ross DA, Lim J, Lin RS, Yang MH (2008) Incremental learning for robust visual tracking. Int J Comput Vis 77(1-3):125–141
Sevilla-Lara L, Learned-Miller E (2012) Distribution fields for tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1910–1917
Shreesha S, MM MP, Verma U, Pai RM (2020) Computer vision based fish tracking and behaviour detection system. In: Proceedings of the 2020 IEEE International conference on distributed computing, VLSI, Electrical circuits and robotics (DISCOVER). IEEE, pp 252–257
Smeulders AW, Chu DM, Cucchiara R, Calderara S, Dehghan A, Shah M (2013) Visual tracking: An experimental survey. IEEE Trans Pattern Anal Mach Intell 36(7):1442–1468
Song Y, Ma C, Wu X, Gong L, bao L, Zuo W, Shen C, Lau RW, Yang MH (2018) Vital: visual tracking via adversarial learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 8990–8999
Wang N, Yeung DY (2013) Learning a deep compact image representation for visual tracking. In: Proceedings of the annual conference on neural information processing systems (NIPS). pp 809–817
Wu Y, Lim J, Yang MH (2013) Online object tracking: a benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2411–2418
Wu Y, Lim J, Yang MH (2015) Object tracking benchmark. IEEE Trans Pattern Anal Mach Intell 37(9):1834–1848
Wu Y, Shen B, Ling H (2012) Online robust image alignment via iterative convex optimization. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1808–1814
Xu T, Feng ZH, Wu X, Kittler J (2019) Learning adaptive discriminative correlation filters via temporal consistency preserving spatial feature selection for robust visual object tracking. IEEE Trans Image Process 28(11):5596–5609
Zhang S, Wang C, Chan SC, Wei X, Ho CH (2014) New object detection, tracking, and recognition approaches for video surveillance over camera network. IEEE Sens J 15(5):2679–2691
Zhang T, Ghanem B, Liu S, Ahuja N (2012) Robust visual tracking via multi-task sparse learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2042–2049
Zhu Z, Wang Q, Li B, Wu W, Yan J, Hu W (2018) Distractor-aware siamese networks for visual object tracking. In: Proceedings of the european conference on computer vision (ECCV), pp 101–117
Zuo W, Wu X, Lin L, Zhang L, Yang MH (2019) Learning support correlation filters for visual tracking. IEEE Trans Pattern Anal 41(5):1158–1172
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was supported by National Natural Science Foundation of China (No. 61972068, 61976042), Liaoning Baiqianwan Talent Program, Dalian Science Foundation for Young Scholars (No. 2017RQ151), Innovative Talents Project of Liaoning Universities (No. LR2019020).
Rights and permissions
About this article
Cite this article
Wang, F., Wang, C., Yin, S. et al. AMTSet: a benchmark for abrupt motion tracking. Multimed Tools Appl 81, 4711–4734 (2022). https://doi.org/10.1007/s11042-021-10947-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-021-10947-4