Multimedia Tools and Applications

, Volume 78, Issue 2, pp 2227–2250 | Cite as

Multi-tracker fusion via adaptive outlier detection

  • Chao Xie
  • Ning Wang
  • Wengang ZhouEmail author
  • Weiping Li
  • Houqiang Li


In visual tracking task, due to the ubiquitous challenging attributes such as illumination changes, occlusion and target deformation, there hardly exists a tracker that works satisfactorily under various circumstances. To cope with different challenging factors, in this paper, we propose a fusion framework to absorb the strength of different tracking algorithms for robust object tracking. Our approach focuses on the output fusion of different trackers, without knowing their specific details, which makes our framework quite general to incorporate any new tracker. The proposed framework consists of three main steps. First, it measures the pair-wise correlation between different tracker pairs based on their appearance and geometric consistency. Then, we introduce two effective strategies to identify the unreliable trackers by analyzing the computed pair-wise relationships. Through this outlier detection process, our fusion framework adaptively discards the potential failure trackers and weights the rest trackers differently. Finally, the fusion result is derived from weighted combination of the outputs from the reliable group of trackers. Extensive experimental results on the challenging OTB-2013 and OTB-2015 datasets demonstrate the effectiveness of the proposed fusion framework.


Visual tracking Pair-wise evaluation Outlier detection Tracker fusion 



  1. 1.
    Bailer C, Pagani A, Stricker D (2014) A superior tracking approach: building a strong tracker through fusion. In: ECCVGoogle Scholar
  2. 2.
    Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr P (2016) Staple: complementary learners for real-time tracking. In: CVPRGoogle Scholar
  3. 3.
    Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PH (2016) Fully-convolutional siamese networks for object tracking. In: ECCVGoogle Scholar
  4. 4.
    Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: CVPRGoogle Scholar
  5. 5.
    Bomze IM (2002) Branch-and-bound approaches to standard quadratic optimization problems. J Glob Optim 22(1-4):17–37MathSciNetCrossRefGoogle Scholar
  6. 6.
    Choi J, Jin Chang H, Jeong J, Demiris Y, Young Choi J (2016) Visual tracking using attention-modulated disintegration and integration. In: CVPRGoogle Scholar
  7. 7.
    Choi J, Jin Chang H, Yun S, Fischer T, Demiris Y, Young Choi J (2017) Attentional correlation filter network for adaptive visual tracking. In: CVPRGoogle Scholar
  8. 8.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: CVPRGoogle Scholar
  9. 9.
    Danelljan M, Bhat G, Shahbaz Khan F, Felsberg M (2017) Eco: Efficient convolution operators for tracking. In: CVPRGoogle Scholar
  10. 10.
    Danelljan M, Häger G, Khan F, Felsberg M (2014) Accurate scale estimation for robust visual tracking. In: BMVCGoogle Scholar
  11. 11.
    Danelljan M, Häger G., Khan FS, Felsberg M (2016) Adaptive decontamination of the training set: a unified formulation for discriminative visual tracking. In: CVPRGoogle Scholar
  12. 12.
    Danelljan M, Hager G, Shahbaz Khan F, Felsberg M (2015) Convolutional features for correlation filter based visual tracking. In: ICCV workshopGoogle Scholar
  13. 13.
    Danelljan M, Hager G, Shahbaz Khan F, Felsberg M (2015) Learning spatially regularized correlation filters for visual tracking. In: ICCVGoogle Scholar
  14. 14.
    Danelljan M, Robinson A, Khan FS, Felsberg M (2016) Beyond correlation filters: learning continuous convolution operators for visual tracking. In: ECCVGoogle Scholar
  15. 15.
    Fan H, Ling H (2017) Parallel tracking and verifying: a framework for real-time and high accuracy visual tracking. In: ICCVGoogle Scholar
  16. 16.
    Fan H, Ling H (2017) Sanet: structure-aware network for visual tracking. In: CVPR workshopGoogle Scholar
  17. 17.
    Galoogahi HK, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. In: ICCVGoogle Scholar
  18. 18.
    Gao Y, Ji R, Zhang L, Hauptmann A (2014) Symbiotic tracker ensemble toward a unified tracking framework. TCSVT 24(7):1122–1131Google Scholar
  19. 19.
    Han B, Sim J, Adam H (2017) Branchout: regularization for online ensemble tracking with convolutional neural networks. In: CVPRGoogle Scholar
  20. 20.
    Hare S, Saffari A, Torr PH (2011) Struck: structured output tracking with kernels. In: ICCVGoogle Scholar
  21. 21.
    Henriques JF, Caseiro R, Martins P, Batista J (2015) High-speed tracking with kernelized correlation filters. TPAMI 37(3):583–596CrossRefGoogle Scholar
  22. 22.
    Hong S, You T, Kwak S, Han B (2015) Online tracking by learning discriminative saliency map with convolutional neural network. In: ICMLGoogle Scholar
  23. 23.
    Jia X, Lu H, Yang MH (2012) Visual tracking via adaptive structural local sparse appearance model. In: CVPRGoogle Scholar
  24. 24.
    Kalal Z, Mikolajczyk K, Matas J (2012) Tracking-learning-detection. TPAMI 34(7):1409–1422CrossRefGoogle Scholar
  25. 25.
    Khalid O, SanMiguel JC, Cavallaro A (2017) Multi-tracker partition fusion. TCSVT 27(7):1527–1539Google Scholar
  26. 26.
    Lee DY, Sim JY, Kim CS (2015) Multihypothesis trajectory analysis for robust visual tracking. In: CVPRGoogle Scholar
  27. 27.
    Li J, Deng C, Xu RYD, Tao D, Zhao B (2017) Robust object tracking with discrete graph based multiple experts. TIP 26(6):2736–2750MathSciNetGoogle Scholar
  28. 28.
    Li K, He F, Yu H, Chen X (2017) A parallel and robust object tracking approach synthesizing adaptive bayesian learning and improved incremental subspace learning. Frontiers Comput SciGoogle Scholar
  29. 29.
    Li K, He FZ, Yu HP (2018) Robust visual tracking based on convolutional features with illumination and occlusion handing. J Comput Sci Technol 33(1):223–236CrossRefGoogle Scholar
  30. 30.
    Li K, He FZ, Yu HP, Chen X (2017) A correlative classifiers approach based on particle filter and sample set for tracking occluded target. Applied Mathematics-A Journal of Chinese Universities 32(3):294–312MathSciNetCrossRefGoogle Scholar
  31. 31.
    Li Y, Zhu J (2014) A scale adaptive kernel correlation filter tracker with feature integration. In: ECCV workshopGoogle Scholar
  32. 32.
    Liu H, Yan S (2010) Robust graph mode seeking by graph shift. In: ICMLGoogle Scholar
  33. 33.
    Liu S, Zhang T, Cao X, Xu C (2016) Structural correlation filter for robust visual tracking. In: CVPRGoogle Scholar
  34. 34.
    Liu T, Wang G, Yang Q (2015) Real-time part-based visual tracking via adaptive correlation filters. In: CVPRGoogle Scholar
  35. 35.
    Lukezic A, Vojir T, Cehovin Zajc L, Matas J, Kristan M (2017) Discriminative correlation filter with channel and spatial reliability. In: CVPRGoogle Scholar
  36. 36.
    Ma C, Huang JB, Yang X, Yang MH (2015) Hierarchical convolutional features for visual tracking. In: ICCVGoogle Scholar
  37. 37.
    Ma C, Yang X, Zhang C, Yang MH (2015) Long-term correlation tracking. In: CVPRGoogle Scholar
  38. 38.
    Nam H, Baek M, Han B (2016) Modeling and propagating cnns in a tree structure for visual tracking. arXiv:1608.07242
  39. 39.
    Nam H, Han B (2016) Learning multi-domain convolutional neural networks for visual tracking. In: CVPRGoogle Scholar
  40. 40.
    Qi Y, Zhang S, Qin L, Yao H, Huang Q, Yang JLMH (2016) Hedged deep tracking. In: CVPRGoogle Scholar
  41. 41.
    Shen M, Zhang Y, Wang R, Yang J, Xue L, Hu M (2018) Robust object tracking via superpixels and keypoints. Multimedia Tools and Applications, pp 1–21Google Scholar
  42. 42.
    Song Y, Ma C, Gong L, Zhang J, Lau R, Yang MH (2017) Crest: convolutional residual learning for visual tracking. In: ICCVGoogle Scholar
  43. 43.
    Sun J, He FZ, Chen YL, Chen X (2016) A multiple template approach for robust tracking of fast motion target. Applied Mathematics-A Journal of Chinese Universities 31(2):177–197MathSciNetCrossRefGoogle Scholar
  44. 44.
    Tao R, Gavves E, Smeulders AW (2016) Siamese instance search for tracking. In: CVPRGoogle Scholar
  45. 45.
    Valmadre J, Bertinetto L, Henriques JF, Vedaldi A, Torr PH (2017) End-to-end representation learning for correlation filter based tracking. In: CVPRGoogle Scholar
  46. 46.
    Wang H, Liu P, Du Y, Liu X (2018) Online convolution network tracking via spatio-temporal context. Multimedia Tools and Applications, pp 1–14Google Scholar
  47. 47.
    Wang L, Ouyang W, Wang X, Lu H (2016) Stct: Sequentially training convolutional networks for visual tracking. In: CVPRGoogle Scholar
  48. 48.
    Wang M, Liu Y, Huang Z (2017) Large margin object tracking with circulant feature maps. In: CVPRGoogle Scholar
  49. 49.
    Wang N, Yeung DY (2014) Ensemble-based tracking: Aggregating crowdsourced structured time series data. In: ICMLGoogle Scholar
  50. 50.
    Wu Y, Lim J, Yang MH (2013) Online object tracking: a benchmark. In: CVPRGoogle Scholar
  51. 51.
    Wu Y, Lim J, Yang MH (2015) Object tracking benchmark. TPAMI 37 (9):1834–1848CrossRefGoogle Scholar
  52. 52.
    Yan X, He F, Hou N, Ai H (2018) An efficient particle swarm optimization for large-scale hardware/software co-design system. International Journal of Cooperative Information Systems 27(01):1741,001CrossRefGoogle Scholar
  53. 53.
    Yang L, Jiang P, Wang F, Wang X (2018) Robust real-time visual object tracking via multi-scale fully convolutional siamese networks. Multimedia Tools and Applications, pp 1–13Google Scholar
  54. 54.
    Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38(4):13CrossRefGoogle Scholar
  55. 55.
    Yu H, He F, Pan Y (2018) A novel region-based active contour model via local patch similarity measure for image segmentation. Multimedia Tools and Applications, pp 1–23Google Scholar
  56. 56.
    Yun S, Choi J, Yoo Y, Yun K, Young Choi J (2017) Action-decision networks for visual tracking with deep reinforcement learning. In: CVPRGoogle Scholar
  57. 57.
    Zhang J, Ma S, Sclaroff S (2014) Meem: robust tracking via multiple experts using entropy minimization. In: ECCVGoogle Scholar
  58. 58.
    Zhang K, Zhang L, Yang MH (2012) Real-time compressive tracking. In: ECCVGoogle Scholar
  59. 59.
    Zhang K, Zhang L, Yang MH, Zhang D (2013) Fast visual tracking via dense spatio-temporal context learning. In: ECCVGoogle Scholar
  60. 60.
    Zhang T, Bibi A, Ghanem B (2016) In defense of sparse tracking: circulant sparse tracker. In: CVPRGoogle Scholar
  61. 61.
    Zhang T, Xu C, Yang MH (2017) Multi-task correlation particle filter for robust object tracking. In: CVPRGoogle Scholar
  62. 62.
    Zhong W, Lu H, Yang MH (2012) Robust object tracking via sparsity-based collaborative model. In: CVPRGoogle Scholar
  63. 63.
    Zhou Y, He F, Hou N, Qiu Y (2018) Parallel ant colony optimization on multi-core simd cpus. Futur Gener Comput Syst 79:473–487CrossRefGoogle Scholar
  64. 64.
    Zhou Y, He F, Qiu Y (2017) Dynamic strategy based parallel ant colony optimization on gpus for tsps. Science China Information Sciences 068(6):102Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.The CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Department of Electronic Engineering and Information ScienceUniversity of Science and Technology of ChinaHefeiChina

Personalised recommendations