Advertisement

International Journal of Computer Vision

, Volume 118, Issue 3, pp 337–363 | Cite as

Similarity Fusion for Visual Tracking

  • Yu Zhou
  • Xiang Bai
  • Wenyu Liu
  • Longin Jan Latecki
Article

Abstract

Multiple features’ integration and context structure of unlabeled data have proven their effectiveness in enhancing similarity measures in many applications of computer vision. However, in similarity based object tracking, integration of multiple features has been rarely studied. In contrast to conventional tracking approaches that utilize pairwise similarity for template matching, our approach contributes in two different aspects. First, multiple features are integrated into a unified similarity to enhance the discriminative ability of similarity measurements. Second, the neighborhood context of the samples in forthcoming frame are employed to further improve the measurements. We utilize a diffusion process on a tensor product graph to achieve these goals. The obtained approach is validated on numerous challenging video sequences, and the experimental results demonstrate that it outperforms state-of-the-art t racking methods.

Keywords

Visual tracking Similarity measure Fusion 

Notes

Acknowledgments

This work was primarily supported by National Natural Science Foundation of China (NSFC) (Nos. 61222308, 61572207, and 61573160), and in part by Program for New Century Excellent Talents in University (No. NCET-12-0217), NSF grants OIA-1027897 and IIS-1302164, China 973 Program under Grant No. 2012CB316300, National Natural Science Foundation of China (NSFC) (No. 61173120).

References

  1. Adam, A., Rivlin, E., & Shimshoni, I. (2006). Robust fragment-based tracking using the integral histogram. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 798–805).Google Scholar
  2. Avidan, S. (2004). Support vector tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(8), 1064–1072.CrossRefGoogle Scholar
  3. Avidan, S. (2007). Ensemble tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(2), 261–271.CrossRefGoogle Scholar
  4. Babenko, B., Yang, M., & Belongie, S. (2011). Robust object tracking with online multiple instance learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(8), 1619–1632.CrossRefGoogle Scholar
  5. Badrinarayanan, V., Perez, P., Clerc, F. L., & Oisel, L. (2007). Probabilistic color and adaptive multi-feature tracking with dynamically switched priority between cues. In Proceedings of IEEE Interestial Conference on Computer Vision (ICCV).Google Scholar
  6. Bai, X., Wang, B., Yao, C., Liu, W., & Tu, Z. (2010a). Co transduction for shape retrieval. IEEE Transactions on Image Processing, 32(5), 861–874.MathSciNetGoogle Scholar
  7. Bai, X., Yang, X., Latecki, L. J., Liu, W., & Tu, Z. (2010b). Learning context sensitive shape similarity by graph transduction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(5), 861–874.CrossRefGoogle Scholar
  8. Bao, C., Wu, Y., Ling, H., & Ji, H. (2012). Real time robust l1 tracker using accelerated proximal gradient approach. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  9. Belkin, M., & Niyogi, P. (2004). Semi-supervised learning on riemannian manifolds. Machine Learning, 56(special Issue on clustering), 209–239.Google Scholar
  10. Čehovin, L., Kristan, M., & Leonardis, A. (2013). Robust visual tracking using an adaptive coupled-layer visual model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(4), 941–953.CrossRefGoogle Scholar
  11. Collins, R., Liu, Y., & Leordeanu, M. (2005). Online selection of discriminative tracking features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10), 1631–1643.CrossRefGoogle Scholar
  12. Comaniciu, D., Member, V. R., & Meer, P. (2003). Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(5), 564–575.CrossRefGoogle Scholar
  13. Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 886–893).Google Scholar
  14. Fan, J., Shen, X., & Wu, Y. (2012). Scribble tracker: A matting-based approach for robust tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(8), 1633–1644.CrossRefGoogle Scholar
  15. Fan, J., Wu, Y., & Dai, S. (2010). Discriminative spatial attention for robust tracking. In Proceedings of European Conference on Computer Vision (ECCV).Google Scholar
  16. Fan, Z., Yang, M., Wu, Y., Hua, G., & Yu, T. (2006). Efficient optimal kernel placement for reliable visual tracking. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  17. Francesc Moreno-Noguer, A. S., & Samaras, D. (2008). Dependent multiple cue integration for robust tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(4), 670–685.CrossRefGoogle Scholar
  18. Grabner, H., Grabner, M., & Bischof, H. (2006). Real-time tracking via on-line boosting. In British Machine Vision Conference (BMVC) (pp. 47–56).Google Scholar
  19. Grabner, H., Leistner, C., & Bischof, H. (2008). Semi-supervised on-line boosting for robust tracking. In Proceedings of European Conference on Computer Vision (ECCV) (pp. 234–247).Google Scholar
  20. Hong, Z., Mei, X., Prokhorov, D., & Tao, D. (2013). Tracking via robust multi-task multi-view joint sparse representation. In Proceedings of IEEE Interestial Conference on Computer Vision (ICCV).Google Scholar
  21. Hu, W., Li, X., Zhang, X., Shi, X., Maybank, S., & Zhang, Z. (2010). Incremental tensor subspace learning and its applications to foreground segmentation and tracking. International Journal of Computer Vision, 91(3), 303–327.CrossRefMATHGoogle Scholar
  22. Jia, X., Lu, H., & Yang, M. H. (2012). Visual tracking via adaptive structural local sparse appearance model. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  23. Jiang, N., Liu, W., & Wu, Y. (2011). Learning adaptive metric for robust visual tracking. IEEE Transactions on Image Processing, 20(8), 2288–2300.Google Scholar
  24. Jiang, N., Liu, W., & Wu, Y. (2012). Order determination and sparsity-regularized metric learning for adaptive visual tracking. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  25. Kalal, Z., Mikolajczyk, K., & Matas, J. (2012). Tracking-learning-detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(7), 1409–1422.CrossRefGoogle Scholar
  26. Kwon, J., & Lee, K. M. (2010). Visual tracking decomposition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  27. Kwon, J., & Lee, K. M. (2011). Tracking by sampling trackers. In Proceedings of IEEE International Conference on Computer Vision (ICCV).Google Scholar
  28. Kwon, J., & Lee, K. M. (2013). Highly non-rigid object tracking via patch-based dynamic appearance modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(10), 2427–2441.MathSciNetCrossRefGoogle Scholar
  29. Leichter, I. (2012). Mean shift trackers with cross-bin metrics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 695–706.CrossRefGoogle Scholar
  30. Li, X., Dick, A., Shen, C., van den Hengel, A., & Wang, H. (2013). Incremental learning of 3d-dct compact representations for robust visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(3), 863–881.CrossRefGoogle Scholar
  31. Li, X., Dick, A., Shen, C., Zhang, Z., van den Hengel, A., & Wang, H. (2013). Visual tracking with spatio-temporal dempster-shafer information fusion. IEEE Transactions on Image Processing, 22(8), 3028–3040.MathSciNetCrossRefGoogle Scholar
  32. Li, X., Hu, W., Shen, C., Zhang, Z., Dick, A., & van den Hengel, A. (2013). A survey of appearance models in visual object tracking. ACM Transactions on Intelligent Systems and Technology, 4(4), 58.CrossRefGoogle Scholar
  33. Liu, B., Huang, J., Kulikowski, C., & Yang, L. (2012). Robust visual tracking using local sparse appearance model and k-selection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(5), 861–874.Google Scholar
  34. Ma, J., Qiu, W., Zhao, J., Ma, Y., Yuille, A., & Tu, Z. (2015). Robust l2e estimation of transformation for non-rigid registration. IEEE Transactions on Signal Processing, 63(5), 1115–1129.MathSciNetCrossRefGoogle Scholar
  35. Ma, J., Zhao, J., Tian, J., Yuille, A. L., & Tu, Z. (2014). Robust point matching via vector field consensus. IEEE Transactions on Image Processing, 23(4), 1706–1721.MathSciNetCrossRefGoogle Scholar
  36. Mei, X., & Ling, H. (2011). Robust visual tracking and vehicle classification via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(11), 2259–2272.CrossRefGoogle Scholar
  37. Mei, X., Ling, H., Wu, Y., Blasch, E., & Bai, L. (2011). Minimum error bounded efficient l1 tracker with occlusion detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  38. Ojala, T., Pietikäinen, M., & Mäenpää, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), 971–987.CrossRefMATHGoogle Scholar
  39. Porikli, F., Tuzel, O., & Meer, P. (2006). Covariance tracking using model update based on lie algebra. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  40. Ross, D., Kim, J., Lin, R. S., & Yang, M. H. (2008). Incremental learning for robust visual tracking. International Journal of Computer Vision, 77(1), 125–141.CrossRefGoogle Scholar
  41. Santner, J., Leistner, C., Saffari, A., Pock, T., & Bischof, H. (2010). Prost: Parallel robust online simple tracking. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  42. Sinha, K., & Belkin, M. (2009). Semi-supervised learning using sparse eigenfunction bases. In Advances in Neural Information Processing Systems (NIPS).Google Scholar
  43. Spengler, M., & Schiele, B. (2003). Towards robust multi-cue integration for visual tracking. Machine Vision and Applications, 14, 50–58.CrossRefMATHGoogle Scholar
  44. Stenger, B., Woodley, T., & Cipolla, R. (2009). Learning to track with multiple observers. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  45. Tsagkatakis, G., & Savakis, A. (2011). Online distance metric learning for object tracking. IEEE Transactions on Circuits and Systems for Video Technology, 21(12), 1810–1821.CrossRefGoogle Scholar
  46. Vishwanathan, S., Schraudolph, N., Kondor, R., & Borgwardt, K. (2010). Graph kernels. Journal of Machine Learning Research, 11(4), 1201–1242.MathSciNetMATHGoogle Scholar
  47. Wang, D., Lu, H., & Yang, M. H. (2013). Online object tracking with sparse prototypes. IEEE Transactions on Image Processing, 22(1), 314–325.MathSciNetCrossRefGoogle Scholar
  48. Wang, J., & Yagi, Y. (2008). Integrating color and shape-texture features for adaptive real-time object tracking. IEEE Transactions on Image Processing, 17(2), 235–240.CrossRefGoogle Scholar
  49. Wang, M., Qiao, H., & Zhang, B. (2011). A new algorithm for robust pedestrian tracking based on manifold learning and feature selection. IEEE Transactions on Intelligent Transportation Systems, 12(4), 1195–1208.CrossRefGoogle Scholar
  50. Wang, Q., Chen, F., Xu, W., & Yang, M. H. (2012). Object tracking via partial least squares analysis. IEEE Transactions on Image Processing, 21(10), 4454–4465.MathSciNetCrossRefGoogle Scholar
  51. Wang, S., Lu, H., Yang, F., & Yang, M. H. (2011). Superpixel tracking. In Proceedings of IEEE International Conference on Computer Vision (ICCV).Google Scholar
  52. Wang, X., Hua, G., & Han, T. X. (2010). Discriminative tracking by metric learning. In Proceedings of European Conference on Computer Vision (ECCV).Google Scholar
  53. Wu, Y., Cheng, J., Wang, J., Lu, H., Wang, J., Ling, H., et al. (2012). Real-time probabilistic covariance tracking with efficient model update. IEEE Transactions on Image Processing, 21(5), 2824–2837.MathSciNetCrossRefGoogle Scholar
  54. Wu, Y., & Fan, J. (2009). Contextual flow. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  55. Wu, Y., Lim, J., & Yang, M. H. (2013). Online object tracking: A benchmark. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  56. Yang, M., Hua, G., & Wu, Y. (2009). Context-aware visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(7), 1195–1209.CrossRefGoogle Scholar
  57. Yang, M., Yuan, J., & Wu, Y. (2007). Spatial selection for attentional visual tracking. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  58. Yang, X., & Latecki, L. J. (2011). Affinity learning on a tensor product graph with applications to shape and image retrieval. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Google Scholar
  59. Yang, X., Prasad, L., & Latecki, L. J. (2013). Affinity learning with diffusion on tensor product graph. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 28–38.CrossRefGoogle Scholar
  60. Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking: A survey. ACM Computing Surveys (CSUR), 38(4), 1201–1242.CrossRefGoogle Scholar
  61. Yoon, J. H., Kim, D. Y., & Yoon, K. J. (2012). Visual tracking via adaptive tracker selection with multiple features. In Proceedings of European Conference on Computer Vision (ECCV).Google Scholar
  62. Zelnik-Manor, L., & Perona, P. (2004). Self-tuning spectral clustering. In Advances in Neural Information Processing Systems (NIPS) (pp. 1601–1608).Google Scholar
  63. Zhang, K., Zhang, L., & Yang, M. H. (2012). Real-time compressive tracking. In Proceedings of European Conference on Computer Vision (ECCV).Google Scholar
  64. Zhang, T., Ghanem, B., Liu, S., & Ahuja, N. (2013). Robust visual tracking via structured multi-task sparse learning. International Journal of Computer Vision, 101(2), 367–383.MathSciNetCrossRefGoogle Scholar
  65. Zhao, Q., Yang, Z., & Tao, H. (2010). Differential earth mover’s distance with its applications to visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(2), 274–287.Google Scholar
  66. Zhong, W., Lu, H., & Yang, M. H. (2012). Robust object tracking via sparsity-based collaborative model. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  67. Zhou, D., Bousquet, O., Lal, T., Weston, J., & Scholkopf, B. (2004). Learning with local and global consistency. In Advances in Neural Information Processing Systems (NIPS).Google Scholar
  68. Zhou, Y., Bai, X., Liu, W., & Latecki, L. J. (2012). Fusion with diffusion for robust visual tracking. In Advances in Neural Information Processing Systems (NIPS).Google Scholar
  69. Zhu, X. (2006). Semi-supervised learning literature survey. Technical Report 1530, Department of Computer Sciences, University of Wisconsin, Madison.Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Yu Zhou
    • 1
    • 3
  • Xiang Bai
    • 1
  • Wenyu Liu
    • 1
  • Longin Jan Latecki
    • 2
  1. 1.Huazhong University of Science and TechnologyWuhanPeople’s Republic of China
  2. 2.Temple UniversityPhiladelphiaUSA
  3. 3.Beijing University of Posts and TelecommunicationsBeijingPeople’s Republic of China

Personalised recommendations