Skip to main content
SpringerLink
Log in

Soft-assigned bag of features for object tracking

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Hard assignment-based bag of features (BoF) representation inevitably brings in quantization errors, which may lead to inaccuracy, even failure in object tracking. In this paper, we propose a novel soft-assigned BoF tracking approach, in which soft assignment is utilized to improve the robustness and discrimination of BoF representation. After labeling the tracked target, we first randomly sample the circle patches with adaptive size within and outside the labeled target, extract the local features from the patches, and construct the codebooks by k-means clustering. When tracking in a new frame, we generate the BoF representation of each candidate target, and select the most similar candidate target in the previous tracked result based on BoF representation. To improve tracking performance, we also continuously update the codebooks and refine the tracking results. Experiments show that our approach outperforms the state-of-the-art tracking methods under complex tracking conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Zhong, W., Lu, H., Yang M.H.: Robust object tracking via sparsity-based collaborative model. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1838–1845. Providence, USA (2012)

  2. Wang, M., Hong, R., Li, G., Zha, Z.J., Yan, S., Chua, T.S.: Event driven web video summarization by tag localization and key-shot identification. IEEE Trans. Multimedia 14(4), 975–985 (2012)

    Article  Google Scholar 

  3. Zhang, P., Thomas, T., Emmanuel, S.: Privacy enabled video surveillance using a two state Indicatov tracking algorithm. Multimedia Syst. 18(2), 175–199 (2012)

    Article  Google Scholar 

  4. Wachs, J.P., Kölsch, M., Stern, H., Edan, Y.: Vision-based hand-gesture applications. Commun. ACM 54(2), 60–71 (2011)

    Article  Google Scholar 

  5. Wang, M., Hua, X.S., Hong, R., Tang, J., Qi, G.J., Song, Y.: Unified video annotation via multigraph learning. IEEE Trans. Circ. Syst. Video Technol. 19(5), 733–746 (2009)

    Article  Google Scholar 

  6. Park, B.S., Yoo, S.J., Park, J.B., Choi, Y.H.: A simple adaptive control approach for trajectory tracking of electrically driven nonholonomic mobile robots. IEEE Trans. Control Syst. Technol. 18(5), 1199–1206 (2010)

    Article  Google Scholar 

  7. Li, A., Tang, F., Guo, Y., Tao H.: Discriminative nonorthogonal binary subspace tracking. In: European Conference on Computer Vision, pp. 258–271. Heraklion, Greece (2010)

  8. Li, X., Hu, W., Shen, C., Zhang, Z., Dick, A., Hengel, A.: A survey of appearance models in visual object tracking. ACM Trans. Intell. Syst. Technol. 4(4), 1–48 (2013)

  9. Ying, L., Xu, C., Guo W.: Extended MHT algorithm for multiple object tracking. In: International Conference on Internet Multimedia Computing and Service, pp. 75–79. Wuhan, China (2012)

  10. Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., Gong, Y.: Locality-constrained linear coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3360–3367. San Francisco, USA (2010)

  11. Jiang, Y. G., Ngo, C. W., Yang, J.: Towards optimal bag-of-features for object categorization and semantic video retrieval. In: ACM International Conference on Image and Video Retrieval, pp. 494–501. Amsterdam, The Netherlands (2007)

  12. Jégou, H., Douze, M., Schmid, C.: Improving bag-of-features for large scale image search. Int. J. Comput. Vis. 87(3), 316–336 (2010)

    Article  Google Scholar 

  13. Yang, F., Lu, H., Chen, Y.: Bag of features tracking. In: International Conference on Pattern Recognition, pp. 153–156. Istanbul, Turkey (2010)

  14. Yang, F., Lu, H., Zhang, W., Yang, G.: Visual tracking via bag of features. IET Image Process. 6(2), 115–128 (2012)

    Article  MathSciNet  Google Scholar 

  15. Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Lost in quantization: improving particular object retrieval in large scale image databases. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. Anchorage, USA (2008)

  16. Qiu, Z., Yu, T., Ren, T., Liu, Y., Bei, J.: Soft-assigned bag of features tracking. In: International Conference on Internet Multimedia Computing and Service. Huangshan, China (2013)

  17. Pérez, P., Hue, C., Vermaak, J., Gangnet, M.: Color-based probabilistic tracking. In: European Conference on Computer Vision, pp. 661–675. Copenhagen, Denmark (2002)

  18. Allili, M.S., Ziou, D.: Object of interest segmentation and tracking by using feature selection and active contours. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. Minneapolis, USA (2007)

  19. Santner, J., Leistner, C., Saffari, A., Pock, T., Bischof, H.: Prost: Parallel robust online simple tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 723–730. San Francisco, USA (2010)

  20. Hu, W., Li, X., Zhang, X., Shi, X., Maybank, S., Zhang, Z.: Incremental tensor subspace learning and its applications to foreground segmentation and tracking. Int. J. Comput. Vis. 91(3), 303–327 (2011)

    Article  MATH  Google Scholar 

  21. Kim, Z.: Real time object tracking based on dynamic feature grouping with background subtraction. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. Anchorage, USA (2008)

  22. Zhou, H., Yuan, Y., Shi, C.: Object tracking using SIFT features and mean shift. Comput. Vis. Image Underst. 113(3), 345–352 (2009)

    Article  Google Scholar 

  23. He, W., Yamashita, T., Lu, H., Lao, S.: Surf tracking. In: IEEE International Conference on Computer Vision, pp. 1586–1592. Kyoto, Japan (2009)

  24. Wang, S., Lu, H., Yang, F., Yang, M.H.: Superpixel tracking. In: IEEE International Conference on Computer Vision, pp. 1323–1330. Barcelona, Spain (2011)

  25. Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: ECCV Workshop on Statistical Learning in Computer Vision, pp. 1–22. Prague, Czech Republic (2004)

  26. Nowak, E., Jurie, F., Triggs, B.: Sampling strategies for bag-of-features image classification. In: European Conference on Computer Vision, pp. 490–503. Graz, Austria (2006)

  27. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  28. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  29. Manjunath, B.S., Ohm, J.R., Vasudevan, V.V., Yamada, A.: Color and texture descriptors. IEEE Trans. Circ. Syst. Video Technol. 11(6), 703–715 (2001)

    Article  Google Scholar 

  30. Fei-Fei, L., Perona, P.: A bayesian hierarchical model for learning natural scene categories. In: IEEE Computer Vision and Pattern Recognition, pp. 524–531. San Diego, USA (2005)

  31. Wang, M., Hua, X.S., Tang, J., Hong, R.: Beyond distance measurement: Constructing neighborhood similarity for video annotation. IEEE Trans. Multimedia 11(3), 465–476 (2009)

    Article  Google Scholar 

  32. Jiang, Y.G., Yang, J., Ngo, C.W., Hauptmann, A.G.: Representations of keypoint-based semantic concept detection: A comprehensive study. IEEE Trans. Multimedia 12(1), 42–53 (2010)

    Article  Google Scholar 

  33. Gemert, J.C., Geusebroek, J.M., Veenman, C.J., Smeulders, A.W.M.: Kernel codebooks for scene categorization. In: European Conference on Computer Vision, pp. 696–709. Marseille, France (2008)

  34. Zhu, S., Wang, G., Ngo, C. W., Jiang, Y.G.: On the sampling of web images for learning visual concept classifiers. In: ACM International Conference on Image and Video Retrieval, pp. 50–57. Xi’an, China (2010)

  35. Kanungo, T., Mount, D.M., Netanyahu, N.S., Piatko, C.D., Silverman, R., Wu, A.Y.: An efficient k-means clustering algorithm: Analysis and implementation. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 881–892 (2002)

    Article  Google Scholar 

  36. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  Google Scholar 

  37. CAVIAR. http://homepages.inf.ed.ac.uk/rbf/caviar/

  38. Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral histogram. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 798–805. New York, USA (2006)

  39. Ross, D., Lim, J., Lin, R., Yang, M.: Incremental learning for robust visual tracking. Int. J. Comput. Vis. 77(1–3), 125–141 (2008)

    Article  Google Scholar 

  40. Babenko, B., Yang, M.H., Belongie, S.: Visual tracking with online multiple instance learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 983–990. Miami, USA (2009)

  41. Kwon, J., Lee, K.M.: Visual tracking decomposition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1269–1276. San Francisco, USA (2010)

  42. Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking learning detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012)

    Article  Google Scholar 

  43. Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

Download references

Acknowledgments

The authors want to thank the anonymous reviews for their helpful suggestion, and Tao Huang for his contribution in experiment. This paper is supported by Natural Science Foundation of China (61202320), Research Project of Excellent State Key Laboratory (61223003), and Natural Science Foundation of Jiangsu Province (BK2012304).

Author information

Authors and Affiliations

  1. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China

    Tongwei Ren

  2. School of Software, Nanchang University, Nanchang, China

    Zhongyan Qiu

  3. Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China

    Yan Liu

  4. Software Institute, Nanjing University, Nanjing, China

    Tongwei Ren, Tong Yu & Jia Bei

Authors
  1. Tongwei Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhongyan Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jia Bei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, T., Qiu, Z., Liu, Y. et al. Soft-assigned bag of features for object tracking. Multimedia Systems 21, 189–205 (2015). https://doi.org/10.1007/s00530-014-0384-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-014-0384-y

Keywords

Search

Navigation