Efficient Visual Object Tracking with Online Nearest Neighbor Classifier

  • Steve Gu
  • Ying Zheng
  • Carlo Tomasi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6492)


A tracking-by-detection framework is proposed that combines nearest-neighbor classification of bags of features, efficient subwindow search, and a novel feature selection and pruning method to achieve stability and plasticity in tracking targets of changing appearance. Experiments show that near-frame-rate performance is achieved (sans feature detection), and that the state of the art is improved in terms of handling occlusions, clutter, changes of scale, and of appearance. A theoretical analysis shows why nearest neighbor works better than more sophisticated classifiers in the context of tracking.


Near Neighbor Background Clutter Sift Descriptor Voronoi Region Appearance Change 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: IJCAI, pp. 674–679 (1981)Google Scholar
  2. 2.
    Shi, J., Tomasi, C.: Good features to track. In: IEEE CVPR, pp. 593–600 (1994)Google Scholar
  3. 3.
    Isard, M., Blake, A.: A smoothing filter for CONDENSATION. In: Burkhardt, H.-J., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1406, pp. 767–781. Springer, Heidelberg (1998)Google Scholar
  4. 4.
    Comaniciu, D., Ramesh, V., Meer, P.: Real-time tracking of non-rigid objects using mean shift. In: IEEE CVPR, vol. 2, pp. 142–149 (2000)Google Scholar
  5. 5.
    Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral histogram. In: IEEE CVPR, pp. 798–805 (2006)Google Scholar
  6. 6.
    Avidan, S.: Ensemble tracking. IEEE PAMI 29, 261–271 (2007)CrossRefGoogle Scholar
  7. 7.
    Li, Y., Ai, H., Yamashita, T., Lao, S., Kawade, M.: Tracking in low frame rate video: A cascade particle filter with discriminative observers of different life spans. IEEE PAMI 30, 1728–1740 (2008)CrossRefGoogle Scholar
  8. 8.
    Özuysal, M., Calonder, M., Lepetit, V., Fua, P.: Fast keypoint recognition using random ferns. IEEE Trans. Pattern Anal. Mach. Intell. 32, 448–461 (2010)CrossRefGoogle Scholar
  9. 9.
    Lowe, D.: Object recognition from local scale-invariant features. In: ICCV, pp. 1150–1157 (1999)Google Scholar
  10. 10.
    Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Viola, P., Platt, J., Zhang, C.: Multiple instance boosting for object detection. In: NIPS (2005)Google Scholar
  12. 12.
    Dietterich, T., Lathrop, R., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89, 31–71 (1997)CrossRefzbMATHGoogle Scholar
  13. 13.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE CVPR, pp. 886–893 (2005)Google Scholar
  14. 14.
    Felzenszwalb, P., McAllester, D., Ramanan, D.: A discriminatively trained, multiscale, deformable part model. In: IEEE CVPR (2008)Google Scholar
  15. 15.
    Lampert, C., Blaschko, M., Hofmann, T.: Efficient subwindow search: A branch and bound framework for object localization. IEEE PAMI 31, 2129–2142 (2009)CrossRefGoogle Scholar
  16. 16.
    Everingham, M., Gool, L., Williams, C., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge (VOC 2009) Results (2009),
  17. 17.
    Tomasi, C., Petrov, S., Sastry, A.: 3d tracking = classification + interpolation. In: ICCV, pp. 1441–1448 (2003)Google Scholar
  18. 18.
    Grabner, H., Bischof, H.: On-line boosting and vision. In: IEEE CVPR, pp. 260–267 (2006)Google Scholar
  19. 19.
    Grabner, H., Leistner, C., Bischof, H.: Semi-supervised on-line boosting for robust tracking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 234–247. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  20. 20.
    Babenko, B., Yang, M., Belongie, S.: Visual tracking with online multiple instance learning. In: IEEE CVPR, pp. 983–990 (2009)Google Scholar
  21. 21.
    Santner, J., Leistner, C., Saffari, A., Pock, T., Bischof, H.: PROST Parallel Robust Online Simple Tracking. In: IEEE CVPR (2010)Google Scholar
  22. 22.
    Tian, M., Zhang, W., Liu, F.: On-line ensemble SVM for robust object tracking. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007, Part I. LNCS, vol. 4843, pp. 355–364. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  23. 23.
    Zhao, X., Liu, Y.: Generative estimation of 3D human pose using shape contexts matching. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007, Part I. LNCS, vol. 4843, pp. 419–429. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  24. 24.
    Prakash, C., Paluri, B., Nalin Pradeep, S., Shah, H.: Fragments based parametric tracking. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007, Part I. LNCS, vol. 4843, pp. 522–531. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  25. 25.
    Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic huber-l1 optical flow. In: BMVC (2009)Google Scholar
  26. 26.
    Breiman, L.: Random forests. Mach. Learning 45, 5–32 (2001)CrossRefzbMATHGoogle Scholar
  27. 27.
    Boiman, O., Shechtman, E., Irani, M.: In defense of nearest-neighbor based image classification. In: IEEE CVPR (2008)Google Scholar
  28. 28.
    Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: IEEE CVPR (2007)Google Scholar
  29. 29.
    Gu, S., Zheng, Y., Tomasi, C.: Critical nets and beta-stable features for image matching. In: ECCV, pp. 663–676 (2010)Google Scholar
  30. 30.
    Vedaldi, A., Fulkerson, B.: VLFeat: An open and portable library of computer vision algorithms (2008),
  31. 31.
    Birchfield, S.: Elliptical head tracking using intensity gradients and color histograms. In: IEEE CVPR, pp. 232–237 (1998)Google Scholar
  32. 32.
    Ross, D., Lim, J., Lin, R., Yang, M.: Incremental learning for robust visual tracking. IJCV 77, 125–141 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Steve Gu
    • 1
  • Ying Zheng
    • 1
  • Carlo Tomasi
    • 1
  1. 1.Department of Computer ScienceDuke UniversityUSA

Personalised recommendations