Interactive Tracking of 2D Generic Objects with Spacetime Optimization

  • Xiaolin K. Wei
  • Jinxiang Chai
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5302)


We present a continuous optimization framework for interactive tracking of 2D generic objects in a single video stream. The user begins with specifying the locations of a target object in a small set of keyframes; the system then automatically tracks locations of the objects by combining user constraints with visual measurements across the entire sequence. We formulate the problem in a spacetime optimization framework that optimizes over the whole sequence simultaneously. The resulting solution is consistent with visual measurements across the entire sequence while satisfying user constraints. We also introduce prior terms to reduce tracking ambiguity. We demonstrate the power of our algorithm on tracking objects with significant occlusions, scale and orientation changes, illumination changes, sudden movement of objects, and also simultaneous tracking of multiple objects. We compare the performance of our algorithm with alternative methods.


Target Object Object Tracking Illumination Change Candidate Object Interpolation Weight 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-540-88682-2_50_MOESM1_ESM.avi (13.8 mb)
Supplementary material (14,094 KB)


  1. 1.
    Isard, M., Blake, A.: Condensation conditional density propagation for visual tracking. International Journal on Computer Vision 29(1), 5–28 (1998)CrossRefGoogle Scholar
  2. 2.
    Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Trans. Pattern Analysis and Machine Intelligence 25(3), 564–577 (2003)CrossRefGoogle Scholar
  3. 3.
    Witkin, A., Kass, M.: Spacetime constraints. In: Proceedings of ACM SIGGRAPH 1998, pp. 159–168 (1988)Google Scholar
  4. 4.
    Sun, J., Zhang, W., Tang, X., Shum, H.Y.: Bi-directional tracking using trajectory segment analysis. In: Proceedings of ICCV, vol. 1, pp. 717–724 (2005)Google Scholar
  5. 5.
    Buchanan, A., Fitzgibbon, A.: Interactive feature tracking using k-d trees and dynamic programming. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 626–633 (2006)Google Scholar
  6. 6.
    Wei, Y., Sun, J., Tang, X., Shum, H.Y.: Interactive offline tracking for color object. In: Proceedings of ICCV (2007)Google Scholar
  7. 7.
    Elgammal, A., Duraiswami, R., Davis, L.: Probabilistic tracking in joint feature-spatial spaces. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 781–788 (2003)Google Scholar
  8. 8.
    Blackman, S., Popoli, R.: Design and analysis of modern tracking systems. Artech House Publishers (1999)Google Scholar
  9. 9.
    Koller, D., Daniilidis, K., Nage, H.: Model-based object tracking in monocular image sequences of road traffic scenes. International Journal on Computer Vision 10(3), 257–281 (1993)CrossRefGoogle Scholar
  10. 10.
    Rasmussen, C., Hager, G.: Joint probabilistic techniques for tracking multi-part objects. IEEE Trans. Pattern Analysis and Machine Intelligence 23, 560(n)-576 (1993)Google Scholar
  11. 11.
    Jepson, A., Fleet, D., El-Maraghi, T.: Robust online appearance models for visual tracking. IEEE Trans. Pattern Analysis and Machine Intelligence 25(10), 1296–1311 (2003)CrossRefGoogle Scholar
  12. 12.
    Wu, Y., Huang, T.S.: Robust visual tracking by integrating multiple cues based on co-inference learning. International Journal on Computer Vision 58(1), 55–71 (2004)CrossRefGoogle Scholar
  13. 13.
    Guskov, I.: Kernel-based template alignment. In: Proceedings of the IEEE Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 610–617 (2006)Google Scholar
  14. 14.
    Megret, R., Mikram, M., Berthoumieu, Y.: Inverse composition for multi-kernel tracking. In: Computer Vision, Graphics and Image Processing. LNCS, pp. 480–491 (2007)Google Scholar
  15. 15.
    Agarwala, A., Hertzmann, A., Salesin, D.H., Seitz, S.M.: Keyframe-based tracking for rotoscoping and animation. ACM Transactions on Graphics 24(3), 584–591 (2005)CrossRefGoogle Scholar
  16. 16.
    Avidan, S.: Support vector tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(8), 1064–1072 (2004)CrossRefGoogle Scholar
  17. 17.
    Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Analysis and Machine Intelligence 24(5), 603–619 (2002)CrossRefGoogle Scholar
  18. 18.
    Huber, P.: Robust statistics. Wiley, Chichester (1981)CrossRefzbMATHGoogle Scholar
  19. 19.
    Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., Stahel, W.A.: Robust statistics: The approach based on influence functions. Wiley, Chichester (1986)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Xiaolin K. Wei
    • 1
  • Jinxiang Chai
    • 1
  1. 1.Texas A&M UniversityUSA

Personalised recommendations