Skip to main content
SpringerLink
Log in

Incremental Learning for Robust Visual Tracking

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Visual tracking, in essence, deals with non-stationary image streams that change over time. While most existing algorithms are able to track objects well in controlled environments, they usually fail in the presence of significant variation of the object’s appearance or surrounding illumination. One reason for such failures is that many algorithms employ fixed appearance models of the target. Such models are trained using only appearance data available before tracking begins, which in practice limits the range of appearances that are modeled, and ignores the large volume of information (such as shape changes or specific lighting conditions) that becomes available during tracking. In this paper, we present a tracking method that incrementally learns a low-dimensional subspace representation, efficiently adapting online to changes in the appearance of the target. The model update, based on incremental algorithms for principal component analysis, includes two important features: a method for correctly updating the sample mean, and a forgetting factor to ensure less modeling power is expended fitting older observations. Both of these features contribute measurably to improving overall tracking performance. Numerous experiments demonstrate the effectiveness of the proposed tracking algorithm in indoor and outdoor environments where the target objects undergo large changes in pose, scale, and illumination.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Adelson, E. H., & Bergen, J. R. (1991). The plenoptic function and the elements of early vision. In M. Landy & J. A. Movshon (Eds.), Computational models of visual processing (pp. 1–20). Cambridge: MIT Press.

    Google Scholar 

  • Avidan, S. (2001). Support vector tracking. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 184–191).

  • Belhumeur, P., & Kreigman, D. (1997). What is the set of images of an object under all possible lighting conditions. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 270–277).

  • Birchfield, S. (1998). Elliptical head tracking using intensity gradient and color histograms. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 232–237).

  • Black, M. J., & Jepson, A. D. (1996). Eigentracking: robust matching and tracking of articulated objects using view-based representation. In B. Buxton & R. Cipolla (Eds.), LNCS : Vol. 1064. Proceedings of the fourth European conference on computer vision (pp. 329–342). Berlin: Springer.

    Google Scholar 

  • Black, M. J., Fleet, D. J., & Yacoob, Y. (1998). A framework for modeling appearance change in image sequence. In Proceedings of IEEE international conference on computer vision (pp. 660–667).

  • Brand, M. (2002). Incremental singular value decomposition of uncertain data with missing values. In A. Heyden, G. Sparr, M. Nielsen & P. Johansen (Eds.), LNCS : Vol. 2350. Proceedings of the seventh European conference on computer vision (pp. 707–720). Berlin: Springer.

    Google Scholar 

  • Comaniciu, D., Ramesh, V., & Meer, P. (2003). Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(5), 564–577.

    Article  Google Scholar 

  • Cootes, T., Edwards, G., & Taylor, C. (2001). Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 681–685.

    Article  Google Scholar 

  • Georgescu, B., Comaniciu, D., Han, T. X., & Zhou, X. S. (2004). Multi-model component-based tracking using robust information fusion. In 2nd workshop on statistical methods in video processing, May 2004

  • Golub, G. H., & Van Loan, C. F. (1996) Matrix computations. The Johns Hopkins University Press.

  • Hager, G., & Belhumeur, P. (1996) Real-time tracking of image regions with changes in geometry and illumination. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 403–410).

  • Hall, P., Marshall, D., & Martin, R. (1998). Incremental eigenanalysis for classification. In Proceedings of British machine vision conference (pp. 286–295).

  • Hall, P., Marshall, D., & Martin, R. (2002). Adding and subtracting eigenspaces with eigenvalue decomposition and singular value decomposition. Image and Vision Computing, 20(13–14), 1009–1016.

    Article  Google Scholar 

  • Harville, M. (2002). A framework for high-level feedback to adaptive, per-pixel mixture of Gaussian background models. In A. Heyden, G. Sparr, M. Nielsen & P. Johansen (Eds.), LNCS : Vol. 2352. Proceedings of the seventh European conference on computer vision (pp. 531–542). Berlin: Springer.

    Google Scholar 

  • Isard, M., & Blake, A. (1996). Contour tracking by stochastic propagation of conditional density. In B. Buxton & R. Cipolla (Eds.), LNCS : Vol. 1064. Proceedings of the fourth European conference on computer vision (pp. 343–356). Berlin: Springer.

    Google Scholar 

  • Jepson, A. D., Fleet, D. J., & El-Maraghi, T. F. (2003). Robust online appearance models for visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10), 1296–1311.

    Article  Google Scholar 

  • Jolliffe, I. T. (2002). Principal component analysis. Berlin: Springer.

    MATH  Google Scholar 

  • La Cascia, M., & Sclaroff, S. (1999). Fast, reliable head tracking under varying illumination. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 604–608).

  • Levy, A., & Lindenbaum, M. (2000). Sequential Karhunen–Loeve basis extraction and its application to images. IEEE Transactions on Image Processing, 9(8), 1371–1374.

    Article  MATH  Google Scholar 

  • Lim, J., Ross, D., Lin, R.-S., & Yang, M.-H. (2005). Incremental learning for visual tracking. In L. Saul, Y. Weiss & L. Bottou (Eds.), Advances in neural information processing systems (pp. 793–800). Cambridge: MIT Press.

    Google Scholar 

  • Lin, R.-S., Ross, D., Lim, J., & Yang, M.-H. (2005). Adaptive discriminative generative model and its applications. In L. Saul, Y. Weiss & L. Bottou (Eds.), Advances in neural information processing systems (pp. 801–808). Cambridge: MIT Press.

    Google Scholar 

  • Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In Proceedings of international joint conference on artificial intelligence (pp. 674–679).

  • Matthews, I., Ishikawa, T., & Baker, S. (2004). The template update problem. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(6), 810–815.

    Article  Google Scholar 

  • Moghaddam, B., & Pentland, A. (1995). Probabilistic visual learning for object detection. In Proceedings of IEEE international conference on computer vision (pp. 786–793).

  • Murase, H., & Nayar, S. (1995). Visual learning and recognition of 3d objects from appearance. International Journal of Computer Vision, 14(1), 5–24.

    Article  Google Scholar 

  • North, B., & Blake, A. (1998). Learning dynamical models using expectation-maximization. In Proceedings of IEEE international conference on computer vision (pp. 384–389).

  • Rasmussen, C., & Hager, G. (1998). Joint probabilistic techniques for tracking multi-part objects. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 16–21).

  • Ross, D., Lim, J., & Yang, M.-H. (2004). Adaptive probabilistic visual tracking with incremental subspace update. In A. Heyden, G. Sparr, M. Nielsen & P. Johansen (Eds.), LNCS : Vol. 2350. Proceedings of the eighth European conference on computer vision (pp. 707–720). Berlin: Springer.

    Google Scholar 

  • Roweis, S. (1997). EM algorithms for PCA and SPCA. In M. I. Jordan, M. J. Kearns & S. A. Solla (Eds.), Advances in neural information processing systems 10 (pp. 626–632). Cambridge: MIT Press.

    Google Scholar 

  • Tipping, M. E., & Bishop, C. M. (1999). Probabilistic principal component analysis. Journal of the Royal Statistical Society, Series B, 61(3), 611–622.

    Article  MATH  MathSciNet  Google Scholar 

  • Toyama, K., & Blake, A. (2001). Probabilistic tracking in metric space. In Proceedings of IEEE international conference on computer vision (pp. 50–57).

  • Vermaak, J., Lawrence, N., & Perez, P. (2003). Variational inference for visual tracking. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 773–780).

  • Williams, O., Blake, A., & Cipolla, R. (2003). A sparse probabilistic learning algorithms for real-time tracking. In Proceedings of IEEE international conference on computer vision (Vol. 1, pp. 353–360).

Download references

Author information

Authors and Affiliations

  1. University of Toronto, 10 Kings College Road, Toronto, ON, M55 3G4, Canada

    David A. Ross

  2. Honda Research Institute, 800 California Street, Mountain View, CA, 94041, USA

    Jongwoo Lim & Ming-Hsuan Yang

  3. Motorola Labs, 1303 E Algonquin Rd., Schaumburg, IL, 60196, USA

    Ruei-Sung Lin

Authors
  1. David A. Ross
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jongwoo Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruei-Sung Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ming-Hsuan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David A. Ross.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ross, D.A., Lim, J., Lin, RS. et al. Incremental Learning for Robust Visual Tracking. Int J Comput Vis 77, 125–141 (2008). https://doi.org/10.1007/s11263-007-0075-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-007-0075-7

Keywords

Search

Navigation