International Journal of Computer Vision

, Volume 101, Issue 2, pp 367–383

Robust Visual Tracking via Structured Multi-Task Sparse Learning

  • Tianzhu Zhang
  • Bernard Ghanem
  • Si Liu
  • Narendra Ahuja
Article

Abstract

In this paper, we formulate object tracking in a particle filter framework as a structured multi-task sparse learning problem, which we denote as Structured Multi-Task Tracking (S-MTT). Since we model particles as linear combinations of dictionary templates that are updated dynamically, learning the representation of each particle is considered a single task in Multi-Task Tracking (MTT). By employing popular sparsity-inducing \(\ell _{p,q}\) mixed norms \((\text{ specifically} p\in \{2,\infty \}\) and \(q=1),\) we regularize the representation problem to enforce joint sparsity and learn the particle representations together. As compared to previous methods that handle particles independently, our results demonstrate that mining the interdependencies between particles improves tracking performance and overall computational complexity. Interestingly, we show that the popular \(L_1\) tracker (Mei and Ling, IEEE Trans Pattern Anal Mach Intel 33(11):2259–2272, 2011) is a special case of our MTT formulation (denoted as the \(L_{11}\) tracker) when \(p=q=1.\) Under the MTT framework, some of the tasks (particle representations) are often more closely related and more likely to share common relevant covariates than other tasks. Therefore, we extend the MTT framework to take into account pairwise structural correlations between particles (e.g. spatial smoothness of representation) and denote the novel framework as S-MTT. The problem of learning the regularized sparse representation in MTT and S-MTT can be solved efficiently using an Accelerated Proximal Gradient (APG) method that yields a sequence of closed form updates. As such, S-MTT and MTT are computationally attractive. We test our proposed approach on challenging sequences involving heavy occlusion, drastic illumination changes, and large pose variations. Experimental results show that S-MTT is much better than MTT, and both methods consistently outperform state-of-the-art trackers.

Keywords

Visual tracking Particle filter Graph Structure Sparse representation Multi-task learning 

Supplementary material

11263_2012_582_MOESM10_ESM.avi (1.6 mb)
Supplementary material 1 (avi 1635 KB)
11263_2012_582_MOESM11_ESM.avi (1.5 mb)
Supplementary material 2 (avi 1575 KB)
11263_2012_582_MOESM12_ESM.avi (1.8 mb)
Supplementary material 3 (avi 1890 KB)
11263_2012_582_MOESM13_ESM.avi (1.7 mb)
Supplementary material 4 (avi 1777 KB)
11263_2012_582_MOESM14_ESM.avi (4.4 mb)
Supplementary material 5 (avi 4539 KB)
11263_2012_582_MOESM15_ESM.avi (2.9 mb)
Supplementary material 6 (avi 2923 KB)
11263_2012_582_MOESM1_ESM.avi (2.2 mb)
Supplementary material 7 (avi 2204 KB)
11263_2012_582_MOESM2_ESM.avi (3.1 mb)
Supplementary material 8 (avi 3211 KB)
11263_2012_582_MOESM3_ESM.avi (1.6 mb)
Supplementary material 9 (avi 1682 KB)
11263_2012_582_MOESM4_ESM.avi (2.6 mb)
Supplementary material 10 (avi 2641 KB)
11263_2012_582_MOESM5_ESM.avi (4.4 mb)
Supplementary material 11 (avi 4459 KB)
11263_2012_582_MOESM6_ESM.avi (5.3 mb)
Supplementary material 12 (avi 5427 KB)
11263_2012_582_MOESM7_ESM.avi (1.6 mb)
Supplementary material 13 (avi 1603 KB)
11263_2012_582_MOESM8_ESM.avi (2.8 mb)
Supplementary material 14 (avi 2836 KB)
11263_2012_582_MOESM9_ESM.avi (2.7 mb)
Supplementary material 15 (avi 2762 KB)

References

  1. Adam, A., Rivlin, E.,& Shimshoni, I. (2006). Robust fragments-based tracking using the integral histogram. In IEEE conference on computer vision and pattern recognition (pp. 798–805).Google Scholar
  2. Avidan, S. (2005). Ensemble tracking. In IEEE conference on computer vision and pattern recognition (pp. 494–501).Google Scholar
  3. Babenko, B., Yang, M. H.,& Belongie, S. (2009). Visual tracking with online multiple instance learning. In IEEE conference on computer vision and pattern recognition (pp. 983–990).Google Scholar
  4. Bao, C., Wu, Y., Ling, H.,& Ji, H. (2012). Real time robust l1 tracker using accelerated proximal gradient approach. In IEEE conference on computer vision and pattern recognition (pp. 1–8).Google Scholar
  5. Beck, A.,& Teboulle, M. (2009). A fast iterative shrinkagethresholding algorithm for linear inverse problems. SIAM Journal on Imaging Science, 2(1), 183–202.Google Scholar
  6. Black, M. J.,& Jepson, A. D. (1998). Eigentracking: Robust matching and tracking of articulated objects using a view-based representation. International Journal of Computer Vision, 26(1), 63–84.Google Scholar
  7. Blasch, E.,& Kahler, B. (2005). Multiresolution EO/IR target tracking and identification. In International conference on information fusion (Vol. 8, pp. 1–8).Google Scholar
  8. Candès, E. J., Romberg, J. K.,& Tao, T. (2006). Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics, 59(8), 1207–1223.Google Scholar
  9. Chen, X., Pan, W., Kwok, J.,& Carbonell, J. (2009). Accelerated gradient method for multi-task sparse learning problem. In IEEE international conference on data mining (pp. 746–751).Google Scholar
  10. Collins, R. T.,& Liu, Y. (2003). On-line selection of discriminative tracking features. In International conference on computer vision (pp. 346–352).Google Scholar
  11. Comaniciu, D., Ramesh, V.,& Meer, P. (2003). Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(5), 564–575.Google Scholar
  12. Doucet, A., De Freitas, N.,& Gordon, N. (2001). Sequential Monte Carlo methods in practice (1st ed.). Springer.Google Scholar
  13. Grabner, H., Grabner, M.,& Bischof, H. (2006). Real-time tracking via on-line boosting. In British machine vision conference (pp. 1–10).Google Scholar
  14. Grabner, H., Leistner, C.,& Bischof, H. (2008). Semi-supervised on-line boosting for robust tracking. In European conference on computer vision (pp. 234–247).Google Scholar
  15. Isard, M.,& Blake, A. (1998). Condensation—Conditional density propagation for visual tracking. International Journal of Computer Vision, 29(1), 5–28.Google Scholar
  16. Jepson, A., Fleet, D.,& El-Maraghi, T. (2003). Robust on-line appearance models for visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10), 1296–1311.Google Scholar
  17. Jiang, N., Liu, W.,& Wu, Y. (2011). Adaptive and discriminative metric differential tracking. In IEEE conference on computer vision and pattern recognition (pp. 1161–1168).Google Scholar
  18. Khan, Z., Balch, T.,& Dellaert, F. (2004). A rao-blackwellized particle filter for eigentracking. In IEEE conference on computer vision and pattern recognition (pp. 980–986).Google Scholar
  19. Kwon, J.,& Lee, K. M. (2010). Visual tracking decomposition. In IEEE conference on computer vision and pattern recognition (pp. 1269–1276).Google Scholar
  20. Leistner, C., Godec, M., Saffari, A.,& Bischof, H. (2010). Online multi-view forests for tracking. In DAGM (pp. 493–502).Google Scholar
  21. Li, H., Shen, C.,& Shi, Q. (2011). Real-time visual tracking with compressed sensing. In IEEE conference on computer vision and pattern recognition (pp. 1305–1312).Google Scholar
  22. Liu, B., Huang, J., Yang, L.,& Kulikowski, C. (2011). Robust visual tracking with local sparse appearance model and k-selection. In IEEE conference on computer vision and pattern recognition (pp. 1–8).Google Scholar
  23. Liu, B., Yang, L., Huang, J., Meer, P., Gong, L.,& Kulikowski, C. (2010). Robust and fast collaborative tracking with two stage sparse optimization. In European conference on computer vision (pp. 1–14).Google Scholar
  24. Liu, R., Cheng, J.,& Lu, H. (2009). A robust boosting tracker with minimum error bound in a co-training framework. In International conference on computer vision (pp. 1459–1466).Google Scholar
  25. Mei, X.,& Ling, H. (2011). Robust visual tracking and vehicle classification via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(11), 2259–2272.Google Scholar
  26. Mei, X., Ling, H., Wu, Y., Blasch, E.,& Bai, L. (2011). Minimum error bounded efficient l1 tracker with occlusion detection. In IEEE conference on computer vision and pattern recognition (pp. 1257–1264).Google Scholar
  27. Nesterov, Y. (2007). Gradient methods for minimizing composite objective function. In CORE discussion paper.Google Scholar
  28. Peng, Y., Ganesh, A., Wright, J., Xu, W.,& Ma, Y. (2012). RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 2233–2246. Google Scholar
  29. Quattoni, A., Carreras, X., Collins, M.,& Darrell, T. (2009). An efficient projection for l 1, infinity regularization. In International conference on machine learning (pp. 857–864).Google Scholar
  30. Ross, D., Lim, J., Lin, R. S.,& Yang, M. H. (2008). Incremental learning for robust visual tracking. International Journal of Computer Vision, 77(1), 125–141.Google Scholar
  31. Tseng, P. (2008). On accelerated proximal gradient methods for convex–concave optimization. Technical report. http://pages.cs.wisc.edu/~brecht/cs726docs/Tseng.APG.pdf.
  32. Wu, Y.,& Huang, T. S. (2004). Robust visual tracking by integrating multiple cues based on co-inference learning. International Journal of Computer Vision, 58, 55–71.Google Scholar
  33. Yang, C., Duraiswami, R.,& Davis, L. (2005). Fast multiple object tracking via a hierarchical particle filter. In International conference on computer vision (pp. 212–219).Google Scholar
  34. Yang, M., Wu, Y.,& Hua, G. (2009). Context-aware visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(7), 1195–1209.Google Scholar
  35. Yilmaz, A., Javed, O.,& Shah, M. (2006). Object tracking: A survey. ACM Computing Surveys, 38(4), 13.Google Scholar
  36. Yin, Z.,& Collins, R. (2008). Object tracking and detection after occlusion via numerical hybrid local and global mode-seeking. In IEEE conference on computer vision and pattern recognition (pp. 1–8).Google Scholar
  37. Yu, Q., Dinh, T. B.,& Medioni, G. (2008). Online tracking and reacquisition using co-trained generative and discriminative trackers. In European conference on computer vision (pp. 678–691).Google Scholar
  38. Yuan, X.,& Yan, S. (2010). Visual classification with multi-task joint sparse representation. In IEEE conference on computer vision and pattern recognition (pp. 3493–3500).Google Scholar
  39. Zhang, T., Ghanem, B., Liu, S.,& Ahuja, N. (2012a). Low-rank sparse learning for robust visual tracking. In European conference on computer vision (pp. 1–8).Google Scholar
  40. Zhang, T., Ghanem, B., Liu, S.,& Ahuja, N. (2012b). Robust visual tracking via multi-task sparse learning. In IEEE conference on computer vision and pattern recognition (pp. 1–8).Google Scholar
  41. Zhou, S. K., Chellappa, R.,& Moghaddam, B. (2004). Visual tracking and recognition using appearance-adaptive models in particle filters. IEEE Transactions on Image Processing, 11(1), 1491–1506.Google Scholar
  42. Zhu, X. (2008). Semi-supervised learning literature survey. Computer sciences technical report 1530, University of Madison.Google Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  • Tianzhu Zhang
    • 1
  • Bernard Ghanem
    • 2
  • Si Liu
    • 3
  • Narendra Ahuja
    • 4
  1. 1.Advanced Digital Sciences Center (ADSC)SingaporeSingapore
  2. 2.King Abdullah University of Science and Technology (KAUST)ThuwalKingdom of Saudi Arabia
  3. 3.Department of Electrical and Computer EngineeringNational University of SingaporeSingaporeSingapore
  4. 4.Department of Electrical and Computer Engineering, Beckman Institute, and Coordinated Science LaboratoryUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations