International Journal of Computer Vision

, Volume 101, Issue 2, pp 367-383

First online:

Robust Visual Tracking via Structured Multi-Task Sparse Learning

  • Tianzhu ZhangAffiliated withAdvanced Digital Sciences Center (ADSC)
  • , Bernard GhanemAffiliated withKing Abdullah University of Science and Technology (KAUST)
  • , Si LiuAffiliated withDepartment of Electrical and Computer Engineering, National University of Singapore Email author 
  • , Narendra AhujaAffiliated withDepartment of Electrical and Computer Engineering, Beckman Institute, and Coordinated Science Laboratory, University of Illinois at Urbana-Champaign

Rent the article at a discount

Rent now

* Final gross prices may vary according to local VAT.

Get Access


In this paper, we formulate object tracking in a particle filter framework as a structured multi-task sparse learning problem, which we denote as Structured Multi-Task Tracking (S-MTT). Since we model particles as linear combinations of dictionary templates that are updated dynamically, learning the representation of each particle is considered a single task in Multi-Task Tracking (MTT). By employing popular sparsity-inducing \(\ell _{p,q}\) mixed norms \((\text{ specifically} p\in \{2,\infty \}\) and \(q=1),\) we regularize the representation problem to enforce joint sparsity and learn the particle representations together. As compared to previous methods that handle particles independently, our results demonstrate that mining the interdependencies between particles improves tracking performance and overall computational complexity. Interestingly, we show that the popular \(L_1\) tracker (Mei and Ling, IEEE Trans Pattern Anal Mach Intel 33(11):2259–2272, 2011) is a special case of our MTT formulation (denoted as the \(L_{11}\) tracker) when \(p=q=1.\) Under the MTT framework, some of the tasks (particle representations) are often more closely related and more likely to share common relevant covariates than other tasks. Therefore, we extend the MTT framework to take into account pairwise structural correlations between particles (e.g. spatial smoothness of representation) and denote the novel framework as S-MTT. The problem of learning the regularized sparse representation in MTT and S-MTT can be solved efficiently using an Accelerated Proximal Gradient (APG) method that yields a sequence of closed form updates. As such, S-MTT and MTT are computationally attractive. We test our proposed approach on challenging sequences involving heavy occlusion, drastic illumination changes, and large pose variations. Experimental results show that S-MTT is much better than MTT, and both methods consistently outperform state-of-the-art trackers.


Visual tracking Particle filter Graph Structure Sparse representation Multi-task learning