Journal of Signal Processing Systems

, Volume 65, Issue 1, pp 63–79 | Cite as

A Robust Particle Filter-Based Method for Tracking Single Visual Object Through Complex Scenes Using Dynamical Object Shape and Appearance Similarity

  • Zulfiqar Hasan Khan
  • Irene Yu-Hua Gu
  • Andrew G. Backhouse


This paper addresses the issue of tracking a single visual object through crowded scenarios, where a target object may be intersected or partially occluded by other objects for a long duration, experience severe deformation and pose changes, and different motion speed in cluttered background. A robust visual object tracking scheme is proposed that exploits the dynamics of object shape and appearance similarity. The method uses a particle filter where a multi-mode anisotropic mean shift is embedded to improve the initial particles. Comparing with the conventional particle filter and mean shift-based tracking (Shan et al. 2004), our method offers the following novelties: We employ a fully tunable rectangular bounding box described by five parameters (2D central location, width, height, and orientation) and full functionaries in the joint tracking scheme; We derive the equations for the multi-mode version of the anisotropic mean shift where the rectangular bounding box is partitioned into concentric areas, allowing better tracking objects with multiple modes. The bounding box parameters are then computed by using eigen-decomposition of mean shift estimates and weighted averaging. This enables a more efficient re-distributions of initial particles towards locations associated with large weights, hence an efficient particle filter tracking using a very small number of particles (N = 15 is used). Experiments have been conducted on video containing a range of complex scenarios, where tracking results are further evaluated by using two objective criteria and compared with two existing tracking methods. Our results have shown that the propose method is robust in terms of tracking drift, tightness and accuracy of tracked bounding boxes, especially in scenarios where the target object contains long-term partial occlusions, intersections, severe deformation, pose changes, or cluttered background with similar color distributions.


Anisotropic mean shift Particle filters Multiple modes Visual tracking Bounding box partition 



This work was partially supported by National Engineering and Scientific Commission (NESCOM) in Pakistan, and Chalmers University of Technology in Sweden.


  1. 1.
    Shan, C., Wei, Y., Tan, T., & Ojardias, F. (2004). Real time hand tracking by combining particle filtering and mean shift. In Proc. ieee int. conf. automatic face and gesture recognition (pp. 669–674).Google Scholar
  2. 2.
    Comaniciu, D., Ramesh, V., & Meer, P. (2003). Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 5, 564–577.CrossRefGoogle Scholar
  3. 3.
    Wang, T., Gu, I. Y. H., & Shi, P. (2007). Object tracking using incremental 2D-PCA learning and Ml Eestimation. In Proc. IEEE int. conf. acoustics, speech and signal processing.Google Scholar
  4. 4.
    Wang, T., Gu, I. Y. H., Backhouse, A., & Shi, P. (2008). Face tracking using Rao-Blackwellized particle filter and pose-dependent probabilistic PCA. In Proc. IEEE int. conf. image processing (pp. 853–856).Google Scholar
  5. 5.
    Toyama, K., Krumm, J., Brumitt, B., & Meyers, B. (1999). Wallflower: Principles and practice of background maintenance. In Proc. IEEE int. conf. computer vision (pp. 255–261).Google Scholar
  6. 6.
    Gavrila, D.(1999). The visual analysis of human movement: A survey. Computer Vision and Image Understanding, 73(1), 82–98.zbMATHCrossRefGoogle Scholar
  7. 7.
    Li, L., Huang, W., Gu, I. Y. H., & Tian, Q. (2004). Statistical modeling of complex background for foreground object detection. IEEE Transactions on Image Processing, 13(11), 1459–1472.CrossRefGoogle Scholar
  8. 8.
    Welch, G., & Bishop, G. (1997). Scaat: Incremental tracking with incomplete information. In Proc. int. conf. computer graphics and interactive techniques.Google Scholar
  9. 9.
    Shi, J., & Tomasi, C. (1994). Good features to track. In Proc. IEEE computer society conf. comp. vision and pattern recognition (pp. 593–600).Google Scholar
  10. 10.
    Gordon, N. J., Doucet, A., & Freitas, N. D. (2000). On sequential monte carlo sampling methods for bayesian filtering. Statistics and Computing, 10, 197–208.CrossRefGoogle Scholar
  11. 11.
    Rosales, R., & Sclaroff S. (1999). 3D trajectory recovery for tracking multiple objects and trajectory guided recognition of actions. In Proc. IEEE computer society conf. computer vision and pattern recognition (pp. 117–123).Google Scholar
  12. 12.
    Doucet, A., Freitas, N. D., & Gordon, N. (2001). Sequential Monte Carlo methods in practice. New York: Springer.zbMATHGoogle Scholar
  13. 13.
    Bar-Shalom, Y., & Fortmann, T. (1988). Tracking and data association. New York: Academic.zbMATHGoogle Scholar
  14. 14.
    Reid, D. B. (1979). An algorithm for tracking multiple targets. IEEE Transactions on Automatic Control, 24(2), 843–854.CrossRefGoogle Scholar
  15. 15.
    Vermaak, J., Doucet, A., & Perez, P. (2003). Maintaining multimodality through mixture tracking. In Proc. int. conf. computer vision (pp. 1110–1116).Google Scholar
  16. 16.
    Okuma, K., Taleghani, A., Freitas, N., Little, J. J., & Lowe, D. G. (2004). A boosted particle filter: Multitarget detection and tracking. In Proc. european conf. computer. vision (pp. 28–39).Google Scholar
  17. 17.
    Li, L., Huang, W., Gu, I. Y. H., Luo, R., & Tian, Q. (2008). An efficient sequential approach to tracking multiple objects through crowds for real-time intelligent CCTV systems. IEEE Transactions on Systems, Man and Cybernetics. Part B., 23(1), 13–23.Google Scholar
  18. 18.
    Collins, R. T. (2003). Mean-shift blob tracking through scale space. In Proc. IEEE comp. society conf. computer vision and pattern recognition (Vol. 2, pp. 234–240).Google Scholar
  19. 19.
    Bretzner, L., & Lindeberg, T. (1998). Feature tracking with automatic selection of spatial scales. In Proc. comp. vision and image understanding (Vol. 71, pp. 385–392).Google Scholar
  20. 20.
    Yilmaz, A. (2007). Object tracking by asymmetric kernel mean shift with automatic scale and orientation selection. In Proc. IEEE int. conf. computer vision and pattern recognition (pp. 1–6).Google Scholar
  21. 21.
    Sumin, Q., & Xianwu, H. (2008). Hand tracking and gesture gecogniton by anisotropic kernel mean shift. In Proc. IEEE int. conf. neural networks and signal processing (Vol. 25, pp. 581–585).Google Scholar
  22. 22.
    Birchfield, S. T., & Rangarajan, S. (2005). Spatiograms versus histograms for region-based tracking. In Proc. IEEE int. conf. computer vision pattern recognition (pp. 71158–1163).Google Scholar
  23. 23.
    Elgammal, A., Duraiswami, R., & Davis, L. S. (2003). Probabilistic tracking in joint feature-spatial spaces. In Proc. IEEE int. conf. computer vision pattern recognition (Vol. 1, pp. I-781–I-788).Google Scholar
  24. 24.
    Xu, D., Wang, Y., & An, J. (2005). Applying a new spatial color histogram in mean-shift based tracking algorithm. In Proc. image and vision comp. conf. New Zealand.Google Scholar
  25. 25.
    Maggio, E., & Cavallaro, A. (2005). Multi-part target representation for colortracking. In Proc. IEEE int. conf. image processing (pp. 729–732).Google Scholar
  26. 26.
    Parameswaran, V., Ramesh, V., & Zoghlami, I. (2006). Tunable kernels for racking. In Proc. IEEE computer society conf. on comp. vision and pattern recognition (pp. 2179–2186).Google Scholar
  27. 27.
    Isard, M., & Blake, A. (1998). CONDENSATION - conditional density propagation for visual tracking. International Journal of Computer Vision, 29(1), 5–28.CrossRefGoogle Scholar
  28. 28.
    Wang, T., Backhouse, A., & Gu, I. Y. H. (2008). Online subspace learning in Grassmann manifold for moving object tracking in video. In Proc. IEEE int. conf. acoustics, speech and signal processing.Google Scholar
  29. 29.
    Michael, K. P., & Shephard, N. (1999). Filtering via simulation: Auxiliary particle filters. Journal of the American Statistical Association, 94(446), 590–599.MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Shen, C., Hengel, A. V. D., & Brooks, M. J. (2004). Enhanced importance sampling: Unscented auxiliary particle filtering for visual tracking. In Proc. Australian Conference on Artificial Intelligence (pp. 180–191).Google Scholar
  31. 31.
    Maggio, E., & Cavallaro, A. (2005). Hybrid particle filter and mean shift tracker with adaptive transition model. In proc. IEEE int. conf. acoustics, speech, and signal processing (Vol. 2, pp. 221–224).Google Scholar
  32. 32.
    Huang Y., & Llach, J. (2007). Variable number of “Informative” particles for object tracking. In Proc. IEEE int. conf. on multimedia and expo (pp. 1926–1929).Google Scholar
  33. 33.
    Bradski, G. R. (1998). Real time face and object tracking as a component of a perceptual user interface. In Proc. IEEE workshop on application of computer vision (pp. 214–219).Google Scholar
  34. 34.
    Fox, D. (2001). KLD-sampling: Adaptive particle filters. In Proc. advances in neural information processing systems (pp. 713–720).Google Scholar
  35. 35.
    Backhouse, A. G., Khan, Z. H., & Gu, I. Y. H. (2009). Robust object tracking using particle filters and multi-region mean shift. Springer Lecture Notes in Computer Science (LNCS) (Vol. 5879). Advances in Multimedia Information Processing, PCM.Google Scholar
  36. 36.
    Chen, Z. (2003). Bayesian filtering: From Kalman filters to particle filters, and beyond. Adaptive Syst. Lab., McMaster Univ., Hamilton, ON, Canada. available at: zhechen/homepage.htm.
  37. 37.
    Khan, Z. H., Gu, I. Y. H., Wang, T., & Backhouse, A. (2009). Joint anisotropic mean shift and consensus point feature correspondences for object tracking in video. In Proc. IEEE int. conf. multimedia and expo (pp. 1270–1273).Google Scholar
  38. 38.
    Khan, Z. H., & Gu, I. Y. H. (2010). Joint feature correspondences and appearance similarity for robust visual object tracking. In IEEE Transactions on information forensics and security (Vol. 5, No. 3).Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Zulfiqar Hasan Khan
    • 1
  • Irene Yu-Hua Gu
    • 1
  • Andrew G. Backhouse
    • 1
  1. 1.Department of Signals and SystemsChalmers University of TechnologyGothenburgSweden

Personalised recommendations