Robust Head Tracking with Particles Based on Multiple Cues Fusion

  • Yuan Li
  • Haizhou Ai
  • Chang Huang
  • Shihong Lao
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3979)


This paper presents a fully automatic and highly robust head tracking algorithm based on the latest advances in real-time multi-view face detection techniques and multiple cues fusion under particle filter framework. Visual cues designed for general object tracking problem hardly suffice for robust head tracking under diverse or even severe circumstances, making it a necessity to utilize higher level information which is object-specific. To this end we introduce a vector-boosted multi-view face detector [2] as the “face cue” in addition to two other general visual cues targeting the entire head, color spatiogram[3] and contour gradient. Data fusion is done by an extended particle filter which supports multiple distinct yet interrelated state vectors (referring to face and head in our tracking context). Furthermore, pose information provided by the face cue is exploited to help achieve improved accuracy and efficiency in the fusion. Experiments show that our algorithm is highly robust against target position, size and pose change as well as unfavorable conditions such as occlusion, poor illumination and cluttered background.


Face Detection Color Histogram Observation Model Cluttered Background Active Appearance Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Isard, M., Blake, A.: Condensation – conditional density propagation for visual tracking. IJCV 28(1), 5–28 (1998)CrossRefGoogle Scholar
  2. 2.
    Huang, C., Ai, H., Li, Y., Lao, S.: Vector boosting for rotation invariant multi-view face detection. In: ICCV (2005)Google Scholar
  3. 3.
    Birchfield, S., Rangarajan, S.: Spatiograms versus histograms for region-based tracking. In: CVPR (2005)Google Scholar
  4. 4.
    Perez, P., Vermaak, J., Blake, A.: Data fusion for visual tracking with particles. Proceedings of IEEE (issue on State Estimation) (2004)Google Scholar
  5. 5.
    Verma, R.C., Schmid, C., Mikolajczyk, K.: Face detection and tracking in a video by propagating detection probabilities. PAMI 25(10), 1215–1228 (2003)CrossRefGoogle Scholar
  6. 6.
    Okuma, K., Taleghani, A., de Freitas, N., Little, J.J., Lowe, D.G.: A Boosted Particle Filter: Multitarget Detection and Tracking. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 28–39. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Viola, P., Jones, M.: Robust real-time object detection. In: IEEE Workshop on Statistical and Theories of Computer Vision (2001)Google Scholar
  8. 8.
    Wu, B., Ai, H., Huang, C., Lao, S.: Fast rotation invariant multi-view face detection based on real adaboost. In: Intl. Conf. on Automatic Face and Gesture Recognition (2004)Google Scholar
  9. 9.
    Schneiderman, H., Kanade, T.: A statistical to 3d object detection applied to faces and cars. In: CVPR (2000)Google Scholar
  10. 10.
    Rowley, H.A.: Neural Network-based Human Face Detection. PhD thesis, Carnegie Mellon University (1999)Google Scholar
  11. 11.
    Birchfield, S.: Elliptical head tracking using intensity gradients and color histograms. In: CVPR (1998)Google Scholar
  12. 12.
    Comaniciu, D., Ramesh, V., Meer, P.: Real-time tracking of non-rigid objects using mean shift. In: CVPR (2000)Google Scholar
  13. 13.
    Pérez, P., Hue, C., Vermaak, J., Gangnet, M.: Color-based probabilistic tracking. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 661–675. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  14. 14.
    Matthews, I., Baker, S.: Active appearance models revisited. Technical Report CMU-RI-TR-03-02, The Robotics Institute, Carnegie Mellon University (2002)Google Scholar
  15. 15.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. PAMI 23(6), 681–684 (2001)CrossRefGoogle Scholar
  16. 16.
    Cascia, M.L., Sclaroff, S., Athitsos, V.: Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3d models. PAMI 22(4), 322–336 (2000)CrossRefGoogle Scholar
  17. 17.
    Vermaak, J., Doucet, A., Perez, P.: Maintaining multi-modality through mixture tracking. In: ICCV (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yuan Li
    • 1
  • Haizhou Ai
    • 1
  • Chang Huang
    • 1
  • Shihong Lao
    • 2
  1. 1.Department of Computer ScienceTsinghua UniversityBeijingChina
  2. 2.Sensing and Control Technology LabOMRON CorporationKyotoJapan

Personalised recommendations