A Probabilistic Approach to Integrating Multiple Cues in Visual Tracking

  • Wei Du
  • Justus Piater
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5303)


This paper presents a novel probabilistic approach to integrating multiple cues in visual tracking. We perform tracking in different cues by interacting processes. Each process is represented by a Hidden Markov Model, and these parallel processes are arranged in a chain topology. The resulting Linked Hidden Markov Models naturally allow the use of particle filters and Belief Propagation in a unified framework. In particular, a target is tracked in each cue by a particle filter, and the particle filters in different cues interact via a message passing scheme. The general framework of our approach allows a customized combination of different cues in different situations, which is desirable from the implementation point of view. Our examples selectively integrate four visual cues including color, edges, motion and contours. We demonstrate empirically that the ordering of the cues is nearly inconsequential, and that our approach is superior to other approaches such as Independent Integration and Hierarchical Integration in terms of flexibility and robustness.


Hide Markov Model Visual Tracking Target Representation Face Tracking Importance Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Pérez, P., Hue, C., Vermaak, J., Gangnet, M.: Color-based probabilistic tracking. In: European Conference on Computer Vision, Copenhagen, Denmark, vol. 1, pp. 661–675 (2002)Google Scholar
  2. 2.
    Taylor, G., Kleeman, L.: Fusion of multimodal visual cues for model-based object tracking. In: acra (2003)Google Scholar
  3. 3.
    Özuysal, M., Lepetit, V., Fleuret, F., Fua, P.: Feature harvesting for tracking-by-detection. In: ECCV (2006)Google Scholar
  4. 4.
    Yang, C., Duraiswami, R., Davis, L.: Fast multiple object tracking via a hierarchical particle filter. In: ICCV, Beijing, China (2005)Google Scholar
  5. 5.
    Pérez, P., Vermaak, J., Blake, A.: Data fusion for visual tracking with particles. Proceedings of the IEEE 92(3), 495–513 (2004)CrossRefGoogle Scholar
  6. 6.
    Isard, M., Blake, A.: Condensation – conditional density propagation for visual tracking. International Journal of Computer Vision 29(2), 5–28 (1998)CrossRefGoogle Scholar
  7. 7.
    Birchfield, S.: Elliptical head tracking using intensity gradients and color histograms. In: CVPR, Santa Barbara, CA, pp. 232–237 (1998)Google Scholar
  8. 8.
    Triesch, J., Malsburg, C.v.: Self-organized integration of adaptive visual cues for face tracking. In: The Fourth IEEE International Conference on Automatic Face and Gesture Recognition (2000)Google Scholar
  9. 9.
    Giebell, J., Gavrila1, D., Schnörr, C.: A bayesian framework for multi-cue 3d object tracking. In: ECCV, Prague, Czech Republic (2004)Google Scholar
  10. 10.
    Leichter, I., Lindenbaum, M., Rivlin, E.: A general framework for combining visual trackers: The black boxes approach. IJCV 67(3), 343–363 (2006)CrossRefGoogle Scholar
  11. 11.
    Wu, Y., Huang, T.: Robust visual tracking by integrating multiple cues based on co-inference learning. International Journal of Computer Vision 58(1), 55–71 (2004)CrossRefGoogle Scholar
  12. 12.
    Brand, M.: Coupled hidden Markov models for modeling interacting processes. Technical report, MIT Media Lab Perceptual Computing (1997)Google Scholar
  13. 13.
    Brasnett, P., Mihaylova, L., Canagarajah, N., Bull, D.: Particle filtering with multiple cues for object tracking in video sequences. In: Proceeding of SPIE-Image and Video Communications, vol. 5685, pp. 430–441 (2005)Google Scholar
  14. 14.
    Wang, H., Suter, D.: Efficient visual tracking by probabilistic fusion of multiple cues. In: ICPR, HongKong (2006)Google Scholar
  15. 15.
    Spengler, M., Schiele, B.: Towards robust multi-cue integration for visual tracking. In: MVA, vol. 14, pp. 50–58 (2003)Google Scholar
  16. 16.
    Gavrila, D.M., Munder, S.: Multi-cue pedestrian detection and tracking from a moving vehicle. International Journal of Computer Vision (2007)Google Scholar
  17. 17.
    Sudderth, E., Ihler, A., Freeman, W., Willsky, A.: Nonparametric belief propagation. In: IEEE Conference on Computer Vision and Pattern Recognition, Madison, WI, vol. 2, pp. 605–612 (2003)Google Scholar
  18. 18.
    Isard, M.: Pampas: Real-valued graphical models for computer vision. In: IEEE Conference on Computer Vision and Pattern Recognition, Madison, WI, vol. 1, pp. 613–620 (2003)Google Scholar
  19. 19.
    Hua, G., Wu, Y.: Multi-scale visual tracking by sequential belief propagation. In: IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, vol. 1, pp. 826–833 (2004)Google Scholar
  20. 20.
    Briers, M., Doucet, A., Singh, S.: Sequential auxiliary particle belief propagation. In: The Eighth International Conference on Information Fusion (2005)Google Scholar
  21. 21.
    Sun, J., Zheng, N., Harry, S.: Stereo matching using belief propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(7), 787–800 (2003)CrossRefGoogle Scholar
  22. 22.
    Birchfield, S.T., Rangarajan, S.: Spatiograms versus histograms for region-based tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA (2005)Google Scholar
  23. 23.
    Porkili, F.: Integral histogram: A fast way to extract histograms in cartesian spaces. In: CVPR, San Diego, CA (2005)Google Scholar
  24. 24.
    Wang, H., Suter, D., Schindler, K.: Effective appearance model and similarity measure for particle filtering and visual tracking. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 606–618. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Wei Du
    • 1
  • Justus Piater
    • 1
  1. 1.Department of Electrical Engineering and Computer Science Montefiore Institute, B28University of LiègeLiegeBelgium

Personalised recommendations