Tracking Complex Objects Using Graphical Object Models

  • Leonid Sigal
  • Ying Zhu
  • Dorin Comaniciu
  • Michael Black
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3417)


We present a probabilistic framework for component-based automatic detection and tracking of objects in video. We represent objects as spatio-temporal two-layer graphical models, where each node corresponds to an object or component of an object at a given time, and the edges correspond to learned spatial and temporal constraints. Object detection and tracking is formulated as inference over a directed loopy graph, and is solved with non-parametric belief propagation. This type of object model allows object-detection to make use of temporal consistency (over an arbitrarily sized temporal window), and facilitates robust tracking of the object. The two layer structure of the graphical model allows inference over the entire object as well as individual components. AdaBoost detectors are used to define the likelihood and form proposal distributions for components. Proposal distributions provide ‘bottom-up’ information that is incorporated into the inference process, enabling automatic object detection and tracking. We illustrate our method by detecting and tracking two classes of objects, vehicles and pedestrians, in video sequences collected using a single grayscale uncalibrated car-mounted moving camera.


Graphical Model Object Detection Temporal Constraint Appearance Model Proposal Distribution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Black, M.J., Yacoob, Y.: Recognizing facial expressions in image sequences using local parameterized models of image motion. International Journal of Computer Vision 25(1), 23–48 (1997)CrossRefGoogle Scholar
  2. 2.
    Burl, M., Weber, M., Perona, P.: A probabilistic approach to object recognition using local photometry and global geometry. In: ECCV, pp. 628–641 (1998)Google Scholar
  3. 3.
    Coughlan, J., Ferreira, S.: Finding deformable shapes using loopy belief propagation. In: ECCV, vol. 3, pp. 453–468 (2002)Google Scholar
  4. 4.
    Douce, A., de Freitas, N., Gordon, N.: Sequential Monte Carlo methods in practice. Statistics for Engineering and Information Sciences, pp. 3–14. Springer, Heidelberg (2001)Google Scholar
  5. 5.
    Felzenszwalb, P., Huttenlocher, D.: Efficient matching of pictorial structures. CVPR 2, 66–73 (2000)Google Scholar
  6. 6.
    Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. CVPR (2003)Google Scholar
  7. 7.
    Ihler, A., Sudderth, E., Freeman, W., Willsky, A.: Efficient multiscale sampling from products of Gaussian mixtures. Advances in Neural Info. Proc. Sys. 16 (2003)Google Scholar
  8. 8.
    Isard, M.: Pampas: Real-valued graphical models for computer vision. In: CVPR, vol. 1, pp. 613–620 (2003)Google Scholar
  9. 9.
    Jordan, M., Sejnowski, T., Poggio, T.: Graphical models: Foundations of neural computation. MIT Press, Cambridge (2001)zbMATHGoogle Scholar
  10. 10.
    Mikolajczyk, K., Schmid, C., Zisserman, A.: Human detection based on a probabilistic assembly of robust part detectors. ECCV (2004)Google Scholar
  11. 11.
    Mohan, A., Papageorgiou, C., Poggio, T.: Example-based object detection in images by components. IEEE PAMI 23(4), 349–361 (2001)Google Scholar
  12. 12.
    Murphy, K., Torralba, A., Freeman, W.: Using the forest to see the trees: A graphical model relating features, objects, and scenes. Advances in Neural Info. Proc. Sys. 16 (2003)Google Scholar
  13. 13.
    Okuma, K., Teleghani, A., de Freitas, N., Little, J., Lowe, D.: A boosted particle filter: Multitarget detection and tracking. ECCV (2004)Google Scholar
  14. 14.
    Sigal, L., Bhatia, S., Roth, S., Black, M., Isard, M.: Tracking loose-limbed people. CVPR (2004)Google Scholar
  15. 15.
    Sudderth, E., Ihler, A., Freeman, W., Willsky, A.: Nonparametric belief propagation (see also MIT AI Lab Memo 2002-020). In: CVPR, vol. 1, pp. 605–612. MIT Press, Cambridge (2003)Google Scholar
  16. 16.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. CVPR (2001)Google Scholar
  17. 17.
    Viola, P., Jones, M., Snow, D.: Detecting pedestrians using patterns of motion and appearance. In: ICCV, pp. 734–741 (2003)Google Scholar
  18. 18.
    Xie, B., Comaniciu, D., Ramesh, V., Simon, M., Boult, T.: Component fusion for face detection in the presence of heteroscedastic noise. In: Annual Conf. of the German Society for Pattern Recognition (DAGM’03), pp. 434–441 (2003)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Leonid Sigal
    • 1
  • Ying Zhu
    • 3
  • Dorin Comaniciu
    • 2
  • Michael Black
    • 1
  1. 1.Department of Computer Science, Brown University, Providence, RI 02912 
  2. 2.Integrated Data Systems, Siemens Corporate Research, Princeton, NJ 08540 
  3. 3.Real Time Vision & Modeling, Siemens Corporate Research, Princeton, NJ 08540 

Personalised recommendations