Video Sequence Querying Using Clustering of Objects’ Appearance Models

  • Yunqian Ma
  • Ben Miller
  • Isaac Cohen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4842)


In this paper, we present an approach for addressing the ‘query by example’ problem in video surveillance, where a user specifies an object of interest and would like the system to return some images (e.g. top five) of that object or its trajectory by searching a large network of overlapping or non-overlapping cameras. The approach proposed is based on defining an appearance model for every detected object or trajectory in the network of cameras. The model integrates relative position, color, and texture descriptors of each detected object. We present a ‘pseudo track’ search method for querying using a single appearance model. Moreover, the availability of tracking within every camera can further improve the accuracy of such association by incorporating information from several appearance models belonging to the object’s trajectory. For this purpose, we present an automatic clustering technique allowing us to build a multi-valued appearance model from a collection of appearance models. The proposed approach does not require any geometric or colorimetric calibration of the cameras. Experiments from a mass transportation site demonstrate some promising results.


Video Sequence Video Stream Video Surveillance Appearance Model Valid Cluster 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Porikli, F., Tuzel, O., Meer, P.: Covariance Tracking using Model Based on Means on Riemannian Manifolds. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2006)Google Scholar
  2. 2.
    Fletcher, P.T., Lu, C., Pizer, S.M., Joshi, S.: Principal Geodesic Analysis for the Study of Nonlinear Statistics of Shape. IEEE Trans. on Medical Imaging 23(8), 995–1005 (2004)CrossRefGoogle Scholar
  3. 3.
    Forstner, W., Moonen, B.: A Metric for Covariance Matrices, TR Dept. of Geodesy and Geoinformatics, Stuttgart University (1999)Google Scholar
  4. 4.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Willey & Sons, New York (2001)zbMATHGoogle Scholar
  5. 5.
    Ma, Y., Yu, Q., Cohen, I.: Multiple Hypothesis Target Tracking Using Merge and Split of Graph’s Node. In: Bebis, G., et al. (eds.) ISCV 2006, pp. 783–792 (2006)Google Scholar
  6. 6.
    Rote, G.: Computing the minimum Hausdorff distance between two point sets on a line under translation. Information Processing Letters 38, 123–127 (1991)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Tuzel, O., Porikli, F., Meer, P.: Human Detection via Classification on Riemannian Manifolds. In: CVPR (2007)Google Scholar
  8. 8.
    Nister, D., Stewenius, H.: Scalable Recognition with a Vocabulary Tree. In: CVPR (2006)Google Scholar
  9. 9.
    Kang, J., Cohen, I., Medioni, G.: Continuous Tracking Within and across Camera Streams. In: CVPR (2003)Google Scholar
  10. 10.
    Yu, Q., Medioni, G., Cohen, I.: Multiple Target Tracking Using Spatio-Temporal Markov Chain Monte Carlo Data Association. In: CVPR (2007)Google Scholar
  11. 11.
    Christel, M.G.: Carnegie Mellon University Traditional Informedia Digital Video Retrieval System. In: CIVR (2007)Google Scholar
  12. 12.
    Ferencz, A., Learned-Miller, E.G., Malik, J.: Learning Hyper-Features for Visual Identification. NIPS (2004)Google Scholar
  13. 13.
    Tuzel, O., Porikli, F., Meer, P.: Region covariance: A fast descriptor for detection and classification. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 589–600. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  14. 14.
    Liu, T., Rosenberg, C., Rowley, H.: Clustering Billions of Images with Large Scale Nearest Neighbor Search. In: WACV (2007)Google Scholar
  15. 15.
    Park, U., Jain, A.K., Kitahara, I., Kogure, K., Hagita, N.: ViSE: Visual Search Engine Using Multiple Networked Cameras. ICPR (2006)Google Scholar
  16. 16.
    Leoputra, W., Tan, T., Lim, F.: Non-overlapping Distributed Tracking using Particle Filter. ICPR (2006)Google Scholar
  17. 17.
    Cameras Albu, A.B., Laurendeau, D., Comtois, S., et al.: MONNET: Monitoring Pedestrians with a Network of Loosely-Coupled Cameras. ICPR (2006)Google Scholar
  18. 18.
    Karcher, H.: Riemannian center of mass and mollifier smoothing. Communications of Pure and Applied Mathematics 30, 509–541 (1977)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Yunqian Ma
    • 1
  • Ben Miller
    • 1
  • Isaac Cohen
    • 1
  1. 1.Honeywell Labs, 3660 Technology Drive, Minneapolis, MN 55418 

Personalised recommendations