Combining spatial and temporal patches for scalable video indexing
- 63 Downloads
This paper tackles the problem of scalable video indexing. We propose a new framework combining spatial and motion patch descriptors. The spatial descriptors are based on a multiscale description of the image and are called Sparse Multiscale Patches. We propose motion patch descriptors based on block motion that describe the motion in a Group of Pictures. The distributions of these sets of patches are compared combining weighted Kullback-Leibler divergences between spatial and motion patches. These divergences are estimated in a non-parametric framework using a k-th Nearest Neighbor estimator. We evaluate this weighted dissimilarity measure on selected videos from the ICOS-HD ANR project. Experiments show that the spatial part of the measure is relevant to detect different sequences, while its motion part allows to detect clips within a sequence. Experiments combining the spatial and temporal parts of the dissimilarity measure show its robustness to resampling and compression; thus exhibiting the spatial scalability of the method on heterogeneous networks.
KeywordsScalable video indexing Sparse multiscale patches descriptors Motion patches descriptors Kullback-Leibler divergence
The authors would like to acknowledge the contribution of W. Belhajali in the experiments conducted here.
- 1.Boltz S, Debreuve E, Barlaud M (2007) High-dimensional kullback-leibler distance for region-of-interest tracking: application to combining a soft geometric constraint with radiometry. In: CVPR. Minneapolis, USAGoogle Scholar
- 2.Hero AO, Ma B, Michel O, Gorman J (2001) Alpha-divergence for classification, indexing and retrieval. Technical report CSPL-328, University of MichiganGoogle Scholar
- 3.Laptev I, Pérez P (2007) Retrieving actions in movies. In: Proc. int. conf. comp. vis. (ICCV’07). Rio de Janeiro, Brazil, pp 1–8Google Scholar
- 6.Morand C, Benois-Pineau J, Domenger J-Ph, Mansencal B (2007) Object-based indexing of compressed video content: from sd to hd video. In: IEEE VMDL/ICIAP. Modena, ItalyGoogle Scholar
- 7.Piro P, Anthoine S, Debreuve E, Barlaud M (2008) Image retrieval via kullback-leibler divergence of patches of multiscale coefficients in the knn framework. In: CBMI. London, UKGoogle Scholar
- 8.Piro P, Anthoine S, Debreuve E, Barlaud M (2009) Sparse multiscale patches for image processing. In: ETVC. LNCS, vol 5416/2009. Springer, New YorkGoogle Scholar
- 12.Zhai Y, Liu J, Cao X, Basharat A, Hakeem A, Ali S, Shah M (2005) Video understanding and content-based retrieval. In: TRECVID05Google Scholar