Multimedia Tools and Applications

, Volume 48, Issue 1, pp 89–104 | Cite as

Combining spatial and temporal patches for scalable video indexing

  • Paolo Piro
  • Sandrine Anthoine
  • Eric Debreuve
  • Michel Barlaud


This paper tackles the problem of scalable video indexing. We propose a new framework combining spatial and motion patch descriptors. The spatial descriptors are based on a multiscale description of the image and are called Sparse Multiscale Patches. We propose motion patch descriptors based on block motion that describe the motion in a Group of Pictures. The distributions of these sets of patches are compared combining weighted Kullback-Leibler divergences between spatial and motion patches. These divergences are estimated in a non-parametric framework using a k-th Nearest Neighbor estimator. We evaluate this weighted dissimilarity measure on selected videos from the ICOS-HD ANR project. Experiments show that the spatial part of the measure is relevant to detect different sequences, while its motion part allows to detect clips within a sequence. Experiments combining the spatial and temporal parts of the dissimilarity measure show its robustness to resampling and compression; thus exhibiting the spatial scalability of the method on heterogeneous networks.


Scalable video indexing Sparse multiscale patches descriptors Motion patches descriptors Kullback-Leibler divergence 



The authors would like to acknowledge the contribution of W. Belhajali in the experiments conducted here.


  1. 1.
    Boltz S, Debreuve E, Barlaud M (2007) High-dimensional kullback-leibler distance for region-of-interest tracking: application to combining a soft geometric constraint with radiometry. In: CVPR. Minneapolis, USAGoogle Scholar
  2. 2.
    Hero AO, Ma B, Michel O, Gorman J (2001) Alpha-divergence for classification, indexing and retrieval. Technical report CSPL-328, University of MichiganGoogle Scholar
  3. 3.
    Laptev I, Pérez P (2007) Retrieving actions in movies. In: Proc. int. conf. comp. vis. (ICCV’07). Rio de Janeiro, Brazil, pp 1–8Google Scholar
  4. 4.
    Loftsgaarden D, Quesenberry C (1965) A nonparametric estimate of a multivariate density function. AMS 36:1049–1051zbMATHMathSciNetGoogle Scholar
  5. 5.
    Mezaris V, Kompatsiaris I, Strintzis MG (2006) Object-based mpeg-2 video indexing and retrieval in a collaborative environment. Multimed Tools Appl 30:255–272CrossRefGoogle Scholar
  6. 6.
    Morand C, Benois-Pineau J, Domenger J-Ph, Mansencal B (2007) Object-based indexing of compressed video content: from sd to hd video. In: IEEE VMDL/ICIAP. Modena, ItalyGoogle Scholar
  7. 7.
    Piro P, Anthoine S, Debreuve E, Barlaud M (2008) Image retrieval via kullback-leibler divergence of patches of multiscale coefficients in the knn framework. In: CBMI. London, UKGoogle Scholar
  8. 8.
    Piro P, Anthoine S, Debreuve E, Barlaud M (2009) Sparse multiscale patches for image processing. In: ETVC. LNCS, vol 5416/2009. Springer, New YorkGoogle Scholar
  9. 9.
    Rothganger F, Lazebnik S, Schmid C, Ponce J (2007) Segmenting, modeling, and matching video clips containing multiple moving objects. IEEE Trans Pattern Anal Mach Intell 29(3):477–491CrossRefGoogle Scholar
  10. 10.
    Schwarz H, Marpe D, Wiegand T (2007) Overview of the scalable video coding extension of the h.264/avc standard. IEEE Trans Circuits Syst Video Technol 17(9):1103–1120CrossRefGoogle Scholar
  11. 11.
    Terrell DW, Scott GR (1992) Variable kernel density estimation. Ann Stat 20(3):1236–1265zbMATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Zhai Y, Liu J, Cao X, Basharat A, Hakeem A, Ali S, Shah M (2005) Video understanding and content-based retrieval. In: TRECVID05Google Scholar
  13. 13.
    Zong D, Chang SF (1999) An integrated approach for content-based video object segmentation and retrieval. IEEE Trans Circuits Syst Video Technol 9:1259–1268CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Paolo Piro
    • 1
  • Sandrine Anthoine
    • 1
  • Eric Debreuve
    • 1
  • Michel Barlaud
    • 1
  1. 1.I3S lab., Université de Nice Sophia-Antipolis / CNRSSophia Antipolis CedexFrance

Personalised recommendations