Multimedia Tools and Applications

, Volume 58, Issue 3, pp 687–711 | Cite as

Extracting representative motion flows for effective video retrieval

  • Zhe Zhao
  • Bin Cui
  • Gao Cong
  • Zi Huang
  • Heng Tao Shen


In this paper, we propose a novel motion-based video retrieval approach to find desired videos from video databases through trajectory matching. The main component of our approach is to extract representative motion features from the video, which could be broken down to the following three steps. First, we extract the motion vectors from each frame of videos and utilize Harris corner points to compensate the effect of the camera motion. Second, we find interesting motion flows from frames using sliding window mechanism and a clustering algorithm. Third, we merge the generated motion flows and select representative ones to capture the motion features of videos. Furthermore, we design a symbolic based trajectory matching method for effective video retrieval. The experimental results show that our algorithm is capable to effectively extract motion flows with high accuracy and outperforms existing approaches for video retrieval.


Video retrieval Content feature Motion flow Trajectory matching 



This research was supported by the National Natural Science foundation of China under Grant No.60933004, 60811120098 and 61073019, and Grant SKLSDE-2010KF-03.


  1. 1.
  2. 2.
    Bashir FI, Khokhar AA, Schonfeld D (2007) Object trajectory-based activity classification and recognition using hidden markov models. IEEE Trans Image Process 16(7):1912–1919MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bashir FI, Khokhar AA, Schonfeld D (2007) Real-time motion trajectory-based indexing and retrieval of video sequences. IEEE Trans Multimedia 9(1):58–65CrossRefGoogle Scholar
  4. 4.
    Chang SF, Chen W, Meng HJ, Sundaram H, Zhong D (1998) A fully automated content-based video search engine supporting spatiotemporal queries. IEEE Trans Circuits Syst Video Technol 8(3):602–615CrossRefGoogle Scholar
  5. 5.
    Chen L, Özsu MT, Oria V (2004) Symbolic representation and retrieval of moving object trajectories. In: the 6th ACM multimedia workshop on MIR, pp 227–234Google Scholar
  6. 6.
    Chen L, Özsu MT, Oria V (2005) Robust and fast similarity search for moving object trajectories. In: Proc. of ACM SIGMOD conference, pp 491–502Google Scholar
  7. 7.
    Cucchiara R, Grana C, Piccardi M, Prati A (2003) Detecting moving objects, ghosts, and shadows in video streams. IEEE Trans Pattern Anal Mach Intell 25(10):1337–1342CrossRefGoogle Scholar
  8. 8.
    Dagtas S, Al-Khatib W, Ghafoor A, Kashyap RL (2000) Models for motion-based video indexing and retrieval. IEEE Trans Image Process 9(1):88–101CrossRefGoogle Scholar
  9. 9.
    Deng Y, Manjunath BS (1998) Netra-V: toward an object-based video representation. IEEE Trans Circuits Syst Video Technol 8(5):616–627CrossRefGoogle Scholar
  10. 10.
    Fablet R, Bouthemy P, Perez P (2002) Nonparametric motion characterization using causal probabilistic models for video indexing and retrieval. IEEE Trans Image Process 11(4):393–407CrossRefGoogle Scholar
  11. 11.
    Flickner M, Niblack HW, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P (1995) Query by image and video content: the QBIC system. Comput 28(9):23–32CrossRefGoogle Scholar
  12. 12.
    Hamrapur A, Gupta A, Horowitz B, Shu CF, Fuller C, Bach J, Gorkani M, Jain R (1997) Virage video engine. In: SPIE proc. storage and retrieval for image and video databases V, pp 188–197Google Scholar
  13. 13.
    Harris CG, Stephens MJ (1988) A combined corner and edge detector. In: Proc. of 4th Alvey vision conference, pp 147–151Google Scholar
  14. 14.
    Hsieh J-W, Yu S-L, Chen Y-S (2006) Motion-based video retrieval by trajectory matching. IEEE Trans Circuits Syst Video Technol 16:396–409CrossRefGoogle Scholar
  15. 15.
    Keogh E, Chu S, Hart D, Pazzani M (2004) Segmenting time series: a survey and novel approach. In: Data mining in time series databases. World ScientificGoogle Scholar
  16. 16.
    Keogh EJ, Chu S, Hart D, Pazzani MJ (2001) An online algorithm for segmenting time series. In: Proc. of ICDM conference, pp 289–296Google Scholar
  17. 17.
    Le T-L, Boucher A, Thonnat M (2007) Subtrajectory-based video indexing and retrieval. In: Proc. of MMM conference, pp 418–427Google Scholar
  18. 18.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRefGoogle Scholar
  19. 19.
    Ma Y-F, Zhang H-J (2002) Motion texture: a new motion based video representation. In: Proc. of ICPR conference, pp 548–551Google Scholar
  20. 20.
    Manjunath BS, Salembier P, Sikora T (2002) Introduction to mpeg-7: multimedia content description interface. In: Proc. of ICPR conference, pp 548–551Google Scholar
  21. 21.
    Rath GB, Makur A (1999) Iterative least squares and compression based estimations for a four-parameter linear global motion model and global motion compensation. IEEE Trans Circuits Syst Video Technol 9:1075–1099CrossRefGoogle Scholar
  22. 22.
    Sivic J, Schaffalitzky F, Zisserman A (2004) Object level grouping for video shots. In: Proc. of ECCV conference, pp 85–98Google Scholar
  23. 23.
    Su C-W, Liao H-Y, Tyan H-R, Lin C-W, Chen D-Y, Fan K-C (2007) Motion flow-based video retrieval. IEEE Trans Multimedia 9(6):1193–1201CrossRefGoogle Scholar
  24. 24.
    Tomasi C, Kanade T (1991) Detection and tracking of point features. Carnegie Mellon University Technical Report, pp 864–975Google Scholar
  25. 25.
    Tsaig Y, Averbuch A (2002) Automatic segmentation of moving objects in video sequences: a region labeling approach. IEEE Trans Circuits Syst Video Technol 12(7):597–612CrossRefGoogle Scholar
  26. 26.
    Wang F, Jiang Y-G, Ngo C-W (2008) Video event detection using motion relativity and visual relatedness. In: Proc. of ACM MM conference, pp 239–248Google Scholar
  27. 27.
    Wu X, Takimoto M, Satoh S, Adachi J (2008) Scene duplicate detection based on the pattern of discontinuities in feature point trajectories. In: Proc. of ACM MM conference, pp 51–60Google Scholar
  28. 28.
    Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38(4):13CrossRefGoogle Scholar
  29. 29.
    Zhu G, Liang D, Liu Y, Huang Q, Gao W (2005) Improving particle filter with support vector regression for efficient visual tracking. In: Proc. of ICIP conference, pp 422–425Google Scholar
  30. 30.
    Avrithis YS, Doulamis AD, Doulamis ND, Kollias SD (1999) A stochastic framework for optimal key frame extraction from mpeg video databases. Comput Vis Image Underst 75:3–24CrossRefGoogle Scholar
  31. 31.
    Lertrusdachakul T, Aoki T, Yasuda H (2005) Camera motion characterization through image feature analysis. In: Proc. of ICCIMA conferenceGoogle Scholar
  32. 32.
    Yeo B-L, Liu B (1995) Rapid scene analysis on compressed video. IEEE Trans Circuits Syst Video Technol 5:533–544CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Zhe Zhao
    • 1
  • Bin Cui
    • 1
  • Gao Cong
    • 2
  • Zi Huang
    • 3
  • Heng Tao Shen
    • 3
  1. 1.State Key Laboratory of Software Development Environment & Department of Computer SciencePeking UniversityBeijingChina
  2. 2.Nanyang Technological UniversityNanyangSingapore
  3. 3.The University of QueenslandQueenslandAustralia

Personalised recommendations