Video Browsing Using Object Trajectories

  • Felix Lee
  • Werner Bailer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6524)


Video browsing methods are complementary to search and retrieval approaches, as they allow for exploration of unknown content sets. Objects and their motion convey important semantics of video content, which is relevant information for video browsing. We propose extending an existing video browsing tool in order to support clustering of objects with similar motion and visualization of the objects’ positions and trajectories. This requires the automatic extraction of moving objects and estimation of their trajectories, as well as the ability to group objects with similar trajectories. For the first issue we describe the application of a recently proposed motion trajectory clustering algorithm, for the second we use k-medoids clustering and the dynamic time warping distance. We present evaluation results of both steps on real world traffic sequences from the Hopkins155 data set. Finally we describe the description of analysis results using MPEG-7 and the integration into the video browsing tool.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Axelrod, A., Caspi, Y., Gamliel, A., Matsushita, Y.: Interactive video exploration using pose slices. In: ACM SIGGRAPH, p. 132 (2006)Google Scholar
  2. 2.
    Bailer, W., Fassold, H., Lee, F., Rosner, J.: Tracking and clustering salient features in image sequences. In: Proc. 7th European Conference on Visual Media Production, London, UK (November 2010)Google Scholar
  3. 3.
    Bailer, W., Schallauer, P.: The detailed audiovisual profile: Enabling interoperability between MPEG-7 based systems. In: Proc. of 12th Intl. Multi-Media Modeling Conference, Beijing, CN, pp. 217–224 (January 2006)Google Scholar
  4. 4.
    Bailer, W., Weiss, W., Kienast, G., Thallinger, G., Haas, W.: A video browsing tool for content management in post-production. International Journal of Digital Multimedia Broadcasting (March 2010)Google Scholar
  5. 5.
    Bashir, F.I., Khokhar, A.A., Schonfeld, D.: Segmented trajectory based indexing and retrieval of video data. In: International Conference on Image Processing, vol. 2, pp. 623–626 (2003)Google Scholar
  6. 6.
    Basu, S., Bilenko, M., Mooney, R.J.: A probabilistic framework for semi-supervised clustering. In: Proc. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 59–68 (2004)Google Scholar
  7. 7.
    Costeira, J., Kanade, T.: A multi-body factorization method for motion analysis. In: Proc. of the Fifth Intl. Conf. on Computer Vision, p. 1071 (1995)Google Scholar
  8. 8.
    Dragicevic, P., Ramos, G., Bibliowitcz, J., Nowrouzezahrai, D., Balakrishnan, R., Singh, K.: Video browsing by direct manipulation. In: Proc. SIGCHI Conf. on Human Factors in Computing Systems, pp. 237–246 (2008)Google Scholar
  9. 9.
    Fassold, H., Rosner, J., Schallauer, P., Bailer, W.: Realtime KLT feature point tracking for high definition video. In: GravisMa Workshop (2009)Google Scholar
  10. 10.
    Fradet, M., Robert, P., Perez, P.: Clustering point trajectories with various life-spans. In: Proc. of 6th European Conference on Visual Media Production, pp. 7–14 (November 2009)Google Scholar
  11. 11.
    Hsieh, J.-W., Yu, S.-L., Chen, Y.-S.: Motion-based video retrieval by trajectory matching. IEEE Transactions on Circuits and Systems for Video Technology 16(3), 396–409 (2006)CrossRefGoogle Scholar
  12. 12.
    Hu, W., Xie, D., Fu, Z., Zeng, W., Maybank, S.: Semantic-based surveillance video retrieval. IEEE Transactions on Image Processing 16(4), 1168–1181 (2007)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Irani, M., Anandan, P.: Video indexing based on mosaic representations. Proceedings of the IEEE 86(5), 905–921 (1998)CrossRefGoogle Scholar
  14. 14.
    Kanatani, K.: Motion segmentation by subspace separation and model selection. In: Proc. 8th IEEE Intl. Conf. on Computer Vision, vol. 2, pp. 586–591 (2001)Google Scholar
  15. 15.
    Kaufman, L., Rousseeuw, P.J.: Clustering by means of medoids. In: Dodge, Y. (ed.) Statistical Data Analysis Based on the L1-Norm and Related Methods, pp. 405–416 (1987)Google Scholar
  16. 16.
    Kimber, D., Dunnigan, T., Girgensohn, A., Shipman, F., Turner, T., Yang, T.: Trailblazing: Video playback control by direct object manipulation. In: IEEE Intl. Conf. on Multimedia and Expo, pp. 1015–1018 (July 2007)Google Scholar
  17. 17.
    Lie, W.-N., Hsiao, W.-C.: Content-based video retrieval based on object motion trajectory. In: Proc. IEEE Workshop on Multimedia Signal Processing, pp. 237–240 (December 2002)Google Scholar
  18. 18.
    MPEG-7. Information Technology—Multimedia Content Description Interface: Part 3: Visual. ISO/IEC 15938-3 (2001)Google Scholar
  19. 19.
    Myers, C.S., Rabiner, L.R.: A comparative study of several dynamic time-warping algorithms for connected word recognition. The Bell System Technical Journal 60(7), 1389–1409 (1981)CrossRefGoogle Scholar
  20. 20.
    Pachoud, S., Maggio, E., Cavallaro, A.: Grouping motion trajectories. In: Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing, pp. 1477–1480 (2009)Google Scholar
  21. 21.
    Sahouria, E., Zakhor, A.: Motion indexing of video. In: International Conference on Image Processing, vol. 2, p. 526. IEEE Computer Society, Los Alamitos (1997)CrossRefGoogle Scholar
  22. 22.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vision 9(2), 137–154 (1992)CrossRefGoogle Scholar
  23. 23.
    Tron, R., Vidal, R.: A benchmark for the comparison of 3-D motion segmentation algorithms. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1–8 (June 2007)Google Scholar
  24. 24.
    Zhong, D., Chang, S.-F.: Spatio-temporal video search using the object based video representation. In: International Conference on Image Processing, vol. 1, p. 21 (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Felix Lee
    • 1
  • Werner Bailer
    • 1
  1. 1.DIGITAL – Institute of Information and Communication TechnologiesJOANNEUM RESEARCH Forschungsgesellschaft mbHGrazAustria

Personalised recommendations