An Approach to Trajectory Estimation of Moving Objects in the H.264 Compressed Domain

  • Christian Käs
  • Henri Nicolas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5414)


This paper presents a simple and fast method for unsupervised trajectory estimation of multiple moving objects within a video scene. It is entirely based on the motion vectors that are present in compressed H.264/AVC or SVC video streams. We extract these motion vectors, perform robust frame-wise global motion estimation and use these estimates to form outlier masks. Motion segmentation on the spatio-temporally filtered outlier masks is performed to detect moving regions in the scene, which are analyzed over time in order to identify similar objects in adjacent frames. The construction of so-called Object History Images (OHIs) is proposed to stabilize the trajectories, which are finally interpolated with X-splines. The system enables real-time analysis with standard hardware.


Scene Analysis Trajectory estimation H.264-AVC/SVC compressed domain 


  1. 1.
    De Bruyne, S., De Neve, W., De Schrijver, D., Lambert, P., Verhoeve, P., Van de Walle, R.: Shot boundary detection for H.264/AVC bitstreams with frames containing multiple types of slices. In: Ip, H.H.-S., Au, O.C., Leung, H., Sun, M.-T., Ma, W.-Y., Hu, S.-M. (eds.) PCM 2007. LNCS, vol. 4810, pp. 177–186. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Babu, R.V., Ramakrishnan, K.: Content-based video retrieval using motion descriptors extracted from compressed domain. In: IEEE International Symposium on Circuits and Systems (ISCAS 2002), Phoenix, USA, vol. 4, pp. 141–144 (2002)Google Scholar
  3. 3.
    Babu, R.V., Ramakrishnan, K., Srinivasan, S.: Video object segmentation: a compressed domain approach. IEEE Transactions on Circuits Systems for Video Technology 14(4), 462–474 (2004)CrossRefGoogle Scholar
  4. 4.
    Zeng, W., Du, J., Gao, W., Huang, Q.: Robust moving object segmentation on h.264/avc compressed video using the block-based mrf model. Real-Time Imaging 11(4), 290–299 (2005)CrossRefGoogle Scholar
  5. 5.
    Sukmarg, O., Rao, K.: Fast object detection and segmentation in mpeg compressed domain. In: 10th IEEE Region Annual International Conference, Kuala Lumpur, Malaysia, vol. 3, pp. 364–368 (September 2000)Google Scholar
  6. 6.
    Mezaris, V., Kompatsiaris, I., Boulgouris, N.V., Strintzis, M.G.: Real-time compressed-domain spatiotemporal segmentation and ontologies for video indexing and retrieval. IEEE Transactions on Circuits and Systems for Video Technology 14, 606–621 (2004)CrossRefGoogle Scholar
  7. 7.
    Hesseler, W., Eickeler, S.: Mpeg-2 compressed-domain algorithms for video analysis. EURASIP Journal on Applied Signal Processing 2, 1–11 (2006)CrossRefzbMATHGoogle Scholar
  8. 8.
    Lie, W.N., Hsiao, W.C.: Content-based video retrieval based on object motion trajectory. In: IEEE Workshop on Multimedia Signal Processing, pp. 237–240 (December 2002)Google Scholar
  9. 9.
    Radhakrishna, A., Kankanhalli, M., Mulhem, P.: Compressed domain object tracking for automatic indexing of objects in mpeg home video. In: IEEE International Conference in Multimedia and Expo (ICME 2002), Lausanne, Switzerland (August 2002)Google Scholar
  10. 10.
    Park, S.M., Lee, J.: Compressed domain object tracking for automatic indexing of objects in mpeg home video. In: 4th Pacific Rim Conference on Multimedia, Singapore, vol. 2, pp. 748–752 (December 2003)Google Scholar
  11. 11.
    Lie, W.N., Chen, R.L.: Tracking moving objects in mpeg-compressed videos. In: IEEE International Conference on Multimedia and Expo. (ICME 2001), vol. 2001, p. 245 (2001)Google Scholar
  12. 12.
    Favalli, L., Mecocci, A., Moschetti, F.: Object tracking for retrieval applications in mpeg-2. IEEE Transactions on Circuits and Systems for Video Technology 10, 427–432 (2000)CrossRefGoogle Scholar
  13. 13.
    Chen, H., Zhan, Y., Qi, F.: Rapid object tracking on compressed video. In: Shum, H.-Y., Liao, M., Chang, S.-F. (eds.) PCM 2001. LNCS, vol. 2195, pp. 1066–1071. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  14. 14.
    Manerba, F., Benois-Pineau, J., Leonardi, R., Mansencal, B.: Multiple moving object detection for fast video content description in compressed domain. EURASIP J. Adv. Signal Process 2008(1), 1–13 (2008)CrossRefzbMATHGoogle Scholar
  15. 15.
    Aggarwal, A., Biswas, S., Singh, S., Sural, S., Majumdar, A.: Object tracking using background subtraction and motion estimation in MPEG videos. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3852, pp. 121–130. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  16. 16.
    Sutter, R.D., DeWolf, K., Lerouge, S., de Walle, R.V.: Lightweight object tracking in compressed video streams demonstrated in region-of-interest coding. EURASIP J. Appl. Signal Process 2007(1), 59 (2007)CrossRefzbMATHGoogle Scholar
  17. 17.
    You, W., Sabirin, M., Kim, M.: Moving object tracking in H.264/AVC bitstream. In: Sebe, N., Liu, Y., Zhuang, Y.-t., Huang, T.S. (eds.) MCAM 2007. LNCS, vol. 4577, pp. 483–492. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  18. 18.
    Schwarz, H., Marpe, D., Wiegand, T.: Overview of the scalable h.264/mpeg4-avc extension. In: IEEE International Conference on Image Processing (ICIP 2006), Atlanta, USA, October 2006, pp. 161–164 (2006)Google Scholar
  19. 19.
    Software, J.R.: Reference software for h.264/svc,
  20. 20.
    Bouthemy, P., Gelgon, M., Ganansia, F.: A unified approach to shot change detection and camera motion characterization  9, 1030 (1999)Google Scholar
  21. 21.
    Durik, M., Benois-Pineau, J.: Robust motion characterisation for video indexing based on mpeg2 optical flow. In: Proceedings of International Workshop on Content-Based Multimedia Indexing (CBMI 2001), Brescia, Italy, pp. 57–64 (September 2001)Google Scholar
  22. 22.
    Bradski, G.R., Davis, J.W.: Motion segmentation and pose recognition with motion history gradients. Mach. Vision Appl. 13(3), 174–184 (2002)CrossRefGoogle Scholar
  23. 23.
    Blanc, C., Schlick, C.: X-splines: a spline model designed for the end-user. In: SIGGRAPH 1995: Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, pp. 377–386. ACM Press, New York (1995)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Christian Käs
    • 1
  • Henri Nicolas
    • 1
  1. 1.LaBRIUniversity of BordeauxTalenceFrance

Personalised recommendations