Advertisement

Motion Estimation and Signaling Techniques for 2D+t Scalable Video Coding

  • M. TagliasacchiEmail author
  • D. Maestroni
  • S. Tubaro
  • A. Sarti
Open Access
Research Article
Part of the following topical collections:
  1. Video Analysis and Coding for Robust Transmission

Abstract

We describe a fully scalable wavelet-based 2D+t (in-band) video coding architecture. We propose new coding tools specifically designed for this framework aimed at two goals: reduce the computational complexity at the encoder without sacrificing compression; improve the coding efficiency, especially at low bitrates. To this end, we focus our attention on motion estimation and motion vector encoding. We propose a fast motion estimation algorithm that works in the wavelet domain and exploits the geometrical properties of the wavelet subbands. We show that the computational complexity grows linearly with the size of the search window, yet approaching the performance of a full search strategy. We extend the proposed motion estimation algorithm to work with blocks of variable sizes, in order to better capture local motion characteristics, thus improving in terms of rate-distortion behavior. Given this motion field representation, we propose a motion vector coding algorithm that allows to adaptively scale the motion bit budget according to the target bitrate, improving the coding efficiency at low bitrates. Finally, we show how to optimally scale the motion field when the sequence is decoded at reduced spatial resolution. Experimental results illustrate the advantages of each individual coding tool presented in this paper. Based on these simulations, we define the best configuration of coding parameters and we compare the proposed codec with MC-EZBC, a widely used reference codec implementing the t+2D framework.

Keywords

Motion Vector Motion Estimation Video Code Scalable Video Code Motion Field 

References

  1. 1.
    Taubam D, Marcellin MW: JPEG2000: Image Compression Fundamentals, Standards and Practice. Kluwer Academic, Boston, Mass, USA; 2002.Google Scholar
  2. 2.
    Said A, Pearlman WA: A new, fast, and efficient image codec based on set partitioning in hierarchical trees. IEEE Transactions on Circuits and Systems for Video Technology 1996, 6(3):243–250. 10.1109/76.499834CrossRefGoogle Scholar
  3. 3.
    Hsiang S-T, Woods JW: Embedded image coding using zeroblocks of subband/wavelet coefficients and context modeling. Proceedings of IEEE International Symposium on Circuits and Systems (ISCAS '00), May 2000, Geneva, Switzerland 3: 662–665.Google Scholar
  4. 4.
    Xu J, Xiong Z, Li S, Zhang Y-Q: Three-dimensional embedded subband coding with optimized truncation (3-D ESCOT). Applied and Computational Harmonic Analysis 2001, 10(3):290–315. 10.1006/acha.2000.0345MathSciNetCrossRefGoogle Scholar
  5. 5.
    Chen P, Woods JW: Bidirectional MC-EZBC with lifting implementation. IEEE Transactions on Circuits and Systems for Video Technology 2004, 14(10):1183–1194. 10.1109/TCSVT.2004.833165CrossRefGoogle Scholar
  6. 6.
    Secker A, Taubman D: Lifting-based invertible motion adaptive transform (LIMAT) framework for highly scalable video compression. IEEE Transactions on Image Processing 2003, 12(12):1530–1542. 10.1109/TIP.2003.819433CrossRefGoogle Scholar
  7. 7.
    Pau G, Tillier C, Pesquet-Popescu B, Heijmans H: Motion compensation and scalability in lifting-based video coding. Signal Processing: Image Communication 2004, 19(7):577–600. 10.1016/j.image.2004.05.003Google Scholar
  8. 8.
    Ohm J-R: Three-dimensional subband coding with motion compensation. IEEE Transactions on Image Processing 1994, 3(5):559–571. 10.1109/83.334985CrossRefGoogle Scholar
  9. 9.
    Andreopoulos Y, Munteanu A, Barbarien J, van der Schaar M, Cornelis J, Schelkens P: In-band motion compensated temporal filtering. Signal Processing: Image Communication 2004, 19(7):653–673. 10.1016/j.image.2004.05.007Google Scholar
  10. 10.
    Wang Y, Cui S, Fowler JE: 3D video coding using redundant-wavelet multihypothesis and motion-compensated temporal filtering. Proceedings of IEEE International Conference on Image Processing (ICIP '03), September 2003, Barcelona, Spain 2: 755–758.Google Scholar
  11. 11.
    Park H-W, Kim H-S: Motion estimation using low-band-shift method for wavelet-based moving-picture coding. IEEE Transactions on Image Processing 2000, 9(4):577–587. 10.1109/83.841935CrossRefGoogle Scholar
  12. 12.
    Ye JC, van der Schaar M: Fully scalable 3D overcomplete wavelet video coding using adaptive motion compensated temporal filtering. In Visual Communications and Image Processing (VCIP '03), July 2003, Lugano, Switzerland, Proceedings of SPIE Edited by: Ebrahimi T, Sikora T. 5150: 1169–1180.Google Scholar
  13. 13.
    Mallat S: A Wavelet Tour of Signal Processing. Academic Press, San Diego, Calif, USA; 1998.zbMATHGoogle Scholar
  14. 14.
    Mehrseresht N, Taubman D: A flexible structure for fully scalable motion compensated 3D-DWT with emphasis on the impact of spatial scalability. submitted to IEEE Transactions on Image ProcessingGoogle Scholar
  15. 15.
    Mehrseresht N, Taubman D: An efficient content-adaptive motion compensated 3D-DWT with enhanced spatial and temporal scalability. submitted to IEEE Transactions on Image ProcessingGoogle Scholar
  16. 16.
    ITU-T : Information technology—Coding of audio-visual objects—Part 10: Advanced video coding. May 2003, ISO/IEC International Standard 14496-10:2003Google Scholar
  17. 17.
    Maestroni D, Sarti A, Tagliasacchi M, Tubaro S: Fast in-band motion estimation with variable size block matching. Proceedings of IEEE International Conference on Image Processing (ICIP '04), October 2004, Singapore 4: 2287–2290.Google Scholar
  18. 18.
    Maestroni D, Sarti A, Tagliasacchi M, Tubaro S: Scalable coding of variable size blocks motion vectors. Proceedings of IEEE International Conference on Image Processing (ICIP~'04), October 2004, Singapore 2: 1333–1336.Google Scholar
  19. 19.
    Maestroni D, Sarti A, Tagliasacchi M, Tubaro S: Wavelet-based video coding: optimal use of motion information for the decoding of spatially scaled video sequences. Proceedings of the 12th European Signal Processing Conference (EUSIPCO '04), September 2004, Vienna, AustriaGoogle Scholar
  20. 20.
    Andreopoulos Y, Munteanu A, Van der Auwera G, Schelkens P, Cornelis J: A new method for complete-to-overcomplete discrete wavelet transforms. Proceedings of the 14th International Conference on Digital Signal Processing (DSP '02), July 2002, Santorini, Greece 2: 501–504.CrossRefGoogle Scholar
  21. 21.
    Andreopoulos Y, van der Schaar M, Munteanu A, Barbarien J, Schelkens P, Cornelis J: Fully-scalable wavelet video coding using in-band motion compensated temporal filtering. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '03), April 2003, Hong Kong, China 3: 417–420.Google Scholar
  22. 22.
    Cai W, Adjouadi M: An efficient approach of fast motion estimation and compensation in wavelet domain video compression. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '04), May 2004, Montreal, Quebec, Canada 2: 977–980.Google Scholar
  23. 23.
    Choi S-J, Woods JW: Motion-compensated 3-D subband coding of video. IEEE Transactions on Image Processing 1999, 8(2):155–167. 10.1109/83.743851CrossRefGoogle Scholar
  24. 24.
    Xiong R, Wu F, Li S, Xiong Z, Zhang Y-Q: Exploiting temporal correlation with adaptive block-size motion alignment for 3D wavelet coding. Visual Communications and Image Processing 2004, January 2004, San Jose, Calif, USA, Proceedings of SPIE 5308: 144–155.Google Scholar
  25. 25.
    Taubman D, Secker A: Highly scalable video compression with scalable motion coding. Proceedings of IEEE International Conference on Image Processing (ICIP '03), September 2003, Barcelona, Spain 3: 273–276.Google Scholar
  26. 26.
    Secker A, Taubman D: Highly scalable video compression with scalable motion coding. IEEE Transactions on Image Processing 2004, 13(8):1029–1041. 10.1109/TIP.2004.826089CrossRefGoogle Scholar
  27. 27.
    Barbarien J, Munteanu A, Verdicchio F, Andreopoulos Y, Cornelis J, Schelkens P: Scalable motion vector coding. Electronics Letters 2004, 40(15):932–934. 10.1049/el:20040490CrossRefGoogle Scholar
  28. 28.
    Boisson G, Francois E, Guillemot C: Accuracy-scalable motion coding for efficient scalable video compression. Proceedings of IEEE International Conference on Image Processing (ICIP '04), October 2004, Singapore 2: 1309–1312.Google Scholar
  29. 29.
    Xiong R, Xu J, Wu F, Li S, Zhang Y-Q: Layered motion estimation and coding for fully scalable 3D wavelet video coding. Proceeding of IEEE International Conference on Image Processing (ICIP '04), October 2004, Singapore 4: 2271–2274.Google Scholar
  30. 30.
    Lucas BD, Kanade T: An iterative image registration technique with an application to stereo vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI '81), August 1981, Vancouver, BC, Canada 674–679.Google Scholar
  31. 31.
    Horn BKP, Schunck BG: Determining optical flow. Artificial Intelligence 1981, 17(1–3):185–203.CrossRefGoogle Scholar
  32. 32.
    Tekalp MA: Digital Video Processing, Prentice-Hall Signal Processing Series. Prentice-Hall, Upper Saddle River, NJ, USA; 1995.Google Scholar
  33. 33.
    Benzler U, Wien M: Results of SVC CE3 (Quality Evaluation). ISO/IEC JTC1/SC29/WG11 MPEG Document M10931, July 2004Google Scholar

Copyright information

© Tagliasacchi et al. 2006

Authors and Affiliations

  • M. Tagliasacchi
    • 1
    Email author
  • D. Maestroni
    • 1
  • S. Tubaro
    • 1
  • A. Sarti
    • 1
  1. 1.Dipartimento di Elettronica e InformazionePolitecnico di MilanoMilanoItaly

Personalised recommendations