Abstract
This paper studies the issue of reducing the temporal redundancy between consecutive frames of a videoconferencing sequence at low bit-rate transmission. To overcome the drawbacks of the traditional block matching algorithm implemented in the most current video coding standards, we propose to better describe the motion of objects through the deformation of planar rectangular mesh grid adapted to the edges of the moving objects in the scene. The traditional inter coding modes are then replaced by two new classes of encoding algorithms. The first one concerns the B-frames where the problem of motion estimation is solved by a bidirectional prediction algorithm which reconstructs the quadrilateral mesh grids without any coding cost. The second class of algorithm much more complex than the first one is specific to the P-frames based on the principle of merging two hierarchical grids of reference. This algorithm addresses not only the motion estimation problem based on the adaptive quadrilateral mesh grid but also the issue of the relevant information (e.g. the positions of the nodes, the connectivity of each quadrilateral mesh of the grid and the motion compensation) to efficiently encode. The implementation of these algorithms in a complete coding scheme offers good performance compared to the H.264/AVC video coder at low bit-rate transmission.
Similar content being viewed by others
References
H.264/AVC references software model (JM 10.1). Available in http://iphome.hhi.de/suehring/tml/download/.
ISO/IEC JTC1/SC29/WG11. “ISO/IEC CD 11172: Information Technology,” MPEG-1 Committee Draft, December 1991.
ISO/IEC JTC1/SC29/WG11. “ISO/IEC CD 13818: Information technology”. MPEG-1 Committee Draft, December 1993.
International telecommunication union, “video coding for low bit rate communication”. ITU-T Draft H.263, May 1996.
Recommendation H.263—video coding for low bit-rate communications. ITU-T, February 1998.
Information technology-generic coding of moving pictures and associated audio information. ISO/IEC 13818, 2000.
Coding of audio—visual objects—part 2: Visual. ISO/IEC 14496-2, 2001.
H.264 advanced video coding. ISO/IEC 14496-10 and ITU-T Rec, 2003.
Recommendation h.264: Advanced video coding for generic audiovisual services. ISO/IEC 14496-10 and ITU-T Rec, March 2005.
Altunbasak, Y., & Tekalp, A. M. (1997). Closed-form connectivity-preserving solutions for motion compensation using 2D meshes. IEEE Transactions on Image Processing, 6(9), 1255–1269.
Bathe, K. J. (1982). Finite element procedures in engineering analysis. Englewood Cliffs: Prentice-Hall.
Becker, E. B., Gatey, G. F., & Oden, J. T. (1982). Finite elements, an introduction. Englewood Cliffs: Prentice-Hall.
De Boor, C. (2001). A practical guide to splines. New York: Springer.
Brusewitz, H. (1990). Motion compensation with triangles. Processing 3rd International Conference on 64 kbit Coding of Moving Video, September.
Bugwadia, K. A., Petajan, E. D., & Puri, N. N. (1996). Progressive-scan rate up-conversion of 21/30hz source materials for hdtv. IEEE Transaction of Consumer Electronics, 42(3), 312–321.
Castagno, R., Haavisto, P., & Ramponi, G. (1996). A method for motion adaptive frame up-conversion. IEEE Transaction Circuits and Systems for Video Technology, 6(5), 436–446.
Chen, T., Wang, Y., Graf, H. P., & Swain, C. (1995). A new frame interpolation scheme for talking head sequences. IEEE International Conference on Image Processing, 2, 591–594.
Chen, Y. K., Vetro, A., Sun, H., & Kung, S. Y. (1998). Frame rate up conversion using transmitted true motion vectors. IEEE International Workshop on Multimedia Signal Processing, pp 622–627, December.
Chihab, N., Zergaïnoh, A., Duhamel, P., & Astruc, J. P. (2004). The influence of the non-uniform spline basis on the approximation signal. Proceedings of the EUSIPCO 2004 Conferences, September.
Choi, B. T., Lee, S. H., & Ko, S. J. (2000). New frame rate up-conversion using bi-directional motion estimation. IEEE Transactions on Consumer Electronics, 46(3), 603–609.
Gonzalez, R. C., & Woods, R. E. (2008). Digital image processing. Upper Saddle River: Prentice Hall.
Bister, M., Karim, A., & Siddiqi, M. U. (2004). Multiresolution motion estimation for low-rate video frame interpolation. IEEE Transactions of Consumer Electronics, 11.
Huang, C. L., & Hsu, C. Y. (1994). A new motion compensation method for image sequence coding using hierarchical grid interpolation. IEEE Transactions on Circuits and Systems for Video Technology, 4(1), 13–52.
Jehan-Besson, S., Barlaud, M., & Aubert, G. (2001). Video object segmentation using eulerian region-based active contours. ICCV International Conference on Computer Vision, pp 353–360.
Kim, D. W., Kim, J. T., & Ra, I. H. (1999). A new video interpolation technique based on motion-adaptive subsampling. IEEE Transactions of Consumer Electronics, 45(3), 782–787.
Krishnamurthy, R., Woods, J. W., & Moulin, P. (1999). Frame interpolation and bidirectional prediction of video using compactly encoded optical-flow fields and label fields. IEEE Transaction on Circuits and Systems for Video Technology, 9(5), 713–726.
Kuo, T. Y., Kim, J., & Kuo, C. C. J. (1999). Motion compensated frame interpolation scheme for H.263 codec. IEEE International Symposium on Circuits and Systems, ISCAS, 4, 491–494.
Lagendijk, R. L., & Sezan, M. I. (1992). Motion compensated frame rate conversion of motion pictures. Proceedings of the IEEE International Conference on Acoustics, Speech & Signal Processing, ICASSP, 3, 453–456.
Lee, O., & Wang, Y. (1995). Motion compensated prediction using nodal based deformable block matching. Journal of Visual Communication and Image Representation, 6, 26–34.
Marquant, G. (2000). Représentation par maillage adaptatif déformable pour la manipulation et la communication d’objets vidéo. Phd thesis, Université de Rennes I—France, Décembre.
Sayed, M., & Badawy, W. (2006). An affine-based algorithm and SIMD architecture for video compression with low bit-rate applications. IEEE Transactions on Circuits and Systems for Video Technology, 16(4), 457–471.
Muñoz-Jimenez, V., & Zergaïnoh, A. (2007). Computational cost reduction of H.264/AVC video coding standard for video conferencing applications. IEEE International Symposium on Signal Processing and its Applications, p 14, February.
Nakaya, Y., & Harashima, H. (1991). An iterative motion estimation method using triangular patches for motion compensation. Processing SPIE Visual Communication and Image Proceeding, pp 546–557, November.
Nakaya, Y., & Harashima, H. (1994). Motion compensation based on spatial transformation. IEEE Transactions on Circuits and Systems for Video Technology, 4(3), 339356.
Nilsson, M., Nordberg, J., & Claesson, I. (2007). Face detection using local smqt features and split up snow classifier. ICASSP International Conference on Acoustic, Speech and Signal Processing, 2, 589–592.
Nosratinia, A. (2001). New kernels for fast mesh-based motion estimation. IEEE Transactions on Circuits and Systems for Video Technology, 11(1), 40–51.
Nosratinia, A., & Orchard, M. T. (1996). Optimal warping prediction for video coding. IEEE International Conference on Acoustics, Speech, and Signal Processing, pp 1986–1989, May.
Richardson, I. E. G. (2003). H.264 and MPEG-4 Video compression: Video coding for next-generation multimedia. Wiley.
Rosenfeld, A. (1980). Quadtrees and pyramids for pattern recognition and image processing. Processing 5th International Conference on Pattern Recognition, pp 802–811.
Samet, A. (1984). The quadtree and related hierarchical data structures. ACM Computing Surveys, 16, 187–260.
Sullivan, G. J., & Baker, R. L. (1991) Motion compensation for video compression using grid interpolation. IEEE International Conference on Acoustics, Speech and Signal Processing, pp 2713–2716, July.
Sung, W. R., Kang, E. K., & Choi, J. S. (1999). Adaptive motion estimation technique for motion compensated inter frame interpolation. IEEE Transactions on Consumer Electronics, 45(3), 753–761.
Tekalp, A. M., Van Beek, P., Toklu, C., & Gûnsel, B. (1998). Two-dimensional mesh-based visual-object representation for interactive synthetic/natural digital video. Invited paper in Proceedings of the IEEE, 86(6), 1029–1051.
Toklu, C., Erdem, A. T., Sezan, M. I., & Tekalp, A. M. (1996). Tracking motion and intensity variations using hierarchical 2D mesh modelling for synthetic object transfiguration. Graphical Models and Image Processing, 58(6), 553–573.
Wang, Y., & Lee, O. (1996). Use of 2D deformable mesh structures for video coding. Part I-the synthesis problem: Mesh-based function approximation and mapping. IEEE Transactions on Circuits and Systems for Video Technology, 6, 636–646.
Zienkewicz, O. C., & Taylor, R. L. (1989). The finite element method. NJ: Prentice Hall.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Mokraoui, A., Muñoz-Jiménèz, V. & Astruc, JP. Motion Estimation Algorithms Using the Deformation of Planar Hierarchical Mesh Grid for Videoconferencing Applications at Low Bit-rate Transmission. J Sign Process Syst 67, 167–185 (2012). https://doi.org/10.1007/s11265-010-0534-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-010-0534-1