Skip to main content
SpringerLink
Log in

Restricted affine motion compensation and estimation in video coding with particle filtering and importance sampling: a multi-resolution approach

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

In this paper, we propose a multi-resolution affine block-based tracker for motion estimation and compensation, compatible with existing video coding standards such as H.264 and HEVC. We propose three modifications to traditional motion compensation techniques in video coding standards such as H.264 and HEVC. First, we replace traditional search methods with an efficient particle filtering-based method, which incorporates information from both spatial and temporal continuity. Second, we use a higher order linear model in place of the traditional translation motion model in these standards to efficiently represent complex motions such as rotation and zoom. Third, we propose a multi-resolution framework that enables efficient parameter estimation. Results of extensive experimentation show reduced residual energy and better Peak Signal-to-Noise Ratio (PSNR, hereafter) as compared to H.264/HEVC for instance, especially in regions of complex motion such as zooming and rotation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. ITU-T Rec. H.262 and ISO/IEC 13818-2 (MPEG-2): Generic coding of moving pictures and associated audio information - Part 2: Video (1994)

  2. ITU-T Rec. H.264 and ISO/IEC 14496-10 (MPEG4-AVC): Advanced video coding for generic audiovisual services (v1, May 2003; v2, Jan. 2004; v3, Sept. 2004; v4, July 2005)

  3. ITU-T Rec. H.265 and ISO/IEC 23008-2 (HEVC): High Efficiency Video Coding (v1, Apr. 2013; v2, Oct. 2014; v3, Apr. 2015)

  4. Minezawa, A., Sekiguch, S., Sugimoto, K.: Te12.2 report (m18142) on motion vector (MV) prediction AMVP/IMVP. In: Mitsubishi Electric, Technical Report JCTVC-C119 (2010)

  5. Kamp, S.: Video coding using decoder-side motion vector derivation. In: RWTH Aachen University, Germany, Technical Report (Online) (2008)

  6. Zhang, K., Bober, M., Kittler, J.: Video coding using affine motion compensated prediction. In: Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 1978–1981 (1996)

  7. Wiegand, T., Steinbach, E., Girod, B.: Affine multi-picture motion-compensated prediction. IEEE Trans. Circ. Syst. Video Technol. 15(2), 197–209 (2005)

    Article  Google Scholar 

  8. Kordasiewicz, R.C., Gallant, M.D., Shirani, S.: Affine motion prediction based on translational motion vectors. IEEE Trans. Circ. Syst. Video Technol. 17(11), 1388–1394 (2007)

    Article  Google Scholar 

  9. Cheung, H.K., Siu, W.C.: Local affine motion prediction for H.264 without extra overhead. In: IEEE International Symposium on Circuits and Systems, pp. 1555–1558 (2010)

  10. Yuan, H., Liu, J., Sun, J., Liu, H., Li, Y.: Affine model based motion compensation prediction for zoom. IEEE Trans Multimed 14(4), 1370–1375 (2012)

    Article  Google Scholar 

  11. Kuo, C.M., Hsiesg, C., Jou, Y.D., Lin, H.C., Liu, P.C.: Motion estimation for video compression using Kalman filtering. IEEE Trans. Broadcast. 42(2), 111–116 (1996)

    Google Scholar 

  12. Kuo, C.M., Chung, S.C., Shih, P.Y.: Kalman filtering based rate-constrained motion estimation for very low bit rate video coding. IEEE Trans. Circ. Syst. Video Technol. 16(1), 3–18 (2006)

    Article  Google Scholar 

  13. Luo, Y., Celenk, M.: Kalman filtering based motion estimation for video coding with adaptive block partitioning. In: IEEE Workshop on Signal Processing Systems, pp. 129–134 (2008)

  14. Yang, S.: Particle filtering based estimation of consistent motion and disparity with reduced search points. IEEE Trans. Circ. Syst. Video Technol. 22(1), 91–104 (2012)

    Article  Google Scholar 

  15. Kwolek, B.: Face tracking for H.264 encoded video sequences. In: Proceedings of IEEE International Conference on Image Processing (ICIP), pp. 449–452 (2005)

  16. Chung, K.L., Yao, T.J.: New prediction and affine transformation—based three-step search scheme for motion estimation with applications. J. Inf. Sci. Eng. 24, 1095–1109 (2008)

    Google Scholar 

  17. Jing, X., Chau, L.P.: An efficient three-step search algorithm for block motion estimation. IEEE Trans. Multimed. 6, 435–438 (2004)

    Article  Google Scholar 

  18. Muhit, A., Pickering, M.R., Frater, M.R., Arnold, J.F.: Video coding using elastic motion model and larger blocks. IEEE Trans. Circ. Syst. Video Technol. 20(5), 661–672 (2010)

    Article  Google Scholar 

  19. Matthias, N., Swoboda, R.: Extending HEVC by an affine motion model. In: Picture Coding Symposium (PCS), pp. 321–324 (2013)

  20. Huang, H., Woods, J. W., Zhao, Y., Bai, H.: Affine SKIP and DIRECT modes for efficient video coding. Visual Communications and Image Processing (VCIP), pp. 1–6 (2012)

  21. Huang, H., Woods, J.W., Zhao, Y., Bai, H.: Control-point representation and differential coding affine-motion compensation. IEEE Trans. Circ. Syst. Video Technol. 23(10), 1651–1660 (2013)

    Article  Google Scholar 

  22. Chen, H., Liang, F., Lin, S.: Affine SKIP and MERGE modes for video coding. In: IEEE 17th International Workshop on Multimedia Signal Processing (MMSP), pp. 1–5 (2015)

  23. Yoo, J.H.L., Seok, H., Zhang, B.: Evolutionary particle filtering for sequential dependency learning from video data. In: IEEE Congress on Evolutionary Computation, pp. 1–8 (2012)

  24. Li, L., Li, H., Lv, H., Yang, H.: An affine motion compensation framework for high efficiency video coding. In: IEEE International Symposium on Circuits and Systems (ISCAS), pp. 525–528 (2015)

  25. Heithausen, C., Vorwerk, J.H.: Motion compensation with higher order motion models for HEVC. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1438–1442 (2015)

  26. Lee, J.H., Lim, K.W., Song, B.C., Ra, J.B.: A fast multi-resolution block matching algorithm and its LSI architecture for low bit-rate video coding. IEEE Trans. Circ. Syst. Video Technol. 11(12), 1289–1301 (2001)

    Article  Google Scholar 

  27. Gahlot, A., Arya, S., Ghosh, D.: Object-based affine motion estimation. In: Proceedings of IEEE Region 10 Conference, pp. 1343–1347 (2003)

  28. Alwani, M., Chaudhary, R., Mathur, M., Dutta Roy, S., Chaudhury, S.: Restricted affine motion compensation in video coding using particle filtering. In: Proceedings of IAPR-sponsored Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), pp. 479–484 (2010)

  29. Irani, M., Rousso, B., Peleg, S.: Computing occluding and transparent motions. Int. J. Comput. Vis. 12(1), 5–16 (1994)

    Article  Google Scholar 

  30. Isard, M., Blake, A.: CONDENSATION: conditional density propagation for visual tracking. Int. J. Comput. Vis. 28(1), 5–28 (1998)

    Article  Google Scholar 

  31. Sullivan, J., Rittscher, J.: Guiding random particles by deterministic search. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), pp. 1–18 (2001)

  32. Dutta Roy, S., Tran, S.D., Davis, L.S., Vikram, B.S.: Multi-resolution tracking in space and time. In: Proceedings of IAPR-sponsored Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), pp. 352–358 (2008)

  33. Lai, Y. K., Lai, Y. F. Quality enhancement for scalable view window in touchable display systems. In: Proceedings of IEEE International Conference on Consumer Electronics, pp. 539–540 (2011)

  34. Richardson, I.E.G.: H.264 and MPEG-4: Video Compression. Wiley, New York (2003)

  35. Wiegand, T., Sullivan, G.J., Bjontegaard, G., Luthra, A.: Overview of the H.264/AVC video coding standard. IEEE Trans. Circ. Syst. Video Technol. 13(7), 688–703 (2003)

    Article  Google Scholar 

  36. Burt, P.J., Adelson, E.H.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, I.I.T. Delhi, Hauz Khas, New Delhi, 110 016, India

    Mithilesh Kumar Jha, Ravi Chaudhary, Sumantra Dutta Roy, Mona Mathur & Brejesh Lall

Authors
  1. Mithilesh Kumar Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ravi Chaudhary
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sumantra Dutta Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mona Mathur
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Brejesh Lall
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mithilesh Kumar Jha.

Additional information

Communicated by S. Kopf.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jha, M.K., Chaudhary, R., Roy, S.D. et al. Restricted affine motion compensation and estimation in video coding with particle filtering and importance sampling: a multi-resolution approach. Multimedia Systems 24, 271–284 (2018). https://doi.org/10.1007/s00530-017-0543-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-017-0543-z

Keywords

Search

Navigation