Skip to main content
SpringerLink
Log in

Refinement of the recovered motion vectors for error concealment in HEVC

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

For error concealment of the relatively large corrupted areas in High Efficiency Video Coding (HEVC), the available spatial information is far from the corrupted region and cannot be directly exploited for error concealment. In this paper, a method is proposed to use the spatial information in a new manner to refine the already recovered Motion Vectors (MVs). The refinement method works based on boundary matching and adaptively selection among three approaches for fine tuning of the temporal MVs. The experiments show that the refinement leads to a significant improvement, 2–7 dB in PSNR, for some frames, and the highest MS-SSIM against the state of the art methods. Another important feature of the proposed method is its generality; which can be added on top of other MV recovery methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Adeyemi-Ejeye AO, Alreshoodi M, Al-Jobouri L, Fleury M (2019) Impact of packet loss on 4K UHD video for portable devices. Multimed Tools Appl 78:31733–31755

    Article  Google Scholar 

  2. Ahmed AV, Khot UP (2020) An efficient generalized error concealment in video codec. International Journal of Computer Vision and Image Processing 10(4):1–28

    Article  Google Scholar 

  3. Akbari A, Trocan M, Granado B (2017) Sparse recovery-based error concealment. IEEE Transactions on Multimedia 19(6):1339–1350

    Article  Google Scholar 

  4. Belfiore S, Grangetto M, Magli E, Olmo G (2005) Concealment of whole-frame losses for wireless low bit-rate video based on multiframe optical flow estimation. IEEE Trans Multimed 7(2):316–329

    Article  Google Scholar 

  5. Boussard V, Golaghazadeh F, Coulombe S, Coudoux FX, Corlay P (2020) Robust H.264 video decoding using CRC-based single error correction and non-desynchronizing bits validation. IEEE International Conference on Image Processing (ICIP)

  6. Byongsu H, Jonghyon J, Cholsu R (2019) An improved multi-directional interpolation for spatial error concealment. Multimed Tools Appl 78:2587–2598

    Article  Google Scholar 

  7. Chang YL, Reznick YA, Chen Z, Cosman PC (2013) Motion compensated error concealment for HEVC based on block-merging and residual energy. International Packet Video Workshop (PV)

  8. Chen C, Liu Y, Yang Z, Bu J, Deng X (2008) Multi-frame error concealment for H.264/AVC frames with complexity adaptation. IEEE Trans Consum Electron 54(3):1422–1429

    Article  Google Scholar 

  9. Chien JT, Li GL, Chen MJ (2010) Effective error concealment algorithm of whole frame loss for H.264 video coding standard by recursive motion vector refinement. IEEE Trans Consum Electron 56(3):1689–1695

    Article  Google Scholar 

  10. Choe G, Nam C, Chu C (2018) An effective temporal error concealment in H.264 video sequences based on scene change detection-PCA model. Multimed Tools Appl 77(24):31953–31967

    Article  Google Scholar 

  11. Chung B, Yim C (2019) Bi-sequential video error concealment method using adaptive Homography-based registration. IEEE Transactions on circuits and systems for video technology 30(6):1535–1549

    Article  Google Scholar 

  12. Fleury M, Moiron S, Ghanbari M (2011) Innovations in video error resilience and concealment. Recent Patents Signal Process 1(2):1–11

    Google Scholar 

  13. Francisco NYZ, et. al. (2017) Effects of error concealment on HEVC standard over wireless channels designed using Rayleigh fading and Markov model on AODV routed slices. IEEE 9th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management (HNICEM)

  14. Ghanbari M (2011) Standard codecs: image compression to advanced video coding. IET publication, 3rd edition, appendix E

  15. Hojati S, Kazemi M, Moallem P (2019) Error concealment with parallelogram partitioning of the lost area,” Multimed Tools Appl, 79: 7449–7469

  16. : https://media.xiph.org/video/derf/

  17. Huang Z, Cai Q (2018) A video data recovery algorithm in wireless communication networks. IEEE 18th International Conference on Communication Technology: 727–731.

  18. Hwang MC, Kim JH, Duong DT, Ko SJ (2008) Hybrid temporal error concealment methods for block-based compressed video transmission. IEEE Trans Broadcast 54(2):198–207

    Article  Google Scholar 

  19. Kazemi M (2020) In favor of fully intra coding for HEVC video transmission over lossy channels. Signal, Image and Video Processing, doi 15:165–173. https://doi.org/10.1007/s11760-020-01735-y

    Article  Google Scholar 

  20. Kazemi M, Ghanbari M, Shirmohammadi S (2020) Intra coding strategy for video error resiliency: behavioral analysis. IEEE Transactions on Multimedia 22(9):2193–2206

    Article  Google Scholar 

  21. Kazemi M, Ghanbari M, Shirmohammadi S (2020) The performance of quality metrics in assessing error-concealed video quality. IEEE Trans Image Process 29:5937–5952

    Article  Google Scholar 

  22. Kazemi M, Ghanbari M, Shirmohammadi S (2021) A review of temporal video error concealment techniques and their suitability for HEVC and VVC. Multimed Tools Appl 80:12685–12730. https://doi.org/10.1007/s11042-020-10333-6

    Article  Google Scholar 

  23. Kim DH, Kwon YJ, Choi KH (2018) Motion-vector refinement for video error concealment using downhill simplex approach. ETRI J 40(2):266–274

    Article  Google Scholar 

  24. Li Y, Chen R (2017) Motion vector recovery for video error concealment based on the plane fitting. Multimed Tools Appl 76(13):14993–15006

    Article  Google Scholar 

  25. Lie W, Lee C, Yeh C, Gao Z (2014) Motion vector recovery for video error concealment by using iterative dynamic programming optimization. IEEE Transactions on Multimedia 16(1):216–227

    Article  Google Scholar 

  26. Lin TL, Chen WC, Lai CK (2013) Recovery of lost motion vectors using encoded residual signals. IEEE Trans Broadcast 59(4):705–716

    Article  Google Scholar 

  27. Lin T, Yang N, Syu R, Liao CC, Tsai WL (2013) Error concealment algorithm for HEVC coded video using block partition decisions. IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC)

    Book  Google Scholar 

  28. Lin TL, Ding TL, Yang NC, Wu PY, Tung KH, Lai CK, Chang TE (2016) Video motion vector recovery method using decoding partition information. J Disp Technol 12(11):1451–1463

    Article  Google Scholar 

  29. Lin TL, Ding TL, Fan CY, Chen WC (2017) Error concealment algorithm based on sparse optimization. Multimed Tools Appl 76(1):397–413

    Article  Google Scholar 

  30. Liu X, Zhai D, Zhou J, Wang S, Zhao D, Gao H (2017) Sparsity-based image error concealment via adaptive dual dictionary learning and regularization. IEEE Trans Image Process 26(2):782–796

    Article  MathSciNet  Google Scholar 

  31. Nam C, Chu C, Kim T, Han S (2020) A novel motion recovery using temporal and spatial correlation for a fast temporal error concealment over H.264 video sequences. Multimed Tools Appl 79:1221–1240

    Article  Google Scholar 

  32. Peng YT, Cosman PC (2014) Weighted boundary matching error concealment for HEVC using block partition decisions. 48th Asilomar Conference on Signals, Systems and Computers: 921–925

  33. Peng Q, Yang T, Zhu C (2002) Block-based temporal error concealment for video packet using motion vector extrapolation. IEEE Int Conf Commun, Circuits Syst West Sino Expo 1:10–14

    Google Scholar 

  34. Punjabi SA, Katsaggelos AK (2018) Video error concealment using deep neural networks. In Proceedings of the IEEE International Conference on Image Processing, Athens, Greece, pp 380–384

    Google Scholar 

  35. Qian X, Liu G, Wang H (2009) Recovering connected error region based on adaptive error concealment order determination. IEEE Trans Multimedia 11(4):683–695

    Article  Google Scholar 

  36. Sankisa A, Punjabi A, Katsaggelos AK (2019) Optical flow prediction for blind and non-blind video error concealment using deep neural networks. International Journal of Multimedia Data Engineering and Management 10(3):27–46

    Article  Google Scholar 

  37. Sankisa A, Punjabi A, Katsaggelos AK (2020) Temporal capsule networks for video motion estimation and error concealment. SIViP 14:1369–1377

    Article  Google Scholar 

  38. Silva CAGD, Pedroso CM (2019) MAC-layer packet loss models for Wi-fi networks: a survey. IEEE Access 7:180512–180531

    Article  Google Scholar 

  39. Suh JW, Ho YS (2002) Error concealment techniques for digital TV. IEEE Trans Broadcast 48(4):299–306

    Article  Google Scholar 

  40. Tang CW, Chen CH, Yu YH, Tsai CJ (2006) Visual sensitivity guided bit allocation for video coding. IEEE Trans Multimedia 8(1):11–18

    Article  Google Scholar 

  41. Usman M, He X, Xu M, Lam K (2015) Survey of error concealment techniques: research directions and open issues. IEEE Picture Coding Symposium: 233-238

  42. Usman M, He X, Lam KM, Xu M, Bokhari SMM, Chen J (2016) Frame interpolation for cloud-based Mobile video streaming. IEEE Transactions on Multimedia 18(5):831–839

    Article  Google Scholar 

  43. Usman MA, Seong CH, Lee MH, Shin SY (2019) A novel error detection & concealment technique for videos streamed over error prone channels. Multimed Tools Appl 78:22959–22975

    Article  Google Scholar 

  44. Wang Y, Zhu QF (1998) Error control and concealment for video communication: a review. Proc IEEE 86(5):974–997

    Article  Google Scholar 

  45. Xiang J, Xu C, Yan Q, Peng XW (2019) Generative adversarial networks based error concealment for low resolution video. IEEE International Conference On Acoustics, Speech And Signal Processing (ICASSP)

    Book  Google Scholar 

  46. Xu J, Jiang W, Yan C, Peng Q, Wu X (2018) A novel weighted boundary matching error concealment Schema for HEVC. IEEE International Conference on Image Processing (ICIP)

  47. Yan B, Gharavi H (2009) A hybrid frame concealment algorithm for H.264/AVC. IEEE Trans Image Process 19(1):98–107

    Article  MathSciNet  Google Scholar 

  48. Yang SH, Chang CW, Chan CC (2015) An object-based error concealment technique for H.264 coded video. Multimed Tools Appl 74(23):10785–10800

    Article  Google Scholar 

  49. W. Yu, H. Sun, G. He and Z. Zhang (2017) A multi-step temporal error concealment method. International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS)

  50. Zhou J, Yan B, Gharavi H (2001) Efficient motion vector interpolation for error concealment of H.264/AVC. IEEE Trans Broadcast 57(1):57–80

    Google Scholar 

  51. Zhou Z, Dai M, Zhao R, Li B, Zhong H, Wen Y (2016) Video error concealment scheme based on tensor model. Multimed Tools Appl 76(14):16045–16061

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, University of Isfahan, Isfahan, 8174673441, Iran

    Mohammad Kazemi

Authors
  1. Mohammad Kazemi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Kazemi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kazemi, M. Refinement of the recovered motion vectors for error concealment in HEVC. Multimed Tools Appl 80, 27385–27405 (2021). https://doi.org/10.1007/s11042-021-11005-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11005-9

Keywords

Search

Navigation