Skip to main content
SpringerLink
Log in

Visual saliency based redundancy allocation in HEVC compatible multiple description video coding

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video delivery over error prone channels suffer from packet loss, which significantly deteriorates the reconstructed video quality. Multiple description coding (MDC) is an effective error resilient coding scheme for transmission of video over unreliable networks, where video is encoded into multiple descriptions having redundancy between them. The amount of redundancy between these descriptions has an important role in MDC, because it provides error resilience. In this paper, a new redundancy allocation scheme using visual saliency model is presented with an aim of achieving high decoding quality, when one or more than one descriptions are received at the decoder. The proposed MDC method splits the input video sequence into even and odd subsequences, which are independently encoded by using high efficiency video coding (HEVC) encoder. The missing frames in each description are predicted by using pixel interpolation and motion interpolation. The residual information is generated by using the interpolated frame and its adjacent frame from the other description, which represents the redundancy in each description. Residual information is encoded based on a visual saliency mask, which is generated using a global contrast based visual saliency model. In this work, two different modes are proposed for visual saliency based redundancy allocation to provide better perceptual quality, while decoding the descriptions. The proposed scheme is implemented on HEVC reference software HM 16.2 and compared with two state-of-the-art temporal subsampling based multiple description video coding methods. Experiments show that the proposed method performs better than the reference methods, both in lossless and packet erasure channel conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Adedoyin S, Fernando WAC, Karim HA, Hewage CT, Kondoz AM (2008) Scalable multiple description coding with side information using motion interpolation. IEEE Trans Consum Electron 54(4):2045–2052

    Article  Google Scholar 

  2. Bai H, Lin W, Zhang M, Wang A, Zhao Y (2014) Multiple description video coding based on human visual system characteristics. IEEE Trans Circ Syst Video Technol 24(8):1390–1394

    Article  Google Scholar 

  3. Chen J, Cai C, Li L, Li C (2016) Layered multiple description video coding using dual-tree discrete wavelet transform and h. 264/avc. Multimed Tools Appl 75 (5):2801–2814

    Article  Google Scholar 

  4. Cheng M-M, Mitra NJ, Huang X, Torr PH, Hu S-M (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582

    Article  Google Scholar 

  5. Choupani R, Wong S, Tolun M (2014) Multiple description coding for snr scalable video transmission over unreliable networks. Multimed Tools Appl 69(3):843–858

    Article  Google Scholar 

  6. Chung D-M, Wang Y (1998) Multiple description image coding based on lapped orthogonal transforms. In: ICIP 98. Proceedings. 1998 International Conference on Image Processing, vol 1. IEEE, pp 664–668

  7. Crave O, Pesquet-Popescu B, Guillemot C (2010) Robust video coding based on multiple description scalar quantization with side information. IEEE Trans Circ Syst Video Technol 20(6):769–779

    Article  Google Scholar 

  8. Franchi N, Fumagalli M, Lancini R, Tubaro S (2005) Multiple description video coding for scalable and robust transmission over ip. IEEE Trans Circ Syst Video Technol 15(3):321–334

    Article  Google Scholar 

  9. Frank B, David F, Karl S, Suhring K (2014) Hevc reference software hm 16.2 reference manual. Technical report, ITU-T/ISO/IEC JCTVC HEVC reference software manual

  10. Ghahremani S, Ghanbari M (2017) Error resilient video transmission in ad hoc networks using layered and multiple description coding. Multimed Tools Appl 76 (6):9033–9049

    Article  Google Scholar 

  11. Goyal VK (2001) Multiple description coding: Compression meets the network. IEEE Signal Process Mag 18(5):74–93

    Article  Google Scholar 

  12. Goyal VK, Kovacevic J, Arean R, Vetterli M (1998) Multiple description transform coding of images. In: ICIP 98. Proceedings. 1998 International Conference on Image Processing, vol 1. IEEE, pp 674–678

  13. Hellge C, Gomez-Barquero D, Schierl T, Wiegand T (2011) Layer-aware forward error correction for mobile broadcast of layered media. IEEE Trans Multimed 13(3):551–562

    Article  Google Scholar 

  14. Heng BA, Apostolopoulos JG, Lim JS (2006) End-to-end rate-distortion optimized md mode selection for multiple description video coding. EURASIP J Appl Signal Process 2006:261–261

    MATH  Google Scholar 

  15. Huo Y, El-Hajjar M, Hanzo L (2013) Inter-layer fec aided unequal error protection for multilayer video transmission in mobile tv. IEEE Trans Circ Syst Video Technol 23(9):1622–1634

    Article  Google Scholar 

  16. Huo Y, Zhu C, Hanzo L (2013) Spatio-temporal iterative source–channel decoding aided video transmission. IEEE Trans Veh Technol 62(4):1597–1609

    Article  Google Scholar 

  17. Huo Y, Hellge C, Wiegand T, Hanzo L (2015) A tutorial and review on inter-layer fec coded layered video streaming. IEEE Commun Surv Tutorials 17 (2):1166–1207

    Article  Google Scholar 

  18. Karim HA, Hewage CT, Worrall S, Kondoz AM (2008) Scalable multiple description video coding for stereoscopic 3d. IEEE Trans Consum Electron 54(2):745–752

    Article  Google Scholar 

  19. Kazemi M, Sadeghi KH, Shirmohammadi S, Moallem P (2015) Rate/distortion optimization in multiple description video coding. Signal Process Image Commun 36:95–105

    Article  Google Scholar 

  20. Kim IK, Cho NI (2006) Hybrid multiple description video coding using optimal dct coefficient splitting and sd/md switching. Signal Process Image Commun 21(4):293–305

    Article  Google Scholar 

  21. Kim C-K, Lee H-r, Jung T-j, Kim B-G, Seo K-d (2016) An efficient delay-constrained arq scheme for mmt packet-based real-time video streaming over ip networks. J Real-Time Image Process 12(2):257–271

    Article  Google Scholar 

  22. Lin C, Tillo T, Zhao Y, Jeon B (2011) Multiple description coding for h. 264/avc with redundancy allocation at macro block level. IEEE Trans Circ Syst Video Technol 21(5):589–600

    Article  Google Scholar 

  23. Majid M, Abhayaratne C (2009) Multiple description image coding using several multiple description scalar quantizers. In: IS&T/SPIE Electronic Imaging, International Society for Optics and Photonics, pp 725711–725711

  24. Majid M, Abhayaratne C (2009) Multiple description scalar quantization with successive refinement. In: Signal Processing Conference, 2009 17th European, IEEE, pp 2268–2272

  25. Majid M, Abhayaratne C (2010) Successive refinement of overlapped cell side quantizers for scalable multiple description coding. In: Picture Coding Symposium (PCS) 2000. IEEE, pp 286–289

  26. Majid M, Abhayaratne C (2012) Inter-description motion vector redundancy control for scalable multiple description video coding. In: IET Conference on Image Processing (IPR)

  27. Majid M, Abhayaratne C (2012) Redundancy controllable scalable unbalanced multiple description bitstream generation for peer-to-peer video streaming. Signal Process Image Commun 27(5):496–512

    Article  Google Scholar 

  28. Servetto SD, Ramchandran K, Vaishampayan VA, Nahrstedt K (2000) Multiple description wavelet based image coding. IEEE Trans Image Process 9(5):813–826

    Article  Google Scholar 

  29. Stephan W (1999) Error patterns for internet video experiments, ITU-T SG16 Document Q15-I-16-R1, pp 2004–2005

  30. Sullivan GJ, Ohm J, Han W-J, Wiegand T (2012) Overview of the high efficiency video coding (hevc) standard. IEEE Trans Circ Syst Video Technol 22 (12):1649–1668

    Article  Google Scholar 

  31. Sun G, Samarawickrama U, Liang J, Tu C, Tran T (2007) Multiple description image coding with prediction compensation. In: Proceedings of 2007 IEEE International Conference on Image Processing, pp 513–516

  32. Tillo T, Grangetto M, Olmo G (2008) Redundant slice optimal allocation for h. 264 multiple description coding. IEEE Trans Circ Syst Video Technol 18(1):59–70

    Article  Google Scholar 

  33. Vaishampayan VA (1993) Design of multiple description scalar quantizers. IEEE Trans Inf Theory 39(3):821–834

    Article  MathSciNet  MATH  Google Scholar 

  34. Vaishampayan VA (1996) Application of multiple description codes to image and video transmission over lossy networks. In: Proceedings of 7th International Workshop Packet Video, pp 55–60

  35. Verdicchio F, Munteanu A, Gavrilescu AI, Cornelis J, Schelkens P (2006) Embedded multiple description coding of video. IEEE Trans Image Process 15 (10):3114–3130

    Article  Google Scholar 

  36. Wang Y, Orchard MT, Reibman AR (1997) Multiple description image coding for noisy channels by pairing transform coefficients. In: 1997., IEEE First Workshop on Multimedia Signal Processing, IEEE, pp 419–424

  37. Wang Y, Orchard MT, Vaishampayan V, Reibman AR (2001) Multiple description coding using pairwise correlating transforms. IEEE Trans Image Process 10 (3):351–366

    Article  MATH  Google Scholar 

  38. Wang Y, Reibman AR, Lin S (2005) Multiple description coding for video delivery. Proc IEEE 93(1):57–70

    Article  Google Scholar 

  39. Wu J, Yuen C, Chen J (2015) Leveraging the delay-friendliness of tcp with fec coding in real-time video communication. IEEE Trans Commun 63(10):3584–3599

    Article  Google Scholar 

  40. Xie X, Zaitsev Y, Velasquez-Garcia L, Teller S, Livermore C (2014) Compact, scalable, highresolution, mems-enabled tactile displays. In: Proceedings of Solid-State Sensors, Actuators, and Microsystems Workshop, pp 127–30

  41. Xie X, Zaitsev Y, Velásquez-García LF, Teller S, Livermore C (2014) Scalable, mems-enabled, vibrational tactile actuators for high resolution tactile displays. J Micromech Microeng 24(12):125014

    Article  Google Scholar 

  42. Xu Y, Zhu C (2013) End-to-end rate-distortion optimized description generation for h. 264 multiple description video coding. IEEE Trans Circ Sys Video Technol 23(9):1523–1536

    Article  Google Scholar 

  43. Yang M, Gadgil N, Comer ML, Delp EJ (2016) Adaptive error concealment for temporal–spatial multiple description video coding. Signal Process Image Commun 47:313–331

    Article  Google Scholar 

  44. Zhang M, Bai H (2012) Adaptive multiple description video coding and transmission for scene change. EURASIP J Wirel Commun Netw 2012(1):265

    Article  Google Scholar 

Download references

Acknowledgements

We are thankful to Mr. Muhammad Zeeshan for his help in the analysis of visual saliency model for noisy inputs.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, University of Engineering and Technology, Taxila, Pakistan

    Muhammad Majid & Muhammad Owais

  2. Department of Software Engineering, University of Engineering and Technology, Taxila, Pakistan

    Syed Muhammad Anwar

Authors
  1. Muhammad Majid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Owais
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Syed Muhammad Anwar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Majid.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Majid, M., Owais, M. & Anwar, S.M. Visual saliency based redundancy allocation in HEVC compatible multiple description video coding. Multimed Tools Appl 77, 20955–20977 (2018). https://doi.org/10.1007/s11042-017-5499-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5499-7

Keywords

Search

Navigation