Skip to main content
SpringerLink
Log in

HEVC compatible perceptual multiple description video coding for reliable video transmission over packet networks

  • Published:
Telecommunication Systems Aims and scope Submit manuscript

Abstract

Multiple description coding (MDC) has been an effective scheme for reliable transmission of videos over error prone networks but requires higher data rates. Recently, perceptual video coding schemes are able to encode videos at a lower data rate, but with the same perceived decoding quality. Such schemes exploit human visual system (HVS) properties to encode videos. Although HVS characteristics have been used in MDC, but only for residual information and are not fully standard compatible. To this end, we propose a high efficiency video coding (HEVC) standard compatible perceptual multiple description video coding (PMDVC) framework. Our proposed framework temporally sub-samples the input video into even and odd frame sub-sequences, which are individually encoded (using HEVC) in a perceptual manner. We encode each description using a context based visual saliency model to obtain visual saliency mask, which is optimally thresholded into salient and non-salient regions. Coding tree unit (CTU) level perceptual relevance mask of each frame is generated by dividing binary saliency mask into five different groups. The quantization parameter at CTU level of each perceptual relevant group is adjusted in such a manner that data rate is minimized while maintaining the perceived quality. Our proposed HEVC based PMDVC scheme is evaluated under lossless and lossy channel conditions. Our results show better performance in data rate reduction, perceptual peak signal-to-noise-ratio, multi- scale structural similarity index, and difference mean opinion score, when compared with HEVC based temporally sub-sampled multiple description video coding scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Alreshoodi, M., Adeyemi-Ejeye, F., Aljobouri, L., Fleury, M., & Alzahrani, B. (2017). Packet loss visibility for higher resolution video on portable devices. In 2017 IEEE international conference on consumer electronics (ICCE), IEEE (pp. 237–238).

  2. Mendoza, S., & Smith, J. (2018). Mobile video optimization, US Patent 10148512 (4 Dec 2018).

  3. Sullivan, G. J., Ohm, J., Han, W.-J., & Wiegand, T. (2012). Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits and Systems for Video Technology, 22(12), 1649–1668.

    Article  Google Scholar 

  4. Ohm, J. R., Sullivan, G. J., Schwarz, H., Tan, T. K., & Wiegand, T. (2012). Comparison of the coding efficiency of video coding standards including high efficiency video coding (HEVC). IEEE Transactions on Circuits and Systems for Video Technology, 22(12), 1669–1684.

    Article  Google Scholar 

  5. Wang, C., Sklar, D., & Johnson, D. (2001). Forward error-correction coding. Crosslink, 3(1), 26–29.

    Google Scholar 

  6. Huo, Y., El-Hajjar, M., & Hanzo, L. (2013). Inter-layer FEC aided unequal error protection for multilayer video transmission in mobile TV. IEEE Transactions on Circuits and Systems for Video Technology, 23(9), 1622–1634.

    Article  Google Scholar 

  7. Kim, C. K., Lee, H. R., Jung, T. J., Kim, B. G., & Seo, K. D. (2016). An efficient delay-constrained ARQ scheme for MMT packet-based real-time video streaming over IP networks. Journal of Real-Time Image Processing, 12(2), 257–271.

    Article  Google Scholar 

  8. Goyal, V. K. (2001). Multiple description coding: compression meets the network. IEEE Signal processing magazine, 18(5), 74–93.

    Article  Google Scholar 

  9. Kazemi, M., Iqbal, R., & Shirmohammadi, S. (2018). Joint intra and multiple description coding for packet loss resilient video transmission. IEEE Transactions on Multimedia, 20(4), 781–795.

    Article  Google Scholar 

  10. Li, H., Meng, L., Zhang, J., Tan, Y., Ren, Y., & Zhang, H. (2019). Multiple description coding based on convolutional auto-encoder. IEEE Access, 7, 26013–26021.

    Article  Google Scholar 

  11. Zhao, L., Bai, H., Wang, A., & Zhao, Y. (2019). Multiple description convolutional neural networks for image compression. IEEE Transactions on Circuits and Systems for Video Technology, 29(8), 2494–2508.

    Article  Google Scholar 

  12. Chaoub, A., Ennaoui, F. Z., & Ibn-Elhaj, E. (2019). Reliable rate-adaptive video transmission over cognitive cellular networks using multiple description scalable coding. Telecommunication Systems, 71(3), 321–338.

    Article  Google Scholar 

  13. Wang, Y., Reibman, A. R., & Lin, S. (2005). Multiple description coding for video delivery. Proceedings of the IEEE, 93(1), 57–70.

    Article  Google Scholar 

  14. Venkataramani, R., Kramer, G., & Goyal, V. K. (2003). Multiple description coding with many channels. IEEE Transactions on Information Theory, 49(9), 2106–2114.

    Article  Google Scholar 

  15. Kazemi, M., Shirmohammadi, S., & Sadeghi, K. H. (2014). A review of multiple description coding techniques for error-resilient video delivery. Multimedia Systems, 20(3), 283–309.

    Article  Google Scholar 

  16. Gallant, M., Shirani, S., & Kossentini, F. (2001). Standard-compliant multiple description video coding, in: Proceedings 2001 international conference on image processing (Cat. No. 01CH37205), Vol. 1, IEEE, pp. 946–949.

  17. Chung, D. M., & Wang, Y. (1998). Multiple description image coding based on lapped orthogonal transforms. In Proceedings 1998 international conference on image processing. ICIP98 (Cat. No. 98CB36269), IEEE vol. 1, (pp. 664–668).

  18. Wang, Y., Orchard, M. T., Vaishampayan, V., & Reibman, A. R. (2001). Multiple description coding using pairwise correlating transforms. IEEE Transactions on Image Processing, 10(3), 351–366.

    Article  Google Scholar 

  19. Wang, Y., Reibman, A. R., & Lin, S. (2004). Multiple description coding for video delivery. Proceedings of the IEEE, 93(1), 57–70.

    Article  Google Scholar 

  20. Vaishampayan, V. A. (1993). Design of multiple description scalar quantizers. IEEE Transactions on Information Theory, 39(3), 821–834.

    Article  Google Scholar 

  21. Verdicchio, F., Munteanu, A., Gavrilescu, A. I., Cornelis, J., & Schelkens, P. (2006). Embedded multiple description coding of video. IEEE Transactions on Image Processing, 15(10), 3114–3130.

    Article  Google Scholar 

  22. Crave, O., Pesquet-Popescu, B., & Guillemot, C. (2010). Robust video coding based on multiple description scalar quantization with side information. IEEE Transactions on Circuits and Systems for Video Technology, 20(6), 769–779.

    Article  Google Scholar 

  23. Majid, M., & Abhayaratne, C. (2012). Redundancy controllable scalable unbalanced multiple description bitstream generation for peer-to-peer video streaming. Signal Processing: Image Communication, 27(5), 496–512.

    Google Scholar 

  24. Majid, M., Abhayaratne, C. (2009). Multiple description scalar quantization with successive refinement. In 17th European signal processing conference, IEEE (pp. 2268–2272).

  25. Karim, H. A., Hewage, C. T., Worrall, S., & Kondoz, A. M. (2008). Scalable multiple description video coding for stereoscopic 3D. IEEE Transactions on Consumer Electronics, 54(2), 745–752.

    Article  Google Scholar 

  26. Adedoyin, S., Fernando, W. A. C., Karim, H. A., Hewage, C. T., & Kondoz, A. M. (2008). Scalable multiple description coding with side information using motion interpolation. IEEE Transactions on Consumer Electronics, 54(4), 2045–2052.

    Article  Google Scholar 

  27. Majid, M., Owais, M., & Anwar, S. M. (2018). Visual saliency based redundancy allocation in HEVC compatible multiple description video coding. Multimedia Tools and Applications, 77(16), 20955–20977.

    Article  Google Scholar 

  28. Lee, J. S., & Ebrahimi, T. (2012). Perceptual video compression: a survey. IEEE Journal of Selected Topics in Signal Processing, 6(6), 684–697.

    Article  Google Scholar 

  29. Zeeshan, M., & Majid, M. (2019). High efficiency video coding compliant perceptual video coding using entropy based visual saliency model. Entropy, 21(10), 964.

    Article  Google Scholar 

  30. Chen, Y., Wu, K., & Zhang, Q. (2014). From QoS to QoE: a tutorial on video quality assessment. IEEE Communications Surveys and Tutorials, 17(2), 1126–1165.

    Article  Google Scholar 

  31. Kufa, J., Polak, L., Kratochvil, T. (2016). HEVC, H. 265 versus VP9 for full HD and UHD video: Is there any difference in QoE? In International symposium ELMAR, IEEE (pp 51–55).

  32. Zach, O., & Slanina, M. (2014). A Matlab-based tool for video quality evaluation without reference. Radioengineering, 23(1), 405–411.

    Google Scholar 

  33. Sumbal, R., Majid, & Anwar, S. M. (2013) Region of interest (ROI) based scalable multiple description image coding using MDSQ-SR. In 2013 IEEE 9th international conference on emerging technologies (ICET), IEEE (pp. 1–5).

  34. Lin, C., Zhao, Y., Xiao, J., & Tillo, T. (2017). Region-based multiple description coding for multiview video plus depth video. IEEE Transactions on Multimedia, 20(5), 1209–1223.

    Article  Google Scholar 

  35. Bai, H., Lin, W., Zhang, M., Wang, A., & Zhao, Y. (2014). Multiple description video coding based on human visual system characteristics. IEEE Transactions on Circuits and Systems for Video Technology, 24(8), 1390–1394.

    Article  Google Scholar 

  36. Frintrop, S., Rome, E., & Christensen, H. I. (2010). Computational visual attention systems and their cognitive foundations: a survey. ACM Transactions on Applied Perception, 7(1), 6.

    Article  Google Scholar 

  37. Zeeshan, M., Majid, M., Nizami, I. F., Anwar, S. M., Din, I. U., & Khan, M. K. (2018). A newly developed ground truth dataset for visual saliency in videos. IEEE Access, 6, 20855–20867.

    Article  Google Scholar 

  38. Yuan, D., Zhao, T., Xu, Y., Xue, H., & Lin, L. (2019). Visual JND: a perceptual measurement in video coding. IEEE Access, 7, 29014–29022.

    Article  Google Scholar 

  39. H. Jiang, J. Wang, Z. Yuan, T. Liu, N. Zheng, S. Li, (2011). Automatic salient object segmentation based on context and shape prior. In BMVC vol. 6 (pp. 110.1–110.12). Dundee, Scotland, August–September.

  40. Zhang, L., Yang, C., Lu, H., Ruan, X., & Yang, M. (2017). Ranking saliency. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(9), 1892–1904.

    Article  Google Scholar 

  41. Common HM Test conditions and software reference configurations (2012). Tech. rep., document JCT-VC of ISO/IEC MPEG and ITU-T VCEG, JCTVC-K1100. Shanghai, China.

  42. HM Reference Software 16.11, Accessed: 25 Jan 2018. [Online]. Available: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/.

  43. JCT-VC AHG report: Test sequence material (AHG5), Tech. rep., Joint collaborative team on video coding (JCT-VC) (Macao, China, Oct 2017).

  44. P. Series (2008). Subjective video quality assessment methods for multimedia applications, Recommendation ITU-T P.910 (pp. 1–34).

  45. Series, B. (2012). Methodology for the subjective assessment of the quality of television pictures. Recommendation ITU-R BT (pp. 500–513).

  46. Z. Wang, E. P. Simoncelli, A. C. Bovik (2003). Multiscale structural similarity for image quality assessment. In Proceedings of IEEE thirty-seventh asilomar conference on signals, systems and computers, vol. 2 (pp. 1398–1402).

  47. Common conditions for wire-line. (September 2001). low delay IP/UDP/RTP packet loss resilient testing. ITU-T video coding experts group VCEG (Santa Barbara, CA, USA: Tech. rep).

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, University of Engineering and Technology Taxila, Taxila, 47050, Pakistan

    Muhammad Zeeshan & Muhammad Majid

Authors
  1. Muhammad Zeeshan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Majid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Majid.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zeeshan, M., Majid, M. HEVC compatible perceptual multiple description video coding for reliable video transmission over packet networks. Telecommun Syst 76, 63–83 (2021). https://doi.org/10.1007/s11235-020-00702-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11235-020-00702-9

Keywords

Search

Navigation