Abstract
Low-delay hierarchical coding structure (LD-HCS), as one of the most crucial components in the High Efficiency Video Coding (HEVC) standard, substantially complicates the temporal reference relationship and achieves better compression ratios. There have been lots of research work on increasing the error resilience of HEVC coding. However, most of them neglect the characteristics of LD-HCS and cannot achieve the optimum trade-off between coding efficiency and error robustness. To this end, in this paper, a non-feedback reference picture selection algorithm is proposed to improve the error robustness of HEVC streaming under LD-HCS, by reducing temporal error propagation in case of packet loss. Specifically, the temporal relationship among different frames under LD-HCS is firstly analyzed, and then frames with the lowest temporal layer are considered as the potential key-frame which is capable of preventing error propagation. Secondly, the end-to-end distortions of using two different reference schemes for the potential key-frame are compared based on an overall distortion estimation model, and the reference scheme that has less total distortion is chosen ultimately. Moreover, the potential key-frame will be encoded entirely in intra-mode if the error propagation is severe. Extensive experiment results demonstrate that the proposed reference picture selection method significantly reduces the effect of temporal error propagation. Compared to the state-of-the-art approaches, the proposed method is able to improve the video quality over error-prone networks.
Similar content being viewed by others
Data and materials availability
All 18 tested sequences are taken from HEVC standard video set [19].
References
Li, T., Xu, M., Zhu, C., Yang, R., Wang, Z., Guan, Z.: A deep learning approach for multi-frame in-loop filter of HEVC. IEEE Trans. Image Process. 28(11), 5663–5678 (2019)
Gao, Y., Zhu, C., Li, S., Yang, T.: Source distortion temporal propagation analysis for random-access hierarchical video coding optimization. IEEE Trans. Circuits Syst. Video Technol. 29(2), 546–559 (2019)
Wang, D., Zhu, C., Sun, Y., Dufaux, F., Huang, Y.: Efficient multi-strategy intra prediction for quality scalable high efficiency video coding. IEEE Trans. Image Process. 28(4), 2063–2074 (2019)
Gao, Y., Zhu, C., Li, S., Yang, T.: Temporally dependent rate-distortion optimization for low-delay hierarchical video coding. IEEE Trans. Image Process. 26(9), 4457–4470 (2017)
Liu, H., Zhu, S., Zeng, B.: Inter-frame dependency-based rate control for VVC low-delay coding. IEEE Signal Process. Lett. (2023). https://doi.org/10.1109/LSP.2023.3234774
Sullivan, G.J., Ohm, J., Han, W.-J., Wiegand, T.: Overview of the high efficiency video coding (hevc) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012)
Carreira, J., Assunçao, P., de Faria, S., Ekmekcioglu, E., Kondoz, A.: A two-stage approach for robust HEVC coding and streaming. IEEE Trans. Circuits Syst. Video Technol. 28(8), 1960–1973 (2018)
Kazemi, M., Ghanbari, M., Shirmohammadi, S.: The performance of quality metrics in assessing error-concealed video quality. IEEE Trans. Image Process. 29, 5937–5952 (2020)
Sun, X., Wang, M., Lin, R., Sun, Y., Shin Cheng, S.: Deep-learned perceptual quality control for intelligent video communication. IEEE Trans. Consum. Electron. 68(4), 354–365 (2022)
Moiron, S., Ghanbari, M.: Enhanced error resiliency for video with cyclic intra-refresh lines. In: IEEE International Conference on Image Processing (ICIP), pp. 3229–3232 (2011)
Chen, H., Zhao, C., Sun, M.-T., Drake, A.: Adaptive intra-refresh for low-delay error-resilient video coding. J. Vis. Commun. Image. R. 31, 294–304 (2015)
Zhang, R., Regunathan, S.L., Rose, K.: Video coding with optimal inter/intra-mode switching for packet loss resilience. IEEE J. Sel. Areas Commun. 18(6), 966–976 (2000)
He, Z., Cai, J., Chen, C.W.: Joint source channel rate-distortion analysis for adaptive mode selection and rate control in wireless video coding. IEEE Trans. Circuits Syst. Video Technol. 12(6), 511–523 (2002)
Kulupana, G., Talagala, D.S., Arachchi, H.K., Fernando, A.: End user video quality prediction and coding parameters selection at the encoder for robust HEVC video transmission. IEEE Trans. Circuits Syst. Video Technol. 29(11), 3367–3381 (2019)
Moiron, S., Ali, I., Ghanbari, M., Fleury, M.: Limitations of multiple reference frames with cyclic intra-refresh line for H.264/AVC. Electron. Lett. 47, 103–104 (2011)
Kazemi, M., Ghanbari, M., Shirmohammadi, S.: Intra coding strategy for video error resiliency: Behavioral analysis. IEEE Trans. Multimedia 22(9), 2193–2206 (2020)
Kazemi, M.: In favor of fully intra coding for HEVC video transmission over lossy channels. SIViP 15(1), 165–173 (2021)
Peng, Q., Zhang, L., Yang, T., Chen, J., Zhu, C.: Key-frame reference selection for non-feedback video communication. J. China Univ. Posts Telecommun. 16(5), 92–102 (2009)
Bossen, F.: Common test conditions and software reference configurations. document JCTVC-H1100. (2013)
Bjontegaard: Calculation of average PSNR differences between RD-curves, Document VCEG-M33. (2001)
Funding
This work was supported in part by the National Natural Science Foundation of China (Grant No. 62001400), Sichuan Science and Technology Program (Grant No. 2021YJ0364), Foundation for Department of Transportation of Henan Province, China (2022-4-6), Grant of Institute of Applied Physics and Computational Mathematics, Beijing (Grant No. HXO2020-118), Guangxi Science and Technology Program (Grant Nos. 2021AB07044, 2021AB07043 and 2021AB40145) and China Postdoctoral Science Foundation (Grant No. 2021M702713). The Natural Science Foundation of Chongqing, China (Grant No. CSTB2023NSCQ-BHX0224).
Author information
Authors and Affiliations
Contributions
JX was involved in conceptualization, methodology, software, writing and preparing the original draft and visualization; JX and BW had contributed to validation and formal analysis; BW, WL and QP carried out writing, reviewing, editing, supervision and project administration. All authors have read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Xu, J., Wang, B., Peng, Q. et al. Key-frame reference selection for error resilient video coding using low-delay hierarchical coding structure. SIViP 18, 215–222 (2024). https://doi.org/10.1007/s11760-023-02742-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-023-02742-5