Skip to main content
SpringerLink
Log in

Perceptual importance analysis-based rate control method for HEVC

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

High efficiency video coding (HEVC) has achieved high coding efficiency as the video coding standard. For rate control in HEVC, the conventional R-λ scheme is based on mean absolute difference in allocating bits; however, the scheme does not fully utilize the perceptual importance variation to guide rate control, thus the subjective and objective quality of coded videos has room to improve. Therefore, in this paper, we propose a rate control scheme that considers perceptual importance. We first develop a perceptual importance analysis scheme to accurately abstract the spatial and temporal perceptual importance maps of video contents. The results of the analysis are then used to guide the bit allocation. Utilizing this model, a region-level bit allocation procedure is developed to maintain video quality balance. Subsequently, a largest coding unit (LCU)-level bit allocation scheme is designed to obtain the target bit of each LCU. To achieve a more accurate bitrate, an improved R-λ model based on the Broyden-Fletcher-Goldfarb-Shanno model is utilized to update the R-λ parameter. The experimental results showed that our method not only improved subjective and objective video quality with lower bitrate errors compared to the original RC in HEVC, but also outperformed state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. An C, Nguyen TQ (2008) Iterative rate-distortion optimization of H. 264 with constant bit rate constraint. IEEE Trans Image Process 17(9):1605–1615

    Article  MathSciNet  Google Scholar 

  2. Bai L, Song L, Xie R, Xie J, Chen M (Dec 2016) Saliency based rate control scheme for high efficiency video coding. In: Proc. IEEE Int. Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), pp. 1–6

  3. Bazen AM, Gerez SH (2002) Systematic methods for the computation of the directional fields and singular points of fingerprints. IEEE Trans Pattern Anal Mach Intell 24(7):905–919

    Article  Google Scholar 

  4. Bross B, (Oct. 2012) High efficiency video coding (HEVC) text specification draft 9 (SoDIS). In: 11th JCT-VC meeting

  5. Chadha A, Andreopoulos Y, (2021) Deep Perceptual Preprocessing for Video Coding,” In: Proc. IEEE Int. Computer Vision and Pattern Recognition (CVPR), pp. 14852–14861

  6. Chen Z, Pan X (2019) An optimized rate control for low-delay H. 265/HEVC. IEEE Trans Image Process 28(9):4541–4552

    Article  MathSciNet  Google Scholar 

  7. Choi H, Nam J, Yoo J, Sim D, Bajic IV, (2012) Rate control based on unified RQ model for HEVC, ITU-T SG16 contribution, JCTVC-H0213, 1–13

  8. Choi H, Nam J, Yoo J, Sim D, Bajic IV, (April 2012) Improvement of the rate control based on pixel-based URQ model for HEVC, ITU-T/ISO/IEC JCT-VC Document In JCT-VC I0094

  9. Dong J, Ling N (2009) A context-adaptive prediction scheme for parameter estimation in H. 264/AVC macroblock layer rate control. IEEE Trans Circuits Syst Video Technol 19(8):1108–1117

    Article  Google Scholar 

  10. Gao Y, Zhu C, Li S, Yang T (2017) Temporally dependent rate-distortion optimization for low-delay hierarchical video coding. IEEE Trans Image Process 4457-4470:4457–4470

    Article  MathSciNet  Google Scholar 

  11. Gao W, Kwong S, Jia Y (2017) Joint machine learning and game theory for rate control in high efficiency video coding. IEEE Trans Image Process 26(12):6074–6089

    Article  MathSciNet  Google Scholar 

  12. Gao W, Kwong S, Jiang Q, Fong CK, Wong PH, Yuen WY (2018) Data-driven rate control for rate-distortion optimization in HEVC based on simplified effective initial QP learning. IEEE Trans Broadcasting 65(1):94–108

    Article  Google Scholar 

  13. Girod B (1993) What’s wrong with mean-squared error? Digital images and human vision[J]. AB Watson ed, pp 207–220

  14. Gong Y, Wan S, Yang K, Wu HR, Liu Y (2019) Temporal-layer-motivated lambda domain picture level rate control for random-access configuration in H.265/HEVC. IEEE Trans Circuits Syst Video Technol 29(1):156–170

    Article  Google Scholar 

  15. Guo H, Zhu C, Xu M, Li S (2019) Inter-Block Dependency-Based CTU Level Rate Control for HEVC. IEEE Trans Broadcasting 66(1):113–126

    Article  Google Scholar 

  16. svn_HEVCSoftware. HM Reference Software 16.19. [Online]. Available: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.19/. Accessed 2018

  17. ITU-R, Methodology for the subjective assessment of the quality of television pictures, ITU-R Recommendation BT.500–10. 2000.

  18. Lee JS, Ebrahimi T (2012) Perceptual video compression: A survey. IEEE J Sel Top Signal Process 6(6):684–697

    Article  Google Scholar 

  19. Lee B, Kim M, Nguyen TQ (2013) A frame-level rate control scheme based on texture and nontexture rate models for high efficiency video coding. IEEE Trans Circuits Syst Video Technol 24(3):465–479

    Article  Google Scholar 

  20. Li B, Li H, Li L, Zhang J, (Oct. 2012) Rate control by R-lambda model for HEVC, ITU-T/ISO/IEC JCT-VC document JCTVC-K0103, Shanghai, CN

  21. Li B, Li H, Li L, (2013) Adaptive bit allocation for R-lambda modelrate control in HM, ITU-T/ISO/IEC JCT-VC document JCTVC-M0036, Incheon, Korea

  22. Li B, Li H, Li L, Zhang J (2014) λ domain rate control algorithm for high efficiency video coding. IEEE Trans Image Process 23(9):3841–3854

    Article  MathSciNet  Google Scholar 

  23. Li Y, Liao W, Huang J, He D, Chen Z, (July 2014) Saliency based perceptual HEVC. In: Proc. IEEE Int. Multimedia and Expo Workshops (ICMEW), pp. 1–5.

  24. Li S, Xu M, Deng X, Wang Z (2015) Weight-based R-λ rate control for perceptual HEVC coding on conversational videos. Signal Process Image Commun 38:127–140

    Article  Google Scholar 

  25. Li L, Li B, Li H, Chen CW (2016) λ-Domain optimal bit allocation algorithm for high efficiency video coding. IEEE Trans Circuits Syst Video Technol 28(1):130–142

    Article  Google Scholar 

  26. Li S, Xu M, Wang Z, Sun X (2016) Optimal bit allocation for CTU level rate control in HEVC. IEEE Trans Circuits Syst Video Technol 27(11):2409–2424

    Article  Google Scholar 

  27. Liang X, Wang Q, Zhou Y, Luo B, Men, A, (Nov 2013) A novel RQ model based rate control scheme in HEVC. In: Proc. IEEE Int. Visual Communications and Image Processing (VCIP), pp. 1–6

  28. Lim KP, Sullivan G, Wiegand T, (2005) Text description of joint model reference encoding methods and decoding concealment methods. JVT-O079, Busan, Korea

  29. Lin H, He X, Teng QZ, Fu W, Xiong S (2016) Adaptive bit allocation scheme for extremely low-delay intraframe rate control in high efficiency video coding. Journal of Electronic Imaging 25(4):043008

    Article  Google Scholar 

  30. Liu Y, Li ZG, Soh YC (2006) A novel rate control scheme for low delay video communication of H. 264/AVC standard. IEEE Trans Circuits Syst Video Technol 17(1):68–78

    Article  Google Scholar 

  31. Ma YF, Zhang HJ, (Nov 2003) Contrast-based image attention analysis by using fuzzy growing. In: Proc. the eleventh ACM international conference on Multimedia, pp. 374–381

  32. Meddeb M, Cagnazzo M, Pesquet-Popescu B, (2014) Region-of-interest-based rate control scheme for high-efficiency video coding. APSIPA Transactions on Signal and Information Processing, 3

  33. Nami S, Pakdaman F, Hashemi MR, (2020) Juniper: A Jnd-Based Perceptual Video Coding Framework to Jointly Utilize Saliency and JND. In: Proc. IEEE Int. Multimedia & Expo Workshops (ICMEW), July 2020, pp. 1–6.

  34. Oh H, Kim W (2012) Video processing for human perceptual visual quality-oriented video coding. IEEE Trans Image Process 22(4):1526–1535

    Article  MathSciNet  Google Scholar 

  35. Ohm JR, Sullivan GJ, Schwarz H, Tan TK, Wiegand T (2012) Comparison of the coding efficiency of video coding standards—including high efficiency video coding (HEVC). IEEE Trans Circuits Syst Video Technol 22(12):1669–1684

    Article  Google Scholar 

  36. Ou YF, Ma Z, Liu T, Wang Y (2010) Perceptual quality assessment of video considering both frame rate and quantization artifacts. IEEE Trans Circuits Syst Video Technol 21(3):286–298

    Article  Google Scholar 

  37. Park S, Kim M, (Nov 2006) Extracting moving/static objects of interest in video,” In: Proc. Pacific-Rim Conference on Multimedia, pp. 722–729

  38. Ribas-Corbera J, Lei S (1999) Rate control in DCT video coding for low-delay communications. IEEE Trans Circuits Syst Video Technol 9(1):172–185

    Article  Google Scholar 

  39. Seshadrinathan K, Bovik AC (2009) Motion tuned spatio-temporal quality assessment of natural videos. IEEE Trans Image Process 19(2):335–350

    Article  MathSciNet  Google Scholar 

  40. Takeuchi M, Saika S, Sakamoto Y, Nagashima T, Cheng Z, Kanai K, Wei X (2018) Perceptual quality driven adaptive video coding using JND estimation. In: Proc. IEEE Int. Picture Coding Symposium (PCS), pp. 179–183

  41. Wang Z, Zeng H, Chen J, Cai C (June 2014) Key techniques of high efficiency video coding standard and its extension. In: Proc. IEEE Int. Industrial Electronics and Applications, pp. 1169–1173.

  42. Wang H, Song L, Xie R, Luo Z, Wang X (May 2018) Masking Effects Based Rate Control Scheme for High Efficiency Video Coding. In: Proc. IEEE Int. Symposium on Circuits and Systems (ISCAS), pp. 1–5

  43. Wei H et al (2018) A Rate Control Algorithm for HEVC Considering Visual Saliency. asia pacific signal and information processing association annual summit and conference, 36–42.

  44. Wei H, Zhou W, Zhou X, Bai R, Duan Z (2018) Saliency-based coding tree unit-level rate control for high-efficiency video coding. J Electron Imaging 27(4):043009

    Article  Google Scholar 

  45. Wiegand T, Schwarz H, Joch A, Kossentini F, Sullivan GJ (2003) Rate-constrained coder control and comparison of video coding standards. IEEE Trans Circuits Syst Video Techno 13(7):688–703

    Article  Google Scholar 

  46. Wong CW, Au OC, Meng B, Lam HK (2003) Perceptual rate control for low-delay video communications. Proc IEEE Int Multimedia Expo 3:III–361

    Google Scholar 

  47. Yang Z, Xu Q, Bao S, Cao X, Huang Q (2021) Learning with Multiclass AUC: Theory and Algorithms. IEEE Transs Pattern Anal Machine Intelligence PP:1

    Google Scholar 

  48. Ye Y, He X, Teng Q, Qing L, Lin H, Xia D (2018) Adaptive gradient information and BFGS based inter frame rate control for high efficiency video coding. Multimed Tools Appl 77(12):14557–14577

    Article  Google Scholar 

  49. Zeng H, Yang A, Ngan KN, Wang M (2016) Perceptual sensitivity-based rate control method for high efficiency video coding. Multimed Tools Appl 75(17):10383–10396

    Article  Google Scholar 

  50. Zhang W, Martin RR, Liu H (2017) A saliency dispersion measure for improving saliency-based image quality metrics. IEEE Trans Circuits Syst Video Technol 28(6):1462–1466

    Article  Google Scholar 

  51. Zhou M, Wei X, Wang S, Kwong S, Fong CK, Wong P, Gao W (2019) SSIM-based global optimization for CTU-level rate control in HEVC. IEEE Trans Multimedia 21(8):1921–1933

    Article  Google Scholar 

  52. Zhu C, Huang Y, Xie R, Song L (2021) HEVC VMAF-oriented Perceptual Rate Distortion Optimization using CNN. In: Proc. IEEE Int. Picture Coding Symposium (PCS), pp. 1–5.

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant No. 62041109, No. 61861038), the Fundamental Research Funds for the Central Universities (Grant No. 31920210073, 31920180115, 31920190039) and Gansu Province Natural Sciences Fund (21JR1RA206).

Author information

Authors and Affiliations

  1. College of Electrical Engineering, Northwest Minzu University, No.1 Xibeixincun, Lanzhou, 730030, China

    HongWei Lin, Xiangqun Li, Mingliang Gao, Keyan Deng & Yongsheng Xu

Authors
  1. HongWei Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiangqun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mingliang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Keyan Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yongsheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to HongWei Lin.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, H., Li, X., Gao, M. et al. Perceptual importance analysis-based rate control method for HEVC. Multimed Tools Appl 81, 12495–12518 (2022). https://doi.org/10.1007/s11042-022-12146-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12146-1

Keywords

Search

Navigation