Skip to main content
SpringerLink
Log in

Line-based self-referencing string prediction technique for screen content coding in AVS3

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

String Prediction (SP) is a very efficient screen content coding (SCC) tool. In SP, the self-referencing string plays an important role to improve coding efficiency. But general self-referencing string has the problem of very low pixel copying throughput and is prohibited in the non-self-referencing based SP which has been adopted in the third-generation Audio Video Standard (AVS3). To overcome the problem and bring back the coding gain of self-referencing string, a line-based self-referencing string (LSRS) enabled SP technique is proposed. Moreover, to keep the pixel copying throughput and coding complexity of LSRS enabled SP the same as non-self-referencing based SP, an unbroken-line decomposition algorithm is presented to decompose an LSRS into multiple non-self-referencing strings. In this way, LSRS can be treated in the same way as a non-self-referencing string with the best trade-off between coding efficiency and complexity. Compared with non-self-referencing based SP, using AVS3 reference software HPM, for twelve SCC common test condition YUV test sequences in text and graphics with motion category and mixed content category, the proposed LSRS technique achieves the average Y BD-rate reduction of 0.81% and 0.59% as well as the maximum Y BD-rate reduction of 2.04% and 1.31% for All Intra and Low Delay configurations, respectively, with almost no additional encoding and decoding complexity. The proposed LSRS enabled SP technique has been adopted in AVS3.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from ftp://47.93.196.121/Public/Seqs/Video/ but restrictions apply to the availability of these data, which were used under licence for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of the Audio Video Standard Workgroup of China (http://www.avs.org.cn/english/).

References

  1. Bjøntegaard G (2001) Calculation of average PSNR differences between RD-Curves, ITU-T SG16 Q.6 Document, VCEG-M33, Austin, USA

  2. Chen C, Peng W (2017) Intra Line Copy for HEVC Screen Content Intra-Picture Prediction. IEEE Trans Circuits Syst Video Technol 27(7):1568–1579

    Article  Google Scholar 

  3. Feng S, Hu K, Fan E, Zhao L, Wu C (2021) Kalman filter for spatial-temporal regularized correlation filters. IEEE Trans Image Process 30:3263–3278

    Article  MathSciNet  Google Scholar 

  4. Hu K, Ye J, Fan E, Shen S, Huang L, Pi J (2017) A novel object tracking algorithm by fusing color and depth information based on single valued neutrosophic cross-entropy. J Intell Fuzzy Syst 32(3):1775–1786

    Article  Google Scholar 

  5. Li B, Xu J, Wu F (2014). 1-D dictionary mode for screen content coding. In: IEEE Visual Communications and Image Processing Conference, 189–192.

  6. Lin T, Zhang P, Wang S, Zhou K, Chen X (2013) Mixed Chroma Sampling-Rate High Efficiency Video Coding for Full-Chroma Screen Content. IEEE Trans Circuits Syst Video Technol 23(1):173–185

    Article  Google Scholar 

  7. Lin W, He X, Dai W, See J, Shinde T, Xiong H, Duan L (2020) Key-point sequence lossless compression for intelligent video analysis. IEEE MultiMedia 27(3):12–22

    Article  Google Scholar 

  8. Lin W, Shinde T, Dai W, Liu M, He X, Tiwari A, Xiong H (2020) Adaptive lossless compression of skeleton sequences. Signal Process Image Commun 80:115679 1–14

    Article  Google Scholar 

  9. Liu Y, Fang C, Sun J, Huang X (2019) Fast Palette Mode Decision Methods for Coding Game Videos With HEVC-SCC. IEEE Trans Circuits Syst Video Technol 29(10):3061–3067

    Article  Google Scholar 

  10. Ma S, Zhang X, Jia C, Zhao Z, Wang S, Wang S (2020) Image and Video Compression with Neural Networks: A Review. IEEE Trans Circuits Syst Video Technol 30(6):1683–1698

    Article  Google Scholar 

  11. Ma Z, Wang W, Xu M et al Advanced Screen Content Coding Using Color Table and Index Map. IEEE Trans Image Process 23(10):4399–4412

  12. Nguyen T, Xu X, Henry F et al (2021) Overview of the screen content support in VVC: applications, coding tools, and performance. IEEE Trans Circuits Syst Video Technol 31:3801–3817. https://doi.org/10.1109/TCSVT.2021.3074312

    Article  Google Scholar 

  13. Peng W, Walls F, Cohen R et al (2016) Overview of screen content video coding: technologies, standards, and beyond. IEEE J Emerg Sel Top Circuits Syst 6(4):393–408

    Article  Google Scholar 

  14. Wang S, Lin T (2014) United coding method for compound image compression. Multimed Tools Appl 71(3):1263–1282

    Article  Google Scholar 

  15. Wang Y, Zhou Q, Zhao L et al (2020) SCC: Unit Basis Vector String for String Prediction, AVS M5994, China

  16. Wang Y, Xu X, Liu S (2021) Low complexity implementation of intra string copy in AVS3. In: IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 1-4

  17. Xiao W, Shi G, Li B, Xu J, Wu F (2018) Fast Hash-Based Inter-Block Matching for Screen Content Coding. IEEE Trans Circuits Syst Video Technol 28(5):1169–1182

    Article  Google Scholar 

  18. Xiao W, Bin L, Xu J, Shi G, Wu F (2018) Weighted Rate-Distortion Optimization for Screen Content Coding. IEEE Trans Circuits Syst Video Technol 28(2):499–512

    Article  Google Scholar 

  19. Xu X (2020) AVS3-P2 Common Test Conditions for screen content coding, AVS Document, AVS M5928, China

  20. Xu X, Liu S Overview of screen content coding in recently developed video coding standards. IEEE trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2021.3064210

  21. Xu X, Liu S, Chuang T et al (2016) Intra block copy in HEVC screen content coding extensions. IEEE J Emerg Sel Top Circuits Syst 6(4):409–418

    Article  Google Scholar 

  22. Yang Y, Zhou K, Zhao L, Lin T (2021) An ultralow complexity string matching approach to screen content coding in AVS3[J]. IEEE Trans Circuits Syst Video Technol 31(9):3714–3718

    Article  Google Scholar 

  23. Yang Y, Lin T, Zhao L, Zhou K, Wang S (2021) A string matching based ultra-low complexity lossless screen content coding technique. Multimed Tools Appl 81:2043–2063. https://doi.org/10.1007/s11042-021-11418-6

    Article  Google Scholar 

  24. Zhao L, Lin T, Zhou K, Wang S, Chen X (2016) Pseudo 2D String Matching Technique for High Efficiency Screen Content Coding. IEEE Trans Multimedia 18(3):339–350

    Article  Google Scholar 

  25. Zhao L, Lin T, Zhou K (2017) An efficient ISC offset parameter coding algorithm in screen content coding. Chin J Comput 40(5):1–11

    Google Scholar 

  26. Zhao L, Zhou K, Guo J, Wang S, Lin T (2018) A Universal String Matching Approach to Screen Content Coding. IEEE Trans Multimedia 20(4):796–809

    Article  Google Scholar 

  27. Zhao L, Zhou K, Lin T (2018) Pixel string matching for full-chroma screen and mixed content coding in AVS2. Chin J Comput 41(11):2482–2495

    Google Scholar 

  28. Zhao L, Zhou K, Lin T et al (2019) A universal string prediction approach and its application in AVS2 mixed content coding. Chin J Comput 42(41):1–15

    Google Scholar 

  29. Zhao L, Lin T, Guo J et al (2019) Universal string prediction-based inter coding algorithm optimization in AVS2 mixed content coding. Chin J Comput 42(28):1–13

    Google Scholar 

  30. Zhao L, Lin T, Zhang D, Zhou K, Wang S (2020) An Ultra-Low Complexity and High Efficiency Approach for Lossless Alpha Channel Coding. IEEE Trans Multimedia 22(3):786–794

    Article  Google Scholar 

  31. Zhou K, Zhao L, Lin T (2016) A hardware decoder architecture for general string matching technique. IEEE J Emerg Sel Top Circuits Syst 6(4):560–572

    Article  Google Scholar 

  32. Zhou K, Zhao L, Lin T (2018) A flexible and uniform string matching technique for general screen content coding. Multimed Tools Appl 77:23751–23775

    Article  Google Scholar 

  33. Zhou Q, Zhao L, Wang H et al (2020) CE_SCC_related: Encoding and Decoding Optimization for String Vector based on the Inherent Characteristics of Several Above Strings. AVS M5950, China

  34. Zhou Q, Zhao L, Zhou K, Lin T, Wang H, Wang S, Jiao M (2021) String prediction for 4:2:0 format screen content coding and its implementation in AVS3. IEEE Trans Multimedia 23:3867–3876

    Article  Google Scholar 

  35. Zou F, Chen Y, Karczewicz M, Seregin V (2015) Hash based intra string copy for HEVC based screen content coding. In: ICMEW, pp. 1–4.

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (No. 62271321, No. 61871289), the Science and Technology Plan Project in Basic Public Welfare class of Shaoxing city (No. 2022A11002), the Social Sciences and Humanities Youth Foundation of Ministry of Education (No. 21YJCZH039), Zhejiang Provincial Postdoctoral Science Foundation No. ZJ2022066, Zhejiang Provincial Natural Science Foundation of China (No. LTY22F020003), the Ministry of Education Industry University Cooperation Collaborative Education Project (No. 202101011025), and the Humanities and Social Sciences Project of Shaoxing University (No. 2021LJ001).

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Shaoxing University, Shaoxing, 312000, China

    Liping Zhao, Keli Hu, Sheng Feng & Weixing Wang

  2. Information Technology R&D Innovation Center of Peking University, Shaoxing, 312000, China

    Liping Zhao

  3. College of Electronics and Information Engineering, VLSI Lab, Tongji University, Shanghai, 200092, China

    Qingyang Zhou, Kailun Zhou & Tao Lin

Authors
  1. Liping Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingyang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Keli Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sheng Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kailun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Weixing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tao Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Lin.

Ethics declarations

Conflict of interest

The authors declare that there are no conflicts of interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, L., Zhou, Q., Hu, K. et al. Line-based self-referencing string prediction technique for screen content coding in AVS3. Multimed Tools Appl 82, 23693–23708 (2023). https://doi.org/10.1007/s11042-023-14673-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14673-x

Keywords

Search

Navigation