Skip to main content
SpringerLink
Log in

CNN-based ternary tree partition approach for VVC intra-QTMT coding

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In July 2020, the Joint Video Experts Team has published the versatile video coding (VVC) standard. The VVC encoder enhances the coding efficiency compared with his predecessor high-efficiency video coding encoder, thanks to the improved coding modules and the new proposed techniques such as the new block partitioning structure called quadtree with nested multi-type tree (QTMT). However, QTMT induces a significant increase in encoding time mainly at the rate distortion optimization level (RDO) which causes an enormous computational complexity. Instead of RDO-QTMT partition process, a deep-QTMT partition approach based on a fast convolution neural network-ternary tree (CNN-TT) is proposed to predict the best intra-QTMT decision tree in order to reduce the encoding time. A database is initially established containing CU-based TT partition depths with several video contents. Then, a CNN-TT model is developed under three-levels provided by the TT structure to early determine the QTMT partition at 32\(\times \)32. Different threshold values are fixed for each level according to the CNN-TT predicted probabilities to reach a balance between the encoding complexity and the coding efficiency. The experimental results prove that our deep-QTMT partition approach saves a significant encoder time on average between 23% and 58% with an acceptable RD performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

Please contact the author for data requests.

References

  1. Javaid, S., Rizvi, S., Ubaid, M.T., et al.: Vvc/h. 266 intra mode qtmt based cu partition using cnn. IEEE Access 10, 37246–37256 (2022)

    Article  Google Scholar 

  2. Tech, G., Pfaff, J., Schwarz, H., et al.: “Fast partitioning for vvc intra-picture encoding with a cnn minimizing the rate-distortion-time cost,” in Data Compression Conference (DCC), 3–12, IEEE (2021)

  3. Li, T., Xu, M., Tang, R., et al.: Deepqtmt: a deep learning approach for fast qtmt-based cu partition of intra-mode vvc. IEEE Trans. Image Process. 30, 5377–5390 (2021)

    Article  Google Scholar 

  4. Xu, J., Wu, G., Zhu, C., et al.: Cnn-based fast cu partitioning algorithm for vvc intra coding, in 2022 IEEE ICIP, 2706–2710, IEEE (2022)

  5. Abdallah, B., Belghith, F., Ayed, M. A. B., et al.: Low-complexity qtmt partition based on deep neural network for versatile video coding,” Signal, Image and Video Processing , 1–8 (2021)

  6. Abdallah, B., Belghith, F., Ben Ayed, M.A., et al.: Fast qtmt decision tree for versatile video coding based on deep neural network, Multimedia Tools Appl. , 1–17 (2022)

  7. Park, S.-H., Kang, J.-W.: Fast multi-type tree partitioning for versatile video coding using a lightweight neural network. IEEE Trans. Multimedia 23, 4388–4399 (2020)

    Article  Google Scholar 

  8. Amestoy, T., Mercat, A., Hamidouche, W., et al.: Tunable vvc frame partitioning based on lightweight machine learning. IEEE Trans. Image Process. 29, 1313–1328 (2019)

    Article  MathSciNet  Google Scholar 

  9. Fu, T., Zhang, H., Mu, F., et al.: Fast cu partitioning algorithm for h.266/vvc intra-frame coding, in 2019 IEEE International Conference on Multimedia and Expo (ICME), 55–60 (2019)

  10. Saldanha, M., Sanchez, G., Marcon, C., et al.: Configurable fast block partitioning for vvc intra coding using light gradient boosting machine. IEEE Trans. Circuits Syst. Video Technol. 32(6), 3947–3960 (2021)

    Article  Google Scholar 

  11. Shu, Z., Li, J., Peng, Z., et al.: Intra complexity control algorithm for vvc. Electronics 11(16), 2572 (2022)

  12. Wu, G., Huang, Y., Zhu, C., et al.: Svm based fast cu partitioning algorithm for vvc intra coding, in 2021 IEEE (ISCAS), 1–5 (2021)

  13. Zhang, Q., Wang, Y., Huang, L., et al.: Fast cu partition and intra mode decision method for h. 266/vvc. IEEE Access 8, 117539–117550 (2020)

    Article  Google Scholar 

  14. Zhao, J., Wu, A., Zhang, Q.: Svm-based fast cu partition decision algorithm for vvc intra coding. Electronics 11(14), 2147 (2022)

    Article  Google Scholar 

  15. Abdallah, B., Belghith, F., Ayed, M. A. B., et al.: Qtmt partitioning structure in vvc: overview and analysis,” IEEE international Ccnference on Sciences and Techniques of Automatic Control and Computer Engineering (STA) , 331–335 (2022)

  16. Bossen, F., Boyce, J., Li, X., et al.: “Jvet common test conditions and software reference configurations for sdr video,” (JVET) of ITU-T SG16 (2018)

  17. Bjøntegaard, G.: Calculation of average psnr differences between rd-curves (vceg-m33), in VCEG Meeting (ITU-T SG16 Q. 6), 2–4 (2001)

  18. Park, S.-H., Kang, J.-W.: Context-based ternary tree decision method in versatile video coding for fast intra coding. IEEE Access 7, 172597–172605 (2019)

    Article  Google Scholar 

  19. Zhang, Y., Kwong, S., Wang, X., et al.: Machine learning-based coding unit depth decisions for flexible complexity allocation in hevc. IEEE Trans. Image Process. 24(7), 2225–2238 (2015)

    Article  MathSciNet  Google Scholar 

  20. Liu, Z., Yu, X., Gao, Y., et al.: Cu partition mode decision for hevc hardwired intra encoder using convolution neural network. IEEE Trans. Image Process. 25(11), 5088–5103 (2016)

    Article  MathSciNet  Google Scholar 

Download references

Funding

No funding was received for conducting this study.

Author information

Authors and Affiliations

  1. LETI, ENIS, University of Sfax, Sfax, Tunisia

    Fatma Belghith, Bouthaina Abdallah, Sonda Ben Jdidia & Nouri Masmoudi

  2. NTS’COM, ENETCOM, University of Sfax, Sfax, Tunisia

    Mohamed Ali Ben Ayed

Authors
  1. Fatma Belghith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bouthaina Abdallah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sonda Ben Jdidia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohamed Ali Ben Ayed
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nouri Masmoudi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have equally contributed.

Corresponding author

Correspondence to Fatma Belghith.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Belghith, F., Abdallah, B., Ben Jdidia, S. et al. CNN-based ternary tree partition approach for VVC intra-QTMT coding. SIViP 18, 3587–3594 (2024). https://doi.org/10.1007/s11760-024-03023-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-024-03023-5

Keywords

Search

Navigation