Skip to main content
SpringerLink
Log in

CNN-based intra partitioning process for spatial and SNR scalability for SHVC

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The partitioning process in the Scalable extension of the High Efficiency Video Coding (SHVC) provides an exhaustive computational time. However, the maximum time saving is not achieved and the speeding up of the computational process is still a burden. Several studies have discussed the complexity reduction related to the intra partitioning process used to minimize the time encoding. In this paper, a conventional Convolutional Neural Network (CNN) based partitioning process was proposed for SHVC with High Efficiency Video Coding (HEVC) as a base layer (BL) to predict the intra partitioning modes. The preprocessing stage includes the removal of the intensity values. Next, its features are extracted for the lowest level of block partitioning. The CNN-based partitioning process was applied on the upsampling process while encoding the Enhancement Layer (EL). The experimental findings indicate that the suggested approach effectively minimizes the complexity within the intra mode up to 64.89% with an acceptable loss in quality by 0.12db added to an increment in bite rate of about 2.19%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1

Similar content being viewed by others

Data Availability

’Data sharing not applicable to this article as no datasets were generated or analyzed during the current study’

References

  1. Sullivan GJ, Ohm J-R, Han W-J, Wiegand T (2012) Overview of the high efficiency video coding (hevc) standard. IEEE Trans Circuits Syst Video Technol 22(12):1649–1668

    Article  Google Scholar 

  2. Wiegand T, Sullivan GJ, Bjontegaard G, Luthra A (2003) Overview of the h. 264/avc video coding standard. IEEE Trans Circuits Syst Video Technol 13(7):560–576

    Article  Google Scholar 

  3. Boyce JM, Ye Y, Chen J, Ramasubramonian AK (2015) Overview of shvc: scalable extensions of the high efficiency video coding standard. IEEE Trans Circuits Syst Video Technol 26(1):20–34

    Article  Google Scholar 

  4. Wali I, Kessentini A, Ayed MAB, Masmoudi N (2017) Depth partitioning and coding mode selection statistical analysis for shvc. International journal of advanced computer science and applications (IJACSA), vol 8, no. 1,

  5. Xu M, Li T, Wang Z, Deng X, Yang R, Guan Z (2018) Reducing complexity of hevc: a deep learning approach. IEEE Trans Image Process 27(10):5044–5059

    Article  MathSciNet  Google Scholar 

  6. Li N, Zhang Y, Zhu L, Luo W, Kwong S (2019) Reinforcement learning based coding unit early termination algorithm for high efficiency video coding. J Vis Commun Image Represent 60:276–286

    Article  Google Scholar 

  7. Nair PS, Rao K, Nair MS (2019) A machine learning approach for fast mode decision in hevc intra prediction based on statistical features. J Intell Fuzzy Syst 36(3):2095–2106

    Article  Google Scholar 

  8. Lee J-K, Kim N, Cho S, Kang J-W (2020) Deep video prediction network-ased inter-frame coding in hevc. IEEE Access 8:95906–95917

    Article  Google Scholar 

  9. Tariq J, Armghan A, Ijaz A, Ashraf I (2020) Pure intra mode decision in hevc using optimized firefly algorithm. J Vis Commun Image Represent 68:102766

    Article  Google Scholar 

  10. Pan Z, Yi X, Zhang Y, Jeon B, Kwong S (2020) Efficient in-loop filtering based on enhanced deep convolutional neural networks for hevc. IEEE Trans Image Process 29:5352–5366

    Article  Google Scholar 

  11. Tariq J, Armghan A, Ijaz A, Ashraf I (2021) Light weight model for intra mode selection in hevc. Multim Tools Appl, 1–16

  12. Chen J, Boyce J, Ye Y, Hannuksela M (2013) Shvc draft text. Draft

  13. Chiang W-J, Chen J-J, Tsai Y-H (2017) A fast shvc coding scheme based on base layer co-located cu and cross-layer pu mode information. In: 2017 IEEE International conference on multimedia & Expo Workshops (ICMEW), IEEE, pp 381–386

  14. Shen L, Feng G (2019) Content-based adaptive shvc mode decision algorithm. IEEE Trans Multimedia 21(11):2714–2725

    Article  Google Scholar 

  15. Wang D, Sun Y, Zhu C, Li W, Dufaux F, Luo J (2020) Fast depth and mode decision in intra prediction for quality shvc. IEEE Trans Image Process 29:6136–6150

    Article  MathSciNet  Google Scholar 

  16. Wang D, Sun Y, Li W, Zhu C, Dufaux F (2019) Fast inter mode predictions for shvc. In: 2019 IEEE International conference on multimedia and Expo (ICME), IEEE, pp 1696–1701

  17. Lu X, Yu C, Gu Y, Martin G (2018) A fast intra coding algorithm for spatial scalability in shvc. In: 2018 25th IEEE International conference on image processing (ICIP), IEEE, pp 1792–1796

  18. Yeh C-H, Tseng W-Y, Kang L-W, Lee C-W, Muchtar K, Chen M-J (2018) Coding unit complexity-based predictions of coding unit depth and prediction unit mode for efficient hevc-to-shvc transcoding with quality scalability. J Vis Commun Image Represent 55:342–351

    Article  Google Scholar 

  19. Wang D, Zhu C, Sun Y, Dufaux F, Huang Y (2017) Efficient multi-strategy intra prediction for quality scalable high efficiency video coding. IEEE Trans Image Process 28(4):2063–2074

    Article  MathSciNet  Google Scholar 

  20. Li Y, Wang F (2020) An intra complexity reduction algorithm for quality scalable shvc. In: 2020 IEEE 3rd International conference on information communication and signal processing (ICICSP), IEEE, pp 300–305

  21. Fu G, Shen L, Yang H, Hu X, An P (2018) Fast intra coding of high dynamic range videos in shvc. IEEE Signal Process Lett 25(11):1665–1669

    Article  Google Scholar 

  22. Balaji L, Thyagharajan K (2018) An enhanced performance for h. 265/shvc based on combined aegbm3d filter and back-propagation neural network. SIViP 12(5):809–817

    Article  Google Scholar 

  23. Shen L, Feng G, An P (2019) Shvc cu processing aided by a feedforward neural network. IEEE Trans Ind Inform 15(11):5803–5815

    Article  Google Scholar 

  24. Dhanalakshmi A, Nagarajan G (2020) Convolutional neural network-based deblocking filter for shvc in h. 265. SIViP 14:1635–1645

    Article  Google Scholar 

  25. Lu Y, Huang X, Liu H, Yin H, Shen L (2021) Fast shvc inter-coding based on bayesian decision with coding depth estimation. J Real-Time Image Process, pp 1–17

  26. Sanagavarapu KS, Pullakandam M (2022) Object tracking based surgical incision region encoding using scalable high efficiency video coding for surgical telementoring applications. Radioengineering, 31(2)

  27. Wang D, Sun Y, Li W, Xie L, Lu X, Dufaux F, Zhu C (2023) A novel mode selection-based fast intra prediction algorithm for spatial shvc. In: ICASSP 2023 - 2023 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp 1–5

  28. Liu Y, Riviere M, Guionnet T, Roumy A, Guillemot C (2023) Cnn-based prediction of partition path for vvc fast inter partitioning using motion fields. arXiv preprint arXiv:2310.13838

  29. Galpin F, Racapé F, Jaiswal S, Bordes P, Le Léannec F, François E (2019) Cnn-based driving of block partitioning for intra slices encoding. In: 2019 Data Compression Conference (DCC), IEEE pp 162–171

  30. Chen Z, Relic L, Azevedo R, Zhang Y, Gross M, Xu D, Zhou L, Schroers C (2023) Neural video compression with spatio-temporal cross-covariance transformers. In: Proceedings of the 31st ACM International Conference on Multimedia, pp 8543–8551

  31. Hevc test model 12 (hm 16.5) (2015) JCT-VC of ITU-T SG16 WP 3and ISO/IEX JTC. https://vcgit.hhi.fraunhofer.de/jvet/HM//tags

  32. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580

  33. Chen J, Boyce J, Ye Y, Hannuksela M (2018) Scalable hevc (shvc) test model 12 (shm 12). JCT-VC of ITU-T SG16 WP 3and ISO/IEX JTC, vol 1

  34. Seregin V, He Y (2014) Common shm test conditions and software reference configurations. Document JCTVCQ1009, pp 1–4

  35. Bjøntegaard G (2001) Calculation of average psnr differences between rd-curves (vceg-m33). In: VCEG Meeting (ITU-T SG16 Q. 6), pp 2–4

Download references

Author information

Authors and Affiliations

  1. ENI Sfax - LETI, Road of Soukra, 3038, Sfax, Tunisia

    Ibtissem Wali & Nouri Masmoudi

  2. ENI Sfax - LETI and ISIM Gabes, Road of Soukra, 3038, Sfax, Tunisia

    Amina Kessentini

Authors
  1. Ibtissem Wali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amina Kessentini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nouri Masmoudi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ibtissem Wali.

Ethics declarations

Competing interests

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wali, I., Kessentini, A. & Masmoudi, N. CNN-based intra partitioning process for spatial and SNR scalability for SHVC. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-19179-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-19179-8

Keywords

Search

Navigation