Skip to main content
SpringerLink
Log in

An adaptive CU size decision algorithm based on gradient boosting machines for 3D-HEVC inter-coding

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

3D high-efficiency video coding (3D-HEVC) is an extension of the HEVC standard for coding of texture videos and depth maps. 3D-HEVC inherits the same quadtree coding structure as HEVC for both texture and depth components, in which the coding units (CUs) are recursively conducted on different sizes, namely, depth levels. However, the recursive splitting process of the CU causes extensive computational complexity. To reduce this computational burden, this paper presents an adaptive CU size decision algorithm for texture videos and depth maps. The proposed algorithm is divided into three steps. In the first step, the average local variance (ALV) is extracted from each CU size to define their homogeneity. Then, a classification-based gradient boosting machines (GBM) is employed to analyze and build a binary classification model from the extracted ALV features. The GBM model is employed to extract and efficiently get suitable thresholds for texture and depth map CUs. In the last step, a fast CU size decision algorithm is performed based on adaptive thresholds for texture videos and depth maps. The experimental results show that the proposed algorithm reduces a significant amount of encoding time, while the loss in coding efficiency is negligible.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Algorithm 2
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

The proposed algorithm has been performed using HTM-16.2, available online at: https://hevc.hhi.fraunhofer.de/trac/3d-hevc/browser/3DVCSoftware/tags/HTM-16.2. The coding experiments were defined under the common test conditions (CTC) [26]. All JCT-3V documents are available online at: http://phenix.int-evry.fr/jct3v/.

References

  1. Ahn Y, Sim D (2015) Square-type-first inter-CU tree search algorithm for acceleration of HEVC encoder. J Real-Time Image Proc 12:419–432. https://doi.org/10.1007/s11554-015-0487-5

    Article  Google Scholar 

  2. Bahad P, Saxena P (2019) Study of AdaBoost and Gradient Boosting Algorithms for Predictive Analytics. Int Conf Intell Comput Smart Commun 2019:235–244

    Google Scholar 

  3. Bakkouri S, Elyousfi A (2020a) Effective CU size decision algorithm based on depth map homogeneity for 3D-HEVC inter-coding. In: 2020 International Conference on Intelligent Systems and Computer Vision (ISCV), DOI https://doi.org/10.1109/iscv49265.2020.9204037

  4. Bakkouri S, Elyousfi A (2020b) FCM-Based Fast Texture CU Size Decision Algorithm for 3D-HEVC Inter-Coding. In: 2020 IEEE/ACS 17th International Conference on Computer Systems and Applications (AICCSA), DOI https://doi.org/10.1109/aiccsa50499.2020.9316455

  5. Bakkouri S, Elyousfi A (2021) Machine learning-based fast CU size decision algorithm for 3D-HEVC inter-coding

  6. Bakkouri S, Elyousfi A (2022) Early termination of CU partition based on boosting neural network for 3d-HEVC Inter-Coding. IEEE Access 10:13870–13883. https://doi.org/10.1109/access.2022.3147502

    Article  Google Scholar 

  7. Bakkouri S, Elyousfi A, Hamout H (2019) Fast CU size and mode decision algorithm for 3d-HEVC intercoding. Multimed Tools Appl 79:6987–7004. https://doi.org/10.1007/s11042-019-08461-9

    Article  Google Scholar 

  8. Binder H, Gefeller O, Schmid M, Mayr A (2014) Extending statistical boosting. Methods Inf Med 53:428–435. https://doi.org/10.3414/me13-01-0123

    Article  Google Scholar 

  9. Bjntegaard G (2001) Calculation of average PSNR differences between RD curves. In: 13th VCEG Meeting, Document VCEGM33, Austin

  10. Bjntegaard G (2008). In: 35th VCEG Meeting, Document VCEGAI11, Berlin

  11. Bocher P, McCloy K (2006) The fundamentals of average local variance - part i: detecting regular patterns. IEEE Trans Image Process 15:300–310. https://doi.org/10.1109/tip.2005.860623

    Article  Google Scholar 

  12. Bosc E, Pepion R, Le Callet P, et al. (2011) Towards a new quality metric for 3-D synthesized view assessment. IEEE J Sel Top Signal Process 5:1332–1343. https://doi.org/10.1109/jstsp.2011.2166245

    Article  Google Scholar 

  13. Chen J, Wang B, Liao J, Cai C (2018) Fast 3d-HEVC inter mode decision algorithm based on the texture correlation of viewpoints. Multimed Tools Appl 78:29291–29305. https://doi.org/10.1007/s11042-018-6832-5

    Article  Google Scholar 

  14. Chen M, Yang Y, Zhang Q, Zhao X, Huang X, Gan Y (2016) Low complexity depth mode decision for HEVC-based 3D video coding. Optik 127:4758–4767. https://doi.org/10.1016/j.ijleo.2016.01.204

    Article  Google Scholar 

  15. Friedman J (2001) Greedy function approximation: a gradient boosting machine

  16. Guelman L (2012) Gradient boosting trees for auto insurance loss cost modeling and prediction. Expert Syst Appl 39:3659–3667. https://doi.org/10.1016/j.eswa.2011.09.058

    Article  Google Scholar 

  17. Islam N, Shahid Z, Puech W (2016) Denoising and error correction in noisy AES-encrypted images using statistical measures. Signal Process Image Commun 41:15–27. https://doi.org/10.1016/j.image.2015.11.003

    Article  Google Scholar 

  18. Joint Collaborative Team on 3D video coding (JCT-3V) HTM 16.2 Reference Software (2016). Available online at https://hevc.hhi.fraunhofer.de/trac/3d-hevc/browser/3DVCSoftware/tags/HTM-16.2. Accessed 27 May 2016

  19. Lei J, Duan J, Wu F, Ling N, Hou C (2018) Fast mode decision based on grayscale similarity and Inter-View correlation for depth map coding in 3d-HEVC. IEEE Trans Circuits Syst Video Technol 28:706–718. https://doi.org/10.1109/tcsvt.2016.2617332

    Article  Google Scholar 

  20. Li Y, Yang G, Zhu Y, Ding X, Song Y, Zhang D (2019) Hybrid stopping model-based fast PU and CU decision for 3d-HEVC texture coding. J Real-Time Image Proc 17:1227–1238. https://doi.org/10.1007/s11554-019-00876-9

    Article  Google Scholar 

  21. Li Y, Yang G, Zhu Y, Ding X, Sun X (2017) Adaptive inter CU depth decision for HEVC using optimal selection model and encoding parameters. IEEE Trans Broadcast 63:535–546. https://doi.org/10.1109/tbc.2017.2704423

    Article  Google Scholar 

  22. Liao Y, Chen M, Yeh C, Lin J, Chen C (2018) Efficient inter-prediction depth coding algorithm based on depth map segmentation for 3d-HEVC. Multimed Tools Appl 78:10181–10205. https://doi.org/10.1007/s11042-018-6547-7

    Article  Google Scholar 

  23. Lin J, Chen M, Ciou Y, Yeh C, Lin M, Kau L, Chang C (2021) Fast Texture Coding Based on Spatial, Temporal and Inter-View Correlations for 3D Video Coding. IEEE Access 9:100081–100095. https://doi.org/10.1109/access.2021.3093950

    Article  Google Scholar 

  24. Müller K, Merkle P, Wiegand T (2011) 3-D Video Representation Using Depth Maps. Proc IEEE 99:643–656. https://doi.org/10.1109/jproc.2010.2091090

    Article  Google Scholar 

  25. Muller K, Schwarz H, Marpe D, Bartnik C, Bosse S, Brust H, Hinz T, Lakshman H, Merkle P, Rhee F, Tech G, Winken M, Wiegand T (2013) 3D High-Efficiency Video Coding for Multi-View Video and Depth Data. IEEE Trans Image Process 22:3366–3378. https://doi.org/10.1109/tip.2013.2264820

    Article  MathSciNet  MATH  Google Scholar 

  26. Muller K, Vetro A (2014) Common test conditions of 3DV core experiments. In: ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JCT3v, vol G1100. pp 1-7

  27. Safavian S, Landgrebe D (1991) A survey of decision tree classifier methodology. IEEE Trans Syst Man Cybern 21:660–674. https://doi.org/10.1109/21.97458

    Article  MathSciNet  Google Scholar 

  28. Saldanha M, Sanchez G, Marcon C, Agostini L (2020) Fast 3d-HEVC depth map encoding using machine learning. IEEE Trans Circuits Syst Video Technol 30:850–861. https://doi.org/10.1109/tcsvt.2019.2898122

    Article  Google Scholar 

  29. Sapountzoglou N, Lago J, Raison B (2020) Fault diagnosis in low voltage smart distribution grids using gradient boosting trees. Electr Power Syst Res 106254:182. https://doi.org/10.1016/j.epsr.2020.106254

    Article  Google Scholar 

  30. Schapire R (1990) The strength of weak learnability. Mach Learn 5:197–227. https://doi.org/10.1007/bf00116037

    Article  Google Scholar 

  31. Si M, Du K (2020) Development of a predictive emissions model using a gradient boosting machine learning method. Environ Technol Innov 101028:20. https://doi.org/10.1016/j.eti.2020.101028

    Article  Google Scholar 

  32. Smolic A, Muller K, Dix K et al (2008) Intermediate view interpolation based on multiview video plus depth for advanced 3D video systems. In: 2008 15th IEEE International Conference on Image Processing

  33. Tai K, Hsieh M, Chen M, Chen C, Yeh C (2017) A Fast HEVC Encoding Method Using Depth Information of Collocated CUs and RD Cost Characteristics of PU Modes. IEEE Trans Broadcast 63:680–692. https://doi.org/10.1109/tbc.2017.2722239

    Article  Google Scholar 

  34. Tanimoto M, Fujii T, Suzuki K (2008) View synthesis algorithm in view synthesis reference software 2.0 (VSRS2.0). Tech. Rep ISO/IEC JTC1/SC29/WG11 M16090, Lausanne, Switzerland

  35. Tech G, Chen Y, Muller K, Ohm J, Vetro A, Wang Y (2016) Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Trans Circuits Syst Video Technol 26:35–49. https://doi.org/10.1109/tcsvt.2015.2477935

    Article  Google Scholar 

  36. Woodcock C, Strahler A (1987) The factor of scale in remote sensing. Remote Sens Environ 21:311–332. https://doi.org/10.1016/0034-4257(87)90015-0

    Article  Google Scholar 

  37. Zhang X, Quadrianto N, Kersting K, Xu Z, Engel Y, Sammut C, Reid M, Liu B, Webb G, Sammut C, Sipper M, Saitta L, Sebag M, Aggarwal C, Gärtner T, Horváth T, Wrobel S, Chakrabarti D, McAuley J, Caetano T, Buntine W, Jensen T, Sammut C, Holder L, Sharara H, Getoor L (2011) Genetic and evolutionary algorithms. Encycl Mach Learn, 456–457

  38. Zhang Q, Zhang N, Wei T, Huang K, Qian X, Gan Y (2017) Fast depth map mode decision based on depth–texture correlation and edge classification for 3d-HEVC. J Vis Commun Image Represent 45:170–180. https://doi.org/10.1016/j.jvcir.2017.03.004

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Higher School of Technology, Sultan Moulay Slimane University, 23000, Beni Mellal, Morocco

    Siham Bakkouri

  2. Department of Computer Science, National Engineering School of Applied Sciences, IbnZohr University and Computer Systems and Vision Laboratory, Faculty of Sciences, Ibn-Zohr University, Agadir, 80000, Morocco

    Abderrahmane Elyousfi

Authors
  1. Siham Bakkouri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abderrahmane Elyousfi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Siham Bakkouri.

Ethics declarations

Conflict of Interests

No conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bakkouri, S., Elyousfi, A. An adaptive CU size decision algorithm based on gradient boosting machines for 3D-HEVC inter-coding. Multimed Tools Appl 82, 32539–32557 (2023). https://doi.org/10.1007/s11042-023-14540-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14540-9

Keywords

Search

Navigation