Skip to main content
SpringerLink
Log in

Fast CU partition-based machine learning approach for reducing HEVC complexity

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

With the development of video coding technology, the high efficiency video coding (HEVC) provides better coding efficiency compared to its predecessors H.264/AVC. HEVC improves rate distortion (RD) performance significantly with increased encoding complexity. Due to the adoption of a large variety of coding unit (CU) sizes, at RD optimization level, the quadtree partition of the CU consumes a large proportion of the encoding complexity. Hence, the computational complexity cost remains a critical issue that must be properly considered in the optimization task. In this paper, two machine learning-based fast CU partition method for inter-mode HEVC are proposed, to optimize the complexity allocation at CU level. First, we propose an online support vector machine (SVM)-based fast CU algorithm for reducing HEVC complexity. The later was trained in an online way. Second, a deep convolutional neural network (CNN) is designed to predict the CU partition, in which large-scale training database including substantial CU partition data is considered. Experimental results demonstrate that the proposed online SVM can achieve a time saving of 52.28% with a degradation of 1.928% in the bitrate (BR). However, the proposed deep CNN can reduce the encoding time by 53.99% with 0.195% BR degradation. Compared to the state-of-the art, the two proposed approaches outperform the related works in terms of both RD performance and complexity reduction at inter-mode.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Sullivan, G.J., Ohm, J.R., Han, W.J., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012)

    Article  Google Scholar 

  2. Khemiri, R., Kibeya, H., Sayadi, F.E., Bahri, N., Atri, M., Masmoudi, N.: Optimization of HEVC motion estimation exploiting SAD and SSD GPU-based implementation. IET Image Proc. 12(2), 243–253 (2017)

    Article  Google Scholar 

  3. Cebrián-Márquez, G., Martinez, J.L., Cuenca, P.: Adaptive inter CU partitioning based on a look-ahead stage for HEVC. Signal Process. Image Commun. 76, 97–108 (2019)

    Article  Google Scholar 

  4. Wang, S., Luo, F., Ma, S., Zhang, X., Wang, S., Zhao, D., Gao, W.: Low complexity encoder optimization for HEVC. J. Vis. Commun. Image Represent. 35, 120–131 (2016)

    Article  Google Scholar 

  5. Xiong, J., Li, H., Wu, Q., Meng, F.: A fast HEVC inter CU selection method based on pyramid motion divergence. IEEE Trans. Multimed. 16(2), 559–564 (2014)

    Article  Google Scholar 

  6. Cho, S., Kim, M.: Fast CU splitting and pruning for suboptimal CU partitioning in HEVC intra coding. IEEE Trans. Circuits Syst. Video Technol. 23(9), 1555–1564 (2013)

    Article  Google Scholar 

  7. Shen, X., Yu, L., Chen, J.: Fast coding unit size selection for HEVC based on Bayesian decision rule. In: Proceedings of Picture Coding Symposium, pp. 453–456 (2012)

  8. Li, Y., Yang, G., Zhu, Y., Ding, X., Sun, X.: Adaptive inter CU depth decision for HEVC using optimal selection model and encoding parameters. IEEE Trans. Broadcast. 63(3), 535–546 (2017)

    Article  Google Scholar 

  9. Fernández, D.G., Del Barrio, A.A., Botella, G., Garcia, C.: Fast and effective CU size decision based on spatial and temporal homogeneity detection. Multimed. Tools Appl. 77(5), 5907–5927 (2018)

    Article  Google Scholar 

  10. Lee, J.H., Goswami, K., Kim, B.G., Jeong, S., Choi, J.S.: Fast encoding algorithm for high-efficiency video coding (HEVC) system based on spatio-temporal correlation. J. Real Time Image Process. 12(2), 407–418 (2016)

    Article  Google Scholar 

  11. Ahn, Y.J., Sim, D.: Square-type-first inter-CU tree search algorithm for acceleration of HEVC encoder. J. Real Time Image Process. 12(2), 419–432 (2016)

    Article  Google Scholar 

  12. Corrêa, G., Assuncao, P.A., Agostini, L.V., da Silva Cruz, L.A.: Fast HEVC encoding decisions using data mining. IEEE Trans. Circuits Syst. Video Technol. 25(4), 660–673 (2015)

    Article  Google Scholar 

  13. Grellert, M., Zatt, B., Bampi, S., da Silva Cruz, L.A.: Fast coding unit partition decision for HEVC using support vector machines. IEEE Trans. Circuits Syst. Video Technol. 29, 1741–1753 (2018)

    Article  Google Scholar 

  14. Zhang, Y., Kwong, S., Wang, X., Yuan, H., Pan, Z., Xu, L.: Machine learning-based coding unit depth decisions for flexible complexity allocation in high efficiency video coding. IEEE Trans. Image Process. 24(7), 2225–2238 (2015)

    Article  MathSciNet  Google Scholar 

  15. Zhu, L., Zhang, Y., Pan, Z., Wang, R., Kwong, S., Peng, Z.: Binary and multi-class learning based low complexity optimization for HEVC encoding. IEEE Trans. Broadcast. 63(3), 547–561 (2017)

    Article  Google Scholar 

  16. Zhu, L., Zhang, Y., Kwong, S., Wang, X., Zhao, T.: Fuzzy SVM-based coding unit decision in HEVC. IEEE Trans. Broadcast. 64(3), 681–694 (2017)

    Article  Google Scholar 

  17. Liu, Z., Yu, X., Gao, Y., Chen, S., Ji, X., Wang, D.: CU partition mode decision for HEVC hardwired intra encoder using convolution neural network. IEEE Trans. Image Process. 25(11), 5088–5103 (2016)

    Article  MathSciNet  Google Scholar 

  18. Hu, Q., Shi, Z., Zhang, X., Gao, Z.: Fast HEVC intra mode decision based on logistic regression classification. In: Proceedings of IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), pp. 1–4 (2016)

  19. Mallikarachchi, T., Talagala, D.S., Arachchi, H.K., Fernando, A.: Content-adaptive feature-based CU size prediction for fast low-delay video encoding in HEVC. IEEE Trans. Circuits Syst. Video Technol. 28(3), 693–705 (2018)

    Article  Google Scholar 

  20. Amer, H., Rashwan, A., Yang, E.H.: Fully connected network for HEVC CU split decision equipped with Laplacian transparent composite model. In: Picture Coding Symposium (PCS), pp. 189–193 (2018)

  21. Laude, T., Ostermann, J.: Deep learning-based intra prediction mode decision for HEVC. In: Proceedings of Picture Coding Symposium (PCS), pp. 1–5 (2016)

  22. Li, T., Xu, M., Deng, X.: A deep convolutional neural network approach for complexity reduction on intra-mode HEVC. In: Proceedings of IEEE International Conference on Multimedia and Expo (ICME), pp. 1255–1260 (2017)

  23. Wang, Y., Fan, X., Jia, C., Zhao, D., Gao, W.: Neural network based inter prediction for HEVC. In: IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2018)

  24. Khemiri, R., Bahri, N., Belghith, F., Bouaafia, S., Sayadi, F.E., Atri, M., Masmoudi, N.: Fast Motion Estimation’s Configuration Using Diamond Pattern and ECU, CFM, and ESD, Modes for Reducing HEVC Computational Complexity, pp. 1–17. IntechOpen, “Digital Imaging” Book, London (2019)

    Google Scholar 

  25. Khemiri, R., Kibeya, H., Loukil, H., Sayadi, F.E., Atri, M., Masmoudi, N.: Real-time motion estimation diamond search algorithm for the new high efficiency video coding on FPGA. Analog Integr. Circuits Signal Process. 94, 259–276 (2018)

    Article  Google Scholar 

  26. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  27. Huang, S., Cai, N., Pacheco, P.P., Narrandes, S., Wang, Y., Xu, W.: Applications of support vector machine (SVM) learning in cancer genomics. Cancer Genom. Proteom. 15(1), 41–51 (2018)

    Google Scholar 

  28. Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of IEEE International Workshop on Acoustic Signal Enhancement (IWAENC), pp. 315–323 (2011)

  29. Xu, M., Deng, X., Li, S., Wang, Z.: Region-of-interest based conversational HEVC coding with hierarchical perception model of face. IEEE J. Sel. Top. Signal Process. 8(3), 475–489 (2014)

    Article  Google Scholar 

  30. Ohm, J.R., Sullivan, G.J., Schwarz, H., Tan, T.K., Wiegand, T.: Comparison of the coding efficiency of video coding standards—including high efficiency video coding (HEVC). IEEE Trans. Circuits Syst. Video Technol. 22(12), 1669–1684 (2012)

    Article  Google Scholar 

  31. Xiph.org.: Xiph.org Video Test Media. [Online]. https://media.xiph.org/video/derf (2017). Accessed 15 June 2019

  32. Bossen, F.: Common test conditions and software reference configurations. Document JCTVC-L1100, Joint Collaborative Team on Video Coding, (2013)

Download references

Author information

Authors and Affiliations

  1. Electronics and Microelectronics Laboratory, Faculty of Sciences of Monastir, University of Monastir, Environment Street, 5019, Monastir, Tunisia

    Soulef Bouaafia, Randa Khemiri, Fatma Ezahra Sayadi & Mohamed Atri

  2. College of Computer Science, King Khalid University, Abha, Saudi Arabia

    Mohamed Atri

Authors
  1. Soulef Bouaafia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Randa Khemiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fatma Ezahra Sayadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohamed Atri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Soulef Bouaafia.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bouaafia, S., Khemiri, R., Sayadi, F.E. et al. Fast CU partition-based machine learning approach for reducing HEVC complexity. J Real-Time Image Proc 17, 185–196 (2020). https://doi.org/10.1007/s11554-019-00936-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-019-00936-0

Keywords

Search

Navigation