Skip to main content
SpringerLink
Log in

Fine-grain complexity control of HEVC intra prediction in battery-powered video codecs

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The high-efficiency video coding (HEVC) standard improves the coding efficiency at the cost of a significantly more complex encoding process. This is an issue for a large number of video-capable devices that operate on batteries, with limited and varying processing power. A complexity controller enables an encoder to provide the best possible quality at any power quota. This paper proposes a complexity control method for HEVC intra coding, based on a Pareto-efficient rate–distortion–complexity (R–D–C) analysis. The proposed method limits the intra prediction for each block (as opposed to existing methods which limit the block partitioning), on a frame-level basis. This method consists of three steps, namely rate-complexity modeling, complexity allocation, and configuration selection. In the first step, a rate-complexity model is presented which estimates the encoding complexity according to the compression intensity. Then, according to the estimated complexity and target complexity, a complexity budget is allocated to each frame. Finally, an encoding configuration from a set of Pareto-efficient configurations is selected according to the allocated complexity and the video content, which offers the best compression performance. Experimental results indicate that the proposed method can adjust the complexity from 100 to 50%, with a mean error rate of less than 0.1%. The proposed method outperforms many state-of-the-art approaches, in terms of both control accuracy and compression efficiency. The encoding performance loss in terms of BD-rate varies from 0.06 to 3.69%, on average, for 90–60% computational complexity, respectively. The method can also be used for lower than 50% complexity if need be, with a higher BD-rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Sullivan, G.J., Ohm, J.-R., Han, W.-J., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22, 1649–1668 (2012). https://doi.org/10.1109/TCSVT.2012.2221191

    Article  Google Scholar 

  2. Correa, G., Assuncao, P., Agostini, L., Da Silva Cruz, L.A.: Performance and computational complexity assessment of high-efficiency video encoders. IEEE Trans. Circuits Syst. Video Technol. 22, 1899–1909 (2012). https://doi.org/10.1109/TCSVT.2012.2223411

    Article  Google Scholar 

  3. Pakdaman, F., Hashemi, M.-R., Ghanbari, M.: Fast and efficient intra mode decision for HEVC, based on dual-tree complex wavelet. Multimed. Tools Appl. 76, 9891–9906 (2017). https://doi.org/10.1007/s11042-016-3584-y

    Article  Google Scholar 

  4. Jamali, M., Coulombe, S.: Fast HEVC intra mode decision based on RDO cost prediction. IEEE Trans. Broadcast. (2018). https://doi.org/10.1109/TBC.2018.2847464

    Article  Google Scholar 

  5. Liu, X., Liu, Y., Wang, P., Lai, C.F., Chao, H.C.: An adaptive mode decision algorithm based on video texture characteristics for HEVC intra prediction. IEEE Trans. Circuits Syst. Video Technol. 27, 1737–1748 (2017). https://doi.org/10.1109/TCSVT.2016.2556278

    Article  Google Scholar 

  6. Hosseini, E., Pakdaman, F., Hashemi, M.R., Ghanbari, M.: A computationally scalable fast intra coding scheme for HEVC video encoder. Multimed. Tools Appl. 76, 11607–11630 (2019). https://doi.org/10.1007/s11042-018-6713-y

    Article  Google Scholar 

  7. Liu, X., Li, Y., Liu, D., Wang, P., Yang, L.T.: An adaptive CU size decision algorithm for HEVC intra prediction based on complexity classification using machine learning. IEEE Trans. Circuits Syst. Video Technol. 29, 144–155 (2019). https://doi.org/10.1109/TCSVT.2017.2777903

    Article  Google Scholar 

  8. Zhang, J., Kwong, S.T.W., Zhao, T., Ip, H.H.S.: Complexity control in the HEVC intracoding for industrial video applications. IEEE Trans. Ind. Inform. 15, 1437–1449 (2019). https://doi.org/10.1109/TII.2018.2844214

    Article  Google Scholar 

  9. Jimenez-Moreno, A., Martinez-Enriquez, E., Diaz-de-Maria, F.: Complexity control based on a fast coding unit decision method in the HEVC video coding standard. IEEE Trans. Multimed. 18, 563–575 (2016). https://doi.org/10.1109/TMM.2016.2524995

    Article  Google Scholar 

  10. Deng, X., Xu, M., Jiang, L., Sun, X., Wang, Z.: Subjective-driven complexity control approach for HEVC. IEEE Trans. Circuits Syst. Video Technol. 26, 91–106 (2016). https://doi.org/10.1109/TCSVT.2015.2474075

    Article  Google Scholar 

  11. Zhang, J., Kwong, S., Zhao, T., Pan, Z.: CTU-level complexity control for high efficiency video coding. IEEE Trans. Multimed. 20, 29–44 (2018). https://doi.org/10.1109/TMM.2017.2723238

    Article  Google Scholar 

  12. Apple ProRes. https://apple.com/final-cut-pro/docs/Apple_ProRes_White_Paper.pdf

  13. Lei, J., Li, D., Pan, Z., Sun, Z., Kwong, S., Hou, C.: Fast intra prediction based on content property analysis for low complexity HEVC-based screen content coding. IEEE Trans. Broadcast. 63, 48–58 (2017). https://doi.org/10.1109/TBC.2016.2623241

    Article  Google Scholar 

  14. Xu, L., Kwong, S., Zhang, Y., Zhao, D.: Low-complexity encoder framework for window-level rate control optimization. IEEE Trans. Ind. Electron. 60, 1850–1858 (2013). https://doi.org/10.1109/TIE.2012.2190960

    Article  Google Scholar 

  15. Sjovall, P., Oinonen, A., Teuho, M., Vanne, J., Hamalainen, T.D.: Dynamic resource allocation for HEVC encoding in FPGA-accelerated SDN cloud. In: 2019 IEEE Nordic Circuits and Systems Conference (NORCAS): NORCHIP and International Symposium of System-on-Chip (SoC), pp. 1–5. IEEE (2019)

  16. Rodriguez-Sanchez, R., Alonso, M.T., Martinez, J.L., Mayo, R., Quintana-Orti, E.S.: Time and energy modeling of an intra only HEVC encoder. In: Visual Communications and Image Processing, pp. 1–4. IEEE (2015)

  17. Chen, J., Ye, Y., Kim, S.H.: Algorithm description for versatile video coding and test model 6 (VTM 6). Jt. Video Expert. Team ITU-T SG 16 WP 3 ISO/IEC JTC 1/SC 29/WG 11, 15th Meet. Gothenburg, SE (2019)

  18. Shen, L., Zhang, Z., An, P.: Fast CU size decision and mode decision algorithm for HEVC intra coding. IEEE Trans. Consum. Electron. 59, 207–213 (2013). https://doi.org/10.1109/TCE.2013.6490261

    Article  Google Scholar 

  19. Marzuki, I., Ma, J., Ahn, Y.-J., Sim, D.: A context-adaptive fast intra coding algorithm of high-efficiency video coding (HEVC). J. Real Time Image Process. 16, 883–899 (2019). https://doi.org/10.1007/s11554-016-0571-5

    Article  Google Scholar 

  20. Zhang, Y., Pan, Z., Li, N., Wang, X., Jiang, G., Kwong, S.: Effective data driven coding unit size decision approaches for HEVC INTRA coding. IEEE Trans. Circuits Syst. Video Technol. 28, 3208–3222 (2018). https://doi.org/10.1109/TCSVT.2017.2747659

    Article  Google Scholar 

  21. Xu, M., Li, T., Wang, Z., Deng, X., Yang, R., Guan, Z.: Reducing complexity of HEVC: a deep learning approach. IEEE Trans. Image Process. 27, 5044–5059 (2018). https://doi.org/10.1109/TIP.2018.2847035

    Article  MathSciNet  Google Scholar 

  22. Ozcan, E., Kalali, E., Adibelli, Y., Hamzaoglu, I.: A computation and energy reduction technique for HEVC intra mode decision. IEEE Trans. Consum. Electron. 60, 745–753 (2014). https://doi.org/10.1109/TCE.2014.7027351

    Article  Google Scholar 

  23. Azgin, H., Kalali, E., Hamzaoglu, I.: A computation and energy reduction technique for HEVC intra prediction. IEEE Trans. Consum. Electron. 63, 36–43 (2017). https://doi.org/10.1109/TCE.2017.014728

    Article  Google Scholar 

  24. Azgin, H., Mert, A.C., Kalali, E., Hamzaoglu, I.: Reconfigurable intra prediction hardware for future video coding. IEEE Trans. Consum. Electron. 63, 419–425 (2017). https://doi.org/10.1109/TCE.2017.015118

    Article  Google Scholar 

  25. Zhang, Y., Lu, C.: High-performance algorithm adaptations and hardware architecture for HEVC intra encoders. IEEE Trans. Circuits Syst. Video Technol. (2019). https://doi.org/10.1109/TCSVT.2019.2913504

    Article  Google Scholar 

  26. Kalali, E., Adibelli, Y., Hamzaoglu, I.: A low energy intra prediction hardware for high efficiency video coding. J. Real Time Image Process. 15, 221–234 (2018). https://doi.org/10.1007/s11554-014-0471-5

    Article  Google Scholar 

  27. Correa, G., Assuncao, P., Agostini, L., da Silva Cruz, L.: Complexity control of high efficiency video encoders for power-constrained devices. IEEE Trans. Consum. Electron. 57, 1866–1874 (2011). https://doi.org/10.1109/TCE.2011.6131165

    Article  Google Scholar 

  28. Correa, G., Assuncao, P., Agostini, L., da Silva Cruz, L.A.: Complexity scalability for real-time HEVC encoders. J. Real Time Image Process. 12, 107–122 (2016). https://doi.org/10.1007/s11554-013-0392-8

    Article  Google Scholar 

  29. Correa, G., Assuncao, P., Da Silva Cruz, L.A., Agostini, L.: Encoding time control system for HEVC based on rate-distortion-complexity analysis. In: Proc. IEEE Int. Symp. Circuits Syst. 2015-July, pp. 1114–1117 (2015). https://doi.org/10.1109/ISCAS.2015.7168833

  30. Correa, G., Assuncao, P.A., Agostini, L.V., Da Silva Cruz, L.A.: Pareto-based method for high efficiency video coding with limited encoding time. IEEE Trans. Circuits Syst. Video Technol. 26, 1734–1745 (2016). https://doi.org/10.1109/TCSVT.2015.2469533

    Article  Google Scholar 

  31. Kim, J., Kim, J., Kim, G., Kyung, C.-M.: Power–rate–distortion modeling for energy minimization of portable video encoding devices. In: 2011 IEEE 54th International Midwest Symposium on Circuits and Systems (MWSCAS), pp. 1–4. IEEE (2011)

  32. Grellert, M., Zatt, B., Shafique, M., Bampi, S., Henkel, J.: Complexity control of HEVC encoders targeting real-time constraints. J. Real Time Image Process. 13, 5–24 (2017). https://doi.org/10.1007/s11554-016-0602-2

    Article  Google Scholar 

  33. Deng, X., Xu, M., Li, C.: Hierarchical complexity control of HEVC for live video encoding. IEEE Access. 4, 7014–7027 (2016). https://doi.org/10.1109/ACCESS.2016.2612691

    Article  Google Scholar 

  34. Correa, G., Assuncao, P., da Silva Cruz, L.A., Agostini, L.: Adaptive coding tree for complexity control of high efficiency video encoders. In: 2012 Picture Coding Symposium, pp. 425–428. IEEE (2012)

  35. Rosewarne, C., Bross, B., Naccari, M., Sharman, K.: High efficiency video coding (HEVC) test model 16 encoder description. Document: JCTVC-V1002, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Geneva (2015)

  36. Bossen, F.: Common test conditions and software reference configurations. Jt. Collab. Team Video Coding ITU-T SG16 WP3 ISO/IEC JTC1/SC29/WG11, 5th Meet (2011)

  37. Bjontegaard, G.: Calculation of average PSNR differences between RD-curves. ITU-T Q.6/SG16, Doc. VCEG-M33, 15th Meet. Austin, Texas (2001)

  38. Lee, J., Shin, I.H., Park, H.W.: Adaptive intra-frame assignment and bit-rate estimation for variable GOP length in H.264. IEEE Trans. Circuits Syst. Video Technol. 16, 1271–1279 (2006). https://doi.org/10.1109/TCSVT.2006.881856

    Article  Google Scholar 

  39. Glantz, S.A., Slinker, B.K., Neilands, T.B.: Primer of Applied Regression and Analysis of Variance. McGraw-Hill, New York (1990)

    Google Scholar 

  40. Zoni, D., Cremona, L., Fornaciari, W.: All-digital control-theoretic scheme to optimize energy budget and allocation in multi-cores. IEEE Trans. Comput. 69, 706–721 (2020). https://doi.org/10.1109/TC.2019.2963859

    Article  MATH  Google Scholar 

  41. Khan, M.U.K., Shafique, M., Henkel, J.: Power-efficient workload balancing for video applications. IEEE Trans. Very Large Scale Integr. Syst. 24, 2089–2102 (2016). https://doi.org/10.1109/TVLSI.2015.2504415

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Computer Engineering, College of Engineering, University of Tehran, 19466-14886, Tehran, Iran

    Elahe Hosseini, Farhad Pakdaman, Mahmoud Reza Hashemi & Mohammad Ghanbari

  2. Emeritus Professor at the School of Computer Science and Electronic Engineering, University of Essex, Colchester, CO4 3SQ, UK

    Mohammad Ghanbari

Authors
  1. Elahe Hosseini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Farhad Pakdaman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mahmoud Reza Hashemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohammad Ghanbari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mahmoud Reza Hashemi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hosseini, E., Pakdaman, F., Hashemi, M.R. et al. Fine-grain complexity control of HEVC intra prediction in battery-powered video codecs. J Real-Time Image Proc 18, 603–618 (2021). https://doi.org/10.1007/s11554-020-00996-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-00996-7

Keywords

Search

Navigation