Journal of Real-Time Image Processing

, Volume 12, Issue 1, pp 107–122 | Cite as

Complexity scalability for real-time HEVC encoders

  • Guilherme Correa
  • Pedro Assuncao
  • Luciano Agostini
  • Luis A. da Silva Cruz
Original Research Paper


The high efficiency video coding (HEVC) standard achieves improved compression efficiency in comparison to previous standards at the cost of much higher computational complexity and consequently longer processing times, which may compromise real-time software-based video encoding, especially at high resolutions. This article addresses the problem of enabling complexity scalability in HEVC encoders by trading-off processing time for rate–distortion (R–D) performance in a controlled manner. The proposed method is based on dynamic constraining of HEVC coding treeblocks (CTBs) by limiting the prediction block (PB) shapes and the maximum tree depth used in each CTB, to decrease the number of R–D evaluations performed in the optimization process. The complexity-scalable encoder is capable of adjusting the processing time used in each group of pictures, according to a predefined target. The results show that processing times can be scaled down to 50 % with negligible R–D performance losses and down to 20 % at a maximum BD-PSNR decrease of 1.41 dB, which is acceptable in many applications and in power constrained devices. The simplicity of the scaling algorithm and the possibility of continuous adjustment of the scaling factor make it amenable to control real-time software-based HEVC video encoders.


High efficiency video coding (HEVC) Video coding Real time Computational complexity 


  1. 1.
    Budagavi, M.: Real-time image and video processing in portable and mobile devices. J. Real-Time Image Proc. 1(1), 3–7 (2006)CrossRefGoogle Scholar
  2. 2.
    ISO/IEC-JCT1/SC29/WG11: High efficiency video coding (HEVC) text specification draft 9, doc. JCTVC-K1003. Shanghai (2012)Google Scholar
  3. 3.
    Correa, G., Assuncao, P., Agostini, L., da Silva Cruz, L.A.: Performance and computational complexity assessment of high efficiency video encoders. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1899–1909 (2012)CrossRefGoogle Scholar
  4. 4.
    Joint Collaborative Team on Video Coding (JCT-VC). Accessed 23 September 2013
  5. 5.
    ISO/IEC-JCT1/SC29/WG11: Comparison of compression performance of HEVC working draft 4 with AVC high profile, doc. JCTVC-G399. Geneva (2011)Google Scholar
  6. 6.
    Sullivan, G.J., Wiegand, T.: Rate-distortion optimization for video compression. IEEE Signal Process. Mag. 15(6), 74–90 (1998)CrossRefGoogle Scholar
  7. 7.
    Ren, J., Kehtarnavaz, N., Budagavi, M.: A fast feature-assisted adaptive early termination approach for multiple reference frames motion estimation in H.264. J. Real-Time Image Proc. 3(1–2), 77–88 (2008)CrossRefGoogle Scholar
  8. 8.
    Babionitakis, K., et al.: A real-time motion estimation FPGA architecture. J. Real-Time Image Proc. 3(1–2), 3–20 (2008)CrossRefGoogle Scholar
  9. 9.
    Man-Yau, C., Wan-Chi, S.: Computationally-scalable motion estimation algorithm for H.264/AVC video coding. IEEE Trans. Consum. Electron. 56(2), 895–903 (2010)CrossRefGoogle Scholar
  10. 10.
    Li, Z., Wen, G.: Reusable architecture and complexity-controllable algorithm for the integer/fractional motion estimation of H.264. IEEE Trans. Consum. Electron. 53(2), 749–756 (2007)CrossRefGoogle Scholar
  11. 11.
    Nieto, M., Salgado, L., Cabrera, J., García, N.: Fast mode decision on H.264/AVC baseline profile for real-time performance. J. Real-Time Image Proc. 3(1–2), 61–75 (2008)CrossRefGoogle Scholar
  12. 12.
    Ren, J., Kehtarnavaz, N.: Fast adaptive termination mode selection for H.264 scalable video coding. J. Real-Time Image Proc. 4(1), 13–21 (2009)CrossRefGoogle Scholar
  13. 13.
    Tai, S.-C., Chang, C.-Y., Chen, B.-J., Hu, J.-F.: Speeding up the decisions of quad-tree structures and coding modes for HEVC coding units. In: International computer symposium ICS 2012, pp. 393–401 (2013)Google Scholar
  14. 14.
    Xiaolin, S., Lu, Y., Jie, C.: Fast coding unit size selection for HEVC based on Bayesian decision rule. In: 2012 picture coding symposium, pp. 453–456 (2012)Google Scholar
  15. 15.
    Sampaio, F., Bampi, S., Grellert, M., Agostini, L., Mattos, J.: Motion vectors merging: low complexity prediction unit decision heuristic for the inter-prediction of HEVC encoders. In: 2012 IEEE international conference on multimedia and expo, pp. 657–662 (2012)Google Scholar
  16. 16.
    Guifen, T., Goto, S.: Content adaptive prediction unit size decision algorithm for HEVC intra coding. In: 2012 picture coding symposium, pp. 405–408 (2012)Google Scholar
  17. 17.
    Kiho, C., Jang, E.S.: Early TU decision method for fast video encoding in high efficiency video coding. Electron. Lett. 48, 689–691 (2012)CrossRefGoogle Scholar
  18. 18.
    Correa, G., Assuncao, P., Agostini, L., da Silva Cruz, L.A.: Complexity control of high efficiency video encoders for power-constrained devices. IEEE Trans. Consum. Electron. 57(4), 1866–1874 (2011)CrossRefGoogle Scholar
  19. 19.
    Zhao, T., Wang, Z., Kwong, S.: Flexible mode selection and complexity allocation in high efficiency video coding. IEEE J. Sel. Top. Signal Process. 7(6), 1135–1144 (2013)CrossRefGoogle Scholar
  20. 20.
    Ukhanova, A., Milani, S., Forchhammer, S.: Game-theoretic rate-distortion-complexity optimization for HEVC. In: 2013 IEEE international conference on image processing, pp. 1995–1999 (2013)Google Scholar
  21. 21.
    ISO/IEC-JCT1/SC29/WG11: High efficiency video coding (HEVC) test model 9 (HM 9) encoder description, doc. JCTVC-K1002. Shanghai (2012)Google Scholar
  22. 22.
    Bjontegaard, G.: Calculation of average PSNR differences between RD-curves, doc. VCEG-M33. Austin (2001)Google Scholar
  23. 23.
    VTune™ Amplifier XE 2011 from Intel. Accessed 23 September 2013
  24. 24.
    ISO/IEC-JCT1/SC29/WG11: Common test conditions and software reference configurations, doc. JCTVC-J1100. Stockholm (2012)Google Scholar
  25. 25.
    Bossen, F., Bross, B., Suhring, K., Flynn, D.: HEVC complexity and implementation analysis. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1685–1696 (2012)CrossRefGoogle Scholar
  26. 26.
    Correa, G., Assuncao, P., Agostini, L., da Silva Cruz, L. A.: Coding tree depth estimation for complexity reduction of HEVC. In: 2013 data compression conference, pp. 43–52 (2013)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Guilherme Correa
    • 1
  • Pedro Assuncao
    • 2
  • Luciano Agostini
    • 3
  • Luis A. da Silva Cruz
    • 1
  1. 1.Instituto de Telecomunicações (IT), Polo IIUniversidade de CoimbraCoimbraPortugal
  2. 2.Instituto de Telecomunicações (IT)LeiriaPortugal
  3. 3.Group of Architectures and Integrated CircuitsFederal University of PelotasPelotasBrazil

Personalised recommendations