Multi-level complexity reduction for HEVC multiview coding

  • Caoyang Jiang
  • Saeid Nooshabadi
Original Research Paper


Standardized in 2014, multiview extension of high efficiency video coding (MV-HEVC) offers significantly better compression performance of up to 50% for multiview and 3D videos compared to multiple independent single view HEVC coding. However, the extreme high computational complexity of MV-HEVC demands significant optimization of the encoder. In this work, we propose a series of optimization techniques at various levels of abstraction: non-aggregation massively parallel motion estimation (ME) and disparity estimation (DE) for prediction units, fractional and bidirectional ME/DE, quantization parameter-based early termination of coding tree unit (CTU), and optimized resource-scheduled wave front parallel processing for CTU. When evaluated over three views for all available official multiview video coding test sequences, proposed optimization outperforms the anchor encoder by average factor of 5.4 at the cost of 4.4% bitrate (DBR) increase at no loss in PSNR, or alternatively a PSNR degradation of 0.12 dB at no change to the DBR.


HEVC H.265 Multiview Motion and density estimation Quantization Coding tree unit GPU 


  1. 1.
    Bjontegaard, G.: Calculation of average PSNR differences between RD-curves. In: ITU—Telecommunications Standardization Sector Video Coding Expert Group (VCEG) 13th Meeting, ITU-T Study Group 16, Question 6 (ITU-T SG16 Q.6), Doc No. VCEG-M33, Austin, TX (2001)Google Scholar
  2. 2.
    Bossen, F.: Common HM test conditions and software reference configurations. Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 8th Meeting, MPEG No. m24011, JCT-VC No. JCTVC-H1100 (2012)Google Scholar
  3. 3.
    Chi, C.C., Alvarez-Mesa, M., Juurlink, B., Clare, G., Henry, F., Pateux, S., Schierl, T.: Parallel scalability and efficiency of hevc parallelization approaches. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1827–1838 (2012). CrossRefGoogle Scholar
  4. 4.
    Choi, K., Park, S.H., Jang, E.S.: Coding tree pruning based CU early termination. In: Document JCTVC-F092, JCT-VC, Torino, Italy (2011)Google Scholar
  5. 5.
    Fan, R., Zhang, Y., Li, B.: Motion classification-based fast motion estimation for high efficiency video coding. IEEE Trans. Multimed. 99, 1–1 (2016). Google Scholar
  6. 6.
    Flierl, M., Girod, B.: Multiview video compression. IEEE Signal Process. Mag. 24(6), 66–76 (2007). CrossRefGoogle Scholar
  7. 7.
    Gweon, R.H., Lee, Y.L.: Early termination of CU encoding to reduce HEVC complexity. In: Document JCTVC-F045, JCT-VC, Torino, Italy (2011)Google Scholar
  8. 8.
    Heinrich Hertz Institute (HHI) (2014) H.265/MPEG HEVC Multiview Coding Reference Software (HTM) 16.2.
  9. 9.
    ITU-T, JTC I (2012) Advanced video coding for generic audiovisual servicesGoogle Scholar
  10. 10.
    ITU-T and ISO/IEC JTC (2015) H.265, Series H: Audiovisual and Multimedia Systems, Infrastructure of Audiovisual Services—Coding of Moving VideoGoogle Scholar
  11. 11.
    Jiang, C., Nooshabadi, S.: GPU accelerated motion and disparity estimations for multiview coding. In: IEEE International Conference on Image Processing (ICIP), Melbourne, Australia, pp. 2106–2110 (2013).
  12. 12.
    Jiang, C., Nooshabadi, S.: Parallel multiview video coding exploiting group of pictures level parallelism. IEEE Trans. Parallel Distrib. Syst. (2015).
  13. 13.
    Jiang, C., Nooshabadi, S.: A scalable massively parallel motion and disparity estimation scheme for multiview video coding. IEEE Trans. Circuits Syst. Video Technol. 26(2), 346–359 (2016)CrossRefGoogle Scholar
  14. 14.
    Jimnez-Moreno, A., Martnez-Enrquez, E., de Mara, F.D.: Complexity control based on a fast coding unit decision method in the HEVC video coding standard. IEEE Trans. Multimed. 18(4), 563–575 (2016). CrossRefGoogle Scholar
  15. 15.
    Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC 29/WG 11 (2015) HEVC Test Model (HM) 16. Reference Software.
  16. 16.
    Merkle, P., Smolic, A., Müller, K., Wiegand, T.: Efficient prediction structures for multiview video coding. IEEE Trans. Circuits Syst. Video Technol. 17(11), 1461–1473 (2007). CrossRefGoogle Scholar
  17. 17.
    NVIDIA (2015) NVIDIA Compute Unified Device Architecture (CUDA) C Programming Guide 7.5.
  18. 18.
    Pieters, B., Hollemeersch, C.F., J De Cock, P.L., Neve, W.D., Walle, R.V.D.: Parallel deblocking filtering in MPEG-4 AVC/H.264 on massively parallel architectures. IEEE Trans. Circuits Syst. Video Technol. 21(1), 96–100 (2011). CrossRefGoogle Scholar
  19. 19.
    Radicke, S., Hahn, J.U., Wang, Q., Grecos, C.: Many-Core HEVC Encoding Based on Wavefront Parallel Processing and GPU-Accelerated Motion Estimation. Springer, Cham, pp. 393–417 (2015).
  20. 20.
    Rusanovsky, D., Müller, K., Vetro, A.: Common Test Conditions of 3DV Core Experiments. Joint Collaborative Team on 3D Video Coding Extension Development (JCT-3V) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 3rd Meeting, MPEG No. m28363, JCT3V-VC No. JCT3V-C1100 (2013)Google Scholar
  21. 21.
    Shami, M., Hemani, A.: Classification of massively parallel computer architectures. In: IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) PhD Forum, 2012, pp. 344–351 (2012).
  22. 22.
    Sullivan, G., Boyce, J., Chen, Y., Ohm, J.R., Segall, C., Vetro, A.: Standardized extensions of high efficiency video coding (HEVC). IEEE J. Sel. Top. Signal Process. 7(6), 1001–1016 (2013). CrossRefGoogle Scholar
  23. 23.
    Sze, V., Budagavi, M. (eds.): GJS High Efficiency Video Coding Algorithms and Architectures. Springer, Cham (2014)Google Scholar
  24. 24.
    Tech, G., Chen, Y., Muller, K., Ohm, J.R., Vetro, A., Wang, Y.K.: Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 26(1), 35–49 (2016). CrossRefGoogle Scholar
  25. 25.
    Tourapis, A.M.: Enhanced predictive zonal search for single and multiple frame motion estimation. In: Visual Communications and Image Processing, San Jose, CA, pp. 1069–1079 (2002)Google Scholar
  26. 26.
    Vetro, A., Wiegand, T., Sullivan, G.: Overview of the stereo and multiview video coding extensions of the H.264/MPEG-4 AVC standard. Proc. IEEE 99(4), 626–642 (2011). CrossRefGoogle Scholar
  27. 27.
    Wang, X., Song, L., Chen, M., Yang, J.: Paralleling variable block size motion estimation of HEVC on multi-core CPU plus GPU platform. In: IEEE International Conference on Image Processing (ICIP), Tain City, Taiwan, pp. 1836–1839 (2013).
  28. 28.
    Wu, B.F., Peng, H.Y., Yu, T.L.: Efficient hierarchical motion estimation algorithm and its vlsi architecture. IEEE Trans. Very Large Scale Integr. VLSI Syst. 16(10), 1385–1398 (2008). CrossRefGoogle Scholar
  29. 29.
    Xiao, W., Li, B., Xu, J., Shi, G., Wu, F.: HEVC encoding optimization using multicore CPUs and GPUs. IEEE Trans. Circuits Syst. Video Technol. 25(11), 1830–1843 (2015). CrossRefGoogle Scholar
  30. 30.
    Xie, L., Huang, L., Chen, B.: UMHexagonS search algorithm for fast motion estimation. In: IEEE International Conference on Computer Research and Development (ICCRD), Shanghai, China, vol. 1, pp. 483–487 (2011)Google Scholar
  31. 31.
    Xiong, J., Li, H., Wu, Q., Meng, F.: A fast HEVC inter CU selection method based on pyramid motion divergence. IEEE Trans. Multimed. 16(2), 559–564 (2014). CrossRefGoogle Scholar
  32. 32.
    Yang, J., Kim, J., Won, K., Lee, H., Jeon, B.: Early SKIP detection for HEVC. In: document JCTVC-G543, JCT-VC, Geneva, Switzerland (2011)Google Scholar
  33. 33.
    Zhao, L., Zhang, L., Ma, S., Zhao, D.: Fast mode decision algorithm for intra prediction in (hevc). In: IEEE Visual Communications and Image Processing (VCIP), pp. 1–4 (2011).
  34. 34.
    Zhu, C., Lin, X., Chau, L.: Hexagon-based search pattern for fast block motion estimation. IEEE Trans. Circuits Syst. Video Technol. 12(5), 349–355 (2002). CrossRefGoogle Scholar
  35. 35.
    Zhu, S., Ma, K.: A new diamond search algorithm for fast block-matching motion estimation. IEEE Trans. Image Process. 9(2), 287–290 (2000). CrossRefGoogle Scholar
  36. 36.
    Zone ID: Integer Intrinsic Intrinsics for Intel® Streaming SIMD Extensions 2 (Intel® SSE2) (2014).
  37. 37.
    Zupancic, I., Blasi, S.G., Peixoto, E., Izquierdo, E.: Inter-prediction optimizations for video coding using adaptive coding unit visiting order. IEEE Trans. Multimed. 18(9), 1677–1690 (2016). CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringMichigan TechHoughtonUSA

Personalised recommendations