Multi-level complexity reduction for HEVC multiview coding

Original Research Paper
  • 17 Downloads

Abstract

Standardized in 2014, multiview extension of high efficiency video coding (MV-HEVC) offers significantly better compression performance of up to 50% for multiview and 3D videos compared to multiple independent single view HEVC coding. However, the extreme high computational complexity of MV-HEVC demands significant optimization of the encoder. In this work, we propose a series of optimization techniques at various levels of abstraction: non-aggregation massively parallel motion estimation (ME) and disparity estimation (DE) for prediction units, fractional and bidirectional ME/DE, quantization parameter-based early termination of coding tree unit (CTU), and optimized resource-scheduled wave front parallel processing for CTU. When evaluated over three views for all available official multiview video coding test sequences, proposed optimization outperforms the anchor encoder by average factor of 5.4 at the cost of 4.4% bitrate (DBR) increase at no loss in PSNR, or alternatively a PSNR degradation of 0.12 dB at no change to the DBR.

Keywords

HEVC H.265 Multiview Motion and density estimation Quantization Coding tree unit GPU 

References

  1. 1.
    Bjontegaard, G.: Calculation of average PSNR differences between RD-curves. In: ITU—Telecommunications Standardization Sector Video Coding Expert Group (VCEG) 13th Meeting, ITU-T Study Group 16, Question 6 (ITU-T SG16 Q.6), Doc No. VCEG-M33, Austin, TX (2001)Google Scholar
  2. 2.
    Bossen, F.: Common HM test conditions and software reference configurations. Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 8th Meeting, MPEG No. m24011, JCT-VC No. JCTVC-H1100 (2012)Google Scholar
  3. 3.
    Chi, C.C., Alvarez-Mesa, M., Juurlink, B., Clare, G., Henry, F., Pateux, S., Schierl, T.: Parallel scalability and efficiency of hevc parallelization approaches. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1827–1838 (2012).  https://doi.org/10.1109/TCSVT.2012.2223056 CrossRefGoogle Scholar
  4. 4.
    Choi, K., Park, S.H., Jang, E.S.: Coding tree pruning based CU early termination. In: Document JCTVC-F092, JCT-VC, Torino, Italy (2011)Google Scholar
  5. 5.
    Fan, R., Zhang, Y., Li, B.: Motion classification-based fast motion estimation for high efficiency video coding. IEEE Trans. Multimed. 99, 1–1 (2016).  https://doi.org/10.1109/TMM.2016.2642786 Google Scholar
  6. 6.
    Flierl, M., Girod, B.: Multiview video compression. IEEE Signal Process. Mag. 24(6), 66–76 (2007).  https://doi.org/10.1109/MSP.2007.905699 CrossRefGoogle Scholar
  7. 7.
    Gweon, R.H., Lee, Y.L.: Early termination of CU encoding to reduce HEVC complexity. In: Document JCTVC-F045, JCT-VC, Torino, Italy (2011)Google Scholar
  8. 8.
    Heinrich Hertz Institute (HHI) (2014) H.265/MPEG HEVC Multiview Coding Reference Software (HTM) 16.2. https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/
  9. 9.
    ITU-T, JTC I (2012) Advanced video coding for generic audiovisual servicesGoogle Scholar
  10. 10.
    ITU-T and ISO/IEC JTC (2015) H.265, Series H: Audiovisual and Multimedia Systems, Infrastructure of Audiovisual Services—Coding of Moving VideoGoogle Scholar
  11. 11.
    Jiang, C., Nooshabadi, S.: GPU accelerated motion and disparity estimations for multiview coding. In: IEEE International Conference on Image Processing (ICIP), Melbourne, Australia, pp. 2106–2110 (2013).  https://doi.org/10.1109/ICIP.2013.6738434
  12. 12.
    Jiang, C., Nooshabadi, S.: Parallel multiview video coding exploiting group of pictures level parallelism. IEEE Trans. Parallel Distrib. Syst. (2015).  https://doi.org/10.1109/TPDS.2015.2485993
  13. 13.
    Jiang, C., Nooshabadi, S.: A scalable massively parallel motion and disparity estimation scheme for multiview video coding. IEEE Trans. Circuits Syst. Video Technol. 26(2), 346–359 (2016)CrossRefGoogle Scholar
  14. 14.
    Jimnez-Moreno, A., Martnez-Enrquez, E., de Mara, F.D.: Complexity control based on a fast coding unit decision method in the HEVC video coding standard. IEEE Trans. Multimed. 18(4), 563–575 (2016).  https://doi.org/10.1109/TMM.2016.2524995 CrossRefGoogle Scholar
  15. 15.
    Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC 29/WG 11 (2015) HEVC Test Model (HM) 16. Reference Software. https://hevc.hhi.fraunhofer.de/
  16. 16.
    Merkle, P., Smolic, A., Müller, K., Wiegand, T.: Efficient prediction structures for multiview video coding. IEEE Trans. Circuits Syst. Video Technol. 17(11), 1461–1473 (2007).  https://doi.org/10.1109/TCSVT.2007.903665 CrossRefGoogle Scholar
  17. 17.
    NVIDIA (2015) NVIDIA Compute Unified Device Architecture (CUDA) C Programming Guide 7.5. http://docs.nvidia.com/cuda/pdf/CUDA_C_Programming_Guide.pdf
  18. 18.
    Pieters, B., Hollemeersch, C.F., J De Cock, P.L., Neve, W.D., Walle, R.V.D.: Parallel deblocking filtering in MPEG-4 AVC/H.264 on massively parallel architectures. IEEE Trans. Circuits Syst. Video Technol. 21(1), 96–100 (2011).  https://doi.org/10.1109/TCSVT.2011.2105553 CrossRefGoogle Scholar
  19. 19.
    Radicke, S., Hahn, J.U., Wang, Q., Grecos, C.: Many-Core HEVC Encoding Based on Wavefront Parallel Processing and GPU-Accelerated Motion Estimation. Springer, Cham, pp. 393–417 (2015).  https://doi.org/10.1007/978-3-319-25915-4_21
  20. 20.
    Rusanovsky, D., Müller, K., Vetro, A.: Common Test Conditions of 3DV Core Experiments. Joint Collaborative Team on 3D Video Coding Extension Development (JCT-3V) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 3rd Meeting, MPEG No. m28363, JCT3V-VC No. JCT3V-C1100 (2013)Google Scholar
  21. 21.
    Shami, M., Hemani, A.: Classification of massively parallel computer architectures. In: IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) PhD Forum, 2012, pp. 344–351 (2012).  https://doi.org/10.1109/IPDPSW.2012.42
  22. 22.
    Sullivan, G., Boyce, J., Chen, Y., Ohm, J.R., Segall, C., Vetro, A.: Standardized extensions of high efficiency video coding (HEVC). IEEE J. Sel. Top. Signal Process. 7(6), 1001–1016 (2013).  https://doi.org/10.1109/JSTSP.2013.2283657 CrossRefGoogle Scholar
  23. 23.
    Sze, V., Budagavi, M. (eds.): GJS High Efficiency Video Coding Algorithms and Architectures. Springer, Cham (2014)Google Scholar
  24. 24.
    Tech, G., Chen, Y., Muller, K., Ohm, J.R., Vetro, A., Wang, Y.K.: Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 26(1), 35–49 (2016).  https://doi.org/10.1109/TCSVT.2015.2477935 CrossRefGoogle Scholar
  25. 25.
    Tourapis, A.M.: Enhanced predictive zonal search for single and multiple frame motion estimation. In: Visual Communications and Image Processing, San Jose, CA, pp. 1069–1079 (2002)Google Scholar
  26. 26.
    Vetro, A., Wiegand, T., Sullivan, G.: Overview of the stereo and multiview video coding extensions of the H.264/MPEG-4 AVC standard. Proc. IEEE 99(4), 626–642 (2011).  https://doi.org/10.1109/JPROC.2010.2098830 CrossRefGoogle Scholar
  27. 27.
    Wang, X., Song, L., Chen, M., Yang, J.: Paralleling variable block size motion estimation of HEVC on multi-core CPU plus GPU platform. In: IEEE International Conference on Image Processing (ICIP), Tain City, Taiwan, pp. 1836–1839 (2013).  https://doi.org/10.1109/ICIP.2013.6738378
  28. 28.
    Wu, B.F., Peng, H.Y., Yu, T.L.: Efficient hierarchical motion estimation algorithm and its vlsi architecture. IEEE Trans. Very Large Scale Integr. VLSI Syst. 16(10), 1385–1398 (2008).  https://doi.org/10.1109/TVLSI.2008.2000526 CrossRefGoogle Scholar
  29. 29.
    Xiao, W., Li, B., Xu, J., Shi, G., Wu, F.: HEVC encoding optimization using multicore CPUs and GPUs. IEEE Trans. Circuits Syst. Video Technol. 25(11), 1830–1843 (2015).  https://doi.org/10.1109/TCSVT.2015.2406199 CrossRefGoogle Scholar
  30. 30.
    Xie, L., Huang, L., Chen, B.: UMHexagonS search algorithm for fast motion estimation. In: IEEE International Conference on Computer Research and Development (ICCRD), Shanghai, China, vol. 1, pp. 483–487 (2011)Google Scholar
  31. 31.
    Xiong, J., Li, H., Wu, Q., Meng, F.: A fast HEVC inter CU selection method based on pyramid motion divergence. IEEE Trans. Multimed. 16(2), 559–564 (2014).  https://doi.org/10.1109/TMM.2013.2291958 CrossRefGoogle Scholar
  32. 32.
    Yang, J., Kim, J., Won, K., Lee, H., Jeon, B.: Early SKIP detection for HEVC. In: document JCTVC-G543, JCT-VC, Geneva, Switzerland (2011)Google Scholar
  33. 33.
    Zhao, L., Zhang, L., Ma, S., Zhao, D.: Fast mode decision algorithm for intra prediction in (hevc). In: IEEE Visual Communications and Image Processing (VCIP), pp. 1–4 (2011).  https://doi.org/10.1109/VCIP.2011.6115979
  34. 34.
    Zhu, C., Lin, X., Chau, L.: Hexagon-based search pattern for fast block motion estimation. IEEE Trans. Circuits Syst. Video Technol. 12(5), 349–355 (2002).  https://doi.org/10.1109/TCSVT.2002.1003474 CrossRefGoogle Scholar
  35. 35.
    Zhu, S., Ma, K.: A new diamond search algorithm for fast block-matching motion estimation. IEEE Trans. Image Process. 9(2), 287–290 (2000).  https://doi.org/10.1109/83.821744 CrossRefGoogle Scholar
  36. 36.
    Zone ID: Integer Intrinsic Intrinsics for Intel® Streaming SIMD Extensions 2 (Intel® SSE2) (2014). https://software.intel.com/en-us/node/583147
  37. 37.
    Zupancic, I., Blasi, S.G., Peixoto, E., Izquierdo, E.: Inter-prediction optimizations for video coding using adaptive coding unit visiting order. IEEE Trans. Multimed. 18(9), 1677–1690 (2016).  https://doi.org/10.1109/TMM.2016.2579505 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringMichigan TechHoughtonUSA

Personalised recommendations