Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Contents-aware partitioning algorithm for parallel high efficiency video coding

  • 151 Accesses

  • 1 Citations

Abstract

We introduce a new parallelization method for high-efficiency video coding (HEVC), which resolves the shortcomings of the existing tile-based parallel processing method. The parallel HEVC performs encoding by dividing a frame into numerous parallel units. This decreases the compression efficiency compared with sequential HEVC, because it artificially breaks the data correlation within a frame, which is called the parallelization overhead. The traditional parallel techniques such as Tiles and wavefront parallel processing (WPP) inherently introduce a high parallelization overhead because they simply divide a frame statically without considering the contents of the frame. The proposed new parallel encoding scheme resolves such problems by partitioning a frame based on the meaningful contents. In order to analyze the correlations within a frame and define the contents, the features within a frame are first extracted and clustered. In the feature clustering algorithm, two factors are considered to balance the workload between parallel units: (1) the number of features in each cluster and (2) the number of coding tree units (CTU) occupied by each cluster. The frame is partitioned based on the result of clustering, and the partitions are encoded in parallel. The proposed scheme achieves a bit-saving of up to 7.21%, with an average of 3.71%, along with an average time-saving of 20.50% compared to the Tiles technique.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

References

  1. 1.

    Ahn YJ, Hwang TJ, Sim DG, Han WJ (2013) Complexity model based load-balancing algorithm for parallel tools of hevc. In: Visual Communications and Image Processing (VCIP), 2013 IEEE, pp 1–5

  2. 2.

    Bjøntegaard G (2008) ”Improvements of the BD-PSNR model,” document VCEG-AI11 ITU-t SG16/q6

  3. 3.

    Blumenberg C, Palomino D, Bampi S, Zatt B (2013) Adaptive contentbased tile partitioning algorithm for the hevc standard. In: Picture Coding Symposium (PCS), IEEE, pp 185–188

  4. 4.

    Bossen F (2013) ”Common HM Test Conditions and Software Reference Configurations,” document HCTVC-l1100, JCT-VC of ISO/ICE and ITU-t, Turin, Geneva, Switzerland

  5. 5.

    Chan CH, Tu CC, Tsai WJ (2017) Improve load balancing and coding efficiency of tiles in high efficiency video coding by adaptive tile boundary. Journal of Electronic Imaging 26(1):006–013

  6. 6.

    Chen B, Yang Z, Huang S, Du X, Cui Z, Bhimani J, Xie X, Mi N (2017) Cyber-physical system enabled nearby traffic flow modelling for autonomous vehicles. In: 36th International Performance Computing and Communications Conference (IPCCC), 2017 IEEE. IEEE

  7. 7.

    Chi CC, Alvarez-Mesa M, Juurlink B, Clare G, Henry F, Pateux S, Schierl T (2012) Parallel scalability and efficiency of hevc parallelization approaches. IEEE Transactions on Circuits and Systems for Video Technology 22(12):1827–1838

  8. 8.

    Ding M, Guoliang F (2015) Multilayer joint gait-pose manifolds for human gait motion modeling. IEEE Transactions on Cybernetics 45(11):2413–2424

  9. 9.

    Ding M, Guoliang F (2016) Articulated and generalized gaussian kernel correlation for human pose estimation. IEEE Trans Image Process 25(2):776–789

  10. 10.

    Fuldseth A, Horowitz M, Xu S, Zhou M (2011) Tiles, JCTVC, Tech. Rep., JCTVC-E408. [Online]. Available: http://phenix.intevry.fr/jct/doc_end_user/documents/5Geneva/wg11/JCTVC-E408-v2.zip

  11. 11.

    Georgakarakos G, Tsiopoulos L, Lillius J, Haldin J, Falk U, U (2015) Performance evaluation of parallel hevc strategies. In: 23rd Euromicro International Conference on Parallel Distributed and Network-Based Processing (PDP), 2015, pp 137–144

  12. 12.

    Henry F, Pateux S (2011) Wavefront parallel processing, JCTVC, Tech. Rep., JCTVC-E196. [Online]. Available: http://phenix.intevry.fr/jct/doc_end_user/documents/5Geneva/wg11/JCTVC-E196-v4.zip

  13. 13.

    Jin X, Dai Q (2016) Clustering-based content adaptive tiles under onchip memory constraints. IEEE Transactions on Multimedia 18(12):2331–2344

  14. 14.

    Lee J, Shin I, Park H (2006) Adaptive intra-frame assignment and bit-rate estimation for variable GOP length in H. 264. IEEE Transactions on Circuits and Systems for Video Technology 16(10):1271–1279

  15. 15.

    McCann K, Bross B, Han WJ, Kim IK, Sugimoto K, Sullivan GJ (2014) ”High Efficiency Video Coding (HEVC) Test Model 15 (HM 15) Encoder Description,” document JCTVC-q1002, JCT-VC of ISO/IEC and ITU-t Valencia, Spain

  16. 16.

    Migallón H, Piñol P, López-Granado O, Galiano V, Malumbres MP (2017) Performance analysis of frame partitioning in parallel HEVC encoders. The Journal of Supercomputing 73(1):543–556

  17. 17.

    Misra K, Segall A, Horowitz M, Xu S, Fuldseth A, Zhou M (2013) An overview of tiles in hevc. IEEE journal of selected topics in signal processing 7(6):969–977

  18. 18.

    Pearson K (1895) Note on regression and inheritance in the case of two parents. Proc R Soc Lond 58:240–242

  19. 19.

    Pinol P, López-Granado O, Migallón H, Galiano V, Malumbres MP (2016) Tile partition analysis for a parallel HEVC encoder. In: Proceedings of the 16th International Conference on Computational and Mathematical Methods in Science and Engineering (CMMSE), pp 989–998

  20. 20.

    Rosten E, Drummond T (2005) Fusing points and lines for high performance tracking. In: 10th IEEE International Conference on Computer Vision, 2005. ICCV 2005, vol. 2. IEEE, pp 1508–1515

  21. 21.

    Rosten E, Porter R, Drummond T (2010) Faster and better: a machine learning approach to corner detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(1):105–119

  22. 22.

    Sullivan GJ, Ohm JR, Han WJ, Wiegand T (2012) Overview of the high efficiency video coding (hevc) standard. IEEE Transactions on Circuits and Systems for Video Technology 22(12):1649–1668

  23. 23.

    Wang XYJ, He D (2011) On BD-Rate Calculation, JCTVC, Tech. Rep., JCTVC-F270. [Online]. Available: http://phenix.intevry.fr/jct/docenduser/documents/6Torino/wg11/JCTVCF270-v1.zip

  24. 24.

    Xie X, Liu S, Yang C, Yang Z, Xu J, Zhang C, Zhai X (2017) The application of smart materials in tactile actuators for tactile information delivery. arXiv:1708.07077

  25. 25.

    Xie X, Livermore C (2016) A pivot-hinged, multilayer SU-8 micro motion amplifier assembled by a self-aligned approach. In: 29th International Conference on Micro Electro Mechanical Systems (MEMS), 2016 IEEE. IEEE

  26. 26.

    Xie X, Livermore C (2017) Passively self-aligned assembly of compact barrel hinges for high-performance, out-of-plane mems actuators. In: 30th International Conference on Micro Electro Mechanical Systems (MEMS), 2017 IEEE. IEEE

  27. 27.

    Xie X, Zaitsev Y, Velasquez-Garcia L, Teller S, Livermore C (2014) Compact, scalable, high-resolution, MEMS-enabled tactile displays. In: Proceedings of the solid-state sensors, actuators, and microsystems workshop, pp 127–130

  28. 28.

    Xie X, Zaitsev Y, Velásquez-García LF, Teller SJ, Livermore C (2014) Scalable, MEMS-enabled, vibrational tactile actuators for high resolution tactile displays. J Micromech Microeng 24(12):125014

  29. 29.

    Yan C, Xie H, Liu S, Yin J, Zhang Y, Dai Q (2018) Effective Uyghur language text detection in complex background images for traffic prompt identification. IEEE transactions on intelligent transportation systems 19(1):220–229

  30. 30.

    Yan C, Xie H, Yang D, Yin J, Zhang Y, Dai Q (2018) Supervised hash coding with deep neural network for environment perception of intelligent vehicles. IEEE transactions on intelligent transportation systems 19(1):284–295

  31. 31.

    Yan C, Zhang Y, Dai F, Zhang J, Li L, Dai Q (2014) Efficient parallel HEVC intra-prediction on many-core processor. Electron Lett 50(11):805–806

  32. 32.

    Yan C, Zhang Y, Xu J, Dai F, Li L, Dai Q, Wu F (2014) A highly parallel framework for HEVC coding unit partitioning tree decision on many-core processors. IEEE Signal Processing Letters 21(5):573–576

  33. 33.

    Yan C, Zhang Y, Xu J, Dai F, Zhang J, Dai Q, Wu F (2014) Efficient parallel framework for HEVC motion estimation on many-core processors. IEEE Transactions on Circuits and Systems for Video Technology 24(12):2077–2089

  34. 34.

    Zhou M, Sze V, Budagavi M (2012) Parallel tools in HEVC for high-throughput processing. In: Applications of Digital Image Processing XXXV, Vol. 8499, International Society for Optics and Photonics

Download references

Acknowledgements

This work was funded by grants from the Digital Media & Communication R&D Team, Samsung Electronics Co., Ltd. and by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP)(No. NRF-2018R1A2A2A05018941). Won Woo Ro is the corresponding author.

Author information

Correspondence to Won Woo Ro.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kim, K., Ro, W.W. Contents-aware partitioning algorithm for parallel high efficiency video coding. Multimed Tools Appl 78, 11427–11442 (2019). https://doi.org/10.1007/s11042-018-6619-8

Download citation

Keywords

  • HEVC
  • Tiles
  • Clustering
  • Frame partitioning