Skip to main content
SpringerLink
Log in

Group-normalized deep CNN-based in-loop filter for HEVC scalable extension

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

High Efficiency Video Coding (HEVC) is the recent video coding standard that can compress raw video at a higher compression state. The extension of HEVC, Scalable High Efficiency Video Coding (SHVC), also has the similar compression phenomenon of HEVC in addition to the implementation of multiple single-layer HEVC streams along with the interlayer reference modules, although the layer-based SHVC incurs more artifacts after compression compared to HEVC resulting with severe degradation in the video quality. To ease this, in-loop filter is used to remove artifacts in H.265 video coding standard. Although the artifacts will be more severe for multiple-layered codec SHVC compared to single-layer HEVC. With the development in deep learning, a group-normalized deep convolutional neural network (gDCNN) is proposed for SHVC in-loop filter to enhance the performance. Initially, the troubles that are met while modeling the traditional CNN that includes normalization, learning capability and the loss functions are examined. Following, on the basis of statistical analysis, the proposed gDCNN is introduced to remove the artifacts efficiently. It is achieved by a group-wise normalization approach, a feature extraction and fusion and a precise loss function. The simulation setting shows 4.2% BD-BR decrement with 0.46 dB increment in BD-PSNR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Chan-Seob, P., Byung-Gyu, K.: Performance analysis of inter-layer prediction in scalable extension of HEVC (SHVC) for adaptive media service. Displays 44, 27–36 (2016)

    Article  Google Scholar 

  2. Pourazad, M.T., Doutre, C., Azimi, M., Nasiopoulos, P.: HEVC: The new gold standard for video compression. IEEE Consum. Electron. Mag. 1(3), 36–46 (2012)

    Article  Google Scholar 

  3. Test Model for Scalable Extensions of High Efficiency Video Coding (HEVC), ISO/IEC and JTC1/SC29/WG11, 2013.

  4. Pan, Z., Lei, J., Zhang, Y., Wang, F.L.: Adaptive fractionalpixel motion estimation skipped algorithm for efficient HEVC motion estimation. ACM Trans. Multimed. Comput. Commun. Appl. 14(1), 1–19 (2018)

    Article  Google Scholar 

  5. Norkin, A., et al.: HEVC deblocking filter. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1746–1754 (2012)

    Article  Google Scholar 

  6. Fu, C.-M., et al.: Sample adaptive offset in the HEVC standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1755–1764 (2012)

    Article  Google Scholar 

  7. Sze, V., Budagavi, M., Sullivan, G.J.: High Efficiency Video Coding (HEVC) (Integrated Circuits and Systems), pp. 200–203. Springer, London, U.K. (2014)

    Book  Google Scholar 

  8. LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)

    Article  Google Scholar 

  9. Dong, C., Loy, C.C., He, K., Tang, X.: in Learning a deep convolutional network for image super-resolution. Proceedings of European Conference on Computer Vision (ECCV) (Springer, London, 2014), pp. 184–199.

  10. Dong, C., Loy, C.C., Tang, X.: Tang, in Accelerating the super-resolution convolutional neural network. Proceedings of European Conference on Computer Vision (ECCV) (Springer, London, 2016), pp. 391–407.

  11. Kim, J. Lee, J.K., Lee, K.M.: in Accurate image super-resolution using very deep convolutional networks. Proceedings of IEEE Conference on Compututer Vision Pattern Recognit. (CVPR) (2016), pp. 1646–1654.

  12. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)

    Article  MathSciNet  Google Scholar 

  13. Zhang, K., Zuo, W., Zhang, L.: FFDNet: Toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Process. 27(9), 4608–4622 (2018)

    Article  MathSciNet  Google Scholar 

  14. Park, W.-S., Kim, M.: in CNN-based in-loop filtering for coding efficiency improvement. Proceedings of IEEE 12th Image, Video, Multidimensional Signal Process. Workshop (IVMSP) (2016), pp. 1–5.

  15. Dai, Y., Liu, D., Wu, F.: in A convolutional neural network approach for post-processing in HEVC intra coding. Proceedings of MultiMedia Modeling (MMM), (2017), pp. 28–39.

  16. Yang, R., Xu, M., Liu, T., Wang, Z., Guan, Z.: Enhancing quality for HEVC compressed videos. IEEE Trans. Circuits Syst. Video Technol. 29(7), 2039–2054 (2019)

    Article  Google Scholar 

  17. Soh, J.W., et al.: Reduction of video compression artifacts based on deep temporal networks. IEEE Access 6, 63094–63106 (2018)

    Article  Google Scholar 

  18. Zhang, Y., Shen, T., Ji, X., Zhang, Y., Xiong, R., Dai, Q.: Residual highway convolutional neural networks for in-loop filtering in HEVC. IEEE Trans. Image Process. 27(8), 3827–3841 (2018)

    Article  MathSciNet  Google Scholar 

  19. Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult”. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)

    Article  Google Scholar 

  20. LeCun, Y., Bottou, L., Orr, G.B., Muller, K.-R.: in Efficient backprop. Neural Networks: Tricks of the Trade (1998).

  21. Lyu, S., Simoncelli E.P.: Nonlinear image representation using divisive normalization. In CVPR (2008).

  22. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML (2015).

  23. Rebuffi, S.-A., Bilen, H., Vedaldi, A.:. Learning multiple visual domains with residual adapters. In NIPS (2017).

  24. Salimans, T., Kingma, D.P.: Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In NIPS (2016).

  25. Ioffe, S.: Batch renormalization: Towards reducing minibatch dependence in batch-normalized models. In NIPS (2017).

  26. Peng, C., Xiao, T., Li, Z., Jiang, Y., Zhang, X., Jia, K., Yu, G., Sun, J.: MegDet: A large mini-batch object detector. In CVPR (2018).

  27. Dean, J., Corrado, G., Monga, R., Chen, K., Devin, M., Mao, M., Senior, A., Tucker, P., Yang, K., Le, Q.V., et al. Large scale distributed deep networks. In NIPS (2012).

  28. Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. In NIPS (2012).

  29. Xie, S., Girshick, R., Dollar, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In CVPR (2017).

  30. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861, (2017).

  31. Chollet, F.: Xception: Deep learning with depth wise separable convolutions. In CVPR (2017).

  32. Zhang, X., Zhou, X., Lin, M., Sun, J.: ShuffleNet: An extremely efficient convolutional neural network for mobile devices. In CVPR (2018).

  33. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In CVPR (2016).

  34. Lee, S.Y., Yang, Y., Kim, D., Cho S., Oh, B. T.: Offset-based in-loop filtering with a deep network in HEVC. IEEE Access 2020.

  35. Balaji, L., Thyagharajan, K.K.: An enhanced performance for H.265/SHVC based on combined AEGBM3D filter and back-propagation neural network. SIViP 12(5), 809–817 (2018)

    Article  Google Scholar 

  36. Shen, L., An, P., Feng, G.: Low-complexity scalable extension of the high-efficiency video coding (SHVC) encoding system. ACM Trans. Multimedia Comput. Commun. Appl. 15(2), 44 (2019)

    Google Scholar 

  37. Dhanalakshmi, A., Nagarajan, G.: Combined spatial temporal based in-loop filter for scalable extension of HEVC. ICT Express 6(4), 306 (2020)

    Article  Google Scholar 

  38. Dhanalakshmi, A., Nagarajan, G.: Convolutional neural network based deblocking filter for SHVC in H.265. Signal Image Video Processing 14(8), 1635–1645 (2020)

    Article  Google Scholar 

  39. Xie, S., Girshick, R., Dollar, P., Tu, Z., He, K.: in Aggregated residual transformations for deep neural networks. in Proceedings IEEE Conference on Computer Vision Pattern Recognit. (CVPR) (2017), pp. 5987–5995.

  40. He, K., Zhang, X., Ren, S., Sun, J.: in Deep residual learning for image recognition. Proceedings of IEEE Conference on Computer Vision Pattern Recognit. (CVPR), (2016), pp. 770–778.

  41. Li, Y., et al.: Convolutional neural network-based block up-sampling for intra frame coding. IEEE Trans. Circuits Syst. Video Technol. 28(9), 2316–2330 (2018)

    Article  Google Scholar 

  42. SHM-7.0. [Online]. Available: https://hevc.hhi.fraunhofer.de/svn/svn SHVCSoftware/tags/SHM-7.0, accessed Oct. 30, 2015.

  43. Bjontegaard, G.: in Calculation of average psnr differences between rdcurves. ITU-T Q. 6/SG16 VCEG, 15th Meeting, Austin, Texas, USA (2001)

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Electronics, Sathyabama Institute of Science and Technology, Chennai, India

    A. Dhanalakshmi

  2. Department of Computer and Communication Engineering, Panimalar Engineering College, Chennai, India

    A. Dhanalakshmi

  3. School of Computing, Department of Computer Science and Engineering, Sathyabama Institute of Science and Technology, Chennai, India

    G. Nagarajan

Authors
  1. A. Dhanalakshmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. Nagarajan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. Dhanalakshmi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dhanalakshmi, A., Nagarajan, G. Group-normalized deep CNN-based in-loop filter for HEVC scalable extension. SIViP 16, 437–445 (2022). https://doi.org/10.1007/s11760-021-01966-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-01966-7

Keywords

Search

Navigation