Advertisement

Down-Sampling Based Video Coding with Degradation-Aware Restoration-Reconstruction Deep Neural Network

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11961)

Abstract

Recently deep learning techniques have shown remarkable progress in image/video super-resolution. These techniques can be employed in a video coding system for improving the quality of the decoded frames. However, different from the conventional super-resolution works, the compression artifacts in the decoded frames should be concerned with. The straightforward solution is to integrate the artifacts removing techniques before super-resolution. Nevertheless, some helpful features may be removed together with the artifacts, and remaining artifacts can be exaggerated. To address these problems, we design an end-to-end restoration-reconstruction deep neural network (RR-DnCNN) using the degradation-aware techniques. RR-DnCNN is applied to the down-sampling based video coding system. In the encoder side, the original video is down-sampled and compressed. In the decoder side, the decompressed down-sampled video is fed to the RR-DnCNN to get the original video by removing the compression artifacts and super-resolution. Moreover, in order to enhance the network learning capabilities, uncompressed low-resolution images/videos are utilized as a ground-truth. The experimental results show that our work can obtain over 8% BD-rate reduction compared to the standard H.265/HEVC. Furthermore, our method also outperforms in reducing compression artifacts in subjective comparison. Our work is available at https://github.com/minhmanho/rrdncnn.

Keywords

Super-resolution Video compression Deep learning 

References

  1. 1.
    Shen, M., Xue, P., Wang, C.: Down-sampling based video coding using super-resolution technique. IEEE Trans. Circ. Syst. Video Technol. 21(6), 755–765 (2011)CrossRefGoogle Scholar
  2. 2.
    Li, Y., et al.: Convolutional neural network-based block up-sampling for intraframe coding. IEEE Trans. Circ. Syst. Video Technol. 28(9), 2316–2330 (2018)CrossRefGoogle Scholar
  3. 3.
    Lin, J., Liu, D., Yang, H., Li, H., Feng, W.: Convolutional neural network-based block up-sampling for HEVC. IEEE Trans. Circ. Syst. Video Technol. (2018)Google Scholar
  4. 4.
    Feng, L., Zhang, X., Zhang, X., Wang, S., Wang, R., Ma, S.: A Dual-Network Based Super-Resolution for Compressed High Definition Video. In: Hong, R., Cheng, W.-H., Yamasaki, T., Wang, M., Ngo, C.-W. (eds.) PCM 2018. LNCS, vol. 11164, pp. 600–610. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00776-8_55 CrossRefGoogle Scholar
  5. 5.
    Zhao, T., Zhang, C., Ren, W., Ren, D., Hu, Q.: Unsupervised degradation learning for single image super-resolution. arXiv preprint: arXiv:1812.04240 (2018)
  6. 6.
    Bulat, A., Yang, J., Tzimiropoulos, G.: To learn image super-resolution, use a GAN to learn how to do image degradation first. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018, Part VI. LNCS, vol. 11210, pp. 187–202. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01231-1_12CrossRefGoogle Scholar
  7. 7.
    Chen, H., He, X., Ren, C., Qing, L., Teng, Q.: CISRDCNN: super-resolution of compressed images using deep convolutional neural networks. Neurocomputing 285, 204–219 (2018)CrossRefGoogle Scholar
  8. 8.
    Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)CrossRefGoogle Scholar
  9. 9.
    Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  10. 10.
    Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep cnn for image de-noising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018, Part VII. LNCS, vol. 11211, pp. 294–310. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01234-2_18CrossRefGoogle Scholar
  12. 12.
    Zhang, K., Zuo, W., Zhang, L.: Learning a single convolutional super-resolution network for multiple degradations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  13. 13.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint: arXiv:1412.6980 (2014)

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Graduate School of Science and EngineeringHosei UniversityTokyoJapan
  2. 2.Xi’dian UniversityXi’anChina
  3. 3.JST, PRESTOTokyoJapan

Personalised recommendations